Entity relationship extraction method for long document

Abstract

The invention discloses a long document-oriented entity relationship extraction method, which relates to the technical field of machine learning, and comprises the following steps of: constructing a prompt template, and splicing an input document and a special identifier added into a pre-training language model to form a new document; embedding type attention and structure attention in the pre-training language model to obtain a long document coding pre-training language model; and inputting the new document into the long document coding pre-training language model to obtain an entity relationship prediction result. According to the method, on one hand, the understanding effect on the document is improved by enhancing input and encoder capabilities through prompt information and embedding attention, and on the other hand, a complete joint extraction framework integrating entities, anaphora and relationships is constructed by combining the relevance of the entities and the relationships through prompt templates. According to the method, the problems that long document semantics are complex, long-distance dependence exists between entities and error propagation exists in entity and relation extraction in the prior art are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to an entity relationship extraction method for long documents. Background Art

[0002] Document-level entity relationship extraction aims to extract entity nouns and semantic relationships between entities from long documents, which can achieve document structuring and help subsequent applications such as knowledge graph construction. At present, related research has the following difficulties: First, the semantic information of documents is complex, and the number of entities and relationships in them far exceeds that of a single sentence. Entity pairs with relationships are often far apart in documents, and sentences related to them are scattered throughout the document. It is necessary to filter out useless information in the document and screen out key sentences. This phenomenon is called the long-distance dependency problem between entities. Secondly, document-level entity relationship extraction needs to go through multiple steps such as identifying entities, merging different mentions of the same entity, and extracting relationships between entities. There may be a problem of error accumulation. If the extracted entity is incorrect, the judgment of the corresponding relationship will inevitably be affected.

[0003] The existing technology for document-level entity relationship extraction is mainly divided into two categories: pipeline-based methods and joint extraction-based methods. The pipeline-based method mainly implements the three subtasks involved in joint extraction, namely entity recognition, coreference resolution, and relationship extraction, in a logical order. A corresponding module is designed for each task, and they are connected in series to obtain the entire model. This type of method is difficult to alleviate the error accumulation between different subtasks, and it is also difficult to share the knowledge learned by each. The joint extraction-based method solves multiple subtasks involved at the same time by building a joint model. At present, this type of method mainly builds a variety of graph networks and realizes multi-task interaction through convolution updates. However, current work has not considered entity recognition, but only combines coreference resolution and relationship extraction, which has limited effect on alleviating error propagation. Summary of the invention

[0004] The purpose of the present invention is to provide an entity relationship extraction method for long documents in response to the above-mentioned problems in the prior art, which can extract high-quality entities and relationships in complex long documents and eliminate long-distance dependencies and error propagation between entities.

[0005] In order to achieve the above object, the present invention has the following technical solutions: In a first aspect, a method for extracting entity relations for a long document is provided, comprising: Construct a prompt template to concatenate the input document with the special identifier added by the pre-trained language model to form a new document; Embed type attention and structure attention in the pre-trained language model to obtain a long document encoding pre-trained language model; Input the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result.

[0006] As a preferred solution, the expression for concatenating the input document with the special identifier added by the pre-trained language model is as follows:

[0007]

[0008] In the formula, and is the processed input document, and Special identifiers added for pre-trained language models, Represents a concatenation operation.

[0009] As a preferred solution, in the step of constructing a prompt template, splicing the input document with the special identifier added by the pre-trained language model to form a new document: if the two entities are in the same sentence, the sentence where the entity is located and the corresponding upper and lower sentences are selected to form a new document; if the two entities are not in the same sentence, but there is a third entity that is in the same sentence as the head and tail entities respectively, the sentences where the head and tail entities are located and the sentence where the third entity that is associated with them is located are selected as key sentences to form a new document.

[0010] As a preferred solution, the expression for embedding type attention and structural attention in the pre-trained language model is as follows:

[0011] In the formula, It represents type attention, which contains the type information of each position entity or non-entity, and focuses on each specific entity type through this attention; It represents structural attention, which indicates the position or co-reference relationship between two tokens, so that word pairs with similar structural relationships can obtain corresponding representations through specific neural network layers.

[0012] As a preferred solution, the entity relationship prediction results are output in the form of entity sets and triple sets.

[0013] As a preferred solution, in the step of inputting the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result, on the one hand, different types of entity mentions are obtained according to the entity template, and reference resolution is performed to obtain the entity; on the other hand, relationship triples are obtained according to the entity pairs.

[0014] As a preferred solution, in the step of inputting the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result, the entity type scores of the head and tail entities are calculated to promote the relationship score; the connection between the entity and the relationship is established through the prior probability, which alleviates the error accumulation caused by the sequential calculation of the two and realizes joint extraction; and the classifier is used to obtain The entity-relationship triples of are taken as the joint extraction results.

[0015] In a second aspect, a long document-oriented entity relationship extraction system is provided, comprising: The prompt template construction module is used to construct the prompt template, which concatenates the input document with the special identifier added by the pre-trained language model to form a new document; The attention embedding module is used to embed type attention and structural attention in the pre-trained language model to obtain a long document encoding pre-trained language model; The entity relationship prediction module is used to input new documents into the long document encoding pre-trained language model to obtain entity relationship prediction results.

[0016] According to a third aspect, an electronic device is provided, including: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the entity relationship extraction method for long documents.

[0017] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the entity relationship extraction method for long documents is implemented.

[0018] Compared with the prior art, the present invention has the following technical solutions: The entity relationship extraction method for long documents of the present invention, on the one hand, improves the understanding of documents by enhancing the input and encoder capabilities through prompt information and embedded attention, and on the other hand, combines the relevance of entities and relationships through prompt templates to construct a complete joint extraction framework that integrates entities, references, and relationships. The method of the present invention effectively solves the problems of complex semantics of long documents, long-distance dependencies between entities, and error propagation in the extraction of entities and relationships that exist in the prior art. Experiments have proved that the method of the present invention has better performance on complex and long documents, can extract high-quality entities and relationships, and has achieved good performance results in data sets in different fields. It has broad application prospects, is suitable for downstream tasks such as knowledge graph construction and intelligent question and answer, and assists in application scenarios such as smart education. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 Schematic diagram of the overall framework of the entity relationship extraction method for long documents in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, ordinary technicians in this field can also obtain other embodiments without making creative work.

[0022] See also Figure 1 The entity relationship extraction method for a long document according to an embodiment of the present invention comprises the following steps: Construct a prompt template to concatenate the input document with the special identifier added by the pre-trained language model to form a new document; Embed type attention and structure attention in the pre-trained language model to obtain a long document encoding pre-trained language model; Input the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result.

[0023] In one possible implementation, the prompt tuning method is a commonly used method in natural language processing tasks, which can help the pre-trained model improve encoding performance and achieve upstream and downstream task alignment. By constructing entity templates and relationship templates, they are combined with the input text to form new input information. On the one hand, the original document is enhanced, and on the other hand, the encoder can focus more on specific entities or entity pairs. The expression for splicing the input document with the special identifier added by the pre-trained language model is as follows:

[0024]

[0025] In the formula, and is the processed input document, and Special identifiers added for pre-trained language models, Represents a concatenation operation.

[0026] In addition to the constructed template, filtering the original document is also an important means to reduce noise, because the length of the original document may reach hundreds of words, which is far from the length of the template. Therefore, when encoding, the prompting effect of the template is likely to be overwhelmed by the document and cannot achieve the expected effect. The embodiment of the present invention constructs a prompting template, splices the input document with the special identifier added by the pre-trained language model, and forms a new document in the step: if the two entities are in the same sentence, then often this sentence and its context are sufficient to judge the relationship between them, so the sentence where the entity is located and the corresponding upper and lower sentences are selected to form a new document. If the two entities are not in the same sentence, but there is a third entity in the same sentence with the head and tail entities respectively, then this entity can be used as a bridge to connect the head and tail entities. Therefore, the sentences where the head and tail entities are located and the sentences where the third entity associated with them are located are selected as key sentences to form a new document.

[0027] In one possible implementation, a pre-trained language model, such as BERT, is a commonly used encoding tool in natural language processing tasks in recent years. However, in the scenario of long documents, a document often contains hundreds or thousands of words, which is completely different from the situation where there are only dozens of words at the sentence level. The performance of the attention mechanism of the pre-trained language model will be significantly reduced in this scenario. Therefore, the pre-trained model is improved by adding two sets of parameters to affect the weight calculation of attention and focus on specific entities or entity pairs. The expression for embedding type attention and structural attention in the pre-trained language model is as follows:

[0028] In the formula, It represents type attention, which contains the type information of each position entity or non-entity. In the previous step, a question-style entity template was designed to find different entity types. In order to further improve the effect of the prompt template and enhance the representation ability of the entity, so as to facilitate the discovery of the same entity by reference resolution, this attention is used to focus on each specific entity type; It represents structural attention, which indicates the position or co-reference relationship between two tokens, so that word pairs with similar structural relationships can be represented more accurately through calculations in specific neural network layers. After adding structural attention, even if two words are far apart, if they have a co-reference relationship, they can still be represented through The embodiment of the present invention improves the encoding ability of the pre-trained language model for long documents by adding two embedded attentions, type and structure, and alleviates the impact of document noise on the encoder.

[0029] In a possible implementation, after obtaining the encoded document, the entity relationship prediction results are output in the form of entity sets and triple sets. The prediction can be divided into two parts. One is to obtain different types of entity mentions based on the entity template, and perform reference resolution to obtain the entity. The other is to obtain the relationship triplet based on the entity pair. In addition, it is considered that the entity type may have a significant promoting effect on the relationship type. For example, if the correct relationship is "live in", then the relationship of the head entity should be "PERSON", and the relationship of the tail entity should be "LOCATION". If the head and tail entities are not of corresponding types, the predicted score of the "live in" relationship should also be reduced. Therefore, the embodiment of the present invention also calculates the entity type scores of the head and tail entities to promote the relationship score. The connection between entities and relationships is constructed through prior probabilities, which alleviates the error accumulation caused by the successive calculations of the two, and realizes joint extraction. Finally, the classifier can be used to obtain The entity relationship triples are taken as the final joint extraction results.

[0030] The embodiments of the present invention conduct experiments on multiple existing datasets for document-level entity relationship extraction, namely, the dataset DocRED based on a general field and its improved version Re-DocRED, to verify the effectiveness of the proposed method.

[0031] The final output result of the embodiment of the present invention includes multiple contents such as entities and relationships, so the result evaluation index needs to include indicators corresponding to each result, namely entity recognition (NER), coreference resolution (COREF), and relation extraction (RE). Since an entity may appear multiple times in a document, entity recognition is usually evaluated at the mention level, so it can also be called mention recognition (ME). Entity recognition and relationship extraction are classification tasks, and F1 value is used for evaluation. Coreference resolution is evaluated by the average of MUC, B3, and CEAF. In the coreference resolution task, MUC, B3, and CEAF are commonly used evaluation indicators, and the average of these three is usually used to comprehensively evaluate the performance of the system. The MUC (Message Understanding Conference) evaluation indicator is mainly used to measure the minimum number of links required to insert or delete the predicted coreference chain to the annotated coreference chain. The basic idea is to evaluate the performance of the system by calculating the link difference between the predicted result and the standard answer. However, the MUC evaluation indicator has a significant defect, that is, it cannot measure the performance of the system in predicting singleton entities. The B3 algorithm is proposed to overcome the shortcomings of the MUC evaluation indicator. It mainly calculates the precision and recall for each mention, and then takes the average of all mentions as the final indicator. Among them, precision refers to the ratio of the number of coreference chains correctly predicted by the system to the total number of coreference chains predicted by the system; recall refers to the ratio of the number of coreference chains correctly predicted by the system to the total number of coreference chains in the standard answer. The B3 algorithm can more comprehensively evaluate the performance of the system by comprehensively considering precision and recall. CEAF (Constituent-based Alignment for Evaluation of coreference) is an evaluation algorithm based on entity similarity. It evaluates the performance of the system by calculating the similarity between the predicted results and the entities in the standard answer. The CEAF algorithm takes into account the position and contextual information of the entity in the text, so it can more accurately evaluate the system's coreference resolution ability.

[0032] The joint model designed in the embodiment of the present invention is mainly divided into three modules: a prompt template construction module, an embedded attention encoding module and a joint prediction module.

[0033] The comparison methods are divided into three categories: 1) Pipeline-based implementation methods. This type of method includes JOINT-IE and JEREX, which mainly divide the document-level joint extraction task into three subtasks: entity recognition, coreference resolution, and relation extraction. Each task is solved by a module, and then the modules are connected in a pipeline manner to implement the entire model.

[0034] 2) Joint framework-based methods. This type of method includes JOINT-M and TAG, which combine these multiple subtasks through an entire framework. By constructing a graph neural network, with entities as nodes and the associations between entities as edges, it predicts the possible relationships or co-references between entities.

[0035] 3) Methods based on large language models. Considering the powerful performance of large language models, they are also used as one of the benchmark models for comparison. These include ChatGPT, which is implemented by ourselves, and GPT-3.5 from other research.

[0036] The above-mentioned datasets were used for comparison with the reference objects, and the various indicators of NER (named entity recognition), COREF (co-reference resolution / coreference resolution), and RE (relation extraction) were compared. The experimental results are shown in Table 1.

[0037] From the experimental results, the following conclusions can be drawn: a. The proposed JOINT-P model achieved the best results in both entity recognition and coreference resolution, and improved by 0.02%, 1.1%, and 4.1% in the three indicators of ME, RE, and RE ign, respectively, compared with the existing optimal model. The results prove that the joint framework proposed in the embodiment of the present invention can effectively extract entities and relationships. Although suboptimal results were achieved in the coreference resolution indicator, the embodiment of the present invention only used a lightweight classifier and clustering algorithm, which has much less training overhead than the current optimal model based on graph neural network method, so this result is also acceptable.

[0038] Table 1

[0039] b. In order to verify the superiority of the joint model designed in the embodiment of the present invention over the pipeline, the unified encoder for sharing knowledge in the model of the embodiment of the present invention was replaced with an encoder for each module to be trained separately, and the interaction design between different modules was removed, thereby obtaining JOINT-P (Pipeline). The results show that the performance of the pipeline framework after removing the module knowledge sharing will be significantly reduced, among which the relationship extraction will decrease the most, which also shows that in this case, the model's ability to cope with error propagation will be significantly reduced. In addition, by comparing the results of the pipeline model (JOINT-IE, JEREX) and the joint model (JOINT-M, TAG) in other baseline models, it can also be seen that the joint model of the embodiment of the present invention has better performance overall.

[0040] c. Although the large language model has achieved good performance in tasks such as sentence-level relation extraction, it still performs poorly in document-level entity relation joint extraction. The overall reasons are roughly twofold: one is that it is difficult to give accurate prompt commands for complex long-document entity relation extraction; the other is that its ability to understand long documents and reason between entities is average.

[0041] Table 2 also shows the performance on the validation set and test set of Re-DocRED (re-annotation of document-level relation extraction DocRED). Compared with DocRED, the method of the embodiment of the present invention has achieved significant improvement in the relation extraction task, but the gap is not much in entity recognition and coreference resolution. This is mainly because Re-DocRED only further annotates the relationship, while the entity remains unchanged. However, the improvement in relation extraction also verifies the generalization of the method of the embodiment of the present invention. When the quality of data annotation improves, the performance of the method of the embodiment of the present invention is also improved accordingly, and there is no overfitting of the training set.

[0042] Table 2

[0043] In addition to the F1 value, the computational overhead of the model is also something that needs to be considered. Based on the average results of 5 tests, the time required for one round of model training is about 37 minutes, and the total time for a complete training and test is about 39 hours, which is much shorter than that of a large language model. As an end-to-end entity relationship joint extraction model, its computational overhead is also acceptable.

[0044] Ablation experiment analysis In order to verify the effectiveness of each module in the model, the embodiment of the present invention further designs an ablation experiment, and verifies the effectiveness of each module of the model through the control variable method, that is, removes the module to be tested and keeps other parts unchanged, and observes the impact of each module on the model performance. If the performance decreases after removing the module, it proves that this module is effective for the overall model. Since the designed model is relatively complex and has many modules. When conducting ablation experiments, it is divided into three major groups, and each group verifies the effectiveness of the overall model design, entity recognition part, and relationship extraction part modules. The experimental results are shown in Table 3.

[0045] The ablation experiment design in the table is divided into three parts. Overall means taking the entire model as the basis, gradually deleting some key modules and observing the performance changes of the model. Entity part means using only the part of the model for entity recognition and observing its performance for entity extraction. Relation part means using only the module of the model for extracting relations and observing its performance for relation extraction. From the experimental results, the following conclusions can be drawn: Table 3

[0046] From an overall perspective, in order to verify that the joint model can promote entity recognition and relationship extraction at the same time, the embedding attention module, new document construction module and entity-relation mutual enhancement module were removed from the complete model. After removing the attention, the entity recognition result dropped significantly by 4.17% to only 89.2%, which also led to a decrease of 5.33% and 8.14% in coreference resolution and relationship extraction, respectively. This shows that the designed attention module can effectively target long documents and play an important role in both entities and relationships. After removing the new documents for relationship construction, relationship extraction dropped by 3.94%, but entity recognition and coreference resolution also dropped by 0.61% and 1.49%. Similarly, after removing the entity-relation mutual enhancement module, entity recognition, coreference resolution, and relationship extraction dropped by 1.50%, 0.41%, and 0.70%, respectively. This shows that although these two modules are mainly designed for relationships, they can also play a role in entities and coreference. Entity-relation mutual enhancement not only enhances relationships, but also plays a reverse role in promoting entities. Overall, the experiments verified the effectiveness of these three important modules, which are all effective for the three subtasks simultaneously.

[0047] In order to verify the effectiveness of the model for entity recognition, the relationship part was removed and only the effect of the entity part module was tested. On the basis of the benchmark pre-trained model BERT, entity templates, type attention and structural attention were gradually added. After adding these parts, the model improved by 0.99%, 5.39% and 1.81% in entity recognition respectively. This shows that these modules can promote the extraction of entities in documents, among which type attention accounts for the largest proportion, indicating that after adding auxiliary information of entity type, the representation ability of pre-trained templates for document-level entity recognition is significantly enhanced. The structural attention also shows its effect on mining document information and the prompting effect of entity templates on the original document. The experiments in this part prove the effectiveness of this module for entity extraction.

[0048] In order to verify the performance of the model in relation extraction, the entity recognition part was removed, and only the effect of the joint extraction part module was tested. The entity and the real label mentioned were input as the entity recognition result, that is, converted into the form of document-level relation extraction. In the case of original document input, new documents and entity relationship mutual enhancement modules were gradually added. From the results, it can be seen that after adding these modules, the relation extraction results were improved by 1.35% and 0.92% respectively. Among them, the constructed new documents brought a greater improvement, indicating the importance of relation templates and key sentence construction for solving long document information screening. And even compared with some of the latest optimal models for document-level relation extraction, the performance of this part is relatively competitive. The experiments in this part also prove the effectiveness of this module for relation extraction.

[0049] The above ablation experiments prove that the designed model not only performs well in the joint extraction of entity relations, but also that each module in it promotes the extraction of entities and relations.

[0050] Another embodiment of the present invention further provides an entity relationship extraction system for long documents, comprising: The prompt template construction module is used to construct the prompt template, which concatenates the input document with the special identifier added by the pre-trained language model to form a new document; The attention embedding module is used to embed type attention and structural attention in the pre-trained language model to obtain a long document encoding pre-trained language model; The entity relationship prediction module is used to input new documents into the long document encoding pre-trained language model to obtain entity relationship prediction results.

[0051] Another embodiment of the present invention further provides an electronic device, comprising: A memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the entity relationship extraction method for long documents.

[0052] Another embodiment of the present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the entity relationship extraction method for long documents is implemented.

[0053] Exemplarily, the instructions stored in the memory may be divided into one or more modules / units, which are stored in a computer-readable storage medium and executed by the processor to complete the entity relationship extraction method for long documents of the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the server.

[0054] The electronic device may be a computing device such as a smart phone, a notebook, a PDA, and a cloud server. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the electronic device may also include more or fewer components, or a combination of certain components, or different components, for example, the electronic device may also include an input / output device, a network access device, a bus, etc.

[0055] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0056] The memory may be an internal storage unit of the server, such as a hard disk or memory of the server. The memory may also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the server. Furthermore, the memory may include both an internal storage unit of the server and an external storage device. The memory is used to store the computer-readable instructions and other programs and data required by the server. The memory may also be used to temporarily store data that has been output or is to be output.

[0057] It should be noted that the information interaction, execution process and other contents between the above-mentioned module units are based on the same concept as the method embodiment. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0058] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0059] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0060] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0061] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for extracting entity relations from long documents, characterized in that: include: Construct a prompt template to concatenate the input document with the special identifier added by the pre-trained language model to form a new document; Embed type attention and structure attention in the pre-trained language model to obtain a long document encoding pre-trained language model; Input the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result.

2. The entity relationship extraction method for long documents according to claim 1 is characterized in that: The expression for concatenating the input document with the special identifier added by the pre-trained language model is as follows: In the formula, and is the processed input document, and Special identifiers added for pre-trained language models, Represents a concatenation operation.

3. The entity relationship extraction method for long documents according to claim 1 is characterized in that: In the step of constructing a prompt template, splicing the input document with the special identifier added by the pre-trained language model to form a new document: if the two entities are in the same sentence, the sentence where the entity is located and the corresponding upper and lower sentences are selected to form a new document; if the two entities are not in the same sentence, but there is a third entity that is in the same sentence as the head and tail entities respectively, the sentences where the head and tail entities are located and the sentence where the third entity that is associated with them is located are selected as key sentences to form a new document.

4. The entity relationship extraction method for long documents according to claim 1 is characterized in that: The expressions for embedding type attention and structural attention in the pre-trained language model are as follows: In the formula, It represents type attention, which contains the type information of each position entity or non-entity, and focuses on each specific entity type through this attention; It represents structural attention, which indicates the position or co-reference relationship between two tokens, so that word pairs with similar structural relationships can obtain corresponding representations through specific neural network layers.

5. The entity relationship extraction method for long documents according to claim 1 is characterized in that: The entity relationship prediction results are output in the form of entity sets and triple sets.

6. The entity relationship extraction method for long documents according to claim 5 is characterized in that: In the step of inputting the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result, on the one hand, different types of entity mentions are obtained according to the entity template, and reference resolution is performed to obtain the entity; on the other hand, relationship triples are obtained according to the entity pairs.

7. The entity relationship extraction method for long documents according to claim 6 is characterized in that: In the step of inputting the new document into the long document encoding pre-trained language model to obtain the entity relationship prediction result, the entity type scores of the head and tail entities are calculated to promote the relationship score; the connection between the entity and the relationship is constructed through the prior probability, which alleviates the error accumulation caused by the sequential calculation of the two and realizes joint extraction; the classifier is used to obtain The entity-relationship triples of are taken as the joint extraction results.

8. An entity relationship extraction system for long documents, characterized in that: include: The prompt template construction module is used to construct the prompt template, which concatenates the input document with the special identifier added by the pre-trained language model to form a new document; The attention embedding module is used to embed type attention and structural attention in the pre-trained language model to obtain a long document encoding pre-trained language model; The entity relationship prediction module is used to input new documents into the long document encoding pre-trained language model to obtain entity relationship prediction results.

9. An electronic device, characterized in that: include: A memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the entity relationship extraction method for long documents as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the entity relationship extraction method for long documents as described in any one of claims 1 to 7.

Images