Chapter-level relationship extraction method, device, equipment, medium and product

Abstract

The present application discloses a chapter-level relationship extraction method, device, equipment, medium and product, which relates to the field of natural language processing technology. The method includes: obtaining a target chapter document for extracting chapter-level relationships; obtaining a first relationship triple in the target chapter document; obtaining a target rule corresponding to the relationship in a preset rule library according to the relationship in the first relationship triple, the target rule including: a mapping relationship between the relationship and the new inferred relationship, and attribute information of the entity in the new inferred relationship; inferring the new inferred relationship triple contained in the first relationship triple according to the target chapter document, the attribute information of the entity included in the first relationship triple and the target rule; outputting the first relationship triple and the new inferred relationship triple. The present application can improve the accuracy of the inferred new relationship group.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method, device, equipment, medium and product for extracting paragraph-level relations. Background Art

[0002] With the rapid development of the Internet and the explosive growth of information data, more and more unstructured data can be obtained.

[0003] Information extraction is used to extract the required entities and the relationships between entities from a large amount of unstructured data, that is, to extract relationship triples, thereby converting unstructured data into a structured form of knowledge, making it easier for users to interpret and use, such as building knowledge graphs, question-answering systems, and performing semantic analysis.

[0004] Currently, deep learning models are usually used to extract relation triplets from unstructured data, and then new relation triplets are inferred based on the extracted relation triplets and existing relation types. However, this method may cause logical errors in the inferred new relation triplets. Summary of the invention

[0005] The purpose of this application is to provide a chapter-level relationship extraction method, device, equipment, medium and product, which can improve the accuracy of the inferred new relationship group.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for extracting chapter-level relations, the method comprising:

[0008] Obtain the target chapter document for extracting chapter-level relations;

[0009] Acquire the first relation triple in the target chapter document;

[0010] According to the relationship in the first relationship triplet, a target rule corresponding to the relationship is obtained in a preset rule library, wherein the target rule includes: a mapping relationship between the relationship and the newly inferred relationship, and attribute information of an entity in the newly inferred relationship;

[0011] Inferring a new inferred relation triple contained in the first relation triple according to the target chapter document, the attribute information of the entity included in the first relation triple and the target rule;

[0012] The first relation triplet and the new inferred relation triplet are output.

[0013] In a second aspect, the present application provides a chapter-level relationship extraction device, the chapter-level relationship extraction device comprising:

[0014] The first acquisition module is used to acquire the target chapter document for extracting the chapter-level relationship;

[0015] A second acquisition module, used for acquiring a first relation triple in the target chapter document;

[0016] A third acquisition module, configured to acquire a target rule corresponding to the relationship in a preset rule base according to the relationship in the first relationship triplet, wherein the target rule includes: a mapping relationship between the relationship and the newly inferred relationship, and attribute information of an entity in the newly inferred relationship;

[0017] An inference module, configured to infer a new inferred relation triple contained in the first relation triple according to the target chapter document, attribute information of entities included in the first relation triple and the target rule;

[0018] An output module is used to output the first relation triplet and the new inferred relation triplet.

[0019] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described paragraph-level relationship extraction methods.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for extracting paragraph-level relationships.

[0021] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned paragraph-level relationship extraction methods.

[0022] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0023] The present application provides a chapter-level relationship extraction method, apparatus, equipment, medium and product. When inferring new relationship triples from relationship triples, the attribute information of each entity is taken into account. That is, after obtaining the relationship triple, a new inferred relationship corresponding to the relationship is obtained based on the relationship in the relationship triple. However, during reasoning, the new inferred relationship triple is not directly obtained based on the obtained new inferred relationship, but the attribute information of each entity in the relationship triple is also obtained. Only when the attribute information of each entity in the relationship triple is the same as the attribute information of each entity corresponding to the new inferred relationship triple, can it be determined that the new inferred relationship can be inferred from the relationship of the relationship triple, thereby obtaining a new inferred relationship triple based on the new inferred relationship. This solution can avoid the problem of logical errors in the inferred new inferred relationship triples and improve the accuracy of reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 Schematic diagram of relationship inference provided in one embodiment of the present application Figure 1 ;

[0026] Figure 2 Schematic diagram of relationship inference provided in one embodiment of the present application Figure 2 ;

[0027] Figure 3 Schematic diagram of relationship inference provided in one embodiment of the present application Figure 3 ;

[0028] Figure 4 Schematic diagram of relationship inference provided in one embodiment of the present application Figure 4 ;

[0029] Figure 5 is a flow chart of a method for extracting chapter-level relations according to an exemplary embodiment;

[0030] Figure 6 is a schematic diagram of generating a preset database according to an exemplary embodiment;

[0031] Figure 7 is a schematic diagram of a chapter-level relationship extraction method according to an exemplary embodiment;

[0032] Figure 8is a functional module diagram of a chapter-level relationship extraction device according to an exemplary embodiment;

[0033] Fig. 9 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] With the rapid development of the Internet and the explosive growth of information data, people are able to access more and more text and unstructured data. This has greatly expanded our ability to obtain and communicate information, but it has also brought a huge challenge: how to extract the information we need from massive data and organize this information in a way that is easy to understand and use. This challenge reflects the importance of information extraction. Information extraction is to extract and understand application-related information from large amounts of data, converting unstructured data into structured knowledge forms, making it easier to interpret and use in various applications, such as building knowledge graphs, question-answering systems, and performing semantic analysis.

[0037] To better solve this problem, scholars have developed a variety of relation extraction (RE) methods. Relation extraction is the core part of information extraction. It transforms unstructured information into structured knowledge by identifying and classifying the semantic relationships between entities in the original text, thereby providing support for various applications such as knowledge graph construction, question answering systems, and semantic analysis.

[0038] Among them, "entity" refers to an element with a specific meaning in the text, which is usually a name or term with an actual reference. Specifically, entities include but are not limited to the following categories:

[0039] 1. Name of a person.

[0040] 2. Place name (Location): such as "XX City", "XX National Park".

[0041] 3. Organization name: such as "XX University", "XX Company".

[0042] 4. Date / Time: such as "June 25, 2024", "3 p.m."

[0043] 5. Money: such as "500 yuan".

[0044] 6. Percent: such as "20%", "50%".

[0045] 7. Product name (Product).

[0046] These entities are usually the most informative and realistic parts of a text, so identifying and extracting them is of great significance in natural language processing tasks such as information extraction, question-answering systems, and machine translation.

[0047] However, early relation extraction methods mainly focused on the sentence level, which could not capture the entity relationships between texts or sentences, and were prone to missing valuable contextual information in large-scale texts, and therefore could not well understand and identify complex relationships.

[0048] With the advancement of technology, researchers have begun to try to solve these problems, and new methods have emerged, such as document-level relation extraction, which can span the entire document, identify and classify the relationships between entities, make up for the shortcomings of sentence-level relation extraction, and extract richer contextual information while understanding the complexity of relationships.

[0049] However, this method often ignores the attributes of the entity itself when using logical rules. In other words, it ignores the properties of the entity itself, which will affect the confidence of the newly inferred relationship triples, such as Figure 1-Figure 4 The relationship shown is inferred, where Figure 1 is a unary rule, Figure 2-Figure 4 For binary rules, the solid line represents a relationship that has been proven to be true, which is the condition for rule reasoning. The dotted line represents a possible relationship, which corresponds to the result of reasoning. If you do not pay attention to the attributes of each entity and directly obtain the dotted line relationship from the solid line relationship, there will be reasoning errors.

[0050] For example, the binary variable r(eh,et) indicates that there is a relationship between entity eh and entity et, where r ∈ R. The inferred rinfer (ej,ek) is a new relationship binary variable, which is the union of n binary variables. The inference process is as follows:

[0051] r1 (e1,e2) ^ r2 (e2,e3) ...^ r(n-1) (e(n-1),en) rinfer (ej,ek).

[0052] Among them, ei∈ ε, ri∈ Tr, j∈(0,n), k∈(0,n), and n represents the length of the rule. This form can represent a series of common logical relations, such as symmetry and transitivity.

[0053] A symmetry is a unary relation that can reverse direction, such as the relation between spouses: spouse_of(e1,e2) ↔ spouse_of(e2,e1).

[0054] Transitivity is a binary or multi-element relationship that is very suitable for modeling cross-sentence relationships in paragraph-level relationship extraction, which has always been a difficulty in paragraph-level relationship extraction. In the previous traditional methods, graph neural networks are used to model long-distance dependencies across sentences, but this usually lacks logic. Transitivity rules can solve this problem very effectively. The following is an example of a transitivity rule:

[0055] mother_of(e1,e2)^ spouse_of(e2,e3) → father_of(e1,e3).

[0056] Taking the entity of person as an example, the existing relationship triple is: a is the child of b, but if the relationship triple directly inferred from this relationship triple is b is the mother of a, that is, a is the child of b b is the mother of a, which is obviously wrong, because it does not take into account b's gender. Here, a person's gender is an entity attribute. If gender is not considered, the newly inferred relationship triplet will be wrong.

[0057] In order to solve the above technical problems, the present disclosure proposes a chapter-level relationship extraction method, device, equipment, medium and product.

[0058] Figure 5 is a flowchart of a method for extracting chapter-level relations according to an exemplary embodiment. Figure 5 As shown, the chapter-level relationship extraction method includes the following steps S101-S105:

[0059] In step S101, a target chapter document whose chapter-level relationship is to be extracted is obtained.

[0060] In step S102, the first relation triplet in the target chapter document is obtained.

[0061] In step S103, a target rule corresponding to the relationship is obtained in a preset rule library according to the relationship in the first relationship triplet, and the target rule includes: a mapping relationship between the relationship and the new inferred relationship, and attribute information of the entity in the new inferred relationship.

[0062] Among them, each rule in the preset rule base is composed of a relationship triple and the attribute information of the entity in the relationship triple, that is, the relationship triple is refined through the attribute information constraints of the entity.

[0063] In step S104, a new inferred relationship triple contained in the first relationship triple is inferred according to the target chapter document, the attribute information of the entity included in the first relationship triple and the target rule.

[0064] In step S105 , the first relation triplet and the new inferred relation triplet are output.

[0065] Continuing with the above example, the first relation triple is a is the child of b, and the target rule is: a is the child of b b is the mother of a, and b is female. Then, according to the target chapter document, the first relation triplet is obtained, which is the gender of b in a is b's child. If the gender of b in a is b's child is female, then the new inferred relation triplet of b is the mother of a can be obtained. That is, by adding attribute constraints to the entity, such as b is female, the confidence of the reasoning result can be effectively improved.

[0066] In the present disclosure, the attribute constraints of entities are introduced into the chapter-level relationship extraction. When inferring new relationship triples for relationship triples, the attribute information of each entity will be taken into consideration. That is, after obtaining the relationship triplet, the new inferred relationship corresponding to the relationship will be obtained based on the relationship in the relationship triplet. However, during reasoning, the new inferred relationship triplet is not directly obtained based on the obtained new inferred relationship, but the attribute information of each entity in the relationship triplet will also be obtained. Only when the attribute information of each entity in the relationship triplet is the same as the attribute information of each entity corresponding to the new inferred relationship triplet, can it be determined that the new inferred relationship can be inferred from the relationship of the relationship triplet, thereby obtaining a new inferred relationship triplet based on the new inferred relationship. This scheme can avoid the problem of logical errors in the inferred new inferred relationship triplet, and improve the accuracy of reasoning.

[0067] Since document-level relationship extraction introduces a very long context, taking the DocRED dataset as an example, the 96 relationship types and the corresponding relationship type explanations are already hundreds of words of context, which will bring great difficulties to the retrieval of large models. Therefore, some methods need to be adopted to reduce the length of the input text to reduce the burden of reasoning for large models. The present disclosure designs a set of algorithms to preliminarily select rules related to the target chapter document. Specifically, all relationship facts that appear in the target chapter document (the first relationship triples in the above embodiment) are counted, and these first relationship triples are used as queries to select rules related to the existing first relationship triples in the preset rule library. Through this method, rules with no relationship facts for reasoning and relationship facts with no rules for reasoning are removed, which greatly reduces the length of the input text.

[0068] Here, an existing model can be used to obtain the first relation triple in the target chapter document. For example, the first relation triple in the target chapter document is pre-acquired according to a deep learning model (such as ATOLP and DREEAM). This can also be understood as extracting basic relation facts in the target chapter document through a deep learning model, and then retrieving applicable target rules based on these basic relation facts.

[0069] In one embodiment, before obtaining the target chapter document to be extracted from the chapter-level relationship, the following steps A1-A4 are also included:

[0070] A1. Obtain existing relationship triples.

[0071] Acquire existing relationship triplets according to a deep learning model, for example, acquire existing relationship triplets according to a baseline model.

[0072] A2. Obtain the attribute information of each entity in the existing relationship triple.

[0073] Existing relation triples can be input into a large language model for more detailed evaluation, especially from the perspective of entity attribute constraints, to improve and supplement the attribute information of each entity in the existing relation triples.

[0074] A3. Generate a first rule base through the first large language model, wherein the first rule base includes: the second relation triples obtained through the first large language model and attribute information of entities in the second relation triples.

[0075] When generating the first rule base through the first large language model, manually annotated training data can be obtained first, and then the training data can be input into the first large language model so that the first large language model generates the first rule base according to the training data. The training data includes: training documents, entities in the training documents, third relationship triples in the training documents, and attribute information of each entity in the third relationship triples.

[0076] A4. Obtain a preset rule base according to the first rule base and the second rule base, wherein the second rule base includes: a mapping relationship between existing relationship triples and attribute information of each entity in the existing relationship triples.

[0077] Before executing steps S101-S105, it is necessary to construct a preset rule base. In the present disclosure, the above steps A1-A4 can be executed by a rule generator to mine rules to construct the preset rule base.

[0078] The rules disclosed in this disclosure have two sources. The first part of the rules is derived from the rules generated by some previous work (steps A1-A2 above), such as AMIE, which generates a second rule base through model mining. The second part of the rules is derived from inputting labeled training data into a large language model, allowing the large language model to summarize effective rules based on these training data (step A3 above).

[0079] Since the rules in the first part are from previous work, however, the quality of these rules varies, and some may not be directly used in the reasoning process of the new inferred relation triples in the present disclosure, therefore, in one embodiment, the above step A1 includes the following sub-steps A11-A13:

[0080] A11. Obtain existing candidate relationship triples.

[0081] Existing candidate relationship triplets may be obtained through a deep learning model, for example, by obtaining existing candidate relationship triplets through a baseline model.

[0082] A12. Obtain the confidence of each existing candidate relationship triple.

[0083] The confidence of existing candidate relationship triples can be obtained through a deep learning model, for example, the confidence of existing candidate relationship triples can be obtained through a baseline model.

[0084] A13. Determine that the existing candidate relation triplet corresponding to the target confidence is the existing relation triplet, and the target confidence is the confidence of the existing candidate relation triplet that is greater than the preset confidence.

[0085] Since confidence can be used to measure the credibility of rules, the existing candidate relationship triples can be evaluated based on the confidence, and a part of high-quality rules can be screened out to obtain the existing relationship triples.

[0086] In order to further screen the existing rules so as to make the reasoning of the new inferred relation triples in the present disclosure more accurate, the above step A13 includes the following sub-steps A131-A134:

[0087] A131. Obtain a target confidence level whose confidence level of an existing candidate relationship triple is greater than a preset confidence level.

[0088] A132. Obtain the support score of the existing candidate relation triples corresponding to the target confidence.

[0089] The support scores of existing candidate relationship triples corresponding to the target confidence can be obtained through the deep learning model. For example, the support scores of existing candidate relationship triples corresponding to the target confidence can be obtained through the baseline model.

[0090] A133. Obtain a target support score whose support score is greater than a preset score.

[0091] A134. Determine the existing candidate relation triple corresponding to the target support score as the existing relation triple.

[0092] In this disclosure, a portion of higher quality rules will be screened out based on a comprehensive evaluation of confidence and support.

[0093] Furthermore, obtaining a preset rule base according to the first rule base and the second rule base further includes the following sub-steps A41-A42:

[0094] A41. Obtain natural language explanations corresponding to all relationship triples in the first rule base and the second rule base.

[0095] The large language model can be used to generate natural language explanations for each relation triple in the first rule base and the second rule base. The generated natural language explanations of these relation triples help the large language model to better understand their meanings in the relational reasoning stage, so as to more accurately infer new inferred relation triples.

[0096] A42. Obtain a preset rule base according to the natural language interpretations corresponding to the first rule base, the second rule base, and all the relationship triples.

[0097] In order to enable the large language model to better understand the meaning of each relation triple in the preset rule base during the relational reasoning stage, examples corresponding to each relation triple in the preset rule base can also be received and saved in the preset rule base.

[0098] Since the preset rule library also includes the natural language interpretation corresponding to the relationship triples, the target rules in the corresponding step S103 also include: the natural language interpretation of the first relationship triple, and the natural language interpretation of the newly inferred relationship triple; at this time, the first relationship triple and the new inferred relationship triple are output, including: outputting the first relationship triple, the new inferred relationship triple, the natural language interpretation corresponding to the first relationship triple, and the natural language interpretation corresponding to the new inferred relationship triple.

[0099] It is difficult for traditional deep learning models to add entity attribute constraints to rules, but this task is feasible for large language models. Therefore, this public research introduces large language models (LLMs) to use entity attributes for relational reasoning. The emergence of large-scale language models represents a major advancement in the field of natural language processing. These models, such as ChatGPT and Claude, have demonstrated impressive capabilities in various general tasks such as text summarization, translation, and question answering. In the field of information extraction, there are also some studies that use large language models. However, these studies currently focus on the field of sentence-level relation extraction and have not achieved satisfactory results. Problems such as overly long context input, unstable output related to generative models, and logical errors caused by large model hallucinations lead to poor performance of large language models in document-level relation extraction tasks. Therefore, using a large language model to complete this task end-to-end may not be a good choice. Therefore, in this disclosure, a deep learning model is combined with a large language model to avoid the use of a large language model to complete the end-to-end chapter-level relation extraction.

[0100] Specifically, the above step S104 includes the following sub-steps S1041-S1042:

[0101] S1041. Obtain attribute information of entities included in the first relation triplet through the second largest language model.

[0102] S1042. Inferring a new inferred relation triple contained in the first relation triple by using the second largest language model according to the target chapter document, attribute information of entities included in the first relation triple and the target rule.

[0103] Of course, in practical applications, step S103 may also be completed through the second largest language model, that is, through the second largest language model, the corresponding target rule is obtained in the preset rule library according to the relationship in the first relationship triple.

[0104] The solutions in the present disclosure are introduced below through specific implementation methods.

[0105] The execution process consists of two stages.

[0106] The first phase is the preparation phase, whose main goal is to generate a high-quality preset rule base.

[0107] The rules in the preset rule base come from two sources: one source is the rules extracted from previous work (such as AMIE) by re-evaluating the rules extracted from the previous work (such as AMIE) in the above steps A1-A2. Figure 6 ), where feasible rules are directly adopted, while imperfect rules are improved by adding attribute information of entities.

[0108] like Figure 6 As shown in the lower left corner, there is a rule: if entity a is located in the administrative region of entity b, then entity b contains the administrative region entity a. This rule has a high score in previous work, but it ignores the constraints of entity attributes, that is, this rule requires entity a to be an administrative region. If entity a is a building or infrastructure, then the requirements of relationship P150 (including administrative region) cannot be met.

[0109] Another source is to use the first large language model directly from the training data (i.e. Figure 6 The label relation triples in ) summarize the rules, such as Figure 6 As shown in the upper left corner, there are three relationships in the labels of the training data. The first is that A is the director of "Movie B", the second is that A's country is C, and the third is that the production country of "Movie B" is C. By analyzing and summarizing these three relationships, we can get a rule: if the director of entity a is entity b, the country of entity b is entity c, and entity a is a creative work, then the production country of entity a is also entity c.

[0110] After obtaining the available preset rule base, the large language model will generate an explanation for each rule (see Figure 6 The explanation part of the preset rule base in ), and manually annotate an example ( Figure 6 The purpose of this is to facilitate the use of large language model understanding rules during the inference phase.

[0111] In the second stage, the reasoning stage.

[0112] like Figure 7 As shown, the target chapter document to be extracted for the chapter-level relationship is: Person D was born in 1963 at his grandmother's home in F, Country E, ... Person D's most famous work is "G Movie" ...

[0113] First, the baseline model is used to pre-extract basic relational facts (the first relational triple in the above embodiment). These relational facts often only reflect the relationship within a sentence, such as Figure 7 This is shown in the pre-extraction section below.

[0114] Then, according to the content and relationship facts of the target chapter document, the target rules are retrieved from the preset rule library. Figure 7 The target chapter document in the article is about the introduction of a director. "Director" and "country" appear in the pre-extracted basic relationship facts. Then, a target rule involving "director" and "country" can be retrieved from the preset rule library.

[0115] Finally, the target chapter document, relation facts and target rules mentioned above are combined to form prompts for input to the large language model. The large language model will determine whether the attribute information of the entity required by the target rule is satisfied based on the content of the target chapter document, and finally infer the new inferred relation triples and generate corresponding explanations, such as Figure 7 As shown on the right, the large language model first selects the following from the preset rule base based on the two relationship facts that the director of "G Movie" is character D and character D's country is country E. Figure 7 The target rule shown in . Then the target chapter document is retrieved to obtain that "G Movie" is a creative work, specifically a movie, which satisfies the attribute information of the entity required in the target rule. Therefore, based on the above information, the new inference relation triple is inferred that the production country of "G Movie" is E country.

[0116] If this new inferred relation triple does not exist in the original relation fact, it is the relation that was not captured in the pre-extraction stage. This is exactly the reasoning result we need. The new relation triple inferred here is often more complex and can reflect the logical relationship between sentences.

[0117] Finally, the large language model evaluates the new relations it has inferred to maintain self-consistency.

[0118] The present disclosure introduces logical rules (the preset rule base mentioned above) and a large language model to enhance the effect of relational reasoning. The method first constructs a rule generator to mine rules and generate corresponding explanations. There are two sources of rules. One is to inherit some of the rules generated by previous work, and let the large language model screen and improve them. The second is to input labeled training data into the large language model, and let the large language model summarize effective rules based on these data. Each rule consists of a rule body and entity attribute constraints. Then, the large language model applies these rules and constraints to infer possible new inferred relationships based on the relationships identified by the baseline model. Finally, this method requires the large language model to evaluate its reasoning results to ensure its self-consistency. The present disclosure introduces logical constraints into the chapter-level relation extraction, explicitly captures remote dependencies and entity attributes to solve the problem of weak logic in the original model, and introduces a large language model to enhance relational reasoning with its excellent reasoning ability, and achieves remarkable results in the chapter-level relation extraction task.

[0119] The present disclosure proposes a new framework for enhanced reasoning with a large language model, which uses logical constraints (attribute information of entities in the above embodiments) to enhance document-level relationship extraction. By introducing external rules, it enhances the reasoning ability of a large language model. This is the first time that the use of the reasoning ability of a large language model to extract document-level relationships has been proposed. This method effectively solves the problem that traditional models ignore the impact of entity attributes on rules and have poor interpretation capabilities. In addition, logical constraints are introduced during the reasoning process to alleviate the hallucination problem in the large language model. The method and idea of ​​applying rules (or more broadly, external knowledge) to solve problems with a large language model proposed in the present disclosure has the potential to be widely used in many other scenarios such as semantic understanding, natural language processing, information retrieval, and artificial intelligence decision-making, improving the efficiency and accuracy of problem solving in these fields.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a chapter-level relationship extraction device for implementing the above-mentioned chapter-level relationship extraction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more chapter-level relationship extraction device embodiments provided below can refer to the limitations of the chapter-level relationship extraction method above, and will not be repeated here.

[0121] In an exemplary embodiment, Figure 8 As shown, a chapter-level relationship extraction device is provided, comprising:

[0122] The first acquisition module 11 is used to acquire the target chapter document from which the chapter-level relationship is to be extracted;

[0123] A second acquisition module 12 is used to acquire the first relationship triple in the target chapter document;

[0124] A third acquisition module 13 is used to acquire a target rule corresponding to the relationship in a preset rule base according to the relationship in the first relationship triple, wherein the target rule includes: a mapping relationship between the relationship and the newly inferred relationship, and attribute information of an entity in the newly inferred relationship;

[0125] An inference module 14, configured to infer a new inferred relation triple contained in the first relation triple according to the target chapter document, the attribute information of the entity included in the first relation triple and the target rule;

[0126] The output module 15 is used to output the first relation triplet and the new inferred relation triplet.

[0127] As an optional implementation, the chapter-level relationship extraction device further includes:

[0128] A fourth acquisition module, used for acquiring existing relationship triples before the first acquisition module acquires the target chapter document from which the chapter-level relationship is to be extracted;

[0129] A fifth acquisition module, used to acquire attribute information of each entity in the existing relationship triple;

[0130] A generating module, configured to generate a first rule base through a first large language model, wherein the first rule base includes: a second relation triplet obtained through the first large language model and attribute information of entities in the second relation triplet;

[0131] The determination module is used to obtain the preset rule base according to the first rule base and the second rule base, wherein the second rule base includes: a mapping relationship between the existing relationship triples and the attribute information of each entity in the existing relationship triples.

[0132] As an optional implementation, the generating module includes:

[0133] The first generating submodule is used to input training data into the first large language model so that the first large language model generates the first rule base according to the training data, wherein the training data includes: training documents, entities in the training documents, third relation triples in the training documents, and attribute information of each entity in the third relation triples.

[0134] As an optional implementation manner, the fourth acquisition module includes:

[0135] The first acquisition submodule is used to acquire existing candidate relationship triples;

[0136] A second acquisition submodule is used to obtain the confidence of each of the existing candidate relationship triples;

[0137] The third acquisition submodule is used to determine that the existing candidate relationship triplet corresponding to the target confidence is the existing relationship triplet, and the target confidence is the confidence of the existing candidate relationship triplet that is greater than a preset confidence.

[0138] As an optional implementation manner, the third acquisition submodule includes:

[0139] A first acquisition subunit is used to acquire a target confidence level whose confidence level of the existing candidate relationship triple is greater than the preset confidence level;

[0140] A second acquisition subunit is used to obtain the support score of the existing candidate relationship triple corresponding to the target confidence;

[0141] A third acquisition subunit is used to obtain a target support score whose support score is greater than a preset score;

[0142] The first determining subunit is used to determine that the existing candidate relationship triplet corresponding to the target support score is the existing relationship triplet.

[0143] As an optional implementation manner, the determining module includes:

[0144] A sixth acquisition module, configured to acquire natural language explanations corresponding to all the relationship triples in the first rule base and the second rule base;

[0145] A seventh acquisition module is used to acquire the preset rule base according to the first rule base, the second rule base and the natural language interpretations corresponding to all the relationship triples.

[0146] As an optional implementation manner, the target rule further includes: a natural language explanation of the first relation triple and a natural language explanation of the newly inferred relation triple; the output module includes:

[0147] The first output submodule is used to output the first relation triple, the new inferred relation triple, the natural language explanation corresponding to the first relation triple, and the natural language explanation corresponding to the new inferred relation triple.

[0148] As an optional implementation manner, the determining module further includes:

[0149] A receiving module, used for receiving examples corresponding to each relation triple in the preset rule base;

[0150] A saving module is used to save the example into the preset rule base.

[0151] As an optional implementation, the inference module includes:

[0152] a fourth acquisition submodule, configured to acquire attribute information of entities included in the first relation triplet through a second largest language model;

[0153] The inference submodule is used to infer a new inferred relation triple contained in the first relation triple according to the target chapter document, attribute information of the entity included in the first relation triple and the target rule through the second large language model.

[0154] As an optional implementation,

[0155] A first acquisition submodule, specifically used to acquire the existing candidate relationship triples through a deep learning model;

[0156] A second acquisition submodule is specifically used to obtain the confidence value of each of the existing candidate relationship triples through the deep learning model;

[0157] The second acquisition subunit is specifically used to obtain the support score of the existing candidate relationship triple corresponding to the target confidence value through the deep learning model.

[0158] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store chapter-level relationship extraction data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a chapter-level relationship extraction method is implemented.

[0159] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0160] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0161] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0162] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0165] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0166] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A chapter-level relationship extraction method, characterized in that: The chapter-level relationship extraction method comprises: Obtain the target chapter document for extracting chapter-level relations; Acquire the first relation triple in the target chapter document; According to the relationship in the first relationship triplet, a target rule corresponding to the relationship is obtained in a preset rule library, wherein the target rule includes: a mapping relationship between the relationship and the newly inferred relationship, and attribute information of an entity in the newly inferred relationship; Inferring a new inferred relation triple contained in the first relation triple according to the target chapter document, the attribute information of the entity included in the first relation triple and the target rule; outputting the first relation triplet and the new inferred relation triplet; Before obtaining the target chapter document to be extracted from the chapter-level relationship, the method further includes: Get the existing relationship triples; Obtaining attribute information of each entity in the existing relationship triple; Generate a first rule base through the first large language model, wherein the first rule base includes: the second relation triplet obtained through the first large language model and attribute information of entities in the second relation triplet; The preset rule base is obtained according to the first rule base and the second rule base, wherein the second rule base includes: a mapping relationship between the existing relationship triples and the attribute information of each entity in the existing relationship triples; The step of obtaining the preset rule base according to the first rule base and the second rule base includes: Obtaining natural language explanations corresponding to all relationship triples in the first rule base and the second rule base; The preset rule base is obtained according to the natural language interpretations corresponding to the first rule base, the second rule base and all the relationship triples.

2. The chapter-level relationship extraction method according to claim 1, characterized in that: The step of generating a first rule base by using a first large language model includes: Training data is input into the first large language model so that the first large language model generates the first rule base according to the training data, wherein the training data includes: training documents, entities in the training documents, third relation triples in the training documents, and attribute information of each entity in the third relation triples.

3. The chapter-level relationship extraction method according to claim 1, characterized in that: The obtaining of the existing relationship triples includes: Obtain existing candidate relationship triples; Obtaining the confidence of each of the existing candidate relationship triples; The existing candidate relationship triplet corresponding to the target confidence is determined to be the existing relationship triplet, and the target confidence is a confidence among the confidences of the existing candidate relationship triplet that is greater than a preset confidence.

4. The chapter-level relationship extraction method according to claim 3 is characterized in that: The determining that the existing candidate relationship triplet corresponding to the target confidence is the existing relationship triplet includes: Acquire a target confidence level whose confidence level of the existing candidate relationship triples is greater than the preset confidence level; Obtaining a support score of the existing candidate relationship triple corresponding to the target confidence; Obtaining a target support score whose support score is greater than a preset score; The existing candidate relation triplet corresponding to the target support score is determined to be the existing relation triplet.

5. A chapter-level relationship extraction device, characterized in that: The chapter-level relationship extraction device comprises: The first acquisition module is used to acquire the target chapter document from which the chapter-level relationship is to be extracted; A second acquisition module, used for acquiring a first relation triple in the target chapter document; A third acquisition module, configured to acquire a target rule corresponding to the relationship in a preset rule base according to the relationship in the first relationship triplet, wherein the target rule includes: a mapping relationship between the relationship and the newly inferred relationship, and attribute information of an entity in the newly inferred relationship; An inference module, configured to infer a new inferred relation triple contained in the first relation triple according to the target chapter document, attribute information of entities included in the first relation triple and the target rule; An output module, used for outputting the first relation triple and the new inferred relation triple; The chapter-level relationship extraction device also includes: A fourth acquisition module, used for acquiring existing relationship triples before the first acquisition module acquires the target chapter document from which the chapter-level relationship is to be extracted; A fifth acquisition module, used to acquire attribute information of each entity in the existing relationship triple; A generating module, configured to generate a first rule base through a first large language model, wherein the first rule base includes: a second relation triplet obtained through the first large language model and attribute information of entities in the second relation triplet; A determination module, configured to obtain the preset rule base according to the first rule base and the second rule base, wherein the second rule base includes: a mapping relationship between the existing relationship triples and the attribute information of each entity in the existing relationship triples; The determining module comprises: A sixth acquisition module, configured to acquire natural language explanations corresponding to all the relationship triples in the first rule base and the second rule base; A seventh acquisition module is used to acquire the preset rule base according to the first rule base, the second rule base and the natural language interpretations corresponding to all the relationship triples.

6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the paragraph-level relationship extraction method described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the paragraph-level relationship extraction method described in any one of claims 1 to 4 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the paragraph-level relationship extraction method described in any one of claims 1 to 4 are implemented.

Images