Large model-based automatic knowledge extraction method and device, and storage medium

Abstract

The present invention relates to the technical field of artificial intelligence, and relates to a large model-based automatic knowledge extraction method and device, and a storage medium. The method comprises: constructing an automatic knowledge extraction training data set, wherein a basic data set is preprocessed, and a knowledge extraction analysis process is added into the basic data set to form the automatic knowledge extraction training data set; building an automatic knowledge extraction model, wherein the automatic knowledge extraction model has a base model, and a relation extraction module, a subject extraction module, and a fact extraction module embedded into the base model; fine-tuning the relation extraction module, the subject extraction module, and the fact extraction module separately by using training data in the automatic knowledge extraction training data set; and performing knowledge extraction on a segment of text by using the fine-tuned automatic knowledge extraction model. A knowledge extraction analysis process is incorporated into the knowledge extraction process, so that the extracted knowledge has strong interpretability, enabling completion of knowledge extraction tasks for document-level text and facilitating continuous expansion of the model in later stages.

Description

A large model-based automatic knowledge extraction method, device and storage medium Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to a knowledge extraction method, device and storage medium, and in particular to an automatic knowledge extraction method, device and storage medium based on a large model. Background Art

[0002] The core concept of traditional text-based knowledge extraction systems is label classification, such as using the Bert model to classify entities or relationships between entities. However, traditional knowledge extraction systems based on label classification lack interpretability of the extracted knowledge and generalizability to text in different contexts. Once these knowledge extraction systems are applied to different corpora, their performance can be significantly degraded.

[0003] Large language models (LLMs), such as GPT-4, glm, and llama, are developing rapidly, and their logical analysis capabilities have been greatly improved, making it possible to directly extract knowledge through such large language models. However, current knowledge extraction systems based on large language models still lack the ability to explain the extracted knowledge. At the same time, there are still shortcomings in document-level multi-knowledge extraction, and large language models rely on templates during the knowledge extraction process, which have many limitations. For example, in relationship extraction tasks, the choice of relationship is typed into the template, resulting in a small number of insertable relationship options. Finally, the current knowledge extraction model is difficult to expand later, which is not conducive to rapid performance improvement of the model.

[0004] Therefore, in order to address the defects in the above-mentioned existing technologies, it is necessary to develop a new automatic knowledge extraction method based on large models.

[0005] Summary of the Invention

[0006] In order to overcome the defects of the existing technology, the present invention proposes an automatic knowledge extraction method, device and storage medium based on a large model, which adds a knowledge extraction analysis process to the knowledge extraction process, so that the extracted knowledge has strong explanatory power and can complete the knowledge extraction task of document-level text, while making the automatic knowledge extraction model easy to continuously expand in the later stage.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] An automatic knowledge extraction method based on a large model, characterized by comprising the following steps:

[0009] Constructing an automatic knowledge extraction training data set: preprocessing a basic data set, and adding a knowledge extraction analysis process to the preprocessed basic data set to form the automatic knowledge extraction training data set;

[0010] Building an automatic knowledge extraction model, the automatic knowledge extraction model comprising a basic model and a relationship extraction module, a subject extraction module, and a fact extraction module embedded in the basic model;

[0011] Fine-tuning the automatic knowledge extraction model, using the training data in the automatic knowledge extraction training dataset to fine-tune the relationship extraction module, the subject extraction module, and the fact extraction module respectively;

[0012] The fine-tuned automatic knowledge extraction model is used to extract knowledge from a text.

[0013] Preferably, the preprocessing basic data set specifically includes:

[0014] Use the existing document-level relation extraction dataset as the basic dataset;

[0015] Cleaning and enhancing the basic data set;

[0016] The cleaned and enhanced basic data set is subjected to a balancing of the number of relations to obtain the preprocessed basic data set.

[0017] Preferably, when balancing the number of relationships in the cleaned and enhanced basic data set, a total of 96 relationships are determined, and the number of data in each relationship is 1000.

[0018] Preferably, the knowledge extraction and analysis process is obtained by manually annotating the preprocessed training data in the basic data set, and querying the large language model using the manually annotated training data, and generating the knowledge extraction and analysis process from the large language model.

[0019] Preferably, a LoRA model is respectively used as the relationship extraction module, the subject extraction module and the fact extraction module.

[0020] Preferably, when fine-tuning the automatic knowledge extraction model, the training data in the automatic knowledge extraction training data set is divided into three parts, and the relationship extraction module, subject extraction module and fact extraction module are fine-tuned respectively using an efficient parameter fine-tuning method.

[0021] Preferably, extracting knowledge from a text using the automatic knowledge extraction model specifically includes:

[0022] Inputting a text into the automatic knowledge extraction model, and extracting all relations in the text by the relation extraction module;

[0023] Inputting each relationship extracted by the relationship extraction module and the text into the automatic knowledge extraction model, and extracting the subject in each relationship by the subject extraction module;

[0024] Input each relationship extracted by the relationship extraction module, the subject in each relationship extracted by the subject extraction module, and the text into the automatic knowledge extraction model, and extract the facts in each relationship by the fact extraction module and generate a knowledge extraction analysis process corresponding to each relationship;

[0025] Then, each relationship extracted by the relationship extraction module, the subject in each relationship extracted by the subject extraction module, the fact in each relationship extracted by the fact extraction module, the knowledge extraction analysis process corresponding to each generated relationship, and the text are input into the automatic knowledge extraction model, and the automatic knowledge extraction model forms a knowledge triple containing the relationship, the subject and the fact for each relationship.

[0026] In addition, the present invention also provides an automatic knowledge extraction device based on a large model, which is characterized by comprising:

[0027] one or more processors;

[0028] a memory for storing one or more programs;

[0029] When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the large model-based automatic knowledge extraction method as described above.

[0030] Finally, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps in the large model-based automatic knowledge extraction method as described above are implemented.

[0031] Compared with the prior art, the large model-based automatic knowledge extraction method, device, and storage medium of the present invention have one or more of the following beneficial technical effects:

[0032] 1. The present invention adds a knowledge extraction analysis process to the knowledge extraction process, so that the model has a strong interpretation ability for the extracted knowledge and can complete the knowledge extraction task of document-level text.

[0033] 2. The present invention redesigns the process of knowledge extraction. According to the human thinking mode, it determines the three-step extraction method of relationship-subject-fact to ensure the accuracy of knowledge extraction.

[0034] 3. The present invention adopts the parameter-efficient model training idea and uses the LoRA model to implement the three extraction processes with separate LoRA models, which facilitates further improvement of the model performance in the later stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a flow chart of the large model-based automatic knowledge extraction method of the present invention. DETAILED DESCRIPTION

[0036] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components set forth in the following description or illustrated in the following figures. The present invention is capable of other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used herein are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and their variations herein is intended to cover the items and their equivalents set forth below and additional items. Unless otherwise specified or limited, the terms "mounted", "connected", "supported" and "coupled" and their variations are used broadly and cover direct mounting and indirect mounting, connection, support and coupling. In addition, "connected" and "coupled" are not limited to physical or mechanical connections or couplings.

[0037] Furthermore, on the first hand, in the disclosure of the present invention, the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like to indicate orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore the above terms cannot be understood as limitations on the present invention; on the second hand, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" cannot be understood as a limitation on the quantity.

[0038] Traditional knowledge extraction systems based on label classification ideas lack the ability to explain the extracted knowledge and lack generalization for texts in different contexts. Knowledge extraction systems based on generative models still lack the ability to explain the extracted knowledge and have shortcomings in document-level multi-knowledge extraction. In addition, generative models rely on templates in the knowledge extraction process, and these templates have many limitations. For example, in relationship extraction tasks, the choice of relationships is typed into the template, resulting in a small number of relationships that can be inserted. Finally, the current knowledge extraction model is difficult to expand in the later stage, which is not conducive to rapid performance improvement of the model. To this end, the present invention designs an automatic knowledge extraction method, device and storage medium based on a large model, which adds a knowledge extraction analysis process to the knowledge extraction process, so that the model has a strong ability to explain the extracted knowledge and can complete the knowledge extraction task of document-level text. At the same time, the model is easy to expand in the later stage.

[0039] Figure 1 shows a flow chart of the automatic knowledge extraction method based on a large model of the present invention. As shown in Figure 1, the automatic knowledge extraction method based on a large model of the present invention includes the following steps:

[0040] 1. Construct an automatic knowledge extraction training dataset.

[0041] In the present invention, an existing knowledge extraction dataset is used as a basic dataset. Specifically, an open source knowledge extraction dataset, such as fewrel, wiki, NYT, SciERC, trex, GIDS, etc., can be used as a basic dataset.

[0042] Preferably, the present invention uses the Redocred dataset as a basic dataset. The Redocred dataset is a document-level knowledge extraction dataset, which contains multiple pieces of knowledge in a long text.

[0043] At the same time, considering that some low-quality data exists in the basic dataset, the present invention requires data cleaning and enhancement of the basic dataset. For example, text that is too short or too long is removed. Furthermore, partial knowledge is automatically supplemented. For example, "has part" and "part of" are opposite relationships. If one is detected in the text but the other is not retrieved, the other can be directly supplemented.

[0044] Furthermore, considering the imbalance in the distribution of data quantities across relationships in the basic dataset, and the significant disparity in data quantities between some relationships, which can affect subsequent training results, the present invention balances the number of relationships in the cleaned and enhanced basic dataset. This balancing results in a total of 96 relationships in the balanced basic dataset, with approximately 1,000 data items per relationship.

[0045] In order to make the extraction results explainable, the present invention adds a knowledge extraction analysis process to the pre-processed basic data set to form the automatic knowledge extraction training data set.

[0046] In the present invention, the knowledge extraction and analysis process is obtained by manually annotating the training data in the preprocessed basic data set, and querying the large language model using the manually annotated training data, and generating the knowledge extraction and analysis process from the large language model.

[0047] Specifically, the present invention uses ChatGPT3.5-turbo to generate the knowledge extraction and analysis process. That is, ChatGPT3.5-turbo is queried using a designed prompt template using manually annotated training data, and ChatGPT3.5-turbo generates a knowledge extraction and analysis process for the training data, which serves as the analysis basis for the knowledge extraction process of the training data.

[0048] 2. Build an automatic knowledge extraction model.

[0049] The automatic knowledge extraction model comprises a basic model and a relationship extraction module, a subject extraction module and a fact extraction module embedded in the basic model.

[0050] Since the large language model vicuna-7b-v1.5 can achieve a good balance between the number of parameters and the effect, in the present invention, the large language model vicuna-7b-v1.5 can be used as the basic model.

[0051] At the same time, in the present invention, when performing knowledge extraction, the thinking process of people in data annotation is imitated to determine the knowledge extraction model of relationship-entity (i.e., subject)-fact. That is, after a given text, all the relationships contained in the text are first determined. Then, for each relationship, the entities that it satisfies that can serve as the subject are determined. Finally, after determining the relationship and subject, the facts are extracted from the text. Therefore, in the present invention, a relationship extraction module, a subject extraction module, and a fact extraction module are embedded in the basic model to respectively realize the extraction of relationships, subjects, and facts.

[0052] Moreover, the relationship extraction module, subject extraction module and fact extraction module are embedded in the basic model, so that the automatic knowledge extraction model can be trained efficiently and is convenient for subsequent expansion and improvement.

[0053] At the same time, in the present invention, a LoRA model is respectively used as the relationship extraction module, the subject extraction module and the fact extraction module, thereby facilitating the training and fine-tuning of the relationship extraction module, the subject extraction module and the fact extraction module.

[0054] 3. Fine-tune the automatic knowledge extraction model.

[0055] After the automatic knowledge extraction model is built, the training data in the automatic knowledge extraction training data set can be used to fine-tune the relationship extraction module, the subject extraction module, and the fact extraction module respectively.

[0056] At the same time, when fine-tuning the automatic knowledge extraction model, the training data in the automatic knowledge extraction training dataset is divided into three parts, and the parameter efficient fine-tuning method (PEET) is used to fine-tune the relationship extraction module, subject extraction module, and fact extraction module respectively. In this way, since each module uses the LoRA model and each module is trained separately, to further improve the knowledge extraction performance of a particular module, only the data of that module can be modified and the LoRA model of that module can be further adjusted. For example, to improve the subject extraction performance of the country of citizenship, the corresponding data can be added to the subject extraction module and further parameters can be adjusted without affecting the relationship extraction of the relationship extraction module or the fact extraction of the fact extraction module. This accelerates the training of the automatic knowledge extraction model and avoids significantly affecting the model's performance.

[0057] 4. Use the fine-tuned automatic knowledge extraction model to extract knowledge from a text.

[0058] After fine-tuning the automatic knowledge extraction model, the fine-tuned automatic knowledge extraction model can be used to extract knowledge from a text.

[0059] In the present invention, the automatic knowledge extraction model is used to extract knowledge from a text, specifically including the following process:

[0060] 1. Input a text into the automatic knowledge extraction model, and extract all the relations in the text by the relation extraction module.

[0061] Thus, a series of relationship lists can be extracted. Moreover, each relationship is processed separately to prevent all relationships from being unable to proceed to the next step due to a logical problem in a certain extracted relationship.

[0062] 2. Each relationship extracted by the relationship extraction module and the text are input into the automatic knowledge extraction model, and the subject extraction module extracts the subject in each relationship.

[0063] Thus, each relationship in the text can be processed separately, and its corresponding subject can be extracted to form a relationship-subject pair.

[0064] 3. Each relationship extracted by the relationship extraction module, the subject in each relationship extracted by the subject extraction module, and the text are input into the automatic knowledge extraction model, and the fact extraction module extracts the facts in each relationship and generates the knowledge extraction analysis process corresponding to each relationship.

[0065] Thus, for each relationship in the text, the facts in the relationship and the knowledge extraction analysis process corresponding to the relationship can be generated, thereby increasing the explainability of knowledge extraction.

[0066] 4. Each relationship extracted by the relationship extraction module, the subject in each relationship extracted by the subject extraction module, the fact in each relationship extracted by the fact extraction module, the knowledge extraction analysis process corresponding to each generated relationship, and the text are input into the automatic knowledge extraction model. The automatic knowledge extraction model extracts a knowledge triple containing the relationship, the subject, and the fact for each relationship. The knowledge triple is the knowledge extracted from the text.

[0067] For example, if a text "The Museo de Rossini is located in the center of Kifisia, a northern suburb of Athens... The museum was founded in 1997 to preserve and promote the multidimensional works of Rossini (1859-1951)" is input into the automatic knowledge extraction model, the relationship extraction module will first extract all the relationships in the text, including "date of birth", "date of death", "date of establishment", etc.

[0068] Next, one of the relations, for example, "date of birth" and the text are input into the automatic knowledge extraction model, and the subject extraction module extracts the subject "De Rossini" of the relation "date of birth".

[0069] Then, the relationship (i.e., "date of birth"), the subject (i.e., "De Rossini"), and the text are input into the automatic knowledge extraction model. The fact extraction module extracts the fact "De Rossini was born in 1959" of the relationship "date of birth" and generates the knowledge extraction analysis process corresponding to the relationship "date of birth": "Fact analysis: Based on the subject "De Rossini" and the relationship "date of birth", the fact is that De Rossini was born in 1959. This information comes from the following conversation: The museum was founded in 1997 to preserve and promote the multidimensional works of De Rossini (1859-1951). The conversation also mentions that De Rossini lived in the De Rossini Museum for a considerable period of time, indicating that he had a close connection with the institution. Therefore, based on the information provided, it can be concluded that Georgios De Rossini's date of birth is 1859."

[0070] Finally, the relationship, subject, fact, knowledge extraction analysis process and the text are input into the automatic knowledge extraction model, and the fact extraction module extracts the triple "['De Rossini', 'Date of Birth', '1859']".

[0071] Of course, as needed, the knowledge extracted from the text may also include the knowledge extraction and analysis process, thereby increasing the interpretability of knowledge extraction.

[0072] The present invention also relates to a large-model-based automatic knowledge extraction device, comprising: one or more processors; and a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the large-model-based automatic knowledge extraction method described above.

[0073] Moreover, the present invention also relates to a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps in the large model-based automatic knowledge extraction method as described above are implemented.

[0074] Finally, it should be noted that the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art may, based on the principles of the present invention, modify or replace the technical solutions of the present invention with equivalents without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A large model-based automatic knowledge extraction method, characterized in that: The following steps are involved: Constructing an automatic knowledge extraction training data set: preprocessing a basic data set, and adding a knowledge extraction analysis process to the preprocessed basic data set to form the automatic knowledge extraction training data set; Building an automatic knowledge extraction model, the automatic knowledge extraction model comprising a basic model and a relationship extraction module, a subject extraction module, and a fact extraction module embedded in the basic model; Fine-tuning the automatic knowledge extraction model, using the training data in the automatic knowledge extraction training dataset to fine-tune the relationship extraction module, the subject extraction module, and the fact extraction module respectively; The fine-tuned automatic knowledge extraction model is used to extract knowledge from a text.

2. The large model-based automatic knowledge extraction method according to claim 1, characterized in that: The preprocessing basic data set specifically includes: Use the existing document-level relation extraction dataset as the basic dataset; Cleaning and enhancing the basic data set; The cleaned and enhanced basic data set is subjected to a balancing of the number of relations to obtain the preprocessed basic data set.

3. The large model-based automatic knowledge extraction method according to claim 2, characterized in that: When balancing the number of relationships in the cleaned and enhanced basic data set, a total of 96 relationships were determined, and the number of data in each relationship was 1,000.

4. The large model-based automatic knowledge extraction method according to claim 3 is characterized in that: The knowledge extraction and analysis process is obtained by manually annotating the training data in the preprocessed basic data set, and querying the large language model using the manually annotated training data, and generating the knowledge extraction and analysis process from the large language model.

5. The large model-based automatic knowledge extraction method according to any one of claims 1 to 4, characterized in that: A LoRA model is respectively used as the relationship extraction module, the subject extraction module and the fact extraction module.

6. The large model-based automatic knowledge extraction method according to claim 5, characterized in that: When fine-tuning the automatic knowledge extraction model, the training data in the automatic knowledge extraction training data set is divided into three parts, and the relationship extraction module, subject extraction module and fact extraction module are fine-tuned respectively using an efficient parameter fine-tuning method.

7. The large model-based automatic knowledge extraction method according to claim 6, characterized in that: Using the automatic knowledge extraction model to extract knowledge from a text specifically includes: Inputting a text into the automatic knowledge extraction model, and extracting all relations in the text by the relation extraction module; Inputting each relationship extracted by the relationship extraction module and the text into the automatic knowledge extraction model, and extracting the subject in each relationship by the subject extraction module; Input each relationship extracted by the relationship extraction module, the subject in each relationship extracted by the subject extraction module, and the text into the automatic knowledge extraction model, and extract the facts in each relationship by the fact extraction module and generate a knowledge extraction analysis process corresponding to each relationship; Then, each relationship extracted by the relationship extraction module, the subject in each relationship extracted by the subject extraction module, the fact in each relationship extracted by the fact extraction module, the knowledge extraction analysis process corresponding to each generated relationship, and the text are input into the automatic knowledge extraction model, and the automatic knowledge extraction model forms a knowledge triple containing the relationship, the subject and the fact for each relationship.

8. An automatic knowledge extraction device based on a large model, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the large model-based automatic knowledge extraction method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the large model-based automatic knowledge extraction method as described in any one of claims 1 to 7 are implemented.

Images