Document key information extraction method based on large language model

Abstract

The invention discloses a document key information retrieval method based on a large language model, a retrieval result is output according to a multi-stage input document and a question input by a user, and the method comprises the following steps: constructing a domain knowledge graph; constructing and training a question extraction model, and obtaining key information and intention of the question input by the user through the question extraction model; retrieving in a knowledge graph according to the key information and the intention to form paragraph description of the key information; screening the document and splitting the document into a plurality of text blocks to obtain a plurality of text blocks most related to the content input by the user; obtaining a corresponding text block with the highest embedding similarity by using a large language model; and integrating and reasoning the corresponding text blocks and paragraph descriptions by adopting a large language model to obtain information about the user input question in the document. The method has the beneficial effects that the accuracy and efficiency of information extraction can be improved, specific requirements of different fields can be met, and powerful technical support is provided for document automatic processing and knowledge management.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for extracting key information from documents based on a large language model. Background Art

[0002] With the rapid development of information technology, the speed of document data generation and accumulation is accelerating, especially in the fields of business operations, scientific research, legal consulting, etc. These documents usually contain a large amount of unstructured text information, which contains rich knowledge and insights. However, how to effectively extract key information from massive documents to support decision-making, knowledge management and automated processing has always been an important challenge in the field of natural language processing (NLP).

[0003] Traditional document information extraction methods mainly rely on rule-based systems and simple keyword matching techniques. These methods work well when processing highly structured documents, but they are often incapable of dealing with complex, unstructured text data. In addition, with the advancement of artificial intelligence technology, especially the development of deep learning, machine learning-based methods have begun to be applied to document information extraction tasks. Although these methods can provide relatively accurate extraction results in some cases, they usually require a large amount of labeled data to train the model and have limited semantic understanding of documents.

[0004] In recent years, with the emergence of large-scale pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer), the field of natural language processing has ushered in revolutionary changes. These models are able to capture the deep semantic features of language by pre-training on large-scale text data, thereby achieving significant performance improvements in a variety of language tasks. However, despite the excellent performance of these models in text understanding, how to effectively apply them to the extraction of key information in documents, especially in specific fields, remains an open problem.

[0005] In specific fields, such as law, medicine, or finance, extracting key information from documents faces additional challenges. Documents in these fields often contain a large amount of professional terms and complex syntactic structures, which places higher demands on the domain adaptability and understanding ability of the model. In addition, document information extraction in these fields often needs to comply with specific specifications and standards, which further increases the difficulty of technical implementation. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for extracting key information from documents based on a large language model. The method aims to solve the limitations of the prior art in extracting key information from documents, especially how to utilize the powerful capabilities of large language models and combine them with domain knowledge graphs to achieve efficient and accurate extraction of key information from documents. The method is specifically implemented by the following technical solutions:

[0007] The document key information retrieval method based on the large language model outputs retrieval results according to the multi-level input documents and the user input questions, and the method includes:

[0008] Construct a document knowledge dataset, and use a multi-label pointer network annotation framework to extract entity and relationship vectors, and build a domain knowledge graph based on the vectors; construct a question extraction model based on a large language model, BiLSTM, and TextCNN, and train the question extraction model to obtain the key information and intent of the question entered by the user through the question extraction model;

[0009] Construct query statements based on key information and intent to conduct in-depth retrieval in the knowledge graph and form paragraph descriptions of key information;

[0010] At the same time, the document is screened and split into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user; and several text blocks F with the highest embedding similarity of the text blocks are obtained using a large language model. 3S ;

[0011] Adopting a large language model to integrate reasoning text blocks F 3S and paragraph description to get information about the user input question in the document.

[0012] The further design of the document key information retrieval method based on the large language model includes the construction of an input content extraction model and the acquisition of key information N and intent Intent, specifically including:

[0013] The user input is fed into a large language model to obtain the hidden state vector X of each input token and the semantic information T of the entire sentence. TextCNN is used to classify the semantic information T and classify it into pre-designed intent categories to identify the intent of the user query.

[0014] Perform semantic analysis on the hidden state vector X through BiLSTM to generate an output feature vector B;

[0015] Conditional random fields are introduced, and the BIO labeling method is used to model the global characteristics of the label sequence; the feature vector B output by BiLSTM is input into the fully connected layer, the label score of each position is calculated, and finally the label sequence with the highest score is selected as the key information N.

[0016] The further design of the document key information retrieval method based on the large language model is that the in-depth search in the knowledge graph is specifically as follows: Cql statements are constructed according to the key information N and the intent Intent, and a query is performed in the knowledge graph K to obtain triples T; the filtered triples T are integrated through the large language model to form a paragraph description P of the key information.

[0017] A further design of the document key information retrieval method based on a large language model is that the triple T is composed of entity names, relations and objects corresponding to the relations extracted through key information; the entity name represents the subject or concept or event or person to which the method is applied in the corresponding field.

[0018] A further design of the document key information retrieval method based on a large language model is that the triple T retrieved from the knowledge graph is input into the large language model through a prompt template.

[0019] The further design of the document key information retrieval method based on the large language model is to screen the document and split it into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user. Specifically, all documents F are retrieved by using the BM25 algorithm in combination with the question input by the user to obtain the most relevant K documents F1, and the document F1 is divided into blocks using LangChain to obtain the text block F corresponding to F1. 1S ; Use BM25 algorithm to select F again 1S The most relevant C text blocks F are obtained 2S .

[0020] The further design of the document key information retrieval method based on the large language model is that the acquisition process of the document F1 is specifically as follows:

[0021] Calculate the score of each document according to formula (1):

[0022]

[0023] Among them, Q represents the problem, q i represents the i-th word in Q, W i represents its own importance, and F represents the document to be matched; W is calculated according to formula (2): i ;

[0024]

[0025] Among them, FN represents the total number of documents to be matched, that is, the number of F, df i To include q i The total number of documents, for the set qi , including q i The more documents there are, the more likely the q i The less important; i Use and characterize q i Relevance to the document;

[0026] The top K texts with the highest scores are selected as the most relevant texts F1.

[0027] The document key information retrieval method based on the large language model is further designed to use the large language model to obtain several text blocks F with the highest embedding similarity of the text block. 3S The specific process is as follows:

[0028] Step 1) Use a large language model to train the text block F 2S Encode the query Q to obtain a corresponding vector representation, and combine the vectors to form a representation of the entire text block;

[0029] Step 2) Calculate the vector cosine similarity to measure the relevance between the text block and the query;

[0030] Step 3) Based on the calculated similarity scores, select the first Z text blocks F with the highest similarity 3S As the most relevant piece of information.

[0031] The present invention also provides an electronic device, comprising a memory, a processor and a computer program, wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the document key information retrieval method based on a large language model.

[0032] The present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement the document key information retrieval method based on a large language model.

[0033] The advantages of the present invention are as follows:

[0034] The document key information extraction method based on a large language model of the present invention constructs a document key information extraction framework based on a large language model. The present invention can not only improve the accuracy and efficiency of information extraction, but also adapt to the specific needs of different fields, and provide strong technical support for document automation processing and knowledge management.

[0035] This method achieves in-depth understanding and efficient processing of document content by integrating the advanced text analysis capabilities of large language models with the structured data of domain knowledge graphs. This combination enables the method to not only identify keywords and phrases in documents, but also understand their meaning and relationship in a specific context, thereby significantly improving the accuracy and relevance of key information extraction.

[0036] This method can adapt to various document types and domain-specific languages, and ensures accurate capture of professional terms and complex concepts through customized language model training and knowledge graph construction. This flexibility and adaptability greatly expands the application scope of the framework, enabling it to meet the needs of users in different industries and fields.

[0037] This method significantly reduces the workload of manual review and data collation through the automated key information extraction process, while speeding up information retrieval. This not only improves work efficiency, but also reduces operating costs, providing users with a cost-effective document processing solution.

[0038] The method of the present invention supports integration with existing document management systems and knowledge bases, so that the extracted key information can be easily further analyzed, shared and applied, thereby promoting the dissemination of knowledge and the formulation of decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of the multi-label pointer network annotation framework of the present invention.

[0040] Figure 2 This is a model diagram for extracting key information and intent of the present invention.

[0041] Figure 3 The flowchart of the method for extracting key information from a document based on a large language model of the present invention.

[0042] Figure 4 It is a schematic diagram of a specific example in the embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is described in detail below.

[0044] The document key information retrieval method based on the large language model of this embodiment outputs retrieval results according to the multi-level input documents and the questions input by the user, including:

[0045] Construct a document knowledge dataset, and use a multi-label pointer network annotation framework to realize the vector extraction of entities and relationships, extract the triple data X in the text, and construct a domain knowledge graph K based on X. This embodiment takes the application in the field of power grid engineering as an example. The above entities are specific technical indicators and economic indicators in the field of power grid engineering, and the relationship is the relationship between specific technical indicators and their values. The large language model LLM (M3) is used. Here, the large language model uses gpt-4o. Gpt-4o contains 1.8 trillion parameters and has very powerful semantic information. It can complete the task of supplementing the construction of the dataset. The constructed document knowledge dataset (triplets) is supplemented and polished to construct the format data X of question-key information (QN).

[0046] The question extraction model is built based on the large language model M4, BiLSTM, and TextCNN. The large language model here uses the open source qwen2-72b, and the LLM (M4) + BiLSTM + TextCNN joint extraction model is trained through X'. Through continuous iteration and optimization, the model can learn how to extract key information N from the user input question Q. The key information contains its information category; the key information and intent of the user input question are obtained through the question extraction model.

[0047] The construction of the input content extraction model and the acquisition of key information N and intent Intent specifically include:

[0048] The user input is input into the large language model to obtain the hidden state vector X of each input token and the semantic information T of the entire sentence. The input token refers to the word, phrase or punctuation mark in the input text.

[0049] TextCNN is used to classify the semantic information T and classify it into pre-designed intent categories to identify the intent of the user query.

[0050] The hidden state vector X is semantically analyzed through BiLSTM to fill the slots and normalize the entities to generate the output feature vector B.

[0051] Conditional random fields are introduced, and the global features of the label sequence are modeled through the BIO labeling method; the feature vector B output by BiLSTM is input into the fully connected layer, the label score of each position is calculated, and finally the label sequence with the highest score is selected as the key information N.

[0052] Then, query statements are constructed based on the key information and intent to conduct in-depth searches in the knowledge graph and form paragraph descriptions of the key information. Specifically: Cql statements are constructed based on the key information N and the intent Intent, and queries are performed in the knowledge graph K to obtain triples T; the filtered triples T are integrated through the large language model M1 to form paragraph descriptions P of the key information. Here, the large language model uses gpt-4o to integrate many triples and filter useful information descriptions. Strong semantic understanding is required. For example, in the power grid engineering data, for question Q, "In the following engineering documents, what is the building structure form in the technical indicators of each project?" The key information N extraction result is {"technical indicators": "building structure form"}, and the intent reasoning result is {"information retrieval"}

[0053] The triple T is composed of entity names, relations and objects corresponding to the relations extracted through key information; the entity name represents the subject, concept, event or person in the power grid field to which the method is applied. The entity represents the subject, concept, event or person in the field to which the method is applied; the object is used to represent the object specified in the entity and relation. The triple T retrieved from the knowledge graph in this embodiment is input into the large language model through the prompt template.

[0054] At the same time, the document is filtered and split into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user; the large language model M4 is used, and here qwen2-72b is used to compare the similarity calculation of multiple texts and questions. The open source qwen achieves good performance while ensuring speed and cost. Several text blocks F with the highest embedding similarity are obtained 3S .

[0055] Furthermore, the document is screened and split into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user. Specifically, all documents F are retrieved using the BM25 algorithm based on the question input by the user to obtain the most relevant K documents F1. LangChain is used to split document F1 into blocks to obtain the text block F corresponding to F1. 1S ; Use BM25 algorithm to select F again 1S The most relevant C text blocks F are obtained 2S .

[0056] The specific process of obtaining document F1 is as follows:

[0057] Calculate the score of each document according to formula (1):

[0058]

[0059] Among them, Q represents the problem, q irepresents the i-th word in Q, W i represents its own importance, and F represents the document to be matched; W is calculated according to formula (2): i ;

[0060]

[0061] Among them, FN represents the total number of documents to be matched, that is, the number of F, df i To include q i The total number of documents, for the set q i , including q i The more documents there are, the more likely the q i The less important; i Use and characterize q i Relevance to the document;

[0062] The top K texts with the highest scores are selected as the most relevant texts F1.

[0063] Use BM25 again to select F 1S The most relevant C text blocks F are obtained 2S ; Compute question Q and text block F 1S The process is the same as above and will not be repeated here.

[0064] In this embodiment, the large language model M4 is used to obtain several text blocks F with the highest embedding similarity of the text blocks. 3S The specific process is as follows:

[0065] Step 1) Use a large language model to train the text block F 2S Encode it with the query Q to get the corresponding vector representation, and combine the vectors into the representation of the entire text block;

[0066] Step 2) Calculate the vector cosine similarity to measure the relevance between the text block and the query;

[0067] Step 3) Based on the calculated similarity scores, select the first Z text blocks F with the highest similarity 3S As the most relevant piece of information.

[0068] Use a large language model M2 to integrate the reasoning text block F 3S and paragraph description to get information about the user input question in the document.

[0069] Combination Figure 4This embodiment provides a specific example. Three documents, "Yigou 110 kV Substation New Construction Project", "Wanfu 110 kV Substation New Construction Project" and "Qianshuang 110 kV Substation New Construction Project" are input at the same time. The documents are screened and split into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user; the large language model is used to obtain several text blocks with the highest embedding similarity of the text blocks. See Figure 4 At the same time, according to the input question "Please find the building structure form corresponding to each engineering project file in each project", the key information "technical indicators: building structure form" and the intention of the question input by the user are obtained through the question extraction model. According to the key information and intention, the query statement is constructed to perform in-depth retrieval in the knowledge graph, and the triple T is obtained and a paragraph description of the key information is formed. Figure 4 The right half of ; a large language model is used to integrate the reasoning text block and paragraph description, and output information about the user input question in the document, see Figure 4 The present invention also provides an electronic device, comprising a memory, a processor and a computer program, wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the document key information retrieval method based on a large language model.

[0070] The present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to implement the document key information retrieval method based on a large language model.

[0071] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A document key information retrieval method based on a large language model, outputting retrieval results based on multi-level input documents and user input questions, characterized in that The method includes: Construct a document knowledge dataset, and use a multi-label pointer network annotation framework to extract entity and relationship vectors, and build a domain knowledge graph based on the vectors; construct a question extraction model based on a large language model, BiLSTM, and TextCNN, and train the question extraction model to obtain the key information and intent of the question entered by the user through the question extraction model; Construct query statements based on key information and intent to conduct in-depth retrieval in the knowledge graph and form paragraph descriptions of key information; At the same time, the document is screened and split into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user; and several text blocks F with the highest embedding similarity of the text blocks are obtained using a large language model. 3S ; Adopting a large language model to integrate reasoning text blocks F 3S and paragraph description to get information about the user input question in the document.

2. The document key information retrieval method based on a large language model according to claim 1 is characterized in that Including the construction of input content extraction model and the acquisition of key information N and intent Intent, specifically including: The user input is fed into a large language model to obtain the hidden state vector X of each input token and the semantic information T of the entire sentence. TextCNN is used to classify the semantic information T and classify it into pre-designed intent categories to identify the intent of the user query. Perform semantic analysis on the hidden state vector X through BiLSTM to generate an output feature vector B; Conditional random fields are introduced, and the BIO labeling method is used to model the global characteristics of the label sequence; the feature vector B output by BiLSTM is input into the fully connected layer, the label score of each position is calculated, and finally the label sequence with the highest score is selected as the key information N.

3. The document key information retrieval method based on a large language model according to claim 1 is characterized in that The specific steps for in-depth retrieval in the knowledge graph are as follows: construct a Cql statement based on the key information N and the intent Intent, query the knowledge graph K, and obtain the triple T; integrate the filtered triples T through the large language model to form a paragraph description P of the key information.

4. The document key information retrieval method based on a large language model according to claim 1 is characterized in that The triplet T is composed of entity names, relations and objects corresponding to the relations extracted through key information; the entity name represents the subject or concept or event or person to which the method is applied in the corresponding field.

5. The document key information retrieval method based on a large language model according to claim 4 is characterized in that The triples T retrieved from the knowledge graph are input into the large language model through the prompt template.

6. The document key information retrieval method based on a large language model according to claim 1 is characterized in that The document is screened and split into multiple text blocks to obtain several text blocks that are most relevant to the content input by the user. Specifically, all documents F are retrieved using the BM25 algorithm based on the question input by the user to obtain the most relevant K documents F1. The document F1 is divided into blocks using LangChain to obtain the text block F corresponding to F1. 1S ; Use BM25 algorithm to select F again 1S The most relevant C text blocks F are obtained 2S .

7. The document key information retrieval method based on a large language model according to claim 5 is characterized in that The specific process of obtaining document F1 is as follows: Calculate the score of each document according to formula (1): Among them, Q represents the problem, q i represents the i-th word in Q, W i represents its own importance, and F represents the document to be matched; W is calculated according to formula (2): i ; Among them, FN represents the total number of documents to be matched, that is, the number of F, df i To include q i The total number of documents, for the set q i , including q i The more documents there are, the more likely the q i The less important; i Use and characterize q i Relevance to the document; The top K texts with the highest scores are selected as the most relevant texts F1.

8. The document key information retrieval method based on a large language model according to claim 5 is characterized in that Use the large language model to obtain several text blocks F with the highest embedding similarity of the text block 3S The specific process is as follows: Step 1) Use a large language model to train the text block F 2S Encode the query Q to obtain a corresponding vector representation, and combine the vectors to form a representation of the entire text block; Step 2) Calculate the vector cosine similarity to measure the relevance between the text block and the query; Step 3) Based on the calculated similarity scores, select the first Z text blocks F with the highest similarity 3S As the most relevant piece of information.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program, wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the document key information retrieval method based on a large language model as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and the computer program is executed by a processor to implement the document key information retrieval method based on a large language model according to any one of claims 1 to 8.

Images