Multi-level electronic document content organization method

Abstract

The invention discloses a multi-level electronic document content organization method, and relates to the field of document content organization. According to the method, the technology of constructing the multi-level electronic document vector knowledge graph by using the large language model is used, and the large language model document analysis understanding capability is finely adjusted, so that the multi-level electronic document can be analyzed and understood according to document titles, sub-titles, paragraphs, number lists and contents such as pictures, tables and formulas related to the document paragraphs; structured intelligent splitting extraction is carried out; storing the extracted data in a graph database in a graph form; and finally, storing the data of each graph node in a vector database in a vectorization manner by using a vectorization tool, and storing the corresponding relationship of the knowledge graph nodes in each piece of vector data. And finally, a data storage structure with a vectorization form and knowledge graph storage is formed. The constructed result can serve as one of input sources of a retrieval enhancement generation technology knowledge base, and external knowledge is more effectively quoted to solve the large language model illusion problem.

Description

Technical Field

[0001] The present invention relates to the field of document content organization, and in particular to a multi-level electronic document content organization method. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] With the rapid development of information technology, digitalization has become an inevitable trend in all walks of life. In the office field, digitalization means converting paper documents into electronic formats, and electronic documents are mostly composed of multi-level electronic documents, that is, multi-level titles, sublists and paragraphs, as well as unstructured files composed of tables, pictures, formulas and other contents in paragraphs. There are usually two ways to use and store such files. One way is to store them in a file system or object storage system, such as NFS, NAS or OSS, and then use the big data processing tool Elasticsearch to provide data analysis and search. Another way is to use natural language processing tools or models such as NLTK, spaCy, BERT, etc. to extract key entity objects, attributes, relationships, and event information from the original text, store and display them in the form of knowledge graphs, and use graph computing languages ​​to query and analyze data.

[0004] The above two methods of processing text electronic documents also have some disadvantages:

[0005] (1) The file system or object storage system stores complete files, which makes it difficult to accurately query and quote when you need to extract article fragments;

[0006] (2) When using the Elasticsearch search tool to query file content, it can only be based on keyword queries, and cannot be queried from semantic representations, which lacks intelligence;

[0007] (3) When using natural language processing tools or models to extract text content, it is necessary to train a special extraction model, and the extraction results will have errors;

[0008] (4) When converting the original electronic document data into a knowledge graph, a lot of contextual information will be lost, and a large amount of text semantic information will also be lost. At the same time, only key entities, relationships, and events can be stored through knowledge graph queries, and the original document content information is missing.

[0009] (5) As a knowledge base of Retrieval-Augmented Generation, its storage and retrieval are not sophisticated enough to find accurate and comprehensive relevant knowledge for queries.

[0010] In summary, a new method is needed to organize multi-level electronic document content, which can not only realize the query and reference capabilities of local fragments of document content, but also fully retain the context or paragraph semantics of the original document; at the same time, a title or a paragraph of text can be associated with related context content, pictures, tables and other information; and the stored data can be conveniently used as an external knowledge base for retrieval-augmented generation to achieve efficient, accurate and comprehensive data retrieval services. Summary of the invention

[0011] The purpose of the present invention is to provide a multi-level electronic document content organization method in response to the problems existing in the prior art, and a technology for constructing a multi-level electronic document vector knowledge graph using a large language model. By fine-tuning the document analysis and understanding ability of the large language model, it can intelligently split and extract the multi-level electronic document according to the document title, subtitle, paragraph, number list, and pictures, tables, formulas and other contents related to the document paragraphs in a structured manner; then store the extracted data in the form of a graph to a graph database; finally, use a vectorization tool to vectorize the data of each graph node and store it in a vector database, and save the corresponding relationship of the knowledge graph nodes in each vector data; finally, a data storage structure of vectorization form plus knowledge graph storage is formed; the constructed result can be used as one of the input sources of the retrieval-augmented generation technology knowledge base, and more effectively reference external knowledge to solve the large language model hallucination problem.

[0012] The technical solution of the present invention is as follows:

[0013] A multi-level electronic document content organization method, comprising:

[0014] Step S1: fine-tune the classification and analysis capabilities of the large language model;

[0015] Step S2: Input the electronic document for which the knowledge graph needs to be constructed into the fine-tuned large language model to construct a multi-document structure graph;

[0016] Step S3: Based on the multi-document structure graph constructed in step S2, complete the graph vector space construction.

[0017] Furthermore, the step S1 comprises:

[0018] Step S11: fine-tune data annotation; complete the process of converting electronic documents from unstructured to structured, and form the format of large model fine-tune data;

[0019] Step S12: fine-tuning the large language model; using the pre-trained large model Qwen2.5-32B-Instruct as the basic model, adopting supervised fine-tuning, and enabling the LoRA fine-tuning method to improve the efficiency of fine-tuning; through repeated evaluation and parameter adjustment, it has a good output effect in the task of splitting electronic documents.

[0020] Furthermore, the step S11 includes:

[0021] Step S111: performing data cleaning on the electronic document to clean out high-quality multi-level electronic documents that can be annotated;

[0022] Step S112: marking the contents of the multi-level electronic document.

[0023] Furthermore, the step S2 comprises:

[0024] Step S21: electronic document content splitting: electronic documents for which knowledge graphs need to be constructed are sequentially input into the trained large language model, the content in the electronic documents is split, and the content is output to a table for storage;

[0025] Step S22: review the split content; review and correct the erroneous data in the split content;

[0026] Step S23: constructing data relationships between single documents; reconstructing the relationship between data and data, parsing the content of each split electronic document into a tree-like relationship diagram structure for storage;

[0027] Step S24: constructing relationships between multiple documents; establishing similarity relationships between multiple electronic documents to complete the source data construction of the multi-document knowledge graph;

[0028] Step S25: knowledge graph construction; formatting the source data of the multi-document knowledge graph, and finally forming a knowledge graph representation of multiple electronic documents, while also establishing corresponding relationships between multiple documents.

[0029] Furthermore, the step S22 includes:

[0030] Manual correction is required for erroneous data. Data that is too long needs to be segmented into pieces no longer than 4,000 characters, and the relationships between the segmented data are annotated to provide support for the subsequent establishment of graph relationships.

[0031] Furthermore, the step S24 includes:

[0032] Use the encoding model bge-large-zh-v1.5-v2 to cross-traverse each tree node between multiple documents, and use the Yuxuan similarity to calculate the semantic similarity with other nodes. The value is used as the weight directly associated with the node; and set a weight threshold. All relationships with similarity less than the weight threshold are discarded.

[0033] Furthermore, the step S25 includes:

[0034] Use the Neo4j graph database to import data, and ultimately form a knowledge graph representation of multiple electronic documents. At the same time, corresponding relationships are established between multiple documents.

[0035] Furthermore, the step S3 comprises:

[0036] Step S31: vectorizing graph nodes;

[0037] Step S32: Use of vector knowledge graph.

[0038] Furthermore, the step S31 includes:

[0039] Use the language encoding model bge-large-zh-v1.5-v2 to encode each node in the knowledge graph, use the vector database to store the node vector representation, and associate the node as metadata with the node vector.

[0040] Furthermore, the step S32 includes:

[0041] Step S321: In the input phase, the TOP-K content most similar to the input topic is searched in the vector database using the encoding model bge-large-zh-v1.5-v2 according to the topic input by the user; the corresponding node in the knowledge graph is found through the vector database entry of the query result;

[0042] Step S322: For each node of TOP-K, use the breadth-first algorithm in the Neo4j graph database to find the original data related to the node in the knowledge graph;

[0043] Step S323: After the above steps, the original information related to the document corresponding to the input topic can be quickly queried, and finally the document query results are reorganized, which can be used as an external knowledge base generated by retrieval enhancement.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention trains a large language model through fine-tuning to complete the hierarchical splitting task of multi-level electronic documents, and finally converts a single unstructured electronic document into structured storage represented by a tree structure.

[0046] 2. The present invention defines the node relationship through the tree structure of a single electronic document and converts it into a knowledge graph storage format. Then, the encoding model is used to calculate the similarity between the nodes of multiple electronic documents, and a relative weight relationship is established between the nodes of multiple electronic documents, and finally multiple electronic documents are converted into a knowledge graph representation.

[0047] 3. The present invention uses the encoding model to encode the constructed knowledge graph nodes and stores the vector representation of the nodes in the vector database. After the user enters the query topic, the vector database first quickly finds the TOP-K similar nodes from the semantic similarity, and then finds the original data related to the node in the knowledge graph through the node, and finally reorganizes the document query results, which can be used as an external knowledge base for retrieval-augmented generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a multi-level electronic document content organization method. DETAILED DESCRIPTION

[0049] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0050] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0051] Embodiment 1

[0052] See also Figure 1 , a multi-level electronic document content organization method, specifically comprising:

[0053] Step S1: fine-tune the classification and analysis capabilities of the large language model;

[0054] Step S2: Input the electronic document for which the knowledge graph needs to be constructed into the fine-tuned large language model to construct a multi-document structure graph;

[0055] Step S3: Based on the multi-document structure graph constructed in step S2, complete the graph vector space construction.

[0056] In this embodiment, specifically, step S1 includes:

[0057] Step S11: fine-tune data annotation; complete the process of converting electronic documents from unstructured to structured, and form the format of large model fine-tune data;

[0058] Step S12: fine-tuning the large language model; using the pre-trained large model Qwen2.5-32B-Instruct as the basic model, adopting supervised fine-tuning (Supervised Fine-Tuning), and enabling the LoRA (Low-Rank Adaptation) fine-tuning method to improve the efficiency of fine-tuning; through repeated evaluation and parameter adjustment, it has a good output effect in the task of splitting electronic documents.

[0059] In this embodiment, specifically, step S11 includes:

[0060] Step S111: clean the electronic document to obtain a high-quality multi-level electronic document that can be annotated; that is, before fine-tuning the data annotation, first use a data cleaning tool or a manual cleaning method to clean up a high-quality multi-level electronic document that can be annotated;

[0061] Step S112: Annotate the contents of multi-level electronic documents; it should be noted that there are two ways to annotate: the first is manual annotation, using a large language model annotation tool, such as labelstudio. Annotate the first-level title, second-level title, third-level title, etc. of the document, annotate subtitles and content, annotate paragraphs and split them by 4096 characters, and annotate the order, annotate numbered lists, annotate content pictures, tables, formulas, etc.; the second method is to use a large language model with a large number of parameters plus prompt words to assist in extracting titles, subtitles, paragraphs, numbered lists, pictures, tables, formulas, etc. in the document, and then complete the annotation through manual correction. The above two methods complete the process of electronic documents from unstructured to structured, forming a format for fine-tuning data with a large model.

[0062] In this embodiment, specifically, step S2 includes:

[0063] Step S21: electronic document content splitting; the electronic documents for which the knowledge graph needs to be constructed are sequentially input into the trained large language model, the contents in the electronic documents are split, and the contents are output to a table for storage; that is, the electronic documents for which the knowledge graph needs to be constructed are prepared, and then the trained large language model that can handle the document splitting task is sequentially input, so that the document can be quickly extracted from the document as required, including the title, subtitle, paragraph, numbered list, picture, table, formula, etc., and the contents are output to a table for storage;

[0064] Step S22: review the split content; review and correct the erroneous data in the split content;

[0065] Step S23: constructing data relationships between single documents; reconstructing the relationship between data and data, parsing the content of each split electronic document into a tree-like relationship diagram structure for storage; it should be noted that the relationship between data and data includes: parent-child and brother relationships between titles, description relationship between title and text, brother relationship between paragraphs, and continuous relationship, number explanation relationship, picture explanation relationship, formula explanation relationship, etc. within a paragraph;

[0066] Step S24: constructing relationships between multiple documents; establishing similarity relationships between multiple electronic documents, such as the semantic similarity between a certain paragraph in document one and a certain paragraph in document two, thereby completing the source data construction of the multi-document knowledge graph;

[0067] Step S25: knowledge graph construction; formatting the source data of the multi-document knowledge graph, and finally forming a knowledge graph representation of multiple electronic documents, while also establishing corresponding relationships between multiple documents.

[0068] In this embodiment, specifically, step S22 includes:

[0069] Manual correction is performed for erroneous data. Data that is too long needs to be segmented into no more than 4,000 characters, and the relationship between the segmented data is marked to provide support for the subsequent establishment of graph relationships. That is, there may be errors in the split data, and there may also be problems with the split paragraphs being too long. Therefore, manual secondary review and correction is required. Manual correction is performed for erroneous data. Data that is too long needs to be segmented into no more than 4,000 characters, and the relationship between the segmented data is marked to provide support for the subsequent establishment of graph relationships.

[0070] In this embodiment, specifically, step S24 includes:

[0071] The encoding model bge-large-zh-v1.5-v2 is used to cross-traverse each tree node between multiple documents, and the Yuxuan similarity is used to calculate the semantic similarity with other nodes, and its value is used as the weight directly associated with the node; and a weight threshold is set (0.7 in this embodiment), and all relationships with a similarity less than 0.7 are discarded.

[0072] In this embodiment, specifically, step S25 includes:

[0073] Use the Neo4j graph database to import data, and ultimately form a knowledge graph representation of multiple electronic documents. At the same time, corresponding relationships are established between multiple documents.

[0074] It should be noted that steps S1 and S2 only construct the knowledge graph representation of the original electronic document, and its structure is also an important external knowledge base input source for retrieval enhancement generation. In order to use the above data source more conveniently, it is also necessary to vectorize each node in the above knowledge graph to extract key semantic representations to improve the efficiency of knowledge retrieval.

[0075] In this embodiment, specifically, step S3 includes:

[0076] Step S31: vectorizing graph nodes;

[0077] Step S32: Use of vector knowledge graph.

[0078] In this embodiment, specifically, step S31 includes:

[0079] Use the language encoding model bge-large-zh-v1.5-v2 to encode each node in the knowledge graph, use the vector database to store the node vector representation, and associate the node as metadata with the node vector.

[0080] In this embodiment, it should be noted that in the use of the Retrieval-Augmented Generation artificial intelligence technology framework, it is often necessary to provide an external knowledge base to solve the hallucination problem of the large language model in generating question-answering results. Therefore, constructing a vector knowledge graph can also be used as an input of another knowledge source, that is, the step S32, which specifically includes:

[0081] Step S321: In the input phase, the TOP-K content most similar to the input topic is searched in the vector database using the encoding model bge-large-zh-v1.5-v2 according to the topic input by the user; the corresponding node in the knowledge graph is found through the vector database entry of the query result;

[0082] Step S322: For each node of TOP-K, a breadth-first algorithm is used in the Neo4j graph database to find the original data related to the node in the knowledge graph; that is, first query the paragraphs, sub-paragraphs, numbered lists, tables, pictures and other contents related to the sibling nodes, and also query the nodes with high similarity to other documents, as well as the paragraphs, sub-paragraphs, numbered lists, tables, pictures and other contents related to their sibling nodes;

[0083] Step S323: After the above steps, the original information related to the document corresponding to the input topic can be quickly queried, and finally the document query results are reorganized, which can be used as an external knowledge base generated by retrieval enhancement.

[0084] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

[0085] This background section is provided to generally present the context of the invention, and the work of the presently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither explicitly nor implicitly admitted to be prior art to the present invention.

Claims

1. A multi-level electronic document content organization method, characterized in that: include: Step S1: fine-tune the classification and analysis capabilities of the large language model; Step S2: Input the electronic document for which the knowledge graph needs to be constructed into the fine-tuned large language model to construct a multi-document structure graph; Step S3: Based on the multi-document structure graph constructed in step S2, complete the graph vector space construction.

2. A multi-level electronic document content organization method according to claim 1, characterized in that: The step S1 comprises: Step S11: fine-tune data annotation; complete the process of converting electronic documents from unstructured to structured, and form the format of large model fine-tune data; Step S12: fine-tuning the large language model; using the pre-trained large model Qwen2.5-32B-Instruct as the basic model, adopting supervised fine-tuning, and enabling the LoRA fine-tuning method to improve the efficiency of fine-tuning; through repeated evaluation and parameter adjustment, it has a good output effect in the task of splitting electronic documents.

3. A multi-level electronic document content organization method according to claim 2, characterized in that: The step S11 comprises: Step S111: performing data cleaning on the electronic document to clean out high-quality multi-level electronic documents that can be annotated; Step S112: marking the contents of the multi-level electronic document.

4. A multi-level electronic document content organization method according to claim 3, characterized in that: The step S2 comprises: Step S21: electronic document content splitting: electronic documents for which knowledge graphs need to be constructed are sequentially input into the trained large language model, the content in the electronic documents is split, and the content is output to a table for storage; Step S22: review the split content; review and correct the erroneous data in the split content; Step S23: constructing data relationships between single documents; reconstructing the relationship between data and data, parsing the content of each split electronic document into a tree-like relationship diagram structure for storage; Step S24: constructing relationships between multiple documents; establishing similarity relationships between multiple electronic documents to complete the source data construction of the multi-document knowledge graph; Step S25: knowledge graph construction; formatting the source data of the multi-document knowledge graph, and finally forming a knowledge graph representation of multiple electronic documents, while also establishing corresponding relationships between multiple documents.

5. A multi-level electronic document content organization method according to claim 4, characterized in that: The step S22 comprises: Manual correction is required for erroneous data. Data that is too long needs to be segmented into pieces no longer than 4,000 characters, and the relationships between the segmented data are annotated to provide support for the subsequent establishment of graph relationships.

6. A multi-level electronic document content organization method according to claim 4, characterized in that: The step S24 comprises: Use the encoding model bge-large-zh-v1.5-v2 to cross-traverse each tree node between multiple documents, and use the Yuxuan similarity to calculate the semantic similarity with other nodes. The value is used as the weight directly associated with the node; and set a weight threshold. All relationships with similarity less than the weight threshold are discarded.

7. A multi-level electronic document content organization method according to claim 4, characterized in that: The step S25 comprises: Use the Neo4j graph database to import data, and ultimately form a knowledge graph representation of multiple electronic documents. At the same time, corresponding relationships are established between multiple documents.

8. A multi-level electronic document content organization method according to claim 1, characterized in that: The step S3 comprises: Step S31: vectorizing graph nodes; Step S32: Use of vector knowledge graph.

9. A multi-level electronic document content organization method according to claim 8, characterized in that: The step S31 comprises: Use the language encoding model bge-large-zh-v1.5-v2 to encode each node in the knowledge graph, use the vector database to store the node vector representation, and associate the node as metadata with the node vector.

10. A multi-level electronic document content organization method according to claim 9, characterized in that: The step S32 comprises: Step S321: In the input phase, the TOP-K content most similar to the input topic is searched in the vector database using the encoding model bge-large-zh-v1.5-v2 according to the topic input by the user; the corresponding node in the knowledge graph is found through the vector database entry of the query result; Step S322: For each node of TOP-K, use the breadth-first algorithm in the Neo4j graph database to find the original data related to the node in the knowledge graph; Step S323: After the above steps, the original information related to the document corresponding to the input topic can be quickly queried, and finally the document query results are reorganized, which can be used as an external knowledge base generated by retrieval enhancement.

Images