A method for audit knowledge reasoning and analysis combining a large language model and knowledge graph

Abstract

This invention belongs to the field of power inspection technology, specifically involving a method for inspection knowledge reasoning and analysis combining a large language model and a knowledge graph. The method includes: collecting multi-source, multi-modal document knowledge and data, integrating and preprocessing them; slicing the preprocessed document content and establishing a vector index; constructing a knowledge graph with a dual-level tree structure based on the documents with the established vector index; according to the dual-level tree structure knowledge graph, guided by power inspection needs, integrating keyword matching, semantic similarity calculation, and multi-hop retrieval technology of the knowledge graph to filter highly relevant search results and obtain a highly relevant cross-database knowledge set; and based on the highly relevant cross-database knowledge set, organizing and summarizing the search results through a large language model to generate a summary report. This invention, relying on a cross-source knowledge graph constructed using a large language model, achieves deep association and semantic mining of multi-modal data.

Description

Technical Field

[0001] This invention belongs to the field of power inspection technology, specifically involving an inspection knowledge reasoning and analysis method that combines a large language model with a knowledge graph. Background Technology

[0002] Electricity inspection work covers the entire business chain of the power sector, including transmission, distribution, marketing, and dispatch, and has become a core business for regulating the power market order and preventing business compliance risks. Through years of information technology construction and operation, electricity inspection has accumulated a wealth of business knowledge, cases, and rules, and has built corresponding knowledge bases, case libraries, and rule libraries based on this.

[0003] While the construction of relevant thematic databases facilitates the digital transformation of audit operations, digital auditing in the power sector faces significant challenges in cross-database retrieval, integration, and analysis. On one hand, the knowledge formats of the audit case database and the rule database differ fundamentally. The case database primarily consists of textual descriptions and scenario records, while the rule database mainly comprises logical clauses and indicator thresholds. This heterogeneous data structure makes it difficult to establish a direct correlation between the two. On the other hand, knowledge fusion under cross-retrieval requirements faces multiple obstacles. Existing technologies struggle to achieve "precise matching of case scenarios with applicable rules" and "collaborative reasoning between historical experience and compliance requirements," often resulting in fragmented search results, low relevance, and broken reasoning logic. This forces auditors to expend considerable effort manually integrating cross-database information when knowledge services are needed, reducing audit efficiency and increasing the risk of missed or incorrect judgments due to human error. Furthermore, the rapid dynamic updates and complex, diverse business scenarios inherent in power auditing knowledge further complicate the efficient retrieval and intelligent reasoning of cross-database knowledge. Effective linkage and integration of knowledge services across the knowledge base, case database, and rule database are crucial for achieving intelligent and precise auditing.

[0004] To address the aforementioned issues, this invention proposes a method for audit knowledge reasoning and analysis that combines a large language model with a knowledge graph. Summary of the Invention

[0005] To overcome the problems in the existing technology, this invention proposes a method for audit knowledge reasoning and analysis that combines a large language model with a knowledge graph.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0007] This invention provides a method for audit knowledge reasoning and analysis combining a large language model and a knowledge graph, comprising the following steps:

[0008] Step 100: Collect multi-source, multi-modal document knowledge and data, integrate and preprocess them to obtain preprocessed documents;

[0009] Step 200: Slice the preprocessed document content and create a vector index;

[0010] Step 300: Based on the documents with established vector indexes, construct a knowledge graph with a dual-level tree structure, namely a knowledge graph with a document chapter structure and a knowledge graph with an entity abstract clustering structure;

[0011] Step 400: Based on the knowledge graph with a dual-level tree structure, and guided by the needs of power inspection, integrate keyword matching, semantic similarity calculation and knowledge graph multi-hop retrieval technology to screen highly relevant search results and obtain a highly relevant cross-database knowledge set;

[0012] Step 500: Based on a highly relevant cross-database knowledge set, organize and summarize the search results using a large language model to generate a summary report.

[0013] Furthermore, in step 100, the integration and preprocessing to obtain preprocessed document content includes: parsing the document knowledge and data into a document with a consistent structure format, and establishing a document chapter directory index structure to obtain the preprocessed document.

[0014] Furthermore, the establishment of the document chapter directory index structure includes: correcting the chapter titles according to the separators and numbering set specified in the chapter title content, calculating the correct level and number of the chapter titles, and correcting the parsed document title lines.

[0015] Furthermore, in step 300, constructing a knowledge graph of the document chapter structure includes:

[0016] Based on the document with the established vector index, insert a root node representing the document, and insert chapter nodes for each chapter, establishing the dependency relationship between the chapter nodes and their root nodes.

[0017] Insert slice nodes for each chapter's slices and establish the dependency relationship between the slice nodes and their respective chapter nodes;

[0018] The content of the slice is divided into sentences, and a sentence node is inserted for each sentence in the slice to establish the subordinate relationship between the sentence and its slice.

[0019] This led to the creation of a knowledge graph of the file chapter structure, from document to chapter to slice to statement.

[0020] Furthermore, in step 300, the knowledge graph of the entity abstract clustering structure includes: a text element sub-graph constructed for the text content within the document; and a non-text element sub-graph constructed for the non-text elements within the document.

[0021] Furthermore, the text element sub-graph constructed for the text content within the document includes:

[0022] Entities are extracted from each slice using a large language model. Based on the slices and statements involved in the extracted entities, entity-slice associations and entity-statement associations are established.

[0023] Using a large language model, search keywords are created for each sentence, which are then added to the sub-graph as keyword nodes, and the keyword-sentence association is established.

[0024] Entities / keywords are clustered using a dual dimension of semantic similarity and name matching, and a membership relationship is constructed from the entities / keywords to the entity / keyword clusters.

[0025] By globally retrieving semantically similar or identically named entities / keywords across documents and verifying their synonymy using a large language model, unified entity / keyword nodes across documents are generated, and the association relationship from entity / keyword clustering to cross-document entities / keywords is established.

[0026] This led to the construction of a text element sub-graph.

[0027] Furthermore, for non-text elements within the document, the constructed non-text element subgraph includes: for each non-text element within the document, establishing its own root node for insertion into the knowledge graph, and associating the root node with the slice in the document; starting from this root node, constructing the subgraph for each non-text element.

[0028] Furthermore, for non-text elements within the document, the constructed non-text element sub-graph includes: for non-table elements within the document, the constructed document non-table element sub-graph specifically includes:

[0029] Extract the slice containing non-table elements, the chapter and summary of the chapter containing the element, and the chapter names and summaries of all parent chapters, and merge them as chapter information for non-text elements;

[0030] Generate a detailed description of the content of the non-table elements within the slice or chapter containing the non-table elements;

[0031] The detailed description of the content of non-table elements is treated as a plain text fragment and added to the attribute of the root node of the non-table element;

[0032] Entities and relationships are extracted from the detailed descriptions of non-table elements and clustered with other entities and relationships within the document to construct a sub-graph of non-table elements in the document.

[0033] Further, step 400 includes: user inputting a query, generating search keywords based on keyword matching and semantic similarity for preliminary retrieval, specifically including:

[0034] Based on a knowledge graph with a dual-level tree structure, the retrieval scope of the query is determined;

[0035] Extract entities from the query and generate search keywords;

[0036] Based on the search keywords, matching is performed in a knowledge graph with a dual-level tree structure to obtain approximate keywords of the keywords in the query, and related statements of the keywords in the query are collected;

[0037] Based on the entities in the query, matching is performed in a knowledge graph with a dual-level tree structure to obtain similar entities in the query, and associated slices of the entities in the query are collected.

[0038] Calculate the word embedding vector of the query, use this as the vector index of the query slice, and select the most relevant slice.

[0039] Furthermore, step 400 also includes: further retrieving related entities and statements based on knowledge graph multi-hop retrieval technology, specifically including:

[0040] Based on the related statements of the keywords in the query, all entities associated with them are obtained through a knowledge graph structure with a dual-level tree structure. Other statements associated with the entities are then queried to obtain a set of statements for single-hop retrieval.

[0041] Based on similar entities in the query, the associated statements are obtained through a knowledge graph structure with a dual-level tree structure. Other entities mentioned by the associated statements are then queried to obtain all slices associated with the entity single-hop retrieval.

[0042] Calculate the relevance of the query to the set of statements based on single-hop retrieval of statements and the set of all slices associated with single-hop entity retrieval, and then re-rank them.

[0043] Repeat the above steps multiple times to achieve multi-hop retrieval and obtain a highly relevant cross-database knowledge set.

[0044] Compared with the prior art, the present invention has the following technical effects:

[0045] (1) This invention standardizes the data processing flow by automating multi-source data collection, document slicing, and vector indexing. Combined with keyword matching and multi-hop retrieval, it transforms core auditing processes such as case analysis and risk detection into automated execution processes. This design changes the traditional auditing model that relies on human experience, and can efficiently handle the auditing needs of massive users and complex power grid scenarios. It solves the problems of low efficiency and limited coverage of manual processing, and realizes the large-scale and efficient advancement of auditing work.

[0046] (2) Based on the cross-source knowledge graph constructed by the large language model, this invention realizes deep association and semantic mining of multimodal data, and can capture potential risk trends from multi-dimensional data such as electricity load, equipment operation, and user behavior. At the same time, by constructing a pre-warning mechanism, it breaks the regulatory lag of traditional post-event inspection, which can not only accurately identify new types of illegal electricity use behaviors with strong concealment and innovative methods, but also predict operational risks in advance and provide avoidance suggestions, effectively reducing the situation of missed judgments and misjudgments, and improving the reliability of inspection results and the initiative of supervision.

[0047] (3) This invention achieves comprehensive integration of multi-source heterogeneous data such as business documents, historical cases, and audit rules by adapting the collection methods to the characteristics of different data sources. By combining word embedding models and entity alignment technology, a unified knowledge association system is constructed, solving the problems of data fragmentation and insufficient integration capabilities in existing technologies. This advantage enables audit work to make judgments based on complete and related data support, avoiding regulatory blind spots caused by information fragmentation, greatly improving the accuracy of audit judgments, and providing a solid data and knowledge foundation for precise supervision. Attached Figure Description

[0048] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the system processing flow of the present invention;

[0050] Figure 2 This is a partial view of the Markdown file obtained by parsing the table file in the experiment of this invention;

[0051] Figure 3 This refers to the chapter structure data generated for the markdown file in the experiment of this invention;

[0052] Figure 4 This is a partial view of the document chapter hierarchy map constructed in the experiment of this invention;

[0053] Figure 5 This is a partial view of the entity and entity clustering map constructed in the experiment of this invention;

[0054] Figure 6 This is the final result of the relevant regulations for the multi-hop cross-database query case in the experiment of this invention;

[0055] Figure 7For comparison, the results of querying LightRAG are shown for the same cross-database multi-hop problem. Detailed Implementation

[0056] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0057] In this embodiment, refer to Figures 1-7 This paper presents a method for investigative knowledge reasoning and analysis that combines a large language model with a knowledge graph. Based on the large language model, it extracts entities and relationships from various knowledge sources (such as knowledge bases, case bases, and rule bases) in investigative business and constructs a knowledge graph with a dual-level tree structure. Through knowledge graph fusion, it achieves cross-database heterogeneous knowledge structure modeling. With the help of the natural language understanding and logical reasoning capabilities of the large language model, it realizes efficient fusion reasoning across case bases, rule bases, and other knowledge, providing more accurate and efficient knowledge support for investigative work.

[0058] This invention employs a multi-stage collaborative processing strategy, integrating multi-source and multi-modal document knowledge and data, including business document knowledge, historical cases, and audit rules. It establishes a vector index for document content using a word embedding model and builds a knowledge graph from the documents using a large language model. Based on this, it constructs a cross-database knowledge retrieval system based on semantic similarity and inter-entity associations, tailored to specific scenario requirements. The retrieved content is then organized and analyzed, and a summary analysis report is generated. Specifically, this invention includes the following steps:

[0059] Step 100: Collect multi-source, multi-modal document knowledge and data, integrate and preprocess them to obtain preprocessed documents;

[0060] Step 200: Slice the preprocessed document content and create a vector index;

[0061] Step 300: Based on the documents with established vector indexes, construct a knowledge graph with a dual-level tree structure, namely a document chapter structure knowledge graph and an entity abstract clustering structure knowledge graph;

[0062] Step 400: Based on the knowledge graph with a dual-level tree structure, guided by the needs of power inspection, keyword matching, semantic similarity calculation and knowledge graph multi-hop retrieval technology are integrated, and highly relevant retrieval results are screened through a re-ranking model to obtain a highly relevant cross-database knowledge set;

[0063] Step 500: Based on a highly relevant cross-database knowledge set, organize and summarize the search results using a large language model to generate a summary report.

[0064] The following is a detailed explanation of each of the above steps:

[0065] Step 100: Collect multi-source, multi-modal document knowledge and data, integrate and preprocess them to obtain preprocessed document content.

[0066] This step aims to parse multi-source, multi-modal document knowledge and data, and synthesize them into a nearly consistent structural form to facilitate subsequent unified processing and lay the foundation for the effective utilization of knowledge and data. All document types are converted into Markdown (Lightweight Markup Language) documents with a consistent structure and format, so they can be processed using the same method later, eliminating the need to write separate processing rules. Following this, by parsing the multi-source, multi-modal document knowledge and data one by one, analyzing their directory structure and element information, a unified document representation with a hierarchical tree structure is established.

[0067] Step 110: Collect multi-source, multi-modal document knowledge and data, including business documents, historical cases, and audit rules.

[0068] The business documents include various records and operation manuals of the power system operation; historical cases cover typical cases of illegal electricity use and equipment failures handled in past audits; and the audit rules are based on relevant laws and regulations, industry standards and internal rules and regulations of the company.

[0069] During the collection process, appropriate collection methods are adopted based on the characteristics of different data sources, such as directly extracting structured data from databases and extracting unstructured or semi-structured data such as text and charts from documents.

[0070] Step 120: Parse the document knowledge and data into a document with a consistent structure and format.

[0071] For documents such as doc and pdf, document parsing tools like MinerU are used to identify and locate elements within the document using computer vision (CV) algorithms combined with visual models. Optical character recognition (OCR) technology is then used to extract the text, and non-text portions are extracted and stored as images. Finally, the document is converted into a Markdown file with a consistent structure and format.

[0072] ;

[0073] In the above formula, Represents a document; This describes the process of document parsing; This indicates that the text obtained after parsing the document is stored in Markdown format; This represents a collection of all non-text elements extracted from a document, including all images, tables, formulas, code blocks, and other elements that may appear in the document. Except for images, which are stored as image files, in... Apart from file location references within the text, all other elements are embedded directly as plain text with formatted tags. It appears in the corresponding position of the text and always occupies a separate line.

[0074] For documents in tabular format such as .xlsx and .csv, the data within the tables is read and converted into Markdown files with a consistent structure and format.

[0075] ;

[0076] In the above formula, Algorithm representation from table Extract all rows / columns of data; This is the total number of rows in the table. r Represents the row index of the table data; This represents the extracted single data record, which is a function that maps keys to values. domain That is, the text and value range of each column in the table header. It is the collection of all values ​​appearing in the table, these data The data is presented in key-value JSON dictionary format, with each entry occupying a portion of the parsed text. The row in the table belongs to the chapter corresponding to its table.

[0077] Step 130: Based on plain text documents with consistent structure and format, establish a document chapter directory index structure.

[0078] After the above steps, both the document and the table have been standardized into similar Markdown documents. The original document was preserved as much as possible during the document parsing process. f The formatting information in the parsed document is preserved to prevent information loss. Based on this parsing result, the resulting Markdown document can be further analyzed. To determine the chapter and table of contents index structure in the document, specifically including:

[0079] Step 1301: Correct and complete the chapter titles.

[0080] For documents such as doc and pdf, document parsing tools often misinterpret chapter headings, leading to a disordered directory structure. To address this issue, a chapter heading correction algorithm is used. For markdown documents Correct the title hierarchy:

[0081] ;

[0082] In the above formula, is the separator of chapter titles, defaulting to "."; is the set of chapter title numbers, defaulting to all Arabic numerals. For Chinese documents, can be taken as the Chinese punctuation mark "、", and then take all Chinese numerals. The title correction algorithm calculates the hierarchy of chapter titles based on the number of separators in the title line and corrects to a markdown document with correctly labeled chapter levels .

[0083] For documents in table forms such as xlsx and csv, their chapter titles are often missing. Use the capabilities of large language models to summarize the title for each chapter with a missing title name:

[0084] ;

[0085] In the above formula, represents the process of summarization by the large language model, is the markdown document text after adding the chapter title name after summarizing each chapter in .

[0086] Step 1302: Establish the correct chapter structure.

[0087] By analyzing the chapter title lines in the markdown document, establish the correct chapter structure index directory for the document:

[0088] ;

[0089] In the above formula, represents the algorithm for analyzing markdown to establish chapter numbers. Each chapter consists of four pieces of information, is the chapter number, is the number of its direct parent chapter, and are the starting and ending line numbers of the chapter respectively, is the total number of chapters.

[0090] After the above steps, each document is parsed and unified into a Markdown document with a consistent structure and format. All non-text elements are preserved, and a consistent chapter directory hierarchy index is established across file types. This lays the foundation for subsequent unified processing of the documents.

[0091] Step 200: Slice the preprocessed document content and create a vector index.

[0092] The document content is segmented according to its chapter structure while preserving sentence integrity. A selected word embedding model is then used to process the document content, transforming the text into vectors to facilitate computer understanding and semantic processing, thus meeting subsequent semantic retrieval needs.

[0093] This step aims to create vector indexes for each document, enabling semantic similarity retrieval and laying the foundation for subsequent information retrieval and report generation.

[0094] Documents with correctly labeled chapter levels Each chapter in the document is sliced:

[0095] ;

[0096] In the above formula, the slicing operation Chapter Cut into Each shard is a fragment. Based on the fragment index number Chapter number and starting line number End line number composition.

[0097] For PDF and Word documents, to avoid fragmenting related sentences and affecting the agent's understanding of the content due to slicing, the row ranges of each slice will overlap to some extent; while for tabular data such as XLSX and CSV, to avoid being misinterpreted as two data entries, there is no overlap between the slices.

[0098] After slicing, word embedding vectors are calculated based on the text content in each slice using a word embedding model:

[0099] ;

[0100] In the above formula, These are the starting row numbers of the slices. The text content within the line range, It is the word embedding algorithm process. This is the final word embedding vector.

[0101] Step 300: Based on the documents with established vector indexes, construct a knowledge graph with a dual-level tree structure, namely a document chapter structure knowledge graph and an entity abstract clustering structure knowledge graph.

[0102] This step aims to construct a knowledge graph index for each document, providing a more accurate and flexible retrieval tool for subsequent deep knowledge retrieval and reasoning. The knowledge graph constructed in this invention has a dual hierarchical structure: first, a hierarchical graph built from the document's chapter directory structure and element inclusion relationships, used for filtering during retrieval; second, a hierarchical graph extracted from the document content, consisting of entities, keywords, and cluster nodes, serving as the entry point for retrieval and the relational relationships for divergent searches. The generated knowledge graph is stored using graph database management systems such as Neo4j and TuGraph.

[0103] Step 310: Based on the documents with established vector indexes, construct a document chapter structure knowledge graph.

[0104] The purpose of constructing this knowledge graph is to reflect the chapter structure of the document, forming a hierarchical tree. Starting from the root node representing the document, nodes branch out sequentially to represent chapters, sub-chaps, document fragments, and leaf nodes representing sentences. This tree-like graph accurately records the chapter hierarchy within the document, providing a range for filtering in the subsequent retrieval process. Compared to simple semantic similarity retrieval, when the query has a clear relevance, it can better locate related fragments, improving retrieval efficiency while reducing the negative impact of a large number of irrelevant fragments on subsequent large-scale model analysis. Specifically, this includes:

[0105] Step 3101: Based on the document with the established vector index, insert the root node representing the document.

[0106] Insert the root node representing the document into the graph database. This node stores information such as filename, language, content summary, and chapter structure as attributes. It then serves as the root node for that file in the graph, adding child nodes to it in subsequent processes.

[0107] Step 3102: Insert chapter nodes for each chapter, establish the hierarchical relationship between chapter nodes and their parent nodes, and build a hierarchical tree of the file chapter structure.

[0108] Insert chapter nodes for each chapter Write the chapter's information into the chapter node attributes, including the source file, starting line number, node number, parent node number, chapter name, and chapter content summary; then, for each chapter node... Establish its dependency relationship with its parent node. This forms a hierarchical tree that establishes the document chapter structure, where the parent node includes the root node or other superior chapter nodes.

[0109] Step 3103: Insert a slice node for each chapter's slice and establish the dependency relationship between the slice node and its corresponding chapter node.

[0110] Insert a slice node for each chapter's slice. Slice node attributes include slice index number, starting line number, chapter number, and text content. This is achieved by creating slice nodes. The hierarchy of slices and chapters is determined by their subordinate relationship with the chapter node.

[0111] Step 3104: Divide the slice content into sentences, create sentence nodes, and establish the subordinate relationship from sentences to slices.

[0112] The content of the slice is determined and labeled by combining a large language model and a rule engine. The slice content is then segmented into sentences using a sentence segmentation model. The content categories include text, images, tables, formulas, code, etc.

[0113] Insert statement nodes based on clause segmentation The statement content is stored in the attributes of the statement node, and finally the subordinate relationship from the statement to the slice is established.

[0114] After the above steps, a hierarchical tree structure of file-chapter-slice-statement is established.

[0115] Step 320: Based on the documents with established vector indexes, construct an entity abstract clustering structure knowledge graph.

[0116] As an example, step 320 specifically includes:

[0117] Step 3201: Extract entities, keywords, and relationships between entities from the text content of the document to construct a text element sub-graph.

[0118] The purpose of constructing the clustering graph of entities and keywords in this section is to extract entities and relationships between entities from the text, serving as entry points for retrieval and as the basis for multi-hop retrieval.

[0119] Instead of employing the conventional entity relation triplet extraction strategy, this approach extends the traditional binary relations of knowledge graphs to multi-dimensional relations. As a result, entity relations are no longer edges connecting two entities, but rather hyperedges connecting multiple entities. In this embodiment, statements are directly treated as entity hyperedges, representing relations between entities. Since most entity relations are stated within the same statement, the relation extraction step is eliminated, improving the efficiency of graph construction.

[0120] Meanwhile, to align identical entities across multiple documents, a strategy of clustering entities twice was adopted. The first clustering occurs within each document, forming document-level entity clusters. The second clustering occurs across documents, aligning the intra-document clusters again and abstracting them into cross-document entities. This significantly reduces the number of entities processed in each clustering step, making it possible to process large numbers of documents and improving processing efficiency. The specific steps are as follows:

[0121] ① Entities are extracted from each slice using a large language model, stored as knowledge graph nodes, and entity-slice associations and entity-statement associations are established.

[0122] Entities are extracted from each slice using a large language model, and it is determined which statements mention each entity, generating the entity's name and description. Entities are then inserted as nodes in a knowledge graph, with corresponding information stored in the node's attributes, and new entities appearing in the slices. and entities are mentioned in the statement Two kinds of relationships.

[0123] ②Use a large language model to create search keywords for each statement and establish the association between keywords and statements.

[0124] Use a large language model to create search keywords for each sentence, which serve as keyword nodes. Add keywords to the search graph and establish associations between keywords and sentences. Keywords allow for more granular retrieval and prevent verbose sentences from reducing vector similarity differences.

[0125] ③ Cluster entities / keywords using both semantic similarity and name matching to construct a membership relationship from the entities / keywords to the entity / keyword clusters.

[0126] For each entity, retrieve semantically similar entities and entities with the same name, and then use a large language model to determine which have completely identical meanings. Entities with identical meanings are abstracted into an entity cluster, and the entity cluster nodes... Add to the knowledge graph and establish a membership relationship from the original entities to entity clusters. Perform identical clustering on keywords.

[0127] ④ By globally searching for semantically similar or identically named entities / keywords across documents, and verifying their synonymy using a large language model, unified entity / keyword nodes across documents are generated, and the association relationship from entity / keyword clustering to cross-document entities / keywords is established.

[0128] For each entity cluster, retrieve semantically similar or identically named entity clusters extracted from different files. The large language model then identifies completely synonymous entity clusters and generates descriptions and names for cross-file entities, which serve as cross-file entity nodes. Add it to the knowledge graph. Finally, add relationships from entity clustering to cross-document entities. Perform identical cross-document alignment on keywords.

[0129] Through the above steps, a hierarchical structure is established, from sentences and slices to entities and keywords, then to entity and keyword clustering, and finally to cross-file entities and keywords, thus constructing a text element sub-graph.

[0130] Step 3202: For non-text elements within the document, construct a sub-graph of the document's non-text elements.

[0131] For non-text elements within a document, a root node is created for each non-text element and inserted into the knowledge graph. The root node is then associated with the slice in the document. Starting from this root node, sub-graphs for each non-text element are constructed.

[0132] In this way, the entire content and information of each document can be reflected in the graph. Since tabular documents can be fully processed through steps 310 and 3201, the above method is reused to directly generate the table's graph, embedding the entire table graph as a sub-graph within the document's graph. For non-table graphs, a larger model is used for processing. The specific process is as follows:

[0133] For table-type elements, including:

[0134] ① The process involves table parsing, document structure construction, and entity and keyword extraction to obtain a graph with a dual-level structure, similar to the one used for processing table files.

[0135] ② Add all nodes and edges of this graph, along with their attributes, to the overall graph of the document. Finally, add the relationship between the root node of the table and its corresponding document slice. In this way, the subgraph of the table is embedded into the graph of the entire file.

[0136] ③ And perform clustering and alignment of entities and keywords.

[0137] Finally, the entities and keywords in the subgraph need to be aligned with those in the document, using the same method as step ③ of 3201.

[0138] For non-table elements, including:

[0139] ① Extract the slice containing non-table elements as context information for the elements;

[0140] ② Extract the chapter name and chapter summary of the chapter containing the non-table element, as well as the chapter names and summaries of all its parent chapters; merge the chapter names and summaries as the chapter information for the non-table element;

[0141] ③ Use a large language model or visual model to generate a detailed description of the content of non-table elements in the slices and chapters they are in;

[0142] ④ Treat the detailed description of the content of non-table elements as plain text fragments and add it to the attribute of the root node of the non-text element;

[0143] ⑤ Extract entities and relationships from the detailed descriptions of non-table elements, and cluster them with other entities and relationships within the document to construct a sub-graph of non-table elements in the document.

[0144] Step 400: Based on the knowledge graph with a dual-level tree structure, guided by the needs of power inspection, keyword matching, semantic similarity calculation and knowledge graph multi-hop retrieval technology are integrated, and highly relevant retrieval results are filtered through a re-ranking model to obtain a highly relevant cross-database knowledge set, realizing deep cross-retrieval and knowledge association between the inspection case database and the rule database.

[0145] Step 410: For the text query input by the user, generate search keywords based on keyword matching and semantic similarity to conduct a preliminary search.

[0146] Query in text form input by the user The following steps will be performed:

[0147] ① Determine the retrieval scope of the query based on the knowledge graph.

[0148] By leveraging the descriptive and chapter structure information of each document in the knowledge graph, and utilizing the capabilities of a large language model, or user selection, the query can be determined. The search scope is given in the form of a set of multiple files or chapters.

[0149] ② Extract entities from the query and generate search keywords:

[0150] ;

[0151] In the above formula, It is the process of using a large model to extract entities and generate keywords. From asking The set consisting of all entities extracted. It is a set of all keywords generated from the query.

[0152] ③ Based on the search keywords, match them in the knowledge graph to obtain similar keywords in the query, and collect related sentences of the keywords in the query.

[0153] For each generated keyword Matching similar keywords in the knowledge graph And extract related statements containing these keywords within the search scope. Form a set .

[0154] ④ Based on the entities extracted from the query, match them in the knowledge graph to obtain similar entities in the query, and collect the associated slices of the entities in the query.

[0155] For each extracted entity Match entities with the same name and semantic similarity in the knowledge graph. And extract the slices associated with these entities. Form a set .

[0156] ⑤ Calculate the word embedding vector of the query and retrieve relevant slices through semantic similarity.

[0157] Direct calculation query The word embedding vector is used to query the vector index of the slice and retrieve the most relevant words. k Slice into a set .

[0158] Step 420: Based on knowledge graph multi-hop retrieval technology, further retrieve related entities and statements.

[0159] ① Based on the related statements of keywords in the query, all entities associated with them are obtained through the knowledge graph structure query. Then, other statements associated with the entities are queried to obtain the set of statements for single-hop retrieval.

[0160] For each related statement retrieved by keyword approximation matching By querying the knowledge graph structure, all entities associated with it (i.e., those mentioned by it) are obtained. Then, other statements associated with these entities are queried to obtain the set of statements for single-hop statement retrieval. .

[0161] ② Based on similar entities in the query, retrieve the associated statements through the knowledge graph structure, query other entities mentioned by the associated statements, and obtain all slices associated with the single-hop entity retrieval.

[0162] For each similar entity obtained by entity matching By querying the knowledge graph structure, we can obtain the associated statements and then query other entities mentioned by these statements. This retrieves all slices associated with the single-hop entity retrieval. .

[0163] ③ Calculate the relevance of the query to the statement set based on single-hop statement retrieval and the slices associated with single-hop entity retrieval, and then reorder them.

[0164] right and The statements and slices in the query are calculated using a reordering model. Based on relevance, sentences and segments with low relevance are filtered out.

[0165] ④ Repeat steps ① to ③ multiple times to achieve multi-hop retrieval, and obtain... as well as .

[0166] Step 430: Filter highly relevant search results through the re-ranking model to obtain a highly relevant cross-database knowledge set.

[0167] The entire set of statements and slices retrieved in the previous steps are merged, deduplicated, and then filtered using a re-ranking model to select those that match the query. Highest similarity k Each slice and statement forms the final search result. , It is a collection of all related statements. It is a set consisting of all related slices.

[0168] Step 500: Based on a highly relevant cross-database knowledge set, organize and summarize the search results using a large language model to generate a summary report.

[0169] This step aims to summarize all the information extracted in the previous step into an easy-to-read and clear summary.

[0170] Search results The information is passed to a general-purpose large language model, which, through prompt word engineering and appropriate templates, organizes and summarizes the information to form a summary table output.

[0171] Method and effect demonstration:

[0172] Taking user capacity expansion as an example, this invention retrieves, analyzes, and summarizes data from a rule base composed of xlsx files and a case base composed of pdf files, generating a risk report that may result from this operation and providing avoidance suggestions.

[0173] Figure 2The markdown file obtained from parsing the file in step 120, taking an xlsx table file as an example, is each data item parsed into a line of JSON dictionary in the file. Figure 3 This refers to the chapter structure information obtained in step 130. Figure 4 This is a partial diagram of the document chapter structure obtained after step 310, which shows the hierarchical structure within the document. Figure 5 The entities and entity clustering map (partial) obtained from step 3201.

[0174] Figure 6 To determine which rules in the rule base were violated by Case 39, we retrieved the content of Case 39 from the case base and then used multi-hop retrieval to locate the relevant rule with indicator code 5016. For comparison, querying the same question using LightRAG (a widely adopted large model augmentation retrieval framework) yielded the following results: Figure 7 The search results only include content from the case library and cannot locate the corresponding rules in the rule library.

[0175] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for auditing knowledge reasoning and analysis combining a large language model and knowledge graph, characterized in that, Includes the following steps: Step 100: Collect multi-source, multi-modal document knowledge and data, integrate and preprocess them to obtain preprocessed documents; Step 200: Slice the preprocessed document content and create a vector index; Step 300: Based on the documents with established vector indexes, construct a knowledge graph with a dual-level tree structure, namely a knowledge graph with a document chapter structure and a knowledge graph with an entity abstract clustering structure; Step 400: Based on the knowledge graph with a dual-level tree structure, guided by the needs of power inspection, integrate keyword matching, semantic similarity calculation and knowledge graph multi-hop retrieval technology to screen highly relevant search results and obtain a highly relevant cross-database knowledge set; Step 500: Based on a highly relevant cross-database knowledge set, organize and summarize the search results using a large language model to generate a summary report; In step 300, the knowledge graph of the entity abstract clustering structure includes: a text element sub-graph constructed for the text content in the document; and a non-text element sub-graph constructed for the non-text elements in the document. The text element sub-graph constructed for the text content within the document includes: Entities are extracted from each slice using a large language model. Based on the slices and statements involved in the extracted entities, entity-slice associations and entity-statement associations are established. Using a large language model, search keywords are created for each sentence, which are then added to the sub-graph as keyword nodes, and the keyword-sentence association is established. Entities / keywords are clustered using a dual dimension of semantic similarity and name matching, and a membership relationship is constructed from the entities / keywords to the entity / keyword clusters. By globally retrieving semantically similar or identically named entities / keywords across documents and verifying their synonymy using a large language model, unified entity / keyword nodes across documents are generated, and the association relationship from entity / keyword clustering to cross-document entities / keywords is established. This was used to construct a text element subgraph.

2. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph as described in claim 1, characterized in that, In step 100, the integration and preprocessing to obtain preprocessed document content includes: parsing the document knowledge and data into a document with a consistent structure and format, and establishing a document chapter directory index structure to obtain the preprocessed document.

3. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph as described in claim 2, characterized in that, The process of establishing the document chapter directory index structure includes: correcting the chapter titles according to the separators and numbering set specified in the chapter title content, calculating the correct level and number of the chapter titles, and correcting the parsed document title lines.

4. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph as described in claim 1, characterized in that, In step 300, constructing a knowledge graph of the document chapter structure includes: Based on the document with the established vector index, insert a root node representing the document, and insert chapter nodes for each chapter, establishing the dependency relationship between the chapter nodes and their root nodes. Insert slice nodes for each chapter's slices and establish the dependency relationship between the slice nodes and their respective chapter nodes; The content of the slice is divided into sentences, and a sentence node is inserted for each sentence in the slice to establish the subordinate relationship between the sentence and its slice. This led to the creation of a knowledge graph of the file chapter structure, from document to chapter to slice to statement.

5. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph as described in claim 1, characterized in that, For non-text elements within a document, the constructed non-text element subgraph includes: for each non-text element within the document, establishing its own root node for insertion into the knowledge graph, and associating the root node with the slice in the document; starting from this root node, constructing the subgraph for each non-text element.

6. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph as described in claim 5, characterized in that, The non-text elements include non-table elements. For non-table elements, the constructed document non-table element sub-graph specifically includes: Extract the slice containing non-table elements, the chapter and summary of the chapter containing the element, and the chapter names and summaries of all parent chapters, and merge them as chapter information for non-text elements; Generate a detailed description of the content of the non-table elements within the slice or chapter containing the non-table elements; The detailed description of the content of non-table elements is treated as a plain text fragment and added to the attribute of the root node of the non-table element; Entities and relationships are extracted from the detailed descriptions of non-table elements and clustered with other entities and relationships within the document to construct a sub-graph of non-table elements in the document.

7. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph as described in claim 1, characterized in that, Step 400 includes: user input query, generating search keywords based on keyword matching and semantic similarity for preliminary retrieval, specifically including: Based on a knowledge graph with a dual-level tree structure, the retrieval scope of the query is determined; Extract entities from the query and generate search keywords; Based on the search keywords, matching is performed in a knowledge graph with a dual-level tree structure to obtain approximate keywords of the keywords in the query, and related statements of the keywords in the query are collected; Based on the entities in the query, matching is performed in a knowledge graph with a dual-level tree structure to obtain similar entities in the query, and associated slices of the entities in the query are collected. Calculate the word embedding vector of the query, use this as the vector index of the query slice, and select the most relevant slice.

8. The audit knowledge reasoning and analysis method based on a large language model combined with a knowledge graph according to claim 7, characterized in that, Step 400 further includes: based on knowledge graph multi-hop retrieval technology, further retrieving related entities and statements, specifically including: Based on the related statements of the keywords in the query, all entities associated with them are obtained through a knowledge graph structure with a dual-level tree structure. Other statements associated with the entities are then queried to obtain a set of statements for single-hop retrieval. Based on similar entities in the query, the associated statements are obtained through a knowledge graph structure with a dual-level tree structure. Other entities mentioned by the associated statements are then queried to obtain all slices associated with the entity single-hop retrieval. Calculate the relevance of the query to the set of statements based on single-hop retrieval of statements and the set of all slices associated with single-hop entity retrieval, and then re-rank them. Repeat the above steps multiple times to achieve multi-hop retrieval and obtain a highly relevant cross-database knowledge set.

Images