Audit path generation method and device, medium and program product

Abstract

The embodiment of the invention provides an audit path generation method and device, a medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: extracting multi-source heterogeneous data from an audit system document, a historical audit report and / or business system data; identifying auditing entities in the multi-source heterogeneous data and association relationships thereof through a natural language processing model, and generating a structured entity set and an association relationship set; storing the structured entity set and the incidence relation set in a graph database to obtain an audit knowledge graph with a hierarchical relation; and generating an audit path based on the audit knowledge graph. By adopting the method, the auditing path with high coverage degree and high auditing efficiency can be generated based on the auditing knowledge graph, and the auditing coverage rate and the auditing efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an audit path generation method, device, medium and program product. Background Technology

[0002] With the diversification of banking business and increasingly stringent regulatory requirements, internal auditing work in banks is facing considerable pressure.

[0003] Existing audit path generation methods typically rely on the experience of auditors. Because audit systems struggle to effectively integrate audit-related data, they also find it difficult to automatically generate audit paths that offer both high coverage and improved efficiency. Therefore, there is an urgent need for a method that can automatically generate audit paths with both high coverage and high efficiency. Summary of the Invention

[0004] This application provides an audit path generation method, device, medium, and program product, which are used to generate audit paths with high audit coverage and high audit efficiency in audit scenarios.

[0005] In a first aspect, embodiments of this application provide an audit path generation method, the method comprising: extracting multi-source heterogeneous data from audit policy documents, historical audit reports and / or business system data; identifying audit entities and their relationships in the multi-source heterogeneous data through a natural language processing model, generating a structured entity set and a set of relationships; storing the structured entity set and the set of relationships in a graph database to obtain an audit knowledge graph with hierarchical relationships; and generating an audit path based on the audit knowledge graph.

[0006] In one possible implementation, extracting multi-source heterogeneous data includes: splitting audit policy documents into at least one logical block, each logical block corresponding to a predefined audit topic; extracting structured audit entities from business system data through a rule parsing module; and merging at least one logical block and the structured audit entities according to the audit topic to obtain a multi-source heterogeneous data set including multiple multi-source heterogeneous data.

[0007] In one possible implementation, after identifying audit entities and their relationships in multi-source heterogeneous data through a natural language processing model, the method further includes: performing deduplication processing on the audit entities by vector similarity calculation to obtain a deduplicated set of structured entities.

[0008] In one possible implementation, before storing the structured entity set and the set of relationships in the graph database, the method further includes: receiving a user's request to modify the structured entity set; identifying audit entities in the modification request using a natural language processing model to obtain supplementary audit entities; and adding the supplementary audit entities to the structured entity set.

[0009] In one possible implementation, the method further includes: comparing and analyzing multiple audit system documents using a natural language processing model to identify newly added and / or modified target audit entities; and updating the audit entities corresponding to the target audit entities and their relationships in the audit knowledge graph based on the target audit entities to obtain an updated audit knowledge graph.

[0010] In one possible implementation, the natural language processing model includes a large language model and a rule engine. The natural language processing model identifies audit entities in multi-source heterogeneous data, including: extracting entities from multi-source heterogeneous data of unstructured documents using the large language model to obtain audit entities related to audit points and audit indicators; and / or, parsing audit entities related to audit templates and audit data sources from structured multi-source heterogeneous data using the rule engine.

[0011] In one possible implementation, generating an audit path based on an audit knowledge graph includes: retrieving multiple audit path points from the audit knowledge graph through multi-hop queries, and generating at least one audit path based on the multiple audit path points; the method further includes: upon receiving user feedback on the multiple audit path points, dynamically sorting the multiple audit path points according to the user feedback, and updating at least one audit path according to the result of the dynamic sorting.

[0012] Secondly, embodiments of this application provide an audit path generation apparatus, comprising: an extraction module for extracting multi-source heterogeneous data from audit policy documents, historical audit reports, and / or business system data; an identification module for identifying audit entities and their relationships in the multi-source heterogeneous data using a natural language processing model, generating a structured entity set and a set of relationships; a storage module for storing the structured entity set and the set of relationships in a graph database to obtain an audit knowledge graph with hierarchical relationships; and a generation module for generating audit paths based on the audit knowledge graph.

[0013] In one possible implementation, the extraction module is specifically used to: split the audit policy document into at least one logical block, each logical block corresponding to a predefined audit topic; extract structured audit entities from business system data through the rule parsing module; and merge at least one logical block and structured audit entities according to the audit topic to obtain a multi-source heterogeneous data set including multiple multi-source heterogeneous data.

[0014] In one possible implementation, the apparatus further includes a deduplication module, which is used to: perform deduplication processing on the audit entities by vector similarity calculation to obtain a deduplicated set of structured entities.

[0015] In one possible implementation, the apparatus further includes a supplementary module, which is configured to: receive a user's request for correction of the structured entity set; identify the audited entity in the correction request using a natural language processing model to obtain a supplementary audited entity; and add the supplementary audited entity to the structured entity set.

[0016] In one possible implementation, the apparatus further includes an update module, which is used to: compare and analyze multiple audit system documents using a natural language processing model to identify newly added and / or modified target audit entities; and update the audit entities corresponding to the target audit entities and their relationships in the audit knowledge graph according to the target audit entities to obtain an updated audit knowledge graph.

[0017] In one possible implementation, the natural language processing model includes a large language model and a rule engine. The recognition module is specifically used to: extract entities from multi-source heterogeneous data of unstructured documents using the large language model to obtain audit entities related to audit points and audit indicators; and / or, parse audit entities related to audit templates and audit data sources from structured multi-source heterogeneous data using the rule engine.

[0018] In one possible implementation, the generation module is specifically used to: retrieve multiple audit path points from the audit knowledge graph through multi-hop queries, and generate at least one audit path based on the multiple audit path points; the device also includes a feedback module, which is used to: dynamically sort the multiple audit path points according to the user feedback when receiving user feedback on the multiple audit path points, and update at least one audit path according to the result of the dynamic sorting.

[0019] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0021] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0022] The audit path generation method, device, medium, and program products provided in this application extract multi-source heterogeneous data from audit policy documents, historical audit reports, and business system data. Then, using a natural language processing model, they identify audit entities and their relationships within this multi-source heterogeneous data, generating a structured entity set and a set of relationships. This achieves entity integration of audit-related multi-source heterogeneous data, overcoming the low retrieval efficiency caused by silos of heterogeneous data. Furthermore, the structured entity set and the set of relationships are stored in a graph database, resulting in an audit knowledge graph with hierarchical relationships that can be used for audit path generation. Thus, audit paths with high coverage and high audit efficiency can be generated based on the audit knowledge graph, improving both audit coverage and efficiency. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 A flowchart illustrating the audit path generation method provided in this application embodiment;

[0025] Figure 2 A flowchart of the intelligent audit path recommendation system provided in the embodiments of this application;

[0026] Figure 3 A schematic diagram illustrating the audit entities and their relationships provided in the embodiments of this application;

[0027] Figure 4 Example diagram of the audit path provided in the embodiments of this application;

[0028] Figure 5 This is a schematic diagram of the audit path generation device provided in the embodiments of this application;

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0030] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0033] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0034] The following is an explanation of some terms and concepts used in the embodiments of this application:

[0035] Large Language Models (LLMs): Large language models learn to understand and generate language by studying massive amounts of text data, typically involving billions or even tens of billions of parameters. LLMs can perform various natural language processing tasks, including text generation, translation, and question answering. Large language models are also known as large-scale language models, big models, or natural language models.

[0036] Prompt engineering refers to the process in machine learning or natural language processing of designing, optimizing, and testing input prompts to improve model performance. These prompts are instructions or questions provided to the model to guide it in generating specific outputs.

[0037] Graph Retrieval-Augmented Generation (GraphRAG): GraphRAG is an extension of Retrieval-Augmented Generation (RAG). It improves the retrieval and generation process of traditional RAG by introducing a graph structure, enabling it to better understand complex semantic relationships and improve the accuracy and coherence of the generated results.

[0038] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0039] In the embodiments of this application, the use of terms such as "first" and "second" is to distinguish between identical or similar items that have essentially the same function and effect. For example, "first electronic device" and "second electronic device" are merely used to distinguish different electronic devices and do not limit their order of execution. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0040] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0041] Against the backdrop of diversified banking operations and increasingly stringent regulatory requirements, internal auditing in banks faces significant challenges. The scope of audits continues to expand, needing to cover multiple business lines, including finance, credit, bills, and information technology (IT). However, audit data is scattered across independent business systems, creating severe data silos. Auditors (users) must process large amounts of unstructured documents and structured data.

[0042] The existing audit system (hereinafter referred to as the system) lacks an effective knowledge association mechanism, making it difficult to systematically accumulate audit knowledge. For example, in the "Special Audit of Bank Acceptance Bill Discounting Business", auditors need to manually search scattered institutional documents, historical problem cases and audit models, which is time-consuming and prone to missing key risk points.

[0043] Furthermore, there are significant differences in experience among audit team members. Novice auditors often make mistakes due to lack of experience, while experienced personnel may struggle to adapt to new systems or changes in the field due to their established experience. In the audit process, data collection, issue verification, and report writing all rely on manual operations, which are inefficient and prone to errors. While existing systems have digitized the process, fragmented document management, reliance on keyword matching for retrieval logic, and a lack of contextual understanding capabilities fail to meet auditors' urgent needs for "what you search for" and intelligent path recommendations. Therefore, there is an urgent need for a method that can integrate multi-source data, build a dynamic knowledge system, and generate audit paths to improve audit efficiency, reduce labor costs, and enhance risk identification capabilities.

[0044] Existing audit systems typically employ a model that combines online processes with basic knowledge retrieval, but this model has several shortcomings.

[0045] For example, there is a flaw in document management, which is too fragmented. Audit policies, reports, models, and other materials are scattered across different functional modules, lacking a unified knowledge association logic. This forces auditors to search across systems, reducing processing efficiency. Another example is the simplistic search logic. Relying on keyword matching for information retrieval requires users to have a strong grasp of technical terminology, and the search results are fragmented, failing to integrate contextual information to form systematic recommendations.

[0046] For example, there is a weakness in knowledge accumulation. Unstructured documents (such as audit manuals) only support full-text downloads or simple keyword searches, making it impossible to extract core audit points, indicators, and relationships, resulting in difficulty in reusing related knowledge. Furthermore, there are deficiencies in intelligent interaction. The audit system lacks semantic understanding and / or graph reasoning capabilities, making it unable to generate structured audit paths based on user-input natural language questions. For instance, if a user expects knowledge on how to conduct bill discounting audits, but the system lacks the ability to handle natural language questions, it cannot resolve the user's question, reducing audit processing efficiency and user experience.

[0047] Furthermore, the data is not timely. For example, new policies and reports may be continuously added, but the system lacks a dynamic update mechanism, which leads to outdated documents being mixed with new content, affecting the accuracy of retrieval.

[0048] An analysis of the various problems existing in the aforementioned audit system reveals several issues: data silos and fragmented knowledge, simplistic retrieval logic, insufficient experience accumulation, and a lack of dynamic updates. As demonstrated above, audit policy documents, historical reports, and business system data are stored in a scattered manner, lacking a unified logical connection. This forces auditors to search across systems, resulting in inefficiency and data silos and fragmented knowledge. For example, searching for indicators related to "identifying false trade backgrounds" requires manual searching of multiple documents and manual integration. The simplistic retrieval logic, relying on keyword matching, necessitates users' mastery of technical terminology and an inability to integrate contextual information. For instance, entering "bill discounting risk" might return irrelevant documents related to "bill discounting processes."

[0049] Insufficient experience accumulation makes it difficult for senior auditors to systematize their expertise, while less experienced personnel may overlook key risk points due to a lack of guidance. For example, newly hired auditors may neglect the core indicator of "qualification verification of discount applicants." Due to a lack of dynamic updates, the system cannot automatically update after new regulations and reports are implemented, leading to outdated information causing interference. For instance, the old version of the "Bill Discounting Management Measures" may still be incorrectly cited.

[0050] After discovering the aforementioned problems in the auditing scenario, the inventors proposed technical concepts such as multi-source data integration, knowledge graph construction, GraphRAG retrieval enhancement, and dynamic update mechanism to solve these problems.

[0051] In the process of integrating multi-source data, entities (such as "audit points" and "audit indicators") can be extracted from unstructured documents (such as audit manuals and historical reports) using a large language model, and structured data (such as business system logging points) can be parsed according to rules to build a preliminary audit entity set.

[0052] To address the issue of missing connections between entities, intelligent connections can be made by designing sub-models (such as audit key points-indicator connection sub-models) and eliminating redundant content through vector similarity calculations, forming a non-redundant audit knowledge graph with hierarchical relationships. This problem can then be solved by constructing a knowledge graph.

[0053] To address the issues of simplistic retrieval logic and insufficient contextual understanding, this paper introduces graph retrieval to enhance generation. Based on cypher queries, it efficiently retrieves related entities and leverages the semantic parsing capabilities of LLM to map natural language input into structured audit tags (such as "section-module-scenario"). This ultimately generates a complete audit path containing risk points, inspection methods, and regulatory justifications. A dynamic update mechanism is constructed, allowing for continuous iteration of the knowledge graph through user feedback and automatic extraction of new documents, ensuring the timeliness and accuracy of audit path generation.

[0054] In view of this, embodiments of this application provide an audit path generation method. This method constructs a structured knowledge graph of multi-source heterogeneous audit data and combines graph retrieval to enhance generation, thereby realizing the dynamic integration of audit knowledge and intelligent audit path generation.

[0055] The method in this application uses a large language model as its core, forming a hierarchical audit knowledge system through entity extraction and relation modeling. Then, based on graph topology reasoning and natural language understanding capabilities, it maps user-input audit requirements into structured path recommendations. This method solves the problems of fragmented knowledge, simplistic retrieval logic, and insufficient experience accumulation in existing audit systems.

[0056] The method described in this application is applicable to audit scenarios in multiple fields. For example, in the fintech field, it is suitable for scenarios where banks and non-bank financial institutions need to conduct audits frequently. For instance, in a special audit of bank acceptance bill discounting business, auditors need to quickly obtain audit focus, historical issues, audit models, and institutional basis through the system. The system stores audit entities (such as "audit modules-scenarios-focuses-key points-indicators") and their relationships through a graph database, and realizes preset scenario tag selection and natural language query based on GraphRAG enhanced retrieval.

[0057] The system includes preset scenario tags such as corporate lending, bill business, and bank acceptance bill discounting. When querying using natural language, for example, if a user enters "how to conduct a bill discounting audit," the system can output highly accurate responses for the user's reference. The system can parse user input into structured audit tags and, through graph retrieval and topology analysis, generate a complete audit path including risk points, inspection methods, and regulatory basis, achieving automatic and efficient recommendation of audit paths.

[0058] The method provided in this application can be applied to applications, websites, or mini-programs that have audit path generation task processing functions. When implementing the audit path generation task processing function on an application, website, or mini-program, for example, a computer with an audit path generation application deployed can implement the audit path generation task processing function by running the audit path generation application. Another example is a terminal electronic device, such as a mobile phone, with an audit path generation mini-program deployed; by running the audit path generation mini-program, the audit path generation task processing function can be implemented.

[0059] The technical solutions of this application will be described in detail below with reference to specific embodiments. The specific embodiments described below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0060] Figure 1 This is a flowchart illustrating the audit path generation method provided in this application embodiment. The executing entity of this method can be an electronic device with corresponding data storage and computing capabilities, such as a computer, server, or server cluster. Figure 1 As shown, the method includes:

[0061] S101, Extract multi-source heterogeneous data from audit policy documents, historical audit reports and / or business system data.

[0062] For example, multi-source heterogeneous data can be understood as data from different sources and in different formats, such as audit policy documents, historical audit reports, and business system data. For instance, audit policy documents are unstructured text, while business system data is structured tabular data; both have different data formats and are therefore multi-source heterogeneous data.

[0063] By employing appropriate reading tools tailored to different data formats or sources, it is possible to extract multi-source heterogeneous data. For example, document parsing tools can extract multi-source heterogeneous data from audit policy documents. Template parsing tools, table extraction tools, or regular expressions can be used to extract multi-source heterogeneous data from historical audit reports. Database query languages ​​or application programming interfaces (APIs) can be used to extract multi-source heterogeneous data from business system data.

[0064] S102 identifies audit entities and their relationships in multi-source heterogeneous data through a natural language processing model, generating a set of structured entities and a set of relationships.

[0065] For example, a natural language processing model can be understood as a text processing tool based on a large language model, used to identify entities in text (such as "audit points" and "audit indicators") and their relationships (such as "audit points - audit indicators"). For instance, an audit entity for "audit points" can be extracted from a document titled "Identification of False Trade Backgrounds" using an LLM model.

[0066] A structured entity set can be understood as a collection of audit entities extracted and organized using a natural language processing model, such as a list containing multiple types of entities like "audit modules" and "audit indicators." A set of relationships can be understood as a collection of relationships between audit entities identified through a natural language processing model, such as relationships like "audit key points - audit indicators" or "audit scenarios - audit systems."

[0067] S103, store the structured entity set and the set of relationships into a graph database to obtain an audit knowledge graph with hierarchical relationships.

[0068] For example, a graph database can be understood as a database system that supports the storage and querying of graph structures, and can be used to store audit knowledge graphs. Examples include graph databases like JanusGraph. An audit knowledge graph can be understood as a hierarchical graph composed of a set of structured entities and a set of relationships, and can be used to generate audit paths. For example, a knowledge graph containing hierarchical relationships such as "audit section-module-scenario-key point-point-indicator".

[0069] For example, based on the steps described above, audit path generation can be achieved through the following implementation methods. For instance, multi-source data integration can be performed first. The system extracts heterogeneous data from audit policy documents (such as the bill discounting management regulations), historical audit reports (such as the "identification of false trade backgrounds" case), and business system data (such as centralized procurement system logs) using data reading tools to ensure the comprehensiveness of audit knowledge. Entity and relationship extraction can then be performed.

[0070] Unstructured documents are parsed using a natural language processing model to identify audit entities (such as "audit points" and "audit indicators") and redundant content is eliminated through vector similarity calculation to generate a set of structured entities. Simultaneously, relationships between entities (such as "audit points - audit indicators") are established through sub-models (such as the "audit points - audit indicators" association sub-model), forming a set of related relationships.

[0071] Furthermore, a knowledge graph can be constructed and stored. The generated set of structured entities and relationships can be stored in a graph database to construct an audit knowledge graph with hierarchical relationships. For example, the "qualification review of discount applicants" indicator in the "bank acceptance bill discounting" scenario can be linked to the "bill discounting management regulations" and persistently stored in the form of a graph structure to provide a data foundation for the generation of subsequent audit paths.

[0072] S104, Generate audit paths based on audit knowledge graph.

[0073] For example, after storing the audit knowledge graph, the specific plan for audit implementation can be planned based on the current audit requirements to generate audit paths.

[0074] For example, key audit entities can be extracted based on current audit requirements. These extracted audit entities can then be retrieved using GraphRAG retrieval enhancements to obtain other related audit entities and their relationships. This allows for the planning and generation of at least one audit path, which can then be used to recommend optimal audit paths to users.

[0075] For example, the method provided in this application can extract structured audit tags by parsing user input using LLM. Based on natural language input (such as user questions or preset scenario selections), audit requirements can be accurately identified, and structured audit paths can be generated through graph topology reasoning. Executing Cypher queries based on the graph database can recall associated audit entities, such as "audit focus" and "audit template." Furthermore, audit paths can be constructed through topology analysis.

[0076] During path generation, the semantic parsing capabilities of LLM can be combined to dynamically supplement audit entities related to the user's input context. Through LLM's semantic parsing and graph topology analysis, the shortcomings of existing audit systems, such as simplistic retrieval logic and insufficient contextual understanding, can be overcome.

[0077] For example, when a user enters "How to conduct a bill discounting audit at Branch X", the LLM parses the natural language into a scenario label of "corporate credit / bill business / bank acceptance bill discounting", and recalls related audit entities such as "discount applicant qualification review" and "trade contract authenticity verification" through Cypher query statements.

[0078] Subsequently, the system constructs an audit path at the "scenario -> key point -> essential point -> indicator" level through topology analysis, and dynamically supplements it with institutional basis such as the "Bill Discounting Management Measures," ultimately generating a complete audit path covering risk points, inspection methods, and institutional basis, thus realizing audit path generation. This method can improve the accuracy and comprehensiveness of audit path generation and avoid the fragmented results caused by existing keyword matching.

[0079] The audit knowledge graph stored in the graph database supports efficient topology queries and dynamic updates. For example, in the scenario of "bank acceptance bill discounting," the system can quickly retrieve the "discount applicant qualification review" indicator based on the audit knowledge graph and associate it with the "bill discounting management regulations" to achieve structured recommendations for audit paths. This method can improve the standardization and intelligent reuse efficiency of audit knowledge, providing auditors with accurate path recommendation support.

[0080] By combining natural language processing models with graph databases, the problems of fragmented audit knowledge and difficulty in integrating multi-source data are solved. Natural language processing models are used to uniformly parse unstructured documents (such as audit manuals) and structured data (such as logs from business systems), integrating scattered audit information into a set of structured entities. This avoids the low retrieval efficiency caused by data silos in traditional audit systems and achieves the integration of multi-source heterogeneous data.

[0081] The audit path generation method provided in this application extracts multi-source heterogeneous data from audit policy documents, historical audit reports, and business system data. It then uses a natural language processing model to identify audit entities and their relationships within this data, generating a structured entity set and a set of relationships. This method achieves entity integration of audit-related multi-source heterogeneous data, overcoming the low retrieval efficiency caused by silos of heterogeneous data. Furthermore, the structured entity set and the set of relationships are stored in a graph database, resulting in an audit knowledge graph with hierarchical relationships that can be used for audit path generation. This allows for the generation of audit paths with high coverage and efficiency based on the audit knowledge graph, improving both audit coverage and efficiency.

[0082] For example, in order to improve the parsing efficiency of unstructured audit documents through parallel computing while maintaining the integrity of semantic associations, unstructured documents can be split into independent logical blocks according to semantic logic based on natural language processing.

[0083] In one possible implementation, extracting multi-source heterogeneous data includes: splitting audit policy documents into at least one logical block, each logical block corresponding to a predefined audit topic; extracting structured audit entities from business system data through a rule parsing module; and merging at least one logical block and the structured audit entities according to the audit topic to obtain a multi-source heterogeneous data set including multiple multi-source heterogeneous data.

[0084] For example, a logic block can be understood as an independent part of an unstructured document (such as an audit manual) divided according to semantic logic, such as logic blocks for audit topics such as "system clauses," "case descriptions," and "operational procedures." A rule parsing module can be understood as a tool for extracting structured data based on preset rules (such as field matching and regular expressions), such as extracting the "audit template" field from business system logs.

[0085] A multi-source heterogeneous dataset can be understood as a dataset that integrates logical blocks and structured data. It includes extracted heterogeneous data from multiple sources. For example, a multi-source heterogeneous dataset might be a comprehensive dataset containing heterogeneous content such as "rule of law," "audit templates," and "operational procedures."

[0086] For example, audit policy documents (such as the bill discounting management regulations) can be semantically broken down into independent logical blocks such as "policy clauses" and "case descriptions," with audit entities extracted from each block in parallel. This logical block splitting facilitates subsequent processing. Structured audit entities (such as "audit templates" and "audit data sources") can be extracted from business system data (such as centralized procurement system logs) using a rule parsing module (such as regular expression matching), achieving structured data extraction.

[0087] By merging the split logical blocks and extracted structured entities into a unified multi-source heterogeneous data set, the data set can be merged, improving the comprehensiveness of audit knowledge. For example, merging the "institutional clauses" logical block with the "audit template" entity forms a multi-source heterogeneous data set containing text descriptions and structured data.

[0088] Distributed computing frameworks (such as Spark) can be used to call LLM in parallel to parse the logical blocks, allowing for the synchronous extraction of entities and relationships. After block parsing is complete, the logical relationships between the logical blocks can be reconstructed through graph topology analysis.

[0089] Parallel parsing through block partitioning improves the processing efficiency of unstructured documents while avoiding knowledge fragmentation caused by partitioning. For example, when processing a 500-page audit manual, the system can split the document into 50 logical blocks for parallel processing, reducing the parsing time from tens of hours in single-threaded mode to several hours. By reconstructing inter-block associations, the system ensures the integrity of the audit knowledge graph, avoiding semantic breaks caused by block parsing. This method not only improves the processing efficiency of audit knowledge but also maintains the semantic coherence of the knowledge graph, adapting to the needs of large-scale document processing.

[0090] In this embodiment, the efficiency and accuracy of data integration are improved through the coordinated processing of logical block splitting and structured data extraction. For example, logical block splitting can avoid interference from redundant content in unstructured documents, while the rule parsing module can accurately extract key structured data from the business system, thereby ensuring the integrity and consistency of multi-source heterogeneous data sets. This method can provide a high-quality data foundation for the subsequent construction of audit knowledge graphs and significantly reduce the complexity of data integration.

[0091] In one possible implementation, after identifying audit entities and their relationships in multi-source heterogeneous data through a natural language processing model, the method further includes: performing deduplication processing on the audit entities by vector similarity calculation to obtain a deduplicated set of structured entities.

[0092] For example, vector similarity calculation can be understood as an algorithm that calculates the similarity between vectors after converting text into vectors. For instance, cosine similarity can be used to determine whether "discount applicant qualification review" and "discount applicant credit review" are similar entities.

[0093] After the natural language processing model generates a structured entity set, the audit entities within the structured entity set can be further optimized through vector similarity calculation. For example, vectorization can be performed first, converting each audit entity into its own vector representation. For instance, audit entities such as "discount applicant qualification review" and "discount applicant credit review" can be vectorized to obtain their respective entity vectors.

[0094] Furthermore, similarity can be calculated for each entity vector. For example, the cosine similarity between entity vectors can be calculated. If the similarity is greater than or equal to a preset threshold (such as 0.8), they are determined to be similar entities; if the similarity is less than the preset threshold, they are determined to be dissimilar entities.

[0095] Furthermore, deduplication can be performed on similar entities. For each audit entity determined to be similar, the similar entities can be merged into one audit entity through selection, fusion, or other merging rules. This allows for deduplication of similar entities to eliminate ambiguity, resulting in a deduplicated set of structured entities.

[0096] For example, if “Discount Applicant Qualification Review” and “Discount Applicant Credit Review” are similar entities, they can be merged into an audit entity called “Discount Applicant Qualification and Credit Review” according to the entity merging rules.

[0097] After eliminating redundant content through vector similarity calculation, sub-models (such as the audit key points-indicator association sub-model) can be used to establish relationships between the deduplicated audit entities (such as establishing associations between audit key points and audit indicators), forming a hierarchical set of associations. This process can improve the systematic association of audit knowledge and provide a logical foundation for subsequent path recommendation.

[0098] In this embodiment, deduplication is performed using vector similarity calculation, which improves the accuracy and conciseness of the entity set. This avoids similar audit entities being mistakenly identified as different independent entities, thereby reducing the number of redundant audit entities. This method can provide a clearer and more concise structured entity set for the construction of audit knowledge graphs, improving the accuracy of subsequent audit path generation.

[0099] By leveraging the textual inductive capabilities and vector similarity calculations of LLM, scattered audit knowledge can be integrated into structured entities and relationships, thus solving the problem of data silos in existing audit systems.

[0100] For example, in the scenario of "bank acceptance bill discounting," the system can automatically extract "identification of false trade background" as an audit point from unstructured documents and generate audit indicators such as "verification of the authenticity of trade contracts." Furthermore, redundant content can be eliminated through vector similarity calculation, ultimately forming a concise audit knowledge graph with hierarchical relationships. This audit knowledge graph not only achieves standardized accumulation of audit knowledge but also provides a data foundation for subsequent path generation and recommendation through entity relationship modeling, improving the reuse efficiency and systematic integration capabilities of audit knowledge.

[0101] For example, in order to continuously optimize the audit path generation results through a user feedback correction mechanism and solve the problems of insufficient experience accumulation and path omissions, the method provided in this application embodiment can provide a supplementary feedback entry on the recommended results interface. Users can mark missing audit entities in the supplementary feedback entry, such as "Discount applicant qualification review not mentioned".

[0102] Based on this, missing audit entities reported by users can be automatically added to the audit knowledge graph, and relationships can be established with already stored audit entities, such as linking "bill discounting scenario" with the supplemented "discounting applicant qualification review". This allows for the improvement of the completeness of the audit knowledge graph through a user feedback correction mechanism, thereby enhancing the accuracy of audit path generation.

[0103] In one possible implementation, before storing the structured entity set and the set of relationships in the graph database, the method further includes: receiving a user's request to modify the structured entity set; identifying audit entities in the modification request using a natural language processing model to obtain supplementary audit entities; and adding the supplementary audit entities to the structured entity set.

[0104] For example, a user feedback module set up in the system can be used to receive user requests for corrections to a set of structured entities. The user feedback module can be understood as an interactive interface that receives correction requests input by auditors, such as a supplementary feedback entry provided on the recommendation results interface. This supplementary feedback entry can be a text input box, an image input control, and / or an audio input control, etc.

[0105] When optimizing the structured entity set through the user feedback module, users can submit correction requests, such as inputting the natural language text "Discount applicant qualification review not mentioned." LLM is used to perform semantic understanding on "Discount applicant qualification review not mentioned," and based on the semantic analysis results, the missing audit entity "Discount applicant qualification review" is added to the structured entity set. Afterwards, relationships can be established among the audit entities in the optimized structured entity set.

[0106] A user feedback correction mechanism can form a closed loop of user feedback, solving the problems of insufficient experience accumulation and missing paths in the existing audit system. For example, when a user reports "the qualification review of discount applicants is not covered," the system can add the audit entity to the audit knowledge graph and automatically recommend it in similar scenarios in the future.

[0107] Furthermore, by incrementally fine-tuning the LLM, the system can learn implicit needs from user feedback, such as "prioritizing qualification review in bill discounting audits," thereby optimizing the LLM's entity extraction logic and path generation capabilities in similar scenarios. This method not only improves the practicality of audit path generation but also achieves the systematic accumulation of audit experience, reducing the learning cost for users through manual learning.

[0108] In this embodiment of the application, by receiving user requests to modify the structured entity set, the structured entity set can be dynamically modified, thereby improving the comprehensiveness, practicality, and adaptability of audit knowledge.

[0109] For example, in scenarios where auditing systems are frequently updated, if only the affected regions of the auditing knowledge graph are partially updated, rather than the entire auditing knowledge graph being rebuilt, the maintenance cost of the auditing knowledge graph can be reduced and its timeliness ensured.

[0110] In one possible implementation, the method further includes: comparing and analyzing multiple audit system documents using a natural language processing model to identify newly added and / or modified target audit entities; and updating the audit entities corresponding to the target audit entities and their relationships in the audit knowledge graph based on the target audit entities to obtain an updated audit knowledge graph.

[0111] For example, a natural language processing model can be used to perform natural language understanding and analysis on multiple audit policy documents, both old and new, and then compare and analyze them to identify newly added and / or modified target audit entities. These target audit entities are related to the audit entities already stored in the audit knowledge graph. By replacing or supplementing these target audit entities, the audit knowledge graph can be supplemented or updated, thus achieving a partial update of the audit knowledge graph.

[0112] For example, when a new version of the audit policy document is uploaded to the system, a partial update of the audit knowledge graph can be triggered. This update and storage only applies to newly added or modified audit entities and their relationships, avoiding a full reconstruction. This approach can be understood as an incremental update mechanism, which updates only the affected graph regions instead of rebuilding the entire graph. This allows for fine-tuning to update the audit knowledge graph.

[0113] When a new target audit entity (such as "anti-money laundering review") conflicts with an existing audit entity (such as "money laundering risk assessment"), the conflict can be resolved through rules (such as prioritizing the new target audit entity). A conflict resolution strategy can be understood as a strategy that eliminates conflict by setting pre-defined rules to resolve conflicts between a new target audit entity and an existing audit entity. For example, setting "prioritizing the audit entity identified in the latest audit policy document" as a pre-defined rule for conflict resolution can handle conflicts between the new target audit entity and the existing audit entity.

[0114] This method overcomes the resource waste and time delays caused by full reconstruction in audit knowledge graph maintenance through an incremental update mechanism. For example, after the audit policy is updated, the system only needs to partially update the nodes related to the addition and / or modification in the audit knowledge graph, rather than re-parse all documents for full generation, thus avoiding the decrease in processing efficiency caused by full reconstruction.

[0115] Meanwhile, when conflicts arise between the target audit entity and the audit entity in the audit knowledge graph, a conflict resolution strategy can be employed to ensure high accuracy and consistency of the updated audit knowledge graph. This method reduces the maintenance cost of the knowledge graph and ensures the timeliness of audit path generation results, adapting to the needs of audit scenarios with frequent policy updates.

[0116] In this embodiment, an incremental update mechanism reduces the maintenance time cost of the audit knowledge graph and ensures the timeliness of updates. This method only updates the topology of audit entities involved in the changes and their relationships, saving maintenance costs and improving the maintenance efficiency of the audit knowledge graph.

[0117] In one possible implementation, the natural language processing model includes a large language model and a rule engine. The natural language processing model identifies audit entities in multi-source heterogeneous data, including: extracting entities from multi-source heterogeneous data of unstructured documents using the large language model to obtain audit entities related to audit points and audit indicators; and / or, parsing audit entities related to audit templates and audit data sources from structured multi-source heterogeneous data using the rule engine.

[0118] For example, a rules engine can be understood as a tool that parses structured data based on preset rules (such as field matching), such as extracting the "audit template" field from business system logs.

[0119] Large language models can be used to perform semantic understanding and analysis on unstructured documents, such as audit manuals and audit policy documents, to extract audit entities and generate audit entities related to audit key points and audit indicators. Audit key points can be the core areas of focus for auditing, while audit indicators can be specific standards for quantifying these key points.

[0120] The rules engine can parse audit entities related to audit templates and audit data sources from structured data, such as logs from business systems. Audit templates can be standardized audit operation formats, and audit data sources can be the foundation for audit evidence.

[0121] In this embodiment, the comprehensiveness and accuracy of entity extraction are improved through the collaborative processing of a large language model and a rule engine. For example, the large language model can handle complex semantics in unstructured documents, while the rule engine can accurately extract key fields from structured data, thereby ensuring the integrity of the entity set.

[0122] The method provided in this application can efficiently integrate heterogeneous data from multiple sources, such as audit policy documents, historical audit reports, and business system data, to construct a structured audit knowledge graph and solve the problems of knowledge fragmentation and missing connections. This method can extract entities from unstructured documents using a large language model and parse structured data based on a rule engine to generate a set of structured entities. By calculating the vector similarity between entities using sub-models, redundant audit entities can be eliminated and entity relationships can be established. In this way, audit entities and their relationships can be stored in a graph database, forming a hierarchical audit knowledge graph.

[0123] In one possible implementation, generating an audit path based on an audit knowledge graph includes: retrieving multiple audit path points from the audit knowledge graph through multi-hop queries, and generating at least one audit path based on the multiple audit path points; the method further includes: upon receiving user feedback on the multiple audit path points, dynamically sorting the multiple audit path points according to the user feedback, and updating at least one audit path according to the result of the dynamic sorting.

[0124] For example, a multi-hop query can be understood as expanding and planning the coverage of the audit path by traversing the audit knowledge graph multiple times. For instance, a multi-hop query can retrieve multiple audit path points from the audit knowledge graph, such as "audit scenario - audit focus - audit key points - audit indicators." Audit path points can be understood as key or critical nodes in the audit path.

[0125] When performing multi-hop queries, you can start from the audit scenario input by the user (such as "bill discounting") and gradually expand the search for audit path points through multi-hop queries. For example, starting from "bill discounting", you can gradually expand the path coverage to include multiple audit path points related to audit scenarios, audit priorities, audit key points, and audit indicators.

[0126] During this process, the user feedback module can also be used to receive user feedback on the priority of each audit path point. For example, if the user feedback indicates that "the qualification review of the discount applicant is the highest priority," then the audit focus corresponding to "the qualification review of the discount applicant" can be adjusted to the first position in the sorting of each audit path point to ensure that key audit nodes are displayed first.

[0127] For example, in the scenario of bill discounting audit, the audit can be expanded from the "bill discounting scenario" node to multiple audit path points such as "qualification review of discounting applicants" and "authenticity verification of trade contracts," and linked to the "Bill Discounting Management Measures" to form a complete audit path covering risk points, inspection methods, and regulatory basis, avoiding omissions caused by traditional single-click queries. Furthermore, the recommendation order can be dynamically adjusted based on user feedback to ensure that key risk points are presented first, thereby reducing the difficulty of understanding for users.

[0128] In addition, when prioritizing paths, recommended paths can be dynamically ranked based on the graph topology (such as node degree and path length) and user historical behavior data.

[0129] In this embodiment, by using multi-hop queries and a dynamic sorting mechanism based on user feedback, the comprehensiveness, practicality, and accuracy of audit path generation can be improved, highly adaptable to user needs, and flexibly sorted to obtain audit paths that meet user requirements.

[0130] In one possible implementation, the method further includes: adjusting the model parameters of the natural language processing model using an incremental fine-tuning algorithm within a preset period.

[0131] For example, the preset cycle can be any preset duration, such as updating and optimizing once a month or once a quarter. Incremental fine-tuning algorithms can be understood as algorithms that locally adjust the model parameters of a natural language processing model based on user feedback data. For example, based on the supplementary audit entity for "discount applicant qualification review" provided by user feedback, the entity extraction logic of the LLM can be fine-tuned to optimize model parameters, thereby improving subsequent entity extraction and path recommendation capabilities.

[0132] For example, model fine-tuning and optimization can be performed through feedback data acquisition and incremental fine-tuning algorithms. The user feedback module can obtain correction requests from users, and the text, images, audio, and video information included in these requests can all be used as optimization samples.

[0133] Based on the acquired feedback data, incremental fine-tuning algorithms such as Low-Rank Adaptation (LoRA) or Quantized Low-Rank Adaptation (QLoRA) can be used to fine-tune the LLM, thereby optimizing its entity extraction and path recommendation capabilities in similar scenarios. For example, the natural language processing model can learn the implicit needs in user feedback, such as "priority should be given to qualification review in bill discounting audits," thus optimizing subsequent recommendation results.

[0134] For example, during the audit entity extraction process, the parameter weights of the large language model can be dynamically adjusted for different audit scenarios (such as "credit audit" and "IT audit"), prioritizing the extraction of entities strongly related to the current scenario. For instance, in the "credit audit" scenario, entities such as "loan approval process" and "risk exposure" can be extracted first.

[0135] Specifically, a scenario classification model is constructed. The LLM is trained to classify input audit scenarios (such as "section-module-scenario" labels) and output scenario feature vectors. When dynamically adjusting parameters, the attention weights of the LLM during the entity extraction stage can be adjusted based on the scenario feature vectors, prioritizing keywords strongly related to the scenario. Furthermore, during audit entity fusion optimization, scenario feature vectors can be introduced as a weighting factor in vector similarity calculation to more accurately deduplicate redundant audit entities.

[0136] By optimizing audit entity extraction based on scenario awareness, the accuracy and relevance of entity extraction in specific audit scenarios can be improved. For example, in the "IT audit" scenario, the system can prioritize the extraction of key entities such as "system permission configuration" and "data encryption" to reduce interference from irrelevant entities (such as "financial statements"), thereby improving the scenario adaptability of the audit knowledge graph and avoiding the problem of audit entity redundancy caused by general extraction models.

[0137] In this embodiment, the incremental fine-tuning algorithm can optimize the model parameters of the natural language processing model in a timely manner within a preset period, thereby improving the entity extraction and path recommendation capabilities of the natural language processing model and enhancing the accuracy and adaptability of audit path generation.

[0138] Figure 2 This is a flowchart of the intelligent audit path recommendation system provided in the embodiments of this application. The audit path generation method provided in the embodiments of this application can be implemented based on the intelligent audit path recommendation system. The following is in conjunction with... Figure 2The audit path generation method of this application embodiment will be further described with reference to the accompanying drawings.

[0139] This application provides an intelligent audit path recommendation system based on GraphRAG, which aims to assist auditors in quickly understanding the inspection path of relevant audit projects. It may include: audit focus, inspection ideas, inspection methods, relevant systems, commonly used models, relevant audit results, and relevant work prompts.

[0140] The Intelligent Audit Path Recommendation System comprises two core functional modules: an efficient mechanism for extracting and dynamically updating audit knowledge, and an intelligent audit knowledge query and application function. In terms of knowledge management, the system effectively integrates scattered knowledge resources, constructs a structured audit knowledge graph, and enables continuous iterative optimization of knowledge. At the application level, through intelligent interaction technology, the system can accurately extract key information within a limited number of dialogue rounds, providing professional guidance to auditors (such as new auditors), enabling them to quickly establish a systematic audit thinking framework and master core audit methods, thereby significantly improving audit efficiency and shortening project cycles.

[0141] The intelligent audit path recommendation system can achieve the following two capabilities. First, it integrates multi-source heterogeneous data (including structured business data, unstructured audit manuals, and system logging logs) and constructs an automatic extraction framework for a hierarchical and structured audit indicator system with dynamic update capabilities based on a large language model. This framework can not only achieve intelligent identification of audit entities, but also automatically construct a multi-dimensional relationship network between entities and persistently store it in the form of a graph structure.

[0142] Second, by combining the deep semantic parsing capabilities of large language models with the topological reasoning advantages of knowledge graphs, a domain-adaptive Graph Retrieval Enhanced Generation (GraphRAG) technology stack has been formed. The audit intelligent path recommendation assistant built upon this technology stack supports two core functions: Function 1, graph traversal-based path recommendation based on arbitrary audit nodes; Function 2, intent understanding and path generation for natural language queries.

[0143] like Figure 2 As shown, the execution process of the intelligent audit path recommendation system includes the construction of an audit indicator extraction system and the construction of an intelligent audit path recommendation assistant. For scenarios involving new audit entities (audit indicators), after users upload unstructured documents such as audit policy documents, the audit indicator extraction system automatically extracts audit indicators. Optionally, the extracted audit indicators can be corrected. This correction can be done manually. For scenarios involving updated audit indicators, the system can perform batch updates and manual intervention before the extracted audit entities are stored in the database to form an audit knowledge graph (audit indicator library).

[0144] Figure 3 This is a schematic diagram illustrating the audit entities and their relationships provided in the embodiments of this application. Figure 3 This illustrates the audit entities and their interrelationships within the automatic extraction framework of the audit indicator extraction system. The sources for constructing the audit indicator extraction system can be categorized into three parts: audit policy documents (such as audit work manuals), historical audit reports, and business system data. By sorting and extracting frequently used and key documents during the audit process, and processing the embedded data, for example, 12 key audit entities (audit sections, audit modules, audit scenarios, audit focus, audit key points, work prompts, audit policies, audit issues, audit standards, audit reports, audit templates, and audit data sources) and their corresponding 12 relationships can be obtained, as shown in Figure 3.

[0145] For example, Table 1 is an audit entity extraction description table, and the entity names and content descriptions are shown in Table 1.

[0146] Table 1

[0147]

[0148] Among them, audit key points and audit standards are relatively important audit entities, which can be generated using audit standard generation sub-models and audit key point generation sub-models, respectively. The essence of the two sub-models is to write prompt words and combine them with the text summarization capabilities of the larger model to summarize and generalize the text.

[0149] Considering the potential for similar content within the extracted audit standards / audit points set, entity disambiguation is necessary. This can be achieved by converting audit entities into embedding vectors, calculating the similarity between vectors, and classifying them as similar if the similarity exceeds a threshold. The remaining audit entities can then be obtained by parsing from business system data, audit standards manuals, or issue lists according to certain rules.

[0150] Table 2 is an explanation of the construction of audit entity relationships, detailing the relationships between various audit entities. The relationships between audit points and audit standards, audit standards and audit templates, audit scenarios and work prompts, and audit scenarios and audit procedures are difficult to obtain directly from business system data. These relationships can be established through intelligent extraction and correlation analysis using the diverse prompts provided by the system.

[0151] Table 2

[0152]

[0153] Based on the extracted audit entities and their relationships, a hierarchical framework for extracting audit indicators was constructed. This framework was developed and deployed within the audit system, supporting users in constructing audit entities by uploading files and modifying the extracted entities. It also enables automated data integration with a graph database, dynamically updating the audit entity set. Considering the complex node-edge relationships in this scenario, storing the data and relationships in a graph database facilitates deeper retrieval and dynamic data updates.

[0154] The intelligent recommendation system for audit paths offers users two interaction methods: preset scenario selection and natural language input. The preset scenarios are based on a hierarchical tagging system built from an audit business knowledge base, employing a three-level classification structure of "section-module-scenario".

[0155] For example, users can select the "Corporate Credit / Bill Business" section and module, and then further select the "Bank Acceptance Bill Discounting" scenario. For more flexible query needs, users can directly input natural language questions, such as "Please help generate the audit path for the special audit project of bank acceptance bill discounting business of Branch X". The intelligent audit path recommendation system uses the semantic understanding capabilities of a large language model to automatically map natural language queries into structured tags (including audit sections, modules, and scenarios), and the subsequent processing flow remains consistent with the preset scenario mode. This dual-mode design ensures both the query efficiency of standard business scenarios and the flexibility to meet complex audit needs.

[0156] like Figure 2 As shown, after a user asks a question, the intelligent audit path recommendation assistant can determine whether the question falls within a preset scenario. If not, it can perform intent recognition and map the preset scenario based on the intent recognition result, then perform a GraphRAG retrieval. If the scenario does fall within the preset scenario, a GraphRAG retrieval can be performed directly. Entity retrieval is executed based on pre-set Cypher query statements in the graph database, achieving efficient recall and processing of related entities. Subsequently, using these audit entities as nodes, their topological relationship network is further retrieved. Through in-depth processing and multi-dimensional analysis of the graph query results, a complete audit path recommendation scheme is finally constructed, generating the answer. Based on this, a closed-loop processing flow from data retrieval to knowledge reasoning is formed.

[0157] Figure 4 This is a sample diagram of the audit path provided in the embodiments of this application. The sample result returned by the intelligent audit path recommendation assistant is shown below. Figure 4 As shown. The main contents of the audit path include: audit focus, previous audit findings, relevant audit models, relevant audit systems, and relevant reference materials.

[0158] The audit focus can be broken down into a hierarchy of "key points - essential points - indicators" or "scenario - key points - essential points". Previous audit findings can be used as a reference for audit operations, including previously identified audit issues and their descriptions. Relevant audit models provide template data support for operations. Relevant audit policies, both internal and external, serve as the basis for auditing. Relevant reference materials provide audit methodologies for audit operations.

[0159] In addition, the audit path returned by the model can also include a visual representation of the audit path. For example... Figure 4 As shown, a visual representation similar to a mind map or relationship diagram can be used to show users the audit path points and the relationships between them. This can help users understand the audit path more easily and reduce the difficulty of understanding.

[0160] Based on the audit path generation method provided in this application embodiment, the audit knowledge system is systematically integrated. This method can reduce labor costs and comprehensively improve audit efficiency.

[0161] The method provided in this application integrates multimodal audit data resources, including unstructured data such as audit policy documents, historical audit reports, and issue lists, as well as structured data such as business system data, audit templates, and event logs. Through the collaborative application of large language models and entity fusion technology, a structured audit knowledge graph with banking characteristics is constructed, realizing the standardized accumulation and intelligent reuse of audit knowledge.

[0162] The method provided in this application offers a comprehensive and clear audit strategy, enabling rapid audit execution, reducing the audit burden, and effectively addressing efficiency bottlenecks caused by differences in auditor experience. Furthermore, this method can effectively reduce the workload of audit work; for example, obtaining an audit path in a single instance can save several hours of work. In typical application scenarios, the method provided in this application can free up the workload of several full-time auditors annually.

[0163] Figure 5 This is a schematic diagram of the audit path generation device provided in the embodiments of this application, as shown below. Figure 5 As shown in the figure, this application embodiment provides an audit path generation device, which includes: an extraction module 501, used to extract multi-source heterogeneous data from audit policy documents, historical audit reports and / or business system data; an identification module 502, used to identify audit entities and their relationships in the multi-source heterogeneous data through a natural language processing model, and generate a structured entity set and a relationship set; a storage module 503, used to store the structured entity set and the relationship set into a graph database to obtain an audit knowledge graph with hierarchical relationships; and a generation module 504, used to generate audit paths based on the audit knowledge graph.

[0164] In one possible implementation, the extraction module 501 is specifically used to: split the audit system document into at least one logical block, each logical block corresponding to a predefined audit topic; extract structured audit entities from business system data through the rule parsing module; and merge at least one logical block and structured audit entities according to the audit topic to obtain a multi-source heterogeneous data set including multiple multi-source heterogeneous data.

[0165] In one possible implementation, the apparatus further includes a deduplication module, which is used to: perform deduplication processing on the audit entities by vector similarity calculation to obtain a deduplicated set of structured entities.

[0166] In one possible implementation, the apparatus further includes a supplementary module, which is configured to: receive a user's request for correction of the structured entity set; identify the audited entity in the correction request using a natural language processing model to obtain a supplementary audited entity; and add the supplementary audited entity to the structured entity set.

[0167] In one possible implementation, the apparatus further includes an update module, which is used to: compare and analyze multiple audit system documents using a natural language processing model to identify newly added and / or modified target audit entities; and update the audit entities corresponding to the target audit entities and their relationships in the audit knowledge graph according to the target audit entities to obtain an updated audit knowledge graph.

[0168] In one possible implementation, the natural language processing model includes a large language model and a rule engine. The recognition module 502 is specifically used to: extract entities from multi-source heterogeneous data of unstructured documents through the large language model to obtain audit entities related to audit points and audit indicators; and / or, parse audit entities related to audit templates and audit data sources from structured multi-source heterogeneous data through the rule engine.

[0169] In one possible implementation, the generation module 504 is specifically used to: retrieve multiple audit path points from the audit knowledge graph through multi-hop queries, and generate at least one audit path based on the multiple audit path points; the device also includes a feedback module, which is used to: dynamically sort the multiple audit path points according to the user feedback when receiving user feedback on the multiple audit path points, and update at least one audit path according to the result of the dynamic sorting.

[0170] The audit path generation apparatus provided in this application embodiment can be used to execute the technical solution of the audit path generation method in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0171] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device of this embodiment may include: at least one processor 601; and a memory 602 communicatively connected to the at least one processor; wherein the memory 602 stores instructions that can be executed by the at least one processor 601, and the instructions are executed by the at least one processor 601 to cause the electronic device to perform the method as described in any of the above embodiments.

[0172] Optionally, the memory 602 can be either standalone or integrated with the processor 601.

[0173] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0174] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method of any of the foregoing embodiments.

[0175] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the foregoing embodiments.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0177] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0178] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU) or other general-purpose processors. The processor can also be a Digital Signal Processor (DSP) or an Application Specific Integrated Circuit (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0179] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), disks or optical discs.

[0180] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Examples of storage media include Static Random-Access Memory (SRAM) or Electrically Erasable Programmable Read Only Memory (EEPROM).

[0181] Storage media can be, for example, erasable programmable read-only memory (EPROM) or programmable read-only memory (PROM). Storage media can also be read-only memory (ROM), magnetic storage, flash memory, magnetic disks, or optical disks. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0182] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside within an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within an electronic device or host device.

[0183] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0184] The sequence numbers of the embodiments in this application are merely for description and do not represent the superiority or inferiority of the embodiments. Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0185] Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0186] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0187] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0188] It should be further noted that although the steps in the flowchart are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated in this document, there is no strict order requirement for the execution of these steps, and they can be executed in other orders.

[0189] Furthermore, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0190] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0191] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0192] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An audit path generation method, characterized in that, The method includes: Extract heterogeneous data from multiple sources from audit policy documents, historical audit reports, and / or business system data; The audit entities and their relationships in the multi-source heterogeneous data are identified using a natural language processing model, generating a set of structured entities and a set of relationships. The structured entity set and the set of relationships are stored in a graph database to obtain an audit knowledge graph with hierarchical relationships; An audit path is generated based on the aforementioned audit knowledge graph.

2. The method according to claim 1, characterized in that, The extraction of multi-source heterogeneous data includes: The audit policy document is divided into at least one logical block, and each logical block corresponds to a predefined audit topic. The rule parsing module extracts structured audit entities from the data of the business system; The at least one logical block and the structured audit entity are merged according to the audit subject to obtain a multi-source heterogeneous data set including multiple multi-source heterogeneous data.

3. The method according to claim 1, characterized in that, After identifying the audit entities and their relationships in the multi-source heterogeneous data using a natural language processing model, the method further includes: The audit entities are de-specified and deduplicated by calculating vector similarity, resulting in a deduplicated set of structured entities.

4. The method according to claim 1, characterized in that, Before storing the structured entity set and the association set in the graph database, the method further includes: Receive user requests for corrections to the structured entity set; The natural language processing model is used to identify the audit entity in the correction request to obtain a supplementary audit entity. The supplementary audit entity is added to the structured entity set.

5. The method according to claim 1, characterized in that, The method further includes: By comparing and analyzing multiple audit policy documents using the natural language processing model, the target audit entities that have been added or modified can be identified. The audit knowledge graph is updated by updating the audit entities corresponding to the target audit entity and their relationships in the audit knowledge graph.

6. The method according to claim 1, characterized in that, The natural language processing model includes a large language model and a rule engine. The step of identifying audit entities in the multi-source heterogeneous data using the natural language processing model includes: The large language model is used to extract entities from multi-source heterogeneous data of unstructured documents to obtain audit entities related to audit points and audit indicators. And / or, the rule engine can be used to parse audit entities related to audit templates and audit data sources from structured multi-source heterogeneous data.

7. The method according to any one of claims 1-6, characterized in that, The generation of audit paths based on the audit knowledge graph includes: Multiple audit path points are retrieved from the audit knowledge graph through multi-hop queries, and at least one audit path is generated based on the multiple audit path points; The method further includes: Upon receiving user feedback regarding the multiple audit path points, the multiple audit path points are dynamically sorted according to the user feedback, and the at least one audit path is updated based on the result of the dynamic sorting.

8. An electronic device, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images