Auditing domain knowledge graph ontology framework construction method based on large language model

Abstract

The invention discloses an audit domain knowledge graph ontology framework construction method based on a large language model, which comprises the following steps: collecting audit domain related data, and preprocessing the collected data; pre-training the large language model by using the preprocessed audit field related data; based on an existing general entity extraction and relation extraction data set, performing fine adjustment on the pre-trained large language model; performing knowledge extraction on the audit field text data preprocessed in the first step by using a large language model to obtain a knowledge extraction result of the audit field; performing standardization processing on a knowledge extraction result; and based on the knowledge extraction result after standardization processing, constructing a knowledge graph ontology framework in the audit field. According to the method, the knowledge of the auditing field can be automatically extracted, summarized and sorted, and a set of complete auditing field knowledge graph ontology framework is formed.

Description

Technical Field

[0001] The present invention relates to the technical fields of knowledge graphs and natural language processing, and in particular to a method for constructing a knowledge graph ontology framework in the auditing field based on a large language model (LLM). Background Art

[0002] With the rapid growth of global data and the increasing complexity of data, the audit field is facing increasingly severe challenges of large data volume, high complexity and the need to quickly and accurately identify potential risks. Traditional audit methods mainly rely on manual data processing and analysis, which is not only inefficient but also prone to errors, and it is difficult to meet the requirements of modern auditing for efficiency, accuracy and comprehensiveness. In particular, when dealing with massive and complex unstructured data, traditional methods are even more inadequate. Therefore, the audit field urgently needs an innovative method that can extract and construct structured knowledge from unstructured data and achieve efficient reuse of knowledge.

[0003] The ontology framework is a key tool for organizing and managing domain knowledge in the audit field, involving the systematic description of domain knowledge and the clear expression of conceptual relationships. Traditional ontology construction methods usually rely on the knowledge and experience of domain experts, and the process is complex and time-consuming. The specific needs and complexity of the audit field further increase the difficulty of construction. Summary of the invention

[0004] The purpose of this invention is to provide a method for constructing an audit field knowledge graph ontology framework based on a large language model, by utilizing a large language model to automatically extract, summarize and organize the knowledge in the audit field, to form a complete audit field knowledge graph ontology framework, thereby improving the efficiency and accuracy of audit work.

[0005] The technical solution provided by the present invention is:

[0006] In the first aspect, the present application provides a method for constructing a knowledge graph ontology framework in the audit field based on a large language model, comprising:

[0007] Step 1: Collect relevant data in the audit field and pre-process the collected data;

[0008] Step 2: Use the audit-related data preprocessed in step 1 to pre-train the large language model; fine-tune the pre-trained large language model based on the existing general entity extraction and relationship extraction datasets;

[0009] Step 3: Use the large language model to extract knowledge from the audit field text data preprocessed in step 1 to obtain the knowledge extraction results in the audit field;

[0010] Step 4: Standardize the knowledge extraction results; based on the standardized knowledge extraction results, construct a knowledge graph ontology framework in the audit field.

[0011] In a possible implementation manner of the first aspect, the method further includes:

[0012] Step 5: Based on the knowledge graph ontology framework obtained in step 4, score the knowledge extraction results obtained in step 3 to obtain the scoring results; use the scoring results as reward signals to optimize the large language model;

[0013] And iterate steps three to five to complete the iterative optimization of the large language model and the update of the knowledge graph ontology framework.

[0014] In a possible implementation of the first aspect, scoring the knowledge extraction result obtained in step three includes:

[0015] Completeness score, used to evaluate whether all entities and relations in the knowledge graph ontology framework are correctly extracted;

[0016] Redundancy score: used to evaluate whether there are repeated entities or relationships in the knowledge extraction results;

[0017] Accuracy score: used to evaluate the accuracy of the extraction results.

[0018] In a possible implementation manner of the first aspect, the method further includes:

[0019] Step 6: After completing the iterative optimization of the large language model, apply the optimized large language model to the new audit field text data for knowledge extraction to obtain new extraction results; based on the new extraction results, update and expand the knowledge graph ontology framework.

[0020] In a possible implementation of the first aspect, the updating and expanding the knowledge graph ontology framework includes: comparing the newly extracted knowledge with the existing knowledge graph ontology framework, adding new entities and relationships, and removing redundant or erroneous information.

[0021] In the second aspect, the present application provides a system for constructing a knowledge graph ontology framework in the audit field based on a large language model, including:

[0022] Data collection and preprocessing module, used to collect audit-related data and preprocess the collected data;

[0023] Large language model pre-training and fine-tuning module, used to pre-train the large language model using pre-processed audit domain-related data; fine-tune the pre-trained large language model based on the existing general entity extraction and relationship extraction datasets;

[0024] The knowledge extraction module is used to extract knowledge from the pre-processed audit field text data using a large language model to obtain knowledge extraction results in the audit field;

[0025] The knowledge graph ontology framework construction module is used to standardize the knowledge extraction results; based on the standardized knowledge extraction results, the knowledge graph ontology framework in the audit field is constructed.

[0026] In a possible implementation manner of the second aspect, the system further includes:

[0027] The model iteration optimization module is used to score the knowledge extraction results based on the obtained knowledge graph ontology framework to obtain the scoring results; use the scoring results as reward signals to optimize the large language model; and complete the iterative optimization of the large language model and the update of the knowledge graph ontology framework.

[0028] In a possible implementation of the second aspect, scoring the knowledge extraction result includes:

[0029] Completeness score, used to evaluate whether all entities and relations in the knowledge graph ontology framework are correctly extracted;

[0030] Redundancy score: used to evaluate whether there are repeated entities or relationships in the knowledge extraction results;

[0031] Accuracy score: used to evaluate the accuracy of the extraction results.

[0032] In a possible implementation manner of the second aspect, the system further includes:

[0033] The knowledge graph ontology framework update and expansion module is used to apply the optimized large language model to new audit field text data for knowledge extraction after completing the iterative optimization of the large language model to obtain new extraction results; based on the new extraction results, the knowledge graph ontology framework is updated and expanded.

[0034] In a possible implementation of the second aspect, the updating and expanding the knowledge graph ontology framework includes: comparing the newly extracted knowledge with the existing knowledge graph ontology framework, adding new entities and relationships, and removing redundant or erroneous information.

[0035] In a third aspect, the present application provides an electronic device, including: a memory and a processor;

[0036] The memory is used to store computer programs;

[0037] The processor is used to call the computer program to execute the method as described above.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device implements the method described above.

[0039] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is run on an electronic device, the electronic device implements the method as described above.

[0040] The specific implementation methods of the second to fifth aspects of the present application can refer to the implementation methods of the first aspect, which will not be repeated here.

[0041] Beneficial effects:

[0042] Improve audit efficiency: By automatically processing large amounts of text data, the time and cost of manual processing is reduced, thereby improving audit efficiency.

[0043] Improve audit accuracy: Using large language models for entity recognition and relationship extraction can more accurately extract and organize knowledge in the audit field, reducing the possibility of human error.

[0044] Facilitates knowledge sharing and reuse: The constructed ontology framework can clearly represent the concepts, entities and the relationships between them in the audit field, facilitating knowledge sharing and reuse among auditors.

[0045] Strong scalability: As knowledge in the audit field continues to be updated and changed, the ontology framework can be easily updated to adapt to new audit needs. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the present application, the technical solution in this embodiment will be further described in detail below in conjunction with this embodiment.

[0047] As an advanced artificial intelligence technology, the large language model has demonstrated excellent performance in processing complex language tasks with its powerful ability in natural language understanding and generation. The large language model not only has the ability of deep semantic understanding, but also has cross-domain adaptability and self-learning ability, which makes it show great potential in applications in multiple industries. The rise of large language models provides new possibilities for automatically extracting and constructing knowledge graphs from massive text data. However, there is currently no effective method for applying large language models to the construction of ontology frameworks in the audit field. Therefore, it is of great significance to develop a new method for applying large language models to the construction of ontology frameworks in the audit field. This method can not only effectively extract and organize domain knowledge by using the language processing capabilities of large language models, but also realize the dynamic reuse of knowledge through automated knowledge generation and update mechanisms. Through this innovative application, the efficiency and accuracy of audit work can be greatly improved, the intelligent development of audit technology can be promoted, and a solid technical foundation can be provided for coping with future data challenges. This application aims to use large language models to construct knowledge graph ontology in the audit field to improve audit efficiency and accuracy. The core features are: 1) automation, 2) verifiable and updatable), and 3) feedback learning.

[0048] The specific implementation of the present application will be described below based on examples.

[0049] Embodiment 1

[0050] This embodiment provides a method for constructing an audit domain knowledge graph ontology framework based on a large language model, and the steps are as follows:

[0051] Step 1: Collect relevant data in the audit field and pre-process the collected data.

[0052] Through this step, text data that meets the input requirements of the large language model is obtained. This step may include:

[0053] 1.2 Data Collection:

[0054] Collect audit-related data from multiple different sources, including but not limited to policy documents, financial statements, audit reports, regulatory documents, industry standards, audit records, etc. These data may exist in multiple formats, such as PDF documents, spreadsheets, text files, etc. Therefore, it is necessary to systematically collect these multi-source data to ensure that a wide range of audit information is covered to support subsequent analysis and processing.

[0055] 1.2 Data preprocessing:

[0056] After data collection is completed, the obtained data is preprocessed. Preprocessing can include cleaning, denoising, unified format and word segmentation. First, delete unnecessary format information to ensure that only the core text content is retained. At the same time, handle missing values ​​and outliers in the data to improve the integrity and accuracy of the data. After cleaning, all data are unified into text data to ensure data consistency, so as to lay a solid foundation for subsequent analysis and processing. After that, the text data is segmented. Thus, the audit field text data after preliminary preprocessing is obtained.

[0057] The main characteristics of audit data include high specialization and a mixture of structured and unstructured data, which makes the processing and analysis of audit data more complicated.

[0058] First, the texts in the audit field contain a large number of professional terms and industry standards, involving financial statements, auditing standards, compliance requirements and other content. For example, terms such as "financial statement audit", "risk assessment", and "internal control testing" are all common concepts in the audit process. These terms have specific contexts and professional meanings. Understanding these concepts not only relies on the literal meaning of the terms, but also requires in-depth understanding of industry background, standards and practical applications. Existing large-scale language models (such as GPT, etc.) are usually unable to accurately capture these deep professional backgrounds and domain knowledge. Therefore, audit data processing must have a deep understanding of the field to ensure the accuracy of the analysis results.

[0059] Secondly, audit data usually contains both structured data and unstructured text data. Structured data, such as financial statements, transaction records, and audit tracking tables, are organized in a tabular or digital manner with clear fields and data formats. Unstructured text includes audit reports, audit working papers, legal and regulatory documents, etc. These texts do not follow a fixed structure and contain a large number of natural language descriptions and professional terms. This diversity of data forms complicates the processing of audit data, especially the existing large-scale language models usually face two major problems: one is the lack of domain knowledge, which makes it impossible to accurately understand and process the professional terms and background in the audit field; the other is the inability to directly process structured tabular data, especially when faced with complex financial data. The model lacks built-in understanding capabilities and cannot effectively parse the numbers in the table, the relationship between rows and columns, and the actual meaning behind the values.

[0060] In response to these challenges, the embodiments of the present application propose two effective solutions.

[0061] First, in order to help the large language model understand the professional terms in the audit field, this application introduces an interpretability enhancement method. Specifically, the definitions and explanations of relevant concepts are crawled from public resources such as Internet data, industry literature, and audit standards, and these explanations are attached to the professional terms as a supplement to natural language. For example, for the term "internal control testing", you can add "refers to the process of evaluating the effectiveness of an enterprise's internal control system", which can help the model better understand the meaning and context of the term and avoid misunderstanding or omission.

[0062] Secondly, for the processing of structured data, this application constructs a table-to-natural text mapping data set. A large amount of information in the audit exists in tabular form (such as financial statements, risk assessment tables, etc.), and these structured data need to be converted into natural language before they can be processed together with other unstructured data. To this end, this application uses deep learning technology to train a table-to-text mapping model that can convert tabular data into natural language descriptions. For example, a financial statement may correspond to a natural language text describing income, expenditure, assets and liabilities, etc. By training this model, complex financial data can be converted into easy-to-understand natural language, thereby providing support for subsequent analysis and reasoning. After completing this step, structured data can enter the processing flow of the large language model like unstructured text.

[0063] Through these two processing methods, this application realizes the integration of structured and unstructured data, and enhances the ability of large language models to understand audit data. First, by adding detailed explanations to audit terms, the model's understanding of professional terms is improved. Secondly, by converting structured data into natural language text, different types of data can be processed on the same platform, avoiding the problem that structured data cannot be processed directly. These improvements not only improve the model's ability to understand the audit field, but also lay the foundation for subsequent intelligent analysis, risk assessment, and knowledge graph construction. Finally, by combining domain knowledge enhancement and tabular data conversion, this application can process audit data more efficiently and promote the intelligent and automated development of the audit process. This not only helps to improve the accuracy and efficiency of audit analysis, but also helps auditors make more accurate judgments and decisions when faced with complex financial data and regulatory requirements.

[0064] Step 2: Use the audit-related data preprocessed in step 1 to pre-train the large language model; fine-tune the pre-trained large language model based on the existing general entity extraction and relationship extraction datasets.

[0065] This step may include:

[0066] 2.1 Pre-training large language models:

[0067] In this step, you can select an open source large language model (such as Llama or ChatGLM) and pre-train the large language model using the audit domain-related data pre-processed in step 1. The goal of pre-training is to optimize the large language model so that it can prepare basic prediction capabilities for the training data.

[0068] Specifically, the large language model will generate the conditional probability distribution of the next word based on the input context, and optimize the parameters of the large language model by maximizing the probability of the generated word sequence. 1 ,u 2 ,…,u n}, the goal of pre-training the large language model is to adjust the parameters of the large language model by maximizing the following objective function:

[0069]

[0070] Among them, θ represents the parameters of the large language model; u i represents the i-th word in the text sequence; u <i represents all the words before the i-th word in the text sequence (i.e., context); P(u i |u <i ; θ) represents the given context u <i Under the condition of i The conditional probability of .

[0071] By maximizing the above objective function, the large language model will learn how to generate the correct next word with the highest probability under the given context, thereby generating a text sequence that is more in line with expectations. This process will significantly improve the ability of the large language model to generate text in the auditing field, making it more suitable for specific application scenarios.

[0072] 2.2 Based on the existing general entity extraction and relation extraction datasets, fine-tune the pre-trained large language model:

[0073] Select existing general entity extraction and relationship extraction datasets, use the original text as the input of the large language model, and use the corresponding entities and relations as the output of the large language model to fine-tune the large language model. By fine-tuning the parameters of the large language model, its performance in knowledge (entity and relationship) extraction tasks can be improved, so that it has general knowledge extraction capabilities.

[0074] Specifically, it may include:

[0075] 2.2.1 Entity Extraction:

[0076] In order to enable the large language model to have entity prediction capabilities, multiple verified entity extraction datasets are selected. These datasets contain labeled entity information, such as names of people, places, organizations, dates, etc. In the large language model training, the input is the original text, and the output is the entities and their types in the original text. An example input and output is:

[0077] Input: Xiao Ming is studying at Tsinghua University.

[0078] Output: Xiao Ming [name] Tsinghua University [institution].

[0079] By learning from this data, the large language model will be able to predict entities and their types in the text.

[0080] 2.2.2 Relationship Extraction:

[0081] In order to train the relationship extraction capability of the large language model, we select annotated datasets containing relationships between entities. These datasets describe the relationships between different entities in the text, such as "employment relationship", "position relationship", etc. In the training of the large language model, the input is the text labeled with two entities, and the output is the relationship between the entities. An example input and output is:

[0082] Input: Xiaoming [person] studies at Tsinghua University [institution].

[0083] Output: "Study relationship".

[0084] By learning from this data, the large language model will be able to predict the relationship between any two entities in the text.

[0085] Step 3: For the auditing field, perform knowledge extraction based on the large language model, and build a knowledge graph ontology framework in the auditing field based on the knowledge extraction results.

[0086] Step 3: Use the large language model to extract knowledge from the audit field text data preprocessed in step 1 to obtain the knowledge extraction results in the audit field.

[0087] The large language model is used to perform entity recognition and relationship extraction on the audit domain text data preprocessed in step 1. First, all entities in the audit domain text data are audited using the entity extraction model, and then the identified entities are paired and input into the relationship extraction model to predict the relationship between them.

[0088] The large language model can be divided into entity extraction model and entity extraction model.

[0089] This step may specifically include:

[0090] Entity Recognition: Use entity extraction models to identify all entities in the audit domain text data. In the audit domain, possible entity types include company names, personnel, assets, dates, financial statements, regulatory provisions, etc.

[0091] Relationship prediction: Pair the identified entities and input them into the relationship extraction model to predict the relationship between them. In the audit field, possible relationship types include ownership, affiliation, transaction relationship, auditor-audited relationship, related issues, etc.

[0092] Step 4: Standardize the knowledge extraction results; based on the standardized knowledge extraction results, construct a knowledge graph ontology framework in the audit field.

[0093] The extracted entities and relationships are standardized to form a knowledge graph ontology framework in the audit field.

[0094] Specifically, they may include:

[0095] 4.1 Entity Standardization:

[0096] The identified entities are normalized to ensure that different representations of the same entity are consistent.

[0097] For example, "financial statements" and "financial reports" are unified into "financial statements".

[0098] 4.2 Relationship Standardization:

[0099] Standardize the identified relationships to ensure that relationships of the same type use a consistent description.

[0100] For example, unify “include” and “include” into “include”.

[0101] 4.3 Knowledge graph ontology framework construction:

[0102] Based on the standardized entities and relationships, a knowledge graph ontology framework in the audit field is constructed.

[0103] The final knowledge graph ontology framework includes definitions of various entities and the relationships between entities.

[0104] In some embodiments, the above method may further include: Step 5: Based on the knowledge graph ontology framework obtained in Step 4, score the knowledge extraction result obtained in Step 3 to obtain a scoring result; use the scoring result as a reward signal to optimize the large language model. And iterate steps 3 to 5 to complete the iterative optimization of the large language model and the update of the knowledge graph ontology framework.

[0105] The constructed knowledge graph ontology framework is optimized to ensure the accuracy and completeness of the framework. The optimization includes eliminating redundant entities and relationships, merging similar entities and relationships, etc.

[0106] This step may include:

[0107] 5.1: Knowledge Extraction Quality Scoring

[0108] Ideally, the knowledge extraction result should be consistent with the knowledge graph ontology framework structure. However, due to the randomness of the knowledge extraction process, this embodiment proposes a feedback update mechanism based on reinforcement learning to perform training iteratively.

[0109] First, the knowledge extraction results can be scored from three dimensions: completeness, redundancy, and accuracy:

[0110] Completeness: Evaluate whether all entities and relations in the knowledge graph ontology framework are correctly extracted.

[0111] For example, for each correct extraction result (entity or relation), the score is added with a preset value.

[0112] For each missing extraction result, the score is subtracted from the preset value. That is, if an entity or relationship in the knowledge graph ontology framework is not extracted, the score is subtracted from the preset value.

[0113] Redundancy: Evaluate whether there are repeated entities or relations in the knowledge extraction results.

[0114] For example, for each repeated extraction result, the score is reduced by a preset value.

[0115] Accuracy: Evaluate the accuracy of the extraction results.

[0116] For example, for each incorrect extraction result, the score is subtracted by a preset value.

[0117] Through these scoring indicators, the quality of knowledge extraction results can be systematically evaluated and provide a basis for subsequent optimization;

[0118] Exemplarily, the above preset value can be 1.

[0119] 5.2: Reinforcement Learning Training

[0120] Based on the above scoring results, reinforcement learning is introduced for the large language model so that it can adjust itself according to the feedback and scoring results. Specifically, the Proximal Policy Optimization (PPO) algorithm is used for training.

[0121] Reward mechanism: Use the scoring results as reward signals to guide the large language model to improve the quality of knowledge extraction.

[0122] For each correct extraction result (entity or relation), a positive reward is given.

[0123] Negative rewards are given for missing, duplicate, or incorrect extraction results.

[0124] Optimization goal: Through the PPO algorithm, optimize the parameters of the large language model so that the accuracy and completeness of knowledge extraction can be gradually improved and redundancy can be reduced in each iteration.

[0125] In some embodiments, the above method may also include: Step 6: After completing the iterative optimization of the large language model, applying the optimized large language model to new audit field text data for knowledge extraction to obtain new extraction results; based on the new extraction results, updating and expanding the knowledge graph ontology framework.

[0126] This step may include:

[0127] 6.1 Apply the optimized large language model: Use the optimized large language model to extract entities and relationships from new audit domain text data.

[0128] 6.2 Update the knowledge graph ontology framework: compare the newly extracted entities and relationships with the existing knowledge graph ontology framework, add new entities and relationships, and remove redundant or erroneous information.

[0129] 6.3 Apply generalization rules: According to the rules of normalization and standardization, ensure that the updated knowledge graph ontology framework remains consistent and generalized.

[0130] In addition, the constructed knowledge graph ontology framework can also be verified, including manual review with audit experts and verification through actual audit cases, and the knowledge graph ontology framework can be updated based on the verification results.

[0131] Embodiment 3

[0132] This embodiment provides a system for constructing an audit domain knowledge graph ontology framework based on a large language model, including:

[0133] Data collection and preprocessing module, used to collect audit-related data and preprocess the collected data;

[0134] Large language model pre-training and fine-tuning module, used to pre-train the large language model using pre-processed audit domain-related data; fine-tune the pre-trained large language model based on the existing general entity extraction and relationship extraction datasets;

[0135] The knowledge extraction module is used to extract knowledge from the pre-processed audit field text data using a large language model to obtain knowledge extraction results in the audit field;

[0136] The knowledge graph ontology framework construction module is used to standardize the knowledge extraction results; based on the standardized knowledge extraction results, the knowledge graph ontology framework in the audit field is constructed.

[0137] In a possible implementation manner of the second aspect, the system further includes:

[0138] The model iteration optimization module is used to score the knowledge extraction results based on the obtained knowledge graph ontology framework to obtain the scoring results; use the scoring results as reward signals to optimize the large language model; and complete the iterative optimization of the large language model and the update of the knowledge graph ontology framework.

[0139] In a possible implementation of the second aspect, scoring the knowledge extraction result includes:

[0140] Completeness score, used to evaluate whether all entities and relations in the knowledge graph ontology framework are correctly extracted;

[0141] Redundancy score: used to evaluate whether there are repeated entities or relationships in the knowledge extraction results;

[0142] Accuracy score: used to evaluate the accuracy of the extraction results.

[0143] In a possible implementation manner of the second aspect, the system further includes:

[0144] The knowledge graph ontology framework update and expansion module is used to apply the optimized large language model to new audit field text data for knowledge extraction after completing the iterative optimization of the large language model to obtain new extraction results; based on the new extraction results, the knowledge graph ontology framework is updated and expanded.

[0145] In a possible implementation of the second aspect, the updating and expanding the knowledge graph ontology framework includes: comparing the newly extracted knowledge with the existing knowledge graph ontology framework, adding new entities and relationships, and removing redundant or erroneous information.

[0146] Embodiment 3

[0147] This embodiment also provides an electronic device, including: a memory and a processor;

[0148] The memory is used to store computer programs;

[0149] The processor is used to call the computer program to execute the method as described above.

[0150] Embodiment 4

[0151] This embodiment further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed on an electronic device, the electronic device implements the method described above.

[0152] Embodiment 5

[0153] This embodiment further provides a computer program product, including a computer program. When the computer program is executed on an electronic device, the electronic device implements the method described above.

[0154] The specific implementation methods of the systems, electronic devices, computer-readable storage media, and computer program products provided in the second to fifth embodiments can refer to the specific implementation methods of the methods provided in the above-mentioned embodiment one, and will not be repeated here.

[0155] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, electronic equipment, computer-readable storage media or computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the present embodiment may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0156] The present application is described with reference to the flowchart and / or block diagram of the method, system, electronic device, computer-readable storage medium and computer program product according to the present embodiment. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0157] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0159] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0160] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for constructing a knowledge graph ontology framework in the auditing field based on a large language model, characterized in that: include: Step 1: Collect relevant data in the audit field and pre-process the collected data; Step 2: Use the audit-related data preprocessed in step 1 to pre-train the large language model; fine-tune the pre-trained large language model based on the existing general entity extraction and relationship extraction datasets; Step 3: Use the large language model to extract knowledge from the audit field text data preprocessed in step 1 to obtain the knowledge extraction results in the audit field; Step 4: Standardize the knowledge extraction results; Based on the knowledge extraction results after standardization, a knowledge graph ontology framework in the audit field is constructed.

2. The method according to claim 1, characterized in that The method further comprises: Step 5: Based on the knowledge graph ontology framework obtained in step 4, score the knowledge extraction results obtained in step 3 to obtain the scoring results; use the scoring results as reward signals to optimize the large language model; Iterate steps three to five to complete the iterative optimization of the large language model and update the knowledge graph ontology framework.

3. The method according to claim 2, characterized in that The step of scoring the knowledge extraction result obtained in step 3 includes: Completeness score, used to evaluate whether all entities and relations in the knowledge graph ontology framework are correctly extracted; Redundancy score: used to evaluate whether there are repeated entities or relationships in the knowledge extraction results; Accuracy score: used to evaluate the accuracy of the extraction results.

4. The method according to claim 2, characterized in that: The method further comprises: Step 6: After completing the iterative optimization of the large language model, apply the optimized large language model to the new audit field text data for knowledge extraction to obtain new extraction results; based on the new extraction results, update and expand the knowledge graph ontology framework.

5. The method according to claim 1, characterized in that The updating and expanding of the knowledge graph ontology framework includes: comparing the newly extracted knowledge with the existing knowledge graph ontology framework, adding new entities and relationships, and removing redundant or erroneous information.

6. A system for constructing a knowledge graph ontology framework in the audit field based on a large language model, characterized in that: include: Data collection and preprocessing module, used to collect audit-related data and preprocess the collected data; Large language model pre-training and fine-tuning module, used to pre-train the large language model using pre-processed audit domain-related data; Fine-tune the pre-trained large language model based on existing general entity extraction and relation extraction datasets; The knowledge extraction module is used to extract knowledge from the pre-processed audit field text data using a large language model to obtain knowledge extraction results in the audit field; The knowledge graph ontology framework building module is used to standardize the knowledge extraction results; Based on the knowledge extraction results after standardization, a knowledge graph ontology framework in the audit field is constructed.

7. The system according to claim 6, characterized in that The system further comprises: The model iteration optimization module is used to score the knowledge extraction results based on the obtained knowledge graph ontology framework to obtain the scoring results; use the scoring results as reward signals to optimize the large language model; and complete the iterative optimization of the large language model and the update of the knowledge graph ontology framework.

8. The system according to claim 7, characterized in that The scoring of the knowledge extraction results includes: Completeness score, used to evaluate whether all entities and relations in the knowledge graph ontology framework are correctly extracted; Redundancy score: used to evaluate whether there are repeated entities or relationships in the knowledge extraction results; Accuracy score: used to evaluate the accuracy of the extraction results.

9. The system according to claim 7, characterized in that The system further comprises: The knowledge graph ontology framework update and expansion module is used to apply the optimized large language model to new audit field text data for knowledge extraction after completing the iterative optimization of the large language model to obtain new extraction results; based on the new extraction results, the knowledge graph ontology framework is updated and expanded.

10. The system according to claim 1, characterized in that The updating and expanding of the knowledge graph ontology framework includes: comparing the newly extracted knowledge with the existing knowledge graph ontology framework, adding new entities and relationships, and removing redundant or erroneous information.

11. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is configured to call the computer program to execute the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on an electronic device, the electronic device implements the method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed on an electronic device, the electronic device implements the method according to any one of claims 1 to 5.

Images