Auditing method based on large language model and process automation technology and related equipment

Abstract

The invention provides an auditing method based on a large language model and a process automation technology and related equipment, which are applied to the technical field of auditing, and comprise the following steps: acquiring a project file and an auditing instruction by using an RPA auditing robot, and after determining an auditing type of the auditing instruction, executing an auditing process corresponding to the auditing type by the RPA auditing robot, and generating an audit report suggestion document, acquiring an audit problem list of the audit process by the RPA audit robot, and processing the audit problem list by using an audit problem analysis model so as to obtain a rectification suggestion document. By applying the scheme provided by the invention, the auditing report suggestion document and the rectification suggestion document can be obtained by using the RPA auditing robot and the auditing problem analysis model, business personnel participate in the whole process, the manual participation degree is reduced, the time and energy for obtaining the auditing report suggestion document and the rectification suggestion document are reduced, and the cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of auditing technology, and in particular to an auditing method and related equipment based on large language models and process automation technology. Background Technology

[0002] In various audit application scenarios, business personnel manually identify sampled audit risk points by analyzing business reports and risk warning lists on different themes. They then analyze each audit issue and write a final report. Furthermore, from a post-audit prevention perspective, they generate warnings for the identified risks. This involves a comprehensive assessment of multiple audit risks. For example, in a departure audit project, business personnel need to analyze issues from multiple perspectives, including underwriting, claims, and intermediaries, before forming the final report. The entire process requires manual work from business personnel, consuming significant time, effort, and costs. Summary of the Invention

[0003] In view of this, embodiments of this application provide an auditing method and related equipment based on large language models and process automation technology. The solution provided in this application uses an RPA audit robot to obtain project documents and audit instructions. After determining the audit type of the audit instruction, the RPA audit robot executes the audit process corresponding to the audit type, generating an audit plan, audit working papers, and audit report suggestion documents. Furthermore, the RPA audit robot obtains a list of audit issues from the audit process and processes the list using an audit issue analysis model to obtain rectification opinion documents. By applying the solution provided in this application, audit report suggestion documents and rectification opinion documents can be obtained using an RPA audit robot and an audit issue analysis model. The entire process involves business personnel, reducing the degree of manual intervention, reducing the time and effort spent obtaining audit report suggestion documents and rectification opinion documents, and lowering costs.

[0004] To achieve the above objectives, the embodiments of this application provide the following technical solutions:

[0005] An auditing method based on large language models and process automation technology includes:

[0006] The RPA audit robot is used to obtain the project file of the project to be audited and the audit instructions for obtaining the project file. The project file is a file generated by the auditor after filling in the information in the audit project template corresponding to the project to be audited.

[0007] The audit type of the audit instruction is determined, and the RPA audit robot executes the audit process corresponding to the audit type based on the project file, and obtains the process execution information and output results of the audit process corresponding to the audit type. The output results include the audit plan and audit issue working papers.

[0008] The process execution information is input into a preset application big language model to build an audit report generation model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document;

[0009] Obtain the audit issue list of the audit process executed by the RPA audit robot, input the audit issue list into the audit issue analysis model constructed by applying a large language model, so that the audit issue analysis model outputs the rectification opinion document corresponding to the audit issue list.

[0010] The above methods may also include:

[0011] The RPA audit robot adds the audit report suggestion document and the rectification opinion document to the audit report document of the project to be audited.

[0012] Optionally, in the above method, determining the audit type of the audit instruction includes:

[0013] Obtain the audit type identifier of the audit instruction;

[0014] When the audit type identifier is a project-wide identifier, the audit type of the audit instruction is determined to be the project-wide audit type;

[0015] When the audit type identifier is a routine risk sampling inspection identifier, the audit type of the audit instruction is determined to be a routine risk sampling inspection type.

[0016] When the audit type identifier is a designated review identifier, the audit type of the audit instruction is determined to be the designated audit review type.

[0017] Optionally, in the above method, the RPA audit robot executes an audit process corresponding to the audit type based on the project file, including:

[0018] When the audit type of the audit instruction is the overall audit project type, the RPA audit machine determines each review process corresponding to the overall audit project type, determines the execution order of each review process, and executes each review process sequentially using the project file according to the execution order of each review process.

[0019] When the audit type of the audit instruction is the audit routine risk sampling inspection type, the RPA audit robot determines the routine risk sampling inspection process corresponding to the audit routine risk sampling inspection type, and applies the project file to execute the routine risk sampling inspection process;

[0020] When the audit type of the audit instruction is an audit specified review type, the RPA audit robot determines the specified review process corresponding to the audit specified review type and executes the specified review process using the project file.

[0021] The above methods may also include:

[0022] The risk level is determined based on the aforementioned list of audit issues;

[0023] The model sensitivity of the audit report generation model and the audit issue analysis model is adjusted based on the risk level.

[0024] An auditing device based on large language models and process automation technology includes:

[0025] The acquisition unit is used to apply the RPA audit robot to acquire the project file of the project to be audited and the audit instruction to acquire the project file. The project file is a file generated by the auditor after filling in information in the audit project template corresponding to the project to be audited.

[0026] The determining unit is used to determine the audit type of the audit instruction. The RPA audit robot executes the audit process corresponding to the audit type based on the project file and obtains the process execution information of the audit process corresponding to the audit type.

[0027] The execution unit is used to execute the process execution results output after the review process, including the audit plan and audit working papers;

[0028] The first output unit is used to input the process execution information into a preset audit report generation model constructed by the application big language model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document;

[0029] The second output unit is used to obtain a list of audit issues executed by the RPA audit robot, input the list of audit issues into an audit issue analysis model constructed using a large language model, and enable the audit issue analysis model to output a rectification opinion document corresponding to the list of audit issues.

[0030] The aforementioned apparatus may optionally further include:

[0031] The addition unit is used by the RPA audit robot to add the audit report suggestion document and the rectification opinion document to the audit report document of the project to be audited.

[0032] Optionally, in the aforementioned apparatus, the determining unit performs a process for determining the audit type of the audit instruction, including:

[0033] Obtain the audit type identifier of the audit instruction;

[0034] When the audit type identifier is a project-wide identifier, the audit type of the audit instruction is determined to be the project-wide audit type;

[0035] When the audit type identifier is a routine risk sampling inspection identifier, the audit type of the audit instruction is determined to be a routine risk sampling inspection type.

[0036] When the audit type identifier is a designated review identifier, the audit type of the audit instruction is determined to be the designated audit review type.

[0037] Optionally, in the aforementioned apparatus, the process by which the determining unit executes the audit process corresponding to the audit type based on the project file using the RPA audit robot includes:

[0038] When the audit type of the audit instruction is the overall audit project type, the RPA audit machine determines each review process corresponding to the overall audit project type, determines the execution order of each review process, and executes each review process sequentially using the project file according to the execution order of each review process.

[0039] When the audit type of the audit instruction is the audit routine risk sampling inspection type, the RPA audit robot determines the routine risk sampling inspection process corresponding to the audit routine risk sampling inspection type, and applies the project file to execute the routine risk sampling inspection process;

[0040] When the audit type of the audit instruction is an audit specified review type, the RPA audit robot determines the specified review process corresponding to the audit specified review type and executes the specified review process using the project file.

[0041] The aforementioned apparatus may optionally further include:

[0042] An optimization unit is used to determine the risk level based on the audit issue list; and to adjust the model sensitivity of the audit report generation model and the audit issue analysis model based on the risk level.

[0043] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides executes the auditing method described above, which is based on large language model and process automation technology.

[0044] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors to implement the auditing method based on large language model and process automation technology as described above.

[0045] Compared with the prior art, this application has the following advantages:

[0046] This application provides an auditing method based on large language model and process automation technology, comprising: using an RPA audit robot to obtain project files of the project to be audited and audit instructions for obtaining project files, wherein the project files are generated after the auditor fills in information in the audit project template corresponding to the project to be audited; determining the audit type of the audit instructions, the RPA audit robot executing the audit process corresponding to the audit type based on the project files, and obtaining the process execution information and output results of the audit process corresponding to the audit type, the output results including audit plan and audit issue working papers; inputting the process execution information into a preset audit report generation model constructed using a large language model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document; obtaining a list of audit issues executed by the RPA audit robot in the audit process, inputting the list of audit issues into an audit issue analysis model constructed using a large language model, so that the audit issue analysis model outputs a rectification opinion document corresponding to the list of audit issues. By using RPA audit robots and audit issue analysis models, audit report recommendation documents and rectification opinion documents can be obtained. The entire process does not require the involvement of business personnel, which reduces the workload of business personnel and eliminates the need for a large investment of manpower and time, thereby reducing the cost of obtaining audit report recommendation documents and rectification opinion documents. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 A flowchart illustrating an auditing method based on a large language model and process automation technology, provided for embodiments of this application;

[0049] Figure 2 A flowchart illustrating the audit process executed by the RPA audit robot provided in this application embodiment;

[0050] Figure 3A flowchart of the optimization model provided in the embodiments of this application;

[0051] Figure 4 A schematic diagram of the structure of an auditing device based on large language model and process automation technology provided in an embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] Currently, in various audit application scenarios, business personnel use technical means such as analyzing business reports and risk warning lists on different themes to manually screen sampled audit risk points, form audit problem analysis one by one, and write a final report. They also generate reminders for the verified risks from the perspective of post-event prevention. This involves a comprehensive assessment of multiple audit risks. For example, in the departure audit project, business personnel need to analyze problems from multiple perspectives such as underwriting, claims, and intermediaries before forming a final report.

[0056] The existing technology has several problems. First, compiling and writing audit issue analyses and reports is time-consuming, and the investigation of routine audit verification issues involves a large amount of unstructured data, for which current technology has not yet provided an ideal solution. Second, current technology does not tightly couple audit issue analysis with each stage of audit project implementation. Currently, it still relies on manual collection of large amounts of data or reports from various business systems for investigation and analysis, which is not only time-consuming but also prone to errors and costly. Third, traditional audit sampling methods can only cover some risk verification points, failing to provide a comprehensive verification and understanding of the issues, leading to biased audit conclusions.

[0057] To address the aforementioned issues, this application provides an audit solution based on large language models and process automation technology. Applying this solution eliminates the need for business personnel to manually draft audit report recommendation documents and rectification opinion documents. These documents can be obtained using RPA audit robots and audit problem analysis models, reducing the cost of acquiring them and decreasing the workload for business personnel.

[0058] This application can be used in auditing systems comprised of numerous general-purpose or special-purpose computing device environments or configurations. These environments or configurations include, for example, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above, etc.

[0059] Reference Figure 1 The flowchart of an auditing method based on large language model and process automation technology provided in this application embodiment is described in detail below:

[0060] S101. Use the RPA audit robot to obtain the project file of the project to be audited and the audit instructions for obtaining the project file. The project file is a file generated after the auditor fills in the information in the audit project template corresponding to the project to be audited.

[0061] The RPA audit robot retrieves project files for projects to be audited from the audit business processing system. These project files represent the documents for projects requiring auditing by auditors. Audit projects include, but are not limited to, internal projects, external collaborative projects, contracted projects, and various audit-related business projects. The audit project template contains multiple audit fields and descriptions for each field. These descriptions include the audit type, required data type, audit focus (e.g., invoices, tax payments), and data type (e.g., text, images). Different audit projects require different audit content; therefore, the audit fields in different audit project templates will vary.

[0062] In the method provided in this application embodiment, the template library of the audit system stores multiple audit project templates. The project templates in the template library can be templates pre-developed by developers or templates that auditors can customize and generate according to their own needs.

[0063] Auditors can select audit project templates from the template library based on their audit needs, fill in the information according to the selected template, and then generate a project file. If no audit project template in the template library meets the auditor's needs, the auditor can create their own. They can start from scratch, creating a new audit project template from zero, or they can select an audit project template and modify it to obtain one that meets their audit requirements. This project file contains all the information for the audited project, which can be understood as the various audit information required to audit the project. Furthermore, newly generated audit project templates can be stored in the template library.

[0064] In addition to generating project files based on the audit project template, auditors will also input audit instructions, which will specify the type of audit to be performed on the project files.

[0065] S102. Determine the audit type of the audit instruction. The RPA audit robot executes the audit process corresponding to the audit type based on the project file and obtains the process execution information of the audit process corresponding to the audit type.

[0066] In the method provided in this application embodiment, after obtaining the project file and audit instructions, the RPA audit robot first determines the audit type of the audit instructions, and then the RPA audit robot executes the audit process corresponding to the audit type based on the project file, thereby obtaining the process execution information of the audit process corresponding to the audit type.

[0067] Different types of audits have different audit processes, and the process execution information obtained after the execution of different audit processes will be different.

[0068] Optionally, when the RPA audit robot executes the audit process corresponding to the audit type based on the project file, it can also obtain output results, including audit plan and audit issue working papers.

[0069] In another embodiment of the method provided in this application, after the RPA audit robot executes the audit process and obtains the process execution information, it can provide feedback on the process execution information to the staff, for example, by sending the process execution information to the staff via email.

[0070] Reference Figure 2 The flowchart of the audit process executed by the RPA audit robot provided in this application embodiment is described in detail below:

[0071] S201. Obtain the audit type identifier of the audit instruction.

[0072] The audit type identifier is obtained from the audit instruction. The audit type identifier is used to determine the audit type. Different audit types have different identifiers. The audit type here can be understood as the audit scenario.

[0073] In another embodiment of the method provided in this application, the audit type includes three types: audit project overall type (i.e., audit project overall scenario), audit routine risk sampling inspection type (i.e., routine risk sampling inspection scenario), and audit designated review type (i.e., designated review scenario).

[0074] Furthermore, audit types can be defined by the audit engagement and can be broadly categorized into two types: one is an instruction to complete the entire routine audit process, and the other is an instruction to execute specific audit procedures within the routine audit process. Instructions to complete the entire routine audit process are the audit instructions for the overall audit project type mentioned above, while instructions to execute specific audit procedures within the routine audit process are the instructions for the routine risk sampling inspection type and the instructions for the audit-designated review type mentioned above.

[0075] In another embodiment of the method provided in this application, the audit instruction further includes basic data, which includes, but is not limited to, information such as the audited entity and the audit period. The audit type of the audit instruction can be determined through this basic data.

[0076] After determining the audit type of the audit instruction using basic data, this audit type can be compared with the audit type identified by the audit type identifier. If they match, it indicates that the audit type determined for the audit instruction is correct. This verification further improves the reliability and credibility of the audit type determined for the audit instruction. If they do not match, it indicates that there is an anomaly in the audit type determined for the audit instruction. In this case, the audit type determined by the basic data or by the audit type identifier can be provided to the auditors so that they can further determine the audit type of the audit instruction, thereby ensuring the smooth execution of the subsequent audit process for the audited project.

[0077] In another embodiment of the method provided in this application, the audit type of an audit instruction can be determined using either an audit type identifier or the basic data in the audit instruction. Alternatively, using the basic data in the audit instruction to determine the audit type can be used as an alternative method. When the audit instruction does not contain an audit type identifier, or when the audit type identifier in the audit instruction is abnormal, the audit type of the audit instruction is determined by using the basic data of the audit instruction. This provides an alternative solution for determining the audit type of an audit instruction, provides assurance for the process of determining the audit type of an audit instruction, and improves the success rate of auditing the project to be audited.

[0078] S202. When the audit type identifier is the overall project identifier, the audit type of the audit instruction is determined to be the overall audit project type.

[0079] S203. The RPA audit machine identifies the review processes corresponding to the overall type of the audit project and determines the execution order of each review process. According to the execution order of each review process, the application files are used to execute each review process sequentially.

[0080] The RPA audit robot identifies the various review processes corresponding to the overall type of audit project. These review processes are the audit processes described above. Each review process includes, but is not limited to, drafting a plan according to the implementation stage of the audit project, routine risk sampling and inspection, audit issue analysis, and drafting for comments. It should be noted that the routine risk sampling and inspection process includes a variety of routine risks, including but not limited to deleted claims documents, financial deposits, and verification of the number of insured seats.

[0081] The overall type of audit project involves multiple review processes. Each review process is one of the audit processes involved in the audit. Different review processes have different process execution information after execution.

[0082] Furthermore, when executing each review process, the project documents are used to execute the review process. For example, when the review process involves writing a plan according to the implementation steps of the audit project, the process execution information obtained after executing the review process is the audit project implementation plan. When executing the review process, the information required by the review process is extracted from the project documents. For example, it is necessary to extract information on each implementation step of the project from the project documents, and then write relevant plans based on the extracted information, thereby obtaining the audit project implementation plan that includes all implementation steps. This audit project implementation plan is the process execution information of the review process.

[0083] When the review process is a routine risk sampling inspection, the various routine risks included in the routine risk sampling inspection are first identified. Based on pre-set sampling inspection rules, at least one target routine risk is determined from among these routine risks. Then, relevant information for each target routine risk is extracted from the project documents. Based on this information, the risk inspection result for each target routine risk is determined. The risk inspection results for each target routine risk constitute the process execution information for this review process. The sampling inspection rules of this application may include rules for randomly selecting routine risks for inspection using a random selection algorithm, and the number of selected risks can be set according to requirements; the sampling inspection rules may also be rules that determine all routine risks as target routine risks.

[0084] When the review process involves audit issue analysis, an audit issue list is generated based on the project documents during the execution of this process. This audit issue list contains various audit issues that may arise when auditing the project to be audited. The audit issue list is the process execution information obtained after executing this review process.

[0085] Other examination procedures will not be described in this application, but the information obtained after the execution of other examination procedures is still within the scope of protection of this application.

[0086] S204. When the audit type identifier is the routine risk sampling inspection identifier, the audit type of the audit instruction shall be determined to be the routine risk sampling inspection type.

[0087] S205. The RPA audit robot determines the routine risk sampling process corresponding to the type of routine risk sampling inspection in the audit, and executes the routine risk sampling inspection process using project documents.

[0088] When the audit instruction's audit type is the routine risk sampling inspection type, it means that only the routine risk sampling inspection audit process needs to be executed. The process of executing this audit process can be referred to the above explanation, and will not be repeated here.

[0089] S206. When the audit type identifier is a specified review identifier, the audit type of the audit instruction is determined to be the specified audit review type.

[0090] S207. The RPA audit robot determines the designated review process corresponding to the specified audit review type and executes the designated review process using the project file.

[0091] When the audit instruction's audit type is the audit-specified review type, a specified review process needs to be executed. During the execution of the specified review process, a specified risk will be checked to obtain the corresponding check results. The specified risk is at least one of the above-mentioned general risks.

[0092] The designated review process includes designated tasks, which allow staff to assign tasks to staff in an audit project. At least one task must be designated.

[0093] The process of executing the designated review procedure can be referenced from the routine risk sampling inspection procedure described above. The difference between the two is that the risks to be checked when executing the designated review procedure are the risks specified by the staff among the various routine risks in the routine risk sampling inspection procedure. The risks to be checked when executing the routine risk sampling inspection procedure are the risks determined among the various routine risks according to the sampling inspection rules.

[0094] By defining the review type of the review instructions, a variety of audit processing methods are provided for staff, thereby meeting their diverse audit needs.

[0095] In another embodiment of the method provided in this application, S103 can be executed first and then S104, or S104 can be executed first and then S103; that is, the execution order of S103 and S104 is not limited. S103: Input the process execution information into a preset audit report generation model constructed from a large application language model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document.

[0096] This application uses the example of executing S103 followed by S104 for illustration.

[0097] S103. Input the process execution information into the audit report generation model constructed by the preset application big language model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document.

[0098] In another embodiment of this application, the RPA audit robot can send process execution information to the audit report generation model through an interface. The audit report generation model processes the process execution information, including but not limited to: converting unstructured process execution information into structured information; and generating an agent based on the structured information reasoning report framework, thereby generating an audit report suggestion document; furthermore, after generating the audit report suggestion document, the audit report suggestion document is returned to the RPA audit robot.

[0099] S104. Obtain the list of audit issues executed by the RPA audit robot in the audit process, input the list of audit issues into the audit issue analysis model built using the big language model, so that the audit issue analysis model outputs the rectification opinion document corresponding to the list of audit issues.

[0100] After the RPA audit robot obtains the process execution information during the audit process, if the process execution information includes a list of audit issues, the RPA audit robot determines the list of audit issues from the process execution information. Then, the RPA audit robot sends the list of audit issues to the audit issue analysis model via an HTTP interface. The audit issue analysis model can convert the list of audit issues into structured data, and then analyze the structured data to obtain a rectification opinion document. Subsequently, the rectification opinion document can be fed back to the RPA audit robot.

[0101] In another embodiment of the method provided in this application, the process execution information obtained by the RPA audit robot after executing the audit process can be fed back to the staff via email. For example, the running results and statistical data can be fed back to the staff via email. The running results and statistical data include, but are not limited to, project data and working hours, etc., which can be summarized into charts and emailed to system management and business personnel. This helps insurance companies track the use and effectiveness of digital audit methods and track and review them in a timely manner.

[0102] In another embodiment of the method provided in this application, both the audit report generation model and the audit issue analysis model are models built using large language models, which can enhance routine audit processes. Furthermore, the audit issue analysis model can be a model based on 1.5 billion parameters (1.5B base), and the audit report generation model can be a model based on 14 billion parameters (14B base). Both models have undergone fine-tuning with audit-related training materials to improve model performance and reduce computational resource consumption. The audit issue analysis model is used to analyze routine audit issues and propose rectification suggestions, while the audit report generation model can serve as an assistant for writing the final audit report. The application of the above models is integrated into the automated process through Robotic Process Automation (RPA) technology. This application significantly improves the comprehensiveness and accuracy of routine audit issue analysis and the final report by utilizing these two models.

[0103] The method provided in this application involves using an RPA audit robot to obtain project documents and audit instructions. After determining the audit type of the audit instructions, the RPA audit robot executes the audit process corresponding to the audit type, generates an audit report suggestion document using an audit report generation model, and obtains a list of audit issues for the audit process. It then processes this list using an audit issue analysis model to obtain a rectification opinion document. By applying the solution provided in this application, audit report suggestion documents and rectification opinion documents can be obtained using an RPA audit robot and an audit issue analysis model. The entire process involves business personnel, reducing manual intervention and the time and effort spent obtaining audit report suggestion documents and rectification opinion documents, thus lowering costs.

[0104] In another embodiment of the method provided in this application, the RPA audit robot adds audit report suggestion documents and rectification opinion documents to the audit report document of the project to be audited. Subsequently, staff can adjust the audit report document according to the actual audit situation, thereby obtaining a comprehensive and reliable audit report document. This effectively reduces the workload of staff during the audit process, improves staff efficiency, and reduces the time cost of the audit process.

[0105] Reference Figure 3 The flowchart of the optimization model provided in the embodiments of this application is described in detail below:

[0106] S301. Determine the risk level based on the list of audit issues.

[0107] S302. Adjust the model sensitivity of the audit report generation model and the audit issue analysis model based on the risk level.

[0108] In the method provided in this application embodiment, in order to improve the accuracy of the data output by the audit report generation model and the audit issue analysis model, the risk level is determined by the audit issue list, and then the model sensitivity of the audit report generation model and the audit issue analysis model is adjusted based on the risk level, thereby optimizing the two models and balancing the false positive rate and recall rate of the two models.

[0109] The solution provided in this application addresses problems encountered in routine on-site and off-site auditing applications within companies that cannot be solved using existing technologies. The solution utilizes an audit report generation model and an audit issue analysis model to solve two main problems: firstly, it enables the processing of unstructured data for use in audit risk verification scenarios and assists manual investigation; secondly, it facilitates the processing of audit reports, saving manual processing time. The entire solution employs robotic process automation (RPA) technology to optimize routine auditing processes, transforming repetitive and rule-based audit tasks into automated processes, thereby improving efficiency and reducing the risk of human error.

[0110] This application also provides an application scenario for illustration, as detailed below:

[0111] Before this application can be put into use, it is necessary to build an audit report generation model and an audit problem analysis model. The audit report generation model and the audit problem analysis model are built using a large language model. After the model is built, it needs to be trained and optimized using data. The data used needs to be processed, and the processing process is as described in the process of model fine-tuning data and preprocessing.

[0112] Model fine-tuning data and preprocessing: Extract and process the existing structured data in the audit system to form a high-quality corpus; the specific process includes, but is not limited to, cleaning multi-source data and then filtering noise to obtain a high-quality corpus.

[0113] The process of cleaning multi-source data includes: extracting audit issue summaries, issue cause analysis, issue rectification opinions, and issue analysis conclusions from audit projects over the years; and constructing a large model and fine-tuning the corpus through methods such as regular expression matching, anomaly removal, and entity recognition (hereinafter referred to as NER).

[0114] The noise filtering process involves using confidence thresholds (such as Top-k sampling) to select high-confidence corpora from the cleaned data, removing low-quality or conflicting data, and leaving high-quality data.

[0115] After obtaining high-quality data, the two models can be fine-tuned and optimized. The process of fine-tuning and optimizing the models includes domain-adaptive training and dynamic threshold adjustment strategies. Specifically, the domain-adaptive training process is as follows: based on the audit-specific dictionary and rule set, the basic model is fine-tuned using low-rank adaptation technology (hereinafter referred to as LoRA) to retain general semantic understanding capabilities while strengthening audit logical reasoning. The dynamic threshold adjustment strategy process is as follows: the model sensitivity is automatically adjusted according to the risk level of the problems found in the audit to balance the false positive rate and the recall rate.

[0116] After fine-tuning and optimizing the two models, they can be integrated with the already developed RPA audit robot to audit projects. The RPA audit robot obtains audit instructions for the projects to be audited, determines the audit type of the instructions, and then executes the audit process related to the audit type using the project files of the projects to be audited. This yields process execution information. The fine-tuned and optimized audit report generation model is then used to process this information, generating an audit report suggestion document. Finally, the fine-tuned and optimized problem analysis model is used to process the list of audit issues in the process execution information, generating a rectification suggestion document. By combining the audit report generation model, the problem analysis model, and the RPA audit robot into the audit process, staff no longer need to manually collect and analyze issues that arise during the audit. It reduces the workload of staff and allows for targeted verification of risk points in the audit based on needs, making the audit process more flexible and comprehensively covering all risk points in the audit process. This comprehensive audit can effectively improve the accuracy of the final audit conclusions. Using RPA audit robots can improve audit efficiency, avoid omissions that may occur during manual review, improve the accuracy of audit results, and generate reports using models, eliminating the need for staff to spend a lot of time and energy writing audit process reports, thus reducing audit costs.

[0117] Although the present invention describes the operations in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing may be advantageous.

[0118] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0119] and Figure 1 Corresponding to the method shown, this application provides an auditing device based on large language models and process automation technology, which is used to support... Figure 1 For the implementation of the method shown, refer to... Figure 4 The diagram below shows the structure of the auditing device based on large language model and process automation technology provided in this application embodiment. The details are as follows:

[0120] The acquisition unit 401 is used to apply the RPA audit robot to acquire the project file of the project to be audited and the audit instruction to acquire the project file. The project file is a file generated by the auditor after filling in information in the audit project template corresponding to the project to be audited.

[0121] The determining unit 402 is used to determine the audit type of the audit instruction. The RPA audit robot executes the audit process corresponding to the audit type based on the project file and obtains the process execution information of the audit process corresponding to the audit type.

[0122] The first output unit 403 is used to input the process execution information into a preset audit report generation model constructed by an application big language model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document;

[0123] The second output unit 404 is used to obtain the audit issue list of the audit process executed by the RPA audit robot, input the audit issue list into the audit issue analysis model constructed by applying the large language model, so that the audit issue analysis model outputs the rectification opinion document corresponding to the audit issue list.

[0124] The auditing device based on large language model and process automation technology provided in this application embodiment obtains project documents and audit instructions using an RPA audit robot. After determining the audit type of the audit instructions, the RPA audit robot executes the audit process corresponding to the audit type, generates an audit report suggestion document using an audit report generation model, and obtains a list of audit issues for the audit process. It then processes this list using an audit issue analysis model to obtain a rectification opinion document. By applying the solution provided in this application, audit report suggestion documents and rectification opinion documents can be obtained using an RPA audit robot and an audit issue analysis model. The entire process involves business personnel, reducing the degree of manual intervention, the time and effort spent obtaining audit report suggestion documents and rectification opinion documents, and lowering costs.

[0125] In another embodiment of the apparatus provided in this application, the apparatus further includes:

[0126] The addition unit is used by the RPA audit robot to add the audit report suggestion document and the rectification opinion document to the audit report document of the project to be audited.

[0127] In another embodiment of the apparatus provided in this application, the determining unit 402 of the apparatus performs a process of determining the audit type of the audit instruction, including:

[0128] Obtain the audit type identifier of the audit instruction;

[0129] When the audit type identifier is a project-wide identifier, the audit type of the audit instruction is determined to be the project-wide audit type;

[0130] When the audit type identifier is a routine risk sampling inspection identifier, the audit type of the audit instruction is determined to be a routine risk sampling inspection type.

[0131] When the audit type identifier is a designated review identifier, the audit type of the audit instruction is determined to be the designated audit review type.

[0132] In another embodiment of the apparatus provided in this application, the apparatus determining unit 402 executes the process of the RPA audit robot performing an audit process corresponding to the audit type based on the project file, including:

[0133] When the audit type of the audit instruction is the overall audit project type, the RPA audit machine determines each review process corresponding to the overall audit project type, determines the execution order of each review process, and executes each review process sequentially using the project file according to the execution order of each review process.

[0134] When the audit type of the audit instruction is the audit routine risk sampling inspection type, the RPA audit robot determines the routine risk sampling inspection process corresponding to the audit routine risk sampling inspection type, and applies the project file to execute the routine risk sampling inspection process;

[0135] When the audit type of the audit instruction is an audit specified review type, the RPA audit robot determines the specified review process corresponding to the audit specified review type and executes the specified review process using the project file.

[0136] In another embodiment of the apparatus provided in this application, the apparatus further includes:

[0137] An optimization unit is used to determine the risk level based on the audit issue list; and to adjust the model sensitivity of the audit report generation model and the audit issue analysis model based on the risk level.

[0138] This invention also provides a storage medium that includes stored instructions, wherein the execution of the instructions controls the device where the storage medium is located to execute the aforementioned auditing method based on large language model and process automation technology.

[0139] This invention also provides an electronic device, the structural schematic of which is shown below. Figure 5 As shown, it specifically includes a memory 601 and one or more instructions 602, wherein one or more instructions 602 are stored in the memory 601 and configured to be executed by one or more processors 603 to perform the following operations:

[0140] The RPA audit robot is used to obtain the project file of the project to be audited and the audit instructions for obtaining the project file. The project file is a file generated by the auditor after filling in the information in the audit project template corresponding to the project to be audited.

[0141] The audit type of the audit instruction is determined, and the RPA audit robot executes the audit process corresponding to the audit type based on the project file, and obtains the process execution information of the audit process corresponding to the audit type.

[0142] The process execution information is input into a preset application big language model to build an audit report generation model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document;

[0143] Obtain the audit issue list of the audit process executed by the RPA audit robot, input the audit issue list into the audit issue analysis model constructed by applying a large language model, so that the audit issue analysis model outputs the rectification opinion document corresponding to the audit issue list.

[0144] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant regions.

[0145] The specific implementation processes and derivative methods of the above embodiments are all within the protection scope of this invention.

[0146] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0147] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0148] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An auditing method based on large language models and process automation technology, characterized in that, include: The RPA audit robot is used to obtain the project file of the project to be audited and the audit instructions for obtaining the project file. The project file is a file generated by the auditor after filling in the information in the audit project template corresponding to the project to be audited. The audit type of the audit instruction is determined, and the RPA audit robot executes the audit process corresponding to the audit type based on the project file, and obtains the process execution information of the audit process corresponding to the audit type. The process execution information is input into a preset application big language model to build an audit report generation model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document; Obtain the audit issue list of the audit process executed by the RPA audit robot, input the audit issue list into the audit issue analysis model constructed by applying a large language model, so that the audit issue analysis model outputs the rectification opinion document corresponding to the audit issue list.

2. The method according to claim 1, characterized in that, Also includes: The RPA audit robot adds the audit report suggestion document and the rectification opinion document to the audit report document of the project to be audited.

3. The method according to claim 1, characterized in that, Determining the audit type of the audit instruction includes: Obtain the audit type identifier of the audit instruction; When the audit type identifier is a project-wide identifier, the audit type of the audit instruction is determined to be the project-wide audit type; When the audit type identifier is a routine risk sampling inspection identifier, the audit type of the audit instruction is determined to be a routine risk sampling inspection type. When the audit type identifier is a designated review identifier, the audit type of the audit instruction is determined to be the designated audit review type.

4. The method according to claim 3, characterized in that, The RPA audit robot executes the audit process corresponding to the audit type based on the project file, including: When the audit type of the audit instruction is the overall audit project type, the RPA audit machine determines each review process corresponding to the overall audit project type, determines the execution order of each review process, and executes each review process sequentially using the project file according to the execution order of each review process. When the audit type of the audit instruction is the audit routine risk sampling inspection type, the RPA audit robot determines the routine risk sampling inspection process corresponding to the audit routine risk sampling inspection type, and applies the project file to execute the routine risk sampling inspection process; When the audit type of the audit instruction is an audit specified review type, the RPA audit robot determines the specified review process corresponding to the audit specified review type and executes the specified review process using the project file.

5. The method according to claim 1, characterized in that, Also includes: The risk level is determined based on the aforementioned list of audit issues; The model sensitivity of the audit report generation model and the audit issue analysis model is adjusted based on the risk level.

6. An auditing device based on large language models and process automation technology, characterized in that, include: The acquisition unit is used to apply the RPA audit robot to acquire the project file of the project to be audited and the audit instruction to acquire the project file. The project file is a file generated by the auditor after filling in information in the audit project template corresponding to the project to be audited. The determining unit is used to determine the audit type of the audit instruction. The RPA audit robot executes the audit process corresponding to the audit type based on the project file and obtains the process execution information of the audit process corresponding to the audit type. The first output unit is used to input the process execution information into a preset audit report generation model constructed by the application big language model, so that the audit report generation model processes the process execution information and outputs an audit report suggestion document; The second output unit is used to obtain a list of audit issues executed by the RPA audit robot, input the list of audit issues into an audit issue analysis model constructed using a large language model, and enable the audit issue analysis model to output a rectification opinion document corresponding to the list of audit issues.

7. The apparatus according to claim 6, characterized in that, Also includes: The addition unit is used by the RPA audit robot to add the audit report suggestion document and the rectification opinion document to the audit report document of the project to be audited.

8. The apparatus according to claim 6, characterized in that, The process by which the determining unit determines the audit type of the audit instruction includes: Obtain the audit type identifier of the audit instruction; When the audit type identifier is a project-wide identifier, the audit type of the audit instruction is determined to be the project-wide audit type; When the audit type identifier is a routine risk sampling inspection identifier, the audit type of the audit instruction is determined to be a routine risk sampling inspection type. When the audit type identifier is a designated review identifier, the audit type of the audit instruction is determined to be the designated audit review type.

9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform the auditing method based on large language model and process automation technology as described in any one of claims 1-5.

10. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors to implement the auditing method based on large language model and process automation technology as described in any one of claims 1-5.

Images