Audit scheme construction method and device based on reinforcement learning and electronic equipment

Abstract

The invention provides an auditing scheme construction method and device based on reinforcement learning and electronic equipment, and relates to the technical field of auditing. The method is applied to a case selection device, a plurality of audit cases and audit influence factors of each audit case are stored in the case selection device, and the method comprises the following steps: obtaining an audit case selection model; obtaining audit influence factors of the to-be-audited object; calculating the similarity between the audit influence factor of each audit case and the audit influence factor of the to-be-audited object; state data including audit influence factors and similarities of a plurality of audit cases are constructed; according to the state data and an audit case selection model, generating a case selection scheme; selecting a target audit case from the plurality of audit cases according to the case selection scheme; and obtaining an audit scheme for the to-be-audited object according to the target audit case. According to the mode, the comprehensiveness of auditing case query is improved, and the auditing case selection efficiency and the auditing scheme generation efficiency are improved.

Description

Background Art

[0002] In the field of audit technology, after clarifying the object to be audited, an audit plan needs to be formulated for the audit object. When specifying the audit plan, in order to ensure the feasibility and rationality of the formulated audit plan, it is usually necessary to refer to excellent audit cases.

[0003] In the related art, when formulating an audit plan, the audit cases are consulted by looking through documents, and after the target audit case is found, an audit plan for the audited object is formulated by referring to the target audit case.

[0004] However, the method of formulating an audit plan by looking through documents and reviewing audit cases has the problems of low efficiency in plan formulation, difficulty in fully querying audit cases, and logical irrationality in the audit plan. Summary of the invention

[0005] The present application provides a method, device and electronic device for constructing an audit plan based on reinforcement learning, which at least to a certain extent overcomes the problems of low efficiency in formulating audit plans, difficulty in comprehensively querying audit cases, and easy existence of logical irrationality in the related technologies.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of the present application, a method for constructing an audit plan based on reinforcement learning is provided, which is applied to a case selection device, wherein the case selection device stores multiple audit cases and audit influencing factors of each audit case, wherein the audit influencing factors include at least one of object type, organizational structure, audit objective, audit scope and audit standard, and comprises: obtaining an audit case selection model; obtaining the audit influencing factors of the object to be audited; calculating the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited; constructing status data including the audit influencing factors of the multiple audit cases and the similarities; generating a case selection plan based on the status data and the audit case selection model; selecting a target audit case from the multiple audit cases based on the case selection plan; and obtaining an audit plan for the object to be audited based on the target audit case.

[0008] In some embodiments, the audit influencing factors include object type, organizational structure, audit objectives, audit scope and audit standards; the calculation of the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited includes: respectively calculating the average value of the similarity between each audit case and the object to be audited in terms of object type, organizational structure, audit objectives, audit scope and audit standards, and obtaining the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited.

[0009] In some embodiments, generating a case selection plan based on the status data and the audit case selection model includes: generating a feature vector based on the status data; inputting the feature vector into the audit case selection model, and allowing the audit case selection model to generate the case selection plan based on the feature vector.

[0010] In some embodiments, the reinforcement learning-based audit scheme generation method also includes: calculating the feedback reward of the target audit case; constructing historical samples including the feedback reward, storing the historical samples in a replay pool, and the historical samples stored in the replay pool are used to train the audit case selection model to adjust the model parameters of the audit case selection model.

[0011] In some embodiments, the calculation of the feedback reward for the target audit case includes: obtaining an audit case scoring model, the audit case scoring model being used to score audit cases based on audit influencing factors; inputting the audit influencing factors of the audit object and the target audit case into the audit case scoring model, and outputting the scoring value of the target audit case through the audit case scoring model; and generating the feedback reward based on the scoring value.

[0012] In some embodiments, the historical sample also includes the audit influencing factors of the object to be audited, the status data and the case selection plan.

[0013] According to another aspect of the present application, there is also provided an audit scheme construction device based on reinforcement learning, which is applied to a case selection device, wherein the case selection device stores multiple audit cases and the audit influencing factors of each audit case, and the audit influencing factors include at least one of the object type, organizational structure, audit objective, audit scope and audit standard, including: a first acquisition module, used to acquire an audit case selection model; a second acquisition module, used to acquire the audit influencing factors of the object to be audited; a first calculation module, used to calculate the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited; a first construction module, used to construct state data including the audit influencing factors of the multiple audit cases and the similarities; a generation module, used to generate a case selection scheme based on the state data and the audit case selection model; a selection module, used to select a target audit case from the multiple audit cases according to the case selection scheme; a third acquisition module, used to acquire an audit scheme for the object to be audited based on the target audit case.

[0014] In some embodiments, the audit influencing factors include object type, organizational structure, audit objectives, audit scope and audit standards; the first calculation module is used to calculate the average value of the similarity between each audit case and the object to be audited in terms of object type, organizational structure, audit objectives, audit scope and audit standards, and obtain the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited.

[0015] In some embodiments, the generation module is used to generate a feature vector based on the status data; input the feature vector into the audit case selection model, and the audit case selection model generates the case selection plan based on the feature vector.

[0016] In some embodiments, the device also includes: a second calculation module, used to calculate the feedback reward of the target audit case; a second construction module, used to construct historical samples including the feedback reward, and store the historical samples in a replay pool, and the historical samples stored in the replay pool are used to train the audit case selection model to adjust the model parameters of the audit case selection model.

[0017] In some embodiments, the second calculation module is used to obtain an audit case scoring model, and the audit case scoring model is used to score audit cases according to audit influencing factors; the audit influencing factors of the audit object and the target audit case are input into the audit case scoring model, and the scoring value of the target audit case is output through the audit case scoring model; and the feedback reward is generated according to the scoring value.

[0018] According to another aspect of the present application, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned reinforcement learning-based audit scheme construction methods by executing the executable instructions.

[0019] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above-mentioned methods for constructing an audit scheme based on reinforcement learning.

[0020] According to another aspect of the present application, a computer program product is also provided, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for constructing an audit scheme based on reinforcement learning.

[0021] The technical solutions provided in the embodiments of the present application include at least the following intended effects:

[0022] The technical solution provided in the embodiments of the present application utilizes the audit case selection model and the audit influencing factors of the audit object to be audited, and can quickly select target audit cases related to the audit object from multiple audit cases, thereby improving the comprehensiveness of audit case queries, and improving the efficiency of selecting audit cases and generating audit plans.

[0023] Furthermore, by calculating the similarity between the audit object and the audit case in terms of audit influencing factors and adding the similarity to the status data, the selected target audit case can be made more similar in terms of the audit object and the audit object, thereby making the target audit case more referenceable and the audit plan more logically reasonable.

[0024] Furthermore, using the audit influencing factors of the audit case to construct the status data can avoid the huge data content of the audit case from drowning the characteristics of the selected audit case, making the selected audit case more referenceable, compared with directly applying the entire audit case to construct the status data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of a system for constructing an audit solution based on reinforcement learning in one embodiment of the present application is shown.

[0027] Figure 2 A flow chart of a method for constructing an audit scheme based on reinforcement learning in one embodiment of the present application is shown.

[0028] Figure 3 A flowchart of constructing a historical sample in one embodiment of the present application is shown.

[0029] Figure 4 A schematic diagram of a device for constructing an audit scheme based on reinforcement learning in one embodiment of the present application is shown.

[0030] Figure 5 A structural block diagram of an electronic device in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] In addition, the accompanying drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0033] It should be noted that the acquisition, storage, use, and processing of audit cases in the technical solution of this application comply with the relevant provisions of national laws and regulations, and the various types of data related to the audited objects, operation data, behavioral data, etc. obtained in the embodiments of this application have all been authorized.

[0034] For ease of understanding, before introducing the embodiments of the present application, several terms involved in the embodiments of the present application are first explained as follows:

[0035] Audit influencing factors: factors that affect the design of audit programs;

[0036] Object type: the type of object to be audited, for example, a permanent department, enterprise, project department, institution, etc. in an enterprise;

[0037] Organizational structure: personnel composition and management structure of the audit object;

[0038] Audit objectives: such as financial statement audit, compliance audit, operational audit, etc. Audit objectives should be specific, measurable, achievable, relevant and time-bound;

[0039] Audit scope: defines the specific content of the audit work, including the audit object, time range, organization scope, business scope, etc. The audit scope should be determined in accordance with the audit objectives and adjusted according to the specific circumstances of the audit object;

[0040] Audit standards: Audits are conducted in accordance with relevant auditing standards and criteria, such as the International Auditing Standards, the Chinese Certified Public Accountant Auditing Standards, etc.

[0041] The specific implementation of the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0042] Figure 1 FIG. 1 is a schematic diagram showing a system for building an audit solution based on reinforcement learning in one embodiment of the present application. Figure 1 As shown, the system may include: a case selection device 11, a model training device 12, and a solution design terminal 13.

[0043] The case selection device 11 may store a case library 111 , which includes a plurality of audit cases and the audit influencing factors of each case.

[0044] The model training device 12 stores an audit case selection model and can train the audit case selection model. The embodiments of the present application do not limit which model the audit case selection model is. For example, the audit case selection model may include an evaluation network, and the evaluation network may include an actor network. For another example, the audit case selection model may include model-based reinforcement learning (Model-Based RL).

[0045] The case selection device 11 can obtain the audit case selection model from the model training device 12. In one embodiment, the case selection device 11 has an input device 112, through which the audit influencing factors of the audit object can be input into the case selection device 11.

[0046] In another embodiment, the case selection device 11 may receive the audit influencing factors of the object to be audited sent by other devices; or obtain the audit influencing factors of the object to be audited from other devices.

[0047] The case selection device 11 can calculate the similarity between any audit case and the audit object in terms of audit influencing factors, and can use the calculated similarity and audit influencing factors corresponding to each audit case as status data.

[0048] The case selection device 11 may utilize the audit case selection model to process the status data to generate a case selection scheme, and the case selection device 11 may select a target audit case from a plurality of audit cases according to the case selection scheme.

[0049] The solution design terminal 13 can obtain and view the target audit case through the case selection device 11, and develop an audit solution for the audit object based on the target audit case.

[0050] In one embodiment, the solution design terminal 13 is an audit solution designer, and the audit solution designer can directly input the audit solution into the case selection device 11 .

[0051] In another embodiment, the solution design terminal 13 is a device on the audit solution designer side. The audit solution designer can input the audit solution into the solution design terminal 13 , and then the solution design terminal 13 sends the audit solution to the case selection device 11 .

[0052] Taking the scheme design end 13 as an audit scheme designer as an example, the network is used to provide a medium for a communication link between the case selection device 11 and the model training device 12, which can be a wired network or a wireless network.

[0053] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0054] The case selection device 11 and the model training device 12 may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, servers, and the like.

[0055] In one embodiment, the case selection device 11 and the model training device 12 are the same device.

[0056] The server can be a server that provides various services. Optionally, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0057] Under the above system architecture, an embodiment of the present application provides a method for constructing an audit scheme based on reinforcement learning, which can be executed by any electronic device with computing and processing capabilities. For example, the electronic device is a case selection device, which stores multiple audit cases and audit influencing factors of each audit case, and the audit influencing factors include: at least one of object type, organizational structure, audit objective, audit scope and audit standard.

[0058] Figure 2 A flow chart of a method for constructing an audit scheme based on reinforcement learning in an embodiment of the present application is shown as follows: Figure 2 As shown, the audit scheme construction method based on reinforcement learning provided in the embodiment of the present application includes the following S201 to S207.

[0059] S201, obtain the audit case selection model.

[0060] The embodiments of the present application do not limit how to obtain the audit case selection model. In one embodiment, an initial audit case selection model can be directly constructed in the case selection model, and the audit initial case selection model can be trained to obtain an audit case selection model that can be used in the embodiments of the present application.

[0061] In another embodiment, the case selection device may obtain the audit case selection model from other devices. For example, if a trained audit case selection model is stored in the model training device, the case selection device may obtain the audit case selection model from the model training device.

[0062] The embodiments of the present application do not limit which model the audit case selection model is. For example, the audit case selection model may include an evaluation network, and the evaluation network may include an actor network. For another example, the audit case selection model may include model-based reinforcement learning (Model-Based RL).

[0063] S202, obtaining the audit influencing factors of the audit object.

[0064] The embodiments of the present application do not limit how to obtain the audit influencing factors of the audit object. For example, the case selection device has an input device, and the audit influencing factors can be input into the case selection device through the input device.

[0065] S203, calculating the similarity between the audit influencing factors of each audit case and the audit influencing factors of the audit object.

[0066] By calculating the similarity between the audit influencing factors of each audit case and the audit influencing factors of the audited object, the similarity corresponding to each audit case can be obtained, and there are multiple similarities in total.

[0067] The embodiments of the present application do not limit how to calculate the similarity.

[0068] In one embodiment, the audit influencing factors include object type, organizational structure, audit objectives, audit scope and audit standards. Calculating the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited may include: calculating the average value of the similarity between each audit case and the object to be audited in terms of object type, organizational structure, audit objectives, audit scope and audit standards, and obtaining the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited.

[0069] Calculate the similarity between each audit case and the audit object in object type to obtain a first similarity; calculate the similarity between each audit case and the audit object in organizational structure to obtain a second similarity; calculate the similarity between each audit case and the audit object in audit objective to obtain a third similarity; calculate the similarity between each audit case and the audit object in audit scope to obtain a fourth similarity; calculate the similarity between each audit case and the audit object in audit standard to obtain a fifth similarity, and then calculate the average of the first similarity, the second similarity, the third similarity, the fourth similarity and the fifth similarity to obtain the similarity between the audit influencing factors of each audit case and the audit influencing factors of the audit object.

[0070] The embodiments of the present application do not limit how to calculate the similarity between each audit case and the audit object in terms of object type, organizational structure, audit objective, audit scope, and audit standard. For example, the similarity can be calculated by cosine distance, Euclidean distance, etc.

[0071] S204, constructing status data including audit influencing factors and similarities of multiple audit cases.

[0072] With state data as S t , the audit influencing factor of the i-th audit case is u i , the similarity corresponding to the i-th audit case is v i For example, S t ={(u i , v i ), i∈N}, where N is the number of audit cases.

[0073] S205, generating a case selection plan based on the status data and the audit case selection model.

[0074] In one embodiment, generating a case selection scheme based on the state data and the audit case selection model may include: generating a feature vector based on the state data; inputting the feature vector into the audit case selection model, and generating a case selection scheme based on the feature vector by the audit case selection model; t ).

[0075] The embodiment of the present application does not limit the specific form of the selection scheme. For example, if the number of audit cases includes 5 audit cases, then scheme a is selected. t It can be (0, 1, 0, 0, 1), where 0 indicates that the corresponding audit case is not selected, and 1 indicates that the corresponding audit case is selected. Then, among the five audit cases, the second and fifth audit cases are selected.

[0076] In one embodiment, the feature vector is input into the audit case selection model, and the audit case selection model generates a case selection plan based on the feature vector, which may include: inputting the feature vector into the audit case selection model, and the audit case selection model generates a decision probability distribution based on the feature vector; randomly determining a target decision from the decision probability distribution, and using the target decision as the case selection plan.

[0077] S206, selecting a target audit case from multiple audit cases according to the case selection plan.

[0078] The number of target audit cases can be determined according to the parameters of the solution selection model. For example, if the case selection model is set to select 3 audit cases from multiple audit cases each time the state data is processed, the number of target audit cases is 3.

[0079] S207, obtaining an audit plan for the audit object based on the target audit case.

[0080] In one embodiment, obtaining an audit plan for the audit object based on a target audit case may include: an audit plan designer refers to the target case, conceives and determines the audit plan, and then uploads the audit plan to a case selection device. Accordingly, the case selection device completes the acquisition of the audit plan.

[0081] In another embodiment, the solution design end can obtain the target audit case from the case selection device, and the audit solution designer can refer to the target audit case through the solution design end and design an audit solution, and then input the audit solution into the solution design end, which then sends the audit solution to the case selection device.

[0082] The technical solution provided in the embodiments of the present application utilizes the audit case selection model and the audit influencing factors of the audit object to be audited, and can quickly select target audit cases related to the audit object from multiple audit cases, thereby improving the comprehensiveness of audit case queries, and improving the efficiency of selecting audit cases and generating audit plans.

[0083] Furthermore, by calculating the similarity between the audit object and the audit case in terms of audit influencing factors and adding the similarity to the status data, the selected target audit case can be made more similar in terms of the audit object and the audit object, thereby making the target audit case more referenceable and the audit plan more logically reasonable.

[0084] Furthermore, using the audit influencing factors of the audit case to construct the status data can avoid the huge data content of the audit case from drowning the characteristics of the selected audit case, making the selected audit case more referenceable, compared with directly applying the entire audit case to construct the status data.

[0085] In one embodiment, the embodiments of the present application do not limit how to train the case selection model. Figure 3 As shown, each time the audit case selection model generates a case selection plan and selects a target audit case according to the case selection plan, the following S301-S302 can be performed to train the audit case selection model.

[0086] S301, calculate the feedback reward of the target audit case.

[0087] The embodiments of the present application do not limit how to calculate the feedback reward of the target audit case. In one embodiment, the feedback reward of the target audit case is calculated, including: obtaining an audit case scoring model, the audit case scoring model is used to score the audit case according to the audit influencing factors; inputting the audit influencing factors of the audit object and the target audit case into the audit case scoring model, and outputting the scoring value of the target audit case through the audit case scoring model; generating feedback rewards according to the scoring value.

[0088] Among them, the audit case scoring model can be any regression model (for example, linear regression, logistic regression, etc.), classification model (for example, decision tree, support vector machine (SVM) and naive Bayes classifier), integration model (for example, random forest, gradient boosting decision tree (GBDT) and XGBoost, etc.), etc.

[0089] The embodiments of the present application do not limit how to train the audit case scoring model. For example, the training samples may be audit plans and corresponding scoring values ​​(values ​​obtained by professional auditors scoring the audit plans).

[0090] The embodiments of the present application do not limit how to generate feedback rewards based on the rating value. For example, the rating value can be directly used as the feedback reward. For another example, the rating value can be subjected to a preset mathematical process to obtain the feedback reward. For example, the rating value is scaled to between 0 and 1.

[0091] S302, construct historical samples including feedback rewards, store the historical samples in a replay pool, and use the historical samples stored in the replay pool to train the audit case selection model to adjust the model parameters of the audit case selection model.

[0092] In one embodiment, the historical sample (H t ) In addition to including the feedback reward R t In addition, it also includes the audit influencing factors u and status data S of the audit object. t and case selection scheme a t , H t =(R t ,u,S t , a t ).

[0093] The embodiments of the present application do not limit how to use samples in the replay pool to train the audit case selection model. For example, historical samples can be selected from the replay pool to train the case selection model by sampling in a priority experience replay manner.

[0094] When using various feedback reward methods to train the model, it is easy for the case selection model to fall into a local optimal solution, making it difficult for the audit case selection model to generate a case selection plan that meets the requirements. As a result, the selected target audit case has a poor match with the audited object and is difficult to use as a case reference.

[0095] When generating case selection plans, do not manually intervene to select the case selection plan with the highest probability (the plan corresponding to the high probability is the case selection plan that best meets the requirements under the current model parameters and input state data). If you forcibly intervene to select the decision with the highest probability (case selection plan) from the decision probability distribution, the state data during the first training will have a greater impact on the adjustment of the model parameters, so that the first case selection plan will be used as the optimal solution in the subsequent training process, and the trained audit case selection model will easily fall into the local optimal solution. Do not manually intervene, and randomly select the target decision from the decision probability distribution. While considering the impact of feedback rewards on case selection plans, it can also avoid the subsequent training of the audit case selection model using the first target decision as the optimal solution for training, thereby avoiding the trained audit case selection model from falling into the local optimal solution, and thus avoiding the audit case selection model from being unable to generate a case selection plan that meets the requirements.

[0096] In order to avoid the slow convergence of the audit case selection model caused by randomly selecting the target decision from the decision probability distribution and to accelerate the convergence of the audit case selection model, the priority experience replay sampling method is used to select historical samples for adjusting the model parameters, thereby accelerating the convergence of the audit case selection model and improving training efficiency.

[0097] In addition, the sampling method of priority experience replay takes into account the impact of the target decision when selecting historical samples. If the probability of the target decision in the corresponding decision probability distribution is low, the probability of the historical sample where the target decision is located is also low under the sampling method of priority experience replay, so that the historical sample whose target decision is more in line with the preset requirements is more likely to be selected, and the target decision in the selected historical sample is not consistent with the target decision in the historical sample during the first round of training. Therefore, by combining the sampling method of priority experience replay with the stochastic policy gradient, the audit case selection model can be further prevented from falling into the local optimal solution and the convergence of the audit case selection model can be accelerated.

[0098] Based on the same inventive concept, the embodiment of the present application also provides a device for constructing an audit scheme based on reinforcement learning, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0099] Figure 4 A schematic diagram of a device for constructing an audit scheme based on reinforcement learning in an embodiment of the present application is shown. Figure 4 As shown, the device is applied to a case selection device, which stores multiple audit cases and audit influencing factors of each audit case, wherein the audit influencing factors include at least one of object type, organizational structure, audit objective, audit scope and audit standard, including: a first acquisition module 41, used to acquire an audit case selection model; a second acquisition module 42, used to acquire the audit influencing factors of the object to be audited; a first calculation module 43, used to calculate the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited; a first construction module 44, used to construct status data including audit influencing factors and similarities of multiple audit cases; a generation module 45, used to generate a case selection plan based on the status data and the audit case selection model; a selection module 46, used to select a target audit case from multiple audit cases according to the case selection plan; a third acquisition module 47, used to acquire an audit plan for the object to be audited according to the target audit case.

[0100] In some embodiments, the audit influencing factors include object type, organizational structure, audit objectives, audit scope and audit standards; the first calculation module 43 is used to calculate the average value of the similarity between each audit case and the object to be audited in terms of object type, organizational structure, audit objectives, audit scope and audit standards, and obtain the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited.

[0101] In some embodiments, the generation module 45 is used to generate a feature vector based on the status data; the feature vector is input into the audit case selection model, and the audit case selection model generates a case selection plan based on the feature vector.

[0102] In some embodiments, the device also includes: a second calculation module, used to calculate the feedback reward of the target audit case; a second construction module, used to construct historical samples including feedback rewards, store the historical samples in a replay pool, and the historical samples stored in the replay pool are used to train the audit case selection model to adjust the model parameters of the audit case selection model.

[0103] In some embodiments, the second computing module is used to obtain an audit case scoring model, which is used to score audit cases based on audit influencing factors; the audit influencing factors of the audit object and the target audit case are input into the audit case scoring model, and the scoring value of the target audit case is output through the audit case scoring model; and feedback rewards are generated based on the scoring value.

[0104] By utilizing the audit case selection model and the audit influencing factors of the audit object, target audit cases related to the audit object can be quickly selected from multiple audit cases, thereby improving the comprehensiveness of audit case queries, the efficiency of audit case selection, and the efficiency of generating audit plans.

[0105] Furthermore, by calculating the similarity between the audit object and the audit case in terms of audit influencing factors and adding the similarity to the status data, the selected target audit case can be made more similar in terms of the audit object and the audit object, thereby making the target audit case more referenceable and the audit plan more logically reasonable.

[0106] Furthermore, using the audit influencing factors of the audit case to construct the status data can avoid the huge data content of the audit case from drowning the characteristics of the selected audit case, making the selected audit case more referenceable, compared with directly applying the entire audit case to construct the status data.

[0107] It should be noted that the first acquisition module 41, the second acquisition module 42, the first calculation module 43, the first construction module 44, the generation module 45, the selection module 46 and the third acquisition module 47 correspond to S201 to S207 in the method embodiment, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above method embodiment. It should be noted that the above modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0108] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0109] Refer to the following Figure 5 The electronic device 500 according to this embodiment of the present application is described. Figure 5 The electronic device 500 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0110] like Figure 5 As shown, the electronic device 500 is in the form of a general computing device. The components of the electronic device 500 may include but are not limited to: at least one processing unit 510, at least one storage unit 520, and a bus 530 connecting different system components (including the storage unit 520 and the processing unit 510).

[0111] The storage unit stores program codes, which can be executed by the processing unit 510, so that the processing unit 510 performs the steps of various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification. For example, the processing unit 510 can perform the following steps of the above method embodiment: S201-S207.

[0112] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 5201 and / or a cache storage unit 5202 , and may further include a read-only storage unit (ROM) 5203 .

[0113] The storage unit 520 may also include a program / utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0114] Bus 530 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0115] The electronic device 500 may also communicate with one or more external devices 540 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 500, and / or communicate with any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 550. Furthermore, the electronic device 500 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 500 via a bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0116] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0117] In particular, according to an embodiment of the present application, the process described in the above reference flowchart can be implemented as a computer program product, which includes: a computer program, which implements the above-mentioned reinforcement learning-based audit scheme construction method when executed by a processor.

[0118] In an exemplary embodiment of the present application, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. The computer-readable storage medium stores a program product capable of implementing the above method of the present application.

[0119] In some possible implementations, various aspects of the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present application described in the above “Exemplary Method” section of this specification.

[0120] More specific examples of computer-readable storage media in the present application may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0121] In the present application, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0122] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0123] In specific implementation, the program code for performing the operation of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0124] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0125] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0126] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the example implementation modes described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation mode of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation mode of the present application.

[0127] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

Claims

1. A method for constructing an audit scheme based on reinforcement learning, characterized in that: Applied to a case selection device, the case selection device stores a plurality of audit cases and audit influencing factors of each audit case, the audit influencing factors including at least one of object type, organizational structure, audit objective, audit scope and audit standard, including: Obtaining audit case selection models; Obtain the audit influencing factors of the audit object; Calculate the similarity between the audit influencing factors of each audit case and the audit influencing factors of the audit object; Constructing status data including audit influencing factors and similarities of the plurality of audit cases; generating a case selection scheme according to the status data and the audit case selection model; selecting a target audit case from the plurality of audit cases according to the case selection scheme; According to the target audit case, an audit plan for the object to be audited is obtained.

2. The audit scheme generation method based on reinforcement learning according to claim 1 is characterized in that: The audit influencing factors include object type, organizational structure, audit objectives, audit scope and audit standards; the calculation of the similarity between the audit influencing factors of each audit case and the audit influencing factors of the object to be audited includes: The average value of the similarity between each audit case and the audit object in terms of object type, organizational structure, audit objective, audit scope and audit standard is calculated respectively to obtain the similarity between the audit influencing factors of each audit case and the audit influencing factors of the audit object.

3. The audit scheme generation method based on reinforcement learning according to claim 1 is characterized in that: Generating a case selection scheme according to the status data and the audit case selection model includes: generating a feature vector according to the state data; The feature vector is input into the audit case selection model, and the audit case selection model generates the case selection plan according to the feature vector.

4. The audit scheme generation method based on reinforcement learning according to claim 1 is characterized in that: Also includes: Calculating feedback rewards for the target audit cases; Constructing historical samples including the feedback reward, storing the historical samples in a replay pool, and using the historical samples stored in the replay pool to train the audit case selection model to adjust the model parameters of the audit case selection model.

5. The audit scheme generation method based on reinforcement learning according to claim 4 is characterized in that: The calculating of the feedback reward of the target audit case includes: Obtaining an audit case scoring model, wherein the audit case scoring model is used to score audit cases according to audit influencing factors; Inputting the audit influencing factors of the audit object and the target audit case into the audit case scoring model, and outputting the scoring value of the target audit case through the audit case scoring model; The feedback reward is generated according to the rating value.

6. The audit scheme generation method based on reinforcement learning according to claim 4 or 5, characterized in that: The historical sample also includes the audit influencing factors of the object to be audited, the status data and the case selection scheme.

7. An audit scheme construction device based on reinforcement learning, characterized in that: Applied to a case selection device, the case selection device stores a plurality of audit cases and audit influencing factors of each audit case, the audit influencing factors including at least one of object type, organizational structure, audit objective, audit scope and audit standard, including: A first acquisition module is used to acquire an audit case selection model; The second acquisition module is used to obtain the audit influencing factors of the audit object; A first calculation module, used to calculate the similarity between the audit influencing factors of each audit case and the audit influencing factors of the audit object; A first construction module is used to construct status data including the audit influencing factors and the similarities of the plurality of audit cases; A generation module, used for generating a case selection scheme according to the status data and the audit case selection model; A selection module, configured to select a target audit case from the plurality of audit cases according to the case selection scheme; The third acquisition module is used to acquire an audit plan for the object to be audited based on the target audit case.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the audit scheme construction method based on reinforcement learning as described in any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the audit scheme construction method based on reinforcement learning as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for constructing an audit scheme based on reinforcement learning as described in any one of claims 1 to 6 is implemented.

Images