Auditing assistance apparatus and auditing assistance method

Abstract

To provide an auditing assistance apparatus which facilitates specifying a past case example similar to a target to be audited.SOLUTION: An auditing assistance apparatus 1 for evaluating a risk of accounting fraud using a machine learning model includes: a learning model generation unit 32 which generates a machine learning model using a plurality of training records available for machine learning; a learning model storage unit 41 which stores a basic model generated using a first dataset comprising the plurality of training records and a plurality of verification models generated using a second dataset obtained by excluding at least one record from the first dataset; and an influence calculation unit 33 which calculates an influence degree obtained by comparing an output from the basic model and an output from the plurality of verification models obtained by excluding different records, on a new record to be audited; and a similar data specifying unit 34 which specifies a training record similar to the new record based on the influence degree.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an accounting audit support device and an accounting audit support method for evaluating the risk of accounting fraud using a machine learning model. [Background technology]

[0002] In accounting audit work such as internal audits and external audits, the contents of accounting documents such as financial statements are analyzed and checked to see if there are any fraudulent transactions, such as fictitious transactions (see Patent Documents 1 and 2, etc.).

[0003] Also, as disclosed in Patent Literature 3, there are attempts to use the results of machine learning to evaluate the risk of accounting fraud through computer processing. Here, the calculations performed in the learning process of machine learning are complex, and while there is an advantage in that highly accurate evaluation can be realized, it is often difficult for humans to easily understand the basis for the evaluation.

[0004] Therefore, in Patent Document 3, it is possible to determine which account items, which are explanatory variables used in machine learning, are abnormal for the evaluation output by the machine learning model by calculating the SHAP value. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7052135 [Patent Document 2] JP 2019-179531 A [Patent Document 3] Patent No. 7216854 Summary of the Invention [Problem to be solved by the invention]

[0006] However, simply identifying the account items that have a large impact on the evaluation results makes it difficult to grasp the specific circumstances of accounting fraud such as fictitious transactions, and ultimately requires an accountant with extensive experience in accounting audit work to clarify the details of the accounting fraud based on the numerical values ​​of the identified account items. On the other hand, if past cases similar to the audit target can be identified, it will be easier to refer to published past fraud methods, leading to deeper audit procedures.

[0007] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide an accounting audit support device and an accounting audit support method that make it possible to easily identify past cases similar to the audit target. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the accounting audit support device of the present invention is an accounting audit support device for evaluating the risk of accounting fraud using a machine learning model, and is characterized in that it includes a learning model generation unit that generates a machine learning model using a plurality of training records that can be used for machine learning, a learning model memory unit that stores a basic model generated by the learning model generation unit using a first dataset consisting of a plurality of training records and a plurality of verification models generated by the learning model generation unit using a second dataset in which at least one record is excluded from the first dataset, an impact calculation unit that calculates an impact degree for a new record to be audited by comparing the output of the new record based on the basic model with the output of the new record based on a plurality of the verification models in which the excluded records are different, and a similar data identification unit that identifies a training record similar to the new record based on the impact degree.

[0009] Here, it is preferable that the influence degree is the difference between the output from the basic model and the output from the verification model, and the training record excluded from the verification model in which the difference is the largest is identified as being most similar to the new record.

[0010] Moreover, the invention of an accounting audit support method is an accounting audit support method for evaluating the risk of accounting fraud using a machine learning model, comprising the steps of: generating a basic model by machine learning using a first dataset consisting of a plurality of training records; generating a verification model by machine learning using a second dataset in which at least one record is excluded from the first dataset; and generating another verification model by machine learning using a dataset in which training records different from the second dataset are excluded, a desired number of times; calculating the difference between the output by the basic model and the outputs by the plurality of verification models for a new record to be audited; and identifying the training record excluded from the verification model in which the difference is largest as the one most similar to the new record. Effect of the Invention

[0011] The accounting audit support device and accounting audit support method of the present invention configured in this manner can identify training records similar to a new record to be audited from among multiple training records used in machine learning.

[0012] This makes it easy to identify past cases similar to the audit target. In short, if it is possible to identify training records that have a large impact on the assessment results of the risk of accounting fraud from among the individual training records, it becomes possible to refer to past fraud cases. Past cases identified as similar have had their fraudulent methods made public, so referring to them can lead to the deepening of audit procedures. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram illustrating a configuration of an accounting audit support device according to an embodiment of the present invention. [Diagram 2] 1 is a flowchart illustrating a process flow of an accounting audit support method according to an embodiment of the present invention. [Diagram 3]FIG. 1 is an explanatory diagram showing a schematic overview of an accounting audit support method according to an embodiment of the present invention. [Figure 4] FIG. 1 is an explanatory diagram illustrating explanatory variables of training records used in machine learning. [Diagram 5] FIG. 2 is an illustrative diagram illustrating an example data set of training records. [Figure 6] FIG. 13 is an explanatory diagram illustrating an example of an output from a basic model of a new record to be audited. [Figure 7] FIG. 13 is an explanatory diagram illustrating an example of a score difference between an output based on a basic model and an output based on a verification model for a new record. [Figure 8] FIG. 11 is an explanatory diagram illustrating attributes of a training record. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram illustrating the configuration of an accounting audit support device 1 according to the present embodiment. Fig. 2 is a flowchart illustrating the process flow of an accounting audit support method according to the present embodiment.

[0015] The accounting audit support device 1 of this embodiment is a device for evaluating the risk of accounting fraud using a machine learning model for new records (for example, by fiscal year and month) of a company that is the subject of an accounting audit such as an internal audit or an external audit. Here, the evaluation of the risk of accounting fraud includes both the detection of fraud at the present time and the prediction of fraud in the future.

[0016] In the accounting audit support device 1 of this embodiment, as shown in FIG. 1, processing is performed in a calculation processing unit 3 based on data input from an input device 2, a memory unit 4, etc., and evaluation results, etc. are displayed on a display device 5, etc.

[0017] The calculation processing unit 3 of the accounting audit support device 1 is mainly composed of a training data setting unit 31 that sets training data to be used for machine learning, a learning model generation unit 32 that generates a machine learning model using training records included in the training data, an influence calculation unit 33 that calculates the influence of the training records on a new record, a similar data identification unit 34 that identifies training records similar to the new record, and a display control unit 35 that controls the display content on the display device 5, etc.

[0018] The input device 2 for inputting various data is a data input means connected to or equipped on a personal computer (PC), a notebook computer, a tablet terminal, a wearable terminal, a smartphone, etc. Examples of the input device 2 include a keyboard, a mouse, a touch panel, a touch pad, a scanner, a microphone for voice input, a camera, etc.

[0019] On the other hand, the display device 5 may be a liquid crystal display, an organic EL (Electro-Luminescence) display, a printer, or the like.

[0020] Furthermore, the storage unit 4 is a storage medium for recording training data used in processing in the arithmetic processing unit 3, data generated by arithmetic processing, etc., and corresponds to a hard disk, a solid state drive (SSD), a flash memory (such as an SD memory card), a magnetic disk, an optical disk, etc. Also, an external online storage (cloud storage) such as a server connected via a network can be used as the storage unit 4. Here, the area of ​​the storage medium for storing the machine learning model generated by the learning model generation unit 32 will be described by distinguishing it by assigning it the reference symbol "learning model storage unit 41."

[0021] The arithmetic processing unit 3 is configured, as hardware, with a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc., and is equipped with memories such as a random access memory (RAM) and a read only memory (ROM).

[0022] The arithmetic processing unit 3 can execute various functions by applications installed in a computer. Also, a server connected via a network such as the Internet can execute part or all of the arithmetic processing unit 3. The network is composed of part or all of the Internet, a Wide Area Network (WAN), a wired Local Area Network (LAN), a wireless LAN (Wi-Fi), a provider device, a wireless base station, a dedicated line, etc.

[0023] The training data setting unit 31 of the calculation processing unit 3 can arbitrarily set a training record to be used to generate a machine learning model from among multiple training records stored in the memory unit 4 that can be used for machine learning.

[0024] Then, the learning model generation unit 32 generates a basic model by performing machine learning using a first data set consisting of a plurality of training records set by the training data setting unit 31. In this embodiment, the first data set will be described as a set of all training records that can be used for machine learning.

[0025] Here, Fig. 3 is an explanatory diagram showing a schematic overview of the accounting audit support method of the present embodiment. For example, as shown on the left side of Fig. 3, it is assumed that N training records with IDs (identifiers) from 1 to N are stored in the storage unit 4. Note that the storage unit 4 may store other training records.

[0026] Then, the training data setting unit 31 sets the N training records (N records) as all training records (all training records) to be used for machine learning, and causes the learning model generation unit 32 to generate a machine learning model. This machine learning model will be referred to as a "basic model."

[0027] In addition, the learning model generation unit 32 generates a verification model by performing machine learning using a second data set in which at least one training record (one record) is excluded from the first data set (all training records).

[0028] The term "second dataset" is a general term for a dataset that has training records excluded from the first dataset. If the excluded training records are different, a corresponding second dataset is set.

[0029] For example, the right side of Figure 3 shows an example of generating a validation model, which is a machine learning model, using a second dataset (N-1 records) obtained by excluding the training record (record (1)) with an ID (identifier) ​​of 1 from all training records (N records).

[0030] Next, a machine learning model can be generated using a different dataset in which the training record with ID (identifier) ​​2 (record (2)) is excluded from all training records (N records). In this case, a validation model different from the one generated when the training record with ID 1 is excluded will be generated.

[0031] In this way, by determining at least one training record ID to be excluded from all training records (N records) and performing machine learning using the excluded training data (dataset), a validation model for each machine learning corresponding to that training data will be generated. Therefore, the name "second dataset" is used as a general term for datasets that contain these excluded training records.

[0032] Therefore, validation models are generated for each second dataset that is set. In other words, if training records are excluded one by one, a validation model is generated for each excluded training record. Also, if a dataset is set that excludes multiple training records, the validation model will be different if the combination of excluded training records changes.

[0033] The trained machine learning models (basic model, multiple verification models) generated by the learning model generation unit 32 in the above manner are stored in the learning model storage unit 41. Then, the models are read and used during calculations by the influence degree calculation unit 33, which will be described later, and the like.

[0034] The impact calculation unit 33 uses a basic model generated using all training records to output a risk score (hereinafter sometimes simply referred to as "score") that is the evaluation result of the risk of accounting fraud for each new record to be audited.

[0035] The score output using the basic model is a value that indicates the risk of accounting fraud for new records, but just looking at this value does not make it clear how the accounting fraud is being committed, making it difficult to link to specific audit actions.

[0036] Therefore, in the accounting audit support device 1 of this embodiment, the influence calculation unit 33 calculates the influence of a specified record among the training records included in all the training records (first data set).

[0037] Specifically, the impact calculation unit 33 outputs risk scores that are the evaluation results of the risk of accounting fraud for the multiple validation models generated as described above. In short, the impact calculation unit 33 outputs scores for new records using validation models generated for multiple second data sets each having different training records to be excluded from all training records.

[0038] The influence of the excluded training record on the risk score can be grasped by calculating the score of the validation model with the influence calculation unit 33. In short, the influence of each training record supported by the validation model is calculated by comparing the score of the new record based on the basic model with the score of the new record based on the validation model.

[0039] Referring to Figure 3, first, a new record for a certain fiscal year (s) of the company to be audited is input as new data to the basic model generated using all training records. The score (h (full) (x (s) ) is calculated.

[0040] On the other hand, even for the validation model in which the training record with ID 1 was excluded, the score (h (-1) (x (s) The calculation of the score of the verification model can be performed as many times as the number of generated verification models.

[0041] And the impact is the score by the base model (h (full) (x (s) )) and the score by the validation model (e.g., h (-1) (x (s) )) and the difference (for example, I (1→s) = h (full) (x (s) ) - h (-1) (x (s) In other words, this difference indicates the influence of the excluded training record (e.g., the record with ID=1).

[0042] By calculating the degree of influence for each validation model in this manner, it becomes possible to grasp the degree of influence that the training records excluded from each validation model have on new records.

[0043] The similar data identifying unit 34 identifies a training record similar to the new record based on the influence calculated by the influence calculation unit 33. For example, a verification model having the largest difference from the risk output by the basic model is identified, and the training record excluded from the verification model is identified as the training record most similar to the new record.

[0044] The display control unit 35 controls the output format, etc., of the influence of each verification model calculated by the influence calculation unit 33, the training records identified by the similar data identification unit 34, etc., to the display device 5.

[0045] The process flow of the accounting audit support method according to this embodiment will be described below with reference to the flowchart shown in FIG.

[0046] First, in step S1, all training records usable for machine learning are read from the data stored in the storage unit 4, and all training records are set in the training data setting unit 31. For example, as shown on the left side of Fig. 3, when N training records usable for machine learning are read, all training records (N records) are set as training data.

[0047] Each of the N training records is assigned an identifier (ID), which is a combination of an explanatory variable x and a fraud label y. Figure 4 shows examples of items that are considered to be effective as explanatory variables for training records used in machine learning when evaluating the risk of accounting fraud.

[0048] For example, items listed in the financial statements that can be used as explanatory variables include inventory turnover period, inventory-to-net asset ratio, inventory-to-total asset ratio, rate of change in accounts receivable, sales growth rate, rate of change in accounts receivable divided by sales growth rate, amount of change in accounts receivable, accounts receivable turnover period, rate of change in accounts receivable turnover period, balance of tangible fixed assets, net goodwill ratio, retained earnings ratio of deferred tax assets, overseas sales ratio, operating profit margin, non-operating profit and loss, non-operating profit and loss ratio, operating profit to cash flow from operating activities, and tangible fixed asset ratio.

[0049] FIG. 5 is an explanatory diagram illustrating an example of a data set consisting of multiple training records. Records of the same company can be used as separate training records if their accounting years and months are different. A training record is a combination of the values ​​of explanatory variables (x1-x4) selected from the explanatory variable items listed above and a fraud label y assigned to that training record. In short, it is training data in which the solution when the values ​​of the explanatory variables (x1-x4) are input is the fraud label y. Here, the fraud label y assigned to records in which accounting fraud occurred is set to 1, and the fraud label y assigned to records in which no accounting fraud occurred is set to 0.

[0050] In step S1, using all the training records that can be explained in this manner, the learning model generation unit 32 performs machine learning to generate a basic model, and the basic model is stored in the learning model storage unit 41.

[0051] Next, in step S2, the training records to be excluded from all training records are set in the training data setting unit 31. For example, as shown on the right side of Fig. 3, training data (N-1 records) obtained by excluding the training record with ID=1 from all training records is set as a data set.

[0052] In step S3, the learning model generation unit 32 performs machine learning using the training data (N-1 records) to generate a first verification model, and the first verification model is stored in the learning model storage unit 41.

[0053] In step S4, it is determined whether or not to generate a validation model using another data set that excludes different training records. Normally, multiple validation models are generated, so the first time step S4 is passed, the process proceeds to "YES." For training records in which at least the fraudulent label y is 1, a data set is set that excludes the training records from all training records, and each validation model is generated.

[0054] After proceeding to "NO" in step S4, in step S5, a score is calculated by the basic model of the new record to be audited. FIG. 6 is an explanatory diagram illustrating an example of the output by the basic model of the new record to be audited. In this example, when a new record (values ​​of explanatory variables x1-x4) extracted from the financial statements of company z, which is the company to be audited, for the fiscal year ending March 2023, is input, the evaluation result calculated by the basic model is output as a fraud score of 0.326.

[0055] This fraud score can be considered an assessment of the risk of accounting fraud, with 1 being the presence of accounting fraud. However, it is difficult to specifically examine the possible methods by which accounting fraud may be committed using this number alone.

[0056] Therefore, in step S6, the score of the new record is calculated using the verification model. Here, the same number of verification models as generated in steps S2 and S3 are used. Fig. 7 is an explanatory diagram illustrating an example of the score difference between the output using the basic model and the output using the verification model for the new record.

[0057] The base models in the "A) Score" column in Figure 7 all have the same value (fraud score of 0.326). On the other hand, the verification models in the "B) Score" column have different fraud score values ​​for each verification model.

[0058] Therefore, in step S7, the difference in score between the base model and the verification model is calculated, and the verification models are arranged in descending order of the score difference (AB) value, as shown in FIG.

[0059] Here, the validation model is a machine learning model based on a dataset in which certain training records are excluded from all training records. Therefore, if there is only one excluded training record in each dataset, the validation model can be expressed by the company name and fiscal year and month.

[0060] For example, the validation model with the largest score difference (AB) and the highest difference ranking was generated from a dataset that excluded training records from Company A's fiscal year end, September 2006, which shows the extent to which these training records have an impact on new records.

[0061] Therefore, in step S8, the training record with the fiscal year and month of September 2006 of company A is identified as the training record most similar to the new record to be audited. In short, the similar data identification unit 34 assigns difference rankings, and the output of the list shown in Fig. 7 identifies the training record most similar to the new record based on the influence level, which is the score difference.

[0062] By examining past cases of fraud identified in this way for Company A's fiscal year ending in September 2006, those conducting accounting audits will be able to take specific auditing actions for new records.

[0063] So far, for simplicity, we have described the case where a validation model is generated by excluding one training record from all training records. However, it is also possible to create a validation model using a dataset in which multiple training records are excluded from all training records.

[0064] For example, a validation model in which one record is excluded from all training records may not produce a significant score difference. In such cases, consider combining multiple training records to generate a validation model in which one record is excluded from all training records, and then calculating the score difference.

[0065] However, when combining multiple training records, the computational load becomes enormous if it is done comprehensively, so it is preferable to limit the combination method to reduce the computational cost. For example, it is possible to combine them meaningfully by grouping them by the same company or industry.

[0066] FIG. 8 is an explanatory diagram illustrating an example of attributes of training records. For example, when looking at the influence of a group of companies, a validation model is generated by excluding all training records of the relevant companies, and the score difference is calculated. Explaining with reference to FIG. 8, when it is desired to verify the influence of company a, a validation model is generated by excluding all training records with record IDs (1), (2), and (3). When it is desired to verify the influence of company b, a validation model is generated by excluding all training records with record IDs (4), (5), and (6). When it is desired to verify the influence of company c, a validation model is generated by excluding all training records with record IDs (7) and (8). When it is desired to verify the influence of company d, a validation model is generated by excluding all training records with record IDs (9) and (10).

[0067] On the other hand, when looking at the impact between sets of the same industry, a validation model is generated in which the training records of the relevant industry are excluded collectively, and the score difference is calculated. Explaining this with reference to Figure 8, if you want to verify the impact of industry A, you generate a validation model by excluding all the training records with record IDs (1), (2), (3), (4), (5), and (6) collectively. If you want to verify the impact of industry B, you generate a validation model by excluding all the training records with record IDs (7) and (8). Furthermore, if you want to verify the impact of industry C, you generate a validation model by excluding all the training records with record IDs (9) and (10) collectively.

[0068] Next, the operation of the accounting audit support device 1 and the accounting audit support method according to this embodiment will be described. The accounting audit support device 1 and accounting audit support method of this embodiment, configured as described above, include a similar data identification unit 34 that identifies training records similar to a new record to be audited from among the multiple training records used for machine learning.

[0069] This makes it easy to identify past cases similar to the audit target. In other words, if it is possible to identify training records that have a large impact on the evaluation results of a new record from among the individual training records, it becomes possible to refer to past fraud cases.

[0070] For past cases identified as similar, the past fraud methods have been made public, so by referring to them, audit procedures can be deepened. In interpreting accounting fraud risks, it is generally considered effective because in many cases, the methods of accounting fraud are publicly disclosed in investigation reports, etc., and the relevance of accounting data in past cases determined to be similar is often convincing.

[0071] In addition, when generating a verification model, the calculation costs required to generate a machine learning model can be reduced by limiting the records to be excluded to those that have previously been involved in fraudulent cases. In short, the objective of identifying similar cases (fraudulent records) from the past can be achieved even if the records are limited to only those that have previously been involved in fraudulent cases.

[0072] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes that do not deviate from the gist of the present invention are included in the present invention.

[0073] For example, in the above embodiment, the items listed in financial statements are used as explanatory variables, but this is not limited to this, and various data such as those listed in various accounting documents can be used as explanatory variables. [Explanation of symbols]

[0074] 1: Accounting audit support device 32: Learning model generation unit 33: Influence calculation part 34: Similar data identification section 41: Learning model memory unit

Claims

1. An accounting audit support device for evaluating a risk of accounting fraud using a machine learning model, comprising: a learning model generation unit that generates a machine learning model using a plurality of training records that can be used for machine learning; a learning model storage unit that stores a base model generated by the learning model generation unit using a first data set consisting of a plurality of training records and a plurality of validation models generated by the learning model generation unit using a second data set obtained by excluding at least one record from the first data set; an impact degree calculation unit that calculates an impact degree by comparing an output of the new record to be audited by the basic model with an output of the new record by a plurality of the validation models having different records to be excluded; and a similar data specification unit that specifies a training record similar to the new record based on the degree of influence.

2. The accounting audit support device described in claim 1, characterized in that the influence degree is the difference between the output from the basic model and the output from the verification model, and the training record excluded from the verification model in which the difference is the largest is identified as being most similar to the new record.

3. 1. An accounting audit support method for evaluating a risk of accounting fraud using a machine learning model, comprising: generating a base model by machine learning using a first dataset of a plurality of training records; generating a validation model by performing machine learning using a second dataset obtained by excluding at least one record from the first dataset; A step of performing machine learning using a dataset that excludes training records different from the second dataset to generate another validation model a desired number of times; calculating a difference between an output by the base model and an output by each of the plurality of validation models for a new record to be audited; and identifying the training record excluded from the validation model where the difference is the largest as being most similar to the new record.

Images