Training record specifying apparatus and training record specifying method

Abstract

To provide a training record specifying apparatus which facilitates specifying a similar case example from among a plurality of training records used in machine learning.SOLUTION: A training record specifying apparatus includes: a learning model generation unit 32 which generates a machine learning model; a training data setting unit 31 which sets training data obtained by sampling records for use in generating the machine learning model from among a plurality of training records; a learning model storage unit 41 which stores a plurality of generated weak learning models; a group generation unit 33 which divides the plurality of weak learning models into two groups based on at least one training record; an influence calculation unit 34 which calculates a degree of influence obtained by comparing an output of a new record by a weak learning model belonging to a first group and an output of the new record by a weak leaning model belonging to a second group, on the new record to be evaluated; and a similar data specifying unit 36 which specifies a training record similar to the new record based on the degree of influence.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a training record identification device and a training record identification method for identifying a training record similar to a new record to be evaluated from among a plurality of training records used in machine learning. [Background technology]

[0002] For example, in accounting audit work such as internal audits and external audits, the contents of accounting documents such as financial statements are analyzed and checked to see if there are any fraudulent transactions, such as fictitious transactions (see Patent Documents 1 and 2, etc.).

[0003] Also, as disclosed in Patent Literature 3, there are attempts to use the results of machine learning to evaluate the risk of accounting fraud through computer processing. Here, the calculations performed in the learning process of machine learning are complex, and while there is an advantage in that highly accurate evaluation can be realized, it is often difficult for humans to easily understand the basis for the evaluation.

[0004] Therefore, in Patent Document 3, it is possible to determine which account items, which are explanatory variables used in machine learning, are abnormal for the evaluation output by the machine learning model by calculating the SHAP value. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7052135 [Patent Document 2] JP 2019-179531 A [Patent Document 3] Patent No. 7216854 Summary of the Invention [Problem to be solved by the invention]

[0006] However, simply identifying the account items that have a large impact on the evaluation results makes it difficult to grasp the specific circumstances of accounting fraud such as fictitious transactions, and ultimately requires an accountant with extensive experience in accounting audit work to clarify the details of the accounting fraud based on the numerical values ​​of the identified account items. On the other hand, if past cases similar to the audit target can be identified, it will be easier to refer to published past fraud methods, leading to deeper audit procedures.

[0007] Therefore, an object of the present invention is to provide a training record identification device and a training record identification method that make it possible to easily identify similar cases from among multiple training records used in machine learning. [Means for solving the problem]

[0008] In order to achieve the above object, a training record identification device of the present invention is a training record identification device for identifying a training record similar to a new record to be evaluated from among a plurality of training records used for machine learning, and is characterized in that it includes a learning model generation unit that generates a machine learning model using a plurality of training records that can be used for machine learning, a training data setting unit that sets training data obtained by sampling records to be used for generating a machine learning model from the plurality of training records, a learning model storage unit that stores a plurality of weak learning models generated by the learning model generation unit using the training data, a group generation unit that divides the plurality of weak learning models stored in the learning model storage unit into two groups based on at least one training record, an influence calculation unit that calculates an influence degree by comparing an output of the new record by the weak learning model belonging to the first group generated by the group generation unit with an output of the new record by the weak learning model belonging to the second group, with respect to a new record to be evaluated, and a similar data identification unit that identifies a training record similar to the new record based on the influence degree.

[0009] Here, the first group is constituted by the weak learning model generated by the training data including the training record used to divide the group, and the second group is constituted by the weak learning model generated by the training data not including the training record used to divide the group.

[0010] The system may also be configured to include a significance evaluation unit that evaluates the significance of the impact calculated by the impact calculation unit, and the similar data identification unit may be configured to identify training records that are similar to the new record based on the impact and the evaluation results of the significance evaluation unit.

[0011] Furthermore, the influence degree can be the difference between the outputs of the weak learning models belonging to the two groups generated by the group generation unit, and the significance evaluation unit can be configured to perform a significance test of the difference.

[0012] The invention of a training record identification method is a training record identification method for identifying a training record similar to a new record to be evaluated from among multiple training records used in machine learning, comprising the steps of: generating a weak learning model by performing machine learning using training data sampled from the multiple training records a desired number of times; dividing the multiple weak learning models into two groups based on at least one training record; calculating the difference between the outputs of the weak learning models belonging to the two groups for the new record to be evaluated; and identifying the training record used to divide the group with the largest difference as the most similar to the new record. Here, the method may be configured to include a step of evaluating the significance of the difference. Effect of the Invention

[0013] The training record identification device and training record identification method of the present invention configured in this manner can identify training records similar to a new record to be evaluated from among multiple training records used in machine learning.

[0014] This makes it easy to identify past cases similar to the audit target, for example, in accounting audit work. In other words, if it is possible to identify training records that have a large impact on the results of the assessment of the risk of accounting fraud from among the individual training records, it becomes possible to refer to past fraud cases, etc. For cases identified as similar, the past fraud methods, etc. have been made public, so by referring to them, it is possible to deepen the audit procedure. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram illustrating the configuration of a training record identification device according to an embodiment of the present invention. [Diagram 2] 11 is a flowchart illustrating the process flow of a training record identification method according to the present embodiment. [Diagram 3] 1 is an explanatory diagram showing an overview of a training record identification method according to an embodiment of the present invention; [Figure 4] FIG. 1 is an explanatory diagram illustrating a weak learner equipped with a weak learning model. [Diagram 5] FIG. 1 is an explanatory diagram illustrating an example of the relationship between sampling, training data, and a weak learner. [Figure 6] FIG. 1 is an explanatory diagram outlining a method for splitting multiple weak learners into two groups using a single training record. [Figure 7] FIG. 1 is an explanatory diagram illustrating an example of a relationship between training data and a weak learner. [Figure 8] FIG. 11 is an explanatory diagram illustrating two divided groups. [Figure 9] FIG. 1 is an explanatory diagram illustrating a model configured by weak learners belonging to two groups. [Figure 10] FIG. 13 is an explanatory diagram illustrating an example of a score difference that is an influence degree. [Figure 11] FIG. 11 is an explanatory diagram illustrating a method for evaluating the significance of a score difference. [Figure 12] FIG. 11 is an explanatory diagram illustrating a method for testing a significant difference in influence degree. [Figure 13] FIG. 1 is an explanatory diagram illustrating a method for splitting multiple weak learners into two groups based on multiple training records. [Figure 14] FIG. 11 is an explanatory diagram illustrating the results of implementing a training record identification method using a new record. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram illustrating the configuration of a training record identification device 1 according to the present embodiment. Fig. 2 is a flowchart illustrating the process flow of a training record identification method according to the present embodiment.

[0017] In this embodiment, an example will be described in which a machine learning model is used to evaluate the risk of accounting fraud for new records (for example, on a fiscal year / month basis) of a company that is the subject of an accounting audit such as an internal audit or an external audit, but as will be described later, the training record identification device 1 can be applied in various fields. In an accounting audit, the evaluation of the risk of accounting fraud includes both the detection of fraud at the present time and the prediction of fraud in the future.

[0018] In the training record identification device 1 of this embodiment, as shown in Figure 1, processing is performed in an arithmetic processing unit 3 based on data input from an input device 2, a memory unit 4, etc., and evaluation results, etc. are displayed on a display device 5, etc.

[0019] The calculation processing unit 3 of the training record identification device 1 is mainly composed of a training data setting unit 31 that sets training data to be used for machine learning, a learning model generation unit 32 that generates a machine learning model using training records included in the training data, a group generation unit 33 that groups the generated multiple machine learning models, an influence calculation unit 34 that calculates the influence of the training record on a new record, a significance evaluation unit 35 that evaluates the significance of the influence, a similar data identification unit 36 ​​that identifies training records similar to the new record, and a display control unit 37 that controls the display content on the display device 5, etc.

[0020] The input device 2 for inputting various data is a data input means connected to or equipped on a personal computer (PC), a notebook computer, a tablet terminal, a wearable terminal, a smartphone, etc. Examples of the input device 2 include a keyboard, a mouse, a touch panel, a touch pad, a scanner, a microphone for voice input, a camera, etc.

[0021] On the other hand, the display device 5 may be a liquid crystal display, an organic EL (Electro-Luminescence) display, a printer, or the like.

[0022] Furthermore, the storage unit 4 is a storage medium for recording training data used in processing in the arithmetic processing unit 3, data generated by arithmetic processing, etc., and corresponds to a hard disk, a solid state drive (SSD), a flash memory (such as an SD memory card), a magnetic disk, an optical disk, etc. Also, an external online storage (cloud storage) such as a server connected via a network can be used as the storage unit 4. Here, the area of ​​the storage medium for storing the machine learning model generated by the learning model generation unit 32 will be described by distinguishing it by assigning it the reference symbol "learning model storage unit 41."

[0023] The arithmetic processing unit 3 is configured, as hardware, with a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc., and is equipped with memories such as a random access memory (RAM) and a read only memory (ROM).

[0024] The arithmetic processing unit 3 can execute various functions by applications installed in a computer. Also, a server connected via a network such as the Internet can execute part or all of the arithmetic processing unit 3. The network is composed of part or all of the Internet, a Wide Area Network (WAN), a wired Local Area Network (LAN), a wireless LAN (Wi-Fi), a provider device, a wireless base station, a dedicated line, etc.

[0025] The training data setting unit 31 of the calculation processing unit 3 can arbitrarily set a training record to be used to generate a machine learning model from among multiple training records stored in the memory unit 4 that can be used for machine learning.

[0026] In the training record identification device 1 of the present embodiment, a record to be used for generating a machine learning model is sampled from a plurality of training records, and set as training data. The sampling method is not particularly limited, and for example, a bootstrap method can be used.

[0027] The learning model generation unit 32 generates a weak learning model by performing machine learning using the training data (data set) set by sampling in the training data setting unit 31. Any method of machine learning may be used, and any method such as a decision tree may be applied. Then, sampling is repeated a desired number of times, and a weak learning model is generated using the training data set each time.

[0028] 3 is an explanatory diagram showing an overview of the training record specification method of the present embodiment. The ensemble model is a method for constructing a model that performs highly accurate evaluation by integrating the learning results of multiple weak learners, and examples of the ensemble model include bagging and random forest.

[0029] Random forest creates a dataset (training data) by extracting multiple training records from a given dataset using bootstrap sampling, and derives the final evaluation results using an ensemble model that integrates multiple weak learners generated from these multiple training data.

[0030] 4 is an explanatory diagram illustrating a weak learner equipped with a weak learning model. As shown in this figure, it is assumed that N training records with IDs (identifiers) ranging from 1 to N are stored in the storage unit 4. Note that the storage unit 4 may store other training records.

[0031] Then, the training data setting unit 31 reads N training records (N records) as all training records (all training records) to be used for machine learning, and performs sampling from among all the training records.

[0032] The learning model generation unit 32 generates a weak learner that serves as a weak learning model by performing machine learning using training data sampled from all training records. Fig. 4 shows training data from 1 to M, in which a z column is added to the training data table, z=1 is assigned to sampled target records (training records), and z=0 is assigned to non-target records.

[0033] In this way, if machine learning is performed using training data (dataset) set by sampling from all training records (N records), a weak learning model for each machine learning corresponding to the training data will be generated. In short, by repeating sampling and machine learning M times, M weak learners (f [1] ,f [2] ,f [3] ,f[M-1] ,f [M] ) is generated.

[0034] The trained machine learning model (weak learning model) generated by the learning model generation unit 32 in the above manner is stored in the learning model storage unit 41. Then, the model is read and used during calculations by the group generation unit 33, which will be described later, and the like.

[0035] The group generation unit 33 divides a plurality of weak learning models (weak learners) stored in the learning model storage unit 41 into two groups based on at least one training record. In short, in order to see the influence of a specific training record Y on a new record X to be evaluated, a large number of weak learners are created and divided into "(1) a first group of weak learners in which a specific training record Y is used" and "(2) a second group of weak learners in which a specific training record Y is not used."

[0036] Then, the impact of a specific training record Y on the evaluation target (new record X) is measured from the difference between the score of a new record X obtained by a learning device created by bundling weak learners belonging to the first group and the score of a new record X obtained by a learning device created by bundling weak learners belonging to the second group.

[0037] This particular training record Y may be a single training record (single record) or a combination of multiple training records (multiple records). There are multiple ways of using it, as shown in FIG.

[0038] For example, after creating an ensemble model, it is possible to calculate only the influence of single records and identify training records with high influence as related records ((1) → (2) → (3) in Figure 3). Also, it is possible to define in advance how multiple records are combined, calculate the influence of multiple records from the beginning, and identify them as related records ((1) → (4) → (5) in Figure 3). Furthermore, it is also possible to calculate the influence of single records first, and then calculate the influence of multiple records for pairs of records with high influence as single records, and identify them as related records ((1) → (2) → (3) → (4) → (5) in Figure 3).

[0039] First, for the sake of simplicity, a method for dividing into two groups by a single record will be described with reference to Figures 5 and 6. Figure 5 is an explanatory diagram illustrating an example of the relationship between sampling, training data, and weak learners.

[0040] As shown in FIG. 5, there are weak learners in which a certain record (n) is sampled and weak learners in which a record (n) is not sampled. (n)[m] The weak learner [m] with sampled record (n) is set to 1, and the weak learner [m] with unsampled record (n) is set to 0. In this way, if a single record is identified, the weak learner (f [1] ,f [2] ,f [3] ,f [M-1] ,f [M] ) can be divided into which group.

[0041] FIG. 6 is an explanatory diagram showing an overview of a method for splitting multiple weak learners into two groups using a single training record. For example, let us consider the M weak learners (f [1] ,f [2] ,f [3] ,f [M-1] ,f [M]) is divided into group A of weak learners ([1],[3],...,[M]) in which record (1) has been sampled, and group B of weak learners ([2],...,[M-1]) in which record (1) has not been sampled. Then, the score of the new record is calculated for each of group A and group B, and the influence is calculated from the score.

[0042] Specifically, the impact calculation unit 34 outputs a risk score (hereinafter sometimes simply referred to as "score") that is the evaluation result of the risk of accounting fraud for a new record to be audited in a model composed of group A (first group), and also outputs the risk in a model composed of group B (second group).

[0043] The scores output using the models for each group, A and B, indicate the risk of accounting fraud for new records, but just looking at this value does not make it clear how the accounting fraud is being committed, making it difficult to link it to specific audit actions.

[0044] Therefore, in the training record identification device 1 of this embodiment, the influence of the training record identified for dividing the group is calculated by the influence calculation unit 34. Here, the relationship between the training data and the weak learner is shown again in FIG. 7. The influence I of a certain training record (n) on the score of a new record (s) in a certain accounting year and month (s) of the company to be audited is calculated as (n→s) can be formulated as follows:

[0045] First, a training record (n) is selected, and the M weak learners (f [1] ,f [2] ,f [3] ,f [M-1] ,f [M] ) is Group A, as shown in Figure 8. (n) and Group B (n) Here, the number of weak learners in each group is |A (n) | and |B (n) Indicated by |.

[0046] And, as shown in Figure 9, Group A (n) The score of the model that combines weak learners belonging to [A(n)] (x) to g({f [k] (x)|k∈A (n)}) and Group B (n) The score of the model that combines weak learners belonging to [B(n)] (x) to g({f [k] (x)|k∈A (n)}) Here, the function g is not particularly limited and may be any function such as an average value or a median value.

[0047] The score of the model for each group defined in this way (h [A(n)] (x),h [B(n)] (x)) is the influence I of the training record (n) on the score of the new record (s). (n→s) can be calculated for each training record (1, 2, . . . , N) as shown in FIG. 10. For example, the influence I (1→s) h [A(1)] (x (s) ) - h [B(1)] (x (s) ) score difference.

[0048] In this way, by calculating the score difference between the two groups A and B as the impact in the impact calculation unit 34, it becomes possible to grasp the impact on the risk score of the training record used for grouping.

[0049] The significance evaluation unit 35 evaluates the significance of the influence degree thus calculated. (n→s) There are two groups, (n) ,B (n) Score difference between (h [A(n)] (x (s) )-h [B(n)] (x (s) )), but a significance test is used to evaluate whether the numerical value is significant.

[0050] FIG. 11 is an explanatory diagram illustrating a method for evaluating the significance of the score difference. As shown in this figure, the respective influence degrees I (n→s) Calculate the p-value (significance probability) of and add the indicator column of the training record (1, 2, , N). Influence I (n→s) p1, which is the p-value of (n) (x (s) ) for each Group A (n) ,B (n) The number of weak learners in (|A (n) |,|B (n) |) and each Group A (n) ,B (n) score (h [A(n)] (x (s) ),h [B(n)] (x (s) ) can be used to calculate it.

[0051] Next, as shown in Figure 12, the data sequence of the training record (1, 2, . . . , N) is calculated as the influence I (n→s) Sort in descending order using the key, and then use the impact I (n→s) Let the ID of the record with the nth largest value be (rn). By doing this, the influence of record (r1) I (r1→s) and the influence of record (rn) I (rn→s) It is now possible to test the difference between this and the p-value (p2 (r1,rn) (x (s) )) to the indicator column, where p2 (r1,rn) (x (s) ) is the influence I of record (r1) (r1→s) and the influence of record (rn) I (rn→s) Difference from (=I (r1→s) -I (rn→s) ) is the p-value of the significance test.

[0052] In order to determine that a record (r1) is the single most influential relevant record in the training data, the following conditions must be met: <Condition 1> Impact of record (r1) I (r1→s) But it's big enough. In short, the largest influence among all training records is I (r1→s)However, it becomes necessary that this is not simply a sampling error.

[0053] <Condition 2> Impact of record (r1) I (r1→s) However, the influence of other records (rn) I (rn→s) Much larger than that. In short, the largest influence among all training records is I (r1→s) However, the second and subsequent records I (r2→s) The effect of the variance must be significantly greater than that of the variance.

[0054] The similar data identification unit 36 ​​identifies training records similar to the new record based on the influence calculated by the influence calculation unit 34. (n) ,B (n) Score difference between (h [A(n)] (x (s) )-h [B(n)] (x (s) )) is the impact I (n→s) is calculated, and the record (r1) whose significance has been evaluated by the significance evaluation unit 35 is identified as the training record most similar to the new record.

[0055] The display control unit 37 controls the output format, etc., of the influence calculated by the influence calculation unit 34, the results of the significant difference test by the significance evaluation unit 35, and the training records identified by the similar data identification unit 36 ​​to the display device 5.

[0056] The process flow of the training record identification method of this embodiment will be described below with reference to the flowchart shown in FIG.

[0057] First, in step S1, all training records that can be used for machine learning are read from the data stored in the storage unit 4. Then, in step S2, the training data setting unit 31 sets training data.

[0058] When N training records that can be used for machine learning are loaded, the training data is set as training data for subsequent generation of a machine learning model by sampling all training records (N records).

[0059] Each of the N training records is assigned an identifier (ID), which is a combination of explanatory variable x and a fraud label y (see Figure 4). For example, the following items are considered to be effective as explanatory variable x for training records used in machine learning when evaluating the risk of accounting fraud.

[0060] For example, items listed in the financial statements that can be used as explanatory variables include inventory turnover period, inventory-to-net asset ratio, inventory-to-total asset ratio, rate of change in accounts receivable, sales growth rate, rate of change in accounts receivable divided by sales growth rate, amount of change in accounts receivable, accounts receivable turnover period, rate of change in accounts receivable turnover period, balance of tangible fixed assets, net goodwill ratio, retained earnings ratio of deferred tax assets, overseas sales ratio, operating profit margin, non-operating profit and loss, non-operating profit and loss ratio, operating profit to cash flow from operating activities, and tangible fixed asset ratio.

[0061] Even if the records are from the same company, if the accounting years and months are different, they can be used as separate training records. A training record is a combination of explanatory variable values ​​selected from the explanatory variable items listed above and a fraudulent label y assigned to that training record. In short, it is training data in which the solution when the explanatory variable value is input is the fraudulent label y.

[0062] Referring to the training data [1] shown on the left side of Figure 4, the z column indicates whether each training record (1, 2, . . . , N) was sampled (z = 1) or not (z = 0). The training data sampled from all training records in this way is set as the machine learning dataset.

[0063] In step S3, the learning model generation unit 32 performs machine learning using the training data [1] to generate a first weak learning model, and stores the first weak learning model in the learning model storage unit 41 as a weak learner.

[0064] In step S4, it is determined whether or not to generate another weak learning model using different training data. Normally, multiple weak learning models are generated, so the first time step S4 is passed, the process proceeds to "YES". As shown in FIG. 4, in order to generate M weak learners, steps S2 and S3 are executed M times. The M weak learners thus generated are stored in the learning model storage unit 41.

[0065] After proceeding to "NO" in step S4, grouping is performed in step S5 to divide the M weak learners into two groups. The group generation unit 33 sets one or more training records from all training records (1, 2, . . . , N) as training records for division.

[0066] We have already explained the case where a single record is set as the training record for splitting, so here we will explain, with reference to FIG. 13, how to split multiple weak learners into two groups using multiple training records.

[0067] In the following explanation, a combination of two records is taken as an example, but a combination of three or more records can also be explained in the same way. The set of training records to be used for calculating the influence degree can be determined from the following viewpoints.

[0068] <Set 1> All combinations of training records You can create simple combinations, but the number of combinations is N C2, and when the number of total training records N is large, this can cause a combinatorial explosion.

[0069] <Group 2> Combinations that are meaningful in practice For example, an appropriate combination is selected based on domain knowledge, such as combining all fraud periods of companies that committed fraud.

[0070] <Set 3> The combination of records with the smallest p-value (p1, p2) calculated in the process of deriving the influence of a single record In short, the scope is narrowed down to only records whose single record has a relatively large influence, and pairs with a large influence are efficiently searched for.

[0071] Therefore, as shown in Figure 13, we select a set of training records (n1, n2) and assign M weak learners to group A. (n1,n2) and Group B (n1,n2) Here, group A is (n1,n2) is a group of weak learners sampled together on a pair of training records (n1,n2), and group B (n1,n2) is a group of weak learners for which the pair (n1,n2) of training records was not sampled together.

[0072] For two groups A and B divided by a single record or multiple records, group A (A (n) ,A (n1,n2) ),B(B (n1,n2) ,B (n1,n2) ) are defined, and the number of weak learners in each group A and B (|A (n) |,|A (n1,n2) |),(|B (n) |,|B (n1,n2) |) is calculated (Figures 8 and 13).

[0073] In step S6, the score difference between the two groups A and B of new records to be audited is calculated. For example, new records extracted from the financial statements of a certain accounting year of a company to be audited are input into the models of groups A and B, and the scores are calculated. For example, as shown in FIG. 9, for group A, (n) The score of the model consisting of [A(n)] (x)) and Group B (n) The score of the model consisting of h [B(n)](x) is calculated.

[0074] And two groups A (n) ,B (n) Score difference between (h [A(n)] (x)-h [B(n)] (x)) is calculated as the influence. In the next step S7, it is determined whether or not to perform different group division based on different training records. Since group division is usually performed multiple times, the first time step S7 is passed, the process proceeds to "YES".

[0075] For example, when a single record is set and group division is performed, at least all training records that have had fraudulent cases in the past are used as the basis for division, so steps S5 and S6 are repeated according to the number of such records.

[0076] After proceeding to "NO" in step S7, the significance evaluation unit 35 evaluates the significance of the score differences (degrees of influence) calculated for the number of group divisions (step S8).

[0077] For a training record for which a large score difference (impact) is calculated in step S6 and the significance of the score difference is evaluated in step S8, the training record is identified in step S9 as the training record most similar to the new record to be audited.

[0078] By examining the past fraud cases identified in this way, accounting audit workers can take concrete audit actions for new records. Figure 14 is an explanatory diagram illustrating the results of implementing the training record identification method on the financial data of a certain company for a certain fiscal year as new records.

[0079] In this table, the training record of Company A with its fiscal year of September 2006 is displayed with the highest difference ranking as the training record most similar to the new record to be audited. This table not only displays the scores and score difference (AB) of the new records for each group A and B, but also the results of the t-test using the p-value calculated as described above.

[0080] Here, the differential t-value I indicates the t-value of the significance test of the difference between the score of group A with the target differential ranking and the score of group B with the same differential ranking. Also, the differential t-value II indicates the t-value of the significance test of the difference between the score difference of the target differential ranking (2nd place or later) and the score difference of the 1st differential ranking.

[0081] In this way, the similar data identification unit 36 ​​ranks the differences, and the output of the list shown in the example of FIG. 14 identifies the training record that is most similar to the new record based on the evaluation results of the impact and significance, that is, the score difference (difference t value I, difference t value II).

[0082] In identifying training records similar to such new records, it may not be possible to identify a single record with the greatest impact. In such cases, the impact can be calculated as the impact of multiple records that combine multiple training records, as described above with reference to Figure 3. For example, in a company that committed fraud over multiple periods, the impact of a single year's annual report may be small, but the impact calculated from the combination of all annual reports for the fraudulent period may be large.

[0083] Next, the operation of the training record specifying device 1 and the training record specifying method of the present embodiment will be described. The training record identification device 1 and training record identification method of this embodiment, configured as described above, include a similar data identification unit 36 ​​that identifies training records similar to a new record to be audited from among multiple training records used for machine learning.

[0084] This makes it easy to identify past cases similar to the audit target. In other words, if it is possible to identify training records that have a large impact on the evaluation results of a new record from among the individual training records, it becomes possible to refer to past fraud cases.

[0085] For past cases identified as similar, the past fraud methods have been made public, so by referring to them, audit procedures can be deepened. In interpreting accounting fraud risks, it is generally considered effective because in many cases, the methods of accounting fraud are publicly disclosed in investigation reports, etc., and the relevance of accounting data in past cases determined to be similar is often convincing.

[0086] In addition, if a method is used in which a plurality of weak learners are generated in advance, then divided into two groups, and evaluation is performed using an ensemble model in which the weak learners constituting each group are integrated, the model creation time is not increased compared to the case in which a machine learning model is created each time. In addition, the effort required to manage the machine learning model and the file size are not increased each time evaluation is performed.

[0087] Furthermore, when considering the impact of multiple records, there is no need to create a new machine learning model; the impact of multiple records can be easily calculated by simply dividing multiple weak learners that have already been generated into new groups.

[0088] In addition, since the number of weak learners is calculated for group A (first group) that includes the training records used to split the groups, and for group B (second group) that does not, it becomes possible to perform a significance test and determine the significance of the calculated score difference (degree of impact).

[0089] Furthermore, when calculating the degree of influence by applying an influence function, it cannot be applied to non-differentiable models, but when applying an ensemble model, there are no model constraints such as whether the model is differentiable or not, and the range of application can be expanded.

[0090] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes that do not deviate from the gist of the present invention are included in the present invention.

[0091] For example, in the above embodiment, a case has been described in which the training record identification device 1 and training record identification method of the present embodiment are applied mainly to evaluating the risk of accounting fraud in accounting audit work, but this is not limited to this.

[0092] For example, in the field of finance, the training record identifying device 1 and the training record identifying method of the present embodiment can be applied when banks etc. evaluate the creditworthiness of borrowers. For example, when evaluating the risk of bad debts by using financial indicators of a company's financial statements for one fiscal year as new records to be evaluated, similar bad debt cases can be identified and used as reference when considering the content of management support.

[0093] In addition, in the field of manufacturing, the training record identifying device 1 and the training record identifying method of the present embodiment can be applied when evaluating the risk of failure of manufacturing equipment. For example, in the case of one piece of equipment information, when evaluating the presence or absence of a failure for a new record in which the operation log, years of use, weather information, etc. of the equipment are used as explanatory variables, similar failure cases can be identified, appropriate maintenance can be performed in advance, and the stoppage of the production line can be prevented.

[0094] Furthermore, in the field of marketing, the training record identification device 1 and the training record identification method of the present embodiment can also be applied to evaluating the purchasing tendency of an individual. For example, in the purchasing activity of one person, when evaluating the purchasing tendency of a new record in which demographic data such as gender, age, residential area, income, occupation, and family structure are used as explanatory variables, an individual who shows a similar purchasing tendency can be identified and a promotional activity in line with the tendency can be implemented to improve the promotional effect.

[0095] In addition, in the medical field, the training record identification device 1 and the training record identification method of the present embodiment can be applied to the diagnosis of illness using images. For example, when evaluating the presence or absence of a tumor for a new record with explanatory variables such as a CT image, health condition, and medical history for one patient's information, the case of a similar patient can be identified, and the case can be used as a reference for guidelines for diagnosis and treatment consideration. [Explanation of symbols]

[0096] 1: Training record identification device 31: Training data setting section 32: Learning model generation unit 33: Group generation section 34: Influence calculation part 35: Significance evaluation section 36: Similar data identification section 41: Learning model memory unit

Claims

1. A training record identification device for identifying a training record similar to a new record to be evaluated from among a plurality of training records used in machine learning, the device comprising: a learning model generation unit that generates a machine learning model using a plurality of training records that can be used for machine learning; A training data setting unit that sets training data obtained by sampling records used to generate a machine learning model from a plurality of training records; a learning model storage unit in which a plurality of weak learning models generated by the learning model generation unit using the training data are stored; a group generation unit that divides a plurality of weak learning models stored in the learning model storage unit into two groups based on at least one training record; an influence calculation unit that calculates an influence by comparing an output of the new record by the weak learning model belonging to a first group generated by the group generation unit with an output of the new record by the weak learning model belonging to a second group, the output of the new record being an evaluation target; a similar data specifying unit that specifies a training record similar to the new record based on the degree of influence.

2. the first group is constituted by the weak learning model generated by the training data including the training records used to divide the group; The training record identification device according to claim 1 , wherein the second group is constituted by the weak learning models generated by the training data that does not include the training record used to divide the group.

3. a significance evaluation unit that evaluates the significance of the influence degree calculated by the influence degree calculation unit, 3. The training record identifying device according to claim 1, wherein the similar data identifying unit identifies a training record similar to the new record based on the degree of influence and the evaluation result of the significance evaluating unit.

4. The influence degree is a difference between outputs by the weak learning models belonging to the two groups generated by the group generation unit, 4. The training record identification device according to claim 3, wherein the significance evaluation unit performs a significance test on the difference.

5. A training record identification method for identifying a training record similar to a new record to be evaluated from among a plurality of training records used in machine learning, the method comprising: A step of performing machine learning using training data sampled from a plurality of training records to generate a weakly learned model a desired number of times; Dividing the plurality of weakly learned models into two groups based on at least one training record; A step of calculating a difference between outputs of the weak learning models belonging to the two groups for a new record to be evaluated; identifying the training record used to divide the group with the largest difference as being most similar to the new record.

6. 6. The method of claim 5, further comprising the step of evaluating the significance of the differences.

Images