Financial statement illegality estimation device, financial statement illegality estimation method and program

The financial statement fraud estimation device analyzes text information from companies to detect fraud using machine learning, addressing the limitations of existing methods by enhancing detection accuracy and stability through text analysis.

JP2024160043AActive Publication Date: 2024-11-11中島 真澄 +2
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024055350
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-04-29
Filing Date
2024-03-29
Publication Date
2024-11-11
Estimated Expiration
2044-03-29

AI Technical Summary

Technical Problem

Existing methods for detecting financial statement fraud, including those using financial statement data and hybrid methods, are inadequate in identifying fraudulent activities due to the complexity of financial statement fraud and the potential for misleading deviations from Generally Accepted Accounting Principles (GAAP), and text information from companies or their managers is often overlooked as a complementary source for fraud detection.

Method used

A financial statement fraud estimation device and method that analyzes text information disclosed by companies or their managers, generating explanatory variables such as kanji content rate, word diversity, numerical rate, and tone, and employs machine learning techniques like ensemble learning to estimate the presence of financial statement fraud.

Benefits of technology

The device provides accurate estimation of financial statement fraud, improving detection capabilities and reducing investment risks by leveraging text information, which remains unchanged in databases, even after amended reports are submitted.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160043000001_ABST
    Figure 2024160043000001_ABST
Patent Text Reader

Abstract

To provide a financial statement illegality estimation device, a financial statement illegality estimation method and a program that estimate illegality of financial statements based upon text information that a business enterprise or its executive discloses.SOLUTION: A financial statement illegality estimation device has: a text information analysis part which analyzes text information that a business enterprise or its executive discloses to generate one or more explanatory variables; a model storage part which stores a learnt model or statistical model showing correlation between the explanatory variables and whether there is illegality of financial statements associated with the business enterprise; and an illegality estimation part which inputs the explanatory variables that the text information analysis part generates to the learnt model to estimate whether there is illegality of the financial states associated with the business enterprise. The explanatory variables include a number rate of the text information.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a financial statement fraud estimation device, a financial statement fraud estimation method and a program, and in particular to a technology for estimating the presence or absence of financial statement fraud based on text information disclosed by a company or its management. [Background technology]

[0002] If fraudulent information on financial statements is disclosed, investors will not be able to make correct investment decisions, the market will not function, and public goods will be damaged. If investors are unable to detect fraudulent companies or select non-fraudulent companies, the market will not be able to distribute profits appropriately. To address this problem, various fraud detection methods have been proposed.

[0003] Overseas, there are methods that apply logistic regression to financial statement data (Non-Patent Documents 1 to 4) and methods that apply quantitative analysis to text information in annual reports (Non-Patent Documents 5 and 6).

[0004] In addition to these quantitative statistical methods, Non-Patent Documents 7 to 15 are methods that apply machine learning to financial statement data. Non-Patent Documents 16 to 19 are methods that apply machine learning to text information.

[0005] In addition, there are hybrid machine learning methods that use both text and financial statement data, such as those in Non-Patent Documents 20 to 24.

[0006] In Japan, Non-Patent Documents 25 to 27 are methods that apply logistic regression analysis to financial statement fraud, and Non-Patent Documents 28 to 30 are methods that apply statistical analysis to text information.

[0007] In addition, Non-Patent Document 31 is a method that applies machine learning to financial statement data.

[0008] Patent Document 1 describes a device that uses machine learning to detect abnormal account items based on a number of account items included in a company's financial statements.

[0009] Patent Document 2 describes a device that uses machine learning to detect abnormal transactions between multiple companies based on account items and transaction information contained in the financial statements of each company. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] Beasley, M.1996. An empirical analysis of the relation between the board of director composition and financial statement fraud. The Accounting Review, 71 (4): 443-465. [Non-Patent Document 2] Beneath, MD 1999. Incentives and penalties related to earnings overstatements that violate GAAP, and post-Sarbanes Oxley period. The Accounting Review, 83 (3): 757-787. [Non-Patent Document 3] Dechow, PM, RG Sloan, and AP Sweeney. 1996. Causes and Consequences of Earnings Manipulation: An Analysis of Firms Subject to Enforcement Actions by the SEC. Contemporary Accounting Research 13 (l): 1-36. [Non-Patent Document 4] Skousen, C. J, K. R. Smith, and C. J. Wright. 2009. Detecting and predicting financial statement fraud: The effectiveness of the fraud triangle and SAS No. 99, in M. Hirschey, K., John, A. K. Makhija (ed.) Corporate Governance and Firm Performance (Advances in Financial Economics, 13), Emerald Group Publishing Limited, 53-A81. [Non-Patent Document 5] Churyk, N. T., C. C. Lee, and B. D. Clinton. 2009. Early detection of fraud: Evidence from restatements. Advances in Accounting Behavioral Research 12: 25-40. [Non-Patent Document 6] Lee, C.C., N. T. Churyk, and B. D. Clinton. 2013. Validating early fraud prediction using narrative disclosure. Journal of Forensic & Investigative Accounting, 5 (1): January-June.35-57. [Non-Patent Document 7] Fanning, K.M, and K. O. Cogger.1998. Neural network detection of management fraud using published financial data, Intelligent Systems in Accounting, Finance and Management 7(1): 21-41. [Non-Patent Document 8] Green , BP , and JH Choi . 1997. Assessing the risk of management fraud through neural-network technology. Auding: A Journal of Practice & Theory 16:14-28.

Outdoor Tools9

Outdoor Tools 10

Outdoor Content11

Outdoor Tools 12

Non-Patent Document 13

Non-Patent Document 14

Non-Patent Document 15

Non-Patent Document 16

[0011] [Patent Document 1] Patent No. 6667865 [Patent Document 2] Patent No. 7146218 Summary of the Invention [Problem to be solved by the invention]

[0012] However, methods using financial statement data (including hybrid methods) do not necessarily achieve the purpose of detecting financial statement fraud sufficiently. Due to the nature of financial statement fraud, it is difficult to identify the signs of fraud in financial statements when detecting fraud, and some signs of fraud may be indicated even if fraud does not exist. Even if financial statements deviate from Generally Accepted Accounting Principles (GAAP), if the deviation is properly disclosed, it is not said that fraud exists. On the other hand, even if financial statements comply with GAAP, the principles may not be completely followed, which may lead to the misunderstanding that fraud exists. It is insufficient to detect fraud using only financial ratios from financial statement data.

[0013] In addition, if fraud such as false statements in financial statements is detected in a previous fiscal year, the company is required to submit a correction report (Article 24, Paragraph 2 of the Financial Instruments and Exchange Act). When the correction report is submitted, the financial statement data in the securities report database is overwritten with the figures from the correction report.

[0014] On the other hand, text information disclosed by companies or their management has an advantage over quantitative information and contains information that complements quantitative information. The text portion contains useful information other than that stated in the financial statements or (non-financial) explanations and interpretations not included in audited financial statements, and therefore may play an important role in deciding on financing or investment.

[0015] In addition, as a rationalization factor of the fraud triangle theory, since the text information of managers directly reflects their values ​​and beliefs, it is highly likely that common characteristics of fraudulent corporate managers can be discovered from the text information. Furthermore, text analysis makes it easy to consider how text information is created by managers, and enables the analysis of the interaction between the attributes of corporate disclosure and the characteristics of the managers and companies behind them. In order to predict fraud, it is important to have a perspective on the decision-making process of managers.

[0016] In addition, unlike financial statement data, narrative (text) information is not changed and is disclosed as is in the database even if a correction report is submitted. Therefore, in the case of narrative information, the state in which financial statement fraud is reflected is maintained, so the performance analysis of fraud detection using narrative (text) information is stable.

[0017] Therefore, it is desirable to develop a fraud detection method that focuses on text information. In particular, when the purpose is to detect financial statement fraud, it has not yet been clarified what kind of text information should be targeted, what kind of explanatory variables should be generated, and what kind of model should be used for estimation.

[0018] The present invention has been made to solve these problems, and aims to provide a financial statement fraud estimation device, a financial statement fraud estimation method, and a program that estimate the presence or absence of financial statement fraud based on text information disclosed by a company or its management.

[0019] In one embodiment of the present invention, a financial statement fraud estimation device includes a text information analysis unit that analyzes text information disclosed by a company or its management to generate one or more explanatory variables, a model storage unit that stores a trained model or a statistical model that indicates a correlation between the explanatory variables and the presence or absence of financial statement fraud related to the company, and a fraud estimation unit that inputs the explanatory variables generated by the text information analysis unit into the trained model or the statistical model and estimates the presence or absence of financial statement fraud related to the company, and includes a numerical ratio of the text information as the explanatory variable. In one embodiment of the present invention, the financial statement fraud estimation device further includes the expressiveness of the text information as the explanatory variable. In one embodiment of the present invention, the financial statement fraud estimation device further includes a kanji content rate or a grade level of the text information as the explanatory variable. In one embodiment of the present invention, the financial statement fraud estimation device further includes a number of words in the text information as the explanatory variable. In one embodiment of the present invention, the financial statement fraud estimation device further includes, as the explanatory variables, expressiveness, kanji content or grade level, and number of words of the text information. In one embodiment of the present invention, the financial statement fraud estimation device further includes a word diversity and a tone of the text information as the explanatory variables. In one embodiment of the present invention, the financial statement fraud estimation device further includes a machine learning unit that inputs the explanatory variables and the presence or absence of financial statement fraud related to the company to learn the correlation between the two and generate the trained model, and the machine learning unit performs the learning by ensemble learning. In one embodiment of the present invention, the machine learning unit performs the learning using a Gradient Boosting Decision Tree. In one embodiment of the present invention, the machine learning unit performs the learning using a random forest. In one embodiment of the present invention, the financial statement fraud estimation device uses MD&A (Management's Discussion and Analysis of Financial Condition and Results of Operations) as the text information. In one embodiment of the present invention, a financial statement fraud estimation method is a financial statement fraud estimation method executed on a computer, and includes a text information analysis step of analyzing text information disclosed by a company or its management to generate one or more explanatory variables, and a fraud estimation step of inputting the explanatory variables generated in the text information analysis step into a trained model or statistical model that indicates a correlation between the explanatory variables and the presence or absence of financial statement fraud related to the company, and estimating the presence or absence of financial statement fraud related to the company, wherein the explanatory variables include a numerical rate. In one embodiment of the present invention, a program causes a computer to carry out a method. Effect of the Invention

[0020] The present invention can provide a financial statement fraud estimation device, a financial statement fraud estimation method, and a program that estimate the presence or absence of financial statement fraud based on text information disclosed by a company or its management. [Brief description of the drawings]

[0021] [Figure 1] This is a graph showing the number of companies that disclosed inappropriate accounting and bookkeeping practices. [Diagram 2] 1 is a block diagram showing a schematic hardware configuration of a financial statement fraud estimation device 1. FIG. [Diagram 3] 1 is a block diagram showing a schematic functional configuration of a financial statement fraud estimation device 1. FIG. [Figure 4] 1 is a table showing the complexity (grade level) of text disclosed by fraudulent and non-fraudulent companies. [Diagram 5] This is a table showing the results of the t-test for 25 fraudulent and non-fraudulent companies. [Figure 6]1 is a flowchart showing an example of the operation of the financial statement fraud estimation device 1. [Figure 7] 1 is a table showing six explanatory variables used in the experiment. [Figure 8] FIG. 1 shows an ROC curve. [Figure 9] FIG. 1 shows AUROC. [Figure 10] This is a table showing whether there is a significant difference between the fraudulent company group and the non-fraudulent company group for the six explanatory variables used in the experiment. [Figure 11] This figure shows the evaluation indexes of the trained model when six explanatory variables (Kanji content rate, word diversity, number rate, number of words, tone, and expressiveness) are used. [Figure 12A] 13 is a graph showing the average absolute value of the SHAP value when a decision tree is used. [Figure 12B] 13 is a graph showing the distribution of SHAP values ​​when a decision tree is used. [Figure 12C] 13 is a graph showing the distribution of SHAP values ​​when a decision tree is used. [Figure 12D] 13 is a graph showing the distribution of SHAP values ​​when a decision tree is used. [Figure 12E] 13 is a graph showing the distribution of SHAP values ​​when a decision tree is used. [Figure 12F] 13 is a graph showing the distribution of SHAP values ​​when a decision tree is used. [Figure 12G] 13 is a graph showing the distribution of SHAP values ​​when a decision tree is used. [Figure 13A] This is a graph showing the average absolute value of the SHAP value when random forest is used. [Figure 13B] 13 is a graph showing the distribution of SHAP values ​​when random forest is used. [Figure 13C] 13 is a graph showing the distribution of SHAP values ​​when random forest is used. [Figure 13D] 13 is a graph showing the distribution of SHAP values ​​when random forest is used. [Figure 13E]13 is a graph showing the distribution of SHAP values ​​when random forest is used. [Figure 13F] 13 is a graph showing the distribution of SHAP values ​​when random forest is used. [Figure 13G] 13 is a graph showing the distribution of SHAP values ​​when random forest is used. [Figure 14A] 13 is a graph showing the average absolute value of the SHAP value when XGBoost is used. [Figure 14B] 13 is a graph showing the distribution of SHAP values ​​when XGBoost is used. [Figure 14C] 13 is a graph showing the distribution of SHAP values ​​when XGBoost is used. [Figure 14D] 13 is a graph showing the distribution of SHAP values ​​when XGBoost is used. [Figure 14E] 13 is a graph showing the distribution of SHAP values ​​when XGBoost is used. [Figure 14F] 13 is a graph showing the distribution of SHAP values ​​when XGBoost is used. [Figure 14G] 13 is a graph showing the distribution of SHAP values ​​when XGBoost is used. [Figure 15A] 13 is a graph showing the average absolute value of the SHAP value when LightGBM is used. [Figure 15B] 13 is a graph showing the distribution of SHAP values ​​when LightGBM is used. [Figure 15C] 13 is a graph showing the distribution of SHAP values ​​when LightGBM is used. [Figure 15D] 13 is a graph showing the distribution of SHAP values ​​when LightGBM is used. [Figure 15E] 13 is a graph showing the distribution of SHAP values ​​when LightGBM is used. [Figure 15F] 13 is a graph showing the distribution of SHAP values ​​when LightGBM is used. [Figure 15G] 13 is a graph showing the distribution of SHAP values ​​when LightGBM is used. [Figure 16A] 1 is a graph showing the average absolute value of the SHAP value when CatBoost is used. [Figure 16B] 1 is a graph showing the distribution of SHAP values ​​when CatBoost is used. [Figure 16C] 1 is a graph showing the distribution of SHAP values ​​when CatBoost is used. [Figure 16D] 1 is a graph showing the distribution of SHAP values ​​when CatBoost is used. [Figure 16E] 1 is a graph showing the distribution of SHAP values ​​when CatBoost is used. [Figure 16F] 1 is a graph showing the distribution of SHAP values ​​when CatBoost is used. [Figure 16G] 1 is a graph showing the distribution of SHAP values ​​when CatBoost is used. [Figure 17] FIG. 13 is a diagram showing the evaluation indexes of a trained model when four explanatory variables (Kanji content rate, number rate, number of words, and expressiveness) are used. [Figure 18] FIG. 13 is a diagram showing the evaluation indexes of a trained model when two explanatory variables (number rate, number of words) are used. [Figure 19] FIG. 13 is a diagram showing the evaluation indexes of a trained model when two explanatory variables (number rate, kanji content rate) are used. [Figure 20] FIG. 13 is a diagram showing the evaluation indexes of a trained model when two explanatory variables (number rate and expressiveness) are used. [Figure 21] FIG. 13 is a diagram showing the evaluation index of a trained model when only one explanatory variable (number rate) is used. [Figure 22] FIG. 13 is a diagram showing a histogram of sample data for verifying the effectiveness of the numeric rate. [Figure 23A] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 23B] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 23C]FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 24D] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 24A] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 24B] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 24C] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 24D] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 24E] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 25A] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 25B] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 25C] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 25D] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. [Figure 25E] FIG. 13 is a diagram showing the results of a comparison experiment to verify the effectiveness of the numeric rate. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The financial statement fraud estimation device 1 according to the embodiment of the present invention is an information processing device that estimates the presence or absence of fraud regarding a company's financial statements. The financial statements referred to here include consolidated financial statements in the case of a consolidated company and individual financial statements in the case of a single company without consolidated subsidiaries. Financial statement fraud is defined as "intentional fraud committed by management that undermines the confidence of investors and creditors through misleading financial statements" (Elliott and Willingham 1980) and "intentional misrepresentation of a company's financial results, achieved by omitting or intentionally misrepresenting disclosures and amounts in financial statements with the purpose of deceiving financial statement users such as investors and creditors" (Ravisankar et al. 2011; ELBannay et al. 2021, 650). In this embodiment, it is defined as intentional omission or misrepresentation of disclosures and amounts in financial statements with the purpose of misleading financial statement users such as investors and creditors through the involvement or instruction of a company or management. This includes fraud reflected on financial statements, such as fictitious posting of sales and profits that result in figures different from the actual amounts on the financial statements due to circular transactions, etc. However, financial statement fraud does not include fraud not reflected on financial statements, such as fraud by managers or individual employees, or fraud by subsidiaries (consolidated subsidiaries) and affiliated companies.

[0023] Figure 1 shows data compiled by Tokyo Shoko Research based on company disclosures and published materials from the Financial Services Agency, Tokyo Stock Exchange, etc., targeting listed companies and companies that submit securities reports, collating data on the number of companies whose previous year's financial statements have been affected by "inappropriate accounting and bookkeeping" or that have disclosed the possibility of being affected in the future (Tokyo Shoko Research 2021. Prepared by the applicant based on the survey "Data on companies disclosing inappropriate accounting and bookkeeping" for all listed companies in 2021).

[0024] In Figure 1, the parties are classified into four categories: the company, subsidiaries / affiliates, employees, and executives. Among these, cases in which a company is a party refer to "company-wide accounting fraud" and indicate financial statement fraud as defined above (corresponding to the component shown in the darkest color in Figure 1). This is because only company-wide accounting fraud is reflected in financial statement fraud, and managers create and disclose text information in order to conceal this intentional financial statement fraud.

[0025] 2 is a block diagram showing a schematic hardware configuration of the financial statement fraud estimation device 1. The financial statement fraud estimation device 1 is an information processing device having a CPU 11, a volatile memory 13, a non-volatile memory 14, an interface 15, a bus 20, and an input / output device 70.

[0026] A CPU 11 (Central Processing Unit) reads out a program stored in a non-volatile memory 14 via a bus 20, and executes information processing in accordance with the program to realize specific functions.

[0027] The non-volatile memory 14 is a storage device, such as a hard disk or SSD, whose memory state is maintained regardless of the power state of the financial statement fraud estimation device 1. In general, the programs and data stored in the non-volatile memory 14 are expanded in the volatile memory 13 when the programs are executed.

[0028] The volatile memory 13 is a storage device that stores programs and data expanded from the non-volatile memory 14, temporary calculation data, data input or output via the input / output device 70, and the like.

[0029] The input / output device 70 includes a data output device such as a display, a data input device such as a keyboard and a pointing device, a communication interface for controlling communication with the outside, etc. Display data output from the CPU 11 is displayed on the display via the interface 15. Commands and data input from the keyboard are passed to the CPU 11 via the interface 15. The communication interface obtains transmission data output by the CPU 11 via the interface 15 and outputs it to the outside. The communication interface also obtains received data from the outside and passes it to the CPU 11 via the interface 15.

[0030] 3 is a block diagram showing a schematic functional configuration of a financial statement fraud estimation device 1 according to an embodiment of the present invention. The financial statement fraud estimation device 1 includes a text information input unit 101, a text information analysis unit 102, and a fraud estimation unit 103. In addition, the device includes a machine learning unit 104 as necessary.

[0031] The text information input unit 101 acquires text information disclosed by a company or its manager. Examples of text information are shown below, but the present invention is not limited to these examples.

[0032] A particularly suitable textual information is MD&A (management's discussion and analysis of financial condition and results of operations). This textual information disclosed by management in the "Management's Discussion and Analysis of Financial Condition and Results of Operations" section of the securities report includes an analysis of factors that have a significant impact on business results, and an explanation of financial sources related to capital finance and liquidity.

[0033] The reason why we focus on MD&A disclosure is that, first, the Financial Services Agency has issued guidelines on the contents, it is information that listed companies are required to disclose, and many people read it together with financial statements. Second, although MD&A disclosure is not currently subject to external audits, MD&A disclosure provides an "insider's perspective" on the financial performance of listed companies, and MD&A disclosure is very important information in evaluating not only the honesty of management and its consistency with financial information, but also the "tone at the top." Third, the person who prepared the MD&A (management) may have known that financial reporting fraud was reflected in the MD&A, and was under some kind of stress, which reflected its impact on the text. Therefore, it is meaningful to focus on MD&A disclosure.

[0034] In addition, MD&A has the advantage that its format is predetermined, making it easy to collect and process information, and that it is suitable for routine processing because it leaves little room for discretion.

[0035] Alternatively, the text information may be a CEO letter. A CEO letter is a message to investors that is disclosed in electronic media such as annual reports and corporate websites, or in paper media such as shareholder communications, and often contains information similar to MD&A. CEO letters have greater room for discretion than MD&A, and therefore tend to more easily reflect the thoughts, attitudes, and values ​​of management.

[0036] Alternatively, the text information may be risk information listed in a securities report.

[0037] The text information analysis unit 102 analyzes the text information input to the text information input unit 101, and generates one or more explanatory variables. Examples of explanatory variables are shown below, but the present invention is not limited to these.

[0038] Managers who commit financial statement fraud strategically attempt to conceal the financial statement fraud in text information with the intention of hiding it from investors. There are three theories to expose such strategies of managers to conceal financial statement fraud: the Obfuscation Hypothesis, the Information Manipulation Theory, and the Interpersonal Deception Theory. In this embodiment, several explanatory variables are defined based on these theories.

[0039] (1) Obfuscation hypothesis Linguistic complexity can be broken down into two components: information volume and obfuscation. Of these, information volume is negatively correlated with information asymmetry, while obfuscation is positively correlated with information asymmetry. According to one study, loss-making companies tend to make disclosures that increase the volume of information while making sentences obfuscated to conceal poor performance. The obfuscation hypothesis is a hypothesis that managers mislead investors by concealing bad news or presenting more positive information. Regarding the volume of information, this embodiment does not take it into consideration because the purpose is to detect financial statement fraud involving a company or managers, and another study has demonstrated that bad news itself is difficult to read. In this embodiment, we focus only on obfuscation, which is thought to be easier for managers to exercise discretion.

[0040] Fraudulent companies have a higher text complexity (grade level) than non-fraudulent companies, making the text more difficult to understand in order to mislead investors. For example, it has been found that fraudulent annual reports contain a greater number of difficult words and complex syntax. Fraudulent companies increase the complexity of their statements, making the content more difficult to understand. Fraudulent annual reports have a higher vocabulary diversity than non-fraudulent annual reports, making them more difficult to read. Therefore, we set the following hypothesis: H1a: "Management of fraudulent companies make their text more difficult to understand in order to mislead investors, so the content has a higher content of kanji and a greater variety of words."

[0041] Based on the hypothesis H1a, the text information analysis unit 102 can generate the following explanatory variables: Kanji content rate: The number of kanji characters appearing in the text information divided by the sum of the number of katakana characters, the number of hiragana characters, and the number of kanji characters. · Word diversity: The number of different word types that appear in a text divided by the number of words, where the number of different word types refers to the number of words excluding duplicates. Grade: Grade, which indicates the complexity of a text, is an index showing "where a text is located in the distribution of difficulty levels of Japanese texts in the world" (Sato 2008). Based on Sato (2011), the OBI software measures the complexity by dividing the relative difficulty level based on a balanced corpus into 13 levels corresponding to the grade levels in Japanese schools. In other words, the grade level is displayed as one of 13 levels, ranging from first grade of elementary school (complexity level 1) to third grade of high school (complexity level 12) and university (complexity level 13) (Sato 2008).

[0042] Figure 4 shows the results of measuring grade, an indicator of complexity, for the texts of fraudulent and non-fraudulent companies. Nakashima (2023)'s study suggests that the complexity of the texts of fraudulent companies is higher than that of non-fraudulent companies. Grade is thought to be highly correlated with the kanji content rate and is included in the complexity hypothesis. In other words, grade can be used as an explanatory variable as a substitute for the kanji content rate.

[0043] (2) Information manipulation theory According to information manipulation theory, it is possible to use part or all of information disclosure by manipulating the amount (control of information amount), quality (distortion of information), method (presentation in an ambiguous style), and relevance (irrelevant information) of information presented for the purpose of misleading the recipient. According to Nakashima et al. (2022), a study that examined readability as a fraud detection study, no significant difference was observed in length (number of characters) between fraudulent and non-fraudulent companies. It can be expected that fraudulent companies have false figures in their financial statements and conceal these false figures, especially the numbers and number of words. Therefore, we set the hypothesis H1b: "Fraudulent company managers are unable to show false figures and conceal them, so the number rate and number of words will be lower than non-fraudulent companies."

[0044] Based on hypothesis H1b, the text information analysis unit 102 can generate the following explanatory variables: - Number ratio: The ratio of numbers to all characters appearing in the text information. · Word count: The number of words when the text information is divided into words.

[0045] (3) Interpersonal Deception Theory According to one study, fraudulent MD&As contain, on average, three times more positive sentiment and four times more negative sentiment than authentic MD&As, suggesting that both positive and negative sentiment are more prominent in fraudulent MD&As. Fraudulent companies are more likely to engage in impression management and deceptively portray their companies in a more positive light. In fact, fraudulent company executives have been found to convey false impressions by exaggerating positive news and minimizing or concealing negative news.

[0046] According to interpersonal deception theory, fraudsters attempt to diffuse responsibility by reducing specificity and using generic terms to add ambiguity to documents. Therefore, it can be predicted that fraudulent corporate managers will make their MD&A vague and less expressive in order to avoid being held accountable. Therefore, we set hypothesis H1c: "Fraudulent corporate managers will use fewer positive terms, more negative terms, and less expressive."

[0047] Based on hypothesis H1c, the text information analysis unit 102 can generate the following explanatory variables: Tone: The number of positive terms minus the number of negative terms appearing in the text information is divided by the sum of the number of positive terms and the number of negative terms. The number of positive and negative terms can be measured by the following procedure. First, the Japanese sentence as the text information is cut into words, and then the parts of speech are determined. Then, a dictionary is referenced to calculate how many words in the sentence correspond to positive and negative terms. The dictionary can use a publicly known word sentiment polarity correspondence table. · Expressiveness: The sum of the number of adjectives and adverbs divided by the sum of the number of nouns and verbs.

[0048] The fraud estimation unit 103 acquires one or more explanatory variables generated by the text information analysis unit 102, and inputs them to an estimation model stored in the model storage unit 1031. The estimation model is a trained model 10311 or a statistical model 10312 that indicates a correlation between the explanatory variables and the presence or absence of financial statement fraud. When the explanatory variables are input, the estimation model outputs a response variable that indicates the presence or absence of financial statement fraud in light of a predefined correlation. In this embodiment, the response variable is set to two values ​​(whether or not there is fraud). However, the present invention is not limited to this, and for example, a numerical value indicating the possibility of fraud in financial statements may be used as the response variable.

[0049] When the fraud estimation unit 103 performs estimation using the statistical model 10312, it can employ a known statistical analysis method such as logistic regression analysis.

[0050] On the other hand, when making an estimation using the trained model 10311, the machine learning unit 104 typically needs to generate the trained model in advance.

[0051] The machine learning unit 104 learns the correlation between one or more explanatory variables generated by the text information analysis unit 102 and a target variable indicating the presence or absence of financial statement fraud according to an arbitrary machine learning algorithm. The machine learning unit 104 repeatedly inputs a large amount of learning data (a set of one or more explanatory variables generated by the text information analysis unit 102 and a target variable indicating the presence or absence of financial statement fraud) and repeatedly executes the learning process. By repeatedly executing the learning process in this manner, a trained model indicating the correlation between one or more explanatory variables and a target variable indicating the presence or absence of financial statement fraud is constructed. The correlation indicated by the trained model gradually becomes more reliable as the learning process progresses. When a trained model with a sufficiently reliable level is constructed, the trained model can be used to estimate the value of the target variable corresponding to one or more explanatory variables.

[0052] The fraud estimation unit 103 can automatically and accurately determine the objective variable corresponding to one or more explanatory variables, i.e., the presence or absence of financial statement fraud, based on the trained model constructed by the machine learning unit 104. That is, the fraud estimation unit 103 provides one or more explanatory variables generated by the text information analysis unit 102 as input to the trained model, so that the trained model automatically and accurately outputs the objective variable indicating the presence or absence of financial statement fraud.

[0053] In this embodiment, an ensemble-type learning method that is often used in competitions and the like is adopted as the machine learning method. Specifically, random forest, XGBoost, LightGBM, and CatBoost are listed. In addition, a decision tree analysis was also performed for comparison with these methods. Hereinafter, the explanation will be given on the assumption that the objective variable is a binary value of 0 or 1.

[0054] (1) Decision tree analysis (classification) This is a general term for analysis by supervised machine learning using a single tree structure. Compared to the other methods used in this embodiment (Random Forest, XGBoost, LightGBM, CatBoost), the accuracy of the model obtained tends to be inferior. However, it is superior to other methods in terms of visualization of the tree structure of the model.

[0055] In this embodiment, we used CART (Classification and Regression Trees), which is a type of decision tree analysis method that is implemented as standard in scikit-learn. CART is a method of analysis using a tree (binary tree) that divides data into two at each branch point.

[0056] (2) Random Forest It is a type of ensemble learning method. Some data is randomly extracted from the original training data, allowing overlaps, to construct new training data. Decision tree analysis is then performed based on the new training data constructed. By performing the above operations in parallel, multiple models (weak learners) are obtained. When estimating the objective variable, a majority vote is performed based on the output values ​​obtained from multiple weak learners. The estimated result output by the greatest number of weak learners is used as the final output. This method is expected to reduce the risk of overfitting compared to a single model.

[0057] (3) XGBoost The methods that follow are also ensemble-type learning methods, just like random forest. They are also similar to random forest in that each weak learner has a tree structure. However, the methods of constructing (deciding) and aggregating the weak learners are significantly different. While the purpose of random forest was to suppress overfitting, the main purpose of the methods that follow is to improve prediction accuracy. On the other hand, the methods that will be explained below are more prone to overfitting than decision tree analysis or random forest.

[0058] XGBoost estimates the probability that the objective variable is 1 by using the sum of the estimates of each weak learner (strictly speaking, the sum of the products of a hyperparameter called the learning rate and the estimates of each learner is used, but in this explanation, we will assume that the learning rate is always 1). Each learner is constructed sequentially. The first weak learner estimates the probability that the objective variable is 1, while the nth (n is an integer equal to or greater than 2) weak learner estimates the error between the sum of the estimates of the weak learners up to the (n-1)th and the true value. The information gain is used as the criterion for splitting the leaves in each weak learner with a tree structure. However, the information gain used is different from the entropy and Gini coefficient that are often used in decision tree analysis. Although we will not go into details here, we use information gain that minimizes the error with the true value. The order in which the leaf split is considered follows level-wise tree growth. In other words, after all the leaves at the same depth have been considered for splitting, we move on to considering splitting the branch one level deeper.

[0059] (4)LightGBM LightGBM is a method similar to XGBoost in terms of learning algorithm. It is the same as XGBoost in that it estimates the probability that the objective variable is 1 by the sum of the estimates of weak learners with a tree structure. The main difference with XGBoost is in the method of constructing each regression tree.

[0060] The first difference is that each feature is histogrammed. Histogramming eliminates the need to search for strict branching, making it possible to reduce computational costs. The second difference is that the tree structure is determined by "leaf-wise tree growth" rather than "level-wise tree growth". The difference between the two is the order in which leaf splits are considered. Note that the same tree is constructed when the tree is fully grown. Leaf-wise tree growth is a construction method that extends the branches in order starting from the branch with the greatest information gain. When leaf-wise tree growth is used, a pruning algorithm called "pre-pruning" can be applied. Pre-pruning is an algorithm that branches if further branching will reduce the prediction error, and stops tree growth if it does not. Pre-pruning may be less accurate than post-pruning, but it can reduce computational costs. Note that post-pruning is a pruning method that selects the state with the smallest prediction error after fully growing the decision tree.

[0061] Due to histogramming, leaf-wise tree growth, and pre-pruning, the computational cost of lightGBM tends to be "lighter" than XGBoost. However, decision trees generated by leaf-wise tree growth tend to become complicated, so care must be taken to avoid overfitting.

[0062] (5) CatBoost Like XGBoost and LightGBM, CatBoost is a method that estimates the probability that the objective variable is 1 by summing up the estimated values ​​of weak learners with a tree structure. It is a type of Gradient Boosting Decision Tree. When constructing each weak learner, data randomly sampled from the training data is used, so it is considered to be a method that is less susceptible to overfitting than XGBoost and LightGBM.

[0063] <Experiment> Using the financial statement fraud estimation device 1 according to the present embodiment, an experiment was conducted to estimate the presence or absence of financial statement fraud for text information disclosed by an existing company. The method and results are shown below.

[0064] (1) Sample selection and data The companies analyzed were 2,998 companies that applied Japanese standards, excluding 165 companies in the banking and insurance industries and 320 companies that applied US standards, IFRS standards, and other standards, out of 3,483 companies listed on the Japan Exchange. The fraudulent companies were 73 companies that were found to have inappropriate accounting in 2019 in the "Listed Companies that Disclosed Inappropriate Accounting and Finance in 2020" (hereinafter referred to as "inappropriate accounting") by Tokyo Shoko Research Co., Ltd., and included 31 companies with company-wide fraud, 12 companies with employee-related fraud, 3 companies with individual fraud by senior management, and 27 companies with subsidiary-related fraud. Of the 31 companies with company-wide fraud, 6 were delisted companies, and the final sample was 25 companies. A group of companies with similar industries, total assets, and sales was paired as a non-fraudulent company. Figure 5 shows the t-test results for the 25 fraudulent companies and the 25 non-fraudulent companies. Figure 5 shows that there is no significant difference between the total assets and sales of fraudulent and non-fraudulent companies, making paired samples appropriate.

[0065] The MD&A disclosure data for the fraudulent and non-fraudulent companies includes MD&A for the seven years from 2013 to 2019. The analysis is based on MD&A disclosure for the past seven years because, in the cases of Toshiba and Olympus, companies were involved in fraud for a minimum of seven years prior to the year in which it was discovered.

[0066] (2) Analysis method (2-1) Trial of financial statement fraud estimation using machine learning According to the flowchart shown in Fig. 6, an attempt was made to generate a fraud detection model by machine learning and to estimate financial statement fraud using the model, using the financial statement estimation device 1. The machine learning methods used were decision tree, random forest, XGBoost, LightGBM, and CatBoost.

[0067] (2-1-1) Learning Phase S101: Acquisition of text information The text information input unit 101 acquires text information disclosed by a company or its management. In this experiment, MD&As for the 50 companies shown in (1) Sample Selection and Data for six years from 2013 to 2018 were collected and input as text information for learning.

[0068] S102: Generation of explanatory variables The text information analysis unit 102 analyzes the text information input to the text information input unit 101, and generates explanatory variables. In this experiment, six explanatory variables shown in FIG.

[0069] S103: Generation of trained models The machine learning unit 104 repeatedly inputs a set of the six explanatory variables generated in S102 and a target variable (binary) indicating the presence or absence of financial statement fraud as learning data, and generates a trained model. In this experiment, five types of machine learning methods, namely, decision tree, random forest, XGBoost, LightGBM, and CatBoost, were adopted, and trained models were generated for each.

[0070] (2-1-2) Estimation phase S104: Acquisition of text information The text information input unit 101 acquires text information disclosed by a company or its management. In this experiment, the MD&A for fiscal year 2019 of the 50 companies shown in (1) Sample Selection and Data was collected and input as text information for estimation.

[0071] S105: Generation of explanatory variables The text information analysis unit 102 analyzes the text information input to the text information input unit 101, and generates explanatory variables. In this experiment, six explanatory variables shown in FIG.

[0072] S106: Presumption of Financial Statement Fraud The fraud estimation unit 103 inputs the six explanatory variables generated in S105 into the trained model generated in S103, and outputs a target variable (binary value) indicating the presence or absence of financial statement fraud as an estimation result.

[0073] (2-2) Evaluation To evaluate the effectiveness of classifiers generated by multiple machine learning methods (decision tree, random forest, XGBoost, LightGBM, CatBoost), we calculate Accuracy, Precision, Recall, F-measure, and AUROC. Accuracy is the total number of correctly classified MD&A disclosures divided by the total number of analyzed MD&A disclosures. Precision is the ratio of the number of MD&A disclosures correctly classified as fraudulent to the total number of MD&A disclosures classified as fraudulent. Recall is the ratio of the number of MD&A disclosures correctly classified as fraudulent to the total number of actual fraudulent MD&A disclosures. Recall is also called True Positive Rate. In the following, True Positive Rate is abbreviated as TPR. In contrast, the ratio of the number of MD&A disclosures incorrectly classified as fraudulent to the total number of actual non-fraudulent MD&A disclosures is called False Positive Rate. In the following, False Positive Rate is abbreviated as FPR.

[0074] The F measure combines Precision and Recall into a single metric using a weighted harmonic mean of each, with a higher value indicating better estimation performance (Humphery et al. 2011, 591).

[0075] To explain AUROC, we will explain the estimation method of the machine learning models used in this experiment. In this experiment, the machine learning models first calculate the probability of cheating. A threshold (usually 0.5) is set in advance, and if the probability is above the threshold, it is classified as cheating. The lower the threshold, the stronger the tendency for the machine learning models to classify it as cheating. As a result, the TPR increases, but the FPR also increases. Conversely, the lower the threshold, the lower the FPR, but the TPR also decreases. A graph showing the relationship between TPR and FPR when the threshold is changed between 0 and 1 is called an ROC curve (Figure 8, curve a). When random classification (for example, classification where a random number between 0 and 1 is used as the probability of cheating) is performed, the ROC curve will be like the line b in Figure 8. If there is a threshold where TPR = 1 and FPR = 0, by performing classification at that threshold, ideal classification with Accuracy, Precision, and Recall all being 1 can be performed. In an ideal case where such a threshold exists, an ROC curve like the line c in Figure 8 will be obtained.

[0076] AUROC (area under the ROC curve) is a measure of how close the ROC curve is to the ideal ROC curve (line c in Figure 8). AUROC is given by the area of ​​area d (the area between curve b and TPR=0) shown in Figure 9. The closer the ROC curve is to line c in Figures 8 and 9, the larger the value of the AUROC curve becomes, and the AUROC is 1 when ideal classification is performed. In other words, the closer the value of the AUROC curve is to 1, the higher the estimation performance of the machine learning model is suggested. In addition, when random classification is performed, the AUROC is 0.5. In other words, if the AUROC exceeds 0.5, it is suggested that the machine learning model has a higher estimation ability than random classification.

[0077] (3)Analysis results (3-1) Statistical test results We used the Mann-Whitney U test (significance level 5%) to check whether there were significant differences between the fraudulent and non-fraudulent company groups for the six explanatory variables used in this experiment. The results are shown in Figure 10. Compared to non-fraudulent companies, fraudulent companies were found to have a higher kanji content rate, higher word diversity, lower number rate, lower word count, lower tone, and lower expressiveness. However, no significant difference was found in the number rate. However, an evaluation of the (3-2) model described below revealed that the number rate strongly contributed to the estimation of the machine learning model in this experiment.

[0078] (3-2) Model evaluation In this experiment, the data from 2013 to 2018 was used as training data, and the data from 2019 was used as test data. In addition, the six explanatory variables shown in Figure 7 were used as explanatory variables.

[0079] The evaluation indices (Accuracy, Precision, Recall, F measure, AUROC) of the fraud detection models using decision tree, random forest, XGBoost, LightGBM, and CatBoost are shown in Figure 11. The values ​​of each evaluation index were calculated for the test data.

[0080] The SHAP values ​​of these models are shown in Figures 12 to 16, respectively.

[0081] 12A to 12G are diagrams showing the SHAP values ​​of a model generated using a decision tree. FIG. 12A is a graph comparing the average absolute values ​​of the SHAP values ​​of six explanatory variables, "number rate," "number of words," "kanji content rate," "expressiveness," "word diversity," and "tone." FIG. 12B to 12F are graphs showing the distribution of SHAP values ​​when the above six explanatory variables are used. FIG. 12B shows the distribution of SHAP values ​​for the standardized "number rate," FIG. 12C shows the distribution of SHAP values ​​for the standardized "number of words," FIG. 12D shows the distribution of SHAP values ​​for the standardized "kanji content rate," FIG. 12E shows the distribution of SHAP values ​​for the standardized "expressiveness," FIG. 12F shows the distribution of SHAP values ​​for the standardized "word diversity," and FIG. 12G shows the distribution of SHAP values ​​for the standardized "tone."

[0082] FIG. 13A is a diagram showing the SHAP value of a model generated using a random forest. FIG. 13A is a graph comparing the average absolute values ​​of the SHAP values ​​of six explanatory variables, "number rate," "number of words," "kanji content rate," "expressiveness," "word diversity," and "tone." FIG. 13B to FIG. 13F are graphs showing the distribution of SHAP values ​​when the above six explanatory variables are used, respectively. FIG. 13B shows the distribution of SHAP values ​​for the standardized "number rate," FIG. 13C shows the distribution of SHAP values ​​for the standardized "number of words," FIG. 13D shows the distribution of SHAP values ​​for the standardized "kanji content rate," FIG. 13E shows the distribution of SHAP values ​​for the standardized "expressiveness," FIG. 13F shows the distribution of SHAP values ​​for the standardized "word diversity," and FIG. 13G shows the distribution of SHAP values ​​for the standardized "tone."

[0083] FIG. 14A is a diagram showing the SHAP value of a model generated using XGBoost. FIG. 14A is a graph comparing the average absolute values ​​of the SHAP values ​​of six explanatory variables, "number rate," "number of words," "kanji content rate," "expressiveness," "word diversity," and "tone." FIG. 14B to FIG. 14F are graphs showing the distribution of SHAP values ​​when the above six explanatory variables are used, respectively. FIG. 14B shows the distribution of SHAP values ​​for the standardized "number rate," FIG. 14C shows the distribution of SHAP values ​​for the standardized "number of words," FIG. 14D shows the distribution of SHAP values ​​for the standardized "kanji content rate," FIG. 14E shows the distribution of SHAP values ​​for the standardized "expressiveness," FIG. 14F shows the distribution of SHAP values ​​for the standardized "word diversity," and FIG. 14G shows the distribution of SHAP values ​​for the standardized "tone."

[0084] FIG. 15A is a diagram showing the SHAP value of a model generated using LightGBM. FIG. 15A is a graph comparing the average absolute values ​​of the SHAP values ​​of six explanatory variables, "number rate," "number of words," "kanji content rate," "expressiveness," "word diversity," and "tone." FIG. 15B to FIG. 15F are graphs showing the distribution of SHAP values ​​when the above six explanatory variables are used, respectively. FIG. 15B shows the distribution of SHAP values ​​for the standardized "number rate," FIG. 15C shows the distribution of SHAP values ​​for the standardized "number of words," FIG. 15D shows the distribution of SHAP values ​​for the standardized "kanji content rate," FIG. 15E shows the distribution of SHAP values ​​for the standardized "expressiveness," FIG. 15F shows the distribution of SHAP values ​​for the standardized "word diversity," and FIG. 15G shows the distribution of SHAP values ​​for the standardized "tone."

[0085] FIG. 16A is a diagram showing the SHAP value of a model generated using CatBoost. FIG. 16A is a graph comparing the average absolute values ​​of the SHAP values ​​of six explanatory variables, "number rate," "number of words," "kanji content rate," "expressiveness," "word diversity," and "tone." FIG. 16B to FIG. 16F are graphs showing the distribution of SHAP values ​​when the above six explanatory variables are used, respectively. FIG. 16B shows the distribution of SHAP values ​​for the standardized "number rate," FIG. 16C shows the distribution of SHAP values ​​for the standardized "number of words," FIG. 16D shows the distribution of SHAP values ​​for the standardized "kanji content rate," FIG. 16E shows the distribution of SHAP values ​​for the standardized "expressiveness," FIG. 16F shows the distribution of SHAP values ​​for the standardized "word diversity," and FIG. 16G shows the distribution of SHAP values ​​for the standardized "tone."

[0086] The SHAP value indicates the degree of influence (degree of contribution) of each explanatory variable when each machine learning model outputs the objective variable. When explanatory variables are ranked based on the average absolute value of the SHAP value, "digit rate" was ranked first in all five models. On the other hand, "tone" and "word diversity" were ranked fourth or lower in all five models. Figure 17 is a table showing the evaluation indexes of the models when these explanatory variables ("tone" and "word diversity"), which are thought to have little contribution to the estimation, are excluded. As shown in Figure 17, the method excluding the decision tree showed estimation accuracy comparable to that of the six-variable method.

[0087] It is common knowledge for those skilled in the art that the SHAP value is useful as an index showing the contribution of explanatory variables to a machine learning model, as shown in, for example, Non-Patent Documents 31 and 32. Even if there are exceptional cases in which the contribution of an explanatory variable is low even when the SHAP value is high, there is no reasonable reason to infer that the present experiment is an exception.

[0088] Of the four explanatory variables excluding "tone" and "word diversity," we used "digit rate" and one other explanatory variable to estimate using only two explanatory variables. Even with a relatively small number of explanatory variables, we were able to achieve some estimation accuracy. Figure 18 shows the evaluation indexes of the model when only the "number rate" and "number of words" are used. Figure 19 shows the evaluation indexes of the model when only "number rate" and "kanji content rate" are used. Figure 20 shows the evaluation indexes of the model when only "number rate" and "expressiveness" are used. The evaluation when only "digit rate" was used as an explanatory variable was as shown in Figure 21.

[0089] First, in the estimation using machine learning, it was found that the contribution of the number rate among the explanatory variables is the highest, and a certain degree of estimation accuracy can be obtained even when only the number rate is used. In conventional financial statement fraud estimation methods, it was mainstream to generate many explanatory variables and use them for estimation. On the other hand, by using the number rate, which is a newly introduced explanatory variable in the present invention, it is possible to estimate financial statement fraud with a certain degree of accuracy with a single explanatory variable. In other words, by using the new concept of the number rate in the estimation process, it has become possible to ensure estimation accuracy while significantly reducing calculation resources compared to the past.

[0090] In addition, we found that by combining the number rate with at least one of the expressiveness, kanji content rate, and number of words, which have relatively high contributions, we could further improve the estimation accuracy compared to using only the number rate. On the other hand, we found that the contributions of tone and word diversity were relatively low.

[0091] Second, we compared the performance of fraud detection models (decision tree, random forest, XGBoost, LightGBM, and CatBoost) and found that random forest performed best in the six-variable case. However, CatBoost may perform best if its hyperparameters are tuned.

[0092] As described above, the statistical test shown in (3-1) and the machine learning model evaluation shown in (3-2) gave different results for the number rate. However, this result does not negate the usefulness of the number rate as an explanatory variable.

[0093] Figure 22 is a histogram showing the distribution of non-fraudulent companies and fraudulent companies in the sample data. Lighter colors indicate the number of non-fraudulent companies, and darker colors indicate the number of fraudulent companies. In this sample data, the numerical ratio is distributed in the range from 0 to 0.175 or less. The number of non-fraudulent companies and fraudulent companies in each range obtained by subdividing the numerical ratio range is distributed in the range from 0 to 35.

[0094] In this histogram, the trend of increase and decrease in the number rate for non-fraudulent companies is different from that for fraudulent companies. For non-fraudulent companies, a relatively clean distribution is seen, with the number rate increasing and then decreasing after peaking at 0.075. On the other hand, for fraudulent companies, the number rate increases and then decreases after peaking at 0.063, then increases again at 0.100, and then decreases again at 0.125, showing a distinctive trend.

[0095] Statistical tests compare the overall trends of increases and decreases in the two groups being compared. Therefore, local differences in trends of increases and decreases can be ignored. Even if one group has a distinctive and local trend of increases or decreases, as in this sample data, the difference is unlikely to appear as a significant difference.

[0096] On the other hand, machine learning estimates the number ratio more locally. For example, it is possible to make different estimates when the number ratio is 0.100 and when the number ratio is 0.113. Therefore, even when using such sample data, it is possible to make relatively accurate judgments.

[0097] In this experiment, although no significant differences were observed in statistical tests, it is believed that factors such as these are the reasons why a certain level of estimation accuracy was achieved when using the number rate generated by machine learning.

[0098] According to Figures 12B, 13B, 14B, 15B, and 16B, it can be seen that the higher the number rate of the sample data in this experiment, the lower the possibility of fraud, and the lower the number rate, the higher the possibility of fraud. It can also be seen that when the number rate is in the intermediate range, there is a mixture of possibilities of either fraud or non-fraud. This is a characteristic seen when the sample data has a distribution like the above-mentioned histogram. Data with such characteristics corresponds to a case where, even if a statistical test does not show a significant difference, estimation by machine learning can show a certain degree of accuracy.

[0099] In this way, even if a statistical test does not reveal a significant difference for a certain explanatory variable, machine learning may be able to provide a certain degree of estimation accuracy. As mentioned above, in this experiment, the AUROC value of the machine learning model when using the numerical rate exceeds 0.5, suggesting that it has a higher estimation ability than random classification. In addition, the numerical rate has the highest SHAP value, which indicates the degree of influence (degree of contribution) of each explanatory variable when the machine learning model outputs the objective variable. Therefore, it can be concluded that the machine learning model using the numerical rate as an explanatory variable as used in this experiment can estimate the presence or absence of financial statement fraud with a practical degree of accuracy.

[0100] <Comparative experiment 1> In order to further verify the usefulness of the numerical rate in estimating financial statement fraud, a comparative experiment was conducted between the AUROC value of the machine learning model when one or more other explanatory variables were used without using the numerical rate and the AUROC value when the numerical rate was used in combination (Comparative Experiment 1). The results of Comparative Experiment 1 are shown in Figure 23. Note that due to the randomness inherent in the machine learning algorithm used for the verification, there are some differences from the values ​​shown in Figures 17 to 22.

[0101] In Figures 23A to 23D, the "Variable List" shows the explanatory variables used by the machine learning model. This does not include numerical percentages.

[0102] In FIG. 23A, “RandomF_AUROC” represents the AUROC value of a model generated using a random forest and only the explanatory variables shown in the “Variable List” (excluding the digital rate). “RandomF_AUROCwith digital rate” also represents the AUROC value of a model generated using a random forest and the explanatory variables shown in the “Variable List” along with the digital rate.

[0103] In FIG. 23B, “XGBClas_AUROC” represents the AUROC value of a model generated using XGBoost and only the explanatory variables shown in the “Variable List” (excluding the digital rate). “XGBClas_AUROCwith digital rate” also represents the AUROC value of a model generated using XGBoost and the explanatory variables shown in the “Variable List” along with the digital rate.

[0104] In FIG. 23C, “LGBMCla_AUROC” represents the AUROC value of a model generated using LightGBM and only the explanatory variables shown in the “Variable List” (excluding the digital rate). “LGBMCla_AUROCwith digital rate” also represents the AUROC value of a model generated using LightGBM and the explanatory variables shown in the “Variable List” along with the digital rate.

[0105] In FIG. 23D, “<catboo_AUROC” represents the AUROC value of a model generated using CatBoost and only the explanatory variables shown in the “Variable List” (excluding the digital rate). “<catboo_AUROCwith digital rate” also represents the AUROC value of a model generated using CatBoost and the explanatory variables shown in the “Variable List” along with the digital rate.

[0106] In any of FIGS. 23A to 23D, it can be seen that adding the digital rate to other explanatory variables increases the AUROC value and improves the estimation accuracy.

[0107] <Comparative Experiment 2> For further verification, a comparative experiment (Comparative Experiment 2) was conducted to compare the AUROC values between a group including the digit ratio as an explanatory variable and a group not including the digit ratio. In Comparative Experiment 2, the number of explanatory variables in the two groups was made the same. The results of Comparative Experiment 2 are shown in FIGS. 24 and 25. Note that due to the randomness inherent in the machine learning algorithm used for verification, there are some differences from the values shown in FIGS. 17 to 22.

[0108] FIG. 24A is a list of AUROC values observed when using one explanatory variable. The "Variable List" shows the explanatory variables (6 patterns) used for generating the machine learning model. In this experiment, only one of the six explanatory variables, "tone", "number of words", "Chinese character content rate", "expressiveness", "digit ratio", and "diversity", was used. The machine learning methods used for model generation were four types: Random Forest (RandomF_AUROC), XGBoost (XGBClas_AUROC), LightGBM (LGBMCla_AUROC), and CatBoost (<catboo_AUROC). The AUROC values were measured for each of these combinations (24 patterns).

[0109] FIG. 25A is a table in which the six patterns listed in the "Variable List" of FIG. 24A were separated into a group including the digit ratio and a group not including the digit ratio, and the statistical quantities of the AUROC values in each group were calculated and compared. The group including the digit ratio has at least higher average and median values of the AUROC value compared to the group not including the digit ratio. From this, it can be evaluated that, as a general tendency, the group including the digit ratio has higher estimation accuracy.

[0110] Figure 24B shows a list of AUROC values observed when using two explanatory variables. The "Variable List" shows the combinations (15 patterns) of explanatory variables used to generate the machine learning model. In this experiment, out of the six explanatory variables, "tone", "number of words", "Chinese character content rate", "expressiveness", "digit rate", and "diversity", different combinations of two explanatory variables were used. The machine learning methods used for model generation are four types: Random Forest (RandomF_AUROC), XGBoost (XGBClas_AUROC), LightGBM (LGBMCla_AUROC), and CatBoost (<catboo_AUROC). The AUROC values were measured for each of these combinations (60 patterns).

[0111] Figure 25B is a table that separates the 15 patterns listed in the "Variable List" of Figure 24B into a group including the digit rate and a group not including it, and calculates and compares the statistics of the AUROC values in each group. The group including the digit rate has larger average value, median value, and maximum value of the AUROC value compared to the group not including the digit rate. From this, it can be evaluated that the group including the digit rate has higher estimation accuracy both as an overall trend and as peak performance.

[0112] Figure 24C shows a list of AUROC values observed when using three explanatory variables. The "Variable List" shows the combinations (20 patterns) of explanatory variables used to generate the machine learning model. In this experiment, out of the six explanatory variables, "tone", "number of words", "Chinese character content rate", "expressiveness", "digit rate", and "diversity", different combinations of three explanatory variables were used. The machine learning methods used for model generation are four types: Random Forest (RandomF_AUROC), XGBoost (XGBClas_AUROC), LightGBM (LGBMCla_AUROC), and CatBoost (<catboo_AUROC). The AUROC values were measured for each of these combinations (120 patterns).

[0113] Figure 25C is a table that classifies the 20 patterns listed in the "variable list" of Figure 24C into a group including the digit ratio and a group not including it, calculates and compares the statistical quantities of the AUROC values in each group. The group including the digit ratio has at least a higher average value and median value of the AUROC value compared to the group not including the digit ratio. From this, it can be evaluated that the group including the digit ratio has a higher estimation accuracy as a general tendency.

[0114] Figure 24D is a list of the AUROC values observed when using four explanatory variables. The "variable list" shows the combinations (15 patterns) of the explanatory variables used to generate the machine learning model. In this experiment, out of the six explanatory variables "tone", "number of words", "Chinese character content rate", "expressiveness", "digit ratio", and "diversity", different combinations of four explanatory variables were used. The machine learning methods used in model generation are four types: Random Forest (RandomF_AUROC), XGBoost (XGBClas_AUROC), LightGBM (LGBMCla_AUROC), and CatBoost (<catboo_AUROC). The AUROC values were measured for each of these combinations (60 patterns).

[0115] Figure 25D is a table that classifies the 15 patterns listed in the "variable list" of Figure 24D into a group including the digit ratio and a group not including it, calculates and compares the statistical quantities of the AUROC values in each group. The group including the digit ratio has larger values for all of the average value, median value, and maximum value of the AUROC value compared to the group not including the digit ratio. From this, it can be evaluated that the group including the digit ratio has a higher estimation accuracy both as a general tendency and as peak performance.

[0116] Figure 24E shows a list of AUROC values observed when using five explanatory variables. The "Variable List" indicates the combinations (6 patterns) of explanatory variables used to generate the machine learning model. In this experiment, out of the six explanatory variables "tone", "number of words", "Chinese character content rate", "expressiveness", "digit rate", and "diversity", different combinations of five explanatory variables were used. The machine learning methods used for model generation are four types: Random Forest (RandomF_AUROC), XGBoost (XGBClas_AUROC), LightGBM (LGBMCla_AUROC), and CatBoost (<catboo_AUROC). The AUROC values were measured for each of these combinations (24 patterns).

[0117] Figure 25E is a table that classifies the six patterns listed in the "Variable List" of Figure 24E into a group including the digit rate and a group not including it, and calculates and compares the statistical quantities of the AUROC values in each group. The group including the digit rate has larger average value, median value, and maximum value of the AUROC value compared to the group not including the digit rate. From this, it can be evaluated that the group including the digit rate has higher estimation accuracy both as an overall trend and as peak performance.

[0118] According to Comparative Experiments 1 and 2, in the machine learning model using explanatory variables including the digit rate, the presence or absence of financial statement fraud can be estimated with high accuracy and a certain degree of repeatability compared to the case without the digit rate.

[0119] According to this embodiment, the text information input unit 101 acquires text information such as MD&A and CEO letters disclosed by a company or its operator, the text information analysis unit 102 analyzes the text information to generate one or more explanatory variables including the digit rate, and the fraud estimation unit 103 constructs an estimation model by methods such as Random Forest and CatBoost. As a result, it becomes possible to perform highly accurate financial statement fraud estimation, thereby avoiding investment losses and risks of investors and improving the investment effect when investors invest in non-fraudulent companies.

[0120] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and can be modified as appropriate without departing from the spirit of the present invention. Any of the components of the embodiment can be modified or omitted within the scope of the present invention.

[0121] For example, while the above embodiment assumes that the text information is Japanese text, the present invention is not limited to this, and may target text in a foreign language such as English. In this case, for example, the average value of word length (the number of characters constituting a word) can be used as an explanatory variable instead of the kanji content rate. Explanatory variables such as word diversity, number rate, word count, tone, and expressiveness can be calculated in the same way as in the case of Japanese. Note that a dictionary that is referred to when calculating tone is one that has been prepared in advance for the target language.

[0122] Furthermore, the information processing of the present invention may be realized by hardware or by a CPU executing a computer program. The computer program may be provided to a computer by various types of non-transitory computer readable medium or transitory computer readable medium. [Explanation of symbols]

[0123] 1. Financial statement fraud estimation device 11 CPU 13 Volatile Memory 14 Non-volatile memory 15 Interface 16 Interface 20 Bus 70 Input / Output Devices 101 Text information input section 102 Text Information Analysis Department 103 Fraud Estimation Department 1031 Model Storage Unit 10311 Trained Models 10312 Statistical Model 104 Machine Learning Department

Claims

1. a text information analysis unit that analyzes text information disclosed by a company or its management to generate one or more explanatory variables; A model storage unit that stores a trained model or a statistical model showing a correlation between the explanatory variables and the presence or absence of financial statement fraud related to the company; A fraud estimation unit that inputs the explanatory variables generated by the text information analysis unit into the trained model and estimates whether or not there is financial statement fraud related to the company, The explanatory variable includes a numeric ratio of the text information. Financial statement fraud estimation device.

2. The explanatory variable further includes expressiveness of the text information. The financial statement fraud estimation device according to claim 1.

3. The explanatory variables further include a kanji content rate or a grade level of the text information. The financial statement fraud estimation device according to claim 1.

4. The explanatory variable further includes the number of words in the text information. The financial statement fraud estimation device according to claim 1.

5. The explanatory variables further include expressiveness, kanji content or grade level, and number of words of the text information. The financial statement fraud estimation device according to claim 1.

6. The explanatory variables further include vocabulary diversity and tone of the text information. The financial statement fraud estimation device according to claim 5.

7. Further comprising a machine learning unit that inputs the explanatory variables and the presence or absence of financial statement fraud related to the company, learns the correlation between the two, and generates the trained model; The machine learning unit performs the learning by ensemble learning. The financial statement fraud estimation device according to any one of claims 1 to 6.

8. The machine learning unit performs the learning using a gradient boosting decision tree. The financial statement fraud estimation device according to claim 7.

9. The machine learning unit performs the learning using a random forest. The financial statement fraud estimation device according to claim 7.

10. The text information used is Management's Discussion and Analysis of Financial Conditions and Results of Operations (MD&A). The financial statement fraud estimation device according to any one of claims 1 to 6.

11. A method for estimating financial statement fraud implemented on a computer, comprising: A text information analysis step of analyzing text information disclosed by a company or its management to generate one or more explanatory variables; a fraud estimation step of inputting the explanatory variables generated in the text information analysis step into a trained model indicating a correlation between the explanatory variables and the presence or absence of financial statement fraud in the company, and estimating the presence or absence of financial statement fraud in the company; Include the number rate as an explanatory variable Methods for estimating financial statement fraud.

12. A program for causing a computer to execute the method according to claim 11.

Citation Information

Patent Citations

  • Identifying the probability of violative behavior in a market

    US20050004862A1

  • Accounting information processing device, accounting information processing method, and accounting information processing program

    JP6667865B1

  • Information processing device, information processing method, and program

    JP7146218B1