Machine learning-based financial abnormality analysis method for listed companies

Abstract

The application discloses a listed company financial abnormality analysis method based on machine learning, comprising the following steps: acquiring financial data and non-financial data for training, and preprocessing the financial data and the non-financial data to obtain financial feature data and non-financial feature data; constructing a derivative index according to the financial feature data, and performing important feature screening to obtain input features; training a financial abnormality analysis model based on the non-financial feature data and the input features, and predicting the abnormality level of unknown samples based on the trained financial abnormality analysis model; acquiring model result vectors corresponding to historical abnormal samples and unknown samples predicted as abnormal, and judging the financial abnormality category of the unknown samples based on the model result vectors through a k-nearest neighbor algorithm, and performing index analysis. The method enhances the interpretability of the listed company financial abnormality analysis result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of financial data processing, and particularly relates to a listed company financial anomaly analysis method based on machine learning. BACKGROUND

[0002] Traditional listed company financial anomaly analysis generally analyzes from financial accounting subjects and enterprise operation, and is based on business field knowledge, and various accounting subjects are compared at the same level by professional accountants to find financial anomaly risk points. The traditional financial anomaly analysis method based on financial and accounting business needs deep participation of many professionals, consumes a lot of time and energy. In recent years, many financial anomaly analysis methods based on machine learning have appeared, that is, with the help of data mining, machine learning and other methods, a large amount of sample and feature data are analyzed by big data to find individual anomalies. This method is fast in analysis, low in cost, and can predict the financial anomaly degree of a listed company. Although the financial anomaly analysis method based on machine learning can assist the supervisor to find the financial anomaly degree of the listed company to a certain extent, it has the defect of poor explanation, that is, it cannot accurately point out the specific aspects and indicators of the company that exist anomalies. SUMMARY

[0003] The main purpose of the present application is to provide a listed company financial anomaly analysis method based on machine learning, which aims to simplify and improve the explanation of the financial anomaly of the listed company.

[0004] To achieve the above purpose, the present application provides a listed company financial anomaly analysis method based on machine learning, which comprises the following steps:

[0005] Obtain financial data and non-financial data for training, and preprocess the financial data and the non-financial data to obtain financial feature data and non-financial feature data;

[0006] Derivative indexes are constructed according to the financial feature data, and important feature screening is performed to obtain input features;

[0007] A financial anomaly analysis model is trained based on the non-financial feature data and the input features, and unknown samples are predicted for anomaly level based on the trained financial anomaly analysis model;

[0008] Obtain the model result vector corresponding to the historical anomaly samples and the unknown samples predicted to be abnormal, and judge the financial anomaly category of the unknown samples by k-nearest neighbor algorithm based on the model result vector, and perform index analysis.

[0009] Optionally, the financial data includes high-risk abnormal listed company financial data subject to administrative penalties of the securities regulatory commission, self-regulatory supervision and warning letters, medium-risk abnormal listed company financial data with delisting risk warning and other risk warning signs, abnormal listed company financial data of grade C or D in information disclosure in addition to the high-risk abnormal listed company and the medium-risk abnormal listed company, and listed company financial data without obvious abnormalities in addition to the high-risk abnormal listed company financial data, the medium-risk abnormal listed company financial data and the abnormal listed company financial data.

[0010] Optionally, the step of obtaining a model result vector corresponding to the historical abnormal sample and the unknown sample predicted to be abnormal, and judging the financial abnormality category of the unknown sample through a k-nearest neighbor algorithm based on the model result vector and performing index analysis includes:

[0011] Labeling the abnormal category corresponding to the historical abnormal sample;

[0012] Vectorizing the analysis result of the financial abnormality analysis model corresponding to the historical abnormal sample;

[0013] Obtaining a model result vector corresponding to the unknown sample feature data through the financial abnormality analysis model;

[0014] Taking the model result vector corresponding to the historical abnormal sample and the model result vector corresponding to the unknown sample as inputs of the k-nearest neighbor algorithm, and determining the abnormal category of the unknown sample through the k-nearest neighbor algorithm; and

[0015] Based on the historical sample, determining the normal interval and the abnormal interval of each index under different industries;

[0016] Performing index analysis based on the index interpretation library.

[0017] Optionally, the step of obtaining financial data and non-financial data for training, and preprocessing the financial data and the non-financial data to obtain financial feature data and non-financial feature data includes:

[0018] Obtaining the financial data and the non-financial data for training;

[0019] Determining financial indicators and non-financial indicators;

[0020] According to the financial indicators and non-financial indicators, cleaning and reconstructing the financial data and the non-financial data to obtain the financial feature data and the non-financial feature data.

[0021] Optionally, the step of constructing derived indicators according to the financial feature data, and performing important feature screening to obtain input features includes:

[0022] grouping the financial indicators in the financial characteristic data for which it is to be determined whether there is an implicit correlation;

[0023] obtaining a regression equation based on the financial characteristic data and the grouping result of the financial indicators;

[0024] constructing a derived indicator based on the regression equation;

[0025] performing important feature screening to obtain input features.

[0026] The embodiment of the present application proposes a listed company financial anomaly analysis method based on machine learning. Firstly, financial data and important non-financial data used for training are obtained to construct financial characteristics and non-financial characteristics. Then, the linear regression method is used to mine the implicit correlation between financial indicators and financial indicators to construct derived financial characteristics. Next, the LightGBM algorithm is used for feature screening and model training to obtain a set of financial anomaly analysis models. The unknown samples are predicted by the model for anomaly level, and the model output vector results of the historical anomaly samples and unknown samples can be obtained. Finally, the k nearest neighbor algorithm is used to determine the unknown sample anomaly category and perform index analysis, thereby enhancing the interpretability of the listed company financial anomaly analysis result. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 FIG. 1 is a flowchart of an embodiment of the listed company financial anomaly analysis method based on machine learning.

[0028] The implementation, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0029] It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0030] Traditional listed company financial anomaly analysis generally starts from financial accounting subjects and enterprise operation, is based on business field knowledge, finds financial anomaly risk points through comparison of various accounting subjects, and is implemented by professional accountants. The traditional financial anomaly analysis method based on financial and accounting business needs deep participation of many professionals, consumes a large amount of time and energy. In recent years, many financial anomaly analysis methods based on machine learning have appeared, that is, with the help of data mining, machine learning and other methods, a large amount of sample and feature data are analyzed to find individual anomalies. This method is fast in analysis, low in cost, and can predict the financial anomaly degree of a listed company. Although the financial anomaly analysis method based on machine learning can assist the supervisor to find the financial anomaly degree of the listed company to a certain extent, the method has the defect of poor explainability, that is, the company cannot be accurately pointed out in which aspect and which index exist anomalies.

[0031] To solve the above defects, the main solution of an embodiment of the listed company financial anomaly analysis method based on machine learning of the present application is as follows:

[0032] Financial data and non-financial data used for training are acquired, and the financial data and the non-financial data are preprocessed to obtain financial feature data and non-financial feature data;

[0033] Derivative indexes are constructed according to the financial feature data, and important features are screened to obtain input features;

[0034] A financial anomaly analysis model is trained based on the non-financial feature data and the input features, and unknown samples are predicted for anomaly levels based on the trained financial anomaly analysis model;

[0035] Model result vectors corresponding to historical anomaly samples and unknown samples predicted as anomalies are acquired, and the financial anomaly categories of the unknown samples are judged by a k-nearest neighbor algorithm based on the model result vectors, and index analysis is performed.

[0036] The method mainly constructs financial features around profit quality, asset quality and cash flow quality based on financial three big report data, and extracts important non-financial data such as abnormal change of an accounting firm to construct non-financial features. Then, the implicit correlation between financial indexes and financial indexes is mined by a linear regression method, a regression equation is acquired, and derivative financial features are constructed based on the regression equation. Next, a LightGBM algorithm is used for feature screening and model training to obtain a set of financial anomaly analysis models, unknown samples are predicted for anomaly levels by the models, and model output vector results of historical anomaly samples and unknown samples can be acquired. Finally, a k-nearest neighbor algorithm is used to judge the anomaly categories of the unknown samples, and index analysis is performed, thereby enhancing the explainability of the listed company financial anomaly analysis result.

[0037] Reference Figure 1 In an embodiment of the method for analyzing financial anomalies of listed companies based on machine learning, the method comprises the following steps:

[0038] Step S10, obtaining financial data and non-financial data for training, and preprocessing the financial data and the non-financial data to obtain financial feature data and non-financial feature data;

[0039] In this embodiment, the financial data includes high-risk abnormal listed company financial data subject to administrative penalties by the Securities Regulatory Commission, self-regulatory supervision and warning letters, medium-risk abnormal listed company financial data with delisting risk warning and other risk warning signs, abnormal listed company financial data with information disclosure level of C or D, and normal listed company financial data other than the high-risk abnormal listed company financial data, the medium-risk abnormal listed company financial data and the abnormal listed company financial data. For example, the label of the high-risk abnormal listed company financial data can be set to 1, the label of the medium-risk abnormal listed company financial data can be set to 2, the label of the abnormal listed company financial data can be set to 3, and the label of the normal listed company financial data can be set to 0.

[0040] Specifically, the financial data can include the balance sheet, profit table, cash flow table and financial note data of an enterprise, and the non-financial data can include the lawsuit announcement, violation behavior announcement, abnormal change of accounting firm, ranking of accounting and auditing agencies (such as the ranking of each firm by the China Association of Certified Public Accountants or the ranking of the securities system accounting firm portrait system), arbitration defeat, lawsuit defeat and other data of an enterprise. After obtaining the financial data and non-financial data, the financial and non-financial data are preprocessed. For example, the required financial indicators and non-financial indicators can be determined first.

[0041] Specifically, to improve the accuracy and reliability of the financial anomaly analysis result, the financial indicators can be constructed around the aspects of profit quality, asset quality, cash flow quality, etc. For example, the financial indicators can be set as profit growth rate, proportion of monetary funds in total assets, cash flow generated by operating activities, etc. It can be understood that the embodiment does not limit the specific sub-projects corresponding to the financial indicators, and the user can customize and set according to the actual demand, which will not be enumerated here. For non-financial indicators, it can be set as the number of abnormal changes of accounting firms, the number of defeats, etc. Next, the financial data and non-financial data are cleaned and reconstructed, and the financial indicator data and the non-financial indicator data are merged according to the company code and the report year to obtain complete financial features and non-financial features.

[0042] In step S20, the derived indicators are constructed according to the financial feature data, and important features are screened to obtain input features.

[0043] In the embodiment, the derived indicators can be constructed according to the financial feature data, and the linear regression is used to explore the implicit correlation between the financial indicators. First, the financial indicators to be determined whether there is an implicit correlation are grouped, such as the financial subject indicators related to enterprise operation activities such as operating income, operating cost, sales expense, and net cash flow from operating activities are divided into a group.

[0044] For each group, all index combinations are traversed (considering the traversal time overhead and the complexity of the regression equation, the number of independent variables in the regression equation is set to be not more than five indicators). The specific steps are as follows: first, select a certain indicator as the dependent variable in the group, select two to five indicators as the independent variables in the remaining indicators, then perform binary to five regression based on the financial indicator data, and calculate the regression equation fitting degree (the fitting degree can judge the regression effect). By setting the fitting degree R_square threshold, such as setting the lower limit range to 0.7 and the upper limit range to 0.95, the regression equation with a calculation result within the threshold range is screened out. Assuming that a certain regression equation is screened out through calculation: sales expense ≈ k1* monetary funds + k2* operating cost, it is proved that a large number of samples satisfy such a correlation relationship, and if the financial indicator data of a certain company does not conform to the relationship, and the greater the deviation, the greater the possibility that the financial status of the company is abnormal, therefore, the residual value obtained by subtracting the actual value of the financial indicator (the actual value of the dependent variable on the left side of the regression equation) from the predicted value of the regression equation (the calculated value of the right side of the regression equation) can be used as a newly generated derived indicator for model training to improve the model recognition effect.

[0045] Further, after obtaining the derived indicators, considering factors such as time overhead, interpretability, sample size, model effect, etc., the LightGBM model is selected to screen important indicators. The derived features and financial feature data are used as algorithm inputs for multiple training, and the feature set actually used by the model in each training is obtained. Since the decision tree uses the feature with the maximum information gain in construction, the features with relatively more usage times are selected as the model input features.

[0046] Step S30, training a financial anomaly analysis model based on the non-financial feature data and the input features, and predicting the abnormal level of unknown samples based on the trained financial anomaly analysis model;

[0047] In this embodiment, a LightGBM algorithm is used to train a financial anomaly analysis multi-classification model based on the non-financial feature data and the input features, and the grid search parameter tuning method and five-fold cross-validation are used to retain the best model. After the same processing of unknown samples as the training samples, the feature data of the unknown samples is obtained, and then input into the financial anomaly analysis model for prediction to obtain the abnormal level of the unknown samples, wherein the samples with classification results of 1, 2 and 3 are regarded as abnormal.

[0048] Step S40, obtaining the model result vector corresponding to the historical abnormal samples and the unknown samples predicted as abnormal, and judging the financial anomaly category of the unknown samples based on the model result vector through a k-nearest neighbor algorithm, and performing index analysis.

[0049] Specifically, in order to improve the interpretability of the analysis results of listed companies' financial anomalies, index analysis can be performed through methods such as calculating the financial anomaly similarity between companies, constructing an index anomaly explanation library, etc. to assist in interpretation, so that the analysis results are more credible and assist users in understanding and analyzing.

[0050] In terms of financial anomaly similarity calculation, the financial anomaly similarity between companies is calculated by using the path vector of the decision tree output by the financial anomaly analysis model. First, the historical abnormal samples are labeled according to the abnormal categories, including income anomaly, cost and expense anomaly, cash flow anomaly, liability anomaly, asset anomaly and related party interest transfer. Then, the path of the abnormal samples in the trained financial anomaly analysis model is converted into a vector, and each abnormal sample forms a specific dimension vector as the basis for judging the abnormal type of unknown companies.

[0051] Optionally, the path vector of unknown sample feature data can also be obtained through the financial anomaly analysis model, and then the k-nearest neighbor algorithm is used to judge the financial anomaly category of the unknown sample, that is, the K nearest instances to the unknown instance are found in the abnormal samples of the known abnormal categories, and the abnormal category of the unknown sample is predicted as which category. The proximity measurement method can select the distance algorithm to calculate the similarity between the unknown sample vector and the known abnormal sample vector. Taking the Manhattan distance in two-dimensional space as an example, the Manhattan distance between point i with coordinates (x1, y1) and point j with coordinates (x2, y2) is:

[0052] d(i,j) = |x1-x2| + |y1-y2|

[0053] Cosine similarity, Euclidean distance, etc. can also be used for measurement. For the selection of K value, cross-validation method can be used to select the optimal K value based on the known abnormal category samples.

[0054] In the aspect of index analysis, based on the input feature data, the best effect of distinguishing abnormal and non-abnormal samples is taken as the standard to solve the normal interval and abnormal interval of each index in different industries. Taking industry A and index x as an example, first, the historical sample data of industry A is screened, and the x index value is sorted in ascending order. The 25%-75% industry quantile can be selected as the initial normal interval, and the sliding interval of 5%, 10% or other quantiles is expanded to the left and right, and the sliding window is taken as the normal interval. If the sample index actual value is not in the abnormal interval range, it is regarded as an abnormal sample. The recall rate and precision rate of the abnormal sample under this condition are solved, and finally the f1 value is solved by combining the recall rate and the precision rate. After multiple calculations, the normal interval and abnormal interval corresponding to the maximum f1 value are found. For example, the industry 25%-75% quantile can be taken as the normal interval in the first calculation, and less than the industry 25% and greater than the 75% quantile is regarded as the abnormal interval. First, calculate the number of abnormal samples predicted as abnormal samples TP, the number of non-abnormal samples predicted as non-abnormal samples TN, the number of non-abnormal samples predicted as abnormal samples FP, and the number of abnormal samples predicted as non-abnormal samples FN under this condition. Calculate the precision and recall, wherein the precision calculation method is:

[0055]

[0056] The recall calculation method is:

[0057]

[0058] The F1 value is calculated from the precision and recall:

[0059]

[0060] The second calculation can take the 20%-80% quantile as the normal interval, less than the 20% quantile and greater than the 80% quantile of the industry as the abnormal interval, and calculate the F1 value again, and so on, so as to finally find the normal interval and the abnormal interval corresponding to the maximum F1 value as the normal interval and the abnormal interval of the A industry x indicator.

[0061] Based on the indicator explanation library, the actual value of the indicator greater than the right boundary of the normal interval is regarded as being large in the industry, the actual value less than the left boundary of the normal interval is regarded as being small in the industry, and the rest is regarded as normal, and a large or small indication is performed. If the financial expense rate of the indicator is large, it is suggested that the company may have abnormal burden relative to other companies in the industry. It is suggested that the business personnel focus on the loan project of the balance sheet of the company, whether there is additional interest in the other accounts payable, and the change of the cash back policy and the increase of the cash discount; on the other hand, the abnormal path matching is performed, the financial experience of the financial expert is solidified as the abnormal path, and the rule of the path can be defined as “XX feature large-XX feature small-XX feature large”. If the financial data of the enterprise meets the abnormal path, the abnormal path will be used as the business explanation of the possible occurrence of some aspects of abnormality. For example, the path “management fee small-management fee growth rate small-operating expenses large” can be explained as: the current management fee of the company is small, and the management fee growth rate is also small, while the operating expenses are large, so the company may unreasonably regard the period expenses as operating expenses to achieve the purpose of beautifying the operating profit.

[0062] In the technical scheme disclosed in the embodiment, first, financial data and non-financial data used for training are acquired, financial features are constructed around profit quality, asset quality, and cash flow quality, and important non-financial data such as abnormal change of an accounting firm is extracted to construct non-financial features. Then, the linear regression method is used to mine the implicit correlation between financial indicators and financial indicators, obtain a regression equation, and construct derived financial features based on the regression equation. Next, the LightGBM algorithm is used for feature screening and model training to obtain a set of financial anomaly analysis model, and the model is used to predict the abnormal level of unknown samples, and the model output vector results of historical abnormal samples and unknown samples can be obtained. Finally, the k-nearest neighbor algorithm is used to determine the abnormal category of unknown samples, and index analysis is performed, thereby enhancing the explainability of the financial anomaly analysis result of the listed company.

[0063] It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0064] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0065] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) as described above, and includes a number of instructions for making a terminal device (which can be a computer or a server) execute the methods described in the various embodiments of the present application.

[0066] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A machine learning-based method for analyzing financial anomalies in listed companies, characterized in that, The machine learning-based method for analyzing financial anomalies in listed companies includes the following steps: Acquire financial and non-financial data for training, and preprocess the financial and non-financial data to obtain financial feature data and non-financial feature data. Based on the financial feature data, derived indicators are constructed, and important features are filtered to obtain input features. Specifically, financial indicators in the financial feature data that are to be determined whether there is an implicit correlation are grouped; regression equations are obtained based on the financial feature data and the grouping results of the financial indicators; derived indicators are constructed based on the regression equations; and important features are filtered to obtain input features. A financial anomaly analysis model is trained based on the non-financial feature data and the input features, and anomaly level prediction is performed on unknown samples based on the trained financial anomaly analysis model. The process involves obtaining model result vectors corresponding to historical abnormal samples and unknown samples predicted to be abnormal, and determining the financial anomaly category of the unknown samples using the k-nearest neighbor algorithm based on these model result vectors. Then, indicator analysis is performed based on an indicator interpretation library. The steps include: labeling the anomaly categories corresponding to historical abnormal samples; vectorizing the financial anomaly analysis model results corresponding to the historical abnormal samples; obtaining model result vectors corresponding to the feature data of the unknown samples through the financial anomaly analysis model; using the model result vectors corresponding to the historical abnormal samples and the unknown samples as input to the k-nearest neighbor algorithm to determine the anomaly category of the unknown samples; determining the normal and abnormal ranges of various indicators under different industries based on historical samples; and performing indicator analysis based on an indicator interpretation library.

2. The machine learning-based financial anomaly analysis method for listed companies as described in claim 1, characterized in that, The financial data includes financial data of high-risk abnormal listed companies that have been subject to administrative penalties, self-regulatory supervision, and reminder letters from the China Securities Regulatory Commission (CSRC); financial data of medium-risk abnormal listed companies with delisting risk warnings and other risk warning labels; financial data of abnormal listed companies with information disclosure levels of C or D, excluding the high-risk and medium-risk abnormal listed companies; and financial data of listed companies without obvious abnormalities, excluding the high-risk, medium-risk, and abnormal listed company financial data.

3. The method for analyzing financial anomalies in listed companies based on machine learning as described in claim 1, characterized in that, The steps of acquiring financial and non-financial data for training, and preprocessing the financial and non-financial data to obtain financial feature data and non-financial feature data include: Obtain the financial data and the non-financial data used for training; Determine financial and non-financial indicators; Based on the financial and non-financial indicators, the financial data and the non-financial data are cleaned and reconstructed to obtain the financial feature data and the non-financial feature data.

Images