A method for auditing the operation results of financial software systems based on machine learning technology

Abstract

The present invention relates to the field of financial technology, and specifically to a method for auditing the operation results of a financial software system based on machine learning technology. The auditing method includes two types: a single-point anomaly detection method and a collective anomaly detection method. The single-point anomaly detection method is aimed at outlier data in the operation result data, and the collective anomaly detection method is aimed at analyzing the overall distribution in the operation result; the present invention has a certain predictiveness. Since both technologies of the present invention are based on the distribution of historical system data for auditing, data that deviates from historical data can be detected. Compared with traditional auditing, since it is not based on fixed rules, abnormal data beyond existing experience can be found, and the present invention can detect collective anomalies. Since the collective anomaly detection technology detects new data based on the overall distribution of historical data, it can detect data with differences in overall distribution compared with historical data, which is impossible for traditional auditing.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a method for auditing the operation results of a financial software system based on machine learning technology. Background Art

[0002] Audit is the abbreviation of inspection and review. Through audit, the negligence and errors that may occur in the daily operation of the financial system and personnel operations can be corrected or prevented in a timely manner to improve the robustness of the system.

[0003] During the operation of financial software systems, business data audits are often required to prevent data errors. Computer algorithms have been introduced in audits in the financial field. The traditional audit method is to audit using fixed rules, and its development process generally includes three steps: 1) Business personnel propose audit requirements; 2) Developers develop code for audit logic; 3) Development is completed and the version is released online. Developers need to develop code for the rules and solidify the audit rules. When data needs to be audited, it is audited according to the established rule logic and the audit results are output.

[0004] The audit process of the above audit method is as follows: Figure 1 As shown, there are often the following deficiencies:

[0005] 1) Unable to cope with unexpected exceptions: Audit rules written in advance are usually based on existing experience. When new exceptions occur, they often go beyond the established rules, and traditional audits are powerless.

[0006] 2) Unable to detect collective anomalies: Some data appear normal from a single record perspective (e.g., a single application amount of 50,000 yuan), but when the data distribution is obviously abnormal (e.g., the application amount of all customers is 50,000 yuan), traditional methods cannot detect such anomalies.

[0007] In view of this, we propose a method to audit the operation results of financial software systems based on machine learning technology. Summary of the invention

[0008] The purpose of the present invention is to provide a method for auditing the operation results of a financial software system based on machine learning technology to solve the problems raised in the above background technology.

[0009] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for auditing the operation results of a financial software system based on machine learning technology, wherein the auditing method includes two parts: a single-point anomaly detection method and a collective anomaly detection method.

[0010] Preferably, the single-point anomaly detection method is used to analyze outlier data in transaction data.

[0011] Preferably, the single point anomaly detection method comprises the following steps:

[0012] Step 1: Classify the financial system data generated in historical transactions according to the business type, financial subject, source channel, customer group, etc. (the classification of data depends on expert experience) and record its classification information;

[0013] Step 2: Perform feature extraction and feature engineering on the system data of the same category to obtain an N-dimensional vector data set, and then divide the data set into a training set, a validation set, and a test set.

[0014] Step 3: For each group of classified data, use the machine learning anomaly detection algorithm to perform N+1 trainings, build N+1 algorithm models, and form a detector. The Nth time is to traverse each dimension of the vector for single-dimensional training, and the N+1th time is to uniformly train all dimensions (the basis for implementing interpretability);

[0015] Step 4: Perform feature engineering on the data to be tested and perform category identification to obtain a high-dimensional vector set (the set to be tested);

[0016] Step 5: Match the corresponding detector according to the category of the [set to be detected] for detection. During the detection process, mark which algorithm model in the detector detected the abnormal data, locate the dimension of the abnormal data, and finally output the abnormal data entry and the specific abnormal dimension (implementation method of explainability).

[0017] Preferably, the collective anomaly detection method analyzes the overall data distribution in the transaction data.

[0018] Preferably, the collective abnormal data detection method comprises the following steps:

[0019] Step 1: Classify the data generated in historical business according to the business type, financial subject, source channel, customer group, etc. in the system data (the classification of data depends on expert experience), and record its classification information;

[0020] Step 2: Perform segmented statistical processing on the continuous numerical fields of historical transaction data of the same category, and count the proportion of data in each interval to the total data. Assuming there are N continuous numerical fields, there are N segmented statistical data fields, and save the segmented statistical data as the [benchmark data set] (the basis for implementing interpretability);

[0021] Step 3: Perform segmented statistical processing on the continuous numerical fields of the data to be detected, and the processing steps are the same as the previous step;

[0022] Step 4: Match the corresponding [benchmark data set] according to the category of the data to be tested, and perform non-parametric tests on the fields in the benchmark data set of the data set to be tested one by one. Fields with test results less than the significance level are considered to have distribution anomalies;

[0023] Step 5: After detection, if any dimension is identified as having a distribution anomaly, the data to be detected is output as abnormal data, along with the specific abnormal field (implementation method for interpretability).

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. Predictive and better prevent unknown anomalies: Since both technologies of the present invention are based on the distribution of historical system data for auditing, data that deviates from historical data can be detected. Compared with traditional auditing, since it is not based on fixed rules, abnormal data beyond existing experience can be discovered;

[0026] 2. It can detect collective anomalies and prevent large-scale errors: Since collective anomaly detection technology detects new data based on the overall distribution of historical data, it can detect data that has an overall distribution different from historical data, making up for the shortcomings of traditional rule auditing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of the traditional audit method;

[0028] Figure 2 is a first flow chart of the single point anomaly detection method of the present invention;

[0029] Figure 3 is a second flow chart of the single point anomaly detection method of the present invention;

[0030] Figure 4 is a first flow chart of the collective anomaly detection method of the present invention;

[0031] Figure 5 This is a second flow chart of the collective anomaly detection method of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] See also Figures 1 to 5The present invention provides a technical solution: a method for auditing the operation results of a financial software system based on machine learning technology, wherein the auditing method includes two parts: a single-point anomaly detection method and a collective anomaly detection method.

[0034] The audit method of the present invention adopts a data classification calculation method, which can effectively reduce the computing power requirements, improve the audit efficiency, and increase the accuracy of the audit;

[0035] The present invention firstly uses the application of machine learning algorithm in financial software system data audit, and secondly uses the application of non-parametric test algorithm in financial software system data audit, which has an explainable implementation method.

[0036] The single-point anomaly detection method analyzes outlier data in financial software system data;

[0037] The single point anomaly detection method includes the following steps:

[0038] Step 1: Classify the financial system transaction data generated in historical transactions according to the business type, financial subject, source channel, customer group, etc. in the system data (the classification of data depends on expert experience), and record its classification information;

[0039] Step 2: Perform feature extraction and feature engineering on the system data of the same category to obtain an N-dimensional vector data set, and then divide the data set into a training set, a validation set, and a test set.

[0040] Step 3: For each group of classified data, use the machine learning anomaly detection algorithm to perform N+1 trainings, build N+1 algorithm models, and form a detector. The Nth time is to traverse each dimension of the vector for single-dimensional training, and the N+1th time is to uniformly train all dimensions (the basis for implementing interpretability);

[0041] Step 4: Perform feature engineering on the data to be tested and perform category identification to obtain a high-dimensional vector set (the set to be tested);

[0042] Step 5: Match the corresponding detector according to the category of the [set to be detected] for detection. During the detection process, mark which algorithm model in the detector detected the abnormal data, locate the dimension of the abnormal data, and finally output the abnormal data entry and the specific abnormal dimension (implementation method of explainability).

[0043] The single-point anomaly detection method is based on outlier data analysis. In steps 1 to 3, machine learning is mainly performed. After completing data classification, feature extraction is performed on historical data. Subsequently, multiple algorithm models are constructed through modeling to form an outlier database, which is equivalent to a detector. This is the basis for the implementation of explainability.

[0044] Steps 4 and 5 are to put the data to be tested into the detector for specific analysis and detection, use its features to compare with the algorithm model in the detector, and finally output the abnormal data items and the specific abnormal dimensions, which is the implementation method of explainability;

[0045] The collective anomaly detection method analyzes the overall data distribution in the transaction data;

[0046] The collective abnormal data detection method includes the following steps:

[0047] Step 1: Classify the system data generated in historical financial business, classify according to the business type, financial subject, source channel, customer group, etc. in the system data (the classification of data depends on expert experience), and record its classification information;

[0048] Step 2: Perform segmented statistical processing on the continuous numerical fields of historical system data of the same classification, and count the proportion of data in each interval to the total data. Assuming there are N continuous numerical fields, there are N segmented statistical data fields, and save the segmented statistical data as the [benchmark data set] (the basis for implementing interpretability);

[0049] Step 3: Perform segmented statistical processing on the continuous numerical fields of the data to be detected, and the processing steps are the same as the previous step;

[0050] Step 4: Match the corresponding [benchmark data set] according to the category of the data to be tested, and perform non-parametric tests on the fields in the benchmark data set of the data set to be tested one by one. Fields with test results less than the significance level are considered to have distribution anomalies;

[0051] Step 5: After detection, if any dimension is identified as having a distribution anomaly, the data to be detected is output as abnormal data, along with the specific abnormal field (implementation method for interpretability).

[0052] The collective anomaly detection method is based on the analysis of the overall data. In steps 1 to 3, machine learning is also performed. After completing the data classification, the historical data is segmented and statistically analyzed, and the proportion of continuous numerical fields is analyzed to form a benchmark data set, which is the basis for the realization of interpretability.

[0053] Steps 4 and 5 are to put the data to be tested into the benchmark data set to filter out abnormal data;

[0054] Working principle and beneficial effects: First, the audit method of the present invention adopts a data classification calculation method, which can effectively reduce the computing power requirements, improve the audit efficiency, and increase the accuracy of the audit; the present invention first uses the application of machine learning algorithms in the audit of financial software system data, and then uses the application of non-parametric test algorithms in the audit of financial software system data, which has an interpretable implementation method;

[0055] The audit method of the present invention is divided into two parts: a single-point anomaly detection method and a collective anomaly detection method. The single-point anomaly detection method is analyzed based on outlier data. In steps 1 to 3, machine learning is mainly performed. After completing data classification, feature extraction is performed on historical data. Subsequently, multiple algorithm models are constructed through modeling to form an outlier database, which is equivalent to a detector. This is the basis for implementing interpretability. Steps 4 and 5 are to put the data to be detected into the detector for specific analysis and detection, compare its features with the algorithm model in the detector, and finally output the abnormal data entry and the specific abnormal dimension, which is the implementation method of interpretability. The collective anomaly detection method is analyzed based on the overall data. In steps 1 to 3, machine learning is also performed. After completing data classification, segmented statistics are performed on historical data, and continuous numerical field proportion analysis is performed to form a benchmark data set. This is the basis for implementing interpretability. Steps 4 and 5 are to put the data to be tested into the benchmark data set to screen out abnormal data.

[0056] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for auditing the operation results of a financial software system based on machine learning technology, characterized in that: The method includes two parts: a single point anomaly detection method and a collective anomaly detection method; The single point anomaly detection method comprises the following steps: Step 1: Classify the financial system data generated in historical financial business, classify according to the business type, financial subject, source channel, and customer group in the data, and record its classification information; Step 2: Perform feature extraction and feature engineering on the system data of the same category to obtain an N-dimensional vector data set, and then divide the data set into a training set, a validation set, and a test set. Step 3: For each group of classified data, use the machine learning anomaly detection algorithm to perform N+1 trainings, build N+1 algorithm models, and form a detector. The Nth training is to traverse each dimension of the vector for single-dimension training, and the N+1th training is to uniformly train all dimensions. Step 4: Perform feature engineering on the data to be tested and perform category recognition to obtain a high-dimensional vector set; Step 5: Match the corresponding detector to the category of the set to be detected for detection. During the detection process, mark which algorithm model in the detector detected the abnormal data, locate the dimension of the abnormal data, and finally output the abnormal data entry and the specific abnormal dimension; The collective anomaly detection method comprises the following steps: Step 1: Classify the system data generated in historical business, classify according to the business type, financial subject, source channel, and customer group in the system data, and record its classification information; Step 2: Perform segmented statistical processing on the continuous numerical fields of historical transaction data of the same category, and count the proportion of data in each interval segment to the total data. Assuming there are N continuous numerical fields, there are N segmented statistical data fields, and save the segmented statistical data as the benchmark data set; Step 3: Perform segmented statistical processing on the continuous numerical fields of the data to be detected, and the processing steps are the same as the previous step; Step 4: Match the corresponding benchmark data set according to the category of the data to be tested, and perform non-parametric tests on the fields in the benchmark data set of the data set to be tested one by one. The fields with test results less than the significance level are considered to have distribution anomalies; Step 5: After detection, if any dimension is identified as having abnormal distribution, the data to be detected is output as abnormal data, along with the specific abnormal field.

2. A method for auditing the operation results of a financial software system based on machine learning technology according to claim 1, characterized in that: The single-point anomaly detection method is aimed at analyzing outlier data in financial software system data.

3. The method for auditing the operation results of a financial software system based on machine learning technology according to claim 1, characterized in that: The collective anomaly detection method analyzes the overall data distribution in the system data.

Images