Financial data analysis system based on machine learning

Abstract

The invention relates to the technical field of data analysis, in particular to a financial data analysis system based on machine learning, and the system comprises a data set generation module which is used for obtaining historical financial data; the model evaluation module is used for evaluating the performance index of each pre-stored machine learning model according to the historical financial data and generating a performance evaluation result; the data acquisition module is used for acquiring financial data to be processed and corresponding business risk preferences; the model screening module is used for generating the weight of each performance index according to the business risk preference corresponding to the financial data to be processed, and screening a target model according to the performance evaluation result and the weight corresponding to each performance index; the model training module is used for training a target model; and the risk assessment module is used for performing data analysis on the to-be-processed financial data by adopting the trained target model. By adopting the scheme, the model most conforming to the current risk decision target can be screened out for training and deployment according to the actual demand, so that the deviation of the risk assessment result is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular to a financial data analysis system based on machine learning. BACKGROUND

[0002] With the continuous expansion and digital development of the global financial system, credit risk management is increasingly important in the fields of banking, insurance, Internet finance, etc. An effective financial risk control system not only can reduce the risk of institutional losses, but also can improve resource allocation efficiency and optimize customer service experience. In particular, in the fields of credit card business, high-frequency loans, consumer finance, etc., how to accurately assess the default risk of customers has become an important manifestation of the core competitiveness of each financial institution.

[0003] Traditional financial risk control mainly relies on rule-based systems, using fixed indicators and manual experience to evaluate customer credit. However, with the rapid expansion of data size and the diversification of customer behavior patterns, traditional methods face many challenges such as insufficient accuracy, poor flexibility, and weak real-time performance. The rise of machine learning technology has brought new solutions to financial risk control. Through learning and pattern mining on large-scale historical data, machine learning models can automatically extract key features, mine hidden rules, and thus achieve accurate prediction of customer risk.

[0004] However, there is a significant limitation in existing technologies: a single fixed machine learning model is often used to handle all types of financial data analysis and risk assessment tasks. Whether it is logistic regression, support vector machine, random forest, gradient boosting machine, or other classifiers such as K-nearest neighbors or naive Bayes, system designers often select a specific model at the deployment stage and apply it to all business scenarios and data types. This approach is due to multiple real reasons: first, the model selection process is complex and lacks standards, developers often select models based on personal experience or preliminary test results in specific scenarios, lacking a systematic evaluation and comparison mechanism for the performance of different models in diversified financial risk scenarios; second, ignoring the dynamic adaptability of business objectives, different financial businesses have fundamental differences in risk determination tolerance and performance indicators of interest, and a single model is difficult to optimize these potentially conflicting goals.

[0005] This single model application mode has obvious disadvantages, and it is difficult to maintain optimal performance, which may lead to an increase in risk assessment result bias. Therefore, it is urgent to provide a financial data analysis system based on machine learning, which can select the model that best fits the current risk decision-making goal for training and deployment, thereby reducing the bias of risk assessment results by improving model adaptability. SUMMARY

[0006] The application provides a machine learning-based financial data analysis system, which can screen a model most suitable for a current risk decision target according to actual needs, and train and deploy the model, so as to reduce the deviation of risk assessment results by improving the adaptability of the model.

[0007] In order to achieve the above purpose, the application provides the following technical solutions:

[0008] A machine learning-based financial data analysis system comprises a data set generation module, a model evaluation module, a data acquisition module, a model screening module, a model training module and a risk assessment module.

[0009] The data set generation module is configured to acquire historical financial data and generate a training set according to the historical financial data.

[0010] The model evaluation module is configured to evaluate performance indicators of each pre-stored machine learning model according to the historical financial data, and generate a performance evaluation result.

[0011] The data acquisition module is configured to acquire financial data to be processed and a business risk preference corresponding to the financial data to be processed.

[0012] The model screening module is configured to generate weights of the performance indicators according to the business risk preference corresponding to the financial data to be processed, and screen a machine learning model as a target model according to the performance evaluation result and the weights of the performance indicators.

[0013] The model training module is configured to train the target model using the training set.

[0014] The risk assessment module is configured to perform data analysis on the financial data to be processed using the trained target model, and generate a risk assessment result.

[0015] Further, the data set generation module comprises a data acquisition module, a data preprocessing module and a data set division module.

[0016] The data acquisition module is configured to acquire historical financial data.

[0017] The data preprocessing module is configured to preprocess each historical financial data, and the preprocessing comprises standardization processing, missing value filling and data type conversion.

[0018] The data set division module is configured to divide each preprocessed historical financial data into a training set and a test set according to a preset ratio.

[0019] Further, the data preprocessing module comprises a data type identification module, a standardization processing module and a missing value filling module.

[0020] The data type identification module is configured to identify data types contained in each historical financial data, wherein the data types include numerical types and category types, and to convert the data types of the corresponding historical financial data into numerical types if the data types contained in the historical financial data include category types.

[0021] The standardization processing module is configured to perform standardization processing on each historical financial data.

[0022] The missing value filling module is configured to analyze whether there is a missing value in each historical financial data, and to fill the missing value by using a mean filling method if there is a missing value.

[0023] Further, the model training module is further configured to test the trained target model by using a test set.

[0024] Further, the performance indicators include accuracy, precision, recall, F1 score and AUC value.

[0025] Further, the machine learning model includes a logistic regression model, a support vector machine, a random forest model, a gradient boosting machine, K-nearest neighbor and a naive Bayes classifier.

[0026] Further, the data visualization module is further configured to generate a data analysis chart according to the historical financial data.

[0027] The principles and advantages of the present application are as follows:

[0028] 1. The traditional single model scheme often leads to performance degradation due to neglecting data dynamics, while in the present scheme, the dynamic adaptation of the model is realized through a multi-model evaluation-selection mechanism, specifically, the weights of each performance indicator can be automatically allocated according to the business risk preference, and the machine learning model with the best adaptation is selected for data processing.

[0029] 2. The closed-loop design from data preprocessing, model selection to risk assessment reduces the need for manual intervention, thereby reducing the system operation and maintenance cost. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The figure is a logic block diagram of an embodiment of the financial data analysis system based on machine learning.

[0031] Figure 2 The figure is a logic block diagram of the data set generation module in an embodiment of the financial data analysis system based on machine learning.

[0032] Figure 3 The figure is a schematic diagram of the evaluation results of the model performance indicators in an embodiment of the financial data analysis system based on machine learning. DETAILED DESCRIPTION

[0033] The specific embodiments are further described in detail below:

[0034] Example 1:

[0035] Example 1 is substantially as shown in the accompanying Figure 1

[0036] A machine learning-based financial data analysis system, as shown in Figure 1 includes a data set generation module, a model evaluation module, a data acquisition module, a model screening module, a model training module, and a risk assessment module.

[0037] The data set generation module is configured to obtain a plurality of historical financial data and generate a training set based on the historical financial data; as shown in Figure 2 The data set generation module includes a data collection module, a data preprocessing module, and a data set division module.

[0038] The data collection module is configured to collect a plurality of historical financial data; specifically, a user uploads historical financial data in CSV or Excel file format through a front-end interface, and the data collection module performs format verification and analysis after receiving the uploaded historical financial data. Specifically, if the file format is Excel, the multi-level table header is automatically identified and processed, and the read_excel() function is used to read the data and flatten the table header. If it is in CSV format, read_csv() is used to read the data and ensure uniform field names.

[0039] The data preprocessing module is configured to preprocess each historical financial data, and the preprocessing includes standardization processing, missing value filling, and data type conversion; specifically, the data preprocessing module includes a data type identification module, a standardization processing module, and a missing value filling module.

[0040] The data type identification module is configured to identify the data types contained in each historical financial data, including numerical and categorical types; and if the data types contained in the historical financial data include categorical types, the data type of the corresponding historical financial data is converted to numerical type; and LabelEncoder() is used for encoding.

[0041] The standardization processing module is configured to perform standardization processing on each historical financial data.

[0042] ​The missing value filling module is configured to analyze whether there is a missing value in each historical financial data, and if yes, fill the missing value by using a mean value filling method, i.e., using a median value to fill the missing value, specifically, filling the missing value by using a fillna() function, and outputting the historical financial data after filling the missing value as the preprocessed historical financial data; and if no, directly outputting the standardized historical financial data as the preprocessed historical financial data.

[0043] The data set division module is configured to divide each preprocessed historical financial data into a training set and a test set according to a preset ratio. In this embodiment, the training set and the test set are divided according to the preset ratio set by the user by using a train_test_split() function.

[0044] The model evaluation module is configured to evaluate each performance index of each pre-stored machine learning model according to the historical financial data, and generate a performance evaluation result. In this embodiment, the machine learning model includes a logistic regression model, a support vector machine, a random forest model, a gradient boosting machine, K-nearest neighbor and a naive Bayes classifier. The performance index includes accuracy, precision, recall, F1 score and AUC value. In this embodiment, the model evaluation module includes a storage module and an evaluation module.

[0045] In this embodiment, first, a model is constructed on the training set, the model is applied on the test set, the prediction result is compared with the real label, and then the accuracy (the proportion of correctly predicted samples), the precision (the proportion of actual positive examples in the predicted positive examples), the recall (the proportion of correctly predicted actual positive examples), the F1 score (the harmonic mean of the precision and the recall) and the AUC value (the area under the ROC curve) of each model are calculated, and the performance evaluation result is generated.

[0046] In other embodiments of the present application, in order to reduce the threshold for users to master the underlying algorithm implementation and improve the model configuration efficiency, the model evaluation module is used to uniformly manage the construction, training and saving process of each machine learning model. Through the modular encapsulation, the flexibility of model calling is improved, and the algorithm expansion and system maintenance in the later stage are facilitated. Specifically, the model evaluation module provides the following interfaces:

[0047] get_model(model_name): returns the corresponding machine learning model (such as logistic regression, random forest, etc.) according to the user selection.

[0048] train_model(model, X_train, y_train): trains the specified machine learning model using the training set, and uses the default basic parameters by default.

[0049] optimize_model(model, param_grid, X_train, y_train): Hyperparameter tuning based on GridSearchCV, supporting user-defined parameter space.

[0050] save_model(model, filepath) and load_model(filepath): Model serialization and loading interfaces for persistent management.

[0051] All interfaces are designed in the Scikit-learn API style, with strong compatibility and easy embedding into front-end configuration interfaces and control logic. The input data type is NumPy array or Pandas DataFrame. The output result is a trained Scikit-learn classifier, the optimal hyperparameter combination searched by GridSearchCV, training logs or evaluation result objects (can be used for graphical display or table recording), finally saved to the specified local path in joblib format. Through the model evaluation module, users can easily switch algorithms and compare effects, realize rapid iteration and deployment of models, and provide diverse prediction strategy support for the system.

[0052] The model evaluation module supports two training methods: one is to use default parameters for direct training, quickly generating a preliminary model; the other is to use GridSearchCV for hyperparameter optimization, which combines cross-validation strategies to search multiple parameter combinations, select the optimal parameter configuration, improve the model's generalization ability and prediction accuracy, and find the best performance model configuration. The following is the parameter space set for each model:

[0053] Logistic Regression: Regularization parameter C, value range [0.01, 0.1, 1, 10, 100], regularization method L1 or L2.

[0054] Support Vector Machine (SVM): Kernel function linear, poly, rbf. Regularization parameter C, value range: [0.01, 0.1, 1, 10, 100]. Kernel function parameter gamma, value range: [0.01, 0.1, 1, 10].

[0055] Random Forest Model: Number of trees n_estimators, value range: [50, 100, 150, 200] Maximum depth max_depth, value range: [5, 10, 15, 20] Minimum sample split min_samples_split, value range: [2, 5, 10].

[0056] Gradient Boosting Decision Tree (GBDT): learning rate learning_rate, value range: [0.01, 0.05, 0.1, 0.2]. Number of trees n_estimators, value range: [100, 200, 300]. Maximum depth max_depth, value range: [3, 5, 7, 10].

[0057] K-Nearest Neighbors (KNN): Number of neighbors n_neighbors, value range: [3, 5, 7, 10]. Distance metric: Euclidean, Manhattan, Minkowski.

[0058] Naive Bayes Classifier: Distribution used: Gaussian distribution GaussianNB or multinomial distribution MultinomialNB.

[0059] The model evaluation module also includes the following interfaces:

[0060] evaluate_model(model, X_test, y_test): Output evaluation indicators for each class (accuracy, precision, recall, F1, AUC).

[0061] plot_roc(model, X_test, y_test): Plot ROC curve.

[0062] plot_pr(model, X_test, y_test): Plot Precision-Recall curve.

[0063] plot_confusion_matrix(model, X_test, y_test): Generate confusion matrix heatmap.

[0064] The model evaluation module is used to systematically evaluate the performance of a single or multiple trained models. Evaluation indicators include accuracy, precision, recall, F1 score and AUC value. Support drawing ROC curve, Precision-Recall curve, confusion matrix heat map, and comparing the performance of different models in each indicator through bar chart, help users intuitively select the optimal model. In addition, the module provides cross-validation function, further verifies the generalization ability of the model, improves the scientificity and rigor of the evaluation. The evaluation of each model is as follows Figure 3Accuracy: represents the proportion of samples that the model predicts correctly out of the total samples. In the figure, Gradient Boosting and SVM perform well on this metric, approaching or exceeding 0.8 Precision: represents the proportion of samples that are actually positive out of all samples predicted as positive. Gradient Boosting has the highest precision, while KNN has lower precision. Recall: represents the proportion of samples that are actually positive out of all samples predicted as positive. In the figure, Gradient Boosting and SVM also perform well on recall, and are relatively balanced. F1 Score: the harmonic mean of precision and recall, considering both the accuracy and comprehensiveness of the model. Gradient Boosting performs best on F1 score. AUC (Area Under the Curve): the area under the ROC curve, reflecting the model's ability to distinguish between different classes. Gradient Boosting and SVM both have good AUC scores, showing their excellent performance in classification tasks. From the chart, we can see that Gradient Boosting is the best-performing model, especially in balancing precision, recall, and AUC. Therefore, in practical applications, Gradient Boosting may be the preferred model for handling complex datasets and high-dimensional data.

[0065] The data acquisition module is configured to acquire the to-be-processed financial data and a business risk preference corresponding to the to-be-processed financial data, wherein the business risk preference includes a false positive reduction priority, a false negative reduction priority, and a false positive and false negative balance.

[0066] The model screening module is configured to generate weights of each performance indicator according to the business risk preference corresponding to the to-be-processed financial data, and screen a machine learning model as a target model according to the performance evaluation result and the weights of each performance indicator. Specifically, each performance indicator is provided with an initial weight. If the business risk preference is false positive reduction, the weight of precision is increased. If the business risk preference is false negative reduction, the weight of recall is increased. If the business risk preference is false positive and false negative balance, the weight of F1 score is increased. Then, the machine learning model is screened as the target model according to the modified weights.

[0067] The model training module is configured to train the target model using a training set and test the trained target model using a test set.

[0068] The risk assessment module is configured to perform data analysis on the to-be-processed financial data by using the trained target model and generate a risk assessment result. In this embodiment, the risk assessment module comprises the following interfaces:

[0069] predict_proba(model, X_new): Obtain the customer default probability.

[0070] threshold_classify(prob, threshold): Output the predicted label (default / non-default) according to the set threshold.

[0071] assign_risk_level(prob): Divide the risk level (five levels) according to the probability interval.

[0072] generate_report(data, prob, level): Generate an evaluation report containing customer ID, probability, level and other fields.

[0073] The output result is a model evaluation index summary table, various graphical evaluation results, customer default probability, prediction result and risk level label, and risk assessment results in CSV format.

[0074] Embodiment 2:

[0075] Embodiment 2 has the same basic principle as Embodiment 1, and the difference lies in that the data visualization module is further included in Embodiment 2. The data visualization module is configured to generate a data analysis chart according to the historical financial data, and the data analysis chart comprises one or more of a default distribution chart, a feature heat map, a PCA dimensionality reduction visualization, a feature importance chart, a confusion matrix and an ROC / PR curve.

[0076] Specifically, in the financial risk control task, the data dimension is usually high and the relationship between variables is complex, and the user needs to fully understand the data structure, variable distribution and target variable characteristics before modeling. Therefore, the data visualization module can provide intuitive and interactive analysis tools to help users understand data patterns, identify potential anomalies or redundant features, thereby providing a strong reference basis for subsequent modeling. In addition, graphical display can improve system usability and interpretability, making it easier for non-technical users to understand data characteristics.

[0077] The data visualization module provides the following core interface functions:

[0078] show_overview(data): Output the mean, standard deviation, minimum value, maximum value and other descriptive statistical indicators of the data set;

[0079] plot_target_distribution(data, target_column): plot the distribution of the target variable (e.g. default or not) by category;

[0080] plot_correlation_matrix(data): plot the correlation matrix of features to help identify collinearity among variables;

[0081] plot_feature_distribution(data, column): plot the histogram and kernel density plot of a single variable;

[0082] plot_pca_projection(X, y): perform PCA dimensionality reduction and plot a 2D scatter plot to show the underlying distribution structure of samples.

[0083] The above interfaces are implemented based on Seaborn and Matplotlib, and can be directly integrated into the Streamlit front-end interface after encapsulation, and support user-defined field selection. The input data type requires Pandas DataFrame format, which includes complete numerical feature columns and target columns. The final output result is a graph object (Matplotlib Figure) or a dynamic chart directly embedded in the front-end page. If there is an exception (such as an unprocessed non-numeric variable), the system will return a prompt to avoid graph drawing errors.

[0084] With the present scheme, the graphical form can facilitate users or financial managers to understand and make decisions, effectively solving the application obstacles of "black box models".

[0085] The above is only an embodiment of the present application, and the common knowledge of specific structures and characteristics in the scheme is not described in detail. The person skilled in the art knows all the ordinary technical knowledge in the field of the application before the application date or the priority date, can know all the prior art in this field, and has the ability to apply conventional experimental means before that date. The person skilled in the art can improve and implement the present scheme based on their own ability under the guidance of the present application, and some typical known structures or known methods should not be an obstacle for the person skilled in the art to implement the present application. It should be noted that for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should also be considered as the protection scope of the present application, and these will not affect the effect and practicality of the present application. The scope of protection claimed in the present application should be based on the content of its claims, and the specific implementation mode and the like in the specification can be used to explain the content of the claims.

Claims

1. A financial data analysis system based on machine learning, characterized by: It includes data set generation module, model evaluation module, data acquisition module, model screening module, model training module and risk assessment module; The data set generation module is used to obtain a number of historical financial data and generate a training set based on the historical financial data; The model evaluation module is used to evaluate various performance indicators of each pre-stored machine learning model based on the historical financial data and generate a performance evaluation result; The data acquisition module is used to acquire the financial data to be processed and the business risk preference corresponding to the financial data to be processed; The model screening module is configured to generate weights for each performance indicator based on the business risk preference corresponding to the financial data to be processed, and to screen a machine learning model as a target model based on the performance evaluation results and the weights corresponding to the performance indicators; The model training module is used to train the target model using the training set; The risk assessment module is used to perform data analysis on the financial data to be processed using the trained target model and generate risk assessment results.

2. The financial data analysis system based on machine learning according to claim 1, characterized in that: The data set generation module includes a data acquisition module, a data preprocessing module and a data set division module; The data collection module is used to collect a number of historical financial data; The data preprocessing module is used to preprocess the historical financial data, and the preprocessing includes standardization, missing value filling and data type conversion; The data set partitioning module is used to partition the pre-processed historical financial data into a training set and a test set according to a preset ratio.

3. The financial data analysis system based on machine learning according to claim 2, characterized in that: The data preprocessing module includes a data type identification module, a standardization processing module and a missing value filling module; The data type identification module is configured to identify the data type of each historical financial data, wherein the data type includes a numerical type and a categorical type; and is further configured to convert the data type of the corresponding historical financial data into a numerical type if the data type of the historical financial data includes a categorical type; The standardization processing module is used to perform standardization processing on various historical financial data; The missing value filling module is used to analyze whether there are missing values ​​in each historical financial data. If so, the missing values ​​are filled using the mean filling method.

4. The financial data analysis system based on machine learning according to claim 2, characterized in that: The model training module is also used to test the trained target model using a test set.

5. The financial data analysis system based on machine learning according to claim 1, characterized in that: The performance indicators include accuracy, precision, recall, F1 score and AUC value.

6. The financial data analysis system based on machine learning according to claim 1, characterized in that: The machine learning models include logistic regression model, support vector machine, random forest model, gradient boosting machine, K nearest neighbor and naive Bayes classifier.

7. The financial data analysis system based on machine learning according to claim 1, characterized in that: It also includes a data visualization module for generating data analysis charts based on the historical financial data.

Images