Cross-language implementation method and system for internet financial risk control engine

Abstract

The invention discloses a cross-language implementation method and system for an internet financial risk control engine, and the method comprises the following steps: a risk control data analyzer reads data of a business data slave library by using a first database query statement, carries out the model training by using the data as source data of model training, and generates a user tag attribute dictionary and a model file; the online business system triggers a user risk control process, obtains a user tag attribute dictionary, uses a second database query statement to query data from an online business data main library, and generates a corresponding user tag portrait; wherein the logic structure of the second database query statement is the same as that of the first database query statement; the online service system requests a risk control calculation engine, the risk control calculation engine reads the model file, the user tag portrait serves as an input parameter of the risk control calculation engine, and a calculation result is obtained; and outputting a calculation result to an application layer for application. According to the method, the consistency of data and logic is guaranteed when the business system and the analysis system use different languages.

Description

Technical Field

[0001] The present invention belongs to the technical field of financial risk control, and in particular to a cross-language implementation method and system for an Internet financial risk control engine. Background Art

[0002] In the field of Internet finance, the risk control engine, as a core system to ensure business security and compliance, undertakes the key responsibilities of comprehensive risk identification, real-time decision-making and prevention and control. The risk control engine supports the healthy development of financial business by reducing bad debt rates, improving operational efficiency and improving user experience. In order to improve the calculation accuracy and efficiency of the risk control engine, in recent years, machine learning algorithm models including deep learning have been widely used in the field of financial risk control. For example, the Chinese invention patent application with publication number CN116993513A discloses a financial risk control model interpretation method, device and computer equipment, the method comprising: obtaining financial risk control risk samples, the financial risk control risk samples including data information suspected of financial risk; inputting the financial risk control risk samples into an interpretable model, determining the financial risk feature items through the interpretable model, and extracting the eigenvalues ​​corresponding to the financial risk feature items in the interpretable model, wherein the interpretable model is trained based on an interpretable training set, the interpretable training set includes training risk samples and risk prediction results to be explained, the risk prediction results to be explained are obtained by inputting the training risk samples into the financial risk control model to be explained and then outputting them, the risk prediction results to be explained include the probability that the financial risk control risk samples exist in the financial risk control risk samples; determining the interpretation information of the financial risk control model to be explained based on the eigenvalues.

[0003] Existing risk control engines are generally composed of a data collection layer, a data processing layer, a rule engine layer, a model calculation layer, and a decision output layer. They are usually implemented using a combination of Java and Python. For example, the data collection and processing layer usually uses Java to process large-scale data with the help of ETL tools and big data frameworks (such as Spark and Flink); while the model calculation layer usually uses Python to take advantage of Python's rich machine learning algorithm library. However, this multi-language combination architecture faces the following key issues in practical applications:

[0004] Cross-language system calls are complex and difficult to maintain. Since different modules in the risk control engine system are developed in Java and Python, cross-language calls complicate development, testing and maintenance. When problems occur in the system, developers need to troubleshoot in Java and Python environments, which increases the difficulty of troubleshooting and reduces the iteration speed, especially for start-ups or small Internet financial companies.

[0005] Data and logic consistency issues. The entire process from model training to online risk control model application is long and involves multiple nodes, which may cause deviations in data consistency and logical consistency, affecting the accuracy and reliability of the risk control model. Summary of the invention

[0006] The present invention provides a cross-language implementation method and system for an Internet financial risk control engine, aiming to solve the problems of high system call complexity and maintenance difficulty generated when online business and risk control analysis are directly implemented across languages ​​in existing financial risk control systems, as well as consistency problems in data and logic.

[0007] In order to solve the above technical problems, the cross-language implementation method proposed in the present invention comprises the following steps:

[0008] The risk control data analyst uses the first database query statement to read the data from the business data repository as the source data for model training, and generates a user label attribute dictionary and a model file;

[0009] The online business system triggers the user risk control process, obtains the user tag attribute dictionary, uses the second database query statement to query data from the online business data master database, and generates a corresponding user tag portrait; wherein the logical structure of the second database query statement is the same as that of the first database query statement;

[0010] The online business system requests the risk control calculation engine, which reads the model file, uses the user label portrait as an input parameter of the risk control calculation engine, and obtains the calculation result;

[0011] The calculation results are output to the application layer for application.

[0012] Preferably, the data source of the business data slave database is the online business data master database, and the data of the online business data master database is synchronized to the business data slave database in real time.

[0013] Preferably, the user tag attribute dictionary is a structured file containing multiple user tags and user features, and the fields of the structured file include tag name, attribute description, attribute value, and weight.

[0014] Preferably, the model file is a pmml model file, and the structure of the pmml model file includes model type, input fields, model parameters, output fields and calculation logic.

[0015] Preferably, the pmml model file includes a pmml framework integrated with a calculation method. When the risk control calculation engine reads the model file, the model file is parsed through the integrated pmml framework and loaded into the memory; when the risk control calculation engine calculates, it directly calls the calculation method integrated in the pmml framework.

[0016] Preferably, the risk control calculation engine reads one or more model files. In response to multiple model files, the risk control calculation engine traverses the model, calls the calculation method of the framework with the user portrait as an input parameter, obtains the results of each model, and then assembles the results of each model as the calculation result of the risk control calculation engine.

[0017] Preferably, the second database query statement is generated by:

[0018] Obtain a user tag attribute dictionary and a first database query statement, extract a user tag field name from the user tag attribute dictionary, replace a user code placeholder in the first database query statement with the current user code, and use the replaced database query statement as a second database query statement.

[0019] Preferably, the online business system requests the risk control calculation engine through an RPC or HTTP request.

[0020] Preferably, the models of the model training include decision tree, logic analysis, random forest, vector machine, and cubic verification model.

[0021] Another aspect of the present invention further provides a cross-language implementation system for an Internet financial risk control engine, the system being used to implement the above-mentioned implementation method, comprising:

[0022] The risk control analysis module is used by risk control data analysts to read the data from the business data repository and use it as the source data for model training to generate a user label attribute dictionary and model files.

[0023] The online business module obtains the user tag attribute dictionary according to the set rules, uses the second database query statement to query data from the online business data master database, and generates a corresponding user tag portrait; wherein the logical structure of the second database query statement is the same as that of the first database query statement;

[0024] The risk control calculation engine is used to receive online business system requests, load and parse the model file, use the user tag portrait as an input parameter to perform model calculation, and generate risk control calculation results;

[0025] The storage module is used to store the SQL statements generated by risk control data analysts, the user label attribute dictionary generated by training, and the model files;

[0026] The database module includes an online business data master database and a business data slave database. The online business data master database is used to store the data of the online business module, and the business data slave database is used to provide data for model training and analysis for the risk control analysis module, and also synchronizes the data of the online business data master database to the business data slave database in real time.

[0027] Compared with the prior art, the present invention has the following technical effects:

[0028] 1. The cross-language implementation method proposed in the present invention realizes the cross-language system call interaction between the online business module and the risk control analysis module through the user tag attribute dictionary, model file and database query language with the same logical structure, so that the risk control analysts can focus on the risk control model training and analysis of the offline process, and the business system development and testing personnel only need to focus on the iteration and maintenance of the online process system, without worrying about the internal logic of the risk control model. They can focus on their familiar fields to carry out work, and the complexity and maintenance difficulty are much lower, and the iteration efficiency of both can be improved.

[0029] 2. The cross-language implementation method proposed in the present invention synchronizes the online business data master database to the business data slave database used by risk control analysts in real time, ensuring that the data source for risk control analysts' training is highly consistent with the online data source, greatly avoiding data distortion and improving the reliability of the trained model.

[0030] 3. The cross-language implementation method proposed in the present invention uses database query statements with the same logical structure to ensure the consistency of the user tag value calculation logic, and ensures the consistency of the model calculation logic through the pmml model file, thereby ensuring the high consistency of data and logic in the cross-language system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the implementation method of the present invention. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in combination with specific embodiments of the present application and with reference to the accompanying drawings.

[0033] Embodiment 1

[0034] The cross-language implementation method of the Internet financial risk control engine, such as Figure 1 As shown, the steps include steps 1 to 4:

[0035] Step 1: The risk control data analyst uses the first database query statement to read the data from the business data library as the source data for model training, and generates a user label attribute dictionary and a model file.

[0036] In order to use Python's powerful and mature algorithm library to achieve model building, training and calculation, in this embodiment, the risk control data analyst uses Python language to read data from the business data library by writing Python scripts and using database query statements.

[0037] The training result output has two parts:

[0038] A highly discriminative user tag attribute dictionary is generated and uploaded to the cloud storage in an Excel file; the user tag attribute dictionary is a structured file containing multiple user tags and user features, and the fields of the structured file include tag name, attribute description, attribute value, and weight. Tag names such as user risk level and consumption preference are used to identify the specific meaning of each tag; attribute descriptions are descriptive explanations of the tags, which facilitate understanding of the source and function of the tags; attribute values ​​can be numerical values, categories, or Boolean values, etc., indicating the specific numerical values ​​or classifications of the user on the tag; weights indicate the influence weight of this tag in the overall model.

[0039] The model file is a pmml (Predictive Model Markup Language) model file, which is uploaded to the cloud storage. The file contains the user attribute labels required for model calculation and the corresponding calculation logic. The structure of the pmml model file includes model type, input field, model parameters, output field and calculation logic. Model types such as decision tree, logical analysis, random forest, vector machine, cubic verification model, clarify the algorithm type of the model; input field is the user label and its type required by the model, corresponding to the user label attribute dictionary; model parameters include model coefficients, thresholds, tree structures, etc., which are model parameters used for actual calculation and results; output field defines the output of model prediction, such as risk score or credit rating; calculation logic defines how the model processes input fields and outputs the calculation logic of the prediction result. The pmml model file includes a pmml framework integrated with a calculation method. When the risk control calculation engine reads the model file, the model file is parsed through the integrated pmml framework and loaded into the memory; the risk control calculation engine directly calls the calculation method integrated in the pmml framework when calculating.

[0040] Through the high-discrimination user label attribute dictionary and pmml model file, the entire process ensures the accuracy of data content. Risk control analysts only need to focus on their own model training and analysis. They do not need to interact with any online systems or communicate too much with business developers. This can greatly improve the iteration efficiency of risk control analysts' models.

[0041] In this step, the models trained include decision tree, logic analysis, random forest, vector machine, and cubic verification model.

[0042] Step 2: The online business system triggers the user risk control process, obtains the user tag attribute dictionary, uses the second database query statement to query data from the online business data master database, and generates the corresponding user tag portrait.

[0043] In order to make use of common etl and big data framework components (spark, flink), in this embodiment, the online business system is built using the java language.

[0044] In this embodiment, the logical structure of the second database query statement is the same as that of the first database query statement. Specifically, the method for generating the second database query statement is:

[0045] Obtain a user tag attribute dictionary and a first database query statement, extract a user tag field name from the user tag attribute dictionary, replace a user code placeholder in the first database query statement with the current user code, and use the replaced database query statement as a second database query statement.

[0046] Use the second database query statement to obtain the label value of the current user. After all label values ​​are calculated, obtain the complete user label portrait and store the user portrait data for subsequent tracking and risk control analysis.

[0047] The data source of the business data slave database is the online business data master database, and the data of the online business data master database is synchronized to the business data slave database in real time. The online business data master database is synchronized in real time to the business data slave database used by risk control analysts to ensure that the data source of risk control analysts' training is highly consistent with the online data source, which greatly avoids data distortion and improves the reliability of the trained model.

[0048] Step three: The online business system requests the risk control calculation engine. The risk control calculation engine reads the model file, uses the user label portrait as an input parameter of the risk control calculation engine, and obtains the calculation result.

[0049] The risk control engine system pulls the PMML file from the cloud storage and parses the file through the integrated PMML framework. When calculating, the risk control engine system directly calls the calculation method integrated in the PMML framework without the need for additional processing steps, so that the calculation logic of the model maintains a high degree of accuracy and consistency with what the risk control analysts want.

[0050] Specifically, the risk control calculation engine needs to integrate a PMML parsing framework, such as JPMML (for Java environment), PyPMML (for Python environment), etc. Taking JPMML as an example, by integrating JPMML, the system can directly read and understand various models in PMML files (such as decision trees, logical analysis, random forests, vector machines, cubic verification models, etc.).

[0051] At runtime, the risk control calculation engine will pull the latest PMML model file from the cloud storage and load the PMML file into the system memory through the PMML parsing framework. This file contains the model's structure, input and output fields, parameters, and calculation logic. Since the input fields required by the model are defined in the PMML file (usually the tags defined in the user tag attribute dictionary), the risk control engine system will match these input fields with the actual business data. The PMML parsing framework will check the input field type and data format to ensure that they are consistent with the model requirements.

[0052] Once the PMML file is loaded and parsed, the risk control engine system can directly call the scoring method or prediction method provided by the framework. For example, in JPMML, the evaluate() method can be directly called through the Evaluator interface to pass in user data and generate model calculation results. Since the PMML framework already includes the interpretation and execution of the model logic, this call does not require manual writing of the calculation logic, ensuring consistency with the logic used by the risk control analyst during training.

[0053] The final calculation result is the risk score, classification label or other decision result predicted by the model. The risk control engine system returns these results to the application layer for real-time risk control judgment. Due to the standardized support of PMML files and parsing frameworks, the system can flexibly switch models without modifying business logic.

[0054] In this embodiment, the risk control calculation engine reads one or more model files. In response to multiple model files, the risk control calculation engine traverses the model, calls the calculation method of the framework with the user portrait as an input parameter, obtains the results of each model, and then assembles the results of each model as the calculation result of the risk control calculation engine.

[0055] The online business system requests the risk control calculation engine through rpc or http requests.

[0056] In this step, those skilled in the art will appreciate that the risk control calculation engine may set an exception handling mechanism during the calculation process to return a default risk control score or an error mark when the model calculation fails or data is missing.

[0057] The model file of this embodiment can be updated regularly by risk control data analysts, and the online business system can use the latest model file in real time through cloud storage to achieve seamless model iteration.

[0058] Step 4: Output the calculation results to the application layer for application.

[0059] Embodiment 2

[0060] This embodiment is a cross-language implementation system of an Internet financial risk control engine, and the system is used to implement the implementation method described in the first embodiment, including:

[0061] The risk control analysis module is used by risk control data analysts to read the data from the business data repository and use it as the source data for model training to generate a user label attribute dictionary and model files.

[0062] The online business module obtains the user tag attribute dictionary according to the set rules, uses the second database query statement to query data from the online business data master database, and generates a corresponding user tag portrait; wherein the logical structure of the second database query statement is the same as that of the first database query statement;

[0063] The risk control calculation engine is used to receive online business system requests, load and parse the model file, use the user tag portrait as an input parameter to perform model calculation, and generate risk control calculation results;

[0064] The storage module is used to store the SQL statements generated by risk control data analysts, the user label attribute dictionary generated by training, and the model files;

[0065] The database module includes an online business data master database and a business data slave database. The online business data master database is used to store the data of the online business module, and the business data slave database is used to provide data for model training and analysis for the risk control analysis module, and also synchronizes the data of the online business data master database to the business data slave database in real time.

[0066] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A cross-language implementation method for an Internet financial risk control engine, characterized in that: The following steps are involved: The risk control data analyst uses the first database query statement to read the data from the business data repository as the source data for model training, and generates a user label attribute dictionary and a model file; The online business system triggers the user risk control process, obtains the user tag attribute dictionary, uses the second database query statement to query data from the online business data master database, and generates the corresponding user tag portrait; The logical structure of the second database query statement is the same as that of the first database query statement; The online business system requests the risk control calculation engine, which reads the model file, uses the user label portrait as an input parameter of the risk control calculation engine, and obtains the calculation result; The calculation results are output to the application layer for application.

2. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The data source of the business data slave database is the online business data master database, and the data of the online business data master database is synchronized to the business data slave database in real time.

3. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The user tag attribute dictionary is a structured file containing multiple user tags and user features. The fields of the structured file include tag name, attribute description, attribute value, and weight.

4. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The model file is a pmml model file, and the structure of the pmml model file includes model type, input fields, model parameters, output fields and calculation logic.

5. The cross-language implementation method of the Internet financial risk control engine according to claim 4 is characterized in that: The pmml model file includes a pmml framework integrated with a calculation method. When the risk control calculation engine reads the model file, the model file is parsed through the integrated pmml framework and loaded into the memory; when the risk control calculation engine calculates, it directly calls the calculation method integrated in the pmml framework.

6. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The risk control calculation engine reads one or more model files. In response to multiple model files, the risk control calculation engine traverses the model, calls the calculation method of the framework with the user portrait as an input parameter, obtains the results of each model, and then assembles the results of each model as the calculation result of the risk control calculation engine.

7. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The method for generating the second database query statement is: Obtain a user tag attribute dictionary and a first database query statement, extract a user tag field name from the user tag attribute dictionary, replace a user code placeholder in the first database query statement with the current user code, and use the replaced database query statement as a second database query statement.

8. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The online business system requests the risk control calculation engine through rpc or http requests.

9. The cross-language implementation method of the Internet financial risk control engine according to claim 1 is characterized in that: The models trained by the model include decision tree, logic analysis, random forest, vector machine, and cubic verification model.

10. The cross-language implementation system of the Internet financial risk control engine is characterized by: The system is used to implement the implementation method according to any one of claims 1 to 9, including: The risk control analysis module is used by risk control data analysts to read the data from the business data repository and use it as the source data for model training to generate a user label attribute dictionary and model files. The online business module obtains the user tag attribute dictionary according to the set rules, uses the second database query statement to query data from the online business data master database, and generates a corresponding user tag portrait; wherein the logical structure of the second database query statement is the same as that of the first database query statement; The risk control calculation engine is used to receive online business system requests, load and parse the model file, use the user tag portrait as an input parameter to perform model calculation, and generate risk control calculation results; The storage module is used to store the SQL statements generated by risk control data analysts, the user label attribute dictionary generated by training, and the model files; The database module includes an online business data master database and a business data slave database. The online business data master database is used to store the data of the online business module, and the business data slave database is used to provide data for model training and analysis for the risk control analysis module, and also synchronizes the data of the online business data master database to the business data slave database in real time.

Images