A modeling system for rapid development of scorecards, strategy trees, and blended models

Abstract

The application discloses a modeling system for quickly developing a scoring card, a strategy tree and a fusion model, which comprises an item module for saving and opening an item, a data import module for importing data and processing data variables, a data exploration module for obtaining a plurality of variables required to be analyzed by a user to output frequency distribution of each variable, a scoring card module for analyzing variables, generating a scoring card and displaying a result, a strategy tree module for developing a strategy tree model and a strategy matrix, and a machine learning module for providing a machine learning model, wherein the machine learning model comprises developing a gradient boosting tree, a random forest and a neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of modeling systems, in particular to a modeling system for quickly developing a scorecard, a strategy tree and a fusion model. BACKGROUND

[0002] At present, enterprises in the financial, telecommunication, Internet and other industries have massive data. Analysis and modeling of these data can help enterprises better understand customer status and mine customer demand, thereby being applied in risk management, fine marketing, customer lifecycle management and the like.

[0003] In the statistical modeling technology for customers, commonly used models mainly include a scorecard, a strategy tree and other machine learning models. For example, a bank can perform risk management through application scoring, behavior scoring and collection scoring, improve marketing effect through marketing response scoring, and perform fine management on customers through a strategy tree.

[0004] There is a technical solution for modeling through graphic in the prior art, but the applicant has found through research that it has the following disadvantages:

[0005] 1. In the prior art, for model development, modeling personnel usually need to master the corresponding statistical theory basis and need to write programs to realize it, which has the characteristics of high professional threshold and complex operation.

[0006] 2. In the prior art, for scorecard modeling, some necessary functional modules are lacking, such as automatic variable interaction and variable binning intelligent merging functions. If related operations are to be completed, modeling personnel must handle it manually, which makes the modeling efficiency low.

[0007] 3. In the prior art, for strategy tree modeling, there is a lack of a modeling system for developing a strategy tree through human-computer interaction with a mouse only, and the color display information of each node of the strategy tree of the existing modeling system is not comprehensive, such as the graph color of each node is only the color of the training set information, and the color display of the validation set information is lacking. This makes it difficult for modeling personnel to more intuitively observe the validation set information during the development process.

[0008] 4. In the prior art, for machine learning, there is a lack of a modeling system for quickly developing a machine learning model without programming. The selected variables of the machine learning model can be used as a reference for the selected variables of the scorecard or strategy tree model, and the performance indicators of the machine learning model can also be used as the upper limit of the performance of the scorecard or strategy tree model. Without this function, it is difficult to quickly screen the alternative variables of the scorecard or strategy tree, and it is difficult to quickly evaluate whether the scorecard or strategy tree model has room for improvement.

[0009] 5. The prior art lacks the function of rapid model fusion for model development. The developed model cannot be immediately displayed in the candidate variable box as a candidate variable for the next model. If the model needs to be fused, the modeler needs to complete the model fusion in the following steps:

[0010] (1) Develop each sub-model respectively;

[0011] (2) Manually calculate the prediction value of each sub-model outside the modeling software;

[0012] (3) Manually integrate the prediction values of each sub-model in a new data set;

[0013] (4) The modeler imports the new data set in the modeling software;

[0014] (5) Develop a new model based on the new data set to realize the fusion of the model.

[0015] The whole process is complicated and inefficient, and is prone to errors.

[0016] 6. The prior art lacks output of model rule code that can be directly deployed, such as SAS, R language, Python, SQL code, etc., which affects the deployment efficiency after the model development is completed. SUMMARY

[0017] In order to overcome the above technical defects, the present application provides a modeling system for rapidly developing scoring cards, strategy trees and fusion models.

[0018] In order to solve the above problems, the present application is implemented by the following technical scheme:

[0019] The present application discloses a modeling system for rapidly developing scoring cards, strategy trees and fusion models, comprising:

[0020] A project module for saving and opening a project;

[0021] A data import module for importing data and processing data variables;

[0022] A data exploration module for obtaining a plurality of variables required to be analyzed by a user to output the frequency distribution of each variable;

[0023] A scoring card module for analyzing variables, filtering variables, generating scoring cards and displaying results;

[0024] A strategy tree module for developing a strategy tree model and a strategy matrix;

[0025] The machine learning module provides machine learning models, including the development of gradient boosting trees, random forests, and neural networks.

[0026] As a preferred embodiment of the present invention, the project module includes an open project submenu and a save project submenu;

[0027] The "Open Project" submenu is used to open the project that the user wants to start and view the project's details;

[0028] The Save Item submenu is used to save items after adding, deleting, or modifying content.

[0029] As a preferred embodiment of the present invention, the scoring card module includes a variable interaction submenu, a variable binning submenu, a variable clustering submenu, a logistic regression submenu, a scoring card generation submenu, a PSI index calculation submenu, and a KS index calculation submenu.

[0030] The variable interaction submenu is used to allow pairwise interactions between variables and automatically adds the newly generated interactive variables to the training and validation data for subsequent scorecard development.

[0031] The variable binning menu is used to bin variables, calculate WOE values, and measure the predictive power of variables.

[0032] The variable clustering submenu is used to reduce the number of variables by clustering several variables.

[0033] The logistic regression submenu is used to perform logistic regression after removing several variables;

[0034] The "Generate Scoring Card" submenu is used to convert the logistic regression model into a scoring card after the model has been developed.

[0035] The PSI indicator calculation submenu is used to calculate a measure of the difference between actual data and expected data distribution.

[0036] The KS index calculation submenu is used to calculate the KS index for the selected variable and automatically generate a graph.

[0037] In a preferred embodiment of the present invention, the variable binning menu first initializes the binning of variables during runtime. If the intelligent merging option is selected, the system intelligently merges the bins after initialization, resulting in a smooth WOE curve trend that exhibits a monotonically rising, monotonically falling, U-shaped, or inverted U-shaped pattern.

[0038] As a preferred embodiment of the present invention, the machine learning module includes a gradient boosting tree submenu, a random forest submenu, and a neural network submenu;

[0039] The gradient boosting tree submenu is used to transform variables, obtain the optimal number of iterations, and generate a gradient boosting tree for the input variables.

[0040] The Random Forest submenu is used to randomly select candidate variables and generate the selected variables, rule codes, and graphical information for the Random Forest model.

[0041] The neural network submenu is used to generate the results and graphical information of the neural network model.

[0042] In a preferred embodiment of the present invention, the data import module includes an input unit, a raw data unit, a final data unit, an ID and target variable unit, a variable list setting, a data sample and program unit, and a log unit;

[0043] The input unit includes a main dataset or training set file selection box, a validation set file selection box, and a variable label file selection box.

[0044] In a preferred embodiment of the present invention, the input unit includes a main dataset or training set file selection box, a validation set file selection box, and a variable label file selection box, wherein the file format is CSV.

[0045] In a preferred embodiment of the present invention, the strategy tree module includes a strategy tree submenu and a strategy matrix submenu;

[0046] The strategy tree submenu is used to enable interactive development of strategy trees or decision trees based on modeling sample data.

[0047] The strategy matrix submenu is used to generate a cross-tabulation based on the two variables, and to develop an interactive rejection strategy based on the cross-tabulation.

[0048] As a preferred embodiment of the present invention, the strategy tree sub-menu further includes a custom variable unit, a pruning setting unit, a tree diagram color setting unit, an interval setting unit, a manual adjustment unit, a strategy tree result unit, and a log unit;

[0049] The pruning setting unit is used to manually set the minimum proportion of leaf nodes in the whole and the complexity parameter value;

[0050] The manual adjustment unit is used to set the maximum number of branches;

[0051] The tree diagram color setting unit is used to set the gradient color of the strategy tree diagram;

[0052] The strategy tree submenu will automatically generate a strategy tree after running. The strategy tree includes leaf nodes and non-leaf nodes. After the strategy tree is automatically generated, the growth or pruning of the strategy tree can be completed interactively by clicking the mouse.

[0053] The leaf nodes include options for automatically selecting variable growth and options for manually selecting variable growth.

[0054] The non-leaf nodes include pruning options and manual grouping options.

[0055] In a preferred embodiment of the present invention, the strategy matrix submenu further includes a variable grouping setting unit, a graphing setting unit, and a result unit;

[0056] The variable grouping setting unit is used to set variables to be grouped by percentile, by division point or by number of segments.

[0057] The plotting setting unit is used to set the gradient colors of the strategy matrix graph.

[0058] As a preferred embodiment of the present invention, the predicted values ​​of the developed scorecard model, strategy tree, or machine learning model can be directly embedded into the sample data that has been imported into the system. The predicted values ​​of one or more models that have been developed can be directly used as candidate variables for the next new model, thereby realizing model fusion and generating a fused model.

[0059] The fusion model refers to a new model generated by refitting different models using a specific algorithm to improve model performance. Model fusion refers to the operational process required to establish the fusion model. In this invention, model fusion is achieved by automatically adding the developed scorecard, policy tree, and machine learning model as a sub-model variable to the candidate variable boxes of relevant functional modules after the model is developed. These modules include: variable exploration, variable interaction, variable binning, variable clustering, PSI index calculation, KS index calculation, policy tree, policy matrix, and machine learning. When developing new scorecards, policy trees, and machine learning models, the developed sub-model variables can be selected from the candidate variable boxes, thereby developing a new model based on each sub-model, achieving the fusion of the sub-models, and generating a fusion model.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] 1. This invention includes a data import module, a data exploration module, a scorecard module, a strategy tree module, and a machine learning module, which can ensure that existing data can be directly modeled using only a mouse without programming, thus solving the shortcomings of existing technologies that require programming, have high professional thresholds, and are complex to operate.

[0062] 2. This invention enables highly convenient automatic variable interaction to maximize variable effectiveness and improve overall model performance, overcoming the shortcomings of existing technologies that require manual operation, repeated adjustments, and trial and error. This invention allows for highly convenient interactive binning operations and the plotting of WOE curves, intelligently merging WOE curves into one of the following forms: monotonically rising, monotonically falling, U-shaped, or inverted U-shaped. This significantly reduces the workload of manual binning adjustments by modelers and addresses the lack of intelligent binning merging in existing technologies.

[0063] 3. This invention can conveniently draw interactive decision tree (strategy tree) diagrams. The proportion of bad customers in the training set and validation set is displayed on the left and right sides of each node box by color, which solves the problem of the lack of color display of the validation set in the prior art.

[0064] 4. This invention can very conveniently develop machine learning models, quickly display the selected variables and the model's evaluation performance indicators, and serve as a reference for the selected variables and the upper limit of the performance indicators for scorecard and policy tree models.

[0065] 5. This invention enables very convenient model fusion, significantly reducing the workload of modelers in the model fusion process.

[0066] 6. This invention can conveniently output model rule code that can be directly deployed in other corresponding systems, reducing the workload of model deployment under existing technical conditions and improving deployment accuracy. Attached Figure Description

[0067] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0068] Fig. 1 This is the main block diagram of the modeling system for rapidly developing scorecards, strategy trees, and fusion models according to the present invention;

[0069] Fig. 2 This is an overall block diagram of the modeling system for rapidly developing scorecards, strategy trees, and fusion models according to the present invention. Detailed Implementation

[0070] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0071] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0072] Access devices and servers can be connected directly or indirectly via wired or wireless communication. Access devices can be terminals or servers. The target application runs on the access device. The target application is an application capable of making data requests to the server, such as social applications, payment applications, and game applications. The server can be an application server providing services to the target application, or a proxy server distinct from the application server corresponding to the target application. The server identifies whether each access device is malicious and intercepts data packets from malicious devices. When the server acts as a proxy server, it forwards data packets not originating from malicious devices to the application server. Terminals can be desktop terminals or mobile terminals. Mobile terminals can include smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0073] Example 1

[0074] like Figs. 1-2 As shown, this embodiment of the invention discloses a modeling system for rapidly developing scorecards, policy trees, and fusion models. This embodiment is applicable to all operations in establishing scorecard models and fusion models, including:

[0075] The project module M1 is used to save and open projects.

[0076] The project module M1 includes an open project submenu M1.1 and a save project submenu M1.2; the open project submenu is used to open the project that the user wants to start and view the project details; the save project submenu is used to save the project after adding, deleting, or modifying content.

[0077] Specifically, the project module M1 is located in the first menu position in the upper left corner of the homepage of the system described in this invention. When the system is opened, this module will be displayed. The "Save Project" submenu M1.1 allows users to reopen the software and continue working on the saved project if their computer malfunctions, without losing their previously saved work. The "Open Project" submenu M1.2 allows users to open the corresponding model project to view the details of previously developed models if they have forgotten the model details.

[0078] The data import module M2 is used to import data and process data variables.

[0079] The data import module M2 includes an input unit, a raw data unit, a final data unit, an ID and target variable unit, a variable list setting, a data sample and program unit, and a log unit.

[0080] The input unit includes selection boxes for the main dataset or training set file, validation set file, and variable label file. It also includes a selection box for automatically filling in missing values; if this selection box is checked, the missing value must be filled in. The input unit also has the function of automatically generating training and validation sets. If no validation set exists, the system can automatically generate training and validation sets by randomly sampling according to user settings, for example, selecting 70% as the training set and the remainder as the validation set. By enabling the automatic generation of training and validation sets, the system makes it more convenient for users and improves the user experience.

[0081] The raw data unit displays the number of rows and columns of the raw data; the final data unit displays the number of rows and columns of the data after successful import into the system; the ID and target variable unit displays information about the ID and target variables of the imported data; the view or set variable list unit displays information about the imported variables and allows for changes to variable roles, where variable role refers to the role of a variable in the modeling process, including ID variable, candidate variable, and target variable; the view imported data example unit provides a visual overview of the imported data; and the log unit displays information about the data import process.

[0082] The data exploration module M3 is used to obtain several variables that the user needs to analyze, and output the frequency distribution of each variable.

[0083] The data exploration module M3 includes a custom variable unit, a grouping setting unit, a plotting setting unit, a log unit, and a results unit. The custom variable unit displays all variables in the imported data, and the user can select several variables as the variables to be analyzed. The grouping setting unit provides percentile selection, split point selection, and segment number selection. The plotting setting unit allows the user to select whether to generate a frequency distribution graph. The results unit allows the user to view the frequency distribution table of the explored variables. The log unit displays the process log information of the data exploration and directly prints the frequency distribution table of the data exploration in the log.

[0084] The scoring card module M4 is used to analyze variables, generate scoring cards, and display results.

[0085] The scoring card module M4 includes the following submenu: Variable Interaction Submenu M4.1, Variable Binning Submenu M4.2, Variable Clustering Submenu M4.3, Logistic Regression Submenu M4.4, Scoring Card Generation Submenu M4.5, PSI Calculation Submenu M4.6, and KS Calculation Submenu M4.7. The Variable Interaction Submenu is used for pairwise interactions between variables and automatically adds the newly generated interactive variables to the training and validation data for subsequent scoring card development. The Variable Binning Submenu is used for binning variables, calculating WOE values, and measuring the predictive power of variables. The Variable Clustering Submenu is used to reduce the number of variables by clustering several variables. The Logistic Regression Submenu is used to perform logistic regression after removing several variables. The Scoring Card Generation Submenu is used to convert the logistic regression model into a scoring card. The PSI Calculation Submenu is used to calculate a measure of the difference between the actual data and the expected data distribution. The KS Calculation Submenu is used to calculate the KS metric for selected variables individually and automatically plot the results.

[0086] Specifically, the variable interaction submenu M4.1 includes a custom variable unit, an interaction result unit, and a log unit; the interaction result unit includes an interaction result management button and an interaction code button; the interaction result management button is used to view newly generated interactive variables, and clicking on a variable will automatically display the rule code and graphical information of that variable; the interaction code button is used to display interactive code in formats such as SAS, R, Python, and SQL, which is convenient for direct deployment in related system environments such as SAS, R, Python, and databases.

[0087] The variable binning menu M4.2 includes a custom variable unit, an initial binning setting unit, a merge binning setting unit, a binning result unit, a WOE direction unit, and a log unit. The variable binning menu has an IV (Information Value), which measures the correlation between a binary variable and a categorical variable and can be used to measure the predictive power of a variable in scorecard development. The initial binning setting unit includes two options: "Binding by Custom" and "Binding by Maximizing Discrimination." Selecting "Binding by Maximizing Discrimination" will result in a larger IV value than "Binding by Custom." The merge binning setting unit includes options: "Do Not Merge Bins," "Intelligent Merging by Training Set," "Intelligent Merging of Training and Validation Sets," and "Manual Adjustment." If "Intelligent Merging Bins" and "Manual Adjustment" are selected, the bins of numerical variables are intelligently merged, resulting in a smooth WOE curve trend, exhibiting a monotonically rising, monotonically falling, U-shaped, or inverted U-shaped pattern, significantly reducing manual adjustment workload. If the "Manual Adjustment" option is selected, a manual adjustment prompt will appear. The interface is user-friendly, allowing adjustments via mouse clicks. The WOE (Weight of Evidence) is used to calculate the logarithm of the ratio of good to bad customers for a specific value of a character variable or a segment of a continuous variable. The binning result unit includes a variable binning result management button and a binning code button. The binning result management button displays the binning results for all binned variables. It also includes a variable binning overview box, a detail box, and a graphical box. The variable binning overview box provides the training and validation set IV values, WOE correlation coefficient, and PSI value for each variable. The detail box displays the binning details for the selected variable, including the frequency distribution of each group, the bad customer rate, and the WOE value. The graphical box visually displays the binning status of the selected variable. The binning code button displays binning rule codes in formats such as SAS, R, Python, and SQL, which can be directly deployed and run in SAS, R, Python, and database environments. The log unit displays process information for variable binning.

[0088] The variable clustering submenu M4.3 includes a custom variable unit, a candidate variable setting unit, a clustering setting unit, a clustering result unit, and a log unit. The candidate variable setting unit is used to set thresholds for IV, PSI, and WOECor to automatically filter variables that meet the threshold conditions as candidate variables for variable clustering. WOECor is the correlation coefficient between the WOE values ​​of the training set and the validation set. The clustering setting unit is used to set the maximum number of clusters and how many variables to select in each cluster. The variable clustering result unit is used to display the variables selected after variable clustering. The log unit is used to display process information of variable clustering.

[0089] The logistic regression submenu M4.4 includes a custom variable unit, a candidate variable setting unit, a screening method setting unit, a model results unit, and a log unit. The candidate variable unit allows users to select whether to use clustering-based result variables and sets thresholds for IV, PSI, and WOECor to automatically filter variables that meet the corresponding conditions as candidate variables for logistic regression. The screening method setting unit includes radio buttons for forward inclusion, backward elimination, bidirectional screening, and not using stepwise regression. The model results unit displays the model results, including KS table data, PSI table data, selected variable data, and a graphical representation of the model's performance. KS is the Kolmogorov-Smirnov test, used to evaluate the model's risk discrimination ability; the indicator measures the maximum difference between the cumulative ratios of good and bad samples. The log unit displays logistic regression process information, showing the selected variables, the multicollinearity test of the variables, and the model's KS and PSI values.

[0090] The scorecard generation submenu M4.5 includes a variable selection unit, a score setting unit, a graphing setting unit, a scorecard result unit, and a log unit. The score setting unit is used to set the scorecard scale. The graphing setting unit is used to set the segmented range values, allowing user scores to be segmented according to the set values. The scorecard result includes a scorecard result management / view button, a score data view button, and a rule code view button. The scorecard result management / view button displays the KS, PSI, LIFT metrics of the scorecard, as well as the scorecard effect graph. LIFT is a metric for evaluating the effectiveness of a prediction model. The score data view button displays the score details corresponding to each ID. The rule code view button displays score rule codes in SAS, R, Python, and other formats. The log unit displays information about the scorecard generation process.

[0091] The PSI calculation submenu M4.6 includes a custom variable unit, a grouping setting unit, a plotting setting unit, a target variable unit, a result unit, and a log unit. The grouping setting unit refers to setting variable groups. The target variable unit allows you to select whether to calculate only the PSI with a target value of 0, only the PSI with a target variable value of 1, or not consider the target variable value. The PSI calculation formula is:

[0092]

[0093] Wherein, PSI is the group PSI index calculation, i is the i-th bin, and A i It is the actual frequency, the E i It is the expected frequency;

[0094] If 0 < PSI ≤ 0.1, there is no significant change between the actual and the expected, and no actual action needs to be taken; if 0.1 < PSI ≤ 0.25, some changes are found between the actual and the expected, and inspection is recommended; if PSI > 0.25, significant changes are found between the actual and the expected.

[0095] The KS index calculation sub-menu M4.7 includes a custom variable unit, a grouping setting unit, a graphing setting unit, a data set selection unit, a result unit, and a log unit; the data set selection unit refers to checking whether to calculate the KS based only on the training set, only on the validation set, or on all data sets. The larger the KS value, the stronger the ability of the data product to distinguish bad customers. The calculation of the KS value can be achieved by those skilled in the art according to the description of this application.

[0096] The newly generated scoring card model will be displayed in the alternative variable boxes of each functional module of this modeling system and can be directly used as an alternative variable for developing other models, thus facilitating the implementation of model fusion.

[0097] Example 2

[0098] As Figs. 1-2 As shown, the embodiment of the present invention discloses a modeling system for quickly developing a scoring card, a strategy tree, and a fusion model. This embodiment is applicable to all operations for establishing a strategy tree model and a fusion model, and its operations involve a project module M1, a data import module M2, a data exploration module M3, and a strategy tree module M5, where the project module M1, the data import module M2, and the data exploration module M3 are the same as those in Example 1.

[0099] The strategy tree module M5 is used to interactively develop a strategy tree model and a strategy matrix between humans and machines.

[0100] The strategy tree module M5 includes a strategy tree sub-menu M5.1 and a strategy matrix sub-menu M5.2.

[0101] The strategy tree sub-menu M5.1 is used to implement the interactive development of a strategy tree or a decision tree for the modeling sample data between humans and machines. After clicking the run button on the interface of the strategy tree sub-menu, an interactive graphical interface will be automatically popped up, and an initial strategy tree graph will be automatically drawn on the interface. At this time, through the way of clicking the mouse, the further growth or pruning of the strategy tree can be completed interactively between humans and machines. The strategy tree graph includes the following constituent elements: a root node box, an intermediate node box, a leaf node box, lines connecting each box, and variable split point information. The root node box is used to display the set of all data in the modeling data set, the intermediate node box is a judgment condition and displays the set of data in the modeling data set that satisfies all conditions from the root node to this node; the leaf node box is used to display the final decision result.

[0102] Clicking the root node or intermediate node box will bring up pruning and split point adjustment options. Selecting pruning will remove all downstream nodes based on that node, turning it into a leaf node. Selecting split point adjustment will first remove all downstream nodes based on that node, then display the split point information for the corresponding variable, allowing manual adjustment of the split point value. After adjustment, the node remains an intermediate node, and its direct downstream nodes become leaf nodes.

[0103] When the mouse clicks on the leaf node box, two options will pop up: automatic variable growth selection and manual variable growth selection. If the automatic variable growth selection option is selected, the program will automatically choose the optimal variable for growth and draw the strategy tree graph. If the manual variable growth selection option is selected, a variable list will pop up. This list can be sorted by importance from top to bottom or by variable name, providing a suggested selection for the user. After selection, the strategy tree graph will be automatically drawn. The newly generated strategy tree model will be displayed in the candidate variable boxes of each functional module of this modeling system. It can be directly used as candidate variables to develop other models, thus facilitating model fusion.

[0104] Furthermore, the strategy tree submenu also includes color information and node information. The color information is used by developers to observe changes in the data; the left side of each node box displays the color corresponding to the proportion of bad customers in the training set, and the right side displays the color corresponding to the proportion of bad customers in the validation set. The node information is used to display training set data and validation set data, facilitating quick model adjustments. The specific meanings of the values ​​in each node box in the strategy tree graph are as follows:

[0105] The first row shows the percentage of bad customers, the second row shows the number of bad customers, and the third row shows the total number of customers within the node. The training set data and the validation set data are separated by "|", where the left side of "|" is the training set and the right side is the validation set. If there is no validation set, only the training set data is displayed.

[0106] The Strategy Matrix submenu M5.2 is used to generate cross-tabs based on grouping two variables, enabling interactive rejection strategy development. Clicking the "Run" button in the Strategy Matrix submenu will automatically pop up a strategy matrix diagram composed of multiple rectangles. This strategy matrix diagram is created by the user selecting two variables and setting their respective split points. One variable is used as the horizontal axis, and the other as the vertical axis. Lines parallel to the coordinate axes are drawn at the split points, resulting in a strategy matrix diagram composed of multiple small rectangles. When the user clicks on a small rectangle in the strategy matrix diagram, a blue dot is automatically marked to indicate that the customer represented by that rectangle will be rejected. Clicking the rectangle again removes the dot. The rejection rate, acceptance rate, number of rejected customers, and number of accepted customers are displayed above the image. Finally, the corresponding rule code in SAS, R, Python, and SQL formats is generated. The specific meanings of the numerical values ​​in the strategy matrix image are as follows:

[0107] 1. With a target variable, the color gradually changes according to the percentage of bad customers in each rectangle.

[0108] The values ​​in the boxes represent: the first row is the percentage of bad customers, the second row is the number of bad customers, and the third row is the total number of customers in the boxes. The training set data and the validation set data are separated by "|", where the left side of "|" is the training set and the right side is the validation set. If there is no validation set, only the training set data is displayed.

[0109] 2. In the absence of a target variable, the color gradually changes according to the proportion of customers in each rectangle to the total number of customers.

[0110] Meaning of the values ​​in the box: The value represents the total number of customers in the box. The training set data and the validation set data are separated by "|". The left side of "|" is the training set and the right side is the validation set. If there is no validation set, only the training set data will be displayed.

[0111] Example 3

[0112] like Figs. 1-2 As shown, this embodiment of the invention discloses a modeling system for rapidly developing scorecards, policy trees, and fusion models. This embodiment is applicable to all operations of establishing machine learning models and fusion models. Its operations involve project module M1, data import module M2, data exploration module M3, and policy tree module M6, wherein project module M1, data import module M2, and data exploration module M3 are the same as those in embodiment 1.

[0113] The Machine Learning Module M6 is used to develop gradient boosting trees, random forests, and neural network machine learning models.

[0114] The machine learning module M6 includes a gradient boosting tree submenu M6.1, a random forest submenu M6.2, and a neural network submenu M6.3.

[0115] The gradient boosting tree submenu M6.1 is used to develop models using the gradient boosting tree algorithm and output related graphs. The menu interface includes a gradient boosting tree setting unit, which is used to set the relevant parameters of the gradient boosting tree. The relevant parameters include the number of trees, learning rate, interaction depth of each iteration, training ratio of each iteration, ratio used for training, minimum number of samples per node, and number of cross-validation folds.

[0116] The Random Forest submenu M6.2 is used to develop models using the Random Forest algorithm and output related graphs. The menu interface includes a Random Forest settings unit, which is used to set relevant parameters of the Random Forest. The relevant parameters include the number of trees, the number of candidate variables randomly selected for each split of each tree, whether to sample with replacement, prior weights, the number of samples sampled each time, the minimum number of samples per node, and the maximum number of nodes.

[0117] The neural network submenu M6.3 uses the random forest algorithm to develop models and output related graphs. The menu interface includes a neural network settings unit, which is used to set the relevant parameters of the neural network. The relevant parameters include hidden values, limit values, and maximum step size values.

[0118] The newly generated machine learning model will be displayed in the candidate variable boxes of each functional module of this modeling system. It can be directly used as a candidate variable to develop other models, thereby facilitating model fusion.

[0119] In summary, this invention, through the data import module M2 and the data exploration module M3, ensures the accuracy of the variables required for the subsequent development of scorecards, policy trees, and machine learning. The scorecard module M4 enables the development of scorecards with stability and predictive capabilities, and uses the predicted values ​​of the scorecards as candidate variables for developing the fusion model. The policy tree module M5 enables interactive development of policy trees and policy matrices, and uses the predicted values ​​of the policy trees as candidate variables for the fusion model. The machine learning module M6 enables the development of machine learning models, and uses the predicted values ​​of the machine learning models as candidate variables for the fusion model.

[0120] Other aspects of the modeling system for rapidly developing scorecards, strategy trees, and fusion models described in this embodiment can be found in existing technologies.

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] Various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A modeling system for rapidly developing scorecards, policy trees, and fusion models, characterized in that, include: The Projects module is used to save and open projects; The data import module is used to import data and process data variables; The data exploration module is used to obtain several variables that the user needs to analyze, and output the frequency distribution of each variable; The scoring card module is used to analyze variables, generate scoring cards, and display results. The scoring card module includes a variable interaction submenu, a variable binning submenu, a variable clustering submenu, a logistic regression submenu, a scoring card generation submenu, a PSI index calculation submenu, and a KS index calculation submenu. The variable interaction submenu is used to allow pairwise interactions between variables and automatically adds the newly generated interactive variables to the training and validation data for subsequent scorecard development. The variable binning menu is used to measure the predictive power of variables; The variable clustering submenu is used to reduce the number of variables by clustering several variables. The logistic regression submenu is used to perform logistic regression after removing several variables; The "Generate Scoring Card" submenu is used to convert the logistic regression model into a scoring card after the model has been developed. The PSI indicator calculation submenu is used to calculate a measure of the difference between actual data and expected data distribution. The KS index calculation submenu is used to calculate the KS index for the selected variable individually and to automatically draw a graph. The variable binning menu first performs initial binning on the variables during runtime; if the smart merging option is selected, the binning is intelligently merged after the initial binning, making the WOE curve trend smooth and showing a monotonically rising, monotonically falling, U-shaped or inverted U-shaped shape. The strategy tree module is used to develop strategy tree models and strategy matrices. The strategy tree module includes a strategy tree submenu and a strategy matrix submenu. The strategy tree submenu is used to enable interactive development of strategy trees or decision trees based on modeling sample data. The strategy matrix submenu is used to generate a cross-tabulation based on the two variables, and to develop a human-computer interactive rejection strategy based on the cross-tabulation. The strategy tree submenu also includes a custom variable unit, a pruning setting unit, a tree diagram color setting unit, an interval setting unit, a manual adjustment unit, a strategy tree result unit, and a log unit. The pruning setting unit is used to manually set the minimum proportion of leaf nodes in the whole and the complexity parameter value; The manual adjustment unit is used to set the maximum number of branches; The tree diagram color setting unit is used to set the gradient color of the strategy tree diagram; The strategy tree submenu will automatically generate a strategy tree after running. The strategy tree includes leaf nodes and non-leaf nodes. After the strategy tree is automatically generated, the growth or pruning of the strategy tree can be completed interactively by clicking the mouse. The leaf nodes include options for automatically selecting variable growth and options for manually selecting variable growth. The non-leaf nodes include pruning options and manual grouping options; A machine learning module for providing machine learning models, including developing gradient boosting trees, random forests, and neural networks; The predicted values ​​of the developed scorecard model, policy tree, or machine learning model are directly embedded in the sample data imported into the system, and the model will be automatically added as a sub-model variable to the candidate variable box of the relevant functional modules, including: variable exploration, variable interaction, variable binning, variable clustering, PSI index calculation, KS index calculation, policy tree, policy matrix, and machine learning. When developing new scorecards, policy trees, and machine learning models, select the variables of each developed sub-model from the candidate variable box, thereby developing a new model based on each sub-model, achieving the fusion of the sub-models, and generating a fused model.

2. The modeling system for rapidly developing scorecards, strategy trees, and fusion models according to claim 1, characterized in that: The project module includes an "Open Project" submenu and a "Save Project" submenu; The "Open Project" submenu is used to open the project that the user wants to start and view the project's details; The Save Item submenu is used to save items after adding, deleting, or modifying content.

3. The modeling system for rapidly developing scorecards, strategy trees, and fusion models according to claim 1, characterized in that: The machine learning module includes a gradient boosting tree submenu, a random forest submenu, and a neural network submenu. The gradient boosting tree submenu is used to transform variables, obtain the optimal number of iterations, and generate a gradient boosting tree for the input variables. The Random Forest submenu is used to randomly select candidate variables and generate the selected variables, rule codes, and graphical information for the Random Forest model. The neural network submenu is used to generate the results and graphical information of the neural network model.

4. The modeling system for rapidly developing scorecards, strategy trees, and fusion models according to claim 1, characterized in that: The data import module includes an input unit, a raw data unit, a final data unit, an ID and target variable unit, a variable list setting, a data sample and program unit, and a log unit. The input unit includes a main dataset or training set file selection box, a validation set file selection box, and a variable label file selection box.

5. The modeling system for rapidly developing scorecards, strategy trees, and fusion models according to claim 1, characterized in that: The strategy matrix submenu also includes a variable grouping setting unit, a graphing setting unit, and a result unit; The variable grouping setting unit is used to set variables to be grouped by percentile, by division point or by number of segments. The plotting setting unit is used to set the gradient colors of the strategy matrix graph.

Images