Credit scoring system, credit scoring method, and model generation system

Abstract

A credit scoring system is provided that utilizes a variety of data sources, including static and dynamic information about the person being evaluated, to estimate the person's credit score. [Solution] A credit scoring system capable of estimating the credit score of a person being evaluated comprises an acquisition unit that acquires first data including static information of the person being evaluated and second data including dynamic information of the person being evaluated, a processing unit that performs at least one of data cleansing, encoding, scaling, feature selection, and dimensionality reduction on the first data and second data acquired by the acquisition unit, an estimation unit that inputs the data processed by the processing unit into a machine learning model to estimate a credit score, and an output unit that outputs the credit score estimated by the estimation unit.

Description

[Technical Field]

[0001] The present invention generally relates to a technique for estimating a credit score of an appraisee. [Background technology]

[0002] In recent years, financial institutions and other institutions have been increasingly adopting systems to objectively and quickly evaluate applicants' creditworthiness in their credit decision-making processes. Conventional credit decisions have been based primarily on standardized data such as annual income, occupation, number of loans, and deposit balances, but this makes it difficult to accurately reflect the applicant's individual circumstances and fluctuating circumstances. Another problem with conventional systems is that it takes time to obtain credit scoring results.

[0003] Against this background, there has been active development of credit assessment systems that utilize AI and machine learning technologies. Patent Document 1 discloses a system that uses AI and machine learning models to analyze applicant screening data at the time of lease contract signing and past credit assessment data, and estimates credit results. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-114573 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, the data is limited to application data in lease contracts and past credit screening data, and the diversity of data sources is lacking, so there is room for improvement in making comprehensive and highly accurate credit decisions.

[0006] The present invention has been made based on the above, and proposes a credit scoring system etc. that can estimate the credit score of the person being evaluated by utilizing a variety of data sources including static and dynamic information of the person being evaluated. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a credit scoring system capable of estimating the credit score of a person being evaluated, comprising an acquisition unit that acquires first data including static information of the person being evaluated and second data including dynamic information of the person being evaluated, a processing unit that performs at least one of data cleansing, encoding, scaling, feature selection, and dimensionality reduction on the first data and second data acquired by the acquisition unit, an estimation unit that inputs the data processed by the processing unit into a machine learning model to estimate a credit score, and an output unit that outputs the credit score estimated by the estimation unit. [Effects of the Invention]

[0008] According to the present invention, it is possible to estimate the credit score of a person to be appraised by utilizing various data sources including static information and dynamic information of the person to be appraised. Problems, means, and effects not described above will become clear from the following description of the embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a credit scoring system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a model inference server according to the first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a model learning server according to the first embodiment. [Figure 4] FIG. 3 is a diagram illustrating an example of a data item group according to the first embodiment. [Figure 5] FIG. 2 is a diagram illustrating an example of pre-processing in the credit scoring system according to the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of the overall processing in the credit scoring system according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of a screen according to the first embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a screen according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] (I) First embodiment The configurations, processing procedures, and other elements disclosed below are provided to explain embodiments of the present invention and are not intended to limit the present invention. The description based on the drawings is intended to facilitate understanding, and the shape, arrangement, function, and the like of elements may be omitted or simplified as necessary. The present invention is not limited to one or more of the disclosed embodiments, and may be realized by functionally or structurally equivalent means or other means achieving a technically similar purpose, within the scope that can be understood by those skilled in the art from the entire specification. Unless otherwise specified, each element in this specification is interpreted as including "at least one." Furthermore, words described in the singular include the plural, and words described in the plural include the singular. Furthermore, terms used in this specification are not limited to a specific meaning unless the context is clear, and should be interpreted appropriately by those skilled in the art. For example, the expression "including" is interpreted as meaning not limited to the listed items.

[0011] In this specification, "static information" refers to attribute information whose values ​​are unlikely to change in the short term, such as name, date of birth, address, occupation, annual income, home ownership status, and past credit history. "Dynamic information" refers to sequential or behavioral information whose values ​​or distribution may change over time, such as deposits and withdrawals from a savings account, payment history of utility and communication charges, purchase history of goods and services, electronic money usage history, and survey responses. The present embodiment is characterized by utilizing a variety of data sources including such static and dynamic information.

[0012] Here, "learning" refers to the process of using feature vectors obtained by performing preprocessing such as data cleansing, encoding, scaling, feature selection, and dimensionality reduction on training data as input, and estimating model parameters using machine learning algorithms such as gradient boosting decision trees, logistic regression, neural networks, or ensembles of these. Furthermore, "optimization" refers to the process of searching for hyperparameters (learning rate, tree depth, number of leaf nodes, regularization coefficient, etc.) through cross-validation, and selecting settings that satisfy an objective function (e.g., maximizing AUC or minimizing loss weighted for false positives and false negatives) based on performance indices such as AUC (Area Under the Receiver Operating Characteristic Curve), accuracy, precision, recall, and F1 (F1-score).

[0013] In this embodiment, evaluation indices for the entire model, such as Accuracy, AUC, Precision, Recall, and F1, are calculated during the learning and validation processes, and the threshold-independent discrimination ability (AUC) and the balance between false positives and false negatives (F1, etc.) are verified in accordance with business requirements. These evaluation results can be output on the evaluation display screen and result list screen together with individual contribution information (e.g., SHAP value), which contributes to speeding up review decisions and ensuring explainability.

[0014] According to this embodiment, features that integrate static and dynamic information are uniformly applied during inference under the preprocessing parameters and model parameters determined during training, thereby achieving highly efficient inference processing while maintaining consistency in the format and dimensions of the input data. As a result, in this embodiment, the inference processing time per case is approximately 0.05 to 0.2 seconds, which is approximately 5 to 10 times faster than conventional scoring (rule-based or simple regression: 0.5 to 2 seconds) (example of a typical x86 server environment, single-case inference). However, this value may vary depending on the hardware configuration, number of features, whether batching is used, etc.

[0015] Furthermore, this embodiment can be configured to detect performance degradation and changes in data distribution (concept drift) based on periodic saving and monitoring of evaluation indices, and determine whether or not a model needs to be recreated. This avoids unnecessary re-learning and updates the model only when necessary, which is expected to minimize operational costs.

[0016] Additionally, while gradient boosting decision trees (e.g., LightGBM), logistic regression, neural networks, etc. can be used as model types, for feature spaces that include high-dimensional and nonlinear relationships, using gradient boosting decision trees optimized by cross-validation makes it easier to ensure regularized, stable performance with a relatively small number of re-creations. On the other hand, simple regression and simple decision trees tend to require a large number of training samples to achieve stable accuracy under the given conditions.

[0017] The following detailed description will be given with reference to the drawings. In this specification, elements that are identical or functionally similar to those shown in the drawings will be given the same reference numerals. The use of reference numerals in this specification is not limited to the specific embodiment shown in the drawings, but can also be applied to various modified examples that include at least one component.

[0018] FIG. 1 is a diagram illustrating an example of a credit scoring system 100.

[0019] The credit scoring system 100 is configured to include one or more devices interconnected via a network 101. For example, the credit scoring system 100 is configured to include a data providing device 110, a web server 120, a model inference server 130, a database server 140 (DB server), a client terminal 150, and a model learning server 160.

[0020] The data providing device 110 is a device that has the function of retaining data on persons to be evaluated (applicants, applicants) who apply for contracts that involve credit screening, such as loan applications, credit contract applications, service contract applications, and account opening applications, and providing the retained data to other devices. The data providing device 110 is a device operated, for example, by credit data management institutions, mobile communication carriers, utility companies, financial institutions, etc., and stores various data such as applicant attribute data and usage history in a storage device. In this embodiment, "data" is a broad concept that includes information such as measurement values, raw data, processed data, and analysis results.

[0021] The data providing device 110 can extract data corresponding to the applicant based on a scoring request from the client terminal 150 operated by the applicant, and provide the extracted data to the web server 120. The web server 120 relays the data received from the data providing device 110, and uses it as input data for performing inference processing in the model inference server 130.

[0022] Furthermore, the data providing device 110 may be configured to directly transmit and store data relating to applicants to the database server 140. The data stored in the database server 140 can be reused in subsequent scoring processes.

[0023] In addition, the data providing device 110 may have the function of acquiring data entered by an applicant at the client terminal 150, generating new data based on the entered data, and providing the generated data to the web server 120 or the model inference server 130 in a predetermined format.

[0024] In this case, the data providing device 110 may be configured to receive, as input data, for example, questionnaire responses entered by the applicant on the screen of the client terminal 150, a free-text self-introduction, a record of recent activities, or a history of posts on SNS acquired with the applicant's consent, provide the data to the web server 120 or the model inference server 130, and generate features indicating the applicant's personality tendencies, financial sense, willingness to repay, stability of life, etc. by performing analysis processing such as natural language processing in the model inference server 130. The features generated based on the applicant's input as described above can be used by the model inference server 130 to calculate a credit score.

[0025] In this way, the data providing device 110 can provide not only stored data but also data in response to real-time data input by the applicant.

[0026] The data provided by the data providing device 110 can be broadly divided into static data and dynamic data. Static data includes data that is updated infrequently and remains constant over the long term. Examples of static data include the applicant's name, date of birth, address, occupation, annual income, place of employment, home ownership status, family composition, and past credit history. On the other hand, dynamic data includes data that may change over a short period of time. Examples of dynamic data include utility bill payment history, mobile device usage history, location data, bank account deposit and withdrawal history, various subscription contract statuses, social media activity details, and response results to emotion evaluation questionnaires.

[0027] The data providing device 110 can be configured to provide either static data or dynamic data, or both. Note that the configuration described above is an example, and various modifications are possible regarding the configuration of the data providing destination, the timing and method of providing data, etc., depending on the operational design of the entire system.

[0028] The WEB server 120 has a function to receive data related to applicants transmitted from the data providing device 110. The WEB server 120 may also have a function to perform predetermined preprocessing on the received data. The preprocessing may include at least one of data cleansing, encoding, scaling, feature selection, and dimensionality reduction.

[0029] The WEB server 120 transmits the received data to the model inference server 130 and instructs the model inference server 130 to estimate a credit score (scoring process). Furthermore, if the performance index of a pre-trained machine learning model (hereinafter referred to as the model) falls below a predetermined standard, the WEB server 120 instructs the model to be retrained. Furthermore, the WEB server 120 may have a control function for accepting input from the client terminal 150, displaying the inference results on the screen, and controlling the flow of the entire process.

[0030] In this embodiment, the WEB server 120 is configured to function as a core device that manages a series of processing flows, including communication control with the data providing device 110, processing requests to the model inference server 130, re-learning requests to the model learning server 160, and transmission of results to the client terminal 150.

[0031] The preprocessing may be performed integrally with the web server 120, or may be performed by the model inference server 130, or a data processing device dedicated to preprocessing that is provided separately from these. The control function for the entire process is not limited to the web server 120, but may also be realized by an external data processing device or an orchestration mechanism on the cloud.

[0032] The model inference server 130 stores the model in a storage unit and has an inference processing function for estimating a credit score based on the model. The model inference server 130 receives data transmitted from the web server 120 and executes an inference process for estimating a credit score based on the received data.

[0033] The model inference server 130 may be configured to use, as an inference algorithm, a decision tree model, a logistic regression model, a support vector machine, a neural network, a gradient boosting model (XGBoost, LightGBM, etc.), or an ensemble model that combines these.

[0034] The model inference server 130 may be executed by either synchronous processing or asynchronous processing, and may be configured to be executed serverlessly in a cloud environment, or may be configured as a distributed processing system using multiple inference devices.

[0035] The database server 140 stores various data used by the data providing device 110, the web server 120, the model inference server 130, the model learning server 160, etc. The database server 140 can store, for example, applicant data, input data, inference results, learning history, processing logs, etc.

[0036] The client terminal 150 is a device operated by the applicant, and performs input and output to the credit scoring system 100. The client terminal 150 accesses the web server 120 via a web browser or the like, and provides data, receives results, displays screens, etc.

[0037] The model learning server 160 is a device equipped with a function for learning a model. The model learning server 160 executes model training based on training data (learning data set) acquired from the database server 140 or the data providing device 110.

[0038] The model training server 160 may be configured to apply a decision tree model, logistic regression, support vector machine, neural network, gradient boosting model, or ensemble learning method as a learning algorithm. The model training server 160 may also perform cross-validation, accuracy evaluation, feature importance estimation, and the like on the trained model.

[0039] The model training server 160 transmits the trained model to the model inference server 130 in a predetermined format, and updates the model in the model inference server 130. The model training server 160 may be configured to automatically perform a re-learning process at predetermined intervals or in response to a trigger event, thereby continuously improving model performance.

[0040] The model training server 160 may be configured to train and manage multiple types of models not limited to credit scoring, but also for other purposes such as credit evaluation, repayment prediction, and fraud detection. The model training server 160 may be configured in a batch learning environment on the cloud, a virtual machine, or a container execution environment, and its physical implementation form is not limited.

[0041] With the above-described configuration, the credit scoring system 100 is a system that processes multiple types of external data in an integrated manner and enables credit decisions to be made based on a model.

[0042] FIG. 2 is a diagram illustrating an example of the model inference server 130. As shown in FIG.

[0043] The model inference server 130 includes a processor 210, a storage device 220, and an interface device 230. The processor 210 executes data arithmetic and control processing. The storage device 220 stores programs, data, and the like. The programs are read and executed by the processor. The interface device 230 is a device for transmitting and receiving data to and from a network or other devices, or a device for inputting and outputting data between a user and a device. The functions of the model inference server 130 (such as the acquisition unit 221, processing unit 222, estimation unit 223, output unit 224, and reception unit 225) are implemented by software, hardware, or a combination thereof. The functions of the model inference server 130 are implemented, for example, by software execution on a general-purpose computer, hardware implementation using dedicated circuits, or distributed processing using a cloud or virtual machine. Some or all of the functions of the model inference server 130 may be shared by a single device or multiple devices.

[0044] The acquisition unit 221 has a function of receiving data transmitted from the data providing device 110, the web server 120, the database server 140, the client terminal 150, the model learning server 160, and the like.

[0045] The processing unit 222 processes the data received by the acquisition unit 221. For example, the processing unit 222 performs a predetermined data shaping process prior to the scoring process. The process performed by the processing unit 222 may include at least one of data cleansing, encoding, scaling, feature selection, and dimensionality reduction. The processing unit 222 may be configured to perform natural language processing such as morphological analysis, sentiment analysis, and embedded vector generation on text data containing natural language.

[0046] The estimation unit 223 has a function of receiving as input the data shaped by the processing unit 222 and estimating a credit score based on a trained model. The estimation unit 223 can be configured to estimate a credit score using, as a model, for example, a gradient boosting decision tree model, a logistic regression model, a neural network model, or an ensemble model that combines multiple models.

[0047] The estimation unit 223 has a function of estimating the credit score and also estimating the contribution of each feature to the credit score. As the contribution estimation process, the estimation unit 223 can be configured to perform a quantitative evaluation of the contribution using a SHAP (SHapley Additive exPlanations) value, for example.

[0048] Here, the contribution of a feature is not limited to the SHAP value. Other methods for calculating the contribution include, for example, LIME (Local Interpretable Model-agnostic Explanations), Permutation Feature Importance, Partial Dependence Plot, Integrated Gradients, and DeepLIFT.

[0049] The output unit 224 has a function of outputting the credit score estimated by the estimation unit 223 to the client terminal 150, the web server 120, etc. The output unit 224 is responsible for output processing in a predetermined format, such as generating a result display screen, graphing, or transmitting in a file format.

[0050] Receiving unit 225 has a function of receiving a scoring request from client terminal 150. Receiving unit 225 analyzes the applicant ID, time data, type of data to be acquired, etc. included in the scoring request, and generates processing instructions for acquisition unit 221 and processing unit 222. Note that receiving unit 225 may receive the scoring request directly from client terminal 150, or may receive it indirectly via web server 120.

[0051] FIG. 3 is a diagram illustrating an example of the model learning server 160. As shown in FIG.

[0052] The model training server 160 includes a processor 310, a storage device 320, and an interface device 330. The processor 310 is a device that performs data arithmetic and control processing. The storage device 320 is a device for storing programs, data, etc. The programs are read and executed by the processor. The interface device 330 is a device that transmits and receives data to and from a network or other devices, or a device that inputs and outputs data between a user and a device. The functions of the model training server 160 (such as the acquisition unit 321, processing unit 322, learning unit 323, and output unit 324) are realized by software, hardware, or a combination thereof. The functions of the model training server 160 are realized, for example, by software execution on a general-purpose computer, hardware implementation using dedicated circuits, or distributed processing using a cloud or virtual machine. Some or all of the functions of the model training server 160 may be shared by a single device or multiple devices.

[0053] The acquisition unit 321 has a function of acquiring training data for learning from the data providing device 110 or the database server 140. The training data acquired by the acquisition unit 321 may include both static data and dynamic data of the applicant, and is configured to include, for example, attribute data, credit history, income data, payment history, questionnaire response data, etc.

[0054] The processing unit 322 has a function of performing a predetermined shaping process on the training data acquired by the acquisition unit 321. The processing unit 322 performs at least one process from among data cleansing, encoding, scaling, feature selection, and dimensionality reduction, for example, to convert the training data into a format that can be used for learning.

[0055] The learning unit 323 has a function of learning a model capable of estimating a credit score based on the training data shaped by the processing unit 322. The learning unit 323 executes model learning using, for example, a gradient boosting decision tree, a logistic regression, a support vector machine, a neural network, or an ensemble learning algorithm that combines these. The learning unit 323 may also have a function of evaluating accuracy (AUC, F1 score, etc.) after learning and verifying the performance of the trained model.

[0056] The output unit 324 has a function of outputting the credit score estimated by the learning unit 323 to the client terminal 150, the web server 120, etc. The output unit 324 is responsible for output processing in a predetermined format, such as generating a result display screen, graphing, or transmitting in a file format.

[0057] FIG. 4 is a diagram showing data items handled in the credit scoring system 100 and an example of an input format (data item group 400) accepted by the model inference server 130 after preprocessing.

[0058] Each "meaning / example" shown in Figure 4 is not an example of the representation of raw data collected by the acquisition unit (acquisition unit 221 or acquisition unit 321), but an example of a numerical expression or a normalized expression after processing performed by the processing unit (processing unit 222 or processing unit 322).

[0059] In the credit scoring system 100, the input data flow is as follows: raw data is collected by the acquisition unit, processed by the processing unit, and the numerical vector is passed to the estimation unit 223 or the learning unit 323. The credit scoring system 100 uses the same preprocessing specifications for inference and learning, and uses input vectors generated using the same conversion rules.

[0060] The credit scoring system 100 handles static data such as the applicant's (e.g., ID), gender, marital status, age, annual income, job type (industry), occupation (specific job title), educational background, number of dependents, whether the applicant owns a home, nationality, whether the applicant has a landline phone, whether the applicant has a guarantor, and whether the applicant has collateral. The credit scoring system 100 handles dynamic data such as the borrowing ratio, loan application amount, other loan balance, whether the applicant has delinquent or accident data, whether the applicant has a debt restructuring history, and years of employment.

[0061] The credit scoring system 100 stores the applicant ID as an identifier such as a numeric sequential number. The credit scoring system 100 converts gender into a binary category such as "0: male, 1: female." The credit scoring system 100 converts marital status into a binary category such as "0: unmarried, 1: married." The credit scoring system 100 stores age as an integer value and standardizes it as necessary.

[0062] The credit scoring system 100 acquires annual income as an integer value and uses it as an input variable after standardizing currency units, processing outliers, and performing logarithmic transformation. The credit scoring system 100 maps job types (industry categories) to predefined categories and encodes them numerically, and normalizes specific job titles from character strings to categories and encodes them numerically.

[0063] In this embodiment, the term "input variable" refers to the explanatory variable after preprocessing received by the model inference server 130. In this embodiment, the term "explanatory variable" refers to items such as the applicant's attributes, history, and application details used to estimate the credit score. In this embodiment, the term "target variable" refers to the target value given to the model during training as a label representing the credit status (whether or not there is a delinquency, whether or not there is a history of accidents, etc.). In this embodiment, "contribution" refers to a local index (e.g., SHAP value, LIME value) that indicates the magnitude and direction of the influence that each feature has on the credit score for an individual inference target. Meanwhile, "importance" refers to a global index (e.g., permutation feature importance, Feature Importance of a trained model) that indicates the magnitude of the influence that each feature has on the credit score for the entire dataset.

[0064] The credit scoring system 100 calculates the debt ratio by dividing the loan amount by the annual income, and scales it as a real number between 0 and 1. The credit scoring system 100 acquires the loan application amount and other loan balance as integer monetary values, normalizes the currency unit and digit separators, and converts them into numbers. The credit scoring system 100 encodes the presence or absence of delinquency / accident data and the presence or absence of debt consolidation history as a binary flag of "0: no, 1: yes."

[0065] The credit scoring system 100 encodes the industry affiliation from a categorical value to a numeric value, applying one-hot encoding or target encoding as needed. The credit scoring system 100 normalizes the educational background to integers that maintain a hierarchical order, such as "1: high school graduate, 2: vocational school graduate, 3: university graduate, 4: graduate school." The credit scoring system 100 stores the number of dependents as an integer value and performs binning as needed.

[0066] The credit scoring system 100 encodes the following as binary flags: homeownership, nationality, landline, guarantor, and collateral. The credit scoring system 100 stores years of employment as an integer value and applies a logarithmic transformation or standardization to the years.

[0067] The credit scoring system 100 treats the representation shown in Figure 4 as an example of an input format after preprocessing, and during inference, supplies explanatory variables conforming to Figure 4 to the model inference server 130, and during learning, supplies explanatory variables conforming to Figure 4 as well as a target variable to the learning unit 323. With the above configuration, the credit scoring system 100 converts static data and dynamic data into a unified numerical representation, ensuring the estimation accuracy and reproducibility of credit scores.

[0068] FIG. 5 is a diagram showing an example of pre-processing in the credit scoring system 100.

[0069] The preprocessing for the credit scoring system 100 is designed by the administrator or data analyst of the credit scoring system 100 based on the nature of the input data, the characteristics of the model to be used, and practical requirements. The nature of the input data takes into consideration the type, distribution, and missing rate of static and dynamic data, while the characteristics of the model to be used take into consideration the structure that assumes numerical data and resistance to overfitting. Practical requirements also include accuracy, processing speed, explainability, and the form of integration with external systems. When designing based on these perspectives, a series of preprocessing steps are determined, including the method for imputing missing values, the encoding method for categorical variables, the scaling method, whether or not to perform dimensionality reduction and the method used, and the criteria for selecting features.

[0070] In the training process, each preprocessing step is performed using training data to determine the necessary parameters. The parameters determined at this time include an encoding dictionary for converting categorical variables into numerical values, the mean and standard deviation used for scaling, the principal component vector obtained by dimensionality reduction, and identification information for columns to be retained by feature selection. These parameters (preprocessing parameters) are saved together with the trained model.

[0071] In the inference process, the preprocessing parameters determined in the training process are read and a conversion process is applied to the input data. During this process, the preprocessing parameters are not recalculated or reselected during inference, ensuring that the set and order of explanatory variables between training and inference are consistent. Furthermore, when an unknown categorical value is input, the substitute value defined during training is applied, maintaining consistency in the inference process.

[0072] If it becomes necessary to change the preprocessing process, for example, if new features are added or a performance degradation is detected, the preprocessing process and model are retrained using new training data in an offline environment and deployed as a new version. The old version of the preprocessing and model can continue to be used for inference or can be discontinued when switching.

[0073] The preprocessing steps are systematically organized as a preprocessing pipeline, and the structure is arranged in the order of data acquisition, data cleansing, encoding, scaling, dimensionality reduction, and feature selection, as shown in Figure 5. Figure 5 shows the specifications of the preprocessing that are commonly applied to both the training and inference steps, and represents the actual processing flow, but no specific training or inference operations are performed at this stage.

[0074] Step S501: Data acquisition The credit scoring system 100 acquires input data including static and dynamic data related to applicants. The input data may be acquired automatically from a data providing device 110 or may be provided manually or by file transmission from the applicant or a third party. When acquiring data, data is aggregated from multiple data sources with different data formats and units, and is therefore retained in a raw state so that it can be handled uniformly in subsequent pre-processing steps. This input data is classified into multiple data categories, such as attribute data, transaction history data, and credit history data, and is used as input variables for model inference or model learning.

[0075] Step S502: Data cleansing The credit scoring system 100 performs data cleansing on the acquired input data. Data cleansing is performed for the purpose of improving the quality of the input data in order to prevent a decrease in accuracy or erroneous judgment in model inference or model learning.

[0076] The credit scoring system 100 first detects missing values ​​in the input data and applies imputation methods appropriate to the attributes and data type, such as imputing with the mean or median, imputing estimated values ​​based on the same attribute group, or re-obtaining from an external database. Next, it detects abnormal values ​​using statistical outlier detection or rule-based detection, and replaces them with appropriate values ​​or removes them from the dataset. Furthermore, if there are inconsistencies in the data format, such as date format, numeric format, or category label, it converts them into a unified format to ensure consistency.

[0077] The credit scoring system 100 may optimize the cleansing process depending on the different characteristics of static and dynamic data. For static data, the focus is on correcting formatting inconsistencies and input errors, unifying category labels, and checking logical consistency (e.g., matching age and years of employment). For dynamic data, the focus is on detecting numerical outliers, updating to the latest data, unifying time periods (e.g., normalizing to the most recent 12-month history), and unifying transaction dates and amount formats. This ensures that time-varying data is input into the model accurately and consistently.

[0078] Additionally, the credit scoring system 100 identifies duplicate data for the same applicant or transaction and eliminates redundancies and inconsistencies by retaining the most current and accurate data, creating a neat data set suitable for subsequent encoding and scaling.

[0079] Step S503: Encoding The credit scoring system 100 performs encoding to convert categorical data from the cleansed input data into numerical values. The categorical data includes some static data such as gender, marital status, occupation, industry, educational background, foreign nationality, and guarantor status, as well as some dynamic data such as delinquency status, debt restructuring history, homeowner status, landline phone status, and collateral status.

[0080] In the encoding process, the credit scoring system 100 can use multiple techniques to convert categorical variables into numerical vectors that can be processed by the model. For example, if gender is a binary category of "male" or "female," it can be converted to "0" or "1" using label encoding. In addition, for categorical variables with multiple options, such as occupation or industry, it can be expanded into binary vectors corresponding to each option using one-hot encoding.

[0081] Furthermore, the credit scoring system 100 can apply advanced encoding techniques such as target encoding and embedded representation as needed, taking into account the number and distribution of categorical values, thereby enabling data to be formatted into a format suitable for model training and inference while minimizing data loss.

[0082] Step S504: Scaling The credit scoring system 100 performs scaling on the preprocessed numeric data. Scaling aims to reduce the impact of differences in scale between features on prediction accuracy during model learning and inference by adjusting the range and distribution of values ​​of each numeric variable.

[0083] The credit scoring system 100 can apply normalization (processing to convert into a range of minimum value 0 and maximum value 1) or standardization (processing to convert into a mean of 0 and variance of 1) to numeric data items such as age, annual income, debt ratio, loan application amount, other loan balance, and years of service. This allows learning algorithms such as gradient boosting decision trees, logistic regression, and neural networks to converge stably even when the scale of values ​​differs greatly between variables. However, because decision tree algorithms (random forests, gradient boosting decision trees, etc.) are based on partitioning rules, they do not assume numerical scaling, and scaling is usually not necessary when these models are primarily used.

[0084] Furthermore, the credit scoring system 100 selects a scaling method depending on the characteristics of the static and dynamic data. For example, standardization can be applied to static data with limited fluctuation ranges, such as age and annual income, while robust scaling, which suppresses the influence of outliers, can be applied to dynamic data that can fluctuate significantly over a short period of time, such as loan application amounts and other loan balances. In this way, scaling according to the characteristics of the data is performed, thereby simultaneously improving the learning efficiency and generalization performance of the model. Therefore, in a configuration in which a decision tree system is used as the main model, the scaling process in step S504 may be omitted.

[0085] Step S505: Dimension reduction (removal of dimensions) The credit scoring system 100 performs dimensionality reduction on a group of explanatory variables for which input data has been prepared. Here, dimensionality reduction refers to "a process of reducing the number of explanatory variables using statistical methods to suppress the effects of computational load, noise, duplicate data, multicollinearity, etc." This enables the credit scoring system 100 to improve the stability of learning and inference, prevent overlearning, and shorten processing time.

[0086] The credit scoring system 100 uses, for example, principal component analysis (PCA) as an unsupervised learning method to find principal components based on a covariance matrix. It then adopts principal components whose cumulative contribution rate is equal to or greater than a predetermined threshold (e.g., 90%). It also uses linear discriminant analysis (LDA) as a supervised learning method to find a low-dimensional space that most clearly distinguishes between each objective variable. The method to be used is selected through cross-validation, and the number of variables to be adopted is determined within a range that does not degrade metrics such as accuracy and AUC in the validation data.

[0087] Furthermore, for example, the credit scoring system 100 performs dimensionality reduction by taking into account correlations in static data. Among annual income, educational background, occupation, industry, whether or not a person owns a home, whether or not a person has a landline phone, whether or not a person has a guarantor, whether or not a person has collateral, number of dependents, and years of employment, there are strongly correlated combinations, such as between occupation and industry, and between whether or not a person has a guarantor and whether or not a person has collateral. These are combined using PCA or other methods to reduce the number of variables without damaging the data. Furthermore, the correspondence between the transformed principal components and the original variables is recorded so that it is possible to track which original variables have been combined.

[0088] Furthermore, for example, the credit scoring system 100 performs dimension reduction by taking into account the high dimensionality inherent to time-series data, which is often included in dynamic data. For utility bill payment history, communication usage history, deposit and withdrawal history from bank accounts, purchase history, history of late payments and accidents, and evaluation values ​​of survey responses, statistics such as the mean, variance, maximum value, minimum value, and rate of increase or decrease are first created for periods such as the most recent month, three months, or six months. The resulting large number of derived features are then compiled using PCA, or, if the relationship with the objective variable is important, organized using LDA. Furthermore, if the order of the time series is important, the aggregation period and calculation method are adjusted before dimension reduction.

[0089] To maintain the explainability of the model, the credit scoring system 100 saves the relationship between the features after dimension reduction and the original variables as preprocessing parameters (metadata). This allows the effects of the reduced-dimensional features to be explained on an item-by-item basis in the contribution analysis in the subsequent process.

[0090] Furthermore, when using a model that does not require feature scaling, such as a decision tree algorithm, the credit scoring system 100 may be configured to omit the process if dimensionality reduction does not improve accuracy. Furthermore, the system continuously monitors the model's performance, and if it falls below a predetermined standard, reevaluates the method, threshold, and number of variables used. The ultimately obtained low-dimensional features are used for subsequent feature selection and credit score estimation, and the data for each input item is passed to the model while maintaining the required accuracy.

[0091] Step S506: Feature selection The credit scoring system 100 performs feature selection on a group of explanatory variables according to preprocessing parameters. Feature selection is a process of selecting explanatory variables that contribute highly to the objective variable, the credit score, using statistical or machine learning techniques, and excluding unnecessary or insignificant variables. By performing feature selection, the credit scoring system 100 can improve the prediction accuracy and generalization performance of the model, as well as increase computational efficiency and prevent overlearning.

[0092] Here, feature selection and dimensionality reduction are both techniques for reducing the number of explanatory variables, but they differ in their objectives and processing methods. Feature selection is a process of selecting from the original explanatory variables those that contribute most to predictive performance and excluding the others, with the remaining variables retaining their original meaning and interpretation. On the other hand, dimensionality reduction is a process of mapping the original variables into a new variable space using statistical methods or mathematical transformations and replacing them with a small number of synthetic variables (e.g., principal components), and the transformed variables do not necessarily correspond one-to-one to the original variables. Therefore, the difference is that dimensionality reduction aims to compress data, while feature selection aims to extract useful data.

[0093] The credit scoring system 100 may be configured to perform dimensionality reduction first and then feature selection, or vice versa. It may perform feature selection first and then dimensionality reduction, or it may perform either one or the other. For example, it may be configured to perform dimensionality reduction using principal component analysis to aggregate highly correlated attributes of static data, and then perform feature selection to retain principal components with high contributions. For dynamic data, it may be possible to first remove features with low contributions from diverse, highly volatile historical data, and then compress the remaining features using time series patterns or clustering.

[0094] The credit scoring system 100 evaluates the importance of each attribute item to the objective variable in the static data and eliminates items with low importance. The static data are variables that indicate applicant characteristics that are unlikely to change over time, such as age, annual income, educational background, occupation, industry, years of employment, whether or not the applicant owns a home, and whether or not the applicant has a guarantor or collateral. Some of these variables may be highly correlated with each other, resulting in overlapping data. The credit scoring system 100 identifies redundant items using, for example, correlation coefficient analysis or analysis of variance (ANOVA), and reduces the number of variables while maintaining model performance.

[0095] The credit scoring system 100 focuses on the applicant's behavioral history and transaction history, which change over a short period of time, and prioritizes retaining items that reflect the applicant's most recent credit status. Dynamic data includes utility bill payment history, communication usage history, deposit and withdrawal history, purchase history, history of late payments and accidents, and evaluation values ​​of survey responses. The credit scoring system 100 performs feature importance evaluation using models such as gradient boosting decision trees (GBDT) and random forests, or contribution analysis using SHAP values, to remove features that contribute little to predictive performance. The system also reduces the input dimension of the model by grouping variables with similar trends over time.

[0096] In order to ensure explainability in the subsequent inference process and contribution estimation process, the results of feature selection are configured to save the correspondence between the selected features and the original data items as preprocessing parameters. This allows the credit scoring system 100 to track the basis for credit decisions on an item-by-item basis and visualize and output them as needed.

[0097] Additionally, conventional credit scoring systems estimate credit scores using a relatively small number of attribute items, primarily static data such as name, date of birth, address, occupation, annual income, home ownership status, and past credit history. Because this static data fluctuates infrequently and the number of input variables is limited to a few dozen items, the risk of computational load and overfitting due to correlation and redundancy between variables is low. Therefore, high-level preprocessing such as dimensionality reduction and feature selection is rarely performed, and models that emphasize interpretability, such as simple weighting methods and logistic regression, tend to be used.

[0098] However, in recent years, technologies have emerged that collect large amounts of dynamic data, such as transaction history, payment history, social media activity history, and location data, and combine it with static data to use it for credit evaluation. This dynamic data can contain hundreds to thousands of items, and since there is high correlation between variables and noise, inputting the data directly into a model can lead to reduced computational efficiency and overfitting. For this reason, this embodiment employs a configuration that introduces dimensionality reduction and / or feature selection to both static and dynamic data, optimizing the structure of the input data and improving the performance and stability of the model.

[0099] Through the above steps, the credit scoring system 100 can generate formatted input data and provide it to the model inference server 130 or the model training server 160, thereby inferring a credit score or training a model with high accuracy. Furthermore, each step may be executed by at least one of the web server 120, the model inference server 130, the model training server 160, or an external server capable of communicating with the model inference server 130 or the model training server 160, depending on the configuration and operation mode.

[0100] Below, we will explain specific examples of preprocessing according to various data types in the preprocessing pipeline (steps S502 to S506) shown in Fig. 5. Note that these are just examples and can be changed as appropriate depending on data quality, model characteristics, operational requirements, etc.

[0101] For utility bill payment history, the input data includes billing date, payment due date, payment date, amount, charge type (e.g., electricity, gas, water, communications, etc.), and payment method (e.g., direct debit, credit card, etc.). Cleansing processes are then performed, including standardizing date formats and currency units, eliminating duplicate details, and setting imputation rules for unpaid records (e.g., handling overdue payments). Features such as the number of overdue payments, maximum number of days late, average number of days late, on-time payment rate, total amount of late fees, and whether or not automatic payment is enabled are then aggregated for the past one, three, and six months. Charge type and payment method are coded based on a predefined category dictionary, and unknown categories are mapped to default values. Logarithmic transformation or robust scaling is applied to the amount and fees to reduce the influence of outliers. Highly correlated features are compressed or selected using statistical methods, and the results are saved as preprocessing parameters.

[0102] For mobile device usage history, input data includes data traffic volume (e.g., monthly or daily), number of calls and call duration, number of SMS sent, late-night usage rate, roaming status, and billing history. These data undergo cleansing processes, such as unit unification (e.g., converting minutes to seconds, MB to GB), detection and replacement of abnormal spikes, and removal of duplicate records. Features are then calculated for the past one, three, and six months, including totals, averages, and variances, weekday to holiday ratios, daytime to nighttime ratios, number of roaming days, and number of billing events. Rate plan type and roaming status are categorically coded, and robust scaling is applied to features prone to double-tailed distributions, such as data traffic volume and call duration. Furthermore, highly correlated indicators, such as those by time period, are compressed or selected using PCA and importance evaluation, and the selected features are saved.

[0103] For location data, sequential data consisting of longitude and latitude and timestamps is acquired as input data. After cleansing processes such as filling in missing timestamps, removing obvious positioning errors, and normalizing time bands, the locations are anonymized using a specified spatial grid, and feature values ​​are calculated, including the number of grids visited over the past one, three, and six months, the number of main clusters visited, a stability index for main nighttime locations, the number of cluster transitions during the day and night, and the mean, variance, and maximum of travel distance. Cluster IDs are sequentially numbered, and unknown clusters are mapped to default values. Features related to distance and frequency are robustly scaled, and cluster indicators are compressed using PCA or selected based on importance.

[0104] For bank account transaction history, the input data includes transaction date, category (deposit or withdrawal, etc.), amount, balance (if available), and transaction metadata. After cleansing the data by normalizing signs (deposits as positive and withdrawals as negative), standardizing currency units, eliminating duplicate documents, and reconciling transfers and refunds, the following features are calculated: total deposit amount, total withdrawal amount, net cash flow, number of consecutive months of salary-equivalent deposits, average balance, minimum balance, number of times the balance fell below a specified threshold, and number of regular withdrawals for the most recent one, three, and six months. Amount-related features are subjected to logarithmic transformation or robust scaling, and amount-derived features with multicollinearity are compressed or selected using PCA or importance assessment, and the results are saved.

[0105] For the contract status of various subscriptions, input data includes the service name, start date, end date, fee, billing status (active, suspended, canceled), and payment method. Cleansing processes are performed, such as standardizing variations in the spelling of service provider names, merging duplicate contracts, and specifying how free trial periods are handled. Features are then generated, including the current number of active contracts, the number of new contracts initiated in the past month, three months, and six months, the number of contracts canceled, the estimated monthly total, the average contract period, and the auto-pay ratio. Service categories (video, music, software, etc.) are category-coded based on a dictionary, and unknown categories are mapped to default values. Price and number indicators are normalized, and highly correlated indicators are compressed or selected to save the list of adopted features.

[0106] For social media activity, the input data includes the post text, posting time, and reaction indicators (e.g., number of likes, number of replies, etc.). After cleansing the data by normalizing the text (processing symbols, URLs, and mentions), determining language, and removing spam, natural language processing is performed, including tokenization based on morphological analysis, non-word processing, sentiment analysis, and document embedding vector generation. Furthermore, features are calculated, including posting frequency over the past one, three, and six months, percentage by time period, mean and variance of sentiment scores, mean and variance of embedding vectors over a period, and topic distribution indicators. The document embedding vectors are compressed using PCA, and topic indicators are extracted using LDA or other methods. A decision is made as to whether to accept or reject a feature based on its importance, and the accepted features are saved.

[0107] The input data for the response results of the emotion assessment questionnaire are multiple-choice answers such as Likert scales and free-form comments (optional). After cleansing the data by standardizing the scale direction (including processing reversed items) and setting rules for imputing missing values, the system synthesizes scores for each scale (total and average), standardizes scores, and assigns response consistency indicators, as well as text sentiment and embedding indicators to the free-form comments. Categorical responses are coded, and the composite scores are standardized. Furthermore, highly correlated scales are integrated or selected, and the adopted features and conversion rules are saved.

[0108] FIG. 6 is a diagram showing an example of the overall processing in the credit scoring system 100.

[0109] In step S601, the credit scoring system 100 determines whether a new model to be used for estimating a credit score needs to be created. The determination is made based on a number of criteria.

[0110] First, the credit scoring system 100 verifies whether the evaluation index (AUC, F1 score, accuracy rate, etc.) indicating the predictive performance of the existing model is below a predetermined reference value. If the evaluation index is below the reference value, it is determined that the model accuracy has deteriorated and that the model needs to be recreated.

[0111] Next, the credit scoring system 100 analyzes whether the statistical distribution of input data has changed significantly from the distribution at the time of model training to detect the occurrence of concept drift. In particular, features containing dynamic data such as an applicant's transaction history and payment history tend to exhibit large distribution fluctuations over time, which can easily cause the prediction accuracy of existing models to decline in a short period of time. For this reason, the credit scoring system 100 can be configured to consider the need to rebuild the model more frequently than conventional credit models that primarily rely on static data. This prevents a decline in prediction accuracy due to changes in dynamic data and enables the maintenance of highly accurate credit evaluations that reflect the latest data trends.

[0112] Furthermore, the credit scoring system 100 monitors for changes in external factors that affect the assumptions of the model, such as revisions to financial regulations or credit criteria, and determines that the model needs to be recreated if any changes are detected.

[0113] Even if the above conditions are not met, the credit scoring system 100 can apply a periodic update policy that recreates the model every predetermined period (for example, every six months).

[0114] If it is determined that any of the above conditions is met, the credit scoring system 100 proceeds to step S602, where it executes model learning and creation. On the other hand, if none of the conditions is met, it determines that model creation is unnecessary, and proceeds to step S603.

[0115] In step S602, the credit scoring system 100 uses the model training server 160 to acquire training data containing static and dynamic data related to the applicant. The acquired training data is preprocessed in the following order: missing value imputation, categorical variable encoding, numeric variable scaling, statistical dimensionality reduction, and feature selection. Missing value imputation involves setting imputed values ​​based on the training data, and categorical variable encoding involves determining encoding mapping and unknown category processing rules. Scaling involves determining coefficients such as the mean and standard deviation. Dimensionality reduction involves applying, for example, principal component analysis (PCA) to determine the number of principal components, transformation matrix, and lower bounds on contribution rates. Feature selection involves evaluating the importance of feature importance to the objective variable using, for example, a feature importance index such as the SHAP value, and determining the set of features to be adopted. If dynamic data is included, a time window and aggregation method (e.g., total over the last N days, moving average, maximum value, etc.) are set based on the training data, and the sequence data is converted into features for a single record. These preprocessing parameters are saved so that they can be applied during inference using the same procedure as during training. The pre-processing parameters may be re-estimated each time learning is performed, or existing pre-processing parameters may be reused. The decision as to whether to use this method is based on the performance evaluation results described below.

[0116] The credit scoring system 100 can use gradient boosting decision trees, logistic regression, neural networks, or ensembles of these as learning algorithms. In particular, a highly efficient implementation of gradient boosting decision trees can achieve both high classification accuracy and learning efficiency for large feature spaces and nonlinear relationships. Hyperparameters (learning rate, tree depth, number of leaf nodes, regularization coefficient, etc.) are searched for using cross-validation, and settings that maximize classification performance are adopted. In cross-validation, training data is divided into multiple partitions, training is performed on the training set, performance is measured on the validation set, and the results are averaged across the partitions to estimate generalization performance.

[0117] The credit scoring system 100 calculates model performance evaluation indicators to objectively grasp performance as learning progresses. The overall accuracy rate (Accuracy) indicates the ratio of correctly identified cases to the total number of identified cases, and the area under the curve (AUC) evaluates the binary discrimination ability of delinquency probability based on a receiver operating characteristic (ROC) curve, without relying on a threshold. Precision indicates the ratio of cases that actually became delinquent among the number of cases predicted as "at risk of delinquency," and recall indicates the ratio of cases that were correctly predicted among the number of cases that actually became delinquent. The F1-score evaluates the balance between precision and recall by taking the harmonic mean of the two. Support indicates the number of cases in each class, the macro avg is the simple average of the indicators for each class, and the weighted avg is the weighted average of the indicators for each class, and are used to evaluate classification performance from multiple perspectives. If these evaluation indices meet the preset criteria, the model and preprocessing parameters determined in the training are adopted and saved. If the criteria are not met, at least one of the dimensionality reduction method, feature selection method, hyperparameters, and training algorithm is changed and training is rerun. The saved model and preprocessing parameters are used in the subsequent step S603.

[0118] In step S603, the credit scoring system 100 sequentially applies the preprocessing parameters determined and saved in step S602 to the input data related to the applicant via the model inference server 130. These preprocessing parameters include imputed values ​​for missing value imputation, encoding mapping and unknown category processing rules for encoding categorical variables, scaling coefficients for numeric variables, transformation matrices for dimensionality reduction, a list of adopted features for feature selection, and a time window and aggregation method for dynamic data. By applying these parameters in the same procedure as during training, the credit scoring system 100 generates a feature vector from the input data in the same format and with the same number of dimensions as during training.

[0119] The credit scoring system 100 inputs the generated feature vector into a model and estimates the applicant's credit score as the output. The credit score can express the probability of delinquency, credit risk category, etc., in numerical or categorical form. If necessary, explainability techniques (e.g., SHAP values) are used to estimate the contribution of each feature, and explanatory data including the contribution direction (risk increase or decrease) and contribution amount are generated. In this process, the encoding correspondence table and scaling coefficients saved during training are referenced, and the explanatory variable names and category names for display are restored and presented.

[0120] If the input data contains category values ​​that have not yet appeared at the time of learning, unknown category processing rules (for example, mapping to other categories or substituting default values) are applied, and if the numeric input is significantly outside the range expected during learning, outlier processing rules (for example, clipping at upper and lower limits or treating as missing) are applied.If dynamic data is included, the same time window and aggregation method as during learning is applied, and the sequence data is converted into features of a single record and used for inference.

[0121] In step S604, the credit scoring system 100 transmits the credit score and contribution data estimated by the model learning process (step S602) or the model inference process (step S603) to the notification destination via the output unit (output unit 224, output unit 324) and stores them as an examination record. The notification destination includes the client terminal 150 as well as external business systems, credit examination terminals, administrator consoles, etc., and the data is transmitted via the interface device 230 over the network 101.

[0122] The credit scoring system 100 can include as output data individual credit scores, names of features with the highest contribution, the contribution direction (risk increase or decrease) and contribution amount (such as the SHAP value shown in FIG. 7), risk classification, model identifier, version of applied preprocessing parameters, processing execution time, source of input data, whether or not an unknown category occurred, etc. This ensures the reproducibility and explainability of inference results during subsequent audits and operational management.

[0123] The credit scoring system 100 can output both machine-readable structured data (JSON, XML, etc.) for business systems and human-readable report formats (PDF, HTML, etc.) in a single format. The report includes individual results (credit scores and their top contributing factors), evaluation metrics for the entire model (Accuracy, AUC, Precision for each class, Recall, F1-score, macro average, weighted average, etc.), and statistical data, as shown in Figure 7.

[0124] Furthermore, as shown in Figure 8, the system may be configured to output the main attributes of the input data (gender, marital status, age, annual income, borrowing ratio, loan application amount, other loan balance, years of employment, etc.) and estimated credit scores in a list format. This allows the results of multiple applicants to be checked at once, making the screening process more efficient.

[0125] Furthermore, when multiple inference results are output at once using batch processing, aggregated data summarizing the results (overall score distribution, number of cases by risk category, distribution of feature contributions, etc.) can be generated and accumulated as reference data to aid in the model re-creation decision (step S601).

[0126] FIG. 7 is a diagram showing an example of a screen (evaluation display screen 700).

[0127] The evaluation display screen 700 is generated after the credit scoring system 100 completes model training in step S602 or after completing batch inference in step S603. The evaluation display screen 700 is configured to verify the performance of the trained model using objective indicators, present the calculation basis for each applicant's credit score in an explainable manner, and further provide reference information for determining whether to re-create the model in step S601.

[0128] The evaluation display screen 700 includes an individual result display area, an overall model evaluation result display area, and an analysis report display area. The individual result display area displays the predicted score, the individual top SHAP items, and the SHAP value for each item for each applicant being verified or inferred. The predicted score is a credit risk probability ranging from 0.0 to 1.0, with higher values ​​indicating higher risk. Positive SHAP value factors indicate a contribution that increases the score, while negative factors indicate a contribution that decreases the score. The direction and numerical value of the factors are displayed side by side to enable understanding that the larger the absolute value of the SHAP value, the greater the contribution. This allows the examiner to immediately understand the basis for the applicant's estimated credit score.

[0129] The overall model evaluation result display area displays the accuracy rate (Accuracy) and area under the curve (AUC). The accuracy rate is the percentage of correct judgments for the evaluation data, and AUC is an index that shows the ability to distinguish between delinquent and non-delinquent, with the closer to 1 the better the discrimination performance. AUC is used as an overall evaluation that does not depend on a threshold value.

[0130] The analysis report display area displays precision, recall, F1-score, and support for each class 0 (no overdue payments) and class 1 (overdue payments). Precision is the accuracy rate within the prediction set, recall is the percentage of the actual class that was detected, and the F1-score is the harmonic mean of precision and recall. As an indicator to eliminate the impact of imbalance between classes, the macro avg (simple average) and weighted avg (number-weighted average) that reflects the distribution of actual data are calculated and displayed. These indicators are used to set score thresholds and select models according to business requirements.

[0131] The credit scoring system 100 saves the indicators, individual contribution data, model identifiers, and version information of the conversion content displayed on the evaluation display screen 700 as screening records. The saved data is used to track performance changes through regular monitoring during operation, and if a decrease in accuracy or a change in data distribution is detected, it is used to determine whether to recreate the model in step S601. In addition, reports in the same format are periodically generated for new inference results obtained during operation to support decision-making regarding accountability, audit response, and threshold adjustment.

[0132] FIG. 8 is a diagram showing an example of a screen (score calculation result list screen 800).

[0133] The score calculation result list screen 800 is the output result generated after the credit scoring system 100 completes the inference process of step S603. When credit scores are calculated for multiple applicants at once, the calculation results and main input data are organized in a tabular format, allowing the examiner to grasp the results of all cases at a glance.

[0134] The score calculation result list screen 800 assigns one record row to each applicant and displays the identification number, attribute data, credit-related input features, and calculated credit score as columns. Examples of attribute data include personal information items such as gender, marital status, and age. Examples of credit-related input features include key variables used in credit risk assessment such as annual income, borrowing ratio, loan application amount, other loan balance, and years of employment. The credit score column displays the default risk probability for each applicant calculated by the credit scoring system 100 using the trained model, ranging from 0.0 to 1.0, with higher values ​​indicating higher credit risk.

[0135] The credit scoring system 100 uses the score calculation results list screen 800 to help credit examiners compare multiple scores at once and quickly extract applicants who meet specific criteria. Extraction criteria can be set by specifying a credit score range or by combining multiple attribute and feature conditions. For example, complex condition searches can be performed, such as extracting applicants with high credit scores and sending them for further investigation, or identifying applicants with a high debt ratio and short years of employment. In addition, by presenting the results in a list format, it can be used to grasp overall trends and for statistical analysis of risk distribution.

[0136] Based on the data displayed on the score calculation result list screen 800, the credit scoring system 100 generates statistical data such as score distribution, score trends by attribute, and contribution distribution by feature, and accumulates it as auxiliary indicators for determining whether to re-create the model in step S601. The statistical data includes comparison with past distributions, statistical test results, and fluctuation rates, and suggests the need for model re-learning if the score distribution for a specific attribute group has changed significantly or if the overall trend has deviated from the past distribution.

[0137] Furthermore, the credit scoring system 100 saves the content displayed on the score calculation result list screen 800 as a screening record. The saved record includes the input feature amount for each applicant, the calculated score, the model identifier at the time of inference, version information of the conversion content, the processing time, the processing source terminal, whether or not an unknown category occurred and its content, whether or not outlier processing was applied, etc. This allows the basis for judgment at the time of inference to be accurately reproduced in response to later audits or requests for explanation.

[0138] In this way, the score calculation result list screen 800 does more than simply present scores; it also plays a role in consistently supporting the speeding up of review decisions, analysis of overall trends, model performance monitoring, and audit response.

[0139] According to this embodiment, the training process and the inference process in credit scoring can be executed in a unified manner using the pre-processing parameters and model parameters determined during training, thereby matching the format and dimensions of the input data and ensuring the consistency and reproducibility of the inference results.

[0140] In addition, during model training, multiple evaluation indicators such as accuracy rate, AUC, precision, recall, and F1 score are calculated and the acceptance or rejection is determined by comparing them with reference values, thereby preventing over-training and performance degradation while adapting credit risk identification performance to business requirements.

[0141] Furthermore, the evaluation display screen displays each applicant's credit score and contribution information, allowing users to intuitively understand the basis for calculating the score, thereby accelerating screening decisions while ensuring accountability.The score calculation results list screen allows users to get an overview of the inference results for multiple cases, and can be used to understand risk trends and monitor models through condition extraction and statistical analysis.

[0142] (II) Supplementary Note The identifiers of components described in this specification (e.g., prefixes and symbols such as "first" and "second") are for convenience of identification and do not limit the number, order, function, or arrangement of the components. The same identifier may refer to different components in different embodiments, and one component may also perform the function of another component. Therefore, the identifiers of components described in this specification are not intended to limit the technical scope, functional scope, or scope of rights of the components, and each component should be interpreted flexibly according to the context of the embodiment.

[0143] As used herein, the term "interface device" refers to a component that may include one or more interface devices. Such interface devices may include, but are not limited to, an I / O (Input / Output) interface device, a communication interface device, or a combination thereof. For example, an I / O interface device may be configured to function as a user interface and may include at least one input device (e.g., a keyboard, a pointing device) and / or an output device (e.g., a display). These I / O interface devices may be configured to have a communication function corresponding to connection with a remote computing device. In such a case, the I / O interface device can also operate as a communication interface device. Furthermore, a communication interface device may include the same type of communication means (e.g., multiple NICs (Network Interface Cards)) or may combine different types of communication means (e.g., a NIC and an HBA (Host Bus Adapter)). This allows for a flexible configuration that ensures connectivity with heterogeneous systems. An interface device configured in this manner is not limited to a specific hardware configuration and can accommodate future technological advances and diversification of implementations.

[0144] As used herein, the term "storage device" refers to a component that may include at least one storage device. Depending on the intended use and system configuration, such storage devices may be classified into, for example, "memory," which temporarily stores data during operation, and "persistent storage devices," which retain data even after power is turned off. Memory functions as a temporary storage medium accessible by a processor and may include volatile, nonvolatile, or a combination of these memory devices. Examples of such memory devices include, but are not limited to, volatile memory such as dynamic random access memory (DRAM) and static RAM (SRAM), and nonvolatile memory such as magnetoresistive RAM (MRAM) and resistive RAM (ReRAM). Persistent storage devices are components designed for long-term data storage and include devices that use nonvolatile storage media. Specific examples include hard disk drives (HDDs), solid state drives (SSDs), and non-volatile memory express (NVMe) drives. Next-generation storage technologies such as 3D XPoint and phase-change memory may also be included as examples of storage device configurations. A storage device configured in this manner is not limited by the type or architecture of the storage medium, and can accommodate future technological advances and diversification of embodiments.

[0145] As used herein, the term "processor" refers to a component that may include an arithmetic unit or circuitry capable of executing at least one processing function. Depending on the application, such a processor may include a microprocessor device such as a central processing unit (CPU) or a graphics processing unit (GPU), or may be implemented using dedicated circuits such as a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), or an application-specific integrated circuit (ASIC). A processor may be single-core, multi-core, or comprise a single processor core. A processor may be configured to implement processing functions by a computer program, directly implemented by hardware circuits, or even a hybrid configuration combining these. When processing is performed by a program, this includes the processor working in conjunction with other components such as a storage device or an interface device. Although a specific function may be described as a "unit" in this specification, the function may be implemented by a program executed by the processor, by a circuit, or by a combination thereof. Therefore, the function can be considered to be configured as at least a part of the processor. The program may be supplied from an external program source. Examples of program sources include, but are not limited to, a network-connected program distribution server, a computer-readable non-transitory storage medium, etc. A processor configured in this manner is not limited to a specific hardware configuration or implementation form, and can accommodate future technological advances and diversification of embodiments.

[0146] As used herein, the term "system" refers to a component that may include at least one computing resource. Such a system may be configured with one or more physical computers (such as dedicated hardware or an on-premise server) or virtualized computing resources (such as a cloud platform, virtual machines, or containers). Examples of systems include, but are not limited to, cloud computing systems, cluster configurations, and serverless environments. A system may be configured within a single device, or multiple computing resources may be linked via a network. Each component may be logically integrated or physically separated. Such a system may also include components such as a processor, a storage device, and an interface device, and these components may be implemented as physical devices or as virtualized configurations. A system configured in this manner is not limited to a specific hardware configuration, implementation, or layout, and can accommodate future technological advances and diversification of implementations.

[0147] The above-described embodiment has the following features, for example.

[0148] (1) A credit scoring system (e.g., credit scoring system 100) capable of estimating a credit score (a numerical value or category that quantitatively represents the credit risk of the person being evaluated, and an index obtained as a result of inference by providing input data to a machine learning model) of a person being evaluated (e.g., an applicant), includes an acquisition unit (e.g., acquisition unit 221, acquisition unit 321, model inference server 130, model learning server 160) that acquires first data (e.g., static data) including static information of the person being evaluated and second data (e.g., dynamic data) including dynamic information of the person being evaluated, and a process for acquiring the first data and the second data acquired by the acquisition unit, A credit scoring system comprising: a processing unit (e.g., processing unit 222, processing unit 322, model inference server 130, model learning server 160) that performs at least one of data cleansing, encoding, scaling, feature selection, and dimensionality reduction; an estimation unit (e.g., estimation unit 223, learning unit 323, model inference server 130, model learning server 160) that inputs data processed by the processing unit into a machine learning model to estimate a credit score; and an output unit (e.g., output unit 224, output unit 324, model inference server 130, model learning server 160) that outputs the credit score estimated by the estimation unit.

[0149] According to the above configuration, for example, static information related to the attributes of the person being evaluated and dynamic information related to transaction history, usage status, etc. can be combined and acquired, and then at least one preprocessing step from data cleansing, encoding, scaling, feature selection, and dimensionality reduction can be applied to this data to improve the quality of the features input to the machine learning model. This suppresses degradation of learning accuracy due to data variability, missing data, scale differences, etc., and enables stable credit score estimation. Furthermore, by using both static and dynamic information, a dynamic credit evaluation can be realized that reflects not only the fixed attributes of the person being evaluated, but also their latest behavioral trends and history.

[0150] (2) The credit scoring system includes a learning unit (e.g., learning unit 323, model learning server 160) that learns the machine learning model. The learning unit estimates a performance index based on the output result of the machine learning model and the correct answer data corresponding to the output result, and re-learns the machine learning model if the estimated performance index falls below a predetermined standard. Note that the performance index is an index for quantitatively evaluating the predictive performance of a trained machine learning model, and includes, for example, the overall accuracy rate (Accuracy), the area under the curve (AUC), precision, recall, F1 score, number of cases (support), the simple average (macro avg) of each class index, and the weighted average (weighted avg) of each class index.

[0151] With the above configuration, for example, a decline in the accuracy of a machine learning model can be automatically and quantitatively detected based on a performance index obtained by comparing the output results of the machine learning model with ground truth data. This makes it possible to determine when the model needs to be retrained without relying on manual judgment, and maintain the accuracy of credit score estimation over the long term. Furthermore, with the above configuration, for example, multifaceted evaluation that takes into account the balance between false alarms and oversights in delinquency detection becomes possible, enabling credit scoring that maintains high reliability even in the face of overfitting and fluctuations in data distribution.

[0152] (3) The second data includes at least one of deposit and withdrawal information for a deposit account, payment history information for utility or communication charges, purchase history information for goods or services, electronic money usage history information, and questionnaire response information by the person being evaluated.

[0153] According to the above configuration, for example, by including at least one of deposit account deposit and withdrawal information, utility or communication bill payment history information, product or service purchase history information, electronic money usage history information, or questionnaire response information from the person being appraised as the second data, dynamic information reflecting the person being appraised's latest and multifaceted economic activity and consumption trends can be utilized in credit score estimation. As a result, it is possible to accurately detect credit risk fluctuations that could not be captured by conventional credit assessments based on static attribute information, and improve the accuracy of early prediction of delinquency and default. In particular, by continuously collecting and reflecting dynamic information, credit management that responds promptly to changes in the person being appraised's credit status can be realized, achieving both risk reduction and minimization of opportunity loss.

[0154] (4) The above machine learning model is a credit scoring system built using a gradient boosting decision tree algorithm.

[0155] According to the above configuration, for example, by constructing a machine learning model using a gradient boosting decision tree algorithm, high classification performance can be achieved even in a large-scale, high-dimensional feature space, and nonlinear and complex relationships between static and dynamic information can be accurately captured. Furthermore, the gradient boosting decision tree algorithm is relatively robust against missing values ​​and outliers, and it is easy to calculate the contribution of each explanatory variable, making it possible to improve estimation accuracy while ensuring the explainability of credit scores. As a result, in credit screening, credit risk assessment that combines accuracy and transparency can be performed in a short time, improving both business efficiency and reliability.

[0156] (5) The credit scoring system includes a reception unit (e.g., reception unit 225, model inference server 130) that receives requests from an external device (e.g., client terminal 150), and the output unit outputs the credit score estimated by the estimation unit to the external device in response to the request received by the reception unit.

[0157] According to the above configuration, for example, the reception unit can receive requests from external devices and the credit score estimated by the estimation unit can be immediately output to the external devices. This enables real-time or near-real-time integration of credit risk assessment results in business flows such as credit screening and financial transaction application processing. This allows the external system to instantly perform screening decisions and risk management, minimizing processing delays and improving user experience and business efficiency. Furthermore, the system can easily be integrated with external business systems, online application forms, credit management consoles, etc., improving system scalability and operational flexibility.

[0158] (6) The estimation unit estimates the contribution of each feature to the credit score, and the output unit outputs the credit score and contribution estimated by the estimation unit. The contribution is a numerical value that quantifies the magnitude and direction of the contribution of each input feature to the estimated credit score, and includes values ​​calculated using, for example, a SHAP value, a LIME value, a feature contribution rate, or other indicators. For example, a positive value for the contribution indicates that the feature contributes to increasing the credit score, and a negative value indicates that the feature contributes to decreasing the credit score.

[0159] According to the above configuration, for example, the estimation unit estimates the contribution of each feature, and the output unit outputs the contribution together with the credit score, thereby making it possible to present the basis for the credit score evaluation result in a quantitative and explainable manner. This allows credit examiners and risk managers to easily identify the factors that led to a high or low credit score for a particular applicant, ensuring transparency and accountability in judgments. Furthermore, from the perspective of audit response and regulatory compliance, the basis for score calculation can be verified at a later date, improving the reliability and compliance of model operation.

[0160] (7) A model generation system (e.g., credit scoring system 100, model learning server 160) that generates a machine learning model that can predict the credit score of a person being evaluated, comprising: an acquisition unit (e.g., acquisition unit 321, model learning server 160) that acquires training data including static information and dynamic information of the person being evaluated; a processing unit (e.g., processing unit 322, model learning server 160) that performs at least one of data cleansing, encoding, scaling, feature selection, and dimensionality reduction on the training data; and a learning unit (e.g., learning unit 323, model learning server 160) that learns a machine learning model using a machine learning algorithm based on explanatory variables and target variables included in the data processed by the processing unit.

[0161] According to the above configuration, a machine learning model can be generated by pre-processing a variety of features that combine static and dynamic information and optimizing them for learning, thereby enabling the credit risk of the person being evaluated to be predicted more accurately and stably than before, thereby improving the reliability of credit decisions. [Explanation of symbols]

[0162] 100...Credit scoring system, 130...Model inference server, 160...Model learning server.

Claims

1. A credit scoring system capable of estimating a credit score of an assessee, an acquisition unit that acquires first data including static information of the person being evaluated and second data including dynamic information of the person being evaluated; a processing unit that performs preprocessing on the first data and the second data acquired by the acquisition unit; an estimation unit that inputs the data processed by the processing unit into a machine learning model to estimate a credit score; an output unit that outputs the credit score estimated by the estimation unit; a learning unit that learns the machine learning model; Equipped with During learning of the machine learning model, The processing unit aggregating time-series dynamic information included in the second data using a predetermined fixed-length time window and an aggregation method to convert it into a feature quantity of a single record; searching for and selecting a low-dimensional representation based on a predetermined evaluation criterion for the training data to perform dimensionality reduction, searching for and selecting a feature set to be adopted based on the predetermined evaluation criterion for the training data to perform feature selection, and determining and saving preprocessing parameters including conversion information and feature selection information based on the results; During inference using the machine learning model, the processing unit aggregates the time-series dynamic information included in the second data acquired by the acquisition unit using the predetermined fixed-length time window and aggregation method to convert it into a feature quantity of a single record, and performs the same conversion using the stored preprocessing parameters; The estimation unit is a credit scoring system that inputs the data processed by the processing unit into the machine learning model to estimate a credit score.

2. 2. The credit scoring system of claim 1, the learning unit calculates a performance index based on an output result of the machine learning model and correct answer data corresponding to the output result, and starts re-learning when the calculated performance index falls below a predetermined standard or when it is determined that the distribution of the input data has deviated from the distribution at the time of learning by more than a predetermined standard; In the credit scoring system, during the initiated relearning, the processing unit searches for and selects low-dimensional representations based on predetermined evaluation criteria for the training data to perform dimensionality reduction, searches for and selects a feature set to be adopted to perform feature selection, and further reviews the settings of a predetermined fixed-length time window and aggregation method, and updates and saves preprocessing parameters including conversion information, feature selection information, and setting information for the time window and aggregation method based on the results.

3. 2. The credit scoring system of claim 1, A credit scoring system in which the second data includes at least one of deposit account deposit and withdrawal information, utility or communication fee payment history information, product or service purchase history information, electronic money usage history information, and questionnaire response information by the person being evaluated.

4. 2. The credit scoring system of claim 1, The machine learning model is a credit scoring system constructed using a gradient boosting decision tree algorithm.

5. 2. The credit scoring system of claim 1, a reception unit that receives a request from an external device; A credit scoring system in which the output unit outputs the credit score estimated by the estimation unit to the external device in response to a request received by the reception unit.

6. 2. The credit scoring system of claim 1, a recording unit for recording data; the estimation unit estimates the contribution of each feature to the credit score; The output unit outputting the credit score and contribution rate estimated by the estimation unit; outputting audit metadata in a machine-readable structured format, the audit metadata including at least some of the top items of the contribution degree estimated by the estimation unit, the risk category, the model identifier, the version of the applied preprocessing parameter, the processing time, the source of the input data, and whether or not an unknown category has occurred; The recording unit is a credit scoring system that records the credit score and audit metadata output by the output unit.

7. A credit scoring system as described in claim 1, the fixed-length time window includes at least one of the most recent one month, three months, and six months; The credit scoring system, wherein the aggregation method includes at least one of sum, average, maximum, minimum, variance, and rate of increase or decrease.

8. A credit scoring method capable of estimating a credit score of an assessee, comprising: The acquisition unit acquires first data including static information of the assessee and second data including dynamic information of the assessee; a processing unit performing preprocessing on the first data and the second data acquired by the acquisition unit; an estimation unit inputting the data processed by the processing unit into a machine learning model to estimate a credit score; an output unit outputting the credit score estimated by the estimation unit; a learning unit learning the machine learning model; Including, During learning of the machine learning model, The processing unit aggregating time-series dynamic information included in the second data using a predetermined fixed-length time window and an aggregation method to convert it into a feature quantity of a single record; searching for and selecting a low-dimensional representation based on a predetermined evaluation criterion for the training data to perform dimensionality reduction, searching for and selecting a feature set to be adopted based on the predetermined evaluation criterion for the training data to perform feature selection, and determining and saving preprocessing parameters including conversion information and feature selection information based on the results; During inference using the machine learning model, the processing unit aggregates the time-series dynamic information included in the second data acquired by the acquisition unit using the predetermined fixed-length time window and aggregation method to convert it into a feature quantity of a single record, and performs the same conversion using the stored preprocessing parameters; A credit scoring method in which the estimation unit inputs the data processed by the processing unit into the machine learning model to estimate a credit score.

Images