Hyper-parameter optimization method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of artificial intelligence, and relates to a hyper-parameter optimization method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the hyper-parameter initialization operation of a risk control model; performing first model training operation on the initialized risk control model according to the model training data and the model training label; carrying out integration operation on the initialized hyper-parameters and the performance index data; performing hyper-parameter optimization operation on initialization hyper-parameters in the state vector data according to a greedy algorithm; performing a second model training operation on the first risk control model according to the reward function and the optimized state vector data; cyclically executing hyper-parameter optimization operation and second model training operation; and when the second model training operation satisfies a preset iteration suspension condition, obtaining a target risk control model and a target hyper-parameter corresponding to the target risk control model. According to the method, the multiple hyper-parameters are dynamically adjusted through the reinforcement learning agent, and the model prediction performance and efficiency are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a hyperparameter optimization method, apparatus, computer equipment and storage medium. Background Art

[0002] Model hyperparameter optimization is a key task in the field of machine learning, which aims to improve the performance of a model by adjusting its hyperparameters.

[0003] There is an existing model hyperparameter optimization method, namely grid search, which is an exhaustive search method. A discrete search range is pre-set for each hyperparameter, and then all hyperparameter combinations are enumerated for model evaluation.

[0004] However, the applicant found that the above model hyperparameter optimization method has high computational cost, especially when the hyperparameter space dimension is high, the number of combinations increases exponentially, the search efficiency is low, and a lot of computing resources are wasted on invalid combinations. It can be seen that the traditional model hyperparameter optimization method has the problems of high computational cost and poor generalization. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a hyperparameter optimization method, apparatus, computer equipment and storage medium to solve the problems of high computational overhead and poor generalization in traditional model hyperparameter optimization methods.

[0006] In order to solve the above technical problems, the present application embodiment provides a hyperparameter optimization method, which adopts the following technical solution:

[0007] Reading a system database, and obtaining historical user data in the system database;

[0008] Preprocessing the historical user data to obtain model training data and model training labels;

[0009] Calling a risk control model and performing a hyperparameter initialization operation on the risk control model to obtain initialized hyperparameters and an initialized risk control model;

[0010] Performing a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data;

[0011] Integrating the initialization hyperparameters and the performance indicator data to obtain state vector data;

[0012] Performing a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data;

[0013] Performing a second model training operation on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model;

[0014] Loopingly executing the hyperparameter optimization operation and the second model training operation;

[0015] When the second model training operation satisfies the preset iteration termination condition, the target risk control model and the target hyperparameters corresponding to the target risk control model are obtained.

[0016] Furthermore, the step of looping to execute the hyperparameter optimization operation and the second model training operation specifically includes the following steps:

[0017] Calculate the training cost caused by the current hyperparameter optimization operation, where the training cost R complexity It is expressed as:

[0018] R complexity = -α×TrainingTime

[0019] Among them, α represents the hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination;

[0020] A hyperparameter optimization operation is performed on the hyperparameters in the current hyperparameter combination according to the training cost, and the second model training operation is performed.

[0021] Furthermore, the performance indicator data includes an AUC indicator value, a KS indicator value, and an F1-score indicator value, and the steps of performing a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the training overhead and executing the second model training operation specifically include the following steps:

[0022] The total reward function R of the hyperparameter is calculated according to the training cost, the AUC index value, the KS index value, and the F1-score index value, wherein the total reward function R is expressed as:

[0023] R=R AUC +R KS +R F1 +R complesity

[0024] R AUC =(ΔAUC) 1.5

[0025] Among them, ΔAUC represents the performance improvement of the risk control model, R AUC represents the reward for the performance improvement ΔAUC, R KS and R F1Respectively represent the rewards for improving the KS index value and the F1-score index value;

[0026] Perform a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the total reward function, and execute the second model training operation.

[0027] Further, performing a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the total reward function and executing the second model training operation specifically includes the following steps:

[0028] Storing data related to the hyperparameter optimization operation and the second model training operation into an experience pool;

[0029] Performing a random sampling operation in the experience pool to obtain a random sampling combination;

[0030] Performing an update operation on the deep reinforcement learning network according to the random sampling combination and the DDPG algorithm to obtain high-reward state vector data;

[0031] A hyperparameter optimization operation is performed on the high-reward state vector data according to the greedy algorithm, and the second model training operation is performed.

[0032] Furthermore, the step of performing a hyperparameter optimization operation on the high-reward state vector data according to the greedy algorithm and executing the second model training operation specifically includes the following steps:

[0033] After performing hyperparameter optimization on the state vector data of the high reward according to the greedy algorithm in each round, the proportion ε of the greedy algorithm is decayed according to an exponential decay function, wherein the exponential decay function is expressed as:

[0034] ε new =ε initial ×exp(-λ×t)

[0035] Among them, ε initial represents the initial exploration ratio, λ represents the decay rate, and t represents the current iteration round;

[0036] Perform hyperparameter optimization on the state vector data obtained in the previous round according to the greedy algorithm after attenuation processing;

[0037] The step of obtaining a target risk control model and target hyperparameters corresponding to the target risk control model when the second model training operation satisfies a preset iteration termination condition specifically includes the following steps:

[0038] After performing a hyperparameter optimization operation on the state vector data obtained in the previous round according to the greedy algorithm of the minimum attenuation function, the target risk control model and the target hyperparameters corresponding to the target risk control model are obtained.

[0039] Furthermore, the step of preprocessing the historical user data to obtain model training data and model training labels specifically includes the following steps:

[0040] Perform missing value filling and outlier processing operations on the historical user data to obtain pre-processed user data;

[0041] A feature screening operation is performed on the pre-processed user data according to the WOE-IV value to obtain the model training data and the model training label.

[0042] In order to solve the above technical problems, the embodiment of the present application also provides a hyperparameter optimization device, which adopts the following technical solution:

[0043] A historical data acquisition module, used for reading a system database and acquiring historical user data in the system database;

[0044] A preprocessing operation module, used to perform a preprocessing operation on the historical user data to obtain model training data and model training labels;

[0045] A hyperparameter initialization module is used to call the risk control model and perform a hyperparameter initialization operation on the risk control model to obtain initialized hyperparameters and initialize the risk control model;

[0046] A first model training module, configured to perform a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data;

[0047] An integration module, used to integrate the initialization hyperparameters and the performance indicator data to obtain state vector data;

[0048] A hyperparameter optimization module, used for performing a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data;

[0049] A second model training module, configured to perform a second model training operation on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model;

[0050] A loop execution module, used for looping the hyperparameter optimization operation and the second model training operation;

[0051] An iteration termination module is used to obtain a target risk control model and target hyperparameters corresponding to the target risk control model when the second model training operation meets a preset iteration termination condition.

[0052] Furthermore, the loop execution module includes:

[0053] The training cost calculation submodule is used to calculate the training cost caused by the current hyperparameter optimization operation, where the training cost R complexity It is expressed as:

[0054] R complexity = -α×TrainingTime

[0055] Among them, α represents the hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination;

[0056] A hyperparameter optimization submodule is used to perform a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the training overhead, and to execute the second model training operation.

[0057] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0058] The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the hyperparameter optimization method as described above when executing the computer-readable instructions.

[0059] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0060] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the hyperparameter optimization method described above are implemented.

[0061] The present application provides a hyperparameter optimization method, including: reading a system database, obtaining historical user data from the system database; performing a preprocessing operation on the historical user data to obtain model training data and a model training label; calling a risk control model, and performing a hyperparameter initialization operation on the risk control model to obtain initialized hyperparameters and an initialized risk control model; performing a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data; performing an integration operation on the initialized hyperparameters and the performance indicator data to obtain state vector data; performing a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data; performing a second model training operation on the first risk control model according to a reward function and the optimized state vector data to obtain a second risk control model; loopingly executing the hyperparameter optimization operation and the second model training operation; when the second model training operation meets a preset iteration termination condition, obtaining a target risk control model and a target hyperparameter corresponding to the target risk control model. Compared with the existing technology, this application realizes dynamic adjustment of multiple hyperparameters through reinforcement learning agents, significantly improving the model prediction performance and efficiency, especially showing obvious advantages in high-dimensional parameter space: hyperparameter optimization changes from manual adjustment to intelligent automatic adjustment. The action design is based on the increase and decrease ratio, with high exploration efficiency, balancing model performance and computing cost, and improving practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0064] Figure 2 It is a flowchart of the implementation of the hyperparameter optimization method provided in the embodiment of the present application;

[0065] Figure 3 It is a structural schematic diagram of a hyperparameter optimization device provided in an embodiment of the present application;

[0066] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0068] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0069] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0070] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0071] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0072] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0073] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0074] It should be noted that the hyperparameter optimization method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the hyperparameter optimization device is generally set in the server / terminal device.

[0075] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0076] Continue to refer Figure 2 , shows a flow chart of an embodiment of a hyperparameter optimization method according to the present application. The hyperparameter optimization method comprises: step S201, step S202, step S203, step S204, step S205, step S206, step S207, step S208 and step S209.

[0077] In step S201, the system database is read to obtain historical user data from the system database.

[0078] In the embodiment of the present application, the system first connects to its internal database system. This database stores a large amount of historical user data, including user transaction records, behavior patterns, credit scores, historical default records, and other information. The system extracts the required historical user data from these databases by executing SQL queries or other database operation languages, providing a data basis for subsequent model training and parameter optimization.

[0079] In step S202, a preprocessing operation is performed on the historical user data to obtain model training data and model training labels.

[0080] In the embodiments of the present application, the historical user data obtained often contains noise, missing values ​​or inconsistent formats, so preprocessing is required. This step includes data cleaning (such as processing missing values, removing duplicate data), data transformation (such as normalization, standardization), feature selection (selecting the most valuable features for the risk control model) and generating model training labels (such as marking users as "high risk" or "low risk" based on their default situation). The preprocessed data will be divided into model training data and model training labels for subsequent model training.

[0081] In step S203, the risk control model is called, and a hyperparameter initialization operation is performed on the risk control model to obtain initialized hyperparameters and an initialized risk control model.

[0082] In an embodiment of the present application, the system selects or constructs a risk control model, which is constructed based on a machine learning or deep learning algorithm. Subsequently, the system initializes the hyperparameters of the risk control model. Hyperparameters are parameters that need to be set before model training, and they have an important impact on the performance and training efficiency of the model. The process of initializing hyperparameters involves random assignment or setting initial values ​​based on experience.

[0083] In the embodiment of the present application, the hyperparameters of the risk control model refer to random forest hyperparameters, and the hyperparameter initialization operation can be: initializing the hyperparameter range, that is, random forest: the number of trees (n_estimators): initial range [50,500]; maximum depth (max_depth): initial range [5,50]; minimum number of split samples (min_samples_split): initial range [2,10]; minimum number of leaf node samples (min_samples_leaf): initial range [1,5]; MLP: number of hidden layer neurons ( hidden_layer_sizes): [(50,), (100,), (200,)]; activation function (activation): ['relu', 'tanh']; learning rate (learning_rate_init): initial range [0.001, 0.01]; regularization coefficient (alpha): initial range [0.0001, 0.01]; batch size (batch_size): initial range [32, 128]; optimizer (solver): ['adam', 'sgd'].

[0084] In step S204, a first model training operation is performed on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data.

[0085] In the embodiment of the present application, the system trains the initialization risk control model using the preprocessed model training data and model training labels. This training process may involve algorithms such as gradient descent and back propagation, the purpose of which is to adjust the parameters within the model so that the model can better fit the training data. After the training is completed, the system will evaluate the performance indicator data of the model, such as accuracy, recall rate, F1 score, etc., to evaluate the performance of the model.

[0086] In step S205, the initialized hyperparameters and performance indicator data are integrated to obtain state vector data.

[0087] In the embodiment of the present application, in order to perform hyperparameter optimization, the system needs to integrate the initialization hyperparameters and performance index data into a state vector data. This state vector data contains all the key information of the current model training state, which is used to guide the subsequent hyperparameter optimization process.

[0088] In the embodiment of the present application, the current hyperparameter combination is: such as [n_estimators=100, max_depth=10, min_samples_split=2, learning_rate_init=0.001].

[0089] In the embodiments of the present application, the model performance indicators refer to AUC, KS value, and F1-score on the validation set.

[0090] In the embodiment of the present application, the state vector is represented by a combination of hyperparameters and performance indicators.

[0091] In step S206, a hyperparameter optimization operation is performed on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data.

[0092] In the embodiment of the present application, the system uses a greedy algorithm to optimize the initialization hyperparameters in the state vector data. A greedy algorithm is an algorithm that takes the best or optimal (i.e., most favorable) choice at each step, hoping to result in the best or optimal result globally. By continuously adjusting the hyperparameters, the system attempts to find a set of hyperparameter combinations that can improve model performance.

[0093] In this embodiment of the present application, the hyperparameter optimization operation consists of the following options:

[0094] ① Random Forest: n_estimators increase or decrease by 10% (such as adjusting from 100 to 110 or 90); max_depth increase or decrease by 10%; min_samples_split increase or decrease by 10%; min_samples_leaf increase or decrease by 10%; MLP: hidden_layer_sizes increase or decrease the number of neurons by 10%; learning_rate_init increase or decrease by 10%; alpha increase or decrease by 10%; batch_size increase or decrease by 10%;

[0095] ②Switch the activation function (activation) or optimizer (solver).

[0096] In an embodiment of the present application, the mechanism of the hyperparameter optimization operation can be: adopting an ε-greedy strategy (ε-greedy), that is, randomly selecting actions with a probability of ε to explore new hyperparameter combinations, and selecting the action with the highest historical reward for use with a probability of 1-ε. After each action, the state vector is updated to reflect the current hyperparameter combination.

[0097] In step S207, a second model training operation is performed on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model.

[0098] In the embodiment of the present application, after the hyperparameters are optimized, the system will perform a second training on the risk control model based on the optimized state data (including the optimized hyperparameters). At the same time, the system will introduce a reward function to evaluate the performance improvement of the model after each hyperparameter adjustment. The design of the reward function is usually related to the performance indicator data of the model, and aims to encourage the model to develop in the direction of higher performance.

[0099] In the embodiment of the present application, after the hyperparameter optimization operation is performed, the random forest and MLP models are trained respectively using the updated hyperparameter combination, and the performance is evaluated on the validation set:

[0100] Performance indicators: AUC, KS value and F1-score.

[0101] Computational cost: records the training time and resource usage as part of the training overhead.

[0102] Performance Recording: Save the current hyperparameter combination and its corresponding performance.

[0103] In step S208, the hyperparameter optimization operation and the second model training operation are executed in a loop.

[0104] In the embodiment of the present application, in order to improve the performance of the model, the system will cyclically execute steps S206 and S207, and continuously improve the risk control model through hyperparameter optimization and second model training. This cyclic process may continue for multiple iterations until the preset iteration termination condition is met.

[0105] In step S209, when the second model training operation satisfies the preset iteration termination condition, the target risk control model and the target hyperparameters corresponding to the target risk control model are obtained.

[0106] In the embodiment of the present application, when the second model training operation meets the preset iteration termination conditions (such as reaching the maximum number of iterations, the model performance improvement is no longer significant, etc.), the system stops the loop and outputs the final target risk control model and the corresponding target hyperparameters. This target risk control model is obtained after multiple iterations and optimizations, has high performance and stability, and can be used in actual risk control scenarios.

[0107] In some optional implementations of the embodiments of the present application, the iteration termination condition may be that after 9999 iterations, if the sum of the growth rates of AUC, KS, and F1-score for three consecutive steps is less than 0.001, training is stopped and the current hyperparameters are considered to be the target hyperparameters.

[0108] In an embodiment of the present application, a hyperparameter optimization method is provided, including: reading a system database, obtaining historical user data from the system database; performing preprocessing operations on the historical user data to obtain model training data and model training labels; calling a risk control model, and performing a hyperparameter initialization operation on the risk control model to obtain initialized hyperparameters and an initialized risk control model; performing a first model training operation on the initialized risk control model according to the model training data and the model training labels to obtain a first risk control model and performance indicator data; performing an integration operation on the initialized hyperparameters and the performance indicator data to obtain state vector data; performing a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data; performing a second model training operation on the first risk control model according to a reward function and the optimized state vector data to obtain a second risk control model; loopingly executing the hyperparameter optimization operation and the second model training operation; when the second model training operation meets a preset iteration termination condition, obtaining a target risk control model and a target hyperparameter corresponding to the target risk control model. Compared with the existing technology, this application realizes dynamic adjustment of multiple hyperparameters through reinforcement learning agents, significantly improving the model prediction performance and efficiency, especially showing obvious advantages in high-dimensional parameter space: hyperparameter optimization changes from manual adjustment to intelligent automatic adjustment. The action design is based on the increase and decrease ratio, with high exploration efficiency, balancing model performance and computing cost, and improving practicality.

[0109] In some optional implementations of the embodiments of the present application, the above-mentioned loop performs the steps of the hyperparameter optimization operation and the second model training operation, specifically including the following steps:

[0110] Calculate the training cost caused by the current hyperparameter optimization operation, where the training cost R complexity It is expressed as:

[0111] R complexity = -α×TrainingTime

[0112] Among them, α represents the hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination;

[0113] A hyperparameter optimization operation is performed on the hyperparameters in the current hyperparameter combination according to the training cost, and a second model training operation is performed.

[0114] In the embodiment of the present application, the model training cost is calculated according to the number of features to optimize the hyperparameters. The training cost calculation formula is:

[0115] R complexity = -α×N features

[0116] Among them, α is a hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination. This penalty is used to keep the model simple.

[0117] In some optional implementations of the embodiments of the present application, the above-mentioned performance indicator data includes an AUC indicator value, a KS indicator value, and an F1-score indicator value, and a hyperparameter optimization operation is performed on the hyperparameters in the current hyperparameter combination according to the training cost, and the steps of performing the second model training operation specifically include the following steps:

[0118] The total reward function R of the hyperparameters is calculated based on the training cost, AUC index value, KS index value, and F1-score index value. The total reward function R is expressed as:

[0119] R=R AUC +R KS +R F1 +R complesity

[0120] R AUC =(ΔAUC) 1.5

[0121] Among them, ΔAUC represents the performance improvement of the risk control model, R AUC Reward for performance improvement ΔAUC, R KS and R F1Respectively represent the rewards for improving the KS index value and the F1-score index value;

[0122] A hyperparameter optimization operation is performed on the hyperparameters in the current hyperparameter combination according to the total reward function, and a second model training operation is performed.

[0123] In the embodiment of the present application, the purpose of designing the reward function is to encourage the present application to select hyperparameters that improve model performance.

[0124] In the embodiment of the present application, model performance reward: the reward is proportional to the performance improvement, for example:

[0125] R AUC =(AUC) 1.5

[0126] Training Overhead: Calculates the training overhead caused by hyperparameter tuning, for example:

[0127] R complexity =-α·TrainingTime

[0128] Among them, TrainingTime represents the training time of the current hyperparameter combination, and α represents the weight coefficient.

[0129] Total Reward: The reward is defined by the following formula:

[0130] R=R AUC +R KS +R F1 +R complexity

[0131] 2) Reward feedback mechanism:

[0132] After each action, the agent calculates the total reward based on the reward function and uses the result to update the strategy.

[0133] In some optional implementations of the embodiments of the present application, the above-mentioned step of performing a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the total reward function and performing a second model training operation specifically includes the following steps:

[0134] Store the data related to the hyperparameter optimization operation and the second model training operation into the experience pool;

[0135] Perform random sampling operation in the experience pool to obtain random sampling combinations;

[0136] Update the deep reinforcement learning network based on random sampling combinations and the DDPG algorithm to obtain high-reward state vector data;

[0137] A hyperparameter optimization operation is performed on the state vector data with high rewards according to the greedy algorithm, and a second model training operation is performed.

[0138] In the embodiment of the present application, a deep reinforcement learning method based on DDPG is adopted:

[0139] 1.Actor network: outputs continuous actions (such as adjusting n_estimators to increase by 10%).

[0140] 2.Critic network: evaluates the Q value of the action.

[0141] 3. Experience replay: save state-action-reward-new state, and randomly sample for training.

[0142] 4. Target network: Regularly update the target network weights to ensure strategy stability.

[0143] In some optional implementations of the embodiments of the present application, the above-mentioned steps of performing a hyperparameter optimization operation on the high-reward state vector data according to the greedy algorithm and performing a second model training operation specifically include the following steps:

[0144] After each round of hyperparameter optimization of the state vector data with high rewards according to the greedy algorithm, the proportion ε of the greedy algorithm is decayed according to the exponential decay function, where the exponential decay function is expressed as:

[0145] ε new =ε initial ×exp(-λ×t)

[0146] Among them, ε initial represents the initial exploration ratio, λ represents the decay rate, and t represents the current iteration round;

[0147] Perform hyperparameter optimization on the state vector data obtained in the previous round according to the greedy algorithm after attenuation processing;

[0148] When the second model training operation satisfies the preset iteration termination condition, the step of obtaining the target risk control model and the target hyperparameters corresponding to the target risk control model specifically includes the following steps:

[0149] After performing hyperparameter optimization on the state vector data obtained in the previous round according to the greedy algorithm of the minimum attenuation function, the target risk control model and the target hyperparameters corresponding to the target risk control model are obtained.

[0150] In the embodiment of the present application, a decaying ε-greedy strategy is adopted, with a high initial exploration ratio, and the ε value is gradually reduced as the training progresses, ensuring that the present application explores a variety of hyperparameters in the early stage of training, and pays more attention to the use of existing strategies in the later stage. Specifically:

[0151] Initial value: The initial value of the exploration ratio ε is set to 0.9, which means that in the initial stage, this application has a 90% probability of selecting a random action to ensure the diversity of hyperparameters.

[0152] Attenuation method: After each round of iteration, the exploration ratio ε is gradually reduced according to the exponential decay function, the formula is:

[0153] ε new =ε initial ×exp(-λ×t)

[0154] Among them, ε initial represents the initial exploration ratio, λ represents the decay rate, and t represents the current iteration round.

[0155] Minimum exploration value: When ε decays to 0.1, it stops decreasing further and maintains a minimum exploration probability of 10% to ensure that the application can still perform a small amount of exploration in the later stages of training to cope with data changes and small opportunities for improvement in the feature space.

[0156] In some optional implementations of the embodiments of the present application, the above-mentioned step of preprocessing the historical user data to obtain model training data and model training labels specifically includes the following steps:

[0157] Perform missing value filling and outlier processing operations on historical user data to obtain pre-processed user data;

[0158] The pre-processed user data is subjected to feature screening operations according to the WOE-IV value to obtain model training data and model training labels.

[0159] In actual applications, historical user data usually comes from the basic customer information data, order transaction records, payment data and credit behavior data of financial institutions, including: basic information features: the company's operating years, operating status, risk labels, etc.; order information features: order volume, order frequency, order amount, return rate, etc.; credit behavior information features: payment data, number of overdue payments, overdue amount, etc. Among them, the label is the customer's overdue status, for example: 1 means the customer is overdue (for example, more than 30 days without payment), 0 means the customer is not overdue.

[0160] After obtaining historical user data, the features and labels involved in modeling are determined after data preprocessing such as missing value filling and outlier processing and feature screening based on WOE-IV values;

[0161] Select Random Forest as the training model and use reinforcement learning for parameter tuning. The specific steps are as follows:

[0162] ① Initial state:

[0163] Random forest hyperparameters: [n_estimators = 100, max_depth = 10, min_samples_split = 2], MLP hyperparameters: [hidden_layer_sizes = (100,), learning_rate_init = 0.001, activation = 'relu'], and after training the model based on the initial hyperparameters, AUC, KS, and F1-score are calculated on the validation set;

[0164] ②Action execution:

[0165] Agent selection Increase n_estimators by 10%, from 100 to 110. Decrease learning_rate_init by 10%, from 0.001 to 0.0009;

[0166] ③Model training:

[0167] The model was trained using the adjusted hyperparameter combination. The validation set AUC changed to 0.85, the KS value changed to 0.5, and the F1-score value changed to 0.51;

[0168] Determine whether to terminate training:

[0169] Determine whether the following conditions are met: 1. The iteration steps are more than 9999; 2. The sum of the growth rates of AUC, KS, and F1-score for three consecutive steps is less than 0.001. If so, stop training and consider the current hyperparameters as the target hyperparameters. Otherwise, continue iterating.

[0170] Reward feedback: Calculate the total reward based on the reward function, and use the feedback and status to update the optimization strategy;

[0171] Actual effect: After multiple rounds of iterations, the agent found the optimal hyperparameter combination: Random Forest: [n_estimators = 300, max_depth = 15, min_samples_split = 4] MLP: [hidden_layer_sizes = (150,), learning_rate_init = 0.0011, activation = 'relu'];

[0172] The final model effect is: AUC reaches 0.87, KS value is 0.53, and F1-score value is 0.62.

[0173] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0174] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0175] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0176] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0177] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a hyperparameter optimization device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0178] like Figure 3 As shown, the hyperparameter optimization device 200 of the embodiment of the present application includes:

[0179] The historical data acquisition module 210 is used to read the system database and obtain historical user data from the system database;

[0180] A preprocessing operation module 220 is used to perform a preprocessing operation on the historical user data to obtain model training data and model training labels;

[0181] The hyperparameter initialization module 230 is used to call the risk control model and perform a hyperparameter initialization operation on the risk control model to obtain the initialized hyperparameters and the initialized risk control model;

[0182] A first model training module 240 is used to perform a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data;

[0183] An integration module 250 is used to integrate the initialization hyperparameters and performance indicator data to obtain state vector data;

[0184] A hyperparameter optimization module 260 is used to perform a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data;

[0185] A second model training module 270 is used to perform a second model training operation on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model;

[0186] A loop execution module 280, used for looping the hyperparameter optimization operation and the second model training operation;

[0187] The iteration termination module 290 is used to obtain the target risk control model and the target hyperparameters corresponding to the target risk control model when the second model training operation meets the preset iteration termination condition.

[0188] In an embodiment of the present application, a hyperparameter optimization device 200 is provided, including: a historical data acquisition module 210, which is used to read a system database and obtain historical user data in the system database; a preprocessing operation module 220, which is used to perform a preprocessing operation on the historical user data to obtain model training data and a model training label; a hyperparameter initialization module 230, which is used to call a risk control model and perform a hyperparameter initialization operation on the risk control model to obtain an initialized hyperparameter and an initialized risk control model; a first model training module 240, which is used to perform a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data; an integration module 250 , used to integrate the initialization hyperparameters and performance indicator data to obtain state vector data; hyperparameter optimization module 260, used to perform hyperparameter optimization operations on the initialization hyperparameters in the state vector data according to the greedy algorithm to obtain optimized state vector data; second model training module 270, used to perform a second model training operation on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model; loop execution module 280, used to loop the hyperparameter optimization operation and the second model training operation; iteration termination module 290, used to obtain the target risk control model and the target hyperparameters corresponding to the target risk control model when the second model training operation meets the preset iteration termination condition. Compared with the prior art, the present application realizes dynamic adjustment of multiple hyperparameters through reinforcement learning agents, significantly improves the model prediction performance and efficiency, especially in high-dimensional parameter space, showing obvious advantages: hyperparameter optimization changes from manual adjustment to intelligent automatic adjustment. The action design is based on the increase and decrease ratio, with high exploration efficiency, balancing model performance and computational cost, and improving practicality.

[0189] In some optional implementations of the embodiments of the present application, the above-mentioned loop execution module includes:

[0190] The training cost calculation submodule is used to calculate the training cost caused by the current hyperparameter optimization operation. The training cost R complexity It is expressed as:

[0191] R complexity = -α×TrainingTime

[0192] Among them, α represents the hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination;

[0193] The hyperparameter optimization submodule is used to perform a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the training cost, and to perform a second model training operation.

[0194] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0195] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0196] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0197] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions of the hyperparameter optimization method, etc. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0198] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions or process data stored in the memory 310, such as computer-readable instructions for running the hyperparameter optimization method.

[0199] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0200] The computer device provided by this application realizes dynamic adjustment of multiple hyperparameters through reinforcement learning agents, significantly improving the model prediction performance and efficiency, especially showing obvious advantages in high-dimensional parameter space: hyperparameter optimization changes from manual adjustment to intelligent automatic adjustment. The action design is based on the increase and decrease ratio, with high exploration efficiency, balancing model performance and computing cost, and improving practicality.

[0201] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the hyperparameter optimization method as described above.

[0202] The computer-readable storage medium provided by the present application realizes dynamic adjustment of multiple hyperparameters through reinforcement learning agents, significantly improving the model prediction performance and efficiency, especially showing obvious advantages in high-dimensional parameter space: hyperparameter optimization changes from manual adjustment to intelligent automatic adjustment. The action design is based on the increase and decrease ratio, with high exploration efficiency, balancing model performance and computing cost, and improving practicality.

[0203] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0204] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A hyperparameter optimization method, characterized in that: The steps include: Reading a system database, and obtaining historical user data in the system database; Preprocessing the historical user data to obtain model training data and model training labels; Calling a risk control model and performing a hyperparameter initialization operation on the risk control model to obtain initialized hyperparameters and an initialized risk control model; Performing a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data; Integrating the initialization hyperparameters and the performance indicator data to obtain state vector data; Performing a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data; Performing a second model training operation on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model; Loopingly executing the hyperparameter optimization operation and the second model training operation; When the second model training operation satisfies the preset iteration termination condition, the target risk control model and the target hyperparameters corresponding to the target risk control model are obtained.

2. The hyperparameter optimization method according to claim 1, characterized in that: The steps of looping to perform the hyperparameter optimization operation and the second model training operation specifically include the following steps: Calculate the training cost caused by the current hyperparameter optimization operation, where the training cost R complexity It is expressed as: R complexity =-α×TrainingTime Among them, α represents the hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination; A hyperparameter optimization operation is performed on the hyperparameters in the current hyperparameter combination according to the training cost, and the second model training operation is performed.

3. The hyperparameter optimization method according to claim 2, characterized in that: The performance indicator data includes an AUC indicator value, a KS indicator value, and an F1-score indicator value. The step of performing a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the training cost and executing the second model training operation specifically includes the following steps: The total reward function R of the hyperparameter is calculated according to the training cost, the AUC index value, the KS index value, and the F1-score index value, wherein the total reward function R is expressed as: R=R AUC +R KS +R F1 +R complesity R AUC =(ΔAUC) 1.5 Among them, ΔAUC represents the performance improvement of the risk control model, R AUC represents the reward for the performance improvement ΔAUC, R KS and R F1 Respectively represent the rewards for improving the KS index value and the F1-score index value; Perform a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the total reward function, and execute the second model training operation.

4. The hyperparameter optimization method according to claim 3, characterized in that: The step of performing a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the total reward function and executing the second model training operation specifically includes the following steps: Storing data related to the hyperparameter optimization operation and the second model training operation into an experience pool; Performing a random sampling operation in the experience pool to obtain a random sampling combination; Performing an update operation on the deep reinforcement learning network according to the random sampling combination and the DDPG algorithm to obtain high-reward state vector data; A hyperparameter optimization operation is performed on the high-reward state vector data according to the greedy algorithm, and the second model training operation is performed.

5. The hyperparameter optimization method according to claim 4, characterized in that: The step of performing a hyperparameter optimization operation on the high-reward state vector data according to the greedy algorithm and executing the second model training operation specifically includes the following steps: After performing hyperparameter optimization on the state vector data of the high reward according to the greedy algorithm in each round, the proportion ε of the greedy algorithm is decayed according to an exponential decay function, wherein the exponential decay function is expressed as: e new =e initial ×exp(-λ×t) Among them, ε initial represents the initial exploration ratio, λ represents the decay rate, and t represents the current iteration round; Perform hyperparameter optimization on the state vector data obtained in the previous round according to the greedy algorithm after attenuation processing; The step of obtaining a target risk control model and target hyperparameters corresponding to the target risk control model when the second model training operation satisfies a preset iteration termination condition specifically includes the following steps: After performing a hyperparameter optimization operation on the state vector data obtained in the previous round according to the greedy algorithm of the minimum attenuation function, the target risk control model and the target hyperparameters corresponding to the target risk control model are obtained.

6. The hyperparameter optimization method according to claim 1, characterized in that: The step of preprocessing the historical user data to obtain model training data and model training labels specifically includes the following steps: Perform missing value filling and outlier processing operations on the historical user data to obtain pre-processed user data; A feature screening operation is performed on the pre-processed user data according to the WOE-IV value to obtain the model training data and the model training label.

7. A hyperparameter optimization device, characterized in that: include: A historical data acquisition module, used for reading a system database and acquiring historical user data in the system database; A preprocessing operation module, used to perform a preprocessing operation on the historical user data to obtain model training data and model training labels; A hyperparameter initialization module is used to call the risk control model and perform a hyperparameter initialization operation on the risk control model to obtain initialized hyperparameters and initialize the risk control model; A first model training module, configured to perform a first model training operation on the initialized risk control model according to the model training data and the model training label to obtain a first risk control model and performance indicator data; An integration module, used to integrate the initialization hyperparameters and the performance indicator data to obtain state vector data; A hyperparameter optimization module, used for performing a hyperparameter optimization operation on the initialized hyperparameters in the state vector data according to a greedy algorithm to obtain optimized state vector data; A second model training module, configured to perform a second model training operation on the first risk control model according to the reward function and the optimized state vector data to obtain a second risk control model; A loop execution module, used for looping the hyperparameter optimization operation and the second model training operation; An iteration termination module is used to obtain a target risk control model and target hyperparameters corresponding to the target risk control model when the second model training operation meets a preset iteration termination condition.

8. The hyperparameter optimization device according to claim 7, characterized in that: The loop execution module includes: The training cost calculation submodule is used to calculate the training cost caused by the current hyperparameter optimization operation, where the training cost R complexity It is expressed as: R complexity =-α×TrainingTime Among them, α represents the hyperparameter, and TrainingTime represents the training time of the current hyperparameter combination; A hyperparameter optimization submodule is used to perform a hyperparameter optimization operation on the hyperparameters in the current hyperparameter combination according to the training overhead, and to execute the second model training operation.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the hyperparameter optimization method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the hyperparameter optimization method according to any one of claims 1 to 6 are implemented.

Images