Redundant feature detection method, detection device, electronic device and medium

Abstract

The present disclosure provides a redundant feature detection method for a model, which relates to the fields of artificial intelligence and finance. The redundant feature detection method includes: obtaining a first feature set and a first indicator of a model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after the initial model is trained based on a training set using the first feature set and a first machine learning algorithm. A second feature set is obtained by eliminating M features from the first feature set. A reconstructed model is obtained, wherein the reconstructed model includes a model with second parameter information obtained after the initial model is trained based on the training set using the second feature set and the first machine learning algorithm. When the ratio of the second indicator of the reconstructed model to the first indicator is greater than or equal to a first preset threshold, the M features are determined to be redundant features. The present disclosure also provides a redundant feature detection device, an electronic device, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and finance, and more specifically, to a redundant feature detection method, a detection device, an electronic device and a storage medium for a model. Background Art

[0002] Due to the unique advantages of machine learning models in risk prevention and control, smart marketing and other fields, more and more companies are using machine learning models in daily production. For example, in the banking industry, machine learning models can be used in credit review, risk review, smart marketing and other businesses. When building a machine learning model, it is necessary to select a series of features related to the prediction target, such as financial features, credit features, behavioral features, etc. Then train the machine learning model so that the model learns the judgment rules in these features, so as to achieve the prediction of the target after the training is completed. At present, the evaluation of the model after training is mainly based on various evaluation indicators. If the indicator performance is good, the model effect is considered good.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that there are at least the following problems in the prior art:

[0004] Since the relevant evaluation indicators cannot reflect whether the model to be evaluated has introduced redundant features, and there is currently no effective method to detect redundant features for machine learning models after training, the model will be at risk of overfitting and may cause a waste of computing power due to processing redundant features. Summary of the invention

[0005] In view of this, the embodiments of the present disclosure provide a method and apparatus, as well as an electronic device and a storage medium, which are capable of performing redundant feature detection on a machine learning model.

[0006] One aspect of an embodiment of the present disclosure provides a redundant feature detection method for a model. The redundant feature detection method includes: obtaining a first feature set and a first index of a model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after training an initial model based on a training set using the first feature set and a first machine learning algorithm, and the first index is used to characterize the prediction performance of the model to be evaluated on a test set. M features are eliminated from the first feature set to obtain a second feature set, wherein M is an integer greater than or equal to 1. A reconstructed model is obtained, wherein the reconstructed model includes a model with second parameter information obtained after training the initial model based on the training set using the second feature set and the first machine learning algorithm. When the ratio of the second index of the reconstructed model to the first index is greater than or equal to a first preset threshold, the M features are determined to be redundant features, wherein the second index is used to characterize the prediction performance of the reconstructed model on the test set.

[0007] According to an embodiment of the present disclosure, the method includes repeatedly performing the operations of eliminating, obtaining a reconstruction model, and determining redundant features until all features in the first feature set are detected. Then, eliminating M features from the first feature set to obtain a second feature set includes: eliminating features one by one from the first feature set. Wherein, when a feature is determined to be a non-redundant feature, the feature is put back in the next detection process and the operations of eliminating, obtaining a reconstruction model, and determining redundant features are repeated.

[0008] According to an embodiment of the present disclosure, removing features one by one from the first feature set includes: assigning an initial value of i to any value in [1, N], looping and performing the following operations until the loop is N times, wherein N is the number of features in the first feature set, and i is an integer greater than or equal to 1: removing the i-th feature from the first feature set to obtain the corresponding second feature set. When it is determined that the i-th feature is a redundant feature, removing the i-th feature, and then setting i=i+1. When it is determined that the i-th feature is a non-redundant feature, replacing the i-th feature, and then setting i=i+1.

[0009] According to an embodiment of the present disclosure, after removing the i-th feature from the first feature set and obtaining the corresponding second feature set, obtaining the reconstructed model includes: configuring a training process based on the first hyperparameter information of the model to be evaluated, using the second feature set and the first machine learning algorithm to train the initial model based on the training set, and obtaining the reconstructed model.

[0010] According to an embodiment of the present disclosure, the step of removing M features from the first feature set to obtain the second feature set includes: obtaining a proxy model, wherein the proxy model includes a model with third parameter information obtained after the initial model is trained based on the training set using the first feature set and a second machine learning algorithm. When the ratio of the third index of the proxy model to the first index is greater than or equal to the first preset threshold, the importance of each feature in the first feature set is obtained based on the second machine learning algorithm, wherein the third index is used to characterize the predictive performance of the proxy model for the test set. Based on the importance of each feature, M features whose importance is less than or equal to the second preset threshold are removed from the first feature set to obtain the second feature set.

[0011] According to an embodiment of the present disclosure, obtaining the reconstructed model includes: obtaining second hyperparameter information, wherein the second hyperparameter information is different from the first hyperparameter information of the model to be evaluated. Based on the second hyperparameter information, a training process is configured, and the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain the reconstructed model.

[0012] According to an embodiment of the present disclosure, the first feature set includes N features, wherein N is greater than M. After determining that the M features are redundant features, the method further includes: when any feature is removed from the second feature set and the ratio of the second index of the reconstructed model to the first index is less than the first preset threshold, determining that the number of redundant features of the first feature set is M. And performing redundancy evaluation on the model to be evaluated based on the ratio of M to N.

[0013] Another aspect of the embodiment of the present disclosure provides a redundant feature detection device for a model. The redundant feature detection device includes an acquisition module, a removal module, a reconstruction module and a determination module. The acquisition module is used to obtain a first feature set and a first index of the model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after the initial model is trained based on the training set using the first feature set and the first machine learning algorithm, and the first index is used to characterize the prediction performance of the model to be evaluated on the test set. The removal module is used to remove M features from the first feature set to obtain a second feature set, wherein M is an integer greater than or equal to 1. The reconstruction module is used to obtain a reconstructed model, wherein the reconstructed model includes a model with second parameter information obtained after the initial model is trained based on the training set using the second feature set and the first machine learning algorithm. The determination module is used to determine that the M features are redundant features when the ratio of the second index of the reconstructed model to the first index is greater than or equal to a first preset threshold, wherein the second index is used to characterize the prediction performance of the reconstructed model on the test set.

[0014] Another aspect of the embodiment of the present disclosure provides an electronic device. The electronic device includes one or more memories and one or more processors. The memories store executable instructions. The processor executes the executable instructions to implement the above method.

[0015] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0016] Another aspect of the embodiments of the present disclosure provides a computer program product, including a computer program / instruction, which implements the above-mentioned method when executed by a processor.

[0017] One or more of the above-mentioned embodiments have the following advantages or beneficial effects: it can at least partially solve the problem that redundant features of the machine learning model after training cannot be effectively detected, and by using the second feature set and the first machine learning algorithm to train the initial model based on the training set, a reconstructed model with second parameter information is obtained, and then the prediction performance of the model to be evaluated is compared with the reconstructed model to determine whether the features eliminated from the first feature set are redundant features, so that the redundant features of the model to be evaluated can be effectively detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 An exemplary system architecture to which a redundant feature detection method can be applied according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 2 The flowchart of the redundant feature detection method of the model according to the embodiment of the present disclosure is schematically shown;

[0021] Figure 3 A flowchart of a redundant feature detection method for a model according to another embodiment of the present disclosure is schematically shown;

[0022] Figure 4 A flowchart of eliminating features one by one from a first feature set according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 5 The flowchart of obtaining the second feature set according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 6 The flowchart of obtaining the reconstruction model according to the embodiment of the present disclosure is schematically shown;

[0025] Figure 7 A flowchart of performing redundancy evaluation on a model to be evaluated according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 8 A block diagram schematically shows a redundant feature detection device for a model according to an embodiment of the present disclosure;

[0027] Fig. 9A flowchart schematically illustrates an operation of performing redundant feature detection using a redundant feature detection device according to an embodiment of the present disclosure; and

[0028] Fig.10 A block diagram of a computer system suitable for implementing a redundant feature detection method and apparatus according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0030] An embodiment of the present disclosure provides a redundant feature detection method for a model. The redundant feature detection method includes: obtaining a first feature set and a first indicator of a model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after training an initial model based on a training set using the first feature set and a first machine learning algorithm, and the first indicator is used to characterize the prediction performance of the model to be evaluated on a test set. M features are eliminated from the first feature set to obtain a second feature set, wherein M is an integer greater than or equal to 1. A reconstructed model is obtained, wherein the reconstructed model includes a model with second parameter information obtained after training the initial model based on a training set using a second feature set and a first machine learning algorithm. When the ratio of the second indicator of the reconstructed model to the first indicator is greater than or equal to a first preset threshold, the M features are determined to be redundant features, wherein the second indicator is used to characterize the prediction performance of the reconstructed model on the test set.

[0031] According to an embodiment of the present disclosure, redundant feature detection of a machine learning model refers to, for example, detecting the features used by the machine learning model to check whether the features used by the machine learning model are all necessary. If the machine learning model relies on a large number of non-essential features (i.e., redundant features), the performance of the machine learning model will be disturbed, affecting the actual use effect after production.

[0032] Specifically, for simple tasks, good results can be achieved by selecting only a few features to build a simple model. For complex tasks, a large number of features need to be selected from various dimensions to build a complex model. However, although complex models can better fit the training data, they also have the risk of overfitting. Overfitting means that the model over-learns the training data and learns rules that are unique to the training data, and such rules are not universally present in the full amount of data. In other words, when features are redundant, the model is at risk of overfitting. It will not perform well on the test set for the time being, but it is more likely to overfit when faced with more diverse data in the future, resulting in a decrease in the model's prediction performance.

[0033] By using the redundant feature detection method of the embodiment of the present disclosure, the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain a reconstructed model with second parameter information, and then the prediction performance of the model to be evaluated is compared with the reconstructed model to determine whether the features removed from the first feature set are redundant features, so that the redundant features of the model to be evaluated can be effectively detected. After the redundant features are detected, the redundant features can be removed to optimize the model to be evaluated. For example, the reconstructed model can be used to replace the model to be evaluated to avoid the waste of computing power caused by processing redundant features.

[0034] It should be noted that the redundant feature detection method and device of the model in the embodiment of the present disclosure can be used in the financial field, and can also be used in any field other than the financial field. The application field of the redundant feature detection method and device of the model in the embodiment of the present disclosure is not specifically limited.

[0035] Figure 1 The exemplary system architecture 100 to which the redundant feature detection method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0036] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0037] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0038] According to an embodiment of the present disclosure, user data can be collected through terminal devices 101, 102, and 103 to form a training set or a test set and stored in the server 105, wherein the collected user data can be analyzed, counted, and processed from various dimensions to obtain a set containing multiple features. Among them, the server 105 can use the first feature set and the first machine learning algorithm to train the initial model based on the training set to obtain a model to be evaluated with first parameter information, and can also use the second feature set and the first machine learning algorithm to train the initial model based on the training set to obtain a reconstructed model with second parameter information. After the training is completed, the model to be evaluated can be used to predict the user data in the test set to obtain an indicator reflecting the performance of the model to be evaluated.

[0039] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0040] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device, for example, the evaluation index (first index or second index) of the model to be evaluated or the reconstructed model predicting the test set may be fed back to the terminal device.

[0041] It should be noted that the redundant feature detection method of the model provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the redundant feature detection device of the model provided in the embodiment of the present disclosure can generally be set in the server 105. The redundant feature detection method of the model provided in the embodiment of the present disclosure can also be executed by a server or a server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the redundant feature detection device of the model provided in the embodiment of the present disclosure can also be set in a server or a server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0043] Figure 2 The flowchart of the redundant feature detection method of the model according to the embodiment of the present disclosure is schematically shown.

[0044] like Figure 2 As shown, the redundant feature detection method of the model according to the embodiment of the present disclosure may include operations S210 to S240.

[0045] In operation S210, a first feature set and a first indicator of the model to be evaluated are obtained, wherein the model to be evaluated includes a model with first parameter information obtained after an initial model is trained based on a training set using the first feature set and a first machine learning algorithm, and the first indicator is used to characterize the prediction performance of the model to be evaluated on a test set.

[0046] According to an embodiment of the present disclosure, taking a banking credit business risk management model as an example, the first feature set may include feature data obtained by processing financial data, industrial and commercial data, credit data, fund transaction data, public opinion data and other data. The above multiple feature data may reflect the personal financial attributes of a credit user.

[0047] When the risk management model is a binary classification model, for example, if the user is a risky user, "0" is output, and if the user is a normal user, "1" is output. Then the first indicator may be the AUC (Area Under Curve) value of the model.

[0048] When the risk management model is a regression model, the model can output the default probability of each customer. For example, if the default probability is greater than or equal to 60%, it represents a default customer, and if the default probability is less than 50%, it represents a normal customer. If it is between 50% and 60%, it represents a customer to be reviewed (for example only). Then the first indicator can be the mean square error value of the model.

[0049] According to an embodiment of the present disclosure, the first machine learning algorithm may be a linear regression algorithm, a logistic regression algorithm, a naive Bayes algorithm, a KNN algorithm, a random forest algorithm, and the like.

[0050] According to an embodiment of the present disclosure, in addition to obtaining the first feature set and the first indicator of the model to be evaluated, the structural information and hyperparameter information of the model to be evaluated can also be obtained. The hyperparameter information may include the hyperparameters set when training the model, and different algorithms involve different hyperparameters, such as learning rate, random number seed, regularization term, etc.

[0051] In operation S220, M features are removed from the first feature set to obtain a second feature set, where M is an integer greater than or equal to 1.

[0052] In operation S230, a reconstructed model is obtained, wherein the reconstructed model includes a model with second parameter information obtained after an initial model is trained based on a training set using a second feature set and a first machine learning algorithm.

[0053] According to an embodiment of the present disclosure, the first parameter information and the second parameter information are, for example, parameters of a mathematical function in a machine learning model, and the process of training the machine learning model is the process of determining the above parameter information. The first parameter information and the second parameter information can process the input data to obtain an output result when the model makes a prediction.

[0054] According to an embodiment of the present disclosure, for example, when the model to be evaluated is a classification model, the reconstructed model is a classification model accordingly. When the model to be evaluated is a regression model, the reconstructed model is a regression model accordingly. The difference between the reconstructed model and the model to be evaluated is that due to the different features used in the training process, the parameter information obtained after the final training is completed may also be different. Therefore, the prediction performance of the two can be compared on this basis.

[0055] In operation S240, when the ratio of the second index of the reconstructed model to the first index is greater than or equal to a first preset threshold, the M features are determined to be redundant features, wherein the second index is used to characterize the prediction performance of the reconstructed model for the test set.

[0056] According to an embodiment of the present disclosure, for example, when the model to be evaluated is a binary classification model, it can be set that the requirement is met when the ratio of AUC (reconstructed) to AUC (to be evaluated) is greater than or equal to 0.99 (i.e., the first preset threshold), that is, the predictive performance of the reconstructed model declines by less than 1% compared with the model to be evaluated (for illustration only).

[0057] According to an embodiment of the present disclosure, the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain a reconstructed model with second parameter information, and then the prediction performance of the model to be evaluated is compared with the reconstructed model to determine whether the features removed from the first feature set are redundant features. Therefore, it is possible to check whether the model introduces unnecessary features, check whether the model has potential overfitting risks, and avoid performance degradation caused by overfitting after the model is deployed online. Since the processing of redundant features will call for additional computing power, resulting in ineffective use of computing power, while detecting redundant features, it is also possible to check whether the computing power utilization of the model is effective.

[0058] Figure 3A flowchart of a redundant feature detection method for a model according to another embodiment of the present disclosure is schematically shown.

[0059] like Figure 3 As shown, the redundant feature detection method of the model according to the embodiment of the present disclosure includes repeatedly performing the operations of elimination (e.g., operation S220), obtaining a reconstructed model (e.g., operation S230), and determining redundant features (e.g., operation S240) until all features in the first feature set are detected. In addition to operations S210, S230, and S240, the method may also include operations S310 to S330. Among them, operation S310 is one embodiment of operation S220.

[0060] In operation S210, a first feature set and a first indicator of the model to be evaluated are obtained, wherein the model to be evaluated includes a model with first parameter information obtained after an initial model is trained based on a training set using the first feature set and a first machine learning algorithm, and the first indicator is used to characterize the prediction performance of the model to be evaluated on a test set.

[0061] In operation S310 , features are eliminated one by one from the first feature set.

[0062] According to an embodiment of the present disclosure, a set may be randomly removed from the first feature set to obtain the second feature set, or a set may be sequentially removed from the first feature set to obtain the second feature set.

[0063] In operation S230, a reconstructed model is obtained, wherein the reconstructed model includes a model with second parameter information obtained after an initial model is trained based on a training set using a second feature set and a first machine learning algorithm.

[0064] In operation S320, it is determined whether the ratio of the second index to the first index of the reconstructed model is greater than or equal to a first preset threshold. If it is greater than the first preset threshold, operation S240 is performed. If it is less than the first preset threshold, operation S330 is performed.

[0065] In operation S240, when the ratio of the second index of the reconstructed model to the first index is greater than or equal to the first preset threshold, the M features are determined to be redundant features, wherein the second index is used to characterize the prediction performance of the reconstructed model for the test set.

[0066] In operation S330, when a feature is determined to be a non-redundant feature, the feature is put back in the next detection process and the operations of eliminating (eg, operation S310), obtaining a reconstructed model (eg, operation S230), and determining redundant features (eg, operation S240) are repeated.

[0067] According to an embodiment of the present disclosure, for example, the first feature set of the risk management model for the banking credit business may be [financial data, industrial and commercial data, credit data, capital transaction data, public opinion data]. First, the financial data is removed, and the second feature set may be [industrial and commercial data, credit data, capital transaction data, public opinion data]. Then, the initial model is trained using the second feature set to obtain a reconstructed model. If the performance of the reconstructed model is significantly lower than that of the model to be evaluated (for example, most risky users are identified as normal users), then the financial data is a non-redundant feature. Next, the financial data is put back, and the remaining undetected features are continued to be removed from [financial data, industrial and commercial data, credit data, capital transaction data, public opinion data].

[0068] Figure 4 The flowchart of eliminating features one by one from the first feature set in operation S310 according to an embodiment of the present disclosure is schematically shown.

[0069] like Figure 4 As shown, the initial value of i can be assigned to any value in [1, N], and then operations S410 to S440 are executed repeatedly until the cycle is N times, where N is the number of features in the first feature set, and i is an integer greater than or equal to 1.

[0070] In operation S410, the i-th feature is removed from the first feature set to obtain a corresponding second feature set.

[0071] According to an embodiment of the present disclosure, for example, financial data is removed from [financial data, industrial and commercial data, credit data, fund transaction data, public opinion data]. In some embodiments of the present disclosure, the initial value of i can also be assigned to a random value in [1, N], for example, i=3, then the credit data is removed first.

[0072] In operation S420, it is determined whether the i-th feature is a redundant feature. When the i-th feature is not a redundant feature, operation S430 is performed. When the i-th feature is a redundant feature, operation S440 is performed.

[0073] In operation S430, when it is determined that the i-th feature is a non-redundant feature, the i-th feature is put back and then i=i+1.

[0074] According to an embodiment of the present disclosure, when financial data is a non-redundant feature, the industrial and commercial data is removed from [financial data, industrial and commercial data, credit data, capital transaction data, and public opinion data].

[0075] In operation S440 , when it is determined that the i-th feature is a redundant feature, the i-th feature is eliminated, and then i=i+1 is set.

[0076] According to an embodiment of the present disclosure, when financial data is a redundant feature, the industrial and commercial data is further eliminated from [industrial and commercial data, credit data, capital transaction data, and public opinion data].

[0077] Furthermore, when the industrial and commercial data is a redundant feature, the credit data can be further eliminated from [credit data, capital transaction data, public opinion data].

[0078] In some embodiments of the present disclosure, i=i+s may be set, where s is a random number, that is, any one of the remaining features may be randomly eliminated.

[0079] By using the redundant feature detection method of the embodiment of the present disclosure, features can be detected one by one, and the contribution of each feature to the model to be evaluated can be clarified. When the prediction performance of the reconstructed model obtained after removing a feature does not decline, it means that the feature is a redundant feature and does not contribute to the prediction performance of the model to be evaluated, thereby preventing the subsequent waste of computing power to process it.

[0080] According to an embodiment of the present disclosure, after removing the i-th feature from the first feature set and obtaining the corresponding second feature set, obtaining the reconstructed model includes: configuring a training process based on the first hyperparameter information of the model to be evaluated, using the second feature set and the first machine learning algorithm to train the initial model based on the training set, and obtaining the reconstructed model.

[0081] According to the embodiments of the present disclosure, the reconstructed model and the model to be evaluated are trained using the same hyperparameter information, so that the eliminated features serve as the difference between the two. By comparing the prediction performance of the two, the role of each feature can be clarified, which is beneficial for providing a reference in the subsequent training of models with similar functions.

[0082] Figure 5 The flowchart of obtaining the second feature set in operation S220 according to an embodiment of the present disclosure is schematically shown.

[0083] like Figure 5 As shown, removing M features from the first feature set to obtain the second feature set in operation S220 may include operations S510 to S530.

[0084] In operation S510, a proxy model is obtained, wherein the proxy model includes a model with third parameter information obtained after an initial model is trained based on a training set using a first feature set and a second machine learning algorithm.

[0085] According to an embodiment of the present disclosure, the second machine learning algorithm may be a strong machine learning algorithm such as a random forest algorithm or a gradient boosting tree algorithm that can obtain feature importance. In some embodiments of the present disclosure, the first machine learning algorithm may also be a strong machine learning algorithm such as a random forest algorithm or a gradient boosting tree algorithm that can obtain feature importance, in which case the second machine learning algorithm may be the same as the first machine learning algorithm.

[0086] In operation S520, when the ratio of the third indicator of the proxy model to the first indicator is greater than or equal to the first preset threshold, the importance of each feature in the first feature set is obtained based on the second machine learning algorithm, wherein the third indicator is used to characterize the prediction performance of the proxy model for the test set.

[0087] According to an embodiment of the present disclosure, the proxy model is a proxy of the model to be evaluated, comes from the same training set, and has the same input and output. Therefore, in the process of training the proxy model on the same training set, the third index of the proxy model is made close to the first index of the model to be evaluated (that is, the ratio of the third index to the first index is greater than or equal to the first preset threshold). When a proxy model that meets the requirements is obtained, the importance of each feature can be obtained.

[0088] In operation S530, based on the importance of each feature, M features whose importance is less than or equal to a second preset threshold are removed from the first feature set to obtain a second feature set.

[0089] According to an embodiment of the present disclosure, by checking the importance of each feature in the proxy model, for example, the second preset threshold can be set to 0 (just as an example), that is, features with an importance of 0 are removed from the first feature set, and then the second feature set is obtained.

[0090] By using the redundant feature detection method of the embodiment of the present disclosure, since the prediction performance of the proxy model is similar to that of the model to be evaluated, the importance of each feature can reflect the contribution to the model to be evaluated. Therefore, at least one feature with a smaller importance is eliminated, which can improve the efficiency of feature elimination.

[0091] Figure 6 The flowchart of obtaining the reconstructed model in operation S230 according to an embodiment of the present disclosure is schematically shown.

[0092] like Figure 6 As shown, obtaining the reconstructed model in operation S230 may include operations S610 to S620.

[0093] In operation S610, second hyperparameter information is obtained, where the second hyperparameter information is different from the first hyperparameter information of the model to be evaluated.

[0094] In operation S620, a training process is configured based on the second hyperparameter information, and the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain a reconstructed model.

[0095] According to the embodiments of the present disclosure, after obtaining a proxy model that meets the requirements, multiple features with less importance may be removed from the first feature set. Then, in the process of obtaining the reconstruction model using the second feature set, using the first hyperparameter information to train the initial model will lead to reduced training efficiency, and even the obtained reconstruction model may not meet the requirements, and erroneous redundant feature detection results may occur. Therefore, obtaining the second hyperparameter information to adjust the learning ability of the reconstruction model can improve the training efficiency.

[0096] Figure 7 The flowchart of performing redundancy evaluation on a model to be evaluated according to an embodiment of the present disclosure is schematically shown.

[0097] like Figure 7 As shown, performing redundancy evaluation on the model to be evaluated may include operations S710 to S720.

[0098] In operation S710, when any feature is removed from the second feature set and the ratio of the second index of the reconstructed model to the first index is less than a first preset threshold, the number of redundant features of the first feature set is determined to be M.

[0099] According to the embodiments of the present disclosure, for example, from [financial data, industrial and commercial data, credit data, capital transaction data, public opinion data], industrial and commercial data and public opinion data are determined to be redundant data. After removing any feature from [financial data, credit data, capital transaction data], the prediction performance of the obtained reconstructed model is seriously degraded compared to the model to be evaluated, then it can be determined that the number of redundant features is 2.

[0100] In operation S720 , redundancy evaluation is performed on the model to be evaluated based on the ratio of M to N.

[0101] According to an embodiment of the present disclosure, the redundancy evaluation of the model to be evaluated can be calculated using the following formula:

[0102]

[0103] Among them, Redundancy represents redundancy, M is the number of redundant features, and N is the number of features in the first feature set.

[0104] When the redundant feature is 2 and the number of features in the first feature set is 5, the redundancy of the risk management model is 0.4.

[0105] According to another embodiment of the present disclosure, taking the banking credit business risk management model as an example, it is possible to predict whether a target customer has a default risk, which can be a binary classification model. For example, when the model is training the model to be evaluated on the training set, the data features used include 45 features such as financial data, industrial and commercial data, credit data, capital transaction data, and public opinion data (only as an example). Then, the model is predicted on the test set, and the AUC value is calculated based on the prediction results. For example, the AUC value of the first indicator is 0.89.

[0106] First, use the feature reduction method, that is, remove features one by one on the first feature set. For each feature, after removing the feature, calculate the change of the second indicator of the reconstructed model on the test set. If the change is the smallest, it is determined to be a redundant feature. Repeat the above process until the AUC value of the second indicator drops to 0.8811 after removing any feature, and the decline is close to 1% (just an example). In this process, four features such as current asset turnover rate and total number of debts are removed respectively, and the redundancy of the calculated model is about 0.09.

[0107] Then, the proxy model method is used, that is, the proxy model is obtained using the second machine learning algorithm, and the importance of each feature is obtained. For example, the gradient boosting tree algorithm is used to model based on these 45 features, and the hyperparameters are adjusted to make the AUC value of the model result reach 0.89. Then check the feature importance of the model and find that the feature importance of the two features of current asset turnover rate and total number of debts is 0. Then remove these two features and use the remaining features to retrain the model (that is, obtain a reconstructed model). If the AUC value obtained can reach 0.8811 or above, it can be ensured that the difference between the reconstructed model after removing the features and the model to be evaluated does not exceed 1%, then these two features are considered to be redundant features, and the model redundancy is calculated to be 0.04.

[0108] Finally, it can be seen from the above that four features can be eliminated through the feature reduction method, and two features can be eliminated through the proxy model method. Therefore, the reconstructed model corresponding to the feature reduction method can finally replace the model to be evaluated.

[0109] In some embodiments of the present disclosure, after the redundant features are determined using the proxy model method, the feature reduction method may be further used to further detect the remaining features one by one.

[0110] By utilizing the redundant feature detection method of the disclosed embodiment, it is possible to evaluate the model based on its redundancy, provide a decision-making basis for the operational management of the model, and avoid the risk of model attenuation and performance waste caused by feature redundancy.

[0111] Figure 8 A block diagram of a redundant feature detection device 800 for a model according to an embodiment of the present disclosure is schematically shown.

[0112] like Figure 8 As shown, the redundant feature detection device 800 may include an acquisition module 810 , a removal module 820 , a reconstruction module 830 and a determination module 840 .

[0113] The acquisition module 810 can, for example, perform operation S210 to obtain a first feature set and a first indicator of the model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after training an initial model based on a training set using the first feature set and a first machine learning algorithm, and the first indicator is used to characterize the prediction performance of the model to be evaluated on a test set.

[0114] The elimination module 820 may, for example, perform operation S220 to eliminate M features from the first feature set to obtain a second feature set, where M is an integer greater than or equal to 1.

[0115] The reconstruction module 830 can, for example, perform operation S230 to obtain a reconstructed model, wherein the reconstructed model includes a model with second parameter information obtained after training the initial model based on the training set using the second feature set and the first machine learning algorithm.

[0116] The determination module 840 can, for example, perform operation S240, which is used to determine that M features are redundant features when the ratio of the second indicator of the reconstructed model to the first indicator is greater than or equal to a first preset threshold, wherein the second indicator is used to characterize the prediction performance of the reconstructed model for the test set.

[0117] According to an embodiment of the present disclosure, the elimination module 820 can also perform operations S310 and S320, for example, to eliminate features one by one from the first feature set. When a feature is determined to be a non-redundant feature, the feature is put back in the next detection process and the operations of eliminating, obtaining a reconstructed model, and determining redundant features are repeated.

[0118] According to an embodiment of the present disclosure, the elimination module 820 may also assign the initial value of i to any value in [1, N], and loop through operations S410 to S440 until the loop is N times, where N is the number of features in the first feature set, and i is an integer greater than or equal to 1. Specifically, the i-th feature is eliminated from the first feature set to obtain the corresponding second feature set. When the i-th feature is determined to be a redundant feature, the i-th feature is eliminated, and then i=i+1 is set. When the i-th feature is determined to be a non-redundant feature, the i-th feature is replaced, and then i=i+1 is set.

[0119] According to an embodiment of the present disclosure, the elimination module 820 may also perform operations S510 to S230, for example, to obtain a proxy model, wherein the proxy model includes a model with third parameter information obtained after the initial model is trained based on the training set using the first feature set and the second machine learning algorithm. When the ratio of the third index of the proxy model to the first index is greater than or equal to the first preset threshold, the importance of each feature in the first feature set is obtained based on the second machine learning algorithm, wherein the third index is used to characterize the predictive performance of the proxy model for the test set. Based on the importance of each feature, M features whose importance is less than or equal to the second preset threshold are eliminated from the first feature set to obtain a second feature set.

[0120] According to an embodiment of the present disclosure, the reconstruction module 830 can also be used to eliminate the i-th feature from the first feature set, and after obtaining the corresponding second feature set, configure the training process based on the first hyperparameter information of the model to be evaluated, use the second feature set and the first machine learning algorithm to train the initial model based on the training set, and obtain the reconstructed model.

[0121] According to an embodiment of the present disclosure, the reconstruction module 830 may also perform operations S610 to S620, for example, to obtain second hyperparameter information, wherein the second hyperparameter information is different from the first hyperparameter information of the model to be evaluated. A training process is configured based on the second hyperparameter information, and the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain a reconstructed model.

[0122] According to an embodiment of the present disclosure, the redundant feature detection device 800 may further include a redundancy evaluation module. The redundancy evaluation module may, for example, perform operations S710 to S720, and is used to determine that the number of redundant features of the first feature set is M when the ratio of the second index to the first index of the reconstructed model obtained by removing any feature from the second feature set is less than the first preset threshold value. And perform redundancy evaluation on the model to be evaluated based on the ratio of M to N.

[0123] Combine the following Fig. 9 , and describes in detail the detailed process of using the redundant feature detection device 800 to perform redundant feature detection.

[0124] Fig. 9 The flowchart schematically shows an operation of performing redundant feature detection using the redundant feature detection device 800 according to an embodiment of the present disclosure.

[0125] like Fig. 9 As shown, performing redundant feature detection using the redundant feature detection device 800 may include operations S910 to S980.

[0126] In operation S910, a feature set of a model to be evaluated (e.g., a first feature set), a training set of a model to be evaluated, an index of a model to be evaluated (e.g., a first index), and a test set of a model to be evaluated may be obtained through the redundant feature detection device 800. Then, a feature reduction method and a proxy model method may be selected to perform redundant feature detection respectively, wherein the feature reduction method is selected to perform operation S930, and the proxy model method is selected to perform operation S920.

[0127] In operation S920, it may be determined whether the first machine learning algorithm has feature importance. If so, operation S922 is performed; if not, operation S921 is performed.

[0128] In operation S921, if the first machine learning algorithm cannot obtain the feature importance, a proxy model may be obtained using algorithms such as random forest and gradient boosting tree, for details, refer to operation S510.

[0129] In operation S922, if the first machine learning algorithm can obtain the feature importance, then the importance of each feature in the first feature set is directly obtained. Otherwise, each feature importance is obtained from the proxy model, and the details can be referred to operation S520.

[0130] In operation S923, features whose importance is greater than or equal to a preset threshold, for example, M features greater than 0, are obtained. For details, refer to operation S530.

[0131] In operation S930, the feature reduction method is used to eliminate features one by one from the first feature set. For details, please refer to operation S310.

[0132] In operation S940, a second feature set is obtained.

[0133] In operation S950, a reconstructed model is obtained based on the second feature set, and details may refer to operation S230.

[0134] In operation S960, it is determined whether the prediction performance of the reconstructed model relative to the model to be evaluated decreases. For example, it is possible to compare whether the ratio of the second index to the first index of the reconstructed model is greater than or equal to a first preset threshold.

[0135] For the reconstructed model obtained by the feature reduction method, when the prediction performance is significantly reduced, a feature is determined to be a non-redundant feature, and the feature is put back in the next detection process and operations S930, S940, S950, and S960 are repeatedly performed. When the prediction performance is not significantly reduced, operation S970 is performed.

[0136] For the reconstructed model obtained by using the proxy model method, when the prediction performance has not been significantly reduced, operation S970 is performed.

[0137] In operation S970, if a redundant feature is determined according to the feature reduction method, operation S930 (eliminating undetected features), operation S940, operation S950, and operation S960 may be repeatedly performed in the next detection process until all features in the first feature set are detected. If M redundant features are determined according to the proxy model method, the number of redundant features may be directly determined.

[0138] In operation S980, based on the result of operation S970, the number of redundant features is determined, and redundancy evaluation of the model to be evaluated is performed. For details, refer to operations S710 to S720.

[0139] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0140] For example, any multiple of the acquisition module 810, the elimination module 820, the reconstruction module 830 and the determination module 840 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 810, the elimination module 820, the reconstruction module 830 and the determination module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or encapsulating the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 810, the elimination module 820, the reconstruction module 830 and the determination module 840 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0141] Fig.10 A block diagram of a computer system suitable for implementing a redundant feature detection method and apparatus according to an embodiment of the present disclosure is schematically shown. Fig.10 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0142] like Fig.10 As shown, the computer system 1000 according to the embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0143] In RAM 1003, various programs and data required for the operation of system 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other through bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0144] According to an embodiment of the present disclosure, the system 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The system 1000 may further include one or more of the following components connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0145] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0146] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the redundant feature detection method of the model provided by the embodiment of the present disclosure.

[0147] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0148] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0149] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0150] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0151] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A redundant feature detection method for a model, comprising: Obtaining a first feature set and a first index of a model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after an initial model is trained based on a training set using the first feature set and a first machine learning algorithm, and the first index is used to characterize the prediction performance of the model to be evaluated on a test set; Eliminate M features from the first feature set to obtain a second feature set, where M is an integer greater than or equal to 1; Obtaining a reconstructed model, wherein the reconstructed model includes a model with second parameter information obtained after the initial model is trained based on the training set using the second feature set and the first machine learning algorithm; When the ratio of the second index of the reconstructed model to the first index is greater than or equal to a first preset threshold, determining that the M features are redundant features, wherein the second index is used to characterize the prediction performance of the reconstructed model for the test set; The removing M features from the first feature set to obtain a second feature set comprises: Obtaining a proxy model, wherein the proxy model includes a model with third parameter information obtained after the initial model is trained based on the training set using the first feature set and the second machine learning algorithm; When the ratio of the third indicator of the proxy model to the first indicator is greater than or equal to the first preset threshold, obtaining the importance of each feature in the first feature set based on the second machine learning algorithm, wherein the third indicator is used to characterize the prediction performance of the proxy model for the test set; Based on the importance of each feature, M features whose importance is less than or equal to a second preset threshold are removed from the first feature set to obtain the second feature set.

2. The redundant feature detection method according to claim 1, wherein: The method comprises repeatedly performing the operations of eliminating, obtaining a reconstructed model, and determining redundant features until all features in the first feature set are detected; Then, the step of removing M features from the first feature set to obtain a second feature set includes: removing features one by one from the first feature set; When a feature is determined to be a non-redundant feature, the feature is put back in the next detection process and the operations of eliminating, obtaining a reconstructed model, and determining redundant features are repeated.

3. The redundant feature detection method according to claim 2, wherein: Eliminating features one by one from the first feature set includes: Assign the initial value of i to any value in [1, N], and repeat the following operations until the loop is N times, where N is the number of features in the first feature set, and i is an integer greater than or equal to 1: Eliminate the i-th feature from the first feature set to obtain the corresponding second feature set; When it is determined that the i-th feature is a redundant feature, the i-th feature is removed, and then i=i+1; When it is determined that the i-th feature is a non-redundant feature, the i-th feature is put back and then i=i+1.

4. The redundant feature detection method according to claim 3, wherein: After removing the i-th feature from the first feature set to obtain the corresponding second feature set, obtaining the reconstructed model includes: A training process is configured based on the first hyperparameter information of the model to be evaluated, and the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain the reconstructed model.

5. The redundant feature detection method according to claim 1, wherein: The obtaining of the reconstruction model comprises: Obtaining second hyperparameter information, wherein the second hyperparameter information is different from the first hyperparameter information of the model to be evaluated; A training process is configured based on the second hyperparameter information, and the initial model is trained based on the training set using the second feature set and the first machine learning algorithm to obtain the reconstructed model.

6. The redundant feature detection method according to claim 1, wherein: The first feature set includes N features, where N is greater than M. After determining that the M features are redundant features, the method further includes: When any feature is removed from the second feature set and the ratio of the second index of the reconstructed model to the first index is less than the first preset threshold, determining the number of redundant features of the first feature set to be M; and The redundancy of the model to be evaluated is evaluated based on the ratio of M to N.

7. A redundant feature detection device for a model, comprising: An obtaining module, used to obtain a first feature set and a first index of a model to be evaluated, wherein the model to be evaluated includes a model with first parameter information obtained after an initial model is trained based on a training set using the first feature set and a first machine learning algorithm, and the first index is used to characterize the prediction performance of the model to be evaluated on a test set; A removal module, used to remove M features from the first feature set to obtain a second feature set, where M is an integer greater than or equal to 1; A reconstruction module, used to obtain a reconstructed model, wherein the reconstructed model includes a model with second parameter information obtained after the initial model is trained based on the training set using the second feature set and the first machine learning algorithm; A determination module, configured to determine that the M features are redundant features when a ratio of a second indicator of the reconstructed model to the first indicator is greater than or equal to a first preset threshold, wherein the second indicator is used to characterize the prediction performance of the reconstructed model for the test set; The removing M features from the first feature set to obtain a second feature set comprises: Obtaining a proxy model, wherein the proxy model includes a model with third parameter information obtained after the initial model is trained based on the training set using the first feature set and the second machine learning algorithm; When the ratio of the third indicator of the proxy model to the first indicator is greater than or equal to the first preset threshold, obtaining the importance of each feature in the first feature set based on the second machine learning algorithm, wherein the third indicator is used to characterize the prediction performance of the proxy model for the test set; Based on the importance of each feature, M features whose importance is less than or equal to a second preset threshold are removed from the first feature set to obtain the second feature set.

8. An electronic device, comprising: One or more memories storing executable instructions; as well as One or more processors execute the executable instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images