Feature screening method and related device

Abstract

The feature screening method comprises the steps that a training data set is acquired, the training data set comprises a plurality of training data subsets, each training data subset comprises a plurality of training samples, and each training sample comprises feature values of a plurality of features in a to-be-screened feature set; training a target network based on the training data subset to obtain a trained target network; based on feature interaction contribution degree information, screening from the to-be-screened feature set to obtain a screened feature set, the feature interaction contribution degree information representing contribution degrees of each at least two features in the to-be-screened feature set to a trained target network; based on the screened feature set, a target feature set is obtained, and the number of features in the target feature set is the target number. According to the feature screening method for explicitly considering the feature interaction influence, by considering the contribution degree of multiple feature interactions, the accuracy of the feature screening result is higher.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence (AI), in particular to a feature screening method and related device. BACKGROUND

[0002] In the field of artificial intelligence, the processing object of a neural network generally has a large number of features, which cannot all be used as input features of the neural network. Therefore, a feature screening technical solution is needed.

[0003] For example, a recommendation system can record thousands of user features and item features. It is impractical to apply all these features to the input features of the recommendation model. The reason is that more features mean more computing resources are needed, and online latency will also increase accordingly, resulting in an increase in technical implementation costs. In addition, noise features and redundant features can cause the training results of the recommendation model to deteriorate. SUMMARY

[0004] The present application provides a feature screening method and related device, which considers the contribution degree of multiple feature interactions to perform feature screening work and obtain a target feature set with a target number of features.

[0005] In a first aspect, the present application provides a feature screening method, including: obtaining a training data set, the training data set including a plurality of training data subsets, each training data subset including a plurality of training samples, and each training sample including feature values of a plurality of features in a to-be-screened feature set; training a target network based on the training data subsets to obtain a trained target network; screening a screened feature set from the to-be-screened feature set based on feature interaction contribution degree information, the feature interaction contribution degree information indicating a contribution degree of each at least two features in the to-be-screened feature set to the trained target network; and obtaining a target feature set based on the screened feature set, the number of features in the target feature set being a target number.

[0006] In the technical solution of this application, feature importance is generally related to other features. That is, removing a feature changes the importance of other closely interacting features. Direct feature deletion would lead to inaccurate feature selection, ultimately resulting in poor predictions from the neural network. Taking "longitude" and "latitude" features as examples, retaining the "longitude" feature and deleting the "latitude" feature changes the importance of the "longitude" feature, leading to inaccurate feature selection. In this embodiment, feature deletion and retention are performed using feature interaction contribution information to achieve feature selection. Feature interaction contribution information represents the contribution of at least two features in the feature set to be selected to the trained target network. Feature selection considers the contribution of at least two features to the target network, ensuring that the interaction contribution of at least two features among the retained features is large, thus guaranteeing the accuracy of the feature selection results. Specifically, taking "longitude" and "latitude" features as examples, the "longitude" and "latitude" features can jointly determine the user's location, while a single "longitude" or "latitude" feature cannot determine the user's location. When determining the user's location, the interaction between the "longitude" and "latitude" features contributes significantly, while the contribution of a single "longitude" or "latitude" feature is relatively small. Therefore, considering the contribution information of feature interaction leads to higher accuracy in feature selection. Thus, the technical solution of this application considers feature interaction contribution information during the feature selection process, making it an explicit feature selection method that considers the impact of feature interaction. Furthermore, by considering feature interaction contribution information during feature deletion, it avoids feature deletion based on local optima principles and instead uses global optima principles after considering the impact of feature interaction, thereby improving the precision and accuracy of the selection process.

[0007] In one possible implementation, the target network includes a main network, the feature set to be filtered includes an initial feature set to be filtered, the filtered feature set includes a first retained feature set and a first deleted feature set, the training dataset includes a first training data subset, the first retained feature set includes several retained features, and the first deleted feature set includes several deleted features. Based on the training data subset, the target network is trained to obtain a trained target network, including: based on the first training data subset, the main network is trained to obtain a trained main network. Based on feature interaction contribution information, a target feature set is filtered from the feature set to be filtered to obtain a target feature set, including: based on first feature interaction information, a first retained feature set and a first deleted feature set are filtered from the initial feature set to be filtered, where the first feature interaction information represents the contribution of at least two features in the initial feature set to the trained main network.

[0008] This implementation provides a method for determining a first set of retained features and a first set of deleted features. The first set of retained features includes a first set of deleted features, and the first set of deleted features includes a first set of deleted features. For example, a main network can be trained, and based on the trained main network and the first feature interaction information, several features to be selected in the main network are ranked by feature importance to obtain the first set of retained features and the first set of deleted features.

[0009] In one possible implementation, the target network further includes a focusing network and a revival network. The feature set to be filtered further includes a first retained feature set and a first deleted feature set. The filtered feature set includes a second retained feature set and a second deleted feature set. The training dataset further includes a second training data subset. The focusing network is obtained by performing a masking operation on the trained main network. The masking operation is used to mask the parameters corresponding to the features in the first deleted feature set. The revival network is obtained by performing a parameter freezing operation on the trained main network. The parameter freezing operation is used to freeze the parameters of the trained focusing network. The above-mentioned method of training the target network based on a subset of training data to obtain a trained target network further includes: training the focusing network based on the second subset of training data to obtain a trained focusing network; and training the revival network based on the second subset of training data to obtain a trained revival network. The aforementioned process of selecting a filtered feature set from the set of features to be filtered based on feature interaction contribution information further includes: selecting several features from the first retained feature set based on second feature interaction information to obtain features to be deleted, where the second feature interaction information represents the contribution of at least two features in the first retained feature set to the trained focusing network; selecting several features from the first deleted feature set based on third feature interaction information to obtain features to be revived, where the third feature interaction information represents the contribution of at least two features in the first deleted feature set to the trained revival network; moving the features to be revived from the first deleted feature set to the first retained feature set to obtain a second retained feature set; and moving the features to be deleted from the first retained feature set to the first deleted feature set to obtain a second deleted feature set.

[0010] This implementation provides a method for further feature selection based on a first set of retained features and a first set of deleted features. First, features are deleted from the first set of retained features. Specifically, using a trained focusing network, the contribution of the first set of retained features to the trained focusing network (i.e., the second feature interaction information) can be recalculated to obtain a feature importance ranking. Based on the new ranking, several features are identified as features to be deleted. Second, features are revived from the first set of deleted features. Specifically, this implementation can also revive a portion of the deleted features as features to be revived. The parameters of the trained focusing network are frozen to obtain the revival network; the revival network is trained to adjust its parameters. Freezing the focusing network ensures the correlation between the parameters of the revival network and the parameters of the focusing network, thus ensuring the correlation between the first set of deleted features and the first set of retained features. The features to be revived obtained from the first deleted feature set are equivalent to retaining the more important features from the first deleted feature set and those with a strong correlation to the first retained features, ensuring that the second retained feature set contains the more important features for the current iteration. Based on the first retained feature set after feature deletion and several features to be revived, the second retained feature set is obtained, which is equivalent to comprehensively considering the more important features from both the first retained and deleted feature sets. Freezing the focusing network and adjusting the parameters of the revival network can be understood as training the revival network to perform better than a copy of the focusing network, without affecting the updating of the focusing network.

[0011] In one possible implementation, before moving the feature to be deleted from the first set of retained features to the first set of deleted features to obtain the second set of deleted features, the method further includes deleting all features in the first set of deleted features.

[0012] In this implementation, each deleted feature is revived only once. This reduces the number of features in the second set of deleted features, facilitating computation and preventing the number of deleted features from accumulating, thus reducing computational load. In the next iteration, the main network effectively becomes a new main network with fewer input features.

[0013] In one possible implementation, based on feature interaction contribution information, a filtered feature set is obtained from the set of features to be filtered, including: with the goal of maximizing feature interaction sub-information, selecting a number of features to be retained and a number of features to be deleted from the set of features to be filtered, to obtain the filtered feature set. The feature interaction sub-information represents the total contribution of at least two features in the set of features to be filtered after deleting a number of features to the trained target network.

[0014] In this implementation, a criterion for feature selection is provided: the objective is to maximize the feature interaction sub-information; the feature interaction sub-information represents the total contribution of at least two features in the set of features to be selected after deleting several features to the trained target network.

[0015] In one possible implementation, the feature interaction contribution information is obtained based on the multi-order gradients of at least two features in the set of features to be screened, using the loss function of the trained target network.

[0016] This implementation provides a method for calculating feature interaction contribution information: based on the multi-order gradients of at least two features in the feature set to be selected using the loss function of the trained target network, the feature interaction contribution information is obtained. This information represents the contribution of at least two features to the trained target network.

[0017] In one possible implementation, the feature interaction contribution information is obtained based on the second-order gradient of every two features in the feature set to be screened, using the loss function of the trained target network. The feature interaction contribution information represents the contribution matrix of every two features in the feature set to be screened to the trained target network. The above-described process of selecting a filtered feature set from the feature set to be screened based on the feature interaction contribution information includes: selecting K features from several features to be screened, with the objective of maximizing the sum of the contribution matrices after deleting the rows and columns corresponding to K features, to obtain the filtered feature set, where K is a positive integer.

[0018] This implementation provides an example of feature interaction contribution information based on multi-order gradients: second-order gradients. After deleting rows and columns corresponding to K features, the K features are selected from several unselected features with the goal of maximizing the sum of the corresponding contribution matrices, resulting in a set of selected features.

[0019] In one possible implementation, obtaining the target feature set based on the filtered feature set includes: determining whether the number of features in the filtered feature set is the target number; if the number of features in the filtered feature set is the target number, then the filtered feature set is determined as the target feature set; if the number of features in the filtered feature set is greater than the target number, then feature filtering operations are performed on several features in the filtered feature set to obtain the target feature set.

[0020] This implementation provides a method for obtaining a target feature set based on a filtered feature set. If the number of features in the filtered feature set is the target number, then the filtered feature set is determined as the target feature set. If the number of features in the filtered feature set is greater than the target number, then feature filtering operations are performed on several features in the filtered feature set until the target feature set is obtained.

[0021] Secondly, this application provides a feature selection apparatus, comprising: an acquisition module for acquiring a training dataset, each training dataset including several training samples, each training sample including feature values ​​of several features in a feature set to be selected; a training module for training a target network based on a subset of training data to obtain a trained target network; a feature selection module for selecting a selected feature set from the feature set to be selected based on feature interaction contribution information, wherein the feature interaction contribution information represents the contribution of at least two features in the feature set to be selected to the trained target network; and, based on the selected feature set, obtaining a target feature set, wherein the number of features in the target feature set is a target number.

[0022] In one possible implementation, the target network includes a main network, the feature set to be filtered includes an initial feature set to be filtered, the filtered feature set includes a first retained feature set and a first deleted feature set, the training dataset includes a first training data subset, the first retained feature set includes several retained features, and the first deleted feature set includes several deleted features. The training module is specifically used to train the main network based on the first training data subset to obtain the trained main network. The feature filtering module is specifically used to filter the initial feature set to be filtered to obtain the first retained feature set and the first deleted feature set based on first feature interaction information, where the first feature interaction information represents the contribution of at least two features in the initial feature set to the trained main network.

[0023] In one possible implementation, the target network further includes a focusing network and a revival network. The feature set to be filtered further includes a first retained feature set and a first deleted feature set. The filtered feature set includes a second retained feature set and a second deleted feature set. The training dataset further includes a second training data subset. The focusing network is obtained by performing a masking operation on the trained main network. The masking operation is used to mask the parameters corresponding to the features in the first deleted feature set. The revival network is obtained by performing a parameter freezing operation on the trained main network. The parameter freezing operation is used to freeze the parameters of the trained focusing network. Specifically, the training module is used to train the focusing network based on the second training data subset to obtain the trained focusing network; and to train the revival network based on the second training data subset to obtain the trained revival network. The aforementioned feature selection module is specifically used to: select several features from the first retained feature set based on the second feature interaction information to obtain features to be deleted, wherein the second feature interaction information represents the contribution of each at least two features in the first retained feature set to the trained focusing network; and select several features from the first deleted feature set based on the third feature interaction information to obtain features to be revived, wherein the third feature interaction information represents the contribution of each at least two features in the first deleted feature set to the trained revival network; move the features to be revived from the first deleted feature set to the first retained feature set to obtain the second retained feature set; and move the features to be deleted from the first retained feature set to the first deleted feature set to obtain the second deleted feature set.

[0024] In one possible implementation, the aforementioned feature filtering module is also used to delete all features in the first set of deleted features.

[0025] In one possible implementation, the feature selection module is specifically used to select a number of features to be retained and a number of features to be deleted from the feature set to be selected with the goal of maximizing the feature interaction sub-information, so as to obtain a selected feature set. The feature interaction sub-information represents the total contribution of each at least two features in the feature set to be selected after deleting the deleted features to the trained target network.

[0026] In one possible implementation, the feature interaction contribution information is obtained based on the multi-order gradients of at least two features in the set of features to be screened, using the loss function of the trained target network.

[0027] In one possible implementation, the feature interaction contribution information is obtained based on the second-order gradient of every two features in the feature set to be screened, using the loss function of the trained target network. The feature interaction contribution information represents the contribution matrix of every two features in the feature set to be screened to the trained target network. Specifically, the feature screening module aims to maximize the sum of the contribution matrices after deleting the rows and columns corresponding to K features, thus selecting K features from several features to be screened, resulting in a screened feature set where K is a positive integer.

[0028] In one possible implementation, the aforementioned feature filtering module is specifically used to determine whether the number of features in the filtered feature set is the target number; if the number of features in the filtered feature set is the target number, then the filtered feature set is determined as the target feature set; if the number of features in the filtered feature set is greater than the target number, then feature filtering operations are performed on several features in the filtered feature set to obtain the target feature set.

[0029] Thirdly, this application provides a chip system including a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to execute the method described in any possible implementation of the first aspect and / or the algorithmic functions embodied in the apparatus described in any possible implementation of the second aspect.

[0030] Fourthly, this application provides a computing device including a processor and a memory. The processor is used to execute instructions stored in the memory to cause the computing device to perform the methods described in any possible implementation of the first aspect and / or the algorithmic functions embodied by the means described in any possible implementation of the second aspect.

[0031] Fifthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in any possible implementation of the first aspect and / or the algorithmic function embodied by the apparatus described in any possible implementation of the second aspect.

[0032] In a sixth aspect, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first aspect and / or the algorithmic function embodied by the apparatus described in any possible implementation of the second aspect, wherein the cluster of computing devices includes at least one computing device.

[0033] In a seventh aspect, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first aspect and / or the algorithmic function embodied by the apparatus described in any possible implementation of the second aspect, wherein the cluster of computing devices includes at least one computing device.

[0034] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0035] Figure 1 A schematic diagram illustrating the composition of a recommendation system architecture provided in an embodiment of this application;

[0036] Figure 2 A flowchart illustrating a feature filtering method provided in an embodiment of this application;

[0037] Figure 3 A schematic diagram of multiple iterative steps in the feature selection process provided in the embodiments of this application;

[0038] Figure 4a A schematic diagram of the network in the first iteration step of the feature selection process provided in the embodiments of this application;

[0039] Figure 4b A schematic diagram of the network in the second iteration step of the feature selection process provided in the embodiments of this application;

[0040] Figure 4c A schematic diagram of the network in the third iteration step of the feature selection process provided in the embodiments of this application;

[0041] Figure 5 A schematic diagram illustrating an example of a feature interaction contribution matrix provided in an embodiment of this application;

[0042] Figure 6 A schematic diagram of a first example of the feature filtering process provided in an embodiment of this application;

[0043] Figure 7 A schematic diagram of a second example of the feature filtering process provided in the embodiments of this application;

[0044] Figure 8a A schematic diagram of a third example of the feature filtering process provided in the embodiments of this application;

[0045] Figure 8b A schematic diagram of the feature filtering process, which is a third example of the feature filtering process provided in the embodiments of this application;

[0046] Figure 9A schematic diagram illustrating the change in the number of features in a third example of the feature filtering process provided in this application embodiment;

[0047] Figure 10 This is a schematic diagram of the composition of a feature screening device provided in an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0049] Figure 12 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0050] Figure 13 This is a schematic diagram of another computing device cluster structure provided in an embodiment of this application. Detailed Implementation

[0051] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0052] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0053] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0054] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0055] In the description of the embodiments of this application, unless otherwise stated, "several" means one or more, for example, several processing units means one or more processing units, and several elements means one or more elements.

[0056] To facilitate understanding of the solutions provided in the embodiments of this application, some of the terms involved in this solution will be briefly introduced first.

[0057] Machine learning systems: Based on input features and labels, the parameters of a machine learning model are trained using optimization methods such as gradient descent, and the trained model is ultimately used to predict the labels of unknown data.

[0058] Recommendation system: The recommendation system uses machine learning algorithms to analyze and learn from users' historical click behavior data, then predicts new user requests and returns a personalized list of recommended items.

[0059] Click-through rate (CTR): Also known as click probability, it refers to the probability that a user will click on a displayed item in a specific environment.

[0060] Feature selection: The process of selecting a subset of relevant features (i.e., attributes, metrics) for building a model.

[0061] Feature importance index: An index that measures the contribution of a feature to the model's prediction results.

[0062] Generally, recommendation system algorithms can be abstractly modeled as click-through rate (CTR) prediction models or CTR prediction algorithms. For example, a CTR prediction model is used to predict the probability that a user will click on an item in a specific environment. It mainly includes: extracting user features; extracting item features; inputting the user features and item features into the CTR prediction model, which then predicts whether the user will click on the item and outputs the probability that the user will click on the item.

[0063] Alternatively, the click-through rate prediction model can be a deep learning model. Compared to shallow machine learning models, deep learning models have more layers of non-linear variation, which can better fit the non-linear relationship between features (such as user features, item features, etc.) and labels (such as click probabilities, etc.). Therefore, deep learning models are widely used in recommender systems.

[0064] For example, a deep learning model can be abstracted into the mathematical form: y = f(x), where x is the feature, y is the label, and f(·) is the deep learning model used by the recommender system. It can be seen that the deep learning model f(·) and the feature x together determine the upper limit of the prediction performance of the final label y. Based on this form, it is clear that the feature x, as the input to the deep learning model, determines the upper limit of the recommender system. Therefore, in the field of recommender systems, how to select features that significantly improve the recommendation performance from a massive amount of features has become a key focus of the industry. To quantify the impact of each feature on the deep learning model, related solutions have designed different feature importance measurement methods and metrics based on different approaches.

[0065] In one solution, STG (Feature Selection using Stochastic Gates) is a widely used feature selection method in the field of neural networks. A trainable gating parameter is assigned to each feature domain, and after learning, features are ranked based on the magnitude of the gating parameter. For example, to achieve such trainable gating parameters, STG uses a reparameterization technique similar to VAE (Variational Autoencoder), transforming the theoretically Boolean-discrete gating parameters into continuously learnable latent variables, and using a differentiable Hard Sigmoid function to compress the distributed output as much as possible between 0 and 1. Furthermore, different combinations of gating trainability can be used to control whether the model explicitly depends on a certain feature.

[0066] In another solution, Model Class Reliance (MCR) is a widely accepted feature importance interval estimation algorithm in the field of shallow machine learning. This method proposes interval estimation of feature importance to combat the randomness introduced by single-point estimation. Its main contribution lies in proposing the concept of a Rashomon set. Within the entire model space that can be learned from the same model structure, the set of all models whose accuracy is more than ε below the optimal model is called the Rashomon set. This set defines all possible models fitted within the confidence interval ε. Each model in this set is relevant to the target feature X. i There is a feature importance evaluation metric, and the upper bound (MCR+) and lower bound (MCR-) of this metric constitute the feature importance confidence interval under the confidence coefficient ε. Furthermore, there is also an MR (Model Reliance) metric, which is an estimate of feature importance points. By shuffling the target feature and observing the degree of deterioration in the evaluation metric, we can determine whether the model strongly depends on the target feature during the learning period. Generally speaking, the greater the decrease in the metric after shuffling, the more important the target feature is.

[0067] In both solutions described above, the feature selection algorithms are "static." "Static" means that once the feature importance is ranked, the ranking remains fixed and is used in subsequent feature selection processes. A problem to consider is that if a feature is removed and the non-linear relationships of the remaining features change, the importance of those remaining features will also change. Therefore, a problem with both solutions is that the altered importance of the remaining features in subsequent feature selection processes can lead to poor recommendation results.

[0068] In other words, both of the above solutions treat feature importance ranking and feature selection as "one-step" tasks. That is, after training the model, a feature importance ranking is obtained, and feature selection is then performed based on this ranking. However, in reality, feature importance is dynamically changing. Removing a feature alters the importance of other closely related features. If features are selected in a "one-step" manner, it will lead to inaccurate feature selection, ultimately resulting in poor selection results and impacting downstream tasks.

[0069] In one example, a user's location information includes two features: "longitude" and "latitude." Assuming "longitude" ranks 4th in importance and "latitude" ranks 6th, and feature selection prioritizes the top 5 features, then "longitude" is retained while "latitude" is deleted. After deleting "latitude," the importance ranking of "longitude" decreases, potentially falling to 5th or 6th, leading to inaccurate feature selection and ultimately poor results. This also impacts downstream tasks; for example, in recommendation systems, recommending based on a user's location using only their "longitude" feature is ineffective.

[0070] Generally, feature selection involves two main steps: first, feature importance estimation and ranking; second, feature truncation. In both steps, the aforementioned solutions treat each feature as an independent contributor. However, in reality, features do not contribute independently. After a feature is removed in the second stage, the importance of features strongly interacting with it in the first stage will change. For example, with a large feature set, this deletion behavior can significantly reduce the accuracy of feature selection. For instance, with 100 features, deleting 50 in this way will significantly alter the interaction behavior of the remaining 50 features, leading to inaccurate selection.

[0071] In view of this, embodiments of this application provide a feature selection method. Taking into account the phenomenon of feature interaction influence in actual situations, embodiments of this application propose a feature selection method that considers feature interaction. This method mainly includes: a fine-tuning process, specifically a feature interaction fine-tuning process based on an asynchronous sparse training method; and a feature selection process, specifically an interaction-aware feature selection process.

[0072] For example, a feature interaction fine-tuning process based on asynchronous sparse training methods transforms the originally one-step feature selection task into a phased, step-by-step fine-tuning process. The original single-round model training process is transformed into a multi-round model training and fine-tuning process. Between consecutive rounds of model fine-tuning, feature selection and feature revival are performed, ensuring that feature importance estimates change along with feature changes. This allows for re-ranking of importance after a feature is deleted, and subsequent re-execution of feature selection and revival until a satisfactory feature set is obtained.

[0073] For example, a feature selection process based on feature interaction awareness ensures that the model uses second-order interaction information to truncate features from a globally optimal perspective, thus avoiding the local optimum problem caused by static truncation methods based on greedy algorithms.

[0074] To better understand the feature filtering method of this application embodiment, the application scenarios of the feature filtering method of this application embodiment are described below, but the description of the application scenarios does not constitute a limitation on the embodiments of this application.

[0075] See Figure 1 , Figure 1 This is a schematic diagram illustrating the composition of a recommendation system architecture provided in an embodiment of this application. For example... Figure 1 As shown in the embodiments of this application, a recommendation system architecture 100 includes a terminal 110 and a server 120. The input features of this recommendation system architecture 100 are obtained through the feature filtering method of this application embodiment, resulting in more accurate recommendation results.

[0076] Optionally, server 120 may include one or more servers. Figure 1 (The example includes a server) The server 120 can provide the methods or apparatus provided in the embodiments of this application to one or more terminals.

[0077] Optionally, a relevant application may be installed on the terminal 110. This application or webpage can provide an interface that displays the recommendation results. The terminal 110 can receive user input on the interface, operations performed, and other related behavioral data, and send this behavioral data to the server 120. The server 120 is equipped with the recommendation model of this embodiment and can determine new recommendation results based on the behavioral data.

[0078] It should be understood that in some optional implementations, terminal 110 may deploy the recommendation model and complete the work on its own without the cooperation of server 120, and this application embodiment is not limited in this regard. In some optional implementations, server 120 may also rely on user behavior data received from other devices besides the terminal or locally stored, without the cooperation of terminal 110, and this application embodiment is not limited in this regard.

[0079] The following description Figure 1 The product form of the terminal 110. In this application embodiment, the terminal 110 can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0080] The following description Figure 1 The product form of server 120. It can be further understood that server 120 can be various types of servers, such as x89 architecture servers, specifically rack servers, blade servers, high-density servers, platform servers, or high-performance servers, etc. In other words, this application embodiment does not specifically limit the specific type of server. Furthermore, it can be understood that... Figure 1 The server structure shown does not constitute a limitation on the server structure. A server may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0081] Furthermore, server 120 can be configured as an independent physical server, or as a server cluster or distributed system consisting of multiple physical servers. It can also be configured as a cloud server or cloud server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The cloud server cluster is deployed in several cloud data centers. The software can be an application that implements object control methods, but it is not limited to the above forms.

[0082] Next, the communication connection method between terminal 110 and server 120 is described. For example, terminal 110 and server 120 are connected via a network, enabling terminal 110 to access the cloud management platform deployed on the cloud server cluster. The network can be a wired network or a wireless network. For example, a wired network can be a cable network, fiber optic network, Digital Data Network (DDN), etc., while a wireless network can be a telecommunications network, intranet, Internet, Local Area Network (LAN), Wide Area Network (WAN), Wireless Local Area Network (WLAN), Metropolitan Area Network (MAN), Public Service Telephone Network (PSTN), Bluetooth network, ZigBee network, Global System for Mobile Communications (GSM), CDMA (Code Division Multiple Access) network, CPRS (General Packet Radio Service) network, etc., or any combination thereof.

[0083] Understandably, a network can use any known network communication protocol to enable communication between different terminal layers and gateways. These network communication protocols can be various wired or wireless communication protocols, such as Ethernet, Universal Serial Bus (USB), FireWire, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), New Radio (NR), Bluetooth, Wireless Fidelity (Wi-Fi), and other communication protocols.

[0084] In one possible scenario, server 120 can function as a cloud platform (a software platform employing application virtualization technology, integrating multiple functions such as software search, download, use, management, and backup). In practical use, server 120 can deploy a cloud management platform and a data center, with terminal 110 interacting with the cloud through the cloud management platform. Additionally, the data center can deploy nodes, which can be virtual machine instances, container instances, physical servers, etc.

[0085] In another possible scenario, the method provided in this application embodiment can be implemented by software. The software has a terminal and a server; the terminal 110 runs the software, and the server 120 runs the software's server. While the terminal 110 is running the software, it can call the server running on the server 120 to implement the method provided in this application embodiment.

[0086] In other words, the method provided in this application embodiment can be applied to the terminal 110 or the server 120. In specific implementation, it can run as software on the terminal 110 or the server 120; for example, the software can be a service or an application. This application embodiment can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application embodiment can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0087] Furthermore, the systems that can implement the technical solutions of the embodiments of this application include cloud servers, web pages, application programming interfaces (APIs), mobile application software, etc. The recommendation model of the embodiments of this application can be deployed on the server side, website, or mobile application software to meet the text generation needs of enterprise users and individual users, and can also be provided to developers for secondary development and use through open API interfaces. Furthermore, the implementation of the embodiments of this application mainly involves deploying the relevant platforms and applications of the recommendation model of the embodiments of this application, such as cloud platforms, websites, applications (APPs), etc.

[0088] like Figure 1As shown, the basic operating logic of the recommendation system architecture 100 is as follows: Users perform a series of actions based on the recommendation results on terminal 110, such as browsing, clicking, commenting, and downloading, generating behavioral data which is stored in server 120 in the form of logs. For example, the recommendation results may be a list of multiple items, and the multiple items may be, for example, a list of items such as... Figure 1 The items shown are Item 1, Item 2, Item 3, etc. The recommendation model in server 120 uses data, including user behavior logs, to obtain new recommendations for the user. This is the online inference process of the recommendation model. Additionally, the recommendation model also includes an offline training process. For example, the training process refers to the process by which the recommendation system architecture 100 uses data, including user behavior logs, to train the model offline, generating a prediction model after training convergence. Then, the trained model is deployed in an online service environment and provides recommendation results based on user requests, item features, and contextual information. The user then provides feedback on these recommendations, forming user data.

[0089] Exemplarily, the combination process of the recommender system architecture 100 and the feature selection method of this application embodiment is described from the model training side and the model application side. Exemplarily, the feature selection method provided in this application embodiment can be applied to the training process of the aforementioned recommender model. Based on the feature selection method of this application embodiment, the features input to the recommender model are selected to obtain a target feature set; based on the model feature set, the corresponding training data is determined; the recommender model performs symbolic and formal intelligent information modeling, extraction, preprocessing, and training on the training data, ultimately obtaining a trained recommender model. Whether in the model training stage or the inference stage, the method of this application embodiment can be applied to select the input features of the recommender model to reduce the number of input features, reduce the use of computing resources, reduce online latency, and reduce costs.

[0090] It should be noted that the relevant application scenarios of the recommender system are exemplary scenarios of the feature selection method in the embodiments of this application, and do not constitute a limitation on the application scenarios of the embodiments of this application. That is to say, the feature selection method in the embodiments of this application can be applied to any scenario in which input features are selected, such as generative model scenarios, image processing model scenarios, etc. In other words, the system scenario applied in the embodiments of this application is a machine learning-based application scenario. Among them, the click-through rate prediction scenario in the recommender system architecture 100 is an example for description. The click-through rate prediction scenario is a typical scenario in machine learning applications, and its main structure includes a display list, logs, offline training modules, online prediction modules, etc.

[0091] Taking a recommender system architecture 100 as an example, the recommender system can record thousands of user and item features throughout the entire process. Applying all of these features to the input features of the recommender model is impractical because more features mean more computing resources are needed, online latency will increase accordingly, and the technical implementation cost will rise. Furthermore, noisy and redundant features will degrade the training results of the recommender model. Therefore, feature filtering is necessary for all features. In this process, accurately assessing feature importance is crucial to improving model accuracy. The goal of this application's embodiments is to improve the accuracy of feature filtering, that is, to solve the problem that static feature filtering algorithms cannot adjust importance weights in a timely manner according to feature changes.

[0092] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0093] See Figure 2 , Figure 2 This is a flowchart illustrating a feature selection method provided in an embodiment of this application. Figure 2 As shown, a feature selection method mainly includes the following steps:

[0094] Step S210: Obtain the training dataset. The training dataset includes several training data subsets, each training data subset includes several training samples, and each training sample includes feature values ​​of several features from the feature set to be selected.

[0095] Step S220: Train the target network based on a subset of training data to obtain the trained target network.

[0096] Step S230: Based on the feature interaction contribution information, a filtered feature set is obtained from the feature set to be filtered. The feature interaction contribution information represents the contribution of each at least two features in the feature set to be filtered to the trained target network.

[0097] Step S240: Based on the filtered feature set, obtain the target feature set. The number of features in the target feature set is the target number.

[0098] It should be noted that the feature selection method in this embodiment requires multiple iterative steps. Each iterative step includes a model training phase (step S220) and a feature selection phase (step S230). In each iterative step, several retained features from the retained feature set in the previous iterative step are considered as the feature set to be selected, and the corresponding network is considered as the target network. Feature selection is performed to obtain features to be deleted. Similarly, several deleted features from the deleted feature set in the previous iterative step are considered as the feature set to be selected, and the corresponding network is considered as the target network. Feature selection is also performed to obtain features to be retained. The features to be retained are reinstated to the retained feature set, and the features to be deleted are moved to the deleted feature set. The process then proceeds to the next iterative step. After multiple iterative steps, the final target feature set is obtained. Several features in the target feature set can be applied to scenarios such as recommendation systems.

[0099] For example, in the first iteration step, the target network includes a main network, the feature set to be selected includes an initial feature set to be selected, and the training dataset includes a first training data subset. During the training phase, the main network is trained based on the first training data subset to obtain a trained main network. During the feature selection phase, based on first feature interaction information, a first retained feature set and a first deleted feature set are selected from the initial feature set to be selected. The first feature interaction information represents the contribution of each at least two features in the initial feature set to be selected to the trained main network.

[0100] For example, in the second iteration step, the target network further includes a focusing network and a revival network, the feature set to be filtered further includes a first retained feature set and a first deleted feature set, the filtered feature set includes a second retained feature set and a second deleted feature set, and the training dataset further includes a second training data subset. The focusing network is obtained by performing a masking operation on the trained main network, whereby the masking operation is used to mask the parameters corresponding to the features in the first deleted feature set; the revival network is obtained by performing a parameter freezing operation on the trained main network, whereby the parameter freezing operation is used to freeze the parameters of the trained focusing network.

[0101] For example, in the second iteration step, the above step S220 includes: training a focusing network based on the second training data subset to obtain a trained focusing network; and training a revival network based on the second training data subset to obtain a trained revival network.

[0102] For example, in the second iteration step, step S230 includes: filtering several features in the first retained feature set based on the second feature interaction information to obtain features to be deleted, wherein the second feature interaction information represents the contribution of each at least two features in the first retained feature set to the trained focusing network; filtering several features in the first deleted feature set based on the third feature interaction information to obtain features to be retained, wherein the third feature interaction information represents the contribution of each at least two features in the first deleted feature set to the trained revival network; moving the features to be retained from the first deleted feature set to the first retained feature set to obtain a second retained feature set; and moving the features to be deleted from the first retained feature set to the first deleted feature set to obtain a second deleted feature set.

[0103] Optionally, by sorting the first set of deleted features by feature importance, several features to be revived can be obtained.

[0104] Optionally, several features to be retained are obtained from the first set of retained features. Then, based on the several features to be retained and the several features to be revived, a second set of retained features is obtained.

[0105] Optionally, the first set of retained features is sorted by feature importance to obtain several features to be retained. These features to be retained can also be understood as features remaining after deleting some features from the first set of retained features.

[0106] Optionally, one possible implementation of obtaining the first set of retained features and the first set of deleted features includes: training a main network, the input features of which are several features to be selected; based on the trained main network, ranking the features to be selected by feature importance to obtain the first set of retained features and the first set of deleted features. The main network can be any neural network model, and this embodiment does not limit it. In each iteration step, the first set of retained features and the first set of deleted features for the current iteration step can be obtained from the previous iteration step.

[0107] This implementation provides a method for obtaining the first set of retained features and the first set of deleted features. In the first iteration step, there is no previous iteration step, so the main network can be trained. Based on the trained main network, the feature importance of several features to be screened in the main network is ranked to obtain the first set of retained features and the first set of deleted features.

[0108] Optionally, the first set of retained features is sorted by feature importance to obtain several features to be retained and several features to be deleted. The method further includes: based on the several features to be deleted, several second deletion features are obtained, and the several second deletion features are used in the next iteration step to filter and obtain several features to be revived.

[0109] It should be noted that after ranking the features by importance in the first set of retained features, several features to be retained are obtained, along with several features to be deleted. Considering the correlation between the deleted and retained features, the importance of the retained features may change after deletion. Therefore, in the next iteration, some features need to be selected from the deleted features of the previous iteration to be retained.

[0110] This implementation provides a method for determining features to be deleted. Based on several features to be deleted, several second deletion features are obtained. Based on these second deletion features, several features to be retained can be obtained in the next iteration step.

[0111] Optionally, for deleted features, they can be retained for multiple iterations without being revived before being discarded, or retained for one iteration without being revived before being discarded directly. Optionally, after each feature selection, the main network needs to be updated. The input features of the updated main network only include the features in the first retained feature set and the features in the first deleted feature set.

[0112] Optionally, one possible implementation of ranking the first set of retained features by feature importance to obtain several features to be retained includes: training a focusing network, which obtains the first set of retained features from the input features through the main network; and ranking the first set of retained features by feature importance based on the focusing network to obtain several features to be revived.

[0113] This implementation provides a method for determining features to be retained from a first set of retained features. Using a trained focusing network, the feature importance ranking of the input features (i.e., features in the first set of retained features) can be recalculated, resulting in a new ranking of the features in the first set of retained features. Based on this new ranking, several features are identified as features to be deleted.

[0114] Optionally, one possible implementation of ranking the first deleted feature set by feature importance to obtain several features to be revived includes: freezing the parameters of the trained focusing network, training the main network to adjust the parameters of the revival network, and obtaining the revival network by retaining the first deleted feature set in the input features through the main network; and ranking the first deleted feature set by feature importance based on the revival network to obtain several features to be revived.

[0115] This implementation provides a method for determining the features to be revived from the first deleted features. This method involves reviving a subset of deleted features as retained features. The parameters of the trained focusing network are frozen, and the revival network is trained to adjust its parameters. This ensures the correlation between the parameters of the revival network and the focusing network, thus guaranteeing the correlation between the first deleted features and the first retained features. In this way, the correlation between the first retained features and the first deleted features is considered. The second retained feature obtained from the first deleted features is equivalent to a more important feature among the deleted features and a more relevant feature among the retained features, ensuring that the second retained feature is a relatively important feature in the current iteration step. This is equivalent to comprehensively considering the more important features among the retained features and the more important features among the deleted features. Simultaneously, the parameter adjustment process of the focusing network does not involve the parameters of the revival network, achieving the independence of the focusing network's parameter adjustment process.

[0116] Optionally, the focusing network is obtained by using input features outside the first reserved feature set of the masked main network. This implementation provides a method for obtaining the focusing network. Obtaining the focusing network using input features outside the first reserved feature set of the masked main network is simple and easy to implement.

[0117] See Figure 3 , Figure 3 This diagram illustrates multiple iterative steps in the feature selection process provided in an embodiment of this application. Figure 3 As shown, any neural network can be trained to execute the feature selection method of this application embodiment. An example will be provided below for illustration.

[0118] In one example, in the first iteration (t=0), the complete main network is trained, assuming it has four input features: w1, w2, w3, and w4. After training, the features are ranked by importance, resulting in the order w1, w3, w2, w4 from highest to lowest importance. The last two features, w2 and w4, are then removed. The second iteration (t=1) then begins training. In this example, the network corresponding to features w2 and w4 is the revive net, and the network corresponding to features w1 and w3 is the focus net.

[0119] In this example, in the second iteration step (t=1), the revival network corresponding to the mask features w2 and w4 is trained, the focusing network corresponding to the features w1 and w3 is trained, and the parameters of the focusing network are adjusted. After the focusing network is trained, it is frozen, and the main network is trained to adjust the parameters of the revival network. The features w1 and w3 of the trained focusing network are ranked by feature importance; the features w2 and w4 of the trained revival network are ranked by feature importance. Combining the results of the two feature importance rankings, the features ranked higher from the features w2 and w4 to be deleted are selected for revival, resulting in new retained features w1, w3, and w4. Multiple iteration steps, such as the third iteration step (t=2), are executed until the features that meet the requirements are obtained. The subsequent iteration process is not described in detail here. In the third iteration step (t=2), the main network can be considered as a new main network, which includes the focusing network corresponding to the new retained features w1, w3, and w4 and the revival network corresponding to the new deleted feature w2.

[0120] Optionally, features ranked lower from the remaining features w1 and w3 can be deleted again, such as w1. These deleted features are then used as input features for the revival network in the next iteration. In this case, in the third iteration (t=2), the main network can be considered a new main network, which includes the focusing network corresponding to the new retained features w3 and w4, and the revival network corresponding to the deleted features w1 and w2. To avoid the number of features in the revival network becoming excessive with each iteration, the revival network can include only the new deleted feature w1.

[0121] See Figure 4a - Figure 4c , Figure 4a A schematic diagram of the network in the first iteration step of the feature selection process provided in the embodiments of this application. Figure 4b This is a schematic diagram of the network in the second iteration step of the feature selection process provided in the embodiments of this application. Figure 4c This is a schematic diagram of the network in the third iteration step of the feature selection process provided in an embodiment of this application. Figure 4a As shown, in one example, in such Figure 3 In the first iteration step (t=0) shown, the main network includes an input layer, hidden layers, and an output layer. The input to the main network is the four features w1, w2, w3, and w4. Figure 4a As shown, in this example, in such Figure 3 In the second iteration step (t=1) shown, the main network includes a focusing network and a revival network. The input to the focusing network is the two features w1 and w3, and the input to the revival network is the two features w2 and w4. Figure 4b As shown, in this example, in such Figure 3In the third iteration step (t=2) shown, the main network includes a focusing network and a revival network. The input of the focusing network is the three features w1, w3, and w4, and the input of the revival network is the feature w2.

[0122] For example, in one iteration step, the loss function of the main network can be... The loss function of a focus net can be The loss function of a revive net can be... Where θ1∪θ2=θ. θ1 represents the parameters of the focusing network, and θ2 represents the parameters of the revival network.

[0123] In other words, the training process of the network model can be assumed to be implemented through a first module, which splits the original complete model training data into multiple data blocks and improves the process and network structure corresponding to one solution. This one solution refers to training all data once and then performing feature selection once.

[0124] In this embodiment, in the first module, the parameters of the neural network are first split into two parts. One part belongs to the focus network, which is used for feature selection and changes in real time during the optimization process. The other part belongs to the revive network, which is used for feature revival. These parameters do not directly affect the network optimization, but they are updated. This update can be understood as continuously practicing to make the copy of the focus network perform better, but it does not affect the update of the focus network itself. For example, in the process, in data block 1, a neural network is fully trained, feature selection is performed, and the selected features are discarded. In data block 2, the selected features are used to further fine-tune the remaining focus network, while the filtered features are updated along with the deleted revive network, but this does not affect the focus network. After convergence, a feature importance evaluation is performed. Features that are important to the revive network are revived, while features that are not important to the focus network are filtered out. The feature selection process for data blocks 3, 4, and 5 follows the same pattern. This ensures that the importance of features changes in real time as the remaining feature set changes.

[0125] Optionally, the above feature importance ranking represents the ranking of the importance of multi-feature interactions, and the importance of multi-feature interactions represents the degree of influence of multiple features on the output of the target network. The target network is a network whose input features are the features whose importance is ranked.

[0126] Optionally, the multi-feature interaction importance is the second gradient of the target network's loss function with respect to any two features.

[0127] In this embodiment, the process of evaluating the importance of features is assumed to be implemented through a second module. This second module is a feature interaction-aware filtering module. The biggest difference between this second module and previous feature filtering modules is that previous modules used a greedy algorithm to select and delete features with the lowest importance ranking. However, a phenomenon occurs when a feature is deleted; the importance of other features interacting with that feature also changes. The ranking of the remaining features' importance also changes. Therefore, deleting the last k features without considering this change cannot fully account for the interaction relationships between features. The second module solves this problem by first estimating the feature interaction contribution matrix using the second-order gradient method. This second-order gradient refers to the second-order gradient of the corresponding loss function with respect to any two features. Based on the two-dimensional interaction contribution matrix, k features are selected for deletion, maximizing the sum of the remaining interaction contribution matrices. In this way, the k least important features are deleted while considering feature interactions, where k is a positive integer.

[0128] See Figure 5 , Figure 5 This is a schematic diagram illustrating an example of a feature interaction contribution matrix provided in an embodiment of this application. In one example, the feature interaction contribution matrix formed by 26 input features is as follows: Figure 5 As shown in the diagram. Each row represents the second-order gradient of a feature with other features, and each column represents the second-order gradient of other features with a single feature. For example, the i-th row and j-th column represents the loss function L applied to the i-th feature w. i With the j-th feature w j second gradient For example, such as Figure 5 The matrix element corresponding to the box shown is the element in the 6th row and 17th column, representing the loss function L applied to the 6th feature w6 and the 17th feature w6. 17 second gradient For example, the importance value of the i-th feature is the sum of the i-th row or i-th column of the feature interaction contribution matrix.

[0129] Understandably, the larger the first-order gradient of the loss function with respect to feature a, the greater the impact of changes in the value of feature a on the loss function, and the more important feature a is. Similarly, the larger the second-order gradients of the loss function with respect to features a and b, the greater the impact of changes in the values ​​of features a and b on the first-order gradients, and consequently, the greater their impact on the loss function. This results in a higher overall importance for both features a and b, which explains the validity of the feature interaction contribution matrix. Furthermore, based on this reasoning, the nth-order gradient of the loss function with respect to n features can be used to represent the overall importance of the interactions among the n features, where n is a positive integer.

[0130] See Figure 6 , Figure 6 This is a schematic diagram of a first example of a feature filtering process provided in an embodiment of this application. For example... Figure 6 As shown, in the first example, the feature selection process mainly includes the following steps:

[0131] Step S601: First, determine a main network. The input features of the main network are all the features to be filtered.

[0132] Step S602: Train the main network to obtain the trained main network.

[0133] Step S603: Sort the parameters of the trained main network by feature importance, retain the top-ranked features to obtain the first retained features, and delete the bottom-ranked features to obtain the first deleted features.

[0134] Step S604: Fine-tune the focusing network 1 corresponding to the first set of retained features to obtain the fine-tuned focusing network 1.

[0135] Step S605: Freeze the finely tuned focusing network 1, and finely tune the revival network 1 corresponding to the first deleted feature to obtain the finely tuned revival network 1.

[0136] Step S605: Using Focusing Network 1, the first retained features are ranked by feature importance. Features ranked at the top are retained to obtain features to be retained; features ranked at the bottom are deleted to obtain the first deleted features. Using Revival Network 1, the deleted features are ranked by feature importance. Features ranked at the top are retained to obtain revival features; features ranked at the bottom are deleted to obtain the second deleted features.

[0137] Furthermore, by combining the features to be retained and the features to be revived, a second retained feature is obtained. Similarly, by combining the first and second deleted features, a second deleted feature is obtained; alternatively, the first deleted feature can be directly used as the second deleted feature. In other words, each deleted feature is revived only once, reducing the number of features in the second deleted feature, simplifying computation, and avoiding the continuous accumulation of deleted features. This results in a new main network with fewer input features. The new main network includes a focusing network 2 and a revival network 2. The focusing network 2 is the network whose input features are the second retained features, and the revival network 2 is the network whose input features are the second deleted features.

[0138] Step S606: Mask the resurrected network 2, fine-tune the focusing network 2 corresponding to the second preserved feature set, and obtain the fine-tuned focusing network 2.

[0139] Step S607: Freeze the finely tuned focusing network 2, and finely tune the revival network 2 corresponding to the second deleted feature to obtain the finely tuned revival network 2.

[0140] Furthermore, through multiple rounds of network training and feature truncation, the desired feature set is obtained through repeated filtering.

[0141] See Figure 7 , Figure 7 This is a schematic diagram of a second example of the feature filtering process provided in an embodiment of this application. For example... Figure 7 As shown, in the second example, the method of this application embodiment is mainly used in the feature selection part of offline testing in machine learning systems (such as recommendation systems). After the recommendation system retrieves massive features from logs, before entering the subsequent recommendation model, the method of this application embodiment is used to accurately evaluate the feature importance, and based on the feature importance, a target number of features are selected from the massive features to help with subsequent online recommendation services. In this example, the feature selection process mainly includes the following steps:

[0142] Step S701: Divide the training data into N blocks, and use the first data block to train a neural network. N is a positive integer.

[0143] Step S702: Invoke the second module of importance assessment. First, calculate the contribution map of feature interactions based on the second gradient, for example... Figure 5 The diagram shows the feature interaction contribution. K features are selected, and after deleting the rows and columns corresponding to these K features, the remaining interactive features are chosen to maximize their importance (e.g., maximizing the sum of the remaining matrices). These K features are then removed. The remaining MK features are retained. M represents the number of input features to the main network in this iteration step, and both M and K are positive integers.

[0144] In step S703, the parameters of the main network are grouped and masked. The MK retained features are not masked and will participate in subsequent training (as parameters of FocusNet), while the network parameters of the K deleted features are masked (as parameters of ReviveNet).

[0145] Step S704: On data block 2, fine-tune focusnet, then freeze focusnet and fine-tune revivenet.

[0146] Step S705: Based on the fine-tuned FocusNet, perform a new round of feature importance ranking, deleting those with lower rankings. Also, perform feature importance ranking on ReviveNet, reviving the top-ranked features. Update ReviveNet and FocusNet, as well as the feature sets currently in use.

[0147] Step S706: Repeat the feature selection process multiple times until the number of features meets the feature selection requirements.

[0148] See Figure 8a , Figure 8a This is a schematic diagram of a third example of the feature selection process provided in an embodiment of this application. For example... Figure 8a As shown, taking personalized advertising in various mobile scenarios as an example, this project mined a large number of user and item features, totaling over a thousand, to achieve personalized advertising. However, since advertising is conducted through real-time bidding, slow model calculation speed can lead to lag, resulting in poor user experience and revenue loss. Therefore, the system has high latency requirements, and the feature measurement can only support a maximum of about 150 features. Selecting the 150 most useful features offline from thousands of features requires a relatively accurate feature importance evaluation algorithm. In this step, the feature importance evaluation algorithm designed in this embodiment is applied, and the application process includes:

[0149] Step S810: In the offline scenario, first obtain a dataset of 1,000 features and hundreds of millions of samples within one day (as the training dataset).

[0150] Step S820: Split the training dataset into 5 parts and proceed to the feature selection process.

[0151] See Figure 8b , Figure 8b This is a schematic diagram of a feature selection process, representing a third example of the feature selection process provided in this application embodiment. (See attached diagram.) Figure 8bAs shown, the feature selection process mainly includes:

[0152] Step S821: First, train the neural network using the first piece of data, and sort the importance of 1000 features using a second-order gradient-based method to obtain a corresponding 1000x1000 importance interaction contribution matrix. Find the 100 least important features, i.e., the matrix of the remaining features after removing these 100 features from the interaction contribution matrix, where the sum of the elements is maximized.

[0153] In step S822, the parameters corresponding to the 900 retained features are used as the parameters of FocusNet, and the parameters of the remaining 100 features are used as the parameters of ReviveNet.

[0154] Step S823: Using the second data block, FocusNet is further fine-tuned, thus readjusting the interaction relationships of the 900 features, resulting in a more accurate ranking than with 1000 features. Subsequently, ReviveNet is fine-tuned while maintaining FocusNet's foundation, allowing these 100 features to be ranked based on the new 900 features. The last 200 features of the 900 are deleted (as features to be revived in the next round), and the first 20 features from ReviveNet are revived.

[0155] In step S824, using the third data block, FocusNet is further fine-tuned, thus readjusting the interaction relationships of the 720 features for a more accurate ranking than with 900 features. Subsequently, ReviveNet is fine-tuned while maintaining FocusNet's foundation, allowing the 200 features removed in the previous step to be ranked in importance based on the new 720 features. The last 300 features of the 720 are removed (as features to be revived in the next round), and the first 20 features from ReviveNet are revived.

[0156] In step S825, using the fourth data block, FocusNet is further fine-tuned, thus readjusting the interaction relationships of the 440 features for a more accurate ranking than with 720 features. Subsequently, ReviveNet is fine-tuned while maintaining FocusNet's foundation, allowing the 300 features deleted in the previous step to be ranked in importance based on the new 440 features. The last 300 features of the 440 are selected for deletion (as features to be revived in the next round), and the first 20 features from ReviveNet are selected for revival.

[0157] Step S826: Using the fifth data block, FocusNet is further fine-tuned, thus readjusting the interaction relationships of the 160 features, resulting in a more accurate ranking than with 440 features. Subsequently, ReviveNet is fine-tuned while maintaining FocusNet's foundation, allowing the 300 features deleted in the previous step to be ranked in importance based on the new 150 features. The last 30 features of the 160 are selected for deletion (as features to be revived in the next round), and the first 20 features of ReviveNet are selected for revival. The above figures are merely examples and do not constitute a limitation on the embodiments of this application. 54

[0158] See Figure 9 , Figure 9 This is a schematic diagram illustrating the change in the number of features in a third example of the feature selection process provided in this application embodiment. (See attached diagram.) Figure 9 As shown, through multiple feature deletions and feature revivals, 150 features were ultimately selected. This selection process, compared to prioritizing features once and then selecting, increasingly focuses on the remaining feature subset. In other words, using the method of this embodiment, 150 features were selected from over 1000 features for deployment, achieving a 2-thousandth improvement in accuracy compared to the control feature selection algorithm. This model group performed best and successfully deployed to the entire dataset, with an online metric ECPM (effective cost per mile, revenue generated per thousand impressions) increasing by at least 2.49%.

[0159] This application primarily relates to the feature importance evaluation process and the feature truncation process. In some methods, feature importance and feature selection are both completed in one evaluation and one selection using the entire dataset. However, this approach fails to consider the interaction between features. This application improves this process by using a multi-round evaluation method to allow the interaction relationships between remaining features to adjust in real time. It also allows deleted features to be revived based on new feature interaction relationships. For feature truncation (i.e., feature deletion), optimizations, such as using the second-order gradient of the loss function with respect to any two features, ensure that the changes in the interaction relationships of the remaining features after deleting a certain feature are considered during feature reduction.

[0160] This application provides a feature interaction fine-tuning process based on an asynchronous sparse training method. This process breaks down a training round into multiple stages, fine-tuning the results of each stage's selection. In some stages, focusing and resurrection networks are used to re-evaluate interactions after feature changes, ensuring the real-time nature of feature importance assessment.

[0161] In other words, the technical solution of this application embodiment is a deep learning feature selection algorithm that explicitly considers the influence of feature interactions; the technical solution of this application embodiment is independent of the network model itself and can be applied to the feature selection algorithm of any deep learning network model; the technical solution of this application embodiment can significantly improve the algorithm accuracy of deep learning feature selection by considering the influence of feature interactions; compared with other methods, the technical solution of this application embodiment is more efficient and can be completed in one training process.

[0162] In this embodiment, the impact of feature interactions on the importance of individual features is explicitly considered during feature filtering, and filtering is performed based on this consideration, making the feature filtering process also interaction-intensity conscious. In other words, the technical solution of this embodiment considers the contribution of feature interactions during feature reduction, preventing local optima from shifting to global optima during feature reduction, thereby improving filtering accuracy.

[0163] See Figure 10 , Figure 10 This is a schematic diagram illustrating the composition of a feature screening device provided in an embodiment of this application. Figure 10 As shown, this application provides a feature screening device 1000, comprising:

[0164] The acquisition module 1010 is used to acquire training datasets. Each training dataset includes several training samples, and each training sample includes the feature values ​​of several features in the feature set to be screened.

[0165] Training module 1020 is used to train the target network based on a subset of training data to obtain the trained target network.

[0166] The feature selection module 1030 is used to select a set of selected features from the set of features to be selected based on feature interaction contribution information, wherein the feature interaction contribution information represents the contribution of each at least two features in the set of features to be selected to the trained target network; and to obtain a target feature set based on the selected feature set, wherein the number of features in the target feature set is the target number.

[0167] In one possible implementation, the target network includes a main network, the feature set to be filtered includes an initial feature set to be filtered, the filtered feature set includes a first retained feature set and a first deleted feature set, the training dataset includes a first training data subset, the first retained feature set includes several retained features, and the first deleted feature set includes several deleted features. The training module 1020 is specifically used to train the main network based on the first training data subset to obtain the trained main network. The feature filtering module 1030 is specifically used to filter the initial feature set to be filtered to obtain the first retained feature set and the first deleted feature set based on first feature interaction information, where the first feature interaction information represents the contribution of at least two features in the initial feature set to the trained main network.

[0168] In one possible implementation, the target network further includes a focusing network and a revival network. The feature set to be filtered further includes a first retained feature set and a first deleted feature set. The filtered feature set includes a second retained feature set and a second deleted feature set. The training dataset further includes a second training data subset. The focusing network is obtained by performing a masking operation on the trained main network. The masking operation is used to mask the parameters corresponding to the features in the first deleted feature set. The revival network is obtained by performing a parameter freezing operation on the trained main network. The parameter freezing operation is used to freeze the parameters of the trained focusing network. Specifically, the training module 1020 is used to train the focusing network based on the second training data subset to obtain the trained focusing network; and to train the revival network based on the second training data subset to obtain the trained revival network. The aforementioned feature selection module 1030 is specifically used to: select several features from the first retained feature set based on the second feature interaction information to obtain features to be deleted, wherein the second feature interaction information represents the contribution of each at least two features in the first retained feature set to the trained focusing network; and select several features from the first deleted feature set based on the third feature interaction information to obtain features to be revived, wherein the third feature interaction information represents the contribution of each at least two features in the first deleted feature set to the trained revival network; move the features to be revived from the first deleted feature set to the first retained feature set to obtain the second retained feature set; and move the features to be deleted from the first retained feature set to the first deleted feature set to obtain the second deleted feature set.

[0169] In one possible implementation, the feature filtering module 1030 described above is also used to delete all features in the first deleted feature set.

[0170] In one possible implementation, the feature selection module 1030 is specifically used to select a number of features to be retained and a number of features to be deleted from the feature set to be selected with the goal of maximizing the feature interaction sub-information, so as to obtain a selected feature set. The feature interaction sub-information represents the total contribution of each at least two features in the feature set to be selected after deleting the deleted features to the trained target network.

[0171] In one possible implementation, the feature interaction contribution information is obtained based on the multi-order gradients of at least two features in the set of features to be screened, using the loss function of the trained target network.

[0172] In one possible implementation, the feature interaction contribution information is obtained based on the second-order gradient of every two features in the feature set to be screened, using the loss function of the trained target network. The feature interaction contribution information represents the contribution matrix of every two features in the feature set to be screened to the trained target network. Specifically, the feature screening module 1030 is used to select K features from several features to be screened, with the objective of maximizing the sum of the contribution matrices after deleting the rows and columns corresponding to K features, to obtain a screened feature set, where K is a positive integer.

[0173] In one possible implementation, the feature filtering module 1030 is specifically used to determine whether the number of features in the filtered feature set is the target number; if the number of features in the filtered feature set is the target number, then the filtered feature set is determined as the target feature set; if the number of features in the filtered feature set is greater than the target number, then feature filtering operations are performed on several features in the filtered feature set to obtain the target feature set.

[0174] The following is about... Figure 10 The software and hardware implementation of the feature selection device 1010 shown (hereinafter referred to as: related device of feature selection method) needs further explanation.

[0175] As an example of a software functional unit, a module can include code running on a computing instance. A computing instance can include at least one of a physical host (computing device), a virtual machine, or a container. Furthermore, the aforementioned computing instance can be one or more. For example, a module can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed within the same availability zone (AZ) or in different AZs, each AZ comprising one or more geographically proximate data centers. Typically, a region can include multiple AZs.

[0176] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0177] As an example of a hardware functional unit, a module may include at least one computing device, such as a server. Alternatively, a module may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The aforementioned PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0178] The multiple computing devices included in the module can be distributed within the same region or in different regions. Similarly, the multiple computing devices included in the module can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the module can be distributed within the same Virtual Private Cloud (VPC) or multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0179] It should be noted that, in other embodiments, the related apparatus of the feature filtering method is additionally provided with one or more modules for performing any of the steps included in the above implementation. The steps implemented by one or more modules in the related apparatus of the feature filtering method can be specified as needed, and more or fewer modules can be obtained than in the embodiments of this application to implement different steps in the above method, thereby realizing all the functions of the related apparatus of the feature filtering method.

[0180] This application also provides a computing device 1100. For example... Figure 11 As shown, the computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, memory 1106, and communication interface 1108 communicate with each other via the bus 1102. The computing device 1100 can be a server, such as a central server, an edge server, or a local server in a local data center, or it can be an electronic device such as a desktop computer, laptop computer, or smartphone. It should be understood that this application does not limit the number of processors and memories in the computing device 1100.

[0181] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu Bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 11The bus 1104 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1104 may include a path for transmitting information between various components of the computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0182] The processor 1104 may include any one or more of the following computing devices: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP) or digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0183] Memory 1106 may include volatile memory, such as random access memory (RAM). Processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 1106 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0184] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 1106, or two or more types of storage media can be configured to realize the function of memory 1106. This application does not limit this.

[0185] The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the functions of the relevant devices of the feature selection method, thereby implementing the method described in the above embodiments. That is, the memory 1106 stores instructions for executing the method described in the above embodiments.

[0186] Alternatively, the memory 1106 may store executable code, which the processor 1104 executes to implement the functions of the relevant devices of the feature selection method, thereby implementing the method described in the above embodiments. That is, the memory 1106 stores instructions for executing the method described in the above embodiments.

[0187] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0188] As one possible implementation, the computing device 1100 may also include a chip system, which includes a processor and a power supply circuit. The power supply circuit supplies power to the processor, and the processor executes the operation steps corresponding to the methods of the embodiments of this application. For simplicity, further details are omitted here. The processor can be implemented using a GPU, or it can be implemented using computing devices or AI chips such as a DPU, NPU, XPU, SoC, offload card, or accelerator card.

[0189] As one possible implementation, the computing device 1100 may include various types of processors 1104, that is, the computing device 1100 is a heterogeneous device. For example, the computing device 1100 includes a CPU and a GPU, and at least one of the processors 1104 can execute the operation steps corresponding to the method of the embodiments of this application. For the sake of brevity, further details are omitted here.

[0190] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be an electronic device such as a desktop computer, a laptop computer, or a smartphone.

[0191] like Figure 12 As shown, the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions for performing the methods described in the above embodiments.

[0192] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the methods described in the above embodiments. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for executing the methods described in the above embodiments.

[0193] It should be noted that the memory 1106 in different computing devices 1100 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the relevant apparatus of the feature selection method. That is, the instructions stored in the memory 1106 of different computing devices 1100 can implement the functions of one or more modules of the relevant apparatus of the feature selection method.

[0194] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 13 One possible implementation is shown. For example... Figure 13 As shown, the two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 1106 in computing device 1100A stores instructions for the functions of one or more modules of the relevant apparatus of the feature filtering method. Simultaneously, the memory 1106 in computing device 1100B stores instructions for the functions of another one or more modules of the relevant apparatus of the feature filtering method.

[0195] It should be understood that Figure 13 The functions of computing device 1100A shown can also be performed by multiple computing devices 1100. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices 1100.

[0196] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 12 and Figure 13 The connection method of the computing device cluster is different in that the memory 1106 of one or more computing devices 1100 in the computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0197] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the aforementioned data processing method. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for performing the aforementioned method.

[0198] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium capable of being stored in the cluster of computing devices or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0199] Based on the methods in the above embodiments, this application provides a computer program product containing instructions. The computer program product may be software or program products containing instructions, capable of running on a computing device or stored on any available medium. When the instructions are executed by a cluster of computing devices containing at least one computing device, at least one computing device in the cluster of computing devices performs the methods in the above embodiments.

[0200] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0201] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0202] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0203] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A feature selection method, characterized in that, include: Obtain a training dataset, which includes several training data subsets, each training data subset includes several training samples, and each training sample includes feature values ​​of several features in the feature set to be screened. Based on the aforementioned subset of training data, the target network is trained to obtain the trained target network. Based on the feature interaction contribution information, a filtered feature set is obtained from the set of features to be filtered. The feature interaction contribution information represents the contribution of each at least two features in the set of features to be filtered to the trained target network. Based on the filtered feature set, a target feature set is obtained, and the number of features in the target feature set is the target number.

2. The method according to claim 1, characterized in that, The target network includes a main network, the feature set to be screened includes an initial feature set to be screened, the screened feature set includes a first retained feature set and a first deleted feature set, the training dataset includes a first training data subset, the first retained feature set includes several retained features, and the first deleted feature set includes several deleted features. The step of training the target network based on the subset of training data to obtain the trained target network includes: Based on the first subset of training data, the main network is trained to obtain the trained main network. The process of selecting a target feature set from the set of features to be selected based on feature interaction contribution information includes: Based on the first feature interaction information, the first retained feature set and the first deleted feature set are obtained from the initial feature set to be screened. The first feature interaction information represents the contribution of each at least two features in the initial feature set to be screened to the trained main network.

3. The method according to claim 2, characterized in that, The target network further includes a focusing network and a revival network. The feature set to be filtered further includes a first retained feature set and a first deleted feature set. The filtered feature set includes a second retained feature set and a second deleted feature set. The training dataset further includes a second training data subset. The focusing network is obtained by performing a masking operation on the trained main network. The masking operation is used to mask the parameters corresponding to the features in the first deleted feature set. The revival network is obtained by performing a parameter freezing operation on the trained main network. The parameter freezing operation is used to freeze the parameters of the trained focusing network. The step of training the target network based on the subset of training data to obtain the trained target network further includes: The focusing network is trained based on the second subset of training data to obtain the trained focusing network; Based on the second subset of training data, the resurrection network is trained to obtain the trained resurrection network; The step of filtering the set of features from the set of features to be filtered based on feature interaction contribution information further includes: Based on the second feature interaction information, several features in the first retained feature set are filtered to obtain features to be deleted. The second feature interaction information represents the contribution of each at least two features in the first retained feature set to the trained focusing network. Based on the third feature interaction information, several features in the first deleted feature set are filtered to obtain the features to be revived. The third feature interaction information represents the contribution of each at least two features in the first deleted feature set to the trained revival network. The feature to be revived is moved from the first deleted feature set to the first retained feature set to obtain the second retained feature set; The feature to be deleted is moved from the first set of retained features to the first set of deleted features to obtain the second set of deleted features.

4. The method according to claim 3, characterized in that, Before moving the feature to be deleted from the first set of retained features to the first set of deleted features to obtain the second set of deleted features, the method further includes: Delete all features in the first set of deleted features.

5. The method according to any one of claims 1-4, characterized in that, The process of obtaining a filtered feature set from the set of features to be filtered based on feature interaction contribution information includes: With the goal of maximizing feature interaction sub-information, a number of features to be retained and a number of features to be deleted are selected from the set of features to be selected, resulting in a set of selected features. The feature interaction sub-information represents the total contribution of at least two features in the set of features to be selected after deleting the deleted features to the trained target network.

6. The method according to any one of claims 1-5, characterized in that, The feature interaction contribution information is obtained based on the multi-order gradients of at least two features in the set of features to be screened, using the loss function of the trained target network.

7. The method according to claim 6, characterized in that, The feature interaction contribution information is obtained based on the second gradient of the loss function of the trained target network with respect to every two features in the feature set to be screened. The feature interaction contribution information represents the contribution matrix of every two features in the feature set to be screened to the trained target network. The process of obtaining a filtered feature set from the set of features to be filtered based on feature interaction contribution information includes: With the goal of maximizing the sum of the contribution matrix of the feature interaction contribution information after deleting the rows and columns corresponding to K features, the K features are selected from the several features to be selected, and the selected feature set is obtained, where K is a positive integer.

8. The method according to any one of claims 1-7, characterized in that, The process of obtaining the target feature set based on the filtered feature set includes: Determine whether the number of features in the filtered feature set is the target number; If the number of features in the filtered feature set is the target number, then the filtered feature set is determined as the target feature set; If the number of features in the filtered feature set is greater than the target number, then feature filtering operations are performed on several features in the filtered feature set to obtain the target feature set.

9. A feature screening device, characterized in that, include: The acquisition module is used to acquire training datasets, each training dataset including several training samples, and each training sample including feature values ​​of several features in the feature set to be screened. The training module is used to train the target network based on the subset of training data to obtain the trained target network; The feature filtering module is used to filter a set of features from the set of features to be filtered based on feature interaction contribution information, wherein the feature interaction contribution information represents the contribution of at least two features in the set of features to be filtered to the trained target network; and to obtain a target feature set based on the filtered feature set, wherein the number of features in the target feature set is a target number.

10. A computing device, characterized in that, include: At least one memory for storing programs; At least one processor is configured to execute a program stored in the memory; wherein, when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-8.

Images