Training method and device, feature selection method and device, and electronic device

Abstract

The one or more embodiments of the specification provide a training method and device, a feature selection method and device, and an electronic device. The method comprises: for each local model in a plurality of local models, the local model determining an action according to an environment state of a current time step, and determining a reward value and an environment state of a next time step according to the environment state of the current time step and the action; for each local model in the plurality of local models, the local model generating a trajectory according to a trajectory parameter of each time step from a first time step to the current time step in response to the trajectory parameter of the current time step satisfying a preset condition, the trajectory parameter comprising the environment state of the current time step, the action, and the reward value, and the environment state of the next time step; and training parameters of a global model according to the trajectory generated by each local model, the local model and the global model being the same model.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and more particularly to a training method and apparatus, a feature selection method and apparatus, and an electronic device. Background Technology

[0002] With the continuous advancement of science and technology, artificial intelligence (AI) technology has achieved tremendous development, bringing convenience to various fields. Task models in AI technology can predict task metrics based on data, and the prediction accuracy of these models depends to a certain extent on the accuracy of the upstream features they rely on. Therefore, feature selection for task models is crucial. Feature selection is a classic problem in data mining. By removing irrelevant and redundant information from numerous features that can capture the characteristics of predictive metrics, data dimensionality is reduced, and the interference of bias and noise on prediction results is minimized. This prepares clean and easily understandable data for downstream predictive analysis, enhancing the interpretability and prediction accuracy of downstream task models.

[0003] However, the accuracy of feature selection in related technologies still needs to be improved, which in turn affects the prediction accuracy of the task model. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide a training method and apparatus, a feature selection method and apparatus, and an electronic device.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of this specification, a training method is proposed, the method comprising:

[0007] For each of the multiple local models, the local model determines an action based on the environmental state at the current time step, and determines a reward value and the environmental state at the next time step based on the environmental state at the current time step and the action. The environmental state includes the visible task corresponding to the local model, the selected features, and the current feature. The action includes selecting or abandoning.

[0008] For each of the multiple local models, the local model responds to the trajectory parameters of the current time step satisfying preset conditions and generates a trajectory based on the trajectory parameters of each time step from the first time step to the current time step. The trajectory parameters include the environmental state, action and reward value of the current time step, and the environmental state of the next time step.

[0009] Based on the trajectory generated by each local model, the parameters of the global model are trained. The local model and the global model are the same model. The global model is used for feature selection preprocessing for unknown tasks. The feature sets of the visible tasks and the unknown tasks are the same.

[0010] In one embodiment of this specification, the preset conditions include at least one of the following:

[0011] In the trajectory parameters of the current time step, the number of selected features in the environmental state of the current time step has reached a preset threshold, and the action of the current time step is selection;

[0012] In the trajectory parameters of the current time step, the feature currently in the environmental state of the current time step is the last feature in the feature set.

[0013] In one embodiment of this specification, training the parameters of the global model based on the trajectories generated by each local model includes:

[0014] For each of the multiple local models, a reward estimate for each time step is determined based on the trajectory generated by the local model, and a set of reward estimates is determined based on the reward estimate for each time step.

[0015] The parameters of the global model are trained based on the trajectory and reward estimates generated by each local model.

[0016] In one embodiment of this specification, training the parameters of the global model based on the trajectory and reward estimates generated by each local model includes:

[0017] For each of the plurality of local models, trajectory parameters for multiple time steps are selected from the trajectory generated by the local model, and return estimates for the multiple time steps are selected from the set of return estimates.

[0018] The parameters of the global model are trained based on the trajectory parameters and reward estimates for each local model at the multiple time steps.

[0019] In one embodiment of this specification, the method further includes:

[0020] For each of the multiple local models, the local model determines the environment state of the first time step based on the state tree of the visible task corresponding to the local model. The state tree is used to represent the environment state transition path represented by the trajectory generated by the visible task in at least one previous round. The state tree includes multiple nodes, each node representing an environment state. The environment state of the first time step includes the environment state represented by any node in the state tree.

[0021] In one embodiment of this specification, each node of the state tree has attribute information, wherein the attribute information includes access count and value estimate;

[0022] The local model determines the environmental state of the first time step based on the state tree of the visible task corresponding to the local model, including:

[0023] The local model starts from the root node in the state tree and determines the next node at each node based on the attribute information of multiple child nodes, thus obtaining the node trajectory from the root node to the end node.

[0024] The local model determines the environmental state of the first time step in the environmental state represented by each node of the node trajectory.

[0025] In one embodiment of this specification, after generating the trajectory based on the trajectory parameters of each time step from the first time step to the current time step, the method further includes:

[0026] For each of the multiple local models, the state tree of the visible task corresponding to the local model is processed according to the trajectory generated by the local model, including at least one of the following: adding a new node or modifying the attribute information of at least one node.

[0027] In one embodiment of this specification, the method further includes:

[0028] Obtain the feature subset corresponding to the trajectory generated by each of the multiple visible tasks in the most recent rounds;

[0029] Based on the feature subset of each of the plurality of visible tasks, determine the probability that each of the plurality of visible tasks will be selected by the local model;

[0030] Based on the probability that each visible task is selected by the local model among the plurality of visible tasks, the visible task corresponding to each local model among the plurality of visible tasks is determined.

[0031] In one embodiment of this specification, determining the probability that each visible task among the plurality of visible tasks is selected by the local model based on a feature subset of the plurality of visible tasks includes:

[0032] Based on the feature subset of each of the multiple visible tasks, the distance ratio and uncertainty of each visible task are determined accordingly;

[0033] Based on the distance ratio and uncertainty of each of the plurality of visible tasks, the probability of each visible task being selected by the local model is determined.

[0034] In one embodiment of this specification, obtaining the feature subset corresponding to the trajectory generated by each of the multiple visible tasks in the most recent rounds includes:

[0035] In response to the current round reaching a preset number of rounds since the last round in which the visible task corresponding to each local model was determined, a subset of features corresponding to the trajectory generated by each visible task in the most recent rounds is obtained.

[0036] In one embodiment of this specification, after generating the trajectory based on the trajectory parameters of each time step from the first time step to the current time step, the method further includes:

[0037] For each of the multiple local models, the trajectory generated by the local model is saved as the trajectory generated by the visible task corresponding to the local model in the current round.

[0038] According to a second aspect of one or more embodiments of this specification, a feature selection method is proposed, the method comprising:

[0039] A global model is used to select a subset of features for the training task from the full feature set, wherein the global model is pre-trained to convergence using the training method described in the first aspect, and the subset of features is used to train the training task.

[0040] According to a third aspect of one or more embodiments of this specification, a training apparatus is provided, the apparatus comprising:

[0041] The state transition module is used to control each local model in multiple local models to determine an action based on the environmental state of the current time step, and to determine a reward value and the environmental state of the next time step based on the environmental state of the current time step and the action. The environmental state includes the visible task corresponding to the local model, the selected features, and the current feature. The action includes selecting or abandoning.

[0042] The trajectory generation module is used to control the trajectory parameters of each local model in response to the current time step to meet preset conditions for each local model in multiple local models, and to generate a trajectory based on the trajectory parameters of each time step from the first time step to the current time step. The trajectory parameters include the environmental state, action and reward value of the current time step, and the environmental state of the next time step.

[0043] The training module is used to train the parameters of the global model based on the trajectory generated by each local model. The local model and the global model are the same model. The global model is used for feature selection preprocessing for unknown tasks. The feature sets of the visible tasks and the unknown tasks are the same.

[0044] In one embodiment of this specification, the preset conditions include at least one of the following:

[0045] In the trajectory parameters of the current time step, the number of selected features in the environmental state of the current time step has reached a preset threshold, and the action of the current time step is selection;

[0046] In the trajectory parameters of the current time step, the feature currently in the environmental state of the current time step is the last feature in the feature set.

[0047] In one embodiment of this specification, the training module is specifically used for:

[0048] For each of the multiple local models, a reward estimate for each time step is determined based on the trajectory generated by the local model, and a set of reward estimates is determined based on the reward estimate for each time step.

[0049] The parameters of the global model are trained based on the trajectory and reward estimates generated by each local model.

[0050] In one embodiment of this specification, when the training module trains the parameters of the global model based on the trajectory and reward estimation sets generated by each local model, it is specifically used for:

[0051] For each of the plurality of local models, trajectory parameters for multiple time steps are selected from the trajectory generated by the local model, and return estimates for the multiple time steps are selected from the set of return estimates.

[0052] The parameters of the global model are trained based on the trajectory parameters and reward estimates for each local model at the multiple time steps.

[0053] In one embodiment of this specification, the method further includes a state selection module for:

[0054] For each of the multiple local models, the local model is controlled to determine the environment state of the first time step based on the state tree of the visible task corresponding to the local model. The state tree is used to represent the environment state transition path represented by the trajectory generated by the visible task in at least one previous round. The state tree includes multiple nodes, each node representing an environment state. The environment state of the first time step includes the environment state represented by any node in the state tree.

[0055] In one embodiment of this specification, each node of the state tree has attribute information, wherein the attribute information includes access count and value estimate;

[0056] The state selection module is used to control the local model to determine the environment state of the first time step based on the state tree of the visible task corresponding to the local model. Specifically, it is used for:

[0057] The local model starts from the root node in the state tree and determines the next node at each node based on the attribute information of multiple child nodes, thus obtaining the node trajectory from the root node to the end node.

[0058] The local model determines the environmental state of the first time step in the environmental state represented by each node of the node trajectory.

[0059] In one embodiment of this specification, the method further includes a maintenance module for:

[0060] After generating the trajectory based on the trajectory parameters of each time step from the first time step to the current time step, for each of the multiple local models, the state tree of the visible task corresponding to the local model is processed according to the trajectory generated by the local model, including at least one of the following: adding a new node or modifying the attribute information of at least one node.

[0061] In one embodiment of this specification, the apparatus further includes a dispensing module for:

[0062] Obtain the feature subset corresponding to the trajectory generated by each of the multiple visible tasks in the most recent rounds;

[0063] Based on the feature subset of each of the plurality of visible tasks, determine the probability that each of the plurality of visible tasks will be selected by the local model;

[0064] Based on the probability that each visible task is selected by the local model among the plurality of visible tasks, the visible task corresponding to each local model among the plurality of visible tasks is determined.

[0065] In one embodiment of this specification, when the allocation module determines the probability of each visible task being selected by the local model based on a feature subset of the plurality of visible tasks, it is specifically used for:

[0066] Based on the feature subset of each of the multiple visible tasks, the distance ratio and uncertainty of each visible task are determined accordingly;

[0067] Based on the distance ratio and uncertainty of each of the plurality of visible tasks, the probability of each visible task being selected by the local model is determined.

[0068] In one embodiment of this specification, the allocation module, when obtaining the feature subset corresponding to the trajectory generated by each of multiple visible tasks in the most recent multiple rounds, is specifically used for:

[0069] In response to the current round reaching a preset number of rounds since the last round in which the visible task corresponding to each local model was determined, a subset of features corresponding to the trajectory generated by each visible task in the most recent rounds is obtained.

[0070] In one embodiment of this specification, the device further includes a storage module for:

[0071] After generating the trajectory based on the trajectory parameters of each time step from the first time step to the current time step, for each of the multiple local models, the trajectory generated by the local model is saved as the trajectory generated by the visible task corresponding to the local model in the current round.

[0072] According to a fourth aspect of one or more embodiments of this specification, a feature selection apparatus is provided, the apparatus comprising:

[0073] The feature selection module is used to select a subset of features for the training task from the full feature set using a global model, wherein the global model is pre-trained to convergence using the training method described in the first aspect, and the feature subset is used to train the training task.

[0074] According to a fifth aspect of one or more embodiments of this specification, an electronic device is provided, comprising:

[0075] processor;

[0076] Memory used to store processor-executable instructions;

[0077] The processor implements the method as described in the first aspect by running the executable instructions.

[0078] According to a sixth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0079] The technical solutions provided in the embodiments of this specification may include the following beneficial effects:

[0080] In the training method provided in the embodiments of this specification, for each of the multiple local models, the local model determines an action based on the environmental state of the current time step, and determines a reward value and the environmental state of the next time step based on the environmental state of the current time step and the action. Then, in response to the trajectory parameters of the current time step satisfying preset conditions, a trajectory is generated based on the trajectory parameters of each time step from the first time step to the current time step. Finally, the parameters of the global model are trained based on the trajectories generated by each local model. In other words, this method involves multiple local models and multiple visible tasks. Each local model is responsible for feature selection for one visible task, and a trajectory composed of trajectory parameters is formed during the feature selection process. Since the trajectory parameters contain a reward value, the trajectory can represent the feature selection experience of the visible task, or the evaluation of the feature selection result. Therefore, training the global model based on the feature selection experience of multiple visible tasks on the full feature set enables the global model to have a more accurate feature selection capability, that is, it can select a subset of features with high relevance to a specific task from the full feature set, thereby improving the prediction accuracy of the task model. Furthermore, this method involves multiple local models identical to the global model performing feature selection for different tasks in parallel, which improves the training efficiency of the global model. Attached Figure Description

[0081] Figure 1 This is a schematic diagram of the PA-FEAT framework provided in an exemplary embodiment.

[0082] Figure 2 This is a flowchart of a training method provided in an exemplary embodiment.

[0083] Figure 3 This is a flowchart of a training method provided in an exemplary embodiment.

[0084] Figure 4 This is a schematic diagram of the structure of a device provided in an exemplary embodiment.

[0085] Figure 5 This is a block diagram of a training apparatus provided in an exemplary embodiment. Detailed Implementation

[0086] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0087] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0088] With the continuous advancement of science and technology, artificial intelligence (AI) technology has achieved tremendous development, bringing convenience to various fields. Task models in AI technology can predict task metrics based on data, and the prediction accuracy of these models depends to a certain extent on the accuracy of the upstream features they rely on. Therefore, feature selection for task models is crucial. Feature selection is a classic problem in data mining. By removing irrelevant and redundant information from numerous features that can capture the characteristics of predictive metrics, data dimensionality is reduced, and the interference of bias and noise on prediction results is minimized. This prepares clean and easily understandable data for downstream predictive analysis, enhancing the interpretability and prediction accuracy of downstream task models. However, the accuracy of feature selection in related technologies still needs improvement, which in turn affects the prediction accuracy of task models. In particular, the accuracy of feature selection for structured data is poor, failing to meet the prediction needs of task models.

[0089] In real-world applications, structured data is ubiquitous, containing a wealth of knowledge that can support or guide data-driven decision-making. Mining and utilizing this knowledge can help identify potential risks and opportunities in advance, which is crucial for the operations of most businesses. Due to the prevalent correlations and dependencies between attributes in structured data, analyzing it using machine learning algorithms often requires considering the combined effects of multiple features, i.e., modeling the cross-features in the data. However, real-world applications often involve a large number of attributes, and modeling cross-features across all attributes typically requires significant computational overhead. Therefore, feature selection is an essential step in structured data mining. Feature selection can eliminate redundant and irrelevant attributes (features) in advance, thus selecting a well-performing subset of features for downstream tasks. However, current feature selection techniques for structured data not only need improvement in accuracy but also fail to meet timeliness requirements.

[0090] Based on this, in a first aspect, at least one embodiment of this specification provides a training method for training a feature selection model. After training, this feature selection model can select a subset of features with high relevance to a specific task from the entire feature set. The task model for that task can use this feature selection model to select a subset of features from the entire feature set before prediction, either during the training phase or the indicator prediction phase, thereby improving training efficiency and effectiveness, as well as the accuracy of indicator prediction. For example, the feature selection model can be a DNN model, etc.

[0091] This method provides, for example Figure 1 The diagram illustrates PA-FEAT, a multi-task deep reinforcement learning framework with progress-aware capabilities, used to train a feature selection model. First, PA-FEAT includes the FEAT multi-task deep reinforcement learning framework, dedicated to enhancing the feature selection process for new tasks sharing the same feature space, reducing latency while maintaining accuracy. Furthermore, based on this FEAT framework, an Inter-task Scheduler (ITS) and an Inter-task Explorer (ITE) are proposed. ITS balances limited computing resources and task resource demands through adaptive inter-task resource scheduling, while ITE enhances the utilization of past experience within each task using an experience tree structure to improve exploration. These two enhancement techniques fully leverage various progress-related information generated during knowledge summarization, enabling the FEAT framework to possess progress-aware capabilities.

[0092] The fundamental focus of PA-FEAT is to update the global model (i.e., the global agent of the global agent) using experience gained from interactions between local models (e.g., the feature selection model of the local agent, which is a local agent synchronized from the global agent to each computing resource) and each environment corresponding to the visible task. Specifically, after initializing the global model, PA-FEAT invokes the inter-task resource scheduler to assign the probability of each task being selected by computing resources based on the progress information of each visible task, thereby achieving on-demand allocation. That is, each resource is allocated to a visible task through the inter-task resource scheduler. Within each computing resource, an environment is first initialized based on the visibility allocated to that resource. The interaction between the local model and this initialized environment continues. At the start of the interaction, each resource invokes the in-task search enhancer corresponding to the visible task to obtain a custom initial state. The local model begins its exploration from this state to deepen its understanding of the visible task corresponding to the resource. During the interaction, the local model takes an action each time from the initial state specified by the in-task search enhancer, then the environment undergoes a state transition and provides corresponding feedback to the agent. This process is repeated, and the experience gained during this interaction is recorded in the experience replay buffer corresponding to the visible task. PA-FEAT periodically samples a batch of samples from the experience replay buffer corresponding to each visible task and updates the global model accordingly. This process is repeated until the global model achieves good training results. After obtaining a high-performance global model, it can respond quickly to unseen tasks and rapidly provide feature selection results for those tasks.

[0093] Next, we will combine the appendix Figure 2 The process of this method is described in detail, with appendix. Figure 2 An exemplary flowchart of the method is shown, including steps S201 to S203. It is understood that... Figure 2 The illustrated process represents only one round of the training process for the feature selection model using this method; repeated steps are not required. Figure 2 The training of the feature selection model by this method is completed only when the convergence condition is met (such as the number of rounds reaching a preset threshold, the loss value of the feature selection model reaching a preset requirement, etc.).

[0094] In step S201, for each of the multiple local models, the local model determines an action based on the environmental state of the current time step, and determines a reward value and the environmental state of the next time step based on the environmental state of the current time step and the action. The environmental state includes the visible task corresponding to the local model, the selected features, and the current feature. The action includes selecting or abandoning.

[0095] In this model, the feature selection model of the global agent serves as the global model. This model is synchronized to each computing resource to form a local model. Therefore, each computing resource has its own local model, resulting in multiple local models. Each computing resource is responsible for feature selection for one of the multiple visible tasks. All visible tasks share the same feature space; for example, all visible tasks select a subset of features from the same full feature set as training samples.

[0096] The interaction between the local model and its environment within the computing resources refers to traversing the features in the entire feature set (specifically, traversing from the current feature in the environment state of the first time step to subsequent features), and making a selection or abandonment action when traversing each feature, as well as determining the reward value for the action. This step is a time step in which the local model interacts with its environment within the computing resources, that is, the process of obtaining the environment state of the next time step from the environment state of the current time step.

[0097] The environment state can be represented by a vector. The first third of the elements in the vector represent the visible task corresponding to the local model (i.e., the visible task handled by the computing resources where the local model resides). For example, the first third of the elements are the correlation coefficients between each feature in the feature set and the visible task. The middle third of the vector represents the selected features. For example, the middle third of the elements are the encoding of whether each feature in the feature set is selected (e.g., if a feature is selected, its corresponding element is 1; if a feature is not selected, its corresponding element is 0). The last third of the vector represents the current feature. For example, the last third of the elements are the encoding of whether each feature in the feature set is the current feature (e.g., if a feature is the current feature, its corresponding element is 1; if a feature is not the current feature, its corresponding element is 0).

[0098] For example, taking any local model as an example, it corresponds to a visible task u among multiple visible tasks; in this step, the local model can display the environmental state at the current time step. Input the value function V respectively θ and the dominant function A θ Generate value estimates And action advantage estimation This leads to the probability distribution of the action. And determine the action based on this distribution. The specific process is as follows:

[0099]

[0100]

[0101]

[0102] In the above formula, θ represents all learnable parameters, and f V (·) and f A (·) represents a deep neural network, A θ V represents the advantage function. θ Represents the value function, f E The function f represents the scalar broadcast as a vector. N Zero centering refers to the operation of subtracting the mean from each element in a vector.

[0103] For example, continuing with this local model, the actions obtained in the above example can be performed. Earn reward points and the environment state at the next time step.

[0104] Understandably, this step can describe the environmental state at the current time step. action Reward Value and the environment state at the next time step. The trajectory parameters that make up the current time step.

[0105] In step S202, for each of the multiple local models, the local model generates a trajectory based on the trajectory parameters of each time step from the first time step to the current time step, in response to the trajectory parameters of the current time step satisfying a preset condition. The trajectory parameters include the environmental state, action, and reward value of the current time step, as well as the environmental state of the next time step.

[0106] As can be seen from the execution process of step S201 above, trajectory parameters are obtained after the action of each time step is completed; the trajectory parameters of each time step represent the process (i.e., action) and evaluation (i.e., reward value) of the environmental state switching from the current time step to the environmental state of the next time step.

[0107] For example, the preset conditions include at least one of the following:

[0108] In the trajectory parameters of the current time step, the number of selected features in the environment at the current time step has reached a preset threshold, and the action at the current time step is selection. This condition indicates that the number of selected features meets the quantity requirement. For example, if the preset data volume threshold = quantity requirement - 1, then this condition can indicate that after the action at the current time step is executed, the number of features in the feature subset of this round has reached the quantity requirement.

[0109] In the trajectory parameters at the current time step, the current feature in the environmental state at the current time step is the last feature in the feature set. This condition indicates that the local model has already traversed the feature set to the last feature, meaning that the feature subset cannot select any more features.

[0110] It is understandable that step S201 will determine the environmental state at the current time step. action Reward Value and the environment state at the next time step. The trajectory parameters that make up the current time step can be used to obtain the trajectory in this step.

[0111] In the trajectory obtained in this step, each trajectory parameter represents the transition of the environment state from the current time step to the next time step. Therefore, the trajectory obtained in this step can represent the environment state transition path in this round.

[0112] Each visible task has a sample library, and the trajectory obtained in this step can be saved to the sample library of the visible task corresponding to the local model.

[0113] In step S203, the parameters of the global model are trained based on the trajectory generated by each local model. The local model and the global model are the same model. The global model is used for feature selection preprocessing for unknown tasks. The feature sets of the visible task and the unknown task are the same.

[0114] For example, this step can be performed as follows:

[0115] First, for each of the multiple local models, a reward estimate for each time step is determined based on the trajectory generated by the local model, and a set of reward estimates is determined based on the reward estimates for each time step. For example, based on the trajectory... Calculate the cumulative return estimate for a given time step i Obtain the cumulative return estimate set Where γ∈[0,1] is the discount factor.

[0116] Next, the parameters of the global model are trained based on the trajectory and reward estimation sets generated by each local model. For example, firstly, for each of the multiple local models, trajectory parameters for multiple time steps are selected from the trajectory generated by the local model, and reward estimates for the multiple time steps are selected from the reward estimation set; then, the parameters of the global model are trained based on the trajectory parameters and reward estimates for the multiple time steps corresponding to each local model.

[0117] For example, when training the parameters of a global model using trajectory parameters and reward estimates from multiple time steps, batch learning can be used from sample databases corresponding to each task. Collect M samples, based on the loss function L in DuelingDQN t The parameter θ is updated using the following formula:

[0118]

[0119]

[0120]

[0121] Where Q(s, a; θ) is the value function approximated by a deep network (i.e., a global model), which consists of the state value estimate V(s; θ, β) and the zero-center-normalized advantage function A(s, a; θ, α), which respectively estimate the value of the state and the advantage of the action; θ t The target network Q(s, a; θ) represents... t The parameters of θ - It is a frozen parameter when updating the target network parameters using gradient descent; α and β represent the parameters of the value function and the advantage function, respectively.

[0122] After training the global model in this step, the data in the sample library for each visible task can be cleared to reduce the data's footprint on the cache. Furthermore, the training results of the global model's parameters can be synchronized to each local model. This ensures that the local models in each round are based on the training results of the global model from the previous round, thereby improving the accuracy of feature selection experience for each visible task.

[0123] In the training method provided in the embodiments of this specification, for each of the multiple local models, the local model determines an action based on the environmental state of the current time step, and determines a reward value and the environmental state of the next time step based on the environmental state of the current time step and the action. Then, in response to the trajectory parameters of the current time step satisfying preset conditions, a trajectory is generated based on the trajectory parameters of each time step from the first time step to the current time step. Finally, the parameters of the global model are trained based on the trajectories generated by each local model. In other words, this method involves multiple local models and multiple visible tasks. Each local model is responsible for feature selection for one visible task, and a trajectory composed of trajectory parameters is formed during the feature selection process. Since the trajectory parameters contain a reward value, the trajectory can represent the feature selection experience of the visible task, or the evaluation of the feature selection result. Therefore, training the global model based on the feature selection experience of multiple visible tasks on the full feature set enables the global model to have a more accurate feature selection capability, that is, it can select a subset of features with high relevance to a specific task from the full feature set, thereby improving the prediction accuracy of the task model. Furthermore, this method involves multiple local models identical to the global model performing feature selection for different tasks in parallel, which improves the training efficiency of the global model.

[0124] In the training method provided in the above embodiments, step S201 traverses from the current feature in the environmental state of the first time step to the subsequent features, and makes a selection or abandonment action when traversing each feature, and determines the reward value of each action. The environmental state of the first time step can be determined in at least one of the following two ways.

[0125] In the first approach, the current feature in the environment state of the first time step can be the first feature in the feature set. That is, the environment state of the first time step is the visible task corresponding to the local model, the selected features are 0, and the current feature is the first feature in the feature set. In other words, step S201 starts from the first feature in the feature set and traverses to the subsequent features.

[0126] In the second approach, a state tree is stored in the experience replay buffer corresponding to each visible task. This state tree is used to characterize the environmental state transition path represented by the trajectory generated by the visible task in at least one previous round. The state tree includes multiple nodes, each node represents an environmental state, and each node has attribute information, including the number of visits and value estimation. The environmental state of the first time step includes the environmental state represented by any node in the state tree (the specific determination method will be introduced below). For example, the in-task search enhancer reads the environmental state transition paths related to the visible task from the experience replay buffer and uses these past paths to construct the state tree E-Tree corresponding to the task. That is, when a new state appears in the path, the E-Tree expands to a new leaf node, and when an old state appears in the path, the corresponding node in the E-Tree increases its access count by one. For example, when the local model performs action a, causing the environmental state to transition from state s to a new state s′ that has never been visited, the node representing the new state s′ will be added to the E-Tree as a child node of the node corresponding to state s. When the local model performs action a, causing the environmental state to transition from state s to a previously visited state s″, the node representing state ss will increase its access count by one.

[0127] Based on the state tree within the experience replay buffer of the visible task, for each of the multiple local models (i.e., the ITE within the computing resources), the environment state of the first time step can be determined according to the state tree of the visible task corresponding to the local model in the following manner:

[0128] First, the local model starts from the root node in the state tree. At each node, it determines the next node based on the attribute information of multiple child nodes, thus obtaining the node trajectory from the root node to the last node. Optionally, at each node, the ρ value of each child node is determined according to the following formula, and the child node with the largest ρ value is determined as the next node:

[0129]

[0130] In the formula, T F T represents the number of times node F has been visited. F，F′ This indicates the number of times child node F′ was selected. To represent the value estimate of node F′ in the E-Tree, we will It is defined as the sum of the performance of the feature subsets mapped from all trajectories corresponding to the state, where the performance can be the prediction accuracy of the classifier trained using the feature subsets on the test set.

[0131] Next, the local model determines the environment state of the first time step from the environment states represented by each node of the node trajectory. Optionally, from the environment states represented by each node of the node trajectory, the environment state represented by a certain node is randomly determined as the environment state of a certain time step.

[0132] Understandably, if the environment state of the first time step is determined in this way, then after executing the appendix in each round... Figure 2 Following step S202 in the training method shown, for each of the multiple local models, the state tree of the visible task corresponding to the local model is processed according to the trajectory generated by the local model, performing at least one of the following operations: adding a new node, modifying the attribute information of at least one node, that is, adding the environmental state transition trajectory passed in this round to the state tree. For example, in the environmental state transition path represented by the trajectory generated by the local model, if a certain environmental state is a new state that has not appeared in the state tree, then a corresponding new node can be added to the state tree according to the previous environmental state of that environmental state. If a certain environmental state is a state that has appeared in the state tree in the environmental state transition path represented by the trajectory generated by the local model, then the corresponding node in the state tree can be modified, for example, by increasing the number of visits to the environmental state in that node.

[0133] The two methods described above can also be used in combination. For example, the first method can be used to determine the environmental state of the first time step in the first N rounds, and after the first N rounds are completed, the environmental state transition path in the trajectory generated in each round can be used to generate a state tree. In each round after N+1, the environmental state of the first time step can be determined using the state tree, and the trajectory can be used to maintain the state tree after the trajectory is generated. That is, at least one of the following processes can be performed: adding a new node or modifying the attribute information of at least one node.

[0134] In some embodiments of this disclosure, a preset round number threshold (which can be 1, 2, 3, etc.) can be pre-set, and computing resources can be allocated to the visible task at intervals of the preset round number threshold. For example, in the appendix... Figure 1 Before the training method shown is executed, ITS can allocate visible tasks to each computing resource across multiple visible tasks. The local model within each computing resource corresponds to the visible task that the computing resource is responsible for. It is understandable that when the number of computing resources is greater than the number of visible tasks, each visible task can receive computing resources during allocation, and different computing resources can be responsible for the same visible task; when the number of computing resources equals the number of visible tasks, a one-to-one correspondence between visible tasks and computing resources can be established during allocation; and when the number of computing resources is less than the number of visible tasks, as many visible tasks as possible can receive computing resources during allocation.

[0135] Each visible task's experience replay buffer can store at least a subset of features corresponding to the trajectories generated in the last n rounds (e.g., n=10). This subset of features can be a set of selected features from the environment state at the last time step of the trajectory. Therefore, computational resources can be allocated to visible tasks in the following manner:

[0136] First, obtain the feature subset corresponding to the trajectory generated by each of the multiple visible tasks in the most recent rounds. For example, for the k-th visible task, obtain the feature subset corresponding to the trajectory generated in the most recent n rounds.

[0137] Understandably, in response to the current round reaching a preset round number threshold since the last round that determined the visible task corresponding to each local model, the feature subset corresponding to the trajectory generated by each of the multiple visible tasks in the most recent rounds is obtained, i.e., this step is performed every round at a preset round number threshold interval.

[0138] Next, based on the feature subset of each of the plurality of visible tasks, the probability of each visible task being selected by the local model is determined.

[0139] For example, firstly, based on the feature subset of each visible task among the multiple visible tasks, the distance ratio and uncertainty of each visible task are determined accordingly; then, based on the distance ratio and uncertainty of each visible task among the multiple visible tasks, the probability of each visible task being selected by the local model is determined.

[0140] For example, the distance ratio of the kth visible task can be determined as follows:

[0141]

[0142]

[0143] In the formula, This represents the performance of a classifier trained on the entire feature set on the visible task k, where the performance can be represented by the prediction accuracy of the classifier trained on the entire feature set on the test set. Representative Each feature subset in The average performance of the trained classifier, where performance can be defined as the prediction accuracy of the classifier trained using a subset of features on the test set.

[0144] For example, the uncertainty of the k-th visible task can be determined as follows:

[0145]

[0146] In the formula, p(i) represents the i-th feature being set. The probability of selecting a feature subset in the set, where m represents the number of features in the full feature set.

[0147] For example, the probability of each visible task being selected by the local model among the plurality of visible tasks is determined in the following manner:

[0148]

[0149] D = [d1, d2, ..., d] n ]

[0150]

[0151] First, the scores corresponding to the two evaluation metrics are normalized. Then, the two normalized scores for each task are summed. Finally, the sums of the scores for n tasks are organized into a vector of length n. This vector is then input into the softmax function to obtain the probability that each task will be selected by the computational resource. This represents the probability that visible task i is selected by computational resources, or the probability that visible task i is selected by the local model.

[0152] Finally, based on the probability that each visible task is selected by the local model among the plurality of visible tasks, the visible task corresponding to each local model among the plurality of local models is determined among the plurality of visible tasks.

[0153] For example, ITS is based on this probability sequence To achieve on-demand allocation of computing resources, for each computing resource, ITS performs a random sampling from the visible task sequence, with the probability table as follows: The visible tasks obtained from the sampling can be allocated corresponding computing resources. Optionally, the above allocation process can be implemented using the numpy.random.choice function.

[0154] Understandably, after each round, the appendix is ​​executed. Figure 2 Following step S202 in the training method shown, for each of the multiple local models, the trajectory generated by that local model can be saved as the trajectory generated by the visible task corresponding to that local model in the current round. For example, the trajectory can be saved to the experience replay buffer of the visible task corresponding to the local model. Alternatively, after saving the trajectory generated in the current round, the trajectory from the earliest saved round can be deleted to keep the number of saved trajectories consistent and minimize memory usage.

[0155] Please refer to the appendix. Figure 3 The flowchart of the training method obtained by combining the above embodiments is shown as an example. The method includes the following steps:

[0156] Step S1: Initialize the agent parameters θ and the sample database for each task.

[0157] Step S2: Determine if the number of rounds is greater than N.

[0158] Step S3: Use ITS to obtain the task u corresponding to the computing resources.

[0159] Step S4: Clear the sample database for each task

[0160] Step S5: Determine if time step t is greater than T-1.

[0161] Step S6: Determine if time step t is equal to 0.

[0162] Step S7: The agent obtains the state.

[0163] Step S8: The agent obtains the state.

[0164] Step S9: From state or Using the value function V respectively θ and the dominant function A θ Obtain the probability distribution of actions

[0165] Step S10: Agents in each environment execute actions. Earn reward points And the next local observation Proceed to the next time step.

[0166] Step S11: Each agent generates a trajectory Cumulative Reporting Estimates

[0167] Step S12: Use the trajectory Update the E-Tree corresponding to task u that is visible in ITE.

[0168] Step S13: Each agent bases its actions on the sample database. And the joint loss function L to update the parameter θ u Now, let's move on to the next round.

[0169] According to a second aspect of one or more embodiments of this specification, a feature selection method is proposed, the method comprising:

[0170] A global model is used to select a subset of features for the training task from the full feature set, wherein the global model is pre-trained to convergence using the training method described in the first aspect, and the subset of features is used to train the training task.

[0171] For example, the global model iterates through the features in the full feature set starting with the first feature, and makes a selection or rejection action when it comes to each feature until it reaches the last feature or the number of selected features reaches the requirement, thus obtaining a feature subset.

[0172] Figure 4 This is a schematic structural diagram of a device provided in an exemplary embodiment. Please refer to... Figure 4 At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, memory 408, and non-volatile memory 410, and may also include other hardware required for tasks. One or more embodiments of this specification can be implemented in software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into memory 408 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0173] Please refer to Figure 5 The training device can be applied to, for example Figure 4 The device shown is used to implement the technical solution of this specification. The device includes:

[0174] The state transition module 501 is used to control each local model in a plurality of local models to determine an action based on the environmental state of the current time step, and to determine a reward value and the environmental state of the next time step based on the environmental state of the current time step and the action. The environmental state includes the visible task corresponding to the local model, the selected features and the current feature, and the action includes selecting or abandoning.

[0175] The trajectory generation module 502 is used to control each local model in a plurality of local models to meet preset conditions in response to the trajectory parameters of the local model at the current time step, and to generate a trajectory based on the trajectory parameters of each time step from the first time step to the current time step. The trajectory parameters include the environmental state, action and reward value of the current time step, and the environmental state of the next time step.

[0176] Training module 503 is used to train the parameters of the global model based on the trajectory generated by each local model, wherein the local model and the global model are the same model, and the global model is used for feature selection preprocessing for unknown tasks, wherein the feature sets of the visible task and the unknown task are the same.

[0177] In one embodiment of this specification, the preset conditions include at least one of the following:

[0178] In the trajectory parameters of the current time step, the number of selected features in the environmental state of the current time step has reached a preset threshold, and the action of the current time step is selection;

[0179] In the trajectory parameters of the current time step, the feature currently in the environmental state of the current time step is the last feature in the feature set.

[0180] In one embodiment of this specification, the training module is specifically used for:

[0181] For each of the multiple local models, a reward estimate for each time step is determined based on the trajectory generated by the local model, and a set of reward estimates is determined based on the reward estimate for each time step.

[0182] The parameters of the global model are trained based on the trajectory and reward estimates generated by each local model.

[0183] In one embodiment of this specification, when the training module trains the parameters of the global model based on the trajectory and reward estimation sets generated by each local model, it is specifically used for:

[0184] For each of the plurality of local models, trajectory parameters for multiple time steps are selected from the trajectory generated by the local model, and return estimates for the multiple time steps are selected from the set of return estimates.

[0185] The parameters of the global model are trained based on the trajectory parameters and reward estimates for each local model at the multiple time steps.

[0186] In one embodiment of this specification, the method further includes a state selection module for:

[0187] For each of the multiple local models, the local model is controlled to determine the environment state of the first time step based on the state tree of the visible task corresponding to the local model. The state tree is used to represent the environment state transition path represented by the trajectory generated by the visible task in at least one previous round. The state tree includes multiple nodes, each node representing an environment state. The environment state of the first time step includes the environment state represented by any node in the state tree.

[0188] In one embodiment of this specification, each node of the state tree has attribute information, wherein the attribute information includes access count and value estimate;

[0189] The state selection module is used to control the local model to determine the environment state of the first time step based on the state tree of the visible task corresponding to the local model. Specifically, it is used for:

[0190] The local model starts from the root node in the state tree and determines the next node at each node based on the attribute information of multiple child nodes, thus obtaining the node trajectory from the root node to the end node.

[0191] The local model determines the environmental state of the first time step in the environmental state represented by each node of the node trajectory.

[0192] In one embodiment of this specification, the method further includes a maintenance module for:

[0193] After generating the trajectory based on the trajectory parameters of each time step from the first time step to the current time step, for each of the multiple local models, the state tree of the visible task corresponding to the local model is processed according to the trajectory generated by the local model, including at least one of the following: adding a new node or modifying the attribute information of at least one node.

[0194] In one embodiment of this specification, the apparatus further includes a dispensing module for:

[0195] Obtain the feature subset corresponding to the trajectory generated by each of the multiple visible tasks in the most recent rounds;

[0196] Based on the feature subset of each of the plurality of visible tasks, determine the probability that each of the plurality of visible tasks will be selected by the local model;

[0197] Based on the probability that each visible task is selected by the local model among the plurality of visible tasks, the visible task corresponding to each local model among the plurality of visible tasks is determined.

[0198] In one embodiment of this specification, when the allocation module determines the probability of each visible task being selected by the local model based on a feature subset of the plurality of visible tasks, it is specifically used for:

[0199] Based on the feature subset of each of the multiple visible tasks, the distance ratio and uncertainty of each visible task are determined accordingly;

[0200] Based on the distance ratio and uncertainty of each of the plurality of visible tasks, the probability of each visible task being selected by the local model is determined.

[0201] In one embodiment of this specification, the allocation module, when obtaining the feature subset corresponding to the trajectory generated by each of multiple visible tasks in the most recent multiple rounds, is specifically used for:

[0202] In response to the current round reaching a preset number of rounds since the last round in which the visible task corresponding to each local model was determined, a subset of features corresponding to the trajectory generated by each visible task in the most recent rounds is obtained.

[0203] In one embodiment of this specification, the device further includes a storage module for:

[0204] After generating the trajectory based on the trajectory parameters of each time step from the first time step to the current time step, for each of the multiple local models, the trajectory generated by the local model is saved as the trajectory generated by the visible task corresponding to the local model in the current round.

[0205] Feature selection devices can be applied to, for example Figure 4 The device shown is used to implement the technical solution of this specification. The device includes:

[0206] The feature selection module is used to select a subset of features for the training task from the full feature set using a global model, wherein the global model is pre-trained to convergence using the training method described in the first aspect, and the feature subset is used to train the training task.

[0207] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0208] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0209] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0210] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0211] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0212] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0213] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” used in one or more embodiments of this specification and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0214] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0215] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0216] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A training method, comprising: for each of a plurality of local models, determining, by the local model, an action according to an environment state of a current time step, and determining, by the local model, a reward value and an environment state of a next time step according to the environment state of the current time step and the action, wherein the environment state comprises a visible task corresponding to the local model, a selected feature, and a current feature, and the action comprises selection or abandonment; for each of the plurality of local models, generating, by the local model, a trajectory according to a trajectory parameter of each of a first time step to the current time step in response to the trajectory parameter of the current time step satisfying a preset condition, wherein the trajectory parameter comprises the environment state of the current time step, the action, and the reward value, and the environment state of the next time step; training, according to the trajectory generated by each of the local models, a parameter of a global model, wherein the local model and the global model are the same model, and the global model is used for feature selection preprocessing of an unknown task, and features of the visible task and the unknown task are the same.

2. The training method of claim 1, wherein the preset condition comprises at least one of the following: in the trajectory parameter of the current time step, the selected feature in the environment state of the current time step reaches a preset quantity threshold, and the action of the current time step is selection; in the trajectory parameter of the current time step, the current feature in the environment state of the current time step is the last feature in the feature set.

3. The training method of claim 1, wherein the training, according to the trajectory generated by each of the local models, the parameter of the global model, comprises: for each of the plurality of local models, determining, according to the trajectory generated by the local model, a return estimate of each time step, and determining, according to the return estimate of each time step, a return estimate set; training, according to the trajectory generated by each of the local models and the return estimate set, the parameter of the global model.

4. The training method of claim 3, wherein the training, according to the trajectory generated by each of the local models and the return estimate set, the parameter of the global model, comprises: for each of the plurality of local models, selecting, in the trajectory generated by the local model, a trajectory parameter of a plurality of time steps, and selecting, in the return estimate set, a return estimate of the plurality of time steps; training, according to the trajectory parameter and the return estimate of the plurality of time steps corresponding to each of the local models, the parameter of the global model.

5. The training method of claim 1, further comprising: for each of the plurality of local models, determining, by the local model, an environment state of a first time step according to a state tree of the visible task corresponding to the local model, wherein the state tree is used to represent an environment state transition path represented by a trajectory generated in a previous at least one round of the visible task, and the state tree comprises a plurality of nodes, each node representing an environment state, and the environment state of the first time step comprises an environment state represented by any node in the state tree.

6. The training method of claim 5, each node of the state tree having attribute information, wherein, the attribute information comprises access times and value estimates. The local model determines the environment state of the first time step according to a state tree of a visible task corresponding to the local model, including: The local model starts from a root node in the state tree, and determines a next node according to attribute information of a plurality of child nodes at each node to obtain a node track from the root node to a terminal node; The local model determines the environment state of the first time step in the environment state represented by each node of the node track.

7. The training method of claim 6, after the trajectory is generated according to the trajectory parameters of each time step from the first time step to the current time step, the method further comprises: For each of the plurality of local models, at least one of the following is performed on the state tree of the visible task corresponding to the local model according to the trajectory generated by the local model: adding a new node, and modifying attribute information of at least one node.

8. The training method of claim 1, the method further comprises: obtaining a feature subset corresponding to a trajectory generated in a plurality of recent episodes for each of a plurality of visible tasks; determining a probability that each of the plurality of visible tasks is selected by a local model according to the feature subset of each of the plurality of visible tasks; determining the visible task corresponding to each of the plurality of local models among the plurality of visible tasks according to the probability that each of the plurality of visible tasks is selected by a local model.

9. The training method of claim 8, the determining of the probability that each of the plurality of visible tasks is selected by a local model according to the feature subset of the plurality of visible tasks comprises: corresponding determination of a distance ratio and uncertainty of each of the plurality of visible tasks according to the feature subset of each of the plurality of visible tasks; determination of the probability that each of the plurality of visible tasks is selected by a local model according to the distance ratio and uncertainty of each of the plurality of visible tasks.

10. The training method of claim 8, the obtaining of the feature subset corresponding to the trajectory generated in a plurality of recent episodes for each of a plurality of visible tasks comprises: in response to a distance between a current episode and an episode in which each local model corresponding to a visible task is last determined reaching a preset episode quantity threshold, obtaining the feature subset corresponding to the trajectory generated in a plurality of recent episodes for each of a plurality of visible tasks.

11. The training method of claim 8, after the trajectory is generated according to the trajectory parameters of each time step from the first time step to the current time step, the method further comprises: for each of the plurality of local models, saving the trajectory generated by the local model as a trajectory generated in a current episode for the visible task corresponding to the local model.

12. A feature selection method, the method comprising: selecting a feature subset for a to-be-trained task in a feature set using a global model, wherein the global model is pre-trained to convergence using the training method of any one of claims 1 to 11, and the feature subset is used to train the to-be-trained task.

13. A training device, the device comprising: a state transition module configured to, for each of the plurality of local models, determine an action based on an environment state of a current time step, and determine a reward value and an environment state of a next time step based on the environment state of the current time step and the action, wherein the environment state comprises a visible task corresponding to the local model, a selected feature, and a current feature, and the action comprises a selection or a discard; a trajectory generation module configured to, for each of the plurality of local models, generate a trajectory based on trajectory parameters of each of a first time step to a current time step in response to the trajectory parameters of the current time step satisfying a preset condition, wherein the trajectory parameters comprise the environment state of the current time step, the action, and the reward value, and the environment state of the next time step; a training module configured to train parameters of a global model based on the trajectory generated by each of the local models, wherein the local models and the global model are the same model, and the global model is used for feature selection preprocessing of an unknown task, and features of the visible task and the unknown task are the same.

14. A feature selection device, the device comprising: a feature selection module configured to select a feature subset in a feature universe for a to-be-trained task using a global model, wherein the global model is trained to convergence using the training method of any one of claims 1 to 11, and the feature subset is used to train the to-be-trained task.

15. An electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of any one of claims 1 to 12 by executing the executable instructions.

16. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method of any one of claims 1 to 12.

Images