Multi-data-set feature selection method and system based on multi-task evolutionary algorithm

Abstract

The invention discloses a multi-data-set feature selection method and system based on a multi-task evolutionary algorithm, and belongs to the technical field of feature selection. The method comprises the following steps: (1) cleaning and splitting a data set; (2) initializing an evolutionary algorithm; (3) selecting excellent individuals to form sub-populations, and generating filial generation individuals; (4) evaluating the sub-population model; (5) carrying out independent evolution on each sub-data set; (6) stopping checking by the algorithm; and (7) outputting a feature selection result and a regression model of each sub-data set. According to the feature selection method, feature selection can be carried out on the multiple data sets at the same time, a shared feature selection scheme transmitted among the different data sets is supported, collaborative feature selection is carried out on the multiple data sets, and independent evolution of the data sets for respective scenes is also supported. Therefore, the calculation overhead consumed in repeated operation is remarkably reduced, and the calculation efficiency of multi-data-set feature selection is remarkably improved.

Description

technical field [0001] The invention belongs to the technical field of feature selection, and in particular relates to a multi-data set feature selection method and system based on a multi-task evolutionary algorithm. Background technique [0002] With the rapid development of big data and artificial intelligence technology, the importance of mining useful information from massive data has become increasingly prominent. With the exponential growth of data in dimension and depth, the demand for computing resources of various machine learning algorithms has also increased significantly. Therefore, data dimensionality reduction has become one of the effective means to reduce the computational cost of machine learning and improve model accuracy. As one of the common dimensionality reduction methods, feature selection is an effective means to solve the curse of dimensionality, and has important theoretical and application value for classification and regression problems. Feature...

AI Technical Summary

Problems solved by technology

[0006] In order to overcome the problem that the existing feature selection technology based on evolutionary algorithm only supports the processing of a single data set, the computational cost of feature selection for multiple data sets and multi-models is high, and the feature selection of multiple data sets is not shared, the present invention proposes a A multi-dataset feature selection method based on a multi-task evolutionary algorithm. A core multi-task evolutionary module is designed so that the evolutionary algorithm can perform feature selection on multiple data sets at the same time. It not only supports the transfer of shared feature selection schemes between different data sets, It also supports independent evolution of each data set for each scenario

Images