Model detection method based on feature selection

Abstract

The invention discloses a model detection method based on feature selection, and the method comprises the steps: cutting an original data set; training a model through employing a training set; predicting the training set and a verification set through employing the model, and obtaining a prediction error; deleting features, obtaining a new training set, predicting the new training set through the model, and obtaining a prediction error; assigning j to (j+1), returning to the last step, and executing the next step till the value is C; calculating prediction error distances between the new training set and the training set; ordering the distances, and searching features corresponding to G minimum distances; storing sequence numbers of the features into a feature deleting feature sequence, and deleting features in the training set and a verification set; assigning C to (C-G), returning to a second step, and executing the next step till C is not greater than G; obtaining a sequence number K according to the prediction error, and deleting former (K-1) features from the training set and a testing set; training the new model through employing the training set with the features being deleted, predicting the testing set with the features being deleted through employing the new model, and obtaining the prediction error.

Description

technical field [0001] The invention relates to technical fields such as data mining, machine learning, big data, cloud computing, and the Internet, and in particular to a model detection method based on feature selection. Background technique [0002] With the rapid development of the Internet, massive amounts of data have been accumulated, which has also brought many problems to data analysis and data mining: [0003] Large amount of data and high dimensionality: a data set consists of the number of instances P and the number of features N, and the combination of the two brings a huge amount of calculation to the algorithm; [0004] Frequent data changes: Rapid changes on the Internet also produce changing data, which requires regenerating the data model; [0005] Noisy data and missing data: Internet data lacks strict conventions, the data is uneven, some algorithms are sensitive to noisy data, and it is difficult to obtain a stable data model. Contents of the inventio...

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved by the present invention is to provide a model detection method based on feature selection in order to overcome the defects of large amount of Internet data calculation, frequent data changes and difficulty in obtaining a stable data model in the prior art

Images