Automatic feature online processing method and device for log type data, machine readable medium and equipment

Abstract

The invention discloses an automatic feature online processing method for log type data, which comprises the following steps: acquiring original log type data of a target object, and performing feature extraction on the original log type data to obtain one or more original feature columns; performing feature processing on the one or more original feature columns to obtain one or more target feature columns; configuring one or more constraint rules; configuring the one or more constraint rules into one or more composite conditions through a rule tree; and performing feature screening on the oneor more target feature columns by utilizing the one or more composite conditions. The device can replace an automatic feature processing product system of artificial feature engineering. According tothe invention, the efficiency of log type data artificial feature engineering can be improved, it is guaranteed that the processed features have interpretability, and the processed data can be directly used for model training.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to an automatic feature online processing method, device, machine-readable medium and equipment for log-type data. Background technique [0002] In the specific practical tasks of machine learning, it is very important to select a set of representative features for building models. In the real world, data is usually complex, redundant, and full of changes, and it is necessary to discover useful features from raw data. Manually selected features rely on manpower and professional knowledge, which is not conducive to promotion. Therefore, we need to use machines to learn and extract features to promote feature engineering work more quickly and effectively. [0003] Feature engineering refers to the use of a series of engineering methods to filter out better data features from raw data to improve the training effect of the model. There is a widely circulated saying in the indus...

AI Technical Summary

Problems solved by technology

[0005] 1. The data form cannot be modeled: very primitive log data is often obtained, that is, one entity (person / thing) corresponds to multiple records (behavior / event / flow, etc.) data, and model training cannot be performed directly, so it needs to pass Feature engineering processes log data into feature data for model training

[0006] 2. Time-consuming: artificial feature engineering relies on the analysis and observation of algorithm engineers, and the steps are lengthy and time-consuming

[0007] 3. Poor feature consistency: artificial feature engineering relies on the understanding and creativity of algorithm engineers on data, and the level of different algorithm engineers varies greatly, resulting in inconsistent results

[0008] 4. Poor code reusability: Artificial feature engineering depends on specific problems, and the code must be rewritten for each data set during each feature engineering operation before modeling, and the reusability is poor

Images