Feature data coding method and device

Abstract

The invention provides a feature data coding method. The method comprises the following steps: a, converting a data table stored in a row form into a coding mode stored in a column form by taking a hierarchy as a corresponding relation according to a high-low sequence of each node hierarchy, wherein the column form taking the hierarchy as the corresponding relationship at least comprises first column data serving as representation data and N columns of data divided according to the hierarchy; and b, assigning each item in the N columns of data according to a specific sequence according to a pointing path from a high level to a low level, wherein assignment at least reflects label tracing of an upper level relationship corresponding to the current level. The coding structure and the coding mode of the feature data are improved again, the feature structures in the feature data are subjected to structured classification, the treatment difficulty of the feature data is reduced, efficient storage of the feature data is met, the storage cost is reduced, smaller I / O operation is achieved, the feature data are loaded according to needs, the overall data structure is flexible, and the method and the device are higher in expansibility, powerful in function and extremely high in commercial value.

Description

Technical field [0001] The present invention belongs to the field of computer technology, particularly, it relates to a method for encoding data and device characteristics. Background technique [0002] Feature works in machine learning has a more important role, as a pre-process model training, model training feature helps in cleaning vast amounts of data, feature extraction. Online, he also forecast model as a pre-processing model, the model estimates the real-time query feature into the model to an online real-time estimates. Industrial grade features data will consume large computing and storage resources, which is usually the amount of data PB level, several tens of billions, while the characteristics of the data needs to be processed very quickly, and can not have too much impact on business services, in order to meet the characteristic data can be efficiently stored, querying, analyzing, coding features need to transform the data depth. [0003] Wherein the data storage is...

AI Technical Summary

Problems solved by technology

The data is extracted and merged through Map and Reduce, which will cause feature data analysis to process a large amount of invalid data by the way. Such IO, network, and calculation consumption are very huge.

[0004] At present, there is no technical solution that can solve the above technical problems, specifically, there is no encoding method and device for feature data

Images