Method and device for constructing feature engineering calculation topological graph

Abstract

The embodiment of the invention provides a method and device for constructing a feature engineering calculation topological graph, and belongs to the technical field of computers. The method comprises the steps that operator functions in a configuration file are traversed according to an inverted sequence of a front-back dependency sequence between the operator functions in the configuration file of the feature engineering to determine a feature dependency linked list so as to construct a feature engineering calculation topological graph, one operator function corresponds to at least one feature, and the other operator function corresponds to at least one feature; the feature information of any feature comprises the feature, a dependency feature and the operator function, and the feature engineering calculation topological graph comprises a feature dependency linked list; one feature dependency linked list aims at a group of features with a calculation dependency relationship or a single feature without a corresponding calculation dependency relationship when feature engineering calculation is carried out. Therefore, the online feature engineering performance is improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method and device for constructing a feature engineering calculation topology graph. Background technique [0002] Topological graphs in mathematics and computer science are often used in real-life applications, such as common traffic route maps, task assignments, construction schedule calculations, and aviation networks. There are several ordered vertices in the topological graph, and each directed edge points from the top-ranked vertex to the low-ranked vertex. Through topology diagrams and topological sorting, we can find the shortest path or critical path in a task or plan to guide us to make reasonable decisions and arrangements, and finally complete tasks efficiently. [0003] The current deep learning model is the main implementation method of artificial intelligence technology in the industry. A complete offline deep learning model training process includes ...

AI Technical Summary

Problems solved by technology

The characteristic data infrastructure of different Internet companies is different. In terms of characteristic engineering calculation, the industry obtains the characteristics of each material to be sorted through configuration files, and then performs specific characteristic operator calculations for each material, but In the usage scenarios of search and recommendation advertisements, there may be more items to be sorted each time. For example, the total number of items to be sorted exceeds 100, which is time-consuming.

[0004] The current online feature engineering is to calculate the features of multiple materials concurrently through multiple threads, and with the enrichment of the features required by the model, there are more operators in the configuration file, and there are successive dependencies between operators, so each material When performing feature engineering, it can only be calculated linearly, that is, the calculation of operators is performed from first to last according to the dependency relationship, and the performance of online feature engineering is poor.

Images