Data mining method and device based on feature conversion function and readable medium

Abstract

The invention relates to a data mining method and device based on a feature conversion function and a readable medium, and belongs to the technical field of computers, the method comprises the following steps: extracting original data: extracting the original data required by feature mining, and carrying out two times of lossless information compression processing on the complex multi-dimensional original data to obtain the original data; forming a data structure required by the feature conversion function; and format conversion: constructing a format conversion function, and converting the Y * M data table generated in the step 1 into a list and dict format which can be processed in a Python environment as an input item for feature generation. And feature generation: constructing a feature conversion function, and converting the input original data in the specific format into final features. The invention establishes a data mining method based on a feature conversion function structure, and constructs a more advanced and efficient feature data mining method by using the feature conversion function.

Description

Technical field [0001] The present invention relates to the field of computer technology, in particular to a data mining method based on a feature conversion function, apparatus and a readable medium. Background [0002] With the rapid development of machine learning technology, machine learning technology is widely used in the data analysis process of various industries, bringing great convenience to people, the existing technology in the feature mining is to directly use SQL statements to query the original data, and the use of SQL with its own computing power to calculate the features, and finally the query and the operation coupled together, this way with the increase in the number of feature mining, the complexity of SQL statements will rise exponentially, resulting in subqueries that must nest more and more layers, Add temporary intermediate tables to achieve operations, and the development cost becomes higher and the running speed becomes slower. At the same time, due to ...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of machine learning technology, machine learning technology is widely used in the data analysis process of various industries, which brings great convenience to people. In the prior art, feature mining directly uses SQL statements to query raw data, and Use SQL's built-in computing power to calculate features, and finally couple queries and operations together. In this way, as the number of feature mining increases, the complexity of SQL statements will increase exponentially, resulting in the need to nest more and more layers of subcategories. Query, adding a temporary intermediate table to realize the operation, the development cost becomes higher, and the running speed becomes slower

At the same time, due to the operation of SQL itself and the limitation of resources, in the process of feature mining, a large number of useless temporary intermediate tables will be generated, which belongs to the invalid occupation of resources

Images