Self-adaptive structure adjusting method and system of machine learning data index structure

Abstract

The invention provides a self-adaptive structure adjustment method and system for a machine learning data index structure. The method comprises the following steps: a node selection step: selecting nodes in the machine learning data index structure one by one according to a preset node number sequence; a node analysis step: analyzing the selected nodes, and executing corresponding structure adjustment operation according to the cache data volume in the nodes and the error range size: if the cache data volume in the nodes or the prediction error of the machine learning model in the nodes is toolarge, executing a node splitting step; if the cache data volume and the error range in the node and the adjacent node are both too small, the two nodes execute a node merging step; otherwise, endingthe process. According to the fine-grained machine learning index structure adjustment method provided by the invention, compared with retraining of all data, the number of retrained models can be reduced, and the influence on performance of irrelevant models and cache during structure adjustment is avoided.

Description

technical field [0001] The invention relates to the field of data indexing in data storage systems, in particular to a method and system for adaptive structure adjustment of machine learning data indexing structures. Background technique [0002] In today's big data era, the scale of data is getting larger and larger, and databases are being used more and more, and new challenges are constantly encountered in the process, mainly including higher requirements for low-latency and high-throughput indexes, etc. . The index is an additional structure derived from the original data of the database, including many keywords, each keyword points to a piece of data, the index can quickly query the data corresponding to a certain keyword, and the index can be used in the database to speed up data query. [0003] Machine learning studies how to use algorithms and statistical models to enable computer systems to effectively perform a specific task based on the observation and inference ...

AI Technical Summary

Problems solved by technology

Therefore, the previous work on the research of machine learning index structure is not comprehensive, and there is still a certain gap

Images