Method of uniformly segmenting database table data and computer equipment

Abstract

The invention provides a method of uniformly segmenting database table data. Column names, column types and index status in a table are acquired through scanning the table needing to be segmented; columns which have column types as an integer type and have not a duplicate value are picked out, and if no corresponding column is found, a column of a self-increasing type is newly added in the table,and is used as a segmentation column; otherwise, a column is arbitrarily selected to be used as the segmentation column; a pre-partition number M is set; and all numerical values of the segmentation column are read into an array, and M partition intervals of the array are calculated through a quantile algorithm to obtain M numerical intervals. The invention also provides computer equipment. According to the method and the equipment, data segmentation is carried out with lower algorithm complexity and lower database overheads, and problems of database burdens and fast partition are better solved.

Description

technical field [0001] The invention relates to a method and computer equipment for evenly dividing data in a database table. Background technique [0002] Data partitioning is an important part of distributed computing: it is required that data be divided into multiple parts evenly before allocating appropriate computing resources, and the number of partitions usually changes dynamically (such as the current remaining computing resources, the amount of resources that users can control) . The data in a traditional database table is generally not partitioned (even if it is partitioned, it is not the number of partitions required for actual calculation), and partitioning is performed when the calculation task is submitted. The usual practice requires scanning all the data in the split column, which increases the burden on the database, and Due to the uneven distribution of values ​​in the split columns, simple algorithms cannot be implemented, and the real-time performance of...

AI Technical Summary

Problems solved by technology

The data in a traditional database table is generally not partitioned (even if it is partitioned, it is not the number of partitions required for actual calculation), and partitioning is performed when the calculation task is submitted. The usual practice requires scanning all the data in the split column, which increases the burden on the database, and Due to the uneven distribution of split column values, simple algorithms cannot be implemented, and the real-time performance of complex algorithms will be greatly affected

Images