Method of uniformly segmenting database table data and computer equipment

A computer equipment and database technology, applied in the field of evenly segmenting database table data, can solve problems such as the algorithm cannot be implemented, not partitioned, and the real-time effect of complex algorithms, so as to achieve the effect of having no database overhead and satisfying real-time performance.

Pending Publication Date: 2018-08-03
福建星瑞格软件有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The data in a traditional database table is generally not partitioned (even if it is partitioned, it is not the number of partitions required for actual calculation), and partitioning is performed when the calculation task is submitted. The usual practice requires scanning all the data in the split column, which increases the burden on the database, and Due to the uneven distribution of split column values, simple algorithms cannot be implemented, and the real-time performance of complex algorithms will be greatly affected

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method of uniformly segmenting database table data and computer equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The method for uniformly segmenting database table data of the present invention comprises:

[0025] Step 1. Obtain the list, column type and index status in the table by scanning the table to be split;

[0026] Step 2. Select a column whose type is integer and has no repeated values. If the corresponding column cannot be found, add a new column of auto-increment type to the table, and this column will be used as a split column; otherwise, arbitrarily select a column as a split categorize;

[0027] Step 3, set the pre-partition number M;

[0028] Step 4. Read all the values ​​of the split column into the array, and calculate M partition intervals of the array through the quantile point algorithm to obtain M value intervals.

[0029] Step 5. Store the M value intervals in the database or memory, and expose the service interface to the outside for calling; obtain the table to be partitioned and the actual number K of partitions from the application program, obtain the M...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method of uniformly segmenting database table data. Column names, column types and index status in a table are acquired through scanning the table needing to be segmented; columns which have column types as an integer type and have not a duplicate value are picked out, and if no corresponding column is found, a column of a self-increasing type is newly added in the table,and is used as a segmentation column; otherwise, a column is arbitrarily selected to be used as the segmentation column; a pre-partition number M is set; and all numerical values of the segmentation column are read into an array, and M partition intervals of the array are calculated through a quantile algorithm to obtain M numerical intervals. The invention also provides computer equipment. According to the method and the equipment, data segmentation is carried out with lower algorithm complexity and lower database overheads, and problems of database burdens and fast partition are better solved.

Description

technical field [0001] The invention relates to a method and computer equipment for evenly dividing data in a database table. Background technique [0002] Data partitioning is an important part of distributed computing: it is required that data be divided into multiple parts evenly before allocating appropriate computing resources, and the number of partitions usually changes dynamically (such as the current remaining computing resources, the amount of resources that users can control) . The data in a traditional database table is generally not partitioned (even if it is partitioned, it is not the number of partitions required for actual calculation), and partitioning is performed when the calculation task is submitted. The usual practice requires scanning all the data in the split column, which increases the burden on the database, and Due to the uneven distribution of values ​​in the split columns, simple algorithms cannot be implemented, and the real-time performance of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/22G06F16/221G06F16/2228
Inventor 李昭阳
Owner 福建星瑞格软件有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap