Data partition loading method and device for association table

A technology of data partitioning and associated tables, applied in database indexing, database updating, digital data processing, etc., can solve the problems of high overhead for repeated data reading, complex partition loading process, high resource consumption, etc., to reduce computing overhead , Reducing resource consumption, the effect of strong versatility

Active Publication Date: 2021-10-01
WUHAN DAMENG DATABASE
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0015] Aiming at the above defects or improvement needs of the prior art, the present invention provides a data partition loading method and device for an association table, the purpose of which is to determine the partition value of the attached table through the par...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data partition loading method and device for association table
  • Data partition loading method and device for association table
  • Data partition loading method and device for association table

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0061] In order to solve the technical problems of large resource consumption, high overhead of repeated data reading and calculation, and complicated partition loading process when partition loading is traditionally implemented through connection, the present invention provides a data partition loading method based on partition mapping cache . In order to realize the data partition loading of the association table, before the data partition loading, that is, when the partition loader is initialized, it will first construct the tree data structure corresponding to the association table according to the association relationship, and create Node builds an empty partition map cache; where, for each node, the partition loader builds an empty partition map cache for each child node of the node, or an empty partition map cache for each set of associated child nodes of the node .

[0062] The embodiment of the present invention mainly introduces the construction of tree data structu...

Embodiment 2

[0070] On the basis of the tree data structure and the partition map cache built in the above-mentioned embodiment 1, the next step is as follows Image 6 The module division and data flow in the partition loading process are shown, and the table data reading, partition calculation and data distribution are performed in sequence. However, due to the addition of the tree-like association structure and the partition map cache, especially the partition map cache, the method of obtaining data partition values ​​has changed, so in Image 6 A new design has been added to the "Partition Computing" module, and the evolved partition loading module is divided as Figure 8 shown.

[0071] combine Figure 9 , the data partition loading method of the association table provided by the embodiment of the present invention is specifically as follows:

[0072] Step 101, when data partition loading is performed on the master table, calculate the partition value of the data based on the partit...

Embodiment 3

[0085] In the above-mentioned embodiment 2, the data partition loading process of the main table and the supplementary table is introduced respectively. For the associated table model with multiple tables, the tree data structure is usually traversed in the order of depth-first traversal of the tree data structure. Each node in the data partition load, refer to Figure 10 , the specific execution sequence is as follows:

[0086] Step 201, use the master table corresponding to the root node as the current node to perform data partition loading, that is, first perform data partition loading on the master table, and update the partition mapping cache maintained by the master table.

[0087] Step 202, after the data partition loading of the current node is completed, it is judged whether the current node has child nodes. Wherein, the completion of the data partition loading specifically means that the data distribution of each piece of data in the table of the current node has be...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data partition loading method and device for an association table. The method comprises the steps: by introducing a tree data structure, the partition value is calculated based on a partition key value of data when a main table is subjected to data partition loading, then the data is distributed according to the calculated partition value, and a partition mapping cache of the main table is updated; secondly, when data partition loading is carried out on the attached table, partition values are obtained from partition mapping caches of father nodes of the attached table based on associated field values of data between the attached table and the father nodes of the attached table, the data are distributed according to the obtained partition values, and the partition mapping caches of the attached table are updated; finally, for each node, the partition mapping cache stores a mapping relationship between an associated field value of the data between the node and a child node of the node and a corresponding partition value. According to the scheme, each piece of data only needs to calculate the partition value once at the root node, so that the calculation overhead is reduced; wherein, in the whole loading process, all the tables only need to be traversed and read once, operations such as sorting and table connection are not needed, and resource consumption and reading overhead are reduced.

Description

【Technical field】 [0001] The invention belongs to the technical field of data loading, and more particularly relates to a method and device for loading data partitions of an association table. 【Background technique】 [0002] For two tables A and B, assuming that each piece of data in B has the value of one or more fields, there must be a piece of data in A that has the same value in the corresponding field, then A table is called the main table, and B table is called For the attached table, table A and table B form an association table, and the corresponding fields are called association fields. Among them, the main table A can have multiple attached tables, and the attached table B can also have its own attached table, but the attached table can only have one main table, forming a multi-level association table. If the relationship between the associated tables is expressed in a tree structure, the main table is the root node, and the attached tables are all child nodes. Th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/22G06F16/23G06F16/2455
CPCG06F16/2282G06F16/24552G06F16/23
Inventor 梅纲袁松彪张驻西
Owner WUHAN DAMENG DATABASE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap