Shared storage-based MPP database data redistribution system

A shared storage and database technology, applied in the database field, can solve problems such as business conflicts, and achieve the effect of avoiding performance problems and business concurrency problems

Inactive Publication Date: 2017-02-08
TIANJIN NANKAI UNIV GENERAL DATA TECH
View PDF4 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, limited by the existing mechanism, it is difficult to complete the data redistribution of existing large-scale data in a short period of time, and there will be conflicts with the business during the data redistribution period

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Shared storage-based MPP database data redistribution system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0034] A best implementation of the MPP database data redistribution system based on shared storage, comprising the following steps:

[0035] 1) Assuming that the shared storage system adopts a distributed file system, 65536 directories are established as storage units and exported to all computing nodes.

[0036] 2) There is an MPP cluster, in which it is assumed that there are 32 computing nodes, the nodeids are n1-n32, and each node corresponds to 2048 hash buckets.

[0037] 3) MPP management node, nodemap records are as follows:

[0038] n1: h0, h32, h64...

[0039] n2: h1, h33, h65...

[0040] ...

[0041] n32: h31, h63, h95... h65535

[0042] Storagemap is documented as follows:

[0043] h0:d0

[0044] h1:d1

[0045] h2: d2

[0046] ...

[0047] h65535:d65535

[0048] 4) Each node, according to the nodemap, mounts the storage unit corresponding to the distributed file system to realize data storage.

[0049] 5) Assuming that the calculation stage is extended t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a shared storage-based MPP database data redistribution system. The shared storage-based MPP database data redistribution system includes a shared storage system, an MPP cluster management node and MPP cluster distributed computation nodes. The shared storage-based MPP database data redistribution system is used for solving the performance problem of the redistribution of data in the existing MPP databases. Through the shared storage-based MPP database data redistribution system, the redistribution of the data can be rapidly realized by the MPP databases according to a distributed storage system when computation node undergoes expansion, so that the performance problem and the business concurrence problem of the redistribution of the existing MPP system data are avoided and the online businesses are hardly influenced.

Description

technical field [0001] The invention relates to the field of databases, in particular to an MPP database data redistribution system based on shared storage. Background technique [0002] With the development of informatization in various industries, the scale of data is getting larger and larger, which brings great challenges to both computing and storage. Existing MPP database clusters are also constantly facing new challenges, especially when the storage and computing capabilities can no longer meet the needs, there is an urgent need to expand computing nodes or storage nodes for MPP database clusters. Simply adding computing nodes and storage nodes brings new problems, that is, old data needs to be redistributed to new nodes to adapt to the performance and storage capacity improvements brought about by expansion. However, limited by the existing mechanism, it is difficult to complete the data redistribution of existing large-scale data in a short period of time, and ther...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/284G06F16/21
Inventor 武新崔维力李春华
Owner TIANJIN NANKAI UNIV GENERAL DATA TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap