Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method for accelerating cluster data distribution of large-scale distributed database

A data distribution and distributed technology, applied in the direction of database distribution/replication, database update, structured data retrieval, etc., can solve problems affecting the execution efficiency of database clusters, and achieve the effects of improving efficiency, low processing cost, and increasing speed

Pending Publication Date: 2019-11-26
江苏华库数据技术有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This will greatly affect the overall execution efficiency of the database cluster

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for accelerating cluster data distribution of large-scale distributed database
  • Method for accelerating cluster data distribution of large-scale distributed database
  • Method for accelerating cluster data distribution of large-scale distributed database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0015] A method for accelerating the data distribution of large-scale distributed database clusters comprises the following steps: referring to figure 2 As shown, in step 1, the receiving end divides a single table into multiple subtables, subtable 1, subtable 2, subtable 3, and subtable 4, and simultaneously receives data 1, data 2, data 3, and data 4 to reduce lock contention;

[0016] Step 2: After receiving, merge metadata 1, metadata 2, metadata 3 and metadata 4 of each sub-table to form a single table.

[0017] refer to image 3 As shown, the sub-tables are not in one-to-one correspondence with the sending nodes, and each sub-table can also receive data sent by two or more sending nodes at the same time, and can be configured according to needs.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for accelerating cluster data distribution of a large-scale distributed database. The method comprises the following steps: step 1, a receiving end dividing a single table into a plurality of sub-tables and receiving data at the same time to reduce lock scrambling; and step 2, after receiving is completed, combining the metadata of each sub-table to form a single table. Because the plurality of sub-tables can receive the table data sent by the sending node at the same time, the efficiency of receiving the table data can be remarkably improved. Meanwhile, due tothe fact that sub-table data files do not need to be combined, the data size of metadata is very small, and the processing cost for combining multiple sub-tables into a single table in the later period is very low. Therefore, the data distribution speed of the distributed database cluster can be remarkably increased.

Description

technical field [0001] The invention belongs to the technical field of data distribution, in particular to a method for accelerating data distribution of large-scale distributed database clusters. Background technique [0002] In a distributed database, table data will be distributed among multiple nodes according to certain rules. Many database operations require the transmission of table data between nodes, such as: cluster expansion or shrinkage; dynamic redistribution; inter-cluster data migration, and so on. In a large-scale distributed database cluster, it often happens that multiple nodes send data to some few nodes at the same time. For example, data is migrated from N nodes to a cluster of M nodes, and N is much larger than M. When this happens, multiple sending nodes need to queue up and wait for the receiving node to receive table data one by one and complete writing. Even if concurrent write conflicts can be reduced through caching strategies, it is difficult t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/27
CPCG06F16/2343G06F16/27
Inventor 赵伟刘威
Owner 江苏华库数据技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com