Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method for loading parallel compressed mass data of database cluster

A mass data and data loading technology, applied in the database field, can solve the problems of time-consuming data, lack of public methods, and affecting data loading performance, so as to achieve the effect of optimizing performance and improving construction speed

Inactive Publication Date: 2013-03-20
天津神舟通用数据技术有限公司
View PDF3 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Benefits of technology

In this new system that improves efficiency on handling large amounts of data quickly without overloading any server(ies). It achieves these goals by integrating both hardware (CPU) and software modules into one machine called an adapter), which allows more efficient access to memory space than just two levels like traditional systems. By doing things efficiently at different stages within it, we can achieve faster build times while still utilising all available computer power effectively.

Problems solved by technology

This patented problem addressed by this patents relates to efficiently preparation large amounts of data from multiple databases on one machine without having any extra steps needed during installation. Additionally, current techniques like inserting rows can be slowed down due to resource limitations caused by these processes.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for loading parallel compressed mass data of database cluster
  • Method for loading parallel compressed mass data of database cluster
  • Method for loading parallel compressed mass data of database cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] Embodiments of the present invention are described in further detail below in conjunction with the accompanying drawings:

[0041] A parallel compressed mass data loading method for a database cluster, comprising the following steps:

[0042] Step 1: Establish a component architecture for parallel compressed massive data loading including data loading tools, drivers and database clusters.

[0043] When loading data to the database cluster, it is necessary to establish such figure 1 Component architecture shown, including data loader, driver, and database cluster. Through this component architecture, even if the component architecture of mature commercial database cluster products cannot be known, the database cluster can have a driver that directly operates it like a database, executes the SQL statement for data loading through the driver, and provides a call driver A data loading tool that loads data files or other database data into the database cluster. Each part ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a method for loading parallel compressed mass data of a database cluster. The method is mainly technically characterized by comprising the following steps of: 1, establishing an assembly architecture comprising a data loading tool, a driving program and a database cluster for loading parallel compressed mass data; 2, sending a data loading command to the database cluster through the data loading tool and the driving program by using a client, acquiring connection information of each database node, connecting each database and transmitting data required to be loaded; providing connection information of each database node of the cluster and relevant information of each table for the client by using a database cluster server; and receiving data transmitted by the client and saving by using the database nodes. In the method, a mass data communication protocol and parallel optimizing processing and compression optimizing processing methods are adopted, and multiple CPUs (Central Processing Units) and network resources of the server are utilized fully and are merged in a mass data loading process, so that the building speed of the database cluster is increased.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Owner 天津神舟通用数据技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com