Distributed parallel database system based on Infiniband network and data processing method

A data processing and database technology, applied in the direction of database distribution/replication, structured data retrieval, etc., to achieve the effect of breaking through the bottleneck of network bandwidth, strong versatility, and high availability

Pending Publication Date: 2019-06-25
CHINA REALTIME DATABASE +3
View PDF4 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a distributed parallel database system and data processing method based on the Infiniband network, which solves the network bandwidth bottleneck of the existing 10 Gigabit network in a specific scenario in the distributed parallel processing database , memor

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed parallel database system based on Infiniband network and data processing method
  • Distributed parallel database system based on Infiniband network and data processing method
  • Distributed parallel database system based on Infiniband network and data processing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The distributed parallel processing database deployment architecture of this embodiment is as follows figure 1 As shown, the distributed parallel processing database adopts a federated architecture. The database system of the present invention is distributed, and two or more servers are required for installation and deployment. The servers are referred to as nodes below, and the series of N nodes with the same function or characteristics are usually called a cluster. Clusters are divided into scheduling clusters, data clusters, and management clusters, and corresponding nodes are divided into scheduling nodes and data nodes according to their functions. The scheduling cluster of the present invention includes a distributed access layer and a distributed positioning layer; the distributed access layer adopts a unified access interface, so that the distributed parallel processing database can be accessed as a complete logical whole; the distributed positioning layer serve...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed parallel database system based on an Infiniband network and a data processing method. The system comprises a scheduling cluster, a data cluster and a management cluster. Each cluster is composed of at least two data nodes, and the data nodes are connected through the Infiniband network. The data processing method comprises a data distribution method, a data loading method, a data query method and a data de-duplication method. Aiming at the current situation that the number of distributed parallel processing database cluster node servers is large, the characteristics of high bandwidth, low delay and low memory of an Infiniband network are fully utilized, and a mode of applying the network to data is designed, including database system configuration, data loading and storage, data query, data calculation and the like. The method is high in universality, breaks through network bandwidth bottleneck, storage space and calculation delay limitation of a current database system, guarantees high availability of the system, and provides high-performance service for users.

Description

technical field [0001] The invention relates to a distributed parallel database system and a data processing method, in particular to a distributed parallel database system and a data processing method based on an Infiniband network. Background technique [0002] Database technology came into being in the late 20th century, its theory and technology developed extremely rapidly, and its application became more and more extensive. With the wide application of microcomputers, in order to strengthen and expand the functions of microcomputers to process data, it is required to interconnect many microcomputers distributed in different places to participate in common work. In this way, the database has entered the era of distributed databases. Since the 1990s, distributed database systems have entered the stage of commercial application, and traditional relational database products have developed into distributed database products centered on computer networks and multi-tasking op...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/27
Inventor 乔斌蒋树嵩张见豪袁烨查易艺
Owner CHINA REALTIME DATABASE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap