Adaptive data loading method for heterogeneous cluster storage

A data load, heterogeneous cluster technology, applied in data exchange networks, digital transmission systems, input/output to record carriers, etc., can solve problems such as storage cluster heterogeneity

Inactive Publication Date: 2015-01-07
LANGCHAO ELECTRONIC INFORMATION IND CO LTD
View PDF0 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This causes the storage c

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0016] A method for storing adaptive data loads in heterogeneous clusters. By integrating various real-time information such as computing resources, network resources, storage resources, and user request data, the load balancing of the entire distributed system is realized. The specific steps are:

[0017] ①Basic load balancing. When the system is initially built, the data is distributed on each node of the cluster according to the maximum load capacity of the node;

[0018] ②Adaptive distributed incremental load balance, when the system is running, collect the resource occupancy of each node in real time, and adjust the data distribution adaptively and dynamically;

[0019] ③Using the weight-based hash method, the data load size is regarded as the same when the system is initially built, and the performance of heterogeneous servers is used as the weight to evenly distribute the data load among the clusters.

Embodiment 2

[0021] A method for storing adaptive data loads in heterogeneous clusters. By integrating various real-time information such as computing resources, network resources, storage resources, and user request data, the load balancing of the entire distributed system is realized. The specific steps are:

[0022] ①Basic load balancing. The data load is the CPU, hard disk, network usage and new energy parameters. When the system is initially built, the data is distributed on each node of the cluster according to the maximum load capacity of the node;

[0023] ②Adaptive distributed incremental load balance, when the system is running, collect the resource occupancy of each node in real time, and adjust the data distribution adaptively and dynamically;

[0024] ③Using the weight-based hash method, the data load size is regarded as the same when the system is initially built, and the performance of heterogeneous servers is used as the weight to evenly distribute the data load among the cl...

Embodiment 3

[0026] A method for storing adaptive data loads in heterogeneous clusters. By integrating various real-time information such as computing resources, network resources, storage resources, and user request data, the load balancing of the entire distributed system is realized. The specific steps are:

[0027] ①Basic load balancing. The data load is the CPU, hard disk, network usage and new energy parameters. When the system is initially built, the data is distributed on each node of the cluster according to the maximum load capacity of the node;

[0028] ②Adaptive distributed incremental load balance, when the system is running, collect the resource occupancy of each node in real time, and adjust the data distribution adaptively and dynamically;

[0029] ③Adopt the weight-based hash method, regard the data load size as the same when the system is initially built, take the performance of heterogeneous servers as the weight, distribute the data load evenly among the clusters, and us...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an adaptive data loading method for heterogeneous cluster storage, and belongs to the field of computer system storage. The method comprises the following specific steps: (1) basic load balance: at the initial establishment of a system, distributing data on each node of clusters according to the maximum load capacity of each node; (2) adaptive distributed incremental load balance: during running of the system, collecting the resource occupation situation of each node in real time, and adjusting a data distribution situation adaptively and dynamically; and (3) distributing data loads among the clusters in a balanced manner at the initial establishment of the system by considering the load size of data as identical and taking the performance of a heterogeneous server as a weight through a weight-based hash method. The method is suitable for processing heterogeneous data loads in a high-stress, high-concurrent-reading-writing, multi-user and heterogeneous large-scale distributed storage system, and adaptively adjusting the distribution of data among servers according to loads.

Description

technical field [0001] The invention relates to a method for storing adaptive data loads, which belongs to the field of computer system storage, in particular to a method for storing adaptive data loads in heterogeneous clusters. Background technique [0002] The development of storage technology is accompanied by the development of computer technology, that is to say, since the birth of computer technology, people have been striving for a higher performance storage system. In the past few decades, it can be said that the era of continuous innovation and development of storage technology, especially in the last two decades, the cluster storage system has shown an explosive growth trend, including SAN, NAS, Lustre, HDFS, Ceph, etc. Such cluster storage. The cluster storage architecture has strong vitality and broad development prospects in the fields of large-scale enterprise application architecture, Internet, Internet of Things, big data, and high-performance computing. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08H04L12/803G06F3/06
CPCH04L67/1097H04L67/1001
Inventor 陈大雅程瑶刘粉粉
Owner LANGCHAO ELECTRONIC INFORMATION IND CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap