Method for achieving dynamic partitioning and load balancing of data-partitioning distributed environment

A technology of distributed environment and data division, applied in the direction of resource allocation, multi-program device, etc., can solve problems such as excessive granularity, and achieve the effect of improving work efficiency, wide applicability, and increasing resource utilization.

Active Publication Date: 2011-10-05
ZHEJIANG UNIV
View PDF4 Cites 50 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] 1. The present invention dynamically combines basic data units, that is, performs dynamic division to perform local load balancing, so that the division granularity can adapt to the requirements of system operation, and solves the problem of load balancing timeliness caused by the fixed division method due to excessive granularity

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for achieving dynamic partitioning and load balancing of data-partitioning distributed environment
  • Method for achieving dynamic partitioning and load balancing of data-partitioning distributed environment
  • Method for achieving dynamic partitioning and load balancing of data-partitioning distributed environment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] Dynamic partitioning enables the basic data units that make up a data partition to be moved to other partitions at runtime and are no longer fixed. Load balancing is mainly divided into three parts: local load balancing, global load balancing, and local resource utilization. Local load balancing balances the load between partitions by moving basic data units between partitions within a single node. The movement of basic data units occurs mainly in memory, is lightweight, and can be performed quickly. The data division after local load balancing is beneficial to improve the efficiency of global load balancing and prevent the occurrence of load balancing failure. For local resource utilization, if the monitoring finds that the use of node resources is insufficient, consider increasing the number of data partitions; otherwise, reduce the number of data partitions. Such an adjustment can improve resource utilization. For example, when the multi-process rate is too low, it...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for achieving dynamic partitioning and load balancing of a data-partitioning distributed environment. According to the invention, the method can achieve the local load balancing by basic data unit dynamic combination, namely dynamic partitioning, so as to make the partition granularity meet the system operation requirement. Therefore, the method solves the problemof load balancing time limitation resulting from over-large granularity of a fixed partitioning method. The method provided by the invention has wide applicability by using a partition request queue and a scarce resource queue model to analyze node loads. A local monitor provided in the invention can dynamically adjust the number of partitions after the system achieves load balance so as to achieve self-adaptive adjustment of the partitions inside nodes, thereby improving the resource utilization rate inside the nodes and further improving the work efficiency of the entire system.

Description

technical field [0001] The invention relates to a distributed environment based on data division, in particular to a method for realizing dynamic data division and load balancing in the data division environment. Background technique [0002] With the globalization of enterprises and the continuous development of massive data processing needs, distributed systems that support business operations of enterprises / companies are becoming more and more popular. However, the two classic distributed models based on application division, the asymmetric model and the symmetric model have their insurmountable defects: the asymmetric model cannot balance the load between nodes, and is prone to performance problems caused by load imbalance; The traditional model has good load balancing capabilities, but for a large number of dependent requests, a database is required for data synchronization and collection, which will also bring performance problems that cannot be ignored. [0003] In o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
Inventor 王新宇杨小虎李善平王婷邱炜伟
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap