Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Big data real-time enquiry system load balancing method based on copy selection

A load balancing and query system technology, applied in transmission systems, electrical components, etc., can solve problems such as optimal load balancing, disregarding heterogeneity of distributed systems, and failure to obtain optimal load balancing to achieve the effect of ensuring effectiveness

Active Publication Date: 2014-04-16
ZHEJIANG HONGCHENG COMP SYST
View PDF4 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The existing load balancing methods of big data real-time query systems have the following problems. First, it is impossible to obtain better load balancing
When the strategy for selecting replicas is determined each time, the degree of load balancing generated by different sequences of data blocks is varied. It is difficult to obtain better load balancing by only considering the default sequence of data blocks.
Second, the heterogeneity of the distributed system is not considered, such as the difference in the disk read rate of the machine

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data real-time enquiry system load balancing method based on copy selection
  • Big data real-time enquiry system load balancing method based on copy selection
  • Big data real-time enquiry system load balancing method based on copy selection

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] The present invention will be further explained below in conjunction with the drawings:

[0053] The invention is divided into two processes: node load information collection and node load balancing. The node load information collection process is as follows figure 1 As shown, the node load information reporter collects the load information of the node, and periodically sends the load information to the cluster load information collector. In the process of load balancing, the coordinator obtains the load information of all nodes through the cluster load information collector, and makes load balancing decisions based on the cluster status.

[0054] The main steps of the node load information collection part include:

[0055] 1) The node load information reporter registers with the cluster load information collector;

[0056] The node load information reporter sends the node's IP and host name to the cluster load information collector, and the cluster load information collector r...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the field of computer database processing, in particular to a big data real-time enquiry system load balancing method based on copy selection. The method comprises the processes of node load information collection and node load balancing, and the node load balancing process comprises the stages of preprocessing and the copy selection. The method has the advantages that the problems that an existing big data real-time enquiry system load balancing method is too simple and the current state of a machine is not considered are solved, the new big data real-time enquiry system load balancing method based on the copy selection is provided, the load balancing effect is superior to that of the existing big data real-time enquiry system, time complexity is low and is O (n2), and n is the number of blocks; the method is suitable for heterogeneous distributed systems and the conditions of operating other tasks in the systems.

Description

Technical field [0001] The invention relates to the field of computer database processing, in particular to a load balancing method for a big data real-time query system based on copy selection. Background technique [0002] In the era of big data, it is no longer possible to store massive amounts of data in a single server. Existing big data real-time query systems, such as Google Dremel, Cloudera Impala, etc., all adopt a distributed computing architecture to ensure real-time big data query. How to ensure the load balance of each node during operation has always been the focus of distributed systems. [0003] The database table of the existing big data real-time query system is logically composed of stored data and related metadata describing the data form in the table. Data is generally stored in a distributed file system. The existing distributed file system divides files into blocks, stores different data blocks of the same file on multiple nodes, and creates a copy of each...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/08
Inventor 王敬昌吴勇陈岭赵江奇徐精忠李晓平赵宇亮
Owner ZHEJIANG HONGCHENG COMP SYST
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com