Load-balanced distribution method based on data distribution in distributed data flow

A data distribution and load balancing technology, applied in data exchange networks, digital transmission systems, electrical components, etc., can solve the problems of uneven distribution of data flow sources and obvious changes in flow, and achieve the effect of optimizing and improving topology design.

Active Publication Date: 2018-05-22
YANTAI UNIV
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above-mentioned technical problems of the existing real-time streaming big data analysis and processing, this method provides a load balancing distribution method based on data dist

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Load-balanced distribution method based on data distribution in distributed data flow
  • Load-balanced distribution method based on data distribution in distributed data flow
  • Load-balanced distribution method based on data distribution in distributed data flow

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0071] The invention relates to a method for handling the load balancing distribution of data streams of sudden hotspot events.

[0072] Such as figure 1 As shown, the labels A, B, C, and D are the order of the load balancing distribution method based on data distribution in the distributed data stream, and the sliding window of A slides forward by a time stamp, which includes the distributed hotspot emergency data stream processing , Mapping of data points of hotspot emergencies, B query or create a grid index tree, which includes constructing a hierarchical grid index tree containing data points of hotspot emergencies, C query or adjust the data routing table, which includes a hierarchical grid index tree Establish association with the data routing table, D data distribution.

[0073] Such as figure 2 As shown, the three areas 1, 2, and 3 are respectively composed of figure 1 Cluster machine node bolt in 1 、bolt 2 and bolt 3 Real-time analysis and processing, at this ...

Embodiment 2

[0084] The invention relates to an adjustment method when the load is unbalanced in a method for handling load balance distribution of sudden hot event data flow.

[0085] when timestamp t i When the included data points arrive, the cluster node load is unbalanced,

[0086] Such as figure 2 The processing steps for load judgment and adjustment after step 1 are as follows:

[0087] for timestamp t i The above steps 1)~4) are also applied to the hot event data points contained in , and for step 5) due to the emergence of hot event 4 in the current area 1, the number of cluster node data points in the grid numbered h1_1 contained in the sliding window surge, causing the preload of cluster node 1 to exceed the load buffer threshold, such as image 3 shown in t i According to the current data routing record distribution method at all times, the load of cluster node 1 will be too large. At this time, the grid contained in cluster node 1 needs to be adjusted. The steps are as f...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a load-balanced distribution method based on data distribution in a distributed data flow. The method is particularly suitable for solving a problem of monitoring and analyzingan event that data flow source distribution is uneven and flow change is obvious. The method comprises the steps of: processing a distributed hotspot emergency data flow, mapping a hotspot emergency data point, constructing a hierarchical grid index tree including the hotspot emergency data point, establishing association of the hierarchical grid index tree and a data routing table, and performingdata distribution. By means of the method provided by the invention, load balancing of various slave nodes is ensured while emergency hotspot events in the same area cannot be randomly distributed toother slave nodes; and the technical support is provided for real-time performance analysis on the emergency hotspot events.

Description

technical field [0001] The present invention relates to a load balancing distribution method based on data distribution in a distributed data flow. The method is applied to spatio-temporal big data processing and belongs to the technical field of real-time streaming big data analysis and processing in distributed computing technology. The present invention particularly relates to The method for handling the load balancing distribution of data streams of sudden hotspot events. Background technique [0002] The rapid development of emerging information technologies and application models such as cloud computing, the Internet of Things, mobile interconnection, and social media has led to a sharp increase in the amount of global data and pushed human society into the era of big data. Taking mobile social media data as an example, tens of millions of microblogs and check-in data are generated every hour. These data will be important for the analysis of hot topics, public opinion ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/801H04L12/803H04L12/853H04L12/741H04L12/751H04L45/02H04L45/74H04L47/2416
CPCH04L45/02H04L45/54H04L47/125H04L47/2416H04L47/29
Inventor 于彦伟齐建鹏刘兆伟赵金东
Owner YANTAI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap