Supercharge Your Innovation With Domain-Expert AI Agents!

Big data cluster data flow regulation and control method

A data flow and big data technology, applied in the field of big data cluster data flow regulation, can solve problems such as difficulty in guaranteeing reliability and cluster bottlenecks

Pending Publication Date: 2022-05-13
CHINA UNITECHS
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The disadvantage of this solution is: single-point service sequence judgment can easily cause the bottleneck point of the entire cluster, and the reliability is difficult to guarantee

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data cluster data flow regulation and control method
  • Big data cluster data flow regulation and control method
  • Big data cluster data flow regulation and control method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0074] figure 2 It is a preprocessing block diagram of OSPF neighbor Down in an embodiment of the present invention. Such as figure 2 shown, including:

[0075] When a cluster has 4 nodes, the threshold of each node is set to 5000kb / s, the monitoring program sets the global threshold to 5000kb / s*4=20000kb / s, and publishes it to the notification program;

[0076] The monitoring program continues to collect the actual request information of each node, and summarizes and accumulates the data of each node within the same period (one minute by default), which is the global actual request amount; at this time, a large amount of data requests are sent to node A from the outside, and the actual data flow is 7000kb / s; the actual data flow of other nodes is 4000kb / s;

[0077] At this time, the global judgment is that the current global actual request volume is 7000+4000*3=19000kb / s, the actual traffic of node A exceeds the threshold, and the monitor calculation threshold is adjuste...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a big data cluster data flow regulation and control method. The method comprises the steps that each node of a big data cluster carries out data flow statistics, data flow threshold setting and data flow regulation and control triggering; a distributed monitoring program periodically collects each node of the cluster and requests data flow statistical data; the distributed notification publishing program accepts and publishes information; the distributed monitoring program collects data flow data of each node, and calculates a data flow threshold value of each node in the next period; the monitoring program issues a data flow threshold value of each node in the next period to a distributed notification issuing program; the distributed notification publishing program notifies each node of the cluster of the latest data flow threshold message; when each node of the cluster senses the data flow threshold value of the node, resetting the data flow threshold value of the node; processing the request by each node of the cluster, and counting and judging whether the data flow exceeds a threshold value; through distributed regulation and control, the data flow threshold value of the whole cluster is calculated, the data flow regulation and control of the big data cluster are realized, and the flow control processing efficiency is improved.

Description

technical field [0001] The invention relates to the field of data flow control, in particular to a data flow control method of a large data cluster. Background technique [0002] For big data clusters, if it is necessary to regulate the request volume of the entire cluster, there are currently two main solutions: [0003] 1. Each node of the cluster is individually regulated, and the threshold for the entire cluster is (the number of cluster nodes × the threshold of a single node). [0004] The disadvantage of this solution is: when the request in the cluster is skewed, the request amount of a single node is too large, but the request amount of the entire cluster is still small, and the data flow control is not accurate. [0005] 2. Count the cluster request volume through a single point of service. Every request of this cluster needs to go through this service, and the threshold is determined by this service. [0006] The disadvantage of this solution is: single-point se...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/54G06F9/50G06F11/30
CPCG06F9/542G06F11/3051G06F9/5027
Inventor 闫一帅毛春阳
Owner CHINA UNITECHS
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com