Unlock instant, AI-driven research and patent intelligence for your innovation.

A lock-step distribution method based on tree network and data partitioning in big data environment

A tree-shaped network and big data technology, which is applied in electrical digital data processing, special data processing applications, digital data information retrieval, etc., can solve problems such as unfavorable transmission, large network impact, and complex tree-shaped network.

Active Publication Date: 2020-07-28
GUANGDONG UNIV OF TECH
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Compared with the star network, the total length of the tree network is short, the cost is low, and the nodes are easy to expand, but the tree network is complex, and when the link connected to the node fails, it will have a greater impact on the entire network
[0004] Data block: Since the data file is relatively large, the transmission delay is also very large, and once an error occurs, the file will be resent, which is not conducive to transmission; Select data block resend and data splicing according to sequence number

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A lock-step distribution method based on tree network and data partitioning in big data environment
  • A lock-step distribution method based on tree network and data partitioning in big data environment
  • A lock-step distribution method based on tree network and data partitioning in big data environment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0022] Given a descending set of node upload capabilities {3, 3, 2, 2, 2, 1, 1, 1, 1, 1}, according to the above method, the height h of the tree network ranges from [1, 8], and r takes The value range is [3 / 9,10 / 9], because the out-degree k is an integer, and the value range is [1,9], so we can discretize r (r=c / k) to get the value of r The range is {1 / 9,1 / 8,1 / 7,1 / 6,1 / 5,1 / 4,1 / 3,1 / 2,1,2 / 9,2 / 7,2 / 5,2 / 3,2,3 / 8,3 / 7,3 / 5,3 / 4,3 / 2,3}, from the value range of h and r, traverse r and h, and find the completion time D( That is, the time spent on each layer is A total of h layers) the value of the smallest r, assuming B=1, finally you can get r * =1, h=2, establish as Figure 5 The tree network shown. This makes the tree network more compact and minimizes data distribution time.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The big data distribution technology of a tree network is to segment and sequentially number big data, and an optimal tree network structure with the minimum height is established, so that the data distribution of a big data file in the network can be performed in the form of an assembly line, thereby greatly shortening data distribution delay and reducing the computing cost, and meanwhile, by adoption of the method, the transmission time of the data in the tree network can also be predicted. The invention provides a lock-step distribution method of the big data in the tree network. A fixed and identical uploading rate of all nodes is discovered to minimize the whole data distribution time, and meanwhile a new method of improving the data distribution rate by adopting the assembly line technology and reducing the height of the tree network is provided to greatly reduce the data distribution cost of the big data in the tree network.

Description

technical field [0001] The invention relates to data distribution in the network, especially the block distribution of large data in the tree network. Background technique [0002] Big data refers to a collection of data that cannot be captured, managed and processed by conventional software tools within a certain period of time. , high growth rates and diverse information assets. Big data has 5V characteristics (proposed by IBM), which are Volume (mass), Velocity (high speed), Variety (variety), Value (low value density), and Veracity (authenticity). [0003] Tree network: such as figure 1 Shown: The tree network is formed by the vertical connection of multi-level star structures, and each node of the tree is a computer or a switching device. In general, the closer to the root of the tree, the better the performance of the node device. Compared with the star network, the total length of the tree network is short, the cost is low, and the nodes are easy to expand. Howeve...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08G06F16/178
CPCG06F16/178H04L67/1095H04L67/1097
Inventor 李嘉兴陈平华武继刚
Owner GUANGDONG UNIV OF TECH