Unlock instant, AI-driven research and patent intelligence for your innovation.

A coding distributed computing method based on partition structure

A distributed computing and encoding technology, applied in the field of network communication, can solve problems such as impractical application, large number of input files and output functions, etc.

Active Publication Date: 2022-04-19
GUANGXI NORMAL UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In S.Li, M.A. Maddah-Ali, Q. Yu, and A.S. Avestimehr, “Afundamental tradeoff between computation and communication in distributed computing,” IEEE Trans. Inf. Theory, vol.64, no.1, pp.109–128, In the scheme proposed by Jan.2018., the number of input files and output functions is too large to be practically applied, so we hope to sacrifice part of the communication load L to reduce the number of input files N and the number of output functions Q

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A coding distributed computing method based on partition structure
  • A coding distributed computing method based on partition structure
  • A coding distributed computing method based on partition structure

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0047] For the number s of calculations of each output function, most of the literature is researched on the case of s=1, and only a few articles mention the case of s>1, and in this case, the focus is on Solve the problem of how to reduce the number N of input files and the number Q of output functions when s>1.

[0048] An encoding distributed computing method based on the Partition structure. In encoding distributed computing, for N input files, K distributed computing nodes are used to calculate the values ​​of Q output functions. The entire calculation process is carried out in three stages: In Map stage, Shuffle stage and Reduce stage, K nodes jointly calculate Q functions s times distributed computing method, including the following steps:

[0049] A.Map stage:

[0050] Each node passes the Map function Perform calculations on local stored files to generate intermediate values ​​v corresponding to all Q functions q,n :

[0051]

[0052] in Represents a functio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a coded distributed computing method based on a Partition structure. In coded distributed computing, for N input files, K distributed computing nodes are used to calculate the values ​​of Q output functions. The whole computing process is divided into three parts: It is carried out in three stages: Map stage, Shuffle stage and Reduce stage. The method of distributed computing with K nodes jointly calculating Q functions for s times includes the following steps: A.Map stage; B.Shuffle stage; C.Reduce stage. The communication load achieved by this method is almost equal to the optimal communication load, and on this basis, the minimum requirements for the number of input files N and the number of output functions Q are significantly reduced compared with Ali's scheme.

Description

technical field [0001] The invention belongs to network communication technology, in particular to a coding distributed computing method based on a Partition structure. Background technique [0002] Driven by the rapid development of machine learning and data science, modern computing paradigms have shifted from traditional single-processor systems to large-scale distributed computing systems, and the MapReduce framework is a popular framework for distributed computing. Distributed computing shows its great advantages when dealing with large-scale data, so it has become a popular research direction in the past two years. [0003] Distributed computing is a computing method, which is relative to centralized computing. Distributed computing refers to the distribution of centralized computing on one device to distributed computing on multiple devices in the network, thus speeding up the computing process. It can handle large-scale data analysis tasks, such as machine learning...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L1/00
CPCH04L1/0076H04L1/0057
Inventor 钟逸云蒋静曲凌晓周玲玲王文汉董洪岩姬生鹏
Owner GUANGXI NORMAL UNIV
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com