Compilation of graph-based program specifications with automated clustering of graph components based on the identification of particular data port connections

A component and specification technology, applied in the field of compiling graph-based program specifications, can solve problems such as calculation mismatch and difficulty in adaptation, and achieve the effect of improving calculation efficiency and realizing efficiency gains

Active Publication Date: 2017-08-18
INITIO TECH
View PDF8 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although it is possible to efficiently perform computations that can be represented in the MapReduce framework and whose inputs and outputs can be modified for storage in the MapReduce framework's file system, many computations do not fit the framework and / or are not easily adaptable so that all of them Both input and output are in the distributed file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Compilation of graph-based program specifications with automated clustering of graph components based on the identification of particular data port connections
  • Compilation of graph-based program specifications with automated clustering of graph components based on the identification of particular data port connections
  • Compilation of graph-based program specifications with automated clustering of graph components based on the identification of particular data port connections

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] refer to figure 1 , the task-based computing system 100 uses the high-level program specification 110 to control the computing resources and storage resources of the computing platform 150 to perform calculations specified by the program specification 110 . Compiler / interpreter 120 receives high-level program specification 110 and generates task-based specification 130 in a form executable by task-based runtime interface / controller 140 . The compiler / interpreter 120 recognizes one or more "execution sets" of one or more "components", which may be instantiated individually or as a unit as fine-grained tasks to be applied to each of a plurality of data elements . As described in more detail below, part of the compilation or interpretation process involves identifying these execution sets and preparing them for execution. It should be understood that compiler / interpreter 120 may use any of a variety of algorithms including steps such as parsing high-level program specifi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a compilation of graph-based program specifications with automated clustering of graph components based on the identification of particular data port connections. A graph-based program specification (628) includes: a plurality of components, each corresponding to a processing task and including one or more ports, including scalar data ports for sending or receiving a single data element and collection data ports for sending or receiving a collection of multiple data elements; and one or more links, each connecting an output port of an upstream component to an input port of a downstream component. Prepared code is generated representing subsets (630, 631) of the plurality of components, including: identifying one or more subset boundaries, including identifying one or more links connecting a collection data port of a component to a scalar data port of a component; forming the subsets based on the identified subset boundaries; and generating prepared code for each formed subset that when used for execution by a runtime system causes processing tasks corresponding to the components in each formed subset to be performed.

Description

[0001] Cross References to Related Applications [0002] This application claims priority to US Application Serial No. 62 / 044,645, filed September 2, 2014, and US Application Serial No. 62 / 164,175, filed May 20, 2015. technical field [0003] This description relates to a method for compiling a graph-based program specification. Background technique [0004] One approach to dataflow computation utilizes a graph-based representation in which computational components corresponding to the nodes (vertices) of the graph pass through dataflows corresponding to the links (directed edges) of the graph (called a "dataflow graph"). )coupling. A downstream component connected to an upstream component by a dataflow link receives an ordered stream of input data elements and processes the input data elements in the order received, optionally generating one or more corresponding streams of output data elements. A system for performing such graph-based computations is described in prior U...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/45G06F9/50G06F9/44
CPCG06F8/433G06F8/447G06F9/5066G06F13/36G06F8/34G06F8/456G06F9/4494G06F16/22G06F16/9024G06F8/41
Inventor C·W·斯坦菲尔R·夏皮罗S·A·库克利希
Owner INITIO TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap