Storage method and storage system for data stream driving computing

A data flow and computing node technology, applied in the computer field, can solve the problems of lack of dynamic control, support and optimization, and no data flow-driven computing control, and achieve the effect of reducing time, optimizing task scheduling and data storage

Active Publication Date: 2019-07-26
SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
View PDF10 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] To sum up, in view of the challenges brought about by the storage bottlenecks faced by data flow-driven computing in the data center environment, the existing research mainly solves them from the perspectives of "software-defined storage" and "data flow scheduling and management". Achieved certain results, but essentially isolated cluster storage scheduling and data flow computing
At present, most of the research on hybrid software-defined storage focuses on the hot and cold data of a single program, and rarely considers the data dependencies betwe

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage method and storage system for data stream driving computing
  • Storage method and storage system for data stream driving computing
  • Storage method and storage system for data stream driving computing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] In order to make the purpose, technical solution, design method and advantages of the present invention clearer, the present invention will be further described in detail through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0049] In all examples shown and discussed herein, any specific values ​​should be construed as exemplary only, and not as limitations. Therefore, other instances of the exemplary embodiment may have different values.

[0050] Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods and devices should be considered part of the description.

[0051] According to an embodiment of the present invention, a storage system for data flow-driven computing is provided, see fig...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a storage method and a storage system for data stream driving calculation. The method comprises the steps that a task path diagram is constructed according to the incidence relation of the execution sequence of data flow driving calculation, and each task path comprises a plurality of task nodes related to input/output operation; a task path is selected as a first key path according to task execution duration based on the task path diagram; a priority label is marked for a task node on the first key path, and the priority label is scheduled and allocated to a corresponding computing node; and the computing node controls to execute the received task based on the priority label so as to realize data reading. According to the storage method and the storage system, the processing efficiency of data stream driving calculation can be improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a storage method and storage system for data flow-driven calculation. Background technique [0002] Data flow-driven computing is generally composed of a group of complex, data-dependent programs with different functions, which together complete a well-defined computing job. With the advent of big data processing technologies, dataflow-driven computing has become increasingly data-intensive. The data-intensive characteristics of data flow-driven computing determine that the I / O (input / output) operation of scheduling tasks is usually the performance bottleneck of the entire calculation, and the performance of storage access gradually surpasses the limitation of computing resources and becomes the main bottleneck restricting the efficiency of data flow computing. . In order to solve this problem, some new technologies have appeared in storage devices in recent years. Fo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5038
Inventor 王洋杜明哲
Owner SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap