Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A storage method and storage system for data flow-driven computing

A data flow and computing node technology, applied in the computer field, can solve problems such as lack of flexibility, lack of dynamic control, affecting the overall execution performance of computing, and achieve the effect of optimizing data storage and reducing time

Active Publication Date: 2021-08-27
SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] To sum up, in view of the challenges brought about by the storage bottlenecks faced by data flow-driven computing in the data center environment, the existing research mainly solves them from the perspectives of "software-defined storage" and "data flow scheduling and management". Achieved certain results, but essentially isolated cluster storage scheduling and data flow computing
At present, most of the research on hybrid software-defined storage focuses on the hot and cold data of a single program, and rarely considers the data dependencies between programs to optimize storage, and does not provide special control, support and support for data flow-driven computing. optimization
In addition, the management and scheduling of data flow lacks flexibility in the perception and response strategies of the computing environment. The realization of perception strategies is mostly pre-defined in the system or middleware, and the lack of flexible dynamic control increases the difficulty of optimizing in response to environmental changes. , thus affecting the overall execution performance of the calculation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A storage method and storage system for data flow-driven computing
  • A storage method and storage system for data flow-driven computing
  • A storage method and storage system for data flow-driven computing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] In order to make the purpose, technical solution, design method and advantages of the present invention clearer, the present invention will be further described in detail through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0049] In all examples shown and discussed herein, any specific values ​​should be construed as exemplary only, and not as limitations. Therefore, other instances of the exemplary embodiment may have different values.

[0050] Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods and devices should be considered part of the description.

[0051] According to an embodiment of the present invention, a storage system for data flow-driven computing is provided, see fig...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a storage method and a storage system for data flow-driven computing. The method includes: constructing a task path graph according to the association relationship of the execution sequence of the data flow-driven calculation, wherein each task path includes a plurality of task nodes involved in input / output operations; based on the task path graph, selecting according to the task execution duration The task path is used as the first critical path; the task nodes on the first critical path are marked with priority labels and scheduled and assigned to the corresponding computing nodes; the computing nodes control the execution of the received tasks based on the priority labels, so as to achieve data read. The storage method and storage system of the present invention can improve the processing efficiency of data stream-driven computing.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a storage method and storage system for data flow-driven calculation. Background technique [0002] Data flow-driven computing is generally composed of a group of complex, data-dependent programs with different functions, which together complete a well-defined computing job. With the advent of big data processing technologies, dataflow-driven computing has become increasingly data-intensive. The data-intensive characteristics of data flow-driven computing determine that the I / O (input / output) operation of scheduling tasks is usually the performance bottleneck of the entire calculation, and the performance of storage access gradually surpasses the limitation of computing resources and becomes the main bottleneck restricting the efficiency of data flow computing. . In order to solve this problem, some new technologies have appeared in storage devices in recent years. Fo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5038
Inventor 王洋杜明哲
Owner SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com