Distributed resource elastic scheduling model and method, electronic equipment and storage medium

A distributed resource and resource scheduling technology, applied in the field of data processing applications, can solve problems such as inability to flexibly schedule, fail to meet real-time systems, resource scheduling failures, etc., achieve stable operation, stable and efficient work, improve resource utilization, and universal versatility Effect

Active Publication Date: 2019-11-15
GUANGDONG UNIV OF TECH
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the system-level resource scheduling of a distributed stream processing system cannot dynamically and flexibly schedule tasks according to the running status of tasks, and requires manual intervention to add or reclaim resources. That is, the system-level resource scheduling cannot perceive the running status of specific stream processing tasks, and can only be statically for resource allocation
Although task-level scheduling can be flexibly increased or decreased, it is limited by fixed physical resources
When the physical resources are insufficient, the decision made by the task-level scheduling cannot be executed, and the resource scheduling fails, which may cause the system to degrade or even crash
In addition, although system-level scheduling can accurately allocate resources at the initial stage, and manually schedule resources at runtime, it cannot achieve automatic elastic increase or decrease of physical resources. Such resource scheduling cannot meet the needs of real-time systems.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed resource elastic scheduling model and method, electronic equipment and storage medium
  • Distributed resource elastic scheduling model and method, electronic equipment and storage medium
  • Distributed resource elastic scheduling model and method, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the examples.

[0035] The invention discloses a distributed resource elastic scheduling model. The scheduling model is a two-layer resource scheduling model, which includes a task layer and a system layer. The task layer includes a task layer resource scheduling framework and a distributed stream processing system. The system layer includes a System layer resource scheduling framework, the system layer resource scheduling framework virtualizes physical resources into virtualized resources, and provides processing tasks with the task layer.

[0036] Among them, the distributed stream processing system also includes a middleware scheduler, which communicates with the system-level resource scheduling framework, dynamically adjusts its own resources, and responds to the task-level resource sch...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of data processing application. The invention discloses a distributed resource elastic scheduling model and method, electronic equipment and a storage medium. The distributed resource elastic scheduling model and method realize cooperative work of task layer dynamic resource scheduling and system layer resource scheduling, give a response to the resource demand of the stream processing task in real time, allocate reasonable physical resources, achieve automatic elastic resource scheduling in the true sense, elastically increase and decrease physical resourcesoccupied by the distributed stream processing system, meet the resource requirement that stream processing tasks change at any time, guarantee the stable operation of the distributed stream processing system and the stable and efficient work of the stream processing system, and meanwhile, increase the resource utilization rate and reduce the cost.

Description

technical field [0001] The invention relates to the application field of data processing, in particular to a distributed resource flexible scheduling model, method, electronic equipment and storage medium. Background technique [0002] With the continuous integration of technologies such as the Internet of Things, social networks, and cloud computing into our lives and the rapid development of existing computing power, storage space, and network bandwidth, various changes have been brought to people's lives. , Finance, business, medical and many other fields continue to grow and accumulate. Since the accumulation of such data often exceeds the capacity of traditional data processing technologies, it is difficult for a single machine to satisfy such a large amount of calculation, and expensive equipment is not our first choice, and the distributed stream processing mode and distributed stream processing system are thus born. As an application that processes massive real-time...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5088
Inventor 黄泽林蔡瑞初傅正佳林峰极陈炳丰温雯王立娟
Owner GUANGDONG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products