Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Task hybrid scheduling method and system and computer readable storage medium

A technology of task mixing and scheduling method, which is applied in the direction of multi-program device, program startup/switching, resource allocation, etc., which can solve the problems of incomplete resource isolation, unbalanced scheduling, etc., and achieve improved performance, balanced scheduling, and balanced promotion sexual effect

Pending Publication Date: 2022-07-01
ZHEJIANG DAHUA TECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the Hadoop system, the on Yarn scheme can be used, that is, all tasks are scheduled by Yarn, and the task process runs on the node manager (NodeManager) node. This scheme makes the central processing unit (central processing unit, CPU) etc. Resource isolation is not complete, resulting in multiple processes may run on a physical machine; some solutions use the onKubernetes solution, in which tasks run in the form of container groups (Pods), which can achieve resource isolation, but not all distributed frameworks support it on Kubernetes scheme; some schemes use the hybrid scheduling architecture of Yarn and Kubernetes, but this scheme will lead to unbalanced scheduling

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task hybrid scheduling method and system and computer readable storage medium
  • Task hybrid scheduling method and system and computer readable storage medium
  • Task hybrid scheduling method and system and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0014] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is particularly pointed out that the following examples are only used to illustrate the present application, but do not limit the scope of the present application. Similarly, the following embodiments are only some of the embodiments of the present application, but not all of the embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0015] Reference in this application to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a task hybrid scheduling method and system and a computer readable storage medium. The method comprises the steps of obtaining a to-be-scheduled task; a Kubernetes controller and a Yarn container group resource are created, the Kubernetes controller is adopted to process the Yarn container group resource to obtain a container group resource, and the task to be scheduled corresponds to the Yarn container group resource; a Kubernetes controller is adopted to schedule container group resources so as to schedule the container group resources to a preset physical machine node, a data node container group is deployed on the preset physical machine node, and the data node container group is a container group served by a data node in the HDFS cluster. In this way, the balance of task scheduling can be improved.

Description

technical field [0001] The present application relates to the technical field of task scheduling, and in particular to a task hybrid scheduling method, system and computer-readable storage medium. Background technique [0002] The distributed scheduling framework is an important part of the big data system, which is responsible for the distribution and scheduling of various tasks, such as: using Yet Another Resource Negotiator (Yarn) or Kubernetes. In the Hadoop system, the on Yarn scheme can be used, that is, all tasks are scheduled by Yarn, and the task process runs on the node manager (NodeManager) node. This scheme makes the central processing unit (CPU), etc. The resource isolation is not complete, resulting in multiple processes running on one physical machine; some solutions use the onKubernetes solution, in which tasks are run in the form of container groups (Pods), which can achieve resource isolation, but not all distributed frameworks support it on Kubernetes sch...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/455G06F9/50
CPCG06F9/4881G06F9/5083G06F9/45558G06F9/5066G06F2009/45562G06F9/5027G06F2209/502G06F9/5077G06F9/5088G06F2209/503
Inventor 阮良
Owner ZHEJIANG DAHUA TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com