Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method, device and system for realizing task scheduling

A task scheduling and task processing technology, applied in the field of data processing, can solve problems such as insufficient computing cluster resources and low utilization of computing cluster resources, and achieve the effect of improving operating efficiency

Pending Publication Date: 2020-05-22
THE FOURTH PARADIGM BEIJING TECH CO LTD
View PDF7 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If the client only submits data processing tasks to the same computing cluster each time, it may cause insufficient resources of the computing cluster while other computing clusters are idle, making the overall resource utilization of the computing cluster low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device and system for realizing task scheduling
  • Method, device and system for realizing task scheduling
  • Method, device and system for realizing task scheduling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0089] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0090] The following description of at least one exemplary embodiment is merely illustrative in nature and in no way taken as limiting the invention, its application or uses.

[0091] Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods and devices should be considered part of the description.

[0092] In all examples shown and discussed herein, any specific values ​​should be construed as exemplary only, and not as limitations. Therefore, other instances of the exemplary embodiment may have dif...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method, a device and a system for realizing task scheduling. The method comprises the following steps: acquiring a target data processing task submitted by a client and running parameters of the target data processing task; obtaining resource use information of the available computing cluster; selecting a target computing cluster from available computing clusters accordingto the running parameters of the target processing task and the resource use information of the available computing clusters; and scheduling the target data processing task to a target computing cluster for running.

Description

technical field [0001] The present invention relates to the technical field of data processing, and more specifically, to a method for implementing task scheduling, a device for implementing task scheduling, a system including at least one computing device and at least one storage device, and a readable storage device. medium. Background technique [0002] Spark is currently the mainstream open source big data processing framework, which is widely used in big data processing tasks such as data cleaning, data exploration, and feature extraction for machine learning. The computing clusters currently supported by Spark for scheduling data processing tasks include Kubernetes computing clusters, YARN computing clusters, Mesos computing clusters, Standalone computing clusters, and Local computing clusters. [0003] When users submit data processing tasks through the client, they need to specify the computing cluster running the data processing tasks through the "--master" and "--...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/50G06N20/00
CPCG06F9/4881G06F9/5038G06N20/00
Inventor 陈迪豪包新启王太泽范晓亮陈靓穆妮王子贤
Owner THE FOURTH PARADIGM BEIJING TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com