Cross-cloud distributed data task scheduling method and device and storage medium

A distributed data and task scheduling technology, applied in the field of electronic information, can solve the problems of complex implementation and low efficiency, and achieve the effect of improving flexibility, comprehensive coverage, and improving scalability.

Pending Publication Date: 2021-08-06
西安易联趣网络科技有限责任公司
View PDF11 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to overcome the shortcomings of complex and inefficient implementation of job scheduling and dependency management in big data processing in the above-mentioned prior art, and provide a cross-cloud distributed data task scheduling method, equipment and storage medium

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-cloud distributed data task scheduling method and device and storage medium
  • Cross-cloud distributed data task scheduling method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0021] It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of electronic information, and discloses a cross-cloud distributed data task scheduling method and device and a storage medium, and the method comprises the following steps: obtaining a workflow of a data task, and analyzing the workflow to obtain a plurality of jobs with a dependency relationship; analyzing the plurality of jobs in sequence according to the dependency relationship to obtain working node server addresses of the jobs, and sending the jobs to corresponding working node servers according to the working node server addresses; employing the job for triggering the work node server to analyze the job to obtain the job content, the job type, the calling key and the cloud platform category of the job, generating an executor according to the job type, calling the cloud platform corresponding to the cloud platform category through the executor and the calling key to execute the job content, obtaining an execution result and sending the execution result; and receiving an execution result sent by the working node server. Cross-cloud processing of data tasks is realized, the problem that an existing scheduling system cannot cross multiple cloud platforms is solved, and the flexibility is greatly improved.

Description

technical field [0001] The invention belongs to the technical field of electronic information, and relates to a cross-cloud distributed data task scheduling method, equipment and storage medium. Background technique [0002] Big data processing is currently a very common technical means in all walks of life. However, with the increase in the amount of data and business volume of technology companies in various industries, big data tasks have shown the following characteristics: the amount of data is getting larger and larger, and the processing of data There are more and more job tasks and the relationship is complicated. With the popularity of public cloud, data storage locations are diversified, including local storage, public cloud storage, and private cloud storage, etc.; and data jobs depend on different local environments, scheduling tasks Execution machines become diverse. [0003] Faced with the above characteristics, the scheduling of data job tasks has become extr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/46
CPCG06F9/4881G06F9/465
Inventor 刘周龙刘敬帅
Owner 西安易联趣网络科技有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap