Task migration method and device, electronic equipment and storage medium

A task and task conversion technology, applied in the field of data processing, can solve problems such as low efficiency and consuming the workload of business R&D personnel, and achieve the effect of reducing workload, improving offline computing requirements, and improving compatibility.

Pending Publication Date: 2020-08-07
BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In the current processing method, business R&D personnel are usually required to manually c

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task migration method and device, electronic equipment and storage medium
  • Task migration method and device, electronic equipment and storage medium
  • Task migration method and device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0042] In addition, it should be understood that the term "and / or" in this article is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and / or B may mean: A exists alone, and A exists at the same time. and B, there are three cases of B alone. In addition, the character " / " in this articl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a task migration method and device, electronic equipment and a storage medium, and relates to the field of data processing, and the method can comprise the steps: obtaining a task submitted by a user; if the task is a Hadoop task and it is determined that task conversion needs to be conducted, convertingHadoop parameters in the task into parameters capable of being recognized by Spark; and injecting the conversion result into a predetermined toolkit, and submitting the conversion result to a Spark cluster. By applying the scheme provided by the invention, the workload of a user can be reduced, and the processing efficiency is improved.

Description

technical field [0001] The present application relates to computer application technology, in particular to a task migration method, device, electronic equipment and storage medium in the field of data processing. Background technique [0002] The computing power and performance of the Spark model are better than the Hadoop model. Hadoop is a distributed system that makes full use of the power of clusters for high-speed computing and storage. Spark is a fast, general-purpose engine for massive data processing. [0003] In the traditional way, many offline tasks are written and run using the Hadoop model. Considering the advantages of the Spark model, it is hoped that Hadoop tasks can be migrated, that is, Hadoop tasks can be converted into Spark tasks. [0004] In the current processing method, business R&D personnel are usually required to manually perform task conversion, but this method requires a large workload of the business R&D personnel and is inefficient. Content...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/48G06F8/30
CPCG06F9/4806G06F8/30G06F9/541G06F9/5066G06F9/4856G06F9/5072G06F9/4875G06F9/4881
Inventor 郝海蛟吕旭光赵建新刘林宏李豪
Owner BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap