Unlock instant, AI-driven research and patent intelligence for your innovation.

Task execution method and device, electronic device, storage medium

A task execution and executor technology, applied in the field of big data, can solve problems such as low job running stability, high task execution failure, wasting time, etc., to save debugging and retry time, reduce waste, and ensure stable operation.

Active Publication Date: 2020-06-19
BEIJING MEDICAL CROSS THE CLOUD TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Apache Spark is a fast and general-purpose computing engine designed for large-scale data processing. When running a Spark job, if the current executor experiences memory overflow, it needs to be retried continuously in other executors, wasting time
Moreover, during the retry process, the probability of task execution failure is high, and the stability of job operation is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task execution method and device, electronic device, storage medium
  • Task execution method and device, electronic device, storage medium
  • Task execution method and device, electronic device, storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0063] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0064] It should be noted that in the present disclosure, the terms "comprising", "configured with", and "disposed at" are used to express an open and inclusive meaning, and refer to the listed elements / components / etc. There may also be additional elements / components / etc.; the terms "first", "second", etc. are used only as labels, not as limitations on the number or order of their objects.

[0065] Spark is an engine for big data processing, a ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a task execution method and device based on Spark, electronic equipment and a computer readable storage medium, and belongs to the technical field of big data. The task execution method comprises the steps of executing a cyclic process in a Spark system until an execution state of a target task is execution success, wherein the cyclic process comprises the following steps:executing a target task in a current executor, and obtaining an execution state of the target task; when it is determined that the execution state belongs to execution failure, re-creating an actuator; and taking the newly created executor as the current executor to execute the target task. According to the task execution method, the failed tasks can be reallocated to the re-created executor, sothat the stable operation of the operation is guaranteed.

Description

[0001] This application claims the priority of the Chinese patent application with the application number 2020100757286 filed with the China Patent Office on January 22, 2020, and the title of the invention is "task performance method and device, electronic equipment, storage medium", the entire content of which is incorporated by reference In this application. technical field [0002] The present disclosure relates to the technical field of big data, and in particular to a Spark-based task execution method and device, electronic equipment, and a computer-readable storage medium. Background technique [0003] Apache Spark is a fast and general-purpose computing engine designed for large-scale data processing. When running a Spark job, if the current executor experiences memory overflow, it needs to retry continuously in other executors, wasting time. Moreover, during the retry process, the probability of task execution failure is high, and the stability of job operation is l...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/07
CPCG06F11/073G06F11/0754G06F11/0793
Inventor 龙下洋
Owner BEIJING MEDICAL CROSS THE CLOUD TECH CO LTD