Method and device for executing computing tasks

A technology for computing tasks and continuous execution, applied in the computer field, can solve problems such as lack of high availability, inability to take advantage of high-performance nodes, single point of failure, etc., and achieve the effects of high availability, availability, and accuracy.

Pending Publication Date: 2020-06-12
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In the process of realizing the present invention, the inventor found that there are at least the following problems in the prior art: the Spark on YarnClient mode has a single point of failure problem and does not have high availability; while the Spark on Yarn Cluster mode executes computing tasks by any node , cannot take advantage of the high performance of the node

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for executing computing tasks
  • Method and device for executing computing tasks
  • Method and device for executing computing tasks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0033] It should be pointed out that, in the case of no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0034] Spark (Spark is a fast and general-purpose computing engine designed for large-scale data processing. Now it has formed a rapidly developing and widely used ecosystem. Spark is a general-pur...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and device for executing a computing task, and relates to the technical field of computers. One specific embodiment of the method comprises the following steps: sending a calculation instruction to a main node, the calculation instruction carrying a calculation task, and receiving original data returned by the main node and generated by executing the calculation task; if it is monitored that the master node cannot execute the calculation task, selecting a new master node from slave nodes according to a preset rule; and sending a new calculation instruction to the new main node, the new calculation instruction carrying the original data, and receiving the original data returned by the new main node and generated by continuously executing the calculation taskaccording to the original data. According to the embodiment, the high-performance advantage and the high availability of the main node are ensured at the same time.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method and device for performing computing tasks. Background technique [0002] Spark is a computing engine. For Spark clusters, the deployment method is mainly Spark on Yarn, and the Spark on Yarn deployment method is divided into two modes: Cluster (cluster) mode and Client (client) mode. [0003] In the process of realizing the present invention, the inventor found that there are at least the following problems in the prior art: the Spark on YarnClient mode has a single point of failure problem and does not have high availability; while the Spark on Yarn Cluster mode executes computing tasks by any node , cannot take advantage of the high performance of the node. Contents of the invention [0004] In view of this, the embodiments of the present invention provide a method and an apparatus for executing computing tasks, which can simultaneously ensure the high per...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/20
CPCG06F11/203
Inventor 姚思雨
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap