Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

High-availability method and system for MapReduce task, terminal and storage medium

A technology of tasks and storage paths, applied in transmission systems, digital transmission systems, hardware monitoring, etc., can solve problems such as waste of manpower, time, waste of cluster resources, untraceable reasons, etc., to ensure product quality, save human resources, and design principle reliable effect

Inactive Publication Date: 2020-05-19
INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF0 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Currently, MapReduce tasks are executed on the data nodes, but the data nodes are downtime, the network is unavailable, poor performance, etc. will affect the task execution and cause the execution to fail. If there is a downtime, there is a risk that there will be no logs to check; the cause cannot be traced, and waste Manpower, time, waste of cluster resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • High-availability method and system for MapReduce task, terminal and storage medium
  • High-availability method and system for MapReduce task, terminal and storage medium
  • High-availability method and system for MapReduce task, terminal and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0044] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0045] figure 1 is a schematic flowchart of a method in one embodiment of the present invention. in, figure 1 The execution subject can be a highly available system of MapReduce tasks.

[0046] Such as figure 1 As shown, the method 100 includes:

[0047] Step 110, selecting a backup node from the cluster and storing the backup node...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a high-availability method and system for a MapReduce task, a terminal and a storage medium. The method comprises the steps of selecting a standby node from a cluster and storing the standby node; monitoring the state of an execution node of the MapReduce task; and if it is monitored that the state of the execution node is abnormal, forwarding the task of the abnormal execution node to a standby node. According to the invention, the method can ensure that the task is normally executed and is not interrupted; manpower resources are saved; the product quality is ensured; alog is used as a record in the whole process; and subsequent disk recovery is quick and convenient.

Description

technical field [0001] The present invention relates to the technical field of big data Insight platform, in particular to a highly available method, system, terminal and storage medium for MapReduce tasks. Background technique [0002] In the big data Insight platform, MapReduce is one of its core components. If Insight implements distribution, it needs to include two parts, one is the distributed file system HDFS, and the other is the distributed computing framework MapReduce, both of which are indispensable. The key that Insight components rely on to execute tasks is MapReduce, so MapReduce is the top priority for big data platforms. Currently, MapReduce tasks are executed on the data nodes, but the data nodes are downtime, the network is unavailable, poor performance, etc. will affect the task execution and cause the execution to fail. If there is a downtime, there is a risk that there will be no logs to check; the cause cannot be traced, and waste Manpower and time ar...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/24H04L29/08H04L12/26G06F11/34
CPCG06F11/3409H04L41/0663H04L43/08H04L67/10
Inventor 道玉明张东东
Owner INSPUR SUZHOU INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com