Unlock instant, AI-driven research and patent intelligence for your innovation.

Job diagnosis method and equipment for mapreduce distributed system

A distributed system and diagnostic method technology, applied in the field of job diagnosis of MapReduce distributed system, can solve problems such as error-prone, complex operation, reduced efficiency and reliability of job diagnosis, and achieve high accuracy rate, simple operation, and improved efficiency and reliability effects

Active Publication Date: 2019-05-24
BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, during the operation of the job, many job diagnoses, such as the monitoring of the running status of the job and the adjustment of the operation control parameters of the job, require manual control by the operator. The operation is complicated and error-prone, which leads to the efficiency and reliability of job diagnosis. decrease in sex

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Job diagnosis method and equipment for mapreduce distributed system
  • Job diagnosis method and equipment for mapreduce distributed system
  • Job diagnosis method and equipment for mapreduce distributed system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0041] In addition, the term "and / or" in this article is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and / or B, which may mean: A exists alone, A and B exist at the same time, There are three cases of B alone. In addition, the character " / " in this article genera...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an operation diagnosis method and device for a MapReduce distributed system. According to the method, object identifiers of an object to be diagnosed, in the MapReduce distributed system is acquired; the object identifiers include a cluster identifier or include a cluster identifier and an operation identifier; at least one operation log file is acquired via the object identifiers; accordingly, at least one operation can be diagnosed according to the log files and pre-configured diagnostic rules so as to acquire diagnosis results. The whole process is automated, operators need not perform manual control, operation is simple, accuracy is high, and efficiency and reliability in operation diagnosis are improved.

Description

【Technical field】 [0001] The invention relates to communication technology, in particular to a job diagnosis method and equipment for a MapReduce distributed system. 【Background technique】 [0002] In recent years, with the rapid development of broadband network technology and parallel computing theory, a more simplified distributed system, the MapReduce (MapReduce) distributed system, emerged to provide services for various applications, for example, for search engines Provide services. In the MapReduce distributed system, it can also become a MapReduce distributed cluster, for example, in the Hadoop system, a data processing process is called a job (Job). After the Job is submitted, the data to be processed is divided into N parts, each to be processed The data is processed by a mapping (Map) task. The Map task runs on a node device in the MapReduce distributed system. One or more Map tasks can be run on a node device; the output results of all Map tasks are aggregated (R...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/182
CPCG06F16/182
Inventor 杨斐刘旭
Owner BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD