Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Task processing method based on computer cluster

A computer cluster and task processing technology, which is applied to data error detection, response error generation, resource allocation, etc. in the direction of redundancy in computing, and can solve problems such as extra expenditure on system hardware performance, waste of resources, and no good solutions. , to achieve the effect of increasing task running speed, improving task processing efficiency, and reducing resource waste

Inactive Publication Date: 2014-08-06
CHNA ENERGY INVESTMENT CORP LTD
View PDF5 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, when the entire mesh processing node fails, the traditional distributed task scheduling technology cannot restart processing from the current mesh processing node, and can only repeatedly execute the entire task chain from the initial state, which still cannot solve the problem of task processing efficiency. Problems of reduction, waste of resources, and additional expenditure on system hardware performance
[0004] For the above technical problems, there is no good solution in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task processing method based on computer cluster
  • Task processing method based on computer cluster

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0029] In one embodiment, it can be realized by using the task chain development mode and the command mode with the HADOOP cloud computing platform. Its specific implementation is as follows:

[0030] (1) According to business requirements (processing sequence), the task processing chain is divided into functions;

[0031] (2) Determine the sequence and dependencies of tasks in the task chain;

[0032] (3) Register the unified interface, and inherit the interface to implement each task in detail;

[0033] (4) Select multiple (for example, 5 to 10) common servers (LINUX operating system) as nodes for calculation and processing of report data;

[0034] (5) Deploy the HADOOP platform on these common servers, and combine these machines into a distributed computing cluster;

[0035] (6) Initialize the distributed file system (HDFS) on the distributed computing cluster;

[0036] (7) Push the data files required by the processing chain to the HADOOP distributed file system (HDFS); ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a task processing method based on a computer cluster. The task processing method based on the computer cluster comprises the steps that each task node in a task chain is divided into multiple sub task nodes; the calculation needing to be finished by the sub task nodes is distributed to multiple computer nodes in the computer cluster for calculation processing; e state snapshots of calculation processing of the multiple computer nodes are stored; when the sub task nodes are interrupted, the state of the sub task nodes before interruption is recovered according to the state snapshots, and the interrupted sub task nodes continue to be executed. Through the task processing method based on the computer cluster, the calculation of the task nodes is distributed to the multiple computer nodes for processing, the tasks can continue to be executed after being interrupted instead of being executed from the initial state of the task chain, and therefore the task processing efficiency is largely improved.

Description

technical field [0001] The invention relates to the field of computer applications, in particular to a task processing method based on computer clusters. Background technique [0002] The processing of task chains is generally based on linear processing at present, using traditional non-distributed computing technology. Traditional linear processing cannot structurally decompose the processing chain. Task processing can only start from the initial state to the end of the operation, and it is difficult to perform parallel computing and other optimization measures in the middle. At the same time, due to strong dependence, if the processing is interrupted for some reason during the entire task processing process, the previously processed resources cannot be applied, and the task can only be executed repeatedly from the initial state, resulting in reduced task processing efficiency and waste of resources. and additional expenditures on system hardware performance. [0003] The...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50G06F11/14
Inventor 王亮张克慧谭映忠刘新宇刘畅关丹凤陈璇郭磊
Owner CHNA ENERGY INVESTMENT CORP LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com