Method and device for constructing Hadoop application development frame, and electronic device

A technology of application development and framework, applied in the direction of program control devices, etc., can solve the problems of repetitive, cumbersome, and low efficiency of tasks, and achieve the effect of improving efficiency and improving utilization efficiency

A technology of application development and framework, applied in the direction of program control devices, etc., can solve the problems of repetitive, cumbersome, and low efficiency of tasks, and achieve the effect of improving efficiency and improving utilization efficiency

CN105045602AActive Publication Date: 2015-11-11BEIJING KINGSOFT INTERNET SECURITY SOFTWARE CO LTD
6 Cites 15 Cited by

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for constructing Hadoop application development frame, and electronic device
  • Method and device for constructing Hadoop application development frame, and electronic device
  • Method and device for constructing Hadoop application development frame, and electronic device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] The embodiments of the present invention will be described in detail below in conjunction with the drawings.

[0066] It should be clear that the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0067] figure 1 It is a schematic flow diagram of a method for constructing a Hadoop application development framework according to an embodiment of the present invention. See figure 1 , The method includes:

[0068] Step 101: Separate the command line to obtain task scheduling and task parameters for setting a configuration template file, the configuration template file containing a scheduling configuration file for encapsulating separated scheduled tasks and a task list configuration for encapsulating separated ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a method and a device for constructing a Hadoop application development frame, and an electronic device. The method comprises the following steps: separating a command line to obtain task scheduling and task parameters used for setting a template configuration file, wherein the template configuration file comprises a scheduling configuration file used for packaging separated scheduler tasks and a task list configuration file used for packaging separated task parameters; setting a task scheduling sequence which carry out visual editing in the scheduling configuration file; setting a visual editing task parameter in the task list configuration file, and constructing a mapping relationship between the scheduler tasks in the scheduling configuration file and the task parameters; and setting a task parameter detection file which starts the task in the scheduling configuration file and detects the validity of the task list configuration file, and constructing the Hadoop application development frame on the basis of the template configuration file and the task parameter detection file. The method and the device can be applied to improve the construction efficiency of the Hadoop application development frame.

Description

Technical field [0001] The present invention relates to computer application technology, in particular to a method, device and electronic device for constructing a Hadoop application development framework. Background technique [0002] The Hadoop application development framework is a software framework that uses distributed file system (HDFS, HadoopDistributedFileSystem) technology to perform distributed processing of large amounts of data (tasks). By maintaining multiple copies of data (tasks), it can ensure that it can deal with failures. The data is redistributed and processed, so it has high fault tolerance and can provide high throughput for data (task) access, which is widely used in application scenarios that require very large data sets. Using the Hadoop application development framework, users can develop distributed programs without understanding the underlying details, so that they can make full use of the high throughput of the Hadoop application development framewor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
11 Nov 2015
Publication
CN105045602A
IPC
G06F9/44
Inventors
潘洪安