Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and system for automatically deploying Hadoop ecosphere components of operation and maintenance

A component and ecological technology, applied in special data processing applications, instruments, electrical digital data processing, etc., to achieve the effect of increasing deployment efficiency, simple invocation, and saving labor costs

Active Publication Date: 2018-09-18
TAIHUA WISDOM IND GRP CO LTD
View PDF11 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of this, the present invention provides a method and system for automatically deploying and maintaining components of the Hadoop ecosystem, which solves the technical problem of automatically deploying and maintaining the components of the Hadoop ecosystem in the cluster

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for automatically deploying Hadoop ecosphere components of operation and maintenance
  • Method and system for automatically deploying Hadoop ecosphere components of operation and maintenance

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0056] The present invention provides a method for automatically deploying and maintaining Hadoop ecological circle components, which can realize automatic deployment and maintenance of Hadoop ecological circle components for hosts in the cluster, figure 1 It is a flow chart of the method for automatically deploying and maintaining Hadoop ecosystem components provided by Embodiment 1 of the present invention. Such as figure 1 Shown, the method provided by the invention comprises:

[0057] Step S101: selecting a host in the cluster as the master node, and the remaining hosts in the cluster as slave nodes;

[0058] Step S102: The master node calls a preset configuration environment method script, and the configuration environment method script generates multiple single-node configuration function scripts according to the requirements of the deployment environment, wherein the deployment environment is the host in the cluster In the working environment, the host in the cluster ...

Embodiment 2

[0075] Embodiment 2 of the present invention provides a method for automatically deploying and maintaining Hadoop ecosystem components, the method comprising:

[0076] Select a host in the cluster as the master node, and the remaining hosts in the cluster as slave nodes;

[0077] The master node calls a preset configuration environment method script, and the configuration environment method script generates multiple single-node configuration function scripts according to the requirements of the deployment environment, wherein the deployment environment is the working environment of the hosts in the cluster, The host in the cluster executes a single-node configuration function script, which can satisfy a basic function of deploying Hadoop ecosystem components;

[0078] Wherein, the single node configuration function script includes: disable SElinux policy script, disable TransparentHuge Pages script, directory default access authority configuration script, modify linux maximum ...

Embodiment 3

[0094] The present invention also provides a system for automatically deploying and maintaining components of the Hadoop ecosystem, capable of automatically deploying and maintaining the components of the Hadoop ecosystem on hosts in the cluster of the system. figure 2 A block diagram of a system for automatically deploying and maintaining Hadoop ecosystem components provided by Embodiment 3 of the present invention. Such as figure 2 As shown, the system provided by the present invention includes a cluster composed of multiple hosts, the hosts in the cluster are connected through a network, and the cluster includes a master node host and multiple slave node hosts, wherein,

[0095] The master node host includes a memory and a processor, wherein,

[0096] The memory is used to store cluster-free login scripts, configuration environment method scripts, single-node environment configuration scripts, remote distribution scripts, yum source configuration scripts, deployment and ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a system for automatically deploying Hadoop ecosphere components of operation and maintenance. The method comprises the steps of selecting a host as a master nodeand the rest of hosts as slave nodes; generating a single node configuration function script in the master node; according to the single node configuration function script, generating a single node environment configuration script; executing the single node environment configuration script in the master node; executing a cluster password-free login script by the master node; calling a remote distribution script to distribute the single node environment configuration script to the slave nodes by the master node; controlling the slave nodes to execute the single node environment configuration script by the master node; calling a yum source configuration script by the master node; executing a deployment installation script by the master node, and installing the Hadoop ecosphere components inthe master node and the slave nodes; and executing a preset management starting script to perform stop, starting and restarting management operations on the Hadoop ecosphere components by the master node. According to the method and the system, manual deployment configuration is replaced with automated scripts, so that the efficiency of deploying the Hadoop ecosphere components on a large scale isimproved and the later operation and maintenance are simple.

Description

technical field [0001] The present invention relates to the technical field of computer applications, and more specifically, to a method and system for automatically deploying and maintaining Hadoop ecosystem components. Background technique [0002] With the continuous application and popularization of Internet big data technology, the amount of data is growing exponentially, doubling about every two years, which means that the amount of data generated in the last two years is more than the sum of all the data generated before. Many, for such a huge amount of data related to its storage, analysis, circulation, etc., a series of related problems have arisen, so the big data platform came into being. Hadoop has become an important part of the current big data processing data platform. With the growth of business volume and business data volume, it is impossible to rely solely on manual deployment after tens of thousands of Hadoop machines need to be deployed in a cluster sca...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 靳家栋辛国茂马述杰
Owner TAIHUA WISDOM IND GRP CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com