Big data cluster deployment method capable of adapting to hardware configuration

A technology of hardware configuration and cluster deployment, applied to electrical components, transmission systems, etc., can solve problems such as error-prone and low efficiency, and achieve the effect of reducing workload, saving time and cost, and facilitating unified management and operation and maintenance

Inactive Publication Date: 2016-09-21
CHINA ELECTRONICS TECH CYBER SECURITY CO LTD
View PDF2 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The traditional manual deployment method requires repeated manual modification, which is inefficient and error-prone

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data cluster deployment method capable of adapting to hardware configuration
  • Big data cluster deployment method capable of adapting to hardware configuration
  • Big data cluster deployment method capable of adapting to hardware configuration

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] A big data cluster deployment method with adaptive hardware configuration, such as figure 1 shown, including the following steps:

[0017] Step 1, puppet-agent sends a synchronization request to puppet-master;

[0018] Step 2, puppet-master authenticates puppet-agent:

[0019] If it is the first time to request synchronization, the puppet-master needs to authenticate the puppet-agent. This embodiment adopts the automatic registration authentication mode, and the authentication is automatically completed after the synchronization request is issued. The automatic registration method needs to delete the existing certificates of the puppet master and agent side, and then modify the puppet configuration file autosign.conf in the puppet-master, and change the content to: *. In this way, after the synchronization request is issued, the authentication can be completed automatically.

[0020] Step 3: puppet-agent detects server hardware configuration information and sends it...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a large data cluster deployment method for self-adaptive hardware configuration, which sends a synchronization request to a puppet-master through a puppet-agent, detects server hardware configuration information and sends it to the master after the authentication is passed; the master generates a pseudo code and sends it to the agent; The agent executes the pseudo-code content belonging to its own node, completes the automatic deployment of the cluster, and ends this synchronization. The present invention realizes the automatic batch deployment of the big data environment. From hardware information detection and acquisition to software installation and configuration file generation, all are programmed, without the tedious work of manually installing software and changing configuration files. It is a kind of big data with adaptive hardware configuration. Cluster automation deployment method. This method reduces the workload of building and deploying big data clusters, saves the time and cost of cluster deployment, and facilitates subsequent unified management and operation and maintenance.

Description

technical field [0001] The invention relates to a large data cluster deployment method for self-adaptive hardware configuration. Background technique [0002] Big data clusters include data batch processing and real-time stream processing frameworks, as well as authentication and authority control systems, involving many components, such as hadoop, hive, kafka, storm, spark, zookeeper, kerberos, sentry, etc., and different There may be interdependencies between components. The deployment of big data clusters mainly includes the installation of these components and the configuration of dependencies. [0003] Puppet is a cluster configuration management tool. The tool generally adopts the host-agent architecture, and uses its built-in puppet modeling language to describe the final state and dependencies of cluster resources, including user accounts, specific files and folders, software packages and services, etc., to complete automated cluster deployment and configuration. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08H04L29/06
CPCH04L67/1095H04L63/08
Inventor 唐明常梦楠任红雷
Owner CHINA ELECTRONICS TECH CYBER SECURITY CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap