HDP (Hadoop Distributed Protocol) platform deployment method based on kubernetes

A platform and basic image technology, applied in the computer field, can solve the problems of HDP platform single point of failure, the difficulty of rapid deployment of HDP platform in clusters with dynamic number of nodes, etc., to achieve the effect of improving reliability and deployment efficiency.

Active Publication Date: 2020-04-17
HANGZHOU ANHENG INFORMATION TECH CO LTD
View PDF18 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The purpose of this application is to provide a kubernetes-based HDP platform deployment method, device, equipment and readable storage medium to solve the problem that there is a single point of failure in the HDP platform, and it is difficult to quickly deploy the HDP platform in a cluster with a dynamic number of nodes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • HDP (Hadoop Distributed Protocol) platform deployment method based on kubernetes
  • HDP (Hadoop Distributed Protocol) platform deployment method based on kubernetes
  • HDP (Hadoop Distributed Protocol) platform deployment method based on kubernetes

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0042] The following introduces Embodiment 1 of a kubernetes-based HDP platform deployment method provided by this application, see figure 1 , embodiment one includes:

[0043] S101, utilize dockerfile to generate the base image of the service component in the HDP platform, the base image includes the ambari service;

[0044] S102. Obtain the yaml file generated according to the user's configuration operation on the service component;

[0045] S103. Build a kubernetes service on the target server, and use the kubernetes service to load the basic image and the yaml file, so as to implement the deployment of the HDP platform.

[0046]This embodiment is used to implement the deployment of the HDP platform. The specific deployment content includes related service components of the HDP platform, such as hdfs, hbase, zookeeper, kafka, spark, flink, storm, etc., and also includes the basic service ambari, where ambari is a big data platform monitoring management tools.

[0047] Th...

Embodiment 2

[0053] see figure 2 , embodiment two specifically includes:

[0054] S201, utilize dockerfile to generate the base image of service components in the HDP platform, the base image includes ambari service, and also includes container password and container target attribute, and the container target attribute is that the containers corresponding to the same base image are secret-free from each other;

[0055] The above process includes: adding a basic rpm package, and installing the basic rpm package. The basic image of this embodiment integrates ambari-server and ambari-agent, and the container password and self-exemption attributes are set when making the basic image, so that all containers based on the same image are mutually exempt from encryption.

[0056] S202. Add the startup instruction of the ambari service in the initialization startup instruction of the container;

[0057] S203. Modify the default configuration file of the container, modify the storage type of the c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an HDP platform deployment method, device and equipment based on kubernetes and a readable storage medium, and the method comprises the steps: generating a basic mirror image of a service module in an HDP platform through a docker file, wherein the basic mirror image comprises an ambari service; obtaining a yaml file generated according to a configuration operation of a user on the service component; and building a kubernetes service in the target server, and loading the basic mirror image and the yaml file by utilizing the kubernetes service so as to realize the deployment of the HDP platform. According to the invention, the docker file is used to construct the basic mirror image according to the basic service ambari and the HDP related service component; yaml files of all the service components are dynamically generated through configuration before installation, finally automatic deployment of the HDP platform is achieved based on kubernetes, and the deployment efficiency of the HDP platform is remarkably improved.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a kubernetes-based HDP platform deployment method, device, equipment and readable storage medium. Background technique [0002] HDP is an open source big data platform, which provides a stable, safe and reliable big data platform, and has the ability to deploy, install, configure, operate and maintain big data clusters, and has a wide range of applications. [0003] The deployment and operation and maintenance clusters are the most used for HDP. When deploying clusters, most people use the blueprint solution. The blueprint solution is to plan the number and configuration of each service component in advance, such as 3 datanodes, 6 nodemagers, etc. Then apply this blueprint through the ambari-api during installation, and you can install the big data cluster according to the distribution method of the service components set in the blueprint. [0004] However, in the p...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/61
CPCG06F8/63Y02D10/00
Inventor 吴波范渊
Owner HANGZHOU ANHENG INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap