Hadoop cluster system and Hadoop cluster rapid establishment method based on container technology

Abstract

The invention discloses a Hadoop cluster system and Hadoop cluster rapid establishment method based on a container technology and relates to the technical field of big data and cloud computing. The method comprises the steps of configuring that containers between different container operation nodes communicate through a VXLAN, and configuring that virtual network bridges of the container operation nodes are not located in the same network segment; installing SSH and JAVA operating environments, producing packaging files of a DNS service mirror image, a Hadoop master service mirror image and hadoop slave service mirror images, packaging the mirror images, and transmitting the packaged mirror images to a mirror image warehouse for storage; and submitting requests for deploying three pieces of services to a container management node in sequence by a user through a client, scheduling the corresponding requests to the container operating nodes for deployment by the container management node, and registering the corresponding service. According to the system and the method, a Hadoop cluster can be deployed rapidly, the condition that the number of servers is insufficient is solved, and the availability of a Hadoop container cluster deployed on a Docker platform is improved.

Description

technical field [0001] The invention relates to the technical fields of big data and cloud computing, in particular to a container technology-based Hadoop cluster system and a rapid construction method. Background technique [0002] In the field of big data technology, big data technology represented by the Hadoop ecosystem has been used more and more after years of development. Software developers often need to develop and debug multi-node Hadoop clusters. One of the problems they often encounter is that the number of servers in the R&D environment is not enough. Similarly, testers and operation and maintenance personnel often need to quickly build a multi-node Hadoop cluster environment when verifying Hadoop cluster functions, and they will also encounter insufficient number of servers and cumbersome configuration of the deployment environment, resulting in low timeliness Case. In addition, for data analysts, there is often a need to quickly deploy Hadoop clusters for da...

AI Technical Summary

Problems solved by technology

[0005] For the defects existing in the prior art, the purpose of the present invention is to provide a Hadoop cluster system based on container technology and a fast construction method, which can quickly deploy Hadoop clusters, solve the situation that the number of servers is not enough, and improve the deployment of Hadoop container clusters in Docker. Availability on the platform

Images