Method for automatically deploying and managing big data clusters through Docker container

Abstract

The invention particularly relates to a method for automatically deploying and managing big data clusters through a Docker container. Docker mirror images related to Hadoop big data cluster automatic deployment are constructed by compiling a Docker file and include a Server side mirror image and an Agent side mirror image. A distributed framework composed of a Server side, multiple Agent sides and a Web side is constructed by using the Docker mirror images, and service of deploying Hadoop related big data clusters is automatically started. Management and configuration tuning of the clusters are achieved through Web pages, workloads of operation and maintenance staff are reduced, meanwhile operation states of the clusters, a host and assemblies can be visually obtained through visual monitoring, work efficiency of the operation and maintenance staff is improved, accordingly big data cluster deployment speed is remarkably increased, the deployment process is simplified, and the hardware resource utilization rate is reduced.

Description

technical field [0001] The invention relates to the technical field of computer software application development, in particular to a method for automatically deploying and managing large data clusters using Docker containers. Background technique [0002] Docker is an open source application container engine that allows developers to package their applications and dependencies into a portable container, and then publish it to any popular Linux machine, and can also implement virtualization. Containers use a sandbox mechanism completely, and there will be no interfaces between them. There is almost no performance overhead and can be easily run on machines and data centers. Most importantly, they are not dependent on any language, framework or system. [0003] A Dockerfile is a script consisting of a series of commands and parameters that are applied to a base image and ultimately create a new image, which simplifies the process from start to finish and greatly simplifies de...

AI Technical Summary

Problems solved by technology

[0005] Based on the above situation, at the same time, in order to solve the shortcomings of high resource utilization rate of big data clusters, complex installation and management, cumbersome parameter adjustment and inconvenient performance monitoring during use, reduce the workload of operation and maintenance personnel, and quickly and efficiently complete large-scale clusters in the production environment. Data cluster deployment and management

Images