Docker-based quick integrated ElasticMapReduce service system and implementation method

A technology of service system and implementation method, applied in the field of fast integrated ElasticMapReduce service system, to facilitate data analysis, use and share business data insights, and reduce resource occupation

Pending Publication Date: 2021-01-19
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The technical task of the present invention is to provide a Docker-based fast and integrated Elastic MapReduce service system and implementation method to solve the problem of how to quickly deploy, service highly integrated, and components can be plugged and unplugged EMR services to meet the needs of various business scenarios, and effectively reduce the cost

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Docker-based quick integrated ElasticMapReduce service system and implementation method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0036] as attached figure 1 As shown, the Docker-based fast integrated Elastic MapReduce service system of the present invention, the Elastic MapReduce service system includes,

[0037] User management and authentication module (E-MR IAM module), which combines IAM and keycloak, can easily protect application services without requiring additional code, and can undertake user management, user authentication, and main and sub-account functions.

[0038] Front-end page and back-end proxy module (E-MR Web UI module) is used to implement reverse proxy using Nginx service, which can not only reduce resource consumption, but also support high concurrency scenarios, and use Angular framework to write front-end web service pages, Make user request processing more efficient;

[0039] The request processing and distribution module (E-MR Api module), written using the Spring Boot framework, is used to receive service requests from the front end of the web. On the basis of inspection, it ...

Embodiment 2

[0052] The implementation method of the fast integrated Elastic MapReduce service system based on Docker of the present invention is to use the Docker technology to integrate the user management and authentication module (E-MRIAM module), the front-end page and the back-end agent module required by the Elastic MapReduce service system. (E-MR Web UI module), request processing and distribution module (E-MR Api module), cluster resource allocation and operation module (E-MR Service module), data storage module (MySQL module) and cluster visualization interaction module (E-MR module) -MR Hue module) six modules to realize the rapid deployment and high availability of Elastic MapReduce services; and through the integrated design of the Elastic MapReduce service system, the functions of hosting multiple Hadoop clusters at the same time and supporting the horizontal expansion of clusters are realized.

[0053] Among them, the request processing and distribution module (E-MR Api modul...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a Docker-based quick integrated Elastic MapReduce service system and an implementation method, belongs to the field of big data development, and aims to solve the technical problem of how to quickly deploy EMR services, highly integrate the services and randomly plug and unplug components, meet the requirements of various service scenes and effectively reduce the cost. Thetechnical scheme is that the Elastic MapReduce service system comprises a user management and authentication module, a front-end page and rear-end agent module, a request processing and distribution module, a cluster resource allocation and operation module, a data storage module and a cluster visual interaction module. The user management and authentication module is used for user management, user authentication and main sub-account creation; and the front-end page and rear-end agent module is used for realizing reverse agent by using Nginx service and compiling a Web front-end service page by using an Angular framework, so that the user request processing is more efficient. The invention further discloses an implementation method of the quick integrated ElasticMapReduce service system based on the Docker. The implementation method comprises the following steps: obtaining the quick integrated ElasticMapReduce service system based on the Docker;.

Description

technical field [0001] The invention relates to the technical field of big data development, in particular to a Docker-based fast integrated Elastic MapReduce service system and an implementation method. Background technique [0002] With the popularity of the mobile Internet, business data has grown exponentially. In the face of multi-dimensional, multi-type, and massive business data, traditional data management systems will be difficult to handle, and cannot effectively mine the inherent value of business data. The Hadoop ecosystem developed for big data scenarios can efficiently and conveniently handle various challenges brought by massive data. Therefore, the Hadoop system is also widely used in the business systems of enterprises or organizations. In the past, before enterprises started to use the Hadoop big data system, they would purchase necessary hardware devices according to the characteristics of their own business scenarios, and install and deploy the required...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/455
CPCG06F9/45558G06F2009/45587G06F2009/45595G06F2009/45591Y02D10/00
Inventor 李涛肖雪张炜刚刘牧
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap