Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Docker elastic scaling-based big data platform resource scheduling method

A big data platform and elastic scaling technology, applied in the direction of resource allocation, digital data processing, program control design, etc., can solve the problems of affecting business applications, business suspension, complexity and inefficiency, saving resources and labor costs, The effect of meeting business needs and reducing waste of resources

Pending Publication Date: 2021-02-19
北京开物数智科技有限公司
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] At present, the use of Docker in the big data analysis platform has gradually matured. In the Docker environment, the business volume and request volume of the analysis data will increase with the increase of the data volume. Under such a huge business request, more If the resource scheduling is not timely, the most direct impact will be the performance degradation of business applications or even the suspension of business. If resource scheduling is performed manually and the scaling mode is manually configured, it is complicated and inefficient, and at the same time, manual operation is easy. Errors directly affect business applications

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Docker elastic scaling-based big data platform resource scheduling method
  • Docker elastic scaling-based big data platform resource scheduling method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0043] (1) Define the Pod of application A for Kubernetes, and set the resource allocation of the Pod to 3 containers, and predetermine the corresponding load indicator threshold: the upper limit of the CPU usage indicator threshold is above 80%, and the lower limit of the threshold is 30%; Set a continuous detection strategy: detect once every 20s, and if the threshold is reached for 3 consecutive detections, an alarm and scaling operation will be performed;

[0044] (2) Set the Pod scaling threshold: the maximum number of replicas is 10, the minimum number of replicas is 2, and the scaling step is 2. After the setting is completed, the index detection is performed every 20s according to the policy workflow;

[0045] (3) The initial value of the number of Pods for application A is 2, and the CPU usage rate is 25%, which is in a normal working state; with the establishment of the big data analysis platform, the user data continues to increase, and the CPU usage rate of applicat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a docker elastic scaling-based big data platform resource scheduling method, relates to the technical field of big data resource scheduling, and aims to realize automatic capacity expansion and capacity reduction of an application system, remarkably improve the resource utilization rate and save the cost. The method comprises the following steps: S1, setting a threshold value for a resource consumption index of an application load; S2, collecting data of actual resource consumption indexes; S3, judging a relationship between the acquired data and a threshold value of a corresponding index, and triggering a corresponding scaling strategy; when the collected data exceeds the threshold value of the corresponding index, automatic expansion is carried out; and when the collected data is lower than the threshold value of the corresponding index, automatic capacity reduction is carried out. The technical scheme provided by the invention is suitable for a big data resource scheduling process.

Description

【Technical field】 [0001] The invention relates to the technical field of big data resource scheduling, in particular to a method for scheduling big data platform resources based on docker elastic scaling. 【Background technique】 [0002] At present, the use of Docker in the big data analysis platform has gradually matured. In the Docker environment, the business volume and request volume of the analysis data will increase with the increase of the data volume. Under such a huge business request, more If the resource scheduling is not timely, the most direct impact will be the performance degradation of business applications or even the suspension of business. If resource scheduling is performed manually and the scaling mode is manually configured, it is complicated and inefficient, and it is easy to operate manually. Errors directly affect business applications. [0003] Therefore, it is necessary to study an intelligent scheduling method of big data analysis platform resourc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5016G06F9/5027G06F9/505G06F9/5022
Inventor 郭耕良
Owner 北京开物数智科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com