Adaptive configuration method for short-time variable big data job cluster scheduling

A technology of cluster scheduling and configuration method, which is applied in multi-programming devices, neural learning methods, electrical digital data processing and other directions, and can solve problems such as manual adjustment and unified optimal scheduling.
CN110737529AActive Publication Date: 2020-01-31BEIJING INSTITUTE OF TECHNOLOGYGY
4 Cites 20 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Current Assignee / Owner
BEIJING INSTITUTE OF TECHNOLOGYGY
Publication Date
2020-01-31

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses an adaptive configuration method for short-time variable big data job cluster scheduling. The method is a self-adaptive cluster scheduler configuration optimization method provided by aiming at the characteristics of cloud platform isomerism, cluster scheduler configuration optimization of dynamic load, isomerism of cloud platform load and short-time variability. Cloud platform loads can be divided into service applications and analysis applications, and different classifications are different in resource consumption and time requirements. According to the method, the configuration of the cluster scheduler is adjusted according to the state information of the job and the information of the cluster environment; the optimal scheduling configuration is always kept; therefore, the operation performance is improved, the operation delay is reduced, the heterogeneous load of the cloud platform can be better adapted, and the optimal configuration item corresponding to the current cluster state can be better found, so that the waiting time of the cluster operation is close to the minimum, the operation efficiency is improved, and the short-time variable big data operation is effectively scheduled.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the technical field of cluster scheduling, and in particular relates to an adaptive configuration method for cluster scheduling of short-term variable big data operations. Background technique

[0002] Currently, cluster scheduling is a necessary prerequisite for performance optimization and resource management of cloud computing systems. A good scheduler can effectively improve the utilization of the cluster and save the user's investment cost, so cluster scheduling has always been one of the hot research directions. Cluster scheduling for short-term big data jobs faces three major challenges: 1. The heterogeneity and dynamics of short-term jobs; 2. How to configure the scheduler, which will determine the performance of the job; 3. No one is suitable for all situations the optimal configuration. For cloud platforms, cluster jobs can be divided into two types:

[0003] 1. Service applications: such as search engines (Search ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More