Unlock instant, AI-driven research and patent intelligence for your innovation.

System resource scheduling method and device and storage medium

A system resource and scheduling method technology, applied in the field of devices and storage media, and system resource scheduling methods, can solve problems such as low utilization rate of system resources, waste, and insufficient system resources, and achieve accurate system resource configuration parameters, improve utilization, and Realize the effect of automatic optimization

Active Publication Date: 2020-08-18
MIGU CO LTD +1
View PDF21 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the inventors have found that at least the following problems exist in the prior art: due to the difference in the number of tasks of all tasks under this type of user in different periods, the obtained system resources may be insufficient or wasted, resulting in low utilization of system resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • System resource scheduling method and device and storage medium
  • System resource scheduling method and device and storage medium
  • System resource scheduling method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, various implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized.

[0022] In the existing technical solutions, system resources are controlled through queues. Although queues can specify the minimum and maximum values ​​of the percentage of system resources used in the cluster, it is inevitable to face the following scenarios, which will lead to insufficient or wasted system...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention relates to the technical field of data processing, and discloses a system resource scheduling method and device, and a storage medium, wherein the method comprises thesteps of obtaining the current data size of a to-be-executed task; determining whether the to-be-executed task has a historical execution record or not; if the historical execution record exists, determining a system resource configuration parameter according to the current data volume of the to-be-executed task and the historical execution record; and allocating system resources to the to-be-executed task according to the system resource configuration parameters, thereby improving the utilization rate of the system resources.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of data processing, and in particular to a system resource scheduling method, device and storage medium. Background technique [0002] With the deepening of the company's big data development, there are more and more projects on the distributed system (Hadoop Distributed FileSystem, Hadoop), and more and more data warehouse tool hive tasks submitted through the scheduling system, but the Hadoop system resources are limited. [0003] Hadoop system resources are controlled by queues, and a certain proportion of system resources will be pre-allocated to the queues. When the scheduling system submits tasks, it only submits a class of users and all tasks under this class of users to the system. System resource management The server will assign all these tasks to a preset queue, so that all tasks under this type of user will compete for system resources in this queue. [0004] However, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50G06F9/48
CPCG06F9/5027G06F9/4881Y02D10/00
Inventor 杨以通
Owner MIGU CO LTD