Resource processing method and device

Abstract

Embodiments of this specification provide a resource processing method and device, wherein the resource processing method includes: acquiring task processing information of historical tasks, clustering the historical tasks according to the task processing information, and generating a According to the corresponding clustering result, according to the data processing value and resource consumption value corresponding to the historical task included in the clustering result, determine the resource value to be consumed corresponding to the target clustering result, the target clustering result being the clustering result In one of the results, the resources in the resource pool are grouped according to the value of the resource to be consumed, the resource group corresponding to the target clustering result is generated, and the preset task category and the target clustering result corresponding to the target clustering result are established. The association relationship between the resource groups.

Description

technical field [0001] The embodiments of this specification relate to the field of computer technology, and in particular to a resource processing method. One or more embodiments of this specification also relate to a resource processing device, a computing device, a computer-readable storage medium, and a computer program. Background technique [0002] In the Internet field, enterprises / institutions in various fields are pursuing digital transformation, mainly by purchasing big data platform products or using open source big data platforms to support the analysis and processing tasks of massive data. [0003] Due to the large amount of data in enterprises / institutions, the resource requirements of different tasks may also vary greatly. For example, the calculation time of large tasks can reach hours or days, and small tasks may only take a few minutes. The overall computing resources are limited, and the current resource scheduling module cannot automatically and reasonab...

AI Technical Summary

Problems solved by technology

[0003] Due to the large amount of data in enterprises / institutions, the resource requirements of different tasks may also vary greatly. For example, the calculation time of large tasks can reach hours or days, and small tasks may only take a few minutes. The overall computing resources are limited, and the current resource scheduling module cannot automatically and reasonably allocate resources. When there are many tasks on the big data platform, if the large tasks monopolize the resources during execution, the small tasks must wait

Therefore, it is difficult to guarantee the response time of small tasks. On the other hand, the utilization efficiency of resources is not high, which will cause the problem that the overall task running time is too long or even impossible to complete, which seriously affects the generation of data results and affects the daily operation of enterprises / institutions. It takes a lot of manpower to adjust and optimize the business analysis

Images