Dynamic resource allocation method, system and equipment and storage medium

Through resource management nodes, the resource allocation and dynamic adjustment of pod nodes are solved, which solves the resource scheduling problem of Spark software in data calculation, and improves resource utilization and execution efficiency.

CN119718600APending Publication Date: 2025-03-28CHINA UNITED NETWORK COMM GRP CO LTD +2
0 Cites 0 Cited by

Patent Information

Application Number
CN202311257942.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-09-26
Publication Date
2025-03-28

AI Technical Summary

Technical Problem

In the prior art, when using Spark software for data calculation, it is difficult to perform resource scheduling, resulting in waste of resources or inefficient execution.

Method used

The resource management node allocates resources to Spark tasks, monitors the operating status of pod nodes that complete Spark tasks, and dynamically adjusts pod nodes to achieve dynamic allocation and scheduling of resources.

Benefits of technology

There is no need to allocate cluster resources in advance, and the pod nodes of Spark tasks can be dynamically adjusted to improve resource utilization and execution completion rate.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention provides a dynamic resource allocation method, system and device and a storage medium, and relates to the field of distribution, the method is used for a resource management node, and the method comprises the steps: carrying out task allocation according to a Spark task, obtaining task allocation information, and sending the task allocation information to a k8s master node; wherein the k8s master node is used for creating a pod node according to the task allocation information and issuing the Spark task, and the pod node is used for executing the Spark task; monitoring the operation information of the pod node; and obtaining new task allocation information according to the operation information and sending the new task allocation information to the k8s master node, and in response, issuing the Spark task by the k8s master node according to the new task allocation information. The problem that when Spark software is adopted for data calculation, resource scheduling is difficult to carry out can be solved.
Need to check novelty before this filing date? Find Prior Art