Dynamic resource scheduling method for big data platform system architecture and supporting environment

By constructing a three-layer decoupled perception and scheduling framework, unified scheduling and dynamic resource adjustment of heterogeneous workflows in the big data platform are realized, solving the problems of uniformity and timeliness of resource scheduling in existing technologies and improving resource utilization and service quality.

CN122152507APending Publication Date: 2026-06-05CHINA RAILWAY TUNNEL GROUP CO LTD +2
0 Cites 1 Cited by

Patent Information

Application Number
CN202610218904.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-02-24
Publication Date
2026-06-05

AI Technical Summary

Technical Problem

Existing big data platforms lack unified heterogeneous workflow scheduling capabilities in resource allocation, which makes it impossible to accurately guarantee data timeliness constraints. This results in low efficiency of cross-node data transmission and delayed elastic scaling response, affecting service quality.

Method used

By constructing a three-layer decoupled perception and scheduling framework, task description information is obtained for semantic parsing, the types of resources and data access patterns required for workflow execution are identified, and resource allocation is dynamically adjusted by combining critical path analysis and weighted calculation to achieve optimal mapping and elastic scaling from tasks to resource nodes.

Benefits of technology

It enables unified scheduling of heterogeneous workflows, improves resource utilization, reduces management complexity, and proactively adjusts resources before resource bottlenecks occur, ensuring the service quality of time-sensitive applications.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of big data processing, and discloses a dynamic resource scheduling method for a big data platform system architecture and a supporting environment, which comprises the following steps: acquiring task description information, and performing semantic analysis; identifying a resource type range and a data access mode required for executing the workflow; identifying a key task; the matching degree between each task and each resource node is evaluated, and weighted calculation is carried out in combination with a key path analysis result; solving an optimal mapping scheme from the task to the resource node; locking computing resources of the critical path task; predicting data traffic and identifying resource bottlenecks; dynamically adjusting resource allocation; executing task scheduling and resource scaling operation; the execution state is monitored in real time, and deviation analysis and parameter dynamic adjustment are carried out; according to the method, based on a timeliness constraint guarantee mechanism of workflow critical path analysis and an active prediction type elastic scaling strategy of data perception, key tasks influencing the overall completion time can be accurately identified, and resource priority guarantee is provided for the key tasks.
Need to check novelty before this filing date? Find Prior Art