Resource scheduling optimization method and system based on dynamic load awareness
By using dynamic load awareness and scheduling optimization methods, the resource configuration of consumer groups and AI models is adjusted in real time, which solves the problems of data backlog and resource waste caused by fixed settings, and achieves load balancing and efficient business processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 广州市申迪计算机系统有限公司
- Filing Date
- 2026-01-05
- Publication Date
- 2026-05-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In existing technologies, the number of concurrent consumers and the inference batch size of AI models are fixed, which leads to data backlog and resource waste when business data surges or decreases, and cannot meet the needs of real-time business processing.
By collecting real-time operational metrics of consumer groups and AI models through a load-aware agent, resource scheduling rules are constructed, and a dynamic scheduling engine is used to calculate the optimal scheduling action, dynamically adjusting the concurrency of consumer groups and the configuration parameters of AI models to achieve load balancing.
It achieves load balancing in the business processing system, improves business processing efficiency, avoids resource waste, and ensures high responsiveness and low latency under traffic fluctuations.
Smart Images
Figure CN122019145A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of resource scheduling technology, specifically relating to a resource scheduling optimization method and system based on dynamic load awareness. Background Technology
[0002] With the deep application of artificial intelligence technology, enterprises need to seamlessly integrate AI model inference services with real-time business data stream processing systems (such as Kafka-based data processing clusters) to process business data in real time. A typical workflow is as follows: the data processing cluster writes real-time business data provided by the production end into a message queue; then, the consumer group of the data processing cluster extracts data from the message queue and submits it to the AI model for inference; finally, the inference results are returned to the downstream system. Therefore, the data processing cluster's storage and transmission capabilities enable the AI model to process various business processes in a timely manner.
[0003] However, the typical workflow described above uses a fixed number of concurrent consumer groups and a fixed AI inference batch size, which cannot be dynamically adjusted according to real-time traffic changes. When a consumer group simultaneously writes a large amount of data to the AI model, its consumption capacity becomes too large, and the system cannot automatically increase the inference capability of the AI model, resulting in business processing delays and service unavailability. Conversely, when the amount of data written decreases, it leads to a waste of inference resources. That is, the large difference between the consumption capacity of the consumer group and the inference capability of the AI model due to resource mismatch results in low business processing efficiency and wasted inference resources. In addition, because the data processing cluster uses a fixed number of concurrent consumer groups, when there is a surge in data written to the message queue, the consumer group cannot retrieve the data from the message queue in a timely manner, resulting in data backlog and high data acquisition latency in the message queue, making it difficult to guarantee the real-time requirements of critical business operations. Summary of the Invention
[0004] This application proposes a resource scheduling optimization method and system based on dynamic load awareness, which can solve the problem in the prior art that the consumption capacity of the data processing cluster and the inference capacity of the AI model cannot meet the actual needs due to the inability to dynamically allocate resources, resulting in low business processing efficiency and resource waste.
[0005] The first aspect of this application provides a resource scheduling optimization method based on dynamic load awareness, the method comprising: When the consumer group processes real-time business data, the load-aware agent collects metrics from both the consumer group and the AI model to obtain several operational metrics. If the operational metrics meet the preset resource scheduling conditions, then resource allocation is performed based on the operational metrics and the business priority of the real-time business data, and resource scheduling rules are constructed. Based on the aforementioned operational metrics and resource scheduling rules, the optimal scheduling action is calculated using a dynamic scheduling engine. Use the optimal scheduling action to update the concurrency of the consumer group and the configuration parameters of the AI model.
[0006] The aforementioned solution addresses the issue that existing technologies often have fixed settings for both consumer group concurrency and AI model inference batches, leading to data backlog and resource waste during surges or drops in business data. Therefore, when consumer groups transmit business data, operational metrics of both consumer groups and AI models are collected to assess their consumption and inference capabilities, initially determining the current load and providing a foundation for subsequent resource adjustments. Then, resource scheduling conditions are used to identify the types of resources requiring adjustment. Resource scheduling rules are constructed to provide accurate adjustment directions and degrees, ensuring that resource scheduling precisely matches the actual load. Finally, a dynamic scheduling engine obtains the optimal scheduling action, ensuring a balance between consumption and inference resources, maintaining the collaborative operation of consumer groups and AI models, and avoiding uneven resource allocation. The optimal scheduling action updates the resource configurations of consumer groups and AI models, achieving load balancing of the current business processing system, improving business processing efficiency while avoiding resource waste.
[0007] In one possible implementation of the first aspect, a load-aware agent collects metrics from both the consumer group and the AI model to obtain several operational metrics, specifically: After each read / write operation of real-time business data by the consumer group, the load-aware agent mounted on the consumer group is invoked to collect the operating status of the consumer group in real time, and obtain the partition Lag value, memory usage and CPU usage of the consumer group; wherein, the partition Lag value is used to measure the latency of the consumer group in processing data. The load-aware agent is used to collect the running status of the AI model in real time to obtain the inference queue length of the AI model. The operational metrics are obtained based on the partition LAg value, memory usage, CPU usage, and inference queue length.
[0008] The above solution uses an embedded load-aware agent to collect data consumption status indicators of the data processing cluster in real time, enabling complete detection of data processing and data inference. It also introduces partition Lag values as operational indicators to accurately reflect data processing progress and provide a reliable basis for resource scheduling.
[0009] In one possible implementation of the first aspect, the resource scheduling conditions are specifically as follows: Set corresponding scheduling thresholds for the partition Lag value of the consumer group, CPU utilization, and inference queue length of the AI model in the aforementioned operational metrics; When any of the aforementioned operational metrics exceeds the corresponding scheduling threshold, the resource scheduling conditions are met. If none of the aforementioned operational metrics exceed the corresponding scheduling threshold, then the current concurrency of the consumer group and the configuration parameters of the AI model are maintained.
[0010] The above solution precisely triggers the resource scheduling mechanism through threshold comparison, avoiding unnecessary frequent adjustments and reducing system jitter. Furthermore, it maintains the existing configuration when all operating indicators are normal, reducing resource waste and scheduling overhead, and improving system stability.
[0011] In one possible implementation of the first aspect, resource allocation is performed based on the operational metrics and the business priority of the real-time business data, and resource scheduling rules are constructed, specifically as follows: The operational indicators that exceed the scheduling threshold are set as abnormal indicators, and a mapping relationship is constructed between the indicator types of abnormal indicators and the direction of resource adjustment. The service priority of the real-time service data is determined based on its service type. The resource allocation type is determined based on the mapping relationship, the resource allocation ratio is determined based on the business priority, and resources are allocated to consumer groups based on the allocation type and allocation ratio to construct resource scheduling rules.
[0012] The above solution only adjusts resources for operational metrics exceeding thresholds, minimizing unnecessary resource allocation to maintain system stability. A mapping relationship is established to provide direction for subsequent resource allocation adjustments and to determine the types of resources requiring adjustment. Furthermore, business priorities influence the degree of resource allocation bias, ensuring high-priority services are processed quickly to guarantee core business resources. This results in precise resource scheduling rules that provide a foundation for the collaborative operation of consumer groups and AI models.
[0013] In one possible implementation of the first aspect, a mapping relationship is constructed between the type of abnormal indicators and the direction of resource adjustment, specifically as follows: When the partition Lag value exceeds the corresponding scheduling threshold, a mapping relationship is established between the partition Lag value and the ability of the consumer group to increase consumption. When the CPU utilization exceeds the corresponding scheduling threshold, a mapping relationship is established between the CPU utilization and the reduction of consumer group load; When the length of the inference queue exceeds the corresponding scheduling threshold, a mapping relationship is established between the length of the inference queue and the improved inference capability of the AI model.
[0014] In one possible implementation of the first aspect, the optimal scheduling action is calculated by a dynamic scheduling engine based on the aforementioned operational metrics and resource scheduling rules, specifically as follows: All the aforementioned operational metrics are integrated into a single state vector, which describes the current load status of the consumer group and the AI model. Based on the current concurrency of the consumer group and the configuration parameters of the AI model, construct the initial action vector; Each optimization objective of the resource scheduling rule is quantified into a composite reward function. Based on the composite reward function, with the goal of optimizing the state vector, the initial action vector is updated by interactive learning between the dynamic scheduling engine and the external environment. After reaching the preset optimization termination condition, the optimal scheduling action is output.
[0015] The above solution combines the current load of consumer groups and AI models with business priorities, and guides the dynamic scheduling engine to find a global equilibrium point through a composite reward function, achieving global optimization of resource scheduling. This enables the system to quickly and accurately determine the optimal scheduling action even under complex system conditions. The optimal scheduling action can automatically coordinate the configurations between consumer groups, AI models, and the resource layer, avoiding adjustment conflicts between subsystems and achieving long-term system load balancing.
[0016] In one possible implementation of the first aspect, the optimal scheduling action is used to update the concurrency of the consumer group and the configuration parameters of the AI model, specifically: Based on the optimal scheduling action, the concurrency of the consumer group is adjusted through the interface of the data processing cluster, and the configuration parameters of the AI model are updated through the model configuration center. Based on the optimal scheduling action, the instance capacity of consumer groups and AI models can be increased or decreased by modifying the CPU threshold of HPA. The scheduling effect is verified based on the updated results after using the optimal scheduling action.
[0017] In one possible implementation of the first aspect, the scheduling effect is verified based on the update result after using the optimal scheduling action, specifically as follows: Based on the update results, check the changes in metrics of the updated consumer group and AI model. If the changes in metrics do not meet the preset optimization requirements, then trigger another update of resource allocation for the consumer group and AI model.
[0018] The above solution constructs a closed-loop linkage mechanism. After updating resource allocation, the scheduling effect is obtained by detecting changes in operational indicators. When the scheduling effect does not meet expectations, the previous steps are repeated to perform resource scheduling again. This enables automatic adjustment of resource configuration based on real-time traffic changes without manual intervention. It ensures that the business processing system is always in a balanced state of "high response, low latency, and efficient resource utilization" under traffic fluctuations, significantly improving business processing efficiency and avoiding resource waste.
[0019] The second aspect of this application provides a resource scheduling optimization system based on dynamic load awareness, the system comprising: an operation indicator acquisition module, a scheduling rule construction module, a scheduling action calculation module, and a resource configuration update module; Among them, the operation indicator collection module is used to collect indicators from the consumer group and the AI model through the load-aware agent when the consumer group processes real-time business data, and obtain several operation indicators. The scheduling rule construction module is used to construct resource scheduling rules by allocating resources based on the operational indicators and the business priority of the real-time business data if the operational indicators meet the preset resource scheduling conditions. The scheduling action calculation module is used to calculate the optimal scheduling action based on the aforementioned operational indicators and resource scheduling rules, through a dynamic scheduling engine. The resource configuration update module is used to update the concurrency of consumer groups and the configuration parameters of AI models using the optimal scheduling action.
[0020] A third aspect of this application provides a terminal device, the device comprising: a terminal device including a processor and a memory, the memory storing a computer program, wherein the processor executes the computer program to implement the steps of a resource scheduling optimization method based on dynamic load awareness as described in any one of the embodiments of this application. Attached Figure Description
[0021] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0022] Figure 1 This is a schematic diagram illustrating the specific process of a resource scheduling optimization method based on dynamic load awareness, provided in an embodiment of this application. Figure 2 This is a structural diagram of a resource scheduling optimization system based on dynamic load awareness, provided in an embodiment of this application. Figure 3 This is a structural diagram of a terminal device provided in an embodiment of this application. Detailed Implementation
[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.
[0025] First Embodiment Message queues serve as middleware for asynchronous communication. Producers send real-time business data to the message queue, and consumers consume information from the message queue to transmit to AI models for business processing. However, this workflow has a significant drawback: the concurrency of consumer groups and the inference batch size of AI models are fixed. Therefore, when the volume of business data surges, message queue data accumulation and business processing delays can easily occur, failing to meet the real-time requirements of critical business operations.
[0026] To address the aforementioned technical issues, this application embodiment dynamically senses the load during business processing and optimizes the consumption capacity of consumer groups and the inference capacity of AI models respectively, thereby improving the matching degree between consumption capacity and inference capacity. By realizing the global collaborative operation of the data processing cluster, it effectively improves business processing efficiency and reduces resource waste.
[0027] like Figure 1 As shown, to address the problem in existing technologies where the inability to dynamically allocate resources leads to insufficient consumption capacity of data processing clusters and inference capabilities of AI models, resulting in low business processing efficiency and resource waste, the first embodiment of this application provides a detailed flowchart of a resource scheduling optimization method based on dynamic load awareness. This embodiment's resource scheduling optimization method based on dynamic load awareness includes steps S1 to S4, detailed below: Step S1: When the consumer group processes real-time business data, the load-aware agent collects metrics from the consumer group and the AI model respectively to obtain several operating metrics.
[0028] In the business processing flow, real-time business data (including financial transaction data and IoT sensor data, etc.) enters the data processing cluster from the producer end. The consumer group of the data processing cluster stores the real-time business data in the message queue through the poll() function, and then extracts the real-time business data from the message queue through the poll() function and passes it to the AI model for inference and output of the corresponding business processing results.
[0029] Consumer groups are logically grouped data within a message queue. Consumers within the same group jointly consume real-time business data on a specified topic, supporting load balancing and consumption offset management. Here, "topic" refers to the data application topic in the data processing cluster, such as financial transactions and log analysis, related to the corresponding business; "consumed data" refers to the data stored or retrieved from the message queue. AI models are online services based on pre-trained machine learning models, capable of predicting or making decisions based on input data, often provided in API form.
[0030] Optionally, the embodiments of this application employ a data processing cluster based on Kafka.
[0031] Therefore, after each read / write operation of real-time business data from the message queue using the poll() function, the load-aware agent embedded in the consumer group is invoked to collect real-time data on the consumer group's operational status. This data includes the consumer group's partition LAG value, memory usage, and CPU usage. The load-aware agent is a lightweight module embedded in the Kafka-based consumer group.
[0032] The partition LAG value measures the latency of data processing by a consumer group, indicating how quickly the consumer group consumes data; therefore, it is one of the main indicators for evaluating the consumption capacity of a consumer group. The CPU utilization rate measures the load on each consumer group, and the memory utilization rate measures the memory pressure on each consumer group. Therefore, CPU utilization and memory utilization are among the main indicators used to adjust HPA configuration.
[0033] The full name of the HPA configuration is HorizontalPodAutoscaler, which enables automatic scaling of Pods horizontally. It allows you to scale workload requirements by scaling the number of Pods.
[0034] Then, a load-aware agent is used to collect the running status of the AI model in real time to obtain the current inference queue length of the AI model. The inference queue length is used to measure the deviation between the speed at which the AI model processes data and the speed at which data is written to the AI model, and is therefore one of the main indicators for evaluating the inference capability of the AI model.
[0035] Therefore, in this embodiment of the application, the collected partition Lag value, memory usage, CPU usage, and inference queue length are used to construct operating metrics to assess the current load of the consumer group and the AI model.
[0036] Step S2: If the operating indicators meet the preset resource scheduling conditions, then resource allocation is performed based on the operating indicators and the business priority of the real-time business data, and resource scheduling rules are constructed.
[0037] To further determine whether the consumer group's spending power and the AI model's reasoning ability meet current needs, this application embodiment sets resource scheduling conditions to determine whether the resource scheduling mechanism needs to be triggered.
[0038] The resource scheduling conditions set corresponding scheduling thresholds for each operating indicator. When any operating indicator exceeds the corresponding scheduling threshold, the set resource scheduling conditions are met and the resource scheduling mechanism is triggered immediately. When all operating indicators do not exceed the corresponding scheduling thresholds, it indicates that the current system is in a load-balanced state and does not need to update the resource configuration. Therefore, the current concurrency of the consumer group and the configuration parameters of the AI model are maintained.
[0039] For example, in the resource scheduling conditions set in this application embodiment, it is specified that the resource scheduling mechanism is triggered when the partition LAg value exceeds 1000, the CPU utilization rate is greater than 80%, and the inference queue length is greater than 50. It is also specified that a judgment operation is performed every 10 seconds using the resource scheduling conditions.
[0040] Operational metrics that exceed the scheduling threshold are set as abnormal metrics. Then, a mapping relationship is constructed between the metric type of the abnormal metrics and the direction of resource adjustment to specify the scheduling direction for subsequent resource scheduling.
[0041] For example, if the partition LAg value exceeds the corresponding scheduling threshold, it indicates that the consumption capacity of the consumer group should be increased. Therefore, a mapping relationship is constructed between the partition LAg value and the consumption capacity of the consumer group to increase the concurrency of the consumer group. If the CPU utilization exceeds the corresponding scheduling threshold, it indicates that the load on the consumer group should be reduced or its processing efficiency should be improved. Therefore, a mapping relationship is constructed between the CPU utilization and the CPU utilization to increase the concurrency of the consumer group or notify the downstream AI service to adjust parameters to reduce consumer waiting / processing time. If the memory utilization exceeds the corresponding scheduling threshold, it indicates that the memory pressure of the consumer group is high and memory usage needs to be optimized or an alarm should be triggered. Therefore, a mapping relationship is constructed between the memory utilization and the memory utilization. If the inference queue length exceeds the corresponding scheduling threshold, it indicates that the AI service processing speed is lagging behind the request arrival speed and the inference capability needs to be improved or the processing efficiency needs to be optimized. Therefore, a mapping relationship is constructed between the inference queue length and the inference capability of the AI model to increase the number of AI model instances (K8s scaling) and / or adjust the AI inference batch size.
[0042] Then, based on the topic names within the data processing cluster, the business type of the real-time business data is determined, further defining its business priority. For example, a topic name containing "finance" indicates financial business, which has a higher priority; a topic name containing "log" indicates log business, which has a lower priority; other topics are assigned a medium priority by default. This business priority guides resource scheduling decisions across multiple subsystems, including message queue concurrency control, AI inference parameter adjustment, and Kubernetes elastic scaling, ensuring that high-priority businesses receive greater resource guarantees.
[0043] Then, the resource allocation type is determined according to the mapping relationship, the resource allocation ratio is determined according to the business priority, and resources are allocated to the consumer group through the allocation type and allocation ratio to construct resource scheduling rules.
[0044] Specifically, when multiple abnormal indicators appear, the system will coordinate adjustments to consumer groups and AI models based on preset business priorities, rather than targeting a single indicator in isolation, to improve system reliability. For example, when both the partition LAg value and the inference queue length exceed the threshold, the resource scheduling rules will simultaneously increase the number of concurrent consumers and the number of AI model instances. In actual resource scheduling, the mapping relationship affects the direction of resource adjustments, such as scaling up and adjusting concurrency, but the magnitude, speed, and priority of resource competition are determined by the corresponding business priorities. The business priorities directly affect the aggressiveness of the scheduling strategy and the weight of resource allocation, making the system more inclined to actively guarantee the performance of high-priority businesses during optimization. Therefore, the resource adjustment direction represented by the mapping relationship is closely related to the business priorities. Priority does not change "what to do" (direction), but it profoundly affects "how much to do, how fast to do it, and who does it first" (magnitude, speed, and order).
[0045] Step S3: Based on the aforementioned operational metrics and resource scheduling rules, the optimal scheduling action is calculated using a dynamic scheduling engine.
[0046] This step is the core of intelligent resource scheduling. It combines the collected operational indicators with resource scheduling rules and uses intelligent algorithms to calculate a set of optimal resource configuration parameters to obtain the corresponding optimal scheduling action.
[0047] First, all discrete operational metrics are integrated into a normalized state vector, completing the transformation from raw data to a machine-understandable "system profile." This state vector is used to describe the current load of the consumer group and the AI model.
[0048] Then, based on the current concurrency of the consumer group and the configuration parameters of the AI model, an initial action vector is constructed. These configuration parameters include the AI inference batch size and the number of model instances. Subsequently, the initial action vector will be continuously optimized to obtain the optimal scheduling actions that enable the consumer group and the AI model to operate collaboratively.
[0049] The optimization objectives of the resource scheduling rules are then quantified into a composite reward function. This integrates the mutually constraining optimization objectives (such as reducing latency, reducing backlog, controlling costs, and ensuring business priority) into a quantifiable optimization objective, guiding the algorithm to find a global equilibrium point rather than a local optimum of a single indicator.
[0050] Finally, based on the composite reward function, with the goal of obtaining the optimal state vector, the dynamic scheduling engine continuously learns from the external environment to update the initial action vector until a preset optimization termination condition is met, at which point the optimal scheduling action is output. Through this interactive learning, the dynamic scheduling engine can consider more complex and non-linear business processing system states, automatically coordinate the configuration of consumers, AI services, and the Kubernetes resource layer, avoid adjustment conflicts between subsystems, and the final optimal scheduling action is based on the prediction of the long-term effects of the action, avoiding short-sighted frequent oscillations that could cause system instability.
[0051] Optionally, in this embodiment, the PPO (Proximity Policy Optimization) model is used to calculate the optimal scheduling action. This is achieved by multiplying each component of the input state vector by its corresponding priority to obtain a weighted state vector. This weighted state vector is then input into a pre-trained PPO model to output the optimal action vector.
[0052] Step S4: Use the optimal scheduling action to update the concurrency of the consumer group and the configuration parameters of the AI model.
[0053] Use the obtained optimal scheduling action to update the resource configuration of the consumer group, AI model, and HPA respectively.
[0054] Based on the optimal scheduling action, the consumer group configuration is modified via the data processing cluster's management API, triggering a consumer group rebalancing mechanism. After triggering the rebalancing mechanism, partition reallocation and dynamic increases / decreases in the number of consumer instances are forcibly implemented according to the optimal scheduling action. For example, the original 3 consumer instances are increased to 5 consumer instances, increasing the consumer group concurrency to adapt to real-time message processing pressure.
[0055] Based on the optimal scheduling action, the inference batch size of the AI model is updated without restarting the service. For example, the inference batch size is adjusted from "16" to "32", reducing the number of batches in low-latency scenarios and improving resource utilization in high-throughput scenarios.
[0056] In addition, optimal scheduling actions are used to dynamically scale up the HPA. By modifying HPA configuration parameters such as CPU thresholds or target replica counts, the instance capacity of consumer groups and AI models can be increased or decreased. For example, the replica count can be increased from "4" to "6" to quickly respond to sudden traffic surges; or it can be reduced to "2" during periods of low traffic to reduce resource costs.
[0057] The updated consumer group and AI model can achieve a real-time balance between consumption and inference capabilities, eliminate data backlog in message queues and inference queues, effectively avoid the resource allocation defects of "giving one thing and losing another" in traditional solutions, and reduce resource waste while improving business processing efficiency.
[0058] As an improvement to the above solution, this application embodiment also provides a closed-loop optimization mechanism. After the resource configuration is updated, the changes in the consumer group and AI model are collected to obtain the update results after using the optimal scheduling action, so as to check the changes in the indicators of the updated consumer group and AI model.
[0059] If the changes in the metrics do not meet the preset optimization requirements, the resource allocation for the consumer group and AI model will be updated again. This involves repeating the previous steps to calculate resource scheduling and allocation until the optimization requirements are met. The changes in metrics include resource utilization, such as CPU and memory usage of consumers and AI services, to ensure that resource contention or overload is not caused by scaling up; and system health status, such as service error rate, request success rate, and Pod restart count, to ensure that scheduling actions do not introduce anomalies.
[0060] For example, when the concurrency of the consumer group is increased, the main update result is the decreasing trend and rate of Kafka partition backlog; when the inference batch size of the AI model is adjusted, the main update result is the changes in AI inference service latency and throughput; when the model instance is added, the main update result is the reduction in AI inference queue length and the improvement in overall processing capacity.
[0061] In addition to focusing on core performance improvements, the system will also monitor resource consumption and system health to ensure that optimization does not come at the expense of stability or incur excessive costs.
[0062] Implementing the embodiments of this application has the following beneficial effects: This application addresses the issue that in existing technologies, both the concurrency of consumer groups and the inference batch size of AI models are fixed, which can easily lead to data backlog and resource waste when business data surges or drops sharply. Therefore, when consumer groups transmit business data, the operational metrics of both consumer groups and AI models are collected to detect their consumption and inference capabilities, initially determining the current load and providing a foundation for subsequent resource adjustments. Then, resource scheduling conditions are used to determine the types of resources that need adjustment, and resource scheduling rules are constructed to provide accurate adjustment directions and degrees for subsequent resource adjustments, ensuring that resource scheduling precisely matches the actual load. Finally, a dynamic scheduling engine obtains the optimal scheduling action to ensure a balance between consumption and inference resources, maintaining the collaborative operation of consumer groups and AI models and avoiding uneven resource allocation. The optimal scheduling action updates the resource configuration of consumer groups and AI models, achieving load balancing of the current business processing system, improving business processing efficiency while avoiding resource waste.
[0063] Second Embodiment Furthermore, in order to implement the resource scheduling optimization system based on dynamic load awareness corresponding to the above method embodiments, and to achieve the corresponding functions and technical effects, Figure 2 A structural diagram of a resource scheduling optimization system based on dynamic load awareness is provided. For ease of explanation, only the parts relevant to this embodiment are shown. The resource scheduling optimization system based on dynamic load awareness provided in this application embodiment includes: The operation metric collection module 201 is used to collect metrics from the consumer group and the AI model through the load-aware agent when the consumer group processes real-time business data, and obtain several operation metrics.
[0064] In the business processing flow, real-time business data (including financial transaction data and IoT sensor data, etc.) enters the data processing cluster from the producer end. The consumer group of the data processing cluster stores the real-time business data in the message queue through the poll() function, and then extracts the real-time business data from the message queue through the poll() function and passes it to the AI model for inference and output of the corresponding business processing results.
[0065] Consumer groups are logically grouped data within a message queue. Consumers within the same group jointly consume real-time business data on a specified topic, supporting load balancing and consumption offset management. Here, "topic" refers to the data application topic in the data processing cluster, such as financial transactions and log analysis, related to the corresponding business; "consumed data" refers to the data stored or retrieved from the message queue. AI models are online services based on pre-trained machine learning models, capable of predicting or making decisions based on input data, often provided in API form.
[0066] Optionally, the embodiments of this application employ a data processing cluster based on Kafka.
[0067] Therefore, after each read / write operation of real-time business data from the message queue using the poll() function, the load-aware agent embedded in the consumer group is invoked to collect real-time data on the consumer group's operational status. This data includes the consumer group's partition LAG value, memory usage, and CPU usage. The load-aware agent is a lightweight module embedded in the Kafka-based consumer group.
[0068] The partition LAG value measures the latency of data processing by a consumer group, indicating how quickly the consumer group consumes data; therefore, it is one of the main indicators for evaluating the consumption capacity of a consumer group. The CPU utilization rate measures the load on each consumer group, and the memory utilization rate measures the memory pressure on each consumer group. Therefore, CPU utilization and memory utilization are among the main indicators used to adjust HPA configuration.
[0069] The full name of the HPA configuration is HorizontalPodAutoscaler, which enables automatic scaling of Pods horizontally. It allows you to scale workload requirements by scaling the number of Pods.
[0070] Then, a load-aware agent is used to collect the running status of the AI model in real time to obtain the current inference queue length of the AI model. The inference queue length is used to measure the deviation between the speed at which the AI model processes data and the speed at which data is written to the AI model, and is therefore one of the main indicators for evaluating the inference capability of the AI model.
[0071] Therefore, in this embodiment of the application, the collected partition Lag value, memory usage, CPU usage, and inference queue length are used to construct operating metrics to assess the current load of the consumer group and the AI model.
[0072] The scheduling rule construction module 202 is used to construct resource scheduling rules by allocating resources based on the operating indicators and the business priority of the real-time business data if the operating indicators meet the preset resource scheduling conditions.
[0073] In this embodiment of the application, the operation indicators that exceed the scheduling threshold are set as abnormal indicators, and a mapping relationship between the indicator type of abnormal indicators and the direction of resource adjustment is constructed. The service priority of the real-time service data is determined based on its service type. The resource allocation type is determined based on the mapping relationship, the resource allocation ratio is determined based on the business priority, and resources are allocated to consumer groups based on the allocation type and allocation ratio to construct resource scheduling rules.
[0074] The scheduling action calculation module 203 is used to calculate the optimal scheduling action based on the operation indicators and resource scheduling rules through a dynamic scheduling engine.
[0075] In this embodiment of the application, all the aforementioned operating metrics are integrated into a single state vector, which is used to describe the current load status of the consumer group and the AI model. Based on the current concurrency of the consumer group and the configuration parameters of the AI model, construct the initial action vector; Each optimization objective of the resource scheduling rule is quantified into a composite reward function. Based on the composite reward function, with the goal of optimizing the state vector, the initial action vector is updated by interactive learning between the dynamic scheduling engine and the external environment. After reaching the preset optimization termination condition, the optimal scheduling action is output.
[0076] Resource configuration update module 204 is used to update the concurrency of consumer groups and the configuration parameters of AI models using optimal scheduling actions.
[0077] Use the obtained optimal scheduling action to update the resource configuration of the consumer group, AI model, and HPA respectively.
[0078] Based on the optimal scheduling action, the consumer group configuration is modified via the data processing cluster's management API, triggering a consumer group rebalancing mechanism. After triggering the rebalancing mechanism, partition reallocation and dynamic increases / decreases in the number of consumer instances are forcibly implemented according to the optimal scheduling action. For example, the original 3 consumer instances are increased to 5 consumer instances, increasing the consumer group concurrency to adapt to real-time message processing pressure.
[0079] Based on the optimal scheduling action, the inference batch size of the AI model is updated without restarting the service. For example, the inference batch size is adjusted from "16" to "32", reducing the number of batches in low-latency scenarios and improving resource utilization in high-throughput scenarios.
[0080] In addition, optimal scheduling actions are used to dynamically scale up the HPA. By modifying HPA configuration parameters such as CPU thresholds or target replica counts, the instance capacity of consumer groups and AI models can be increased or decreased. For example, the replica count can be increased from "4" to "6" to quickly respond to sudden traffic surges; or it can be reduced to "2" during periods of low traffic to reduce resource costs.
[0081] The updated consumer group and AI model can achieve a real-time balance between consumption and inference capabilities, eliminate data backlog in message queues and inference queues, effectively avoid the resource allocation defects of "giving one thing and losing another" in traditional solutions, and reduce resource waste while improving business processing efficiency.
[0082] As an improvement to the above solution, this application embodiment also provides a closed-loop optimization mechanism. After the resource configuration is updated, the changes in the consumer group and AI model are collected to obtain the update results after using the optimal scheduling action, so as to check the changes in the indicators of the updated consumer group and AI model.
[0083] If the changes in the metrics do not meet the preset optimization requirements, the resource allocation for the consumer group and AI model will be updated again. This involves repeating the previous steps to calculate resource scheduling and allocation until the optimization requirements are met. The changes in metrics include resource utilization, such as CPU and memory usage of consumers and AI services, to ensure that resource contention or overload is not caused by scaling up; and system health status, such as service error rate, request success rate, and Pod restart count, to ensure that scheduling actions do not introduce anomalies.
[0084] For example, when the concurrency of the consumer group is increased, the main update result is the decreasing trend and rate of Kafka partition backlog; when the inference batch size of the AI model is adjusted, the main update result is the changes in AI inference service latency and throughput; when the model instance is added, the main update result is the reduction in AI inference queue length and the improvement in overall processing capacity.
[0085] In addition to focusing on core performance improvements, the system will also monitor resource consumption and system health to ensure that optimization does not come at the expense of stability or incur excessive costs.
[0086] In some embodiments, the scheduling rule construction module 202 specifically comprises: To further determine whether the consumer group's spending power and the AI model's reasoning ability meet current needs, this application embodiment sets resource scheduling conditions to determine whether the resource scheduling mechanism needs to be triggered.
[0087] The resource scheduling conditions set corresponding scheduling thresholds for each operating indicator. When any operating indicator exceeds the corresponding scheduling threshold, the set resource scheduling conditions are met and the resource scheduling mechanism is triggered immediately. When all operating indicators do not exceed the corresponding scheduling thresholds, it indicates that the current system is in a load-balanced state and does not need to update the resource configuration. Therefore, the current concurrency of the consumer group and the configuration parameters of the AI model are maintained.
[0088] For example, in the resource scheduling conditions set in this application embodiment, it is specified that the resource scheduling mechanism is triggered when the partition LAg value exceeds 1000, the CPU utilization rate is greater than 80%, and the inference queue length is greater than 50. It is also specified that a judgment operation is performed every 10 seconds using the resource scheduling conditions.
[0089] Operational metrics that exceed the scheduling threshold are set as abnormal metrics. Then, a mapping relationship is constructed between the metric type of the abnormal metrics and the direction of resource adjustment to specify the scheduling direction for subsequent resource scheduling.
[0090] For example, if the partition LAg value exceeds the corresponding scheduling threshold, it indicates that the consumption capacity of the consumer group should be increased. Therefore, a mapping relationship is constructed between the partition LAg value and the consumption capacity of the consumer group to increase the concurrency of the consumer group. If the CPU utilization exceeds the corresponding scheduling threshold, it indicates that the load on the consumer group should be reduced or its processing efficiency should be improved. Therefore, a mapping relationship is constructed between the CPU utilization and the CPU utilization to increase the concurrency of the consumer group or notify the downstream AI service to adjust parameters to reduce consumer waiting / processing time. If the memory utilization exceeds the corresponding scheduling threshold, it indicates that the memory pressure of the consumer group is high and memory usage needs to be optimized or an alarm should be triggered. Therefore, a mapping relationship is constructed between the memory utilization and the memory utilization. If the inference queue length exceeds the corresponding scheduling threshold, it indicates that the AI service processing speed is lagging behind the request arrival speed and the inference capability needs to be improved or the processing efficiency needs to be optimized. Therefore, a mapping relationship is constructed between the inference queue length and the inference capability of the AI model to increase the number of AI model instances (K8s scaling) and / or adjust the AI inference batch size.
[0091] Then, based on the topic names within the data processing cluster, the business type of the real-time business data is determined, further defining its business priority. For example, a topic name containing "finance" indicates financial business, which has a higher priority; a topic name containing "log" indicates log business, which has a lower priority; other topics are assigned a medium priority by default. This business priority guides resource scheduling decisions across multiple subsystems, including message queue concurrency control, AI inference parameter adjustment, and Kubernetes elastic scaling, ensuring that high-priority businesses receive greater resource guarantees.
[0092] Then, the resource allocation type is determined according to the mapping relationship, the resource allocation ratio is determined according to the business priority, and resources are allocated to the consumer group through the allocation type and allocation ratio to construct resource scheduling rules.
[0093] Specifically, when multiple abnormal indicators appear, the system will coordinate adjustments to consumer groups and AI models based on preset business priorities, rather than targeting a single indicator in isolation, to improve system reliability. For example, when both the partition LAg value and the inference queue length exceed the threshold, the resource scheduling rules will simultaneously increase the number of concurrent consumers and the number of AI model instances. In actual resource scheduling, the mapping relationship affects the direction of resource adjustments, such as scaling up and adjusting concurrency, but the magnitude, speed, and priority of resource competition are determined by the corresponding business priorities. The business priorities directly affect the aggressiveness of the scheduling strategy and the weight of resource allocation, making the system more inclined to actively guarantee the performance of high-priority businesses during optimization. Therefore, the resource adjustment direction represented by the mapping relationship is closely related to the business priorities. Priority does not change "what to do" (direction), but it profoundly affects "how much to do, how fast to do it, and who does it first" (magnitude, speed, and order).
[0094] In some embodiments, the scheduling action calculation module 203 specifically comprises: This step is the core of intelligent resource scheduling. It combines the collected operational indicators with resource scheduling rules and uses intelligent algorithms to calculate a set of optimal resource configuration parameters to obtain the corresponding optimal scheduling action.
[0095] First, all discrete operational metrics are integrated into a normalized state vector, completing the transformation from raw data to a machine-understandable "system profile." This state vector is used to describe the current load of the consumer group and the AI model.
[0096] Then, based on the current concurrency of the consumer group and the configuration parameters of the AI model, an initial action vector is constructed. These configuration parameters include the AI inference batch size and the number of model instances. Subsequently, the initial action vector will be continuously optimized to obtain the optimal scheduling actions that enable the consumer group and the AI model to operate collaboratively.
[0097] The optimization objectives of the resource scheduling rules are then quantified into a composite reward function. This integrates the mutually constraining optimization objectives (such as reducing latency, reducing backlog, controlling costs, and ensuring business priority) into a quantifiable optimization objective, guiding the algorithm to find a global equilibrium point rather than a local optimum of a single indicator.
[0098] Finally, based on the composite reward function, with the goal of obtaining the optimal state vector, the dynamic scheduling engine continuously learns from the external environment to update the initial action vector until a preset optimization termination condition is met, at which point the optimal scheduling action is output. Through this interactive learning, the dynamic scheduling engine can consider more complex and non-linear business processing system states, automatically coordinate the configuration of consumers, AI services, and the Kubernetes resource layer, avoid adjustment conflicts between subsystems, and the final optimal scheduling action is based on the prediction of the long-term effects of the action, avoiding short-sighted frequent oscillations that could cause system instability.
[0099] Optionally, in this embodiment, the PPO (Proximity Policy Optimization) model is used to calculate the optimal scheduling action. This is achieved by multiplying each component of the input state vector by its corresponding priority to obtain a weighted state vector. This weighted state vector is then input into a pre-trained PPO model to output the optimal action vector.
[0100] Implementing the embodiments of this application has the following beneficial effects: This application addresses the issue that in existing technologies, both the concurrency of consumer groups and the inference batch size of AI models are fixed, which can easily lead to data backlog and resource waste when business data surges or drops sharply. Therefore, when consumer groups transmit business data, the operational metrics of both consumer groups and AI models are collected to detect their consumption and inference capabilities, initially determining the current load and providing a foundation for subsequent resource adjustments. Then, resource scheduling conditions are used to determine the types of resources that need adjustment, and resource scheduling rules are constructed to provide accurate adjustment directions and degrees for subsequent resource adjustments, ensuring that resource scheduling precisely matches the actual load. Finally, a dynamic scheduling engine obtains the optimal scheduling action to ensure a balance between consumption and inference resources, maintaining the collaborative operation of consumer groups and AI models and avoiding uneven resource allocation. The optimal scheduling action updates the resource configuration of consumer groups and AI models, achieving load balancing of the current business processing system, improving business processing efficiency while avoiding resource waste.
[0101] Furthermore, Figure 3 This is a structural diagram of a terminal device provided in one embodiment of this application. Figure 3 As shown, the terminal device 3 of this embodiment includes: at least one processor 30 (in... Figure 3 (Only one is shown in the image) and a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor, wherein when the processor 30 executes the computer program 32, it can implement the steps of a resource scheduling optimization method based on dynamic load awareness as described in any one of the embodiments of this application.
[0102] The terminal device 3 may be a computing device such as a desktop computer, a cloud server, or a laptop computer, and the computing device may include, but is not limited to, a processor 30 and a memory 31. Figure 3 This is merely an example of terminal device 3 and does not constitute a limitation on terminal device 3. It may include more or fewer components than those shown in the figure.
[0103] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, or improvements made by those skilled in the art within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A resource scheduling optimization method based on dynamic load awareness, characterized in that, include: When the consumer group processes real-time business data, the load-aware agent collects metrics from both the consumer group and the AI model to obtain several operational metrics. If the operational metrics meet the preset resource scheduling conditions, then resource allocation is performed based on the operational metrics and the business priority of the real-time business data, and resource scheduling rules are constructed. Based on the aforementioned operational metrics and resource scheduling rules, the optimal scheduling action is calculated using a dynamic scheduling engine. Use the optimal scheduling action to update the concurrency of the consumer group and the configuration parameters of the AI model.
2. The resource scheduling optimization method based on dynamic load awareness according to claim 1, characterized in that, The process involves collecting metrics from the consumer group and the AI model via a load-aware agent to obtain several operational metrics, specifically: After each read / write operation of real-time business data by the consumer group, the load-aware agent mounted on the consumer group is invoked to collect the operating status of the consumer group in real time, and obtain the partition Lag value, memory usage and CPU usage of the consumer group; wherein, the partition Lag value is used to measure the latency of the consumer group in processing data. The load-aware agent is used to collect the running status of the AI model in real time to obtain the inference queue length of the AI model. The operational metrics are obtained based on the partition LAg value, memory usage, CPU usage, and inference queue length.
3. The resource scheduling optimization method based on dynamic load awareness according to claim 1, characterized in that, The resource scheduling conditions are as follows: Set corresponding scheduling thresholds for the consumer group's partition Lag value, CPU utilization, and AI model's inference queue length in the aforementioned operational metrics; When any of the aforementioned operational metrics exceeds the corresponding scheduling threshold, the resource scheduling conditions are met. If none of the aforementioned operational metrics exceed the corresponding scheduling thresholds, then the current concurrency of the consumer group and the configuration parameters of the AI model are maintained.
4. The resource scheduling optimization method based on dynamic load awareness according to claim 3, characterized in that, The process of allocating resources based on the operational metrics and the service priorities of the real-time business data, and constructing resource scheduling rules, specifically involves: The operational indicators that exceed the scheduling threshold are set as abnormal indicators, and a mapping relationship is constructed between the indicator types of abnormal indicators and the direction of resource adjustment. The service priority of the real-time service data is determined based on its service type. The resource allocation type is determined based on the mapping relationship, the resource allocation ratio is determined based on the business priority, and resources are allocated to consumer groups based on the allocation type and allocation ratio to construct resource scheduling rules.
5. The resource scheduling optimization method based on dynamic load awareness according to claim 4, characterized in that, The mapping relationship between the types of abnormal indicators and the direction of resource adjustment is as follows: When the partition Lag value exceeds the corresponding scheduling threshold, a mapping relationship is established between the partition Lag value and the ability of the consumer group to increase consumption. When the CPU utilization exceeds the corresponding scheduling threshold, a mapping relationship is established between the CPU utilization and the reduction of consumer group load; When the length of the inference queue exceeds the corresponding scheduling threshold, a mapping relationship is established between the length of the inference queue and the improved inference capability of the AI model.
6. The resource scheduling optimization method based on dynamic load awareness according to claim 1, characterized in that, The optimal scheduling action is calculated by the dynamic scheduling engine based on the aforementioned operational metrics and resource scheduling rules, specifically as follows: All the aforementioned operational metrics are integrated into a single state vector, which describes the current load status of the consumer group and the AI model. Based on the current concurrency of the consumer group and the configuration parameters of the AI model, construct the initial action vector; Each optimization objective of the resource scheduling rule is quantified into a composite reward function. Based on the composite reward function, with the goal of optimizing the state vector, the initial action vector is updated by interactive learning between the dynamic scheduling engine and the external environment. After reaching the preset optimization termination condition, the optimal scheduling action is output.
7. The resource scheduling optimization method based on dynamic load awareness according to claim 1, characterized in that, The step of using optimal scheduling actions to update the concurrency of the consumer group and the configuration parameters of the AI model specifically involves: Based on the optimal scheduling action, the concurrency of the consumer group is adjusted through the interface of the data processing cluster, and the configuration parameters of the AI model are updated through the model configuration center. Based on the optimal scheduling action, the instance capacity of consumer groups and AI models can be increased or decreased by modifying the CPU threshold of HPA. The scheduling effect is verified based on the updated results after using the optimal scheduling action.
8. The resource scheduling optimization method based on dynamic load awareness according to claim 7, characterized in that, The verification of scheduling effectiveness based on the update result after using the optimal scheduling action is as follows: Based on the update results, check the changes in metrics of the updated consumer group and AI model. If the changes in metrics do not meet the preset optimization requirements, then trigger another update of resource allocation for the consumer group and AI model.
9. A resource scheduling optimization system based on dynamic load awareness, characterized in that, include: The system includes a performance indicator collection module, a scheduling rule construction module, a scheduling action calculation module, and a resource configuration update module. Among them, the operation indicator collection module is used to collect indicators from the consumer group and the AI model respectively through the load-aware agent when the consumer group processes real-time business data, and obtain several operation indicators. The scheduling rule construction module is used to construct resource scheduling rules by allocating resources based on the operational indicators and the business priority of the real-time business data if the operational indicators meet the preset resource scheduling conditions. The scheduling action calculation module is used to calculate the optimal scheduling action based on the aforementioned operational indicators and resource scheduling rules, through a dynamic scheduling engine. The resource configuration update module is used to update the concurrency of consumer groups and the configuration parameters of AI models using the optimal scheduling action.
10. A terminal device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the resource scheduling optimization method based on dynamic load awareness as described in any one of claims 1 to 8.