Dynamic scheduling method and system of generative ai heterogeneous computing resources of pc terminal

Abstract

The application relates to the technical field of PC terminal AI computing, and discloses a PC terminal generative AI heterogeneous computing resource dynamic scheduling method and system.The system comprises four modules, namely a resource state acquisition module, a scheduling atlas generation module, a resource fluctuation entropy analysis module and a scheduling decision engine module.The resource state acquisition module captures the running state parameters of GPU kernels, CPU threads and memory blocks in real time, and generates resource state feature tensors after normalization processing; the scheduling atlas generation module analyzes the computing node connection topology, extracts the correlation between devices, and constructs a multi-dimensional scheduling atlas; the resource fluctuation entropy analysis module separates the load feature vectors, calculates the entropy values of each computing unit, and generates a heterogeneous resource entropy value matrix; and the scheduling decision engine module jointly analyzes the above atlas and matrix, identifies the resource competition characteristics of the bottleneck nodes, generates a dynamic scheduling instruction set, adapts to the generative AI task demand, and guarantees efficient and stable operation of the task.

Description

Technical Field

[0001] This invention relates to the field of AI computing technology for PC terminals, specifically to a method and system for dynamic scheduling of generative AI heterogeneous computing resources for PC terminals. Background Technology

[0002] With the rapid penetration of generative AI technology into PC terminals, the demand for computing resources from various AI applications is showing significant characteristics of dynamic and heterogeneous nature. Currently, PC terminals are generally equipped with heterogeneous computing architectures consisting of GPUs, CPUs, and multi-level memory. However, existing resource scheduling mechanisms are difficult to adapt to the complex requirements of generative AI tasks.

[0003] Traditional scheduling systems often employ a single-dimensional resource status acquisition approach, focusing only on basic parameters such as CPU utilization and GPU memory usage. This lacks a comprehensive grasp of the data interaction status between heterogeneous devices, resulting in an inaccurate reflection of the actual operating status of computing nodes. Furthermore, scheduling decisions often rely on static priority allocation strategies, neglecting the dynamically changing dependencies and data transmission requirements between computing units in generative AI tasks. For example, fluctuations in data transmission bandwidth between GPU cores and changes in instruction dependency chains between CPU threads lead to discrepancies between scheduling instructions and actual computational needs.

[0004] Existing systems lack effective analytical methods for resource load fluctuations, failing to quantify and assess the dynamic changes in GPU core memory usage, the stability of CPU thread instruction pipelines, and the density of memory block data access. This makes it difficult to predict resource contention risks in advance when facing frequent load peaks in generative AI tasks, leading to problems such as computational task stuttering and data transmission latency. Furthermore, traditional scheduling mechanisms have failed to build an effective cross-device collaborative scheduling framework. Resource allocation between GPUs, CPUs, and memory is independent, making it impossible to dynamically adapt heterogeneous resources according to the real-time computational needs of tasks, significantly limiting the running efficiency and response speed of generative AI tasks on PC terminals. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for dynamic scheduling of heterogeneous computing resources for generative AI on PC terminals, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a dynamic scheduling system for generative AI heterogeneous computing resources on a PC terminal, the system comprising:

[0007] The resource status acquisition module is used to capture the running status parameters of the GPU kernel, CPU thread and memory block in the PC terminal in real time, and perform heterogeneous data format normalization processing on the running status parameters to generate resource status feature tensors.

[0008] The scheduling graph generation module is used to parse the computing node connection topology of the resource state feature tensor, extract the data transmission bandwidth between GPU cores, the dependency chain between CPU threads and the memory block shared area mapping relationship, and construct a multi-dimensional scheduling graph that includes computing node connection weights and cross-device interaction latency.

[0009] The resource fluctuation entropy analysis module is used to separate the real-time load feature vector of each computing unit in the resource state feature tensor, calculate the GPU core's memory occupancy entropy, the CPU thread's instruction pipeline entropy and the memory block's data access entropy based on the corresponding real-time load feature vector, and summarize them to generate a heterogeneous resource entropy value matrix.

[0010] The scheduling decision engine module is used to perform joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy matrix to identify the resource contention characteristics of bottleneck nodes in computing tasks and generate a dynamic scheduling instruction set that includes GPU kernel scheduling priority vectors, CPU thread preemption strategy sequences and memory block locking identifiers.

[0011] Preferably, the resource status acquisition module specifically includes:

[0012] The heterogeneous parameter capture unit synchronously collects the shader core utilization of the GPU kernel, the instruction cache hit rate of the CPU thread, and the page fault rate of the memory block.

[0013] The data fusion unit converts the shader core utilization, instruction cache hit rate, and page error rate into a time-series data stream with a unified timestamp.

[0014] The tensor generation unit extracts the spatial correlation features of the time-series data stream through a three-dimensional convolution kernel and outputs a resource status feature tensor with dimensions of [device type × time window × performance index].

[0015] Preferably, the scheduling map generation module specifically includes:

[0016] The topology parsing unit identifies cross-device synchronization semaphores between the GPU kernel and CPU thread in the resource state feature tensor and marks the direct memory access channel from the memory block to the GPU kernel.

[0017] The weight calculation unit quantifies the fluctuation coefficient of data transfer bandwidth between GPU cores, the waiting latency of dependency chains between CPU threads, and the probability of conflict in the mapping relationship of shared memory blocks.

[0018] The graph construction unit uses the cross-device interaction latency as the weight of the connection edge and the computing node as the vertex to construct a multi-dimensional scheduling graph containing vertex attributes and edge weights.

[0019] Preferably, the resource fluctuation entropy analysis module specifically includes:

[0020] The feature separation unit extracts the GPU kernel's video memory read / write throughput sequence, the CPU thread's instruction execution cycle sequence, and the memory block's cache line contention count sequence from the resource state feature tensor.

[0021] The entropy calculation unit calculates the video memory occupancy entropy based on the video memory read / write throughput sequence, calculates the instruction pipeline entropy based on the instruction execution cycle sequence, and calculates the data access entropy based on the cache line contention count sequence.

[0022] The matrix generation unit organizes the video memory occupancy entropy, instruction pipeline entropy, and data access entropy into a heterogeneous resource entropy matrix according to device type.

[0023] Preferably, the scheduling decision engine module specifically includes:

[0024] The bottleneck identification unit detects computing nodes in the multidimensional scheduling graph whose connection weights exceed a preset threshold, and locates resource competition characteristics by combining the abrupt gradient of the corresponding entropy value in the heterogeneous resource entropy matrix.

[0025] The policy generation unit generates scheduling priority vectors for GPU kernels with resource contention characteristics, generates thread preemption policy sequences for associated CPU threads, and generates memory lock identifiers for conflicting memory blocks.

[0026] The instruction set construction unit integrates the scheduling priority vector, thread preemption strategy sequence, and memory lock flag to generate a dynamic scheduling instruction set.

[0027] Preferably, the system further includes:

[0028] The scheduling policy execution module is used to parse the GPU kernel scheduling priority vector of the dynamic scheduling instruction set, perform computation instruction reordering operations on the target GPU kernel, suspend the specified CPU thread according to the thread preemption policy sequence, and isolate conflicting memory block access requests according to the memory lock identifier.

[0029] The resource reallocation module is used to monitor the GPU kernel instruction throughput, CPU thread wake-up latency, and memory block access conflict rate after the execution of the scheduling policy. When the instruction throughput is lower than the performance threshold or the wake-up latency exceeds the tolerance window, the computing resource load migration operation is triggered.

[0030] Preferably, the resource reallocation module specifically includes:

[0031] The performance monitoring unit collects the instruction queue depth of the GPU kernel, the wait-to-wake queue length of the CPU thread, and the backlog of memory block access requests.

[0032] The migration decision unit migrates GPU computing tasks to idle kernels when the instruction queue depth continuously exceeds the full load threshold; reallocates CPU thread affinity when the wake-up queue length exceeds the latency threshold; and copies memory blocks to free storage areas when the access request backlog reaches the conflict threshold.

[0033] The load migration unit performs GPU computing task migration operations based on the PCIe bus, adjusts CPU thread affinity through the operating system scheduler, and completes memory block copying operations using the memory controller.

[0034] Preferably, the system further includes:

[0035] The elastic scaling module is used to analyze the computational stage characteristics of generative AI tasks, predict the number of GPU cores, CPU thread scale, and memory block capacity required for the next stage, and generate a resource scaling demand sequence.

[0036] The resource supply module is used to respond to the resource expansion and contraction demand sequence, dynamically start and stop the computing instances of the GPU kernel, allocate the CPU thread pool size on demand, and adjust the memory block pre-allocation strategy.

[0037] Preferably, the elastic telescopic module specifically includes:

[0038] The task parsing unit identifies the time consumption ratios of the matrix multiplication stage, backpropagation stage, and gradient update stage in generative AI tasks.

[0039] The demand forecasting unit outputs the GPU core demand increment, CPU thread expansion factor, and memory block expansion coefficient based on the current resource consumption rate and the computational complexity of the next stage.

[0040] The sequence generation unit encodes the GPU kernel demand increment, CPU thread expansion factor, and memory block expansion coefficient into a resource expansion / shrinkage demand sequence.

[0041] Preferably, the present invention also includes a method for dynamic scheduling of generative AI heterogeneous computing resources on a PC terminal, the method comprising all modules and method flow of the generative AI heterogeneous computing resource dynamic scheduling system for PC terminals described above.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] By constructing a multi-dimensional resource status acquisition mechanism, the running status parameters of GPU kernels, CPU threads and memory blocks in PC terminals can be captured in real time. The resource status feature tensor is generated through heterogeneous data format normalization processing, breaking the limitation of single-dimensional data acquisition in traditional scheduling systems. This enables a comprehensive and accurate perception of the running status of heterogeneous computing resources, allowing scheduling decisions to be based on more complete resource status information.

[0044] The scheduling graph generation module analyzes the computing node connection topology of the resource state feature tensor, extracts the data transmission bandwidth between GPU cores, the dependency chain between CPU threads, and the mapping relationship of shared memory blocks, and constructs a multi-dimensional scheduling graph that includes computing node connection weights and cross-device interaction latency. This clearly presents the correlation and data interaction characteristics between heterogeneous computing units, providing an intuitive and comprehensive topology reference for subsequent scheduling decisions. It effectively solves the problem of neglecting the dynamic dependencies between devices in traditional scheduling mechanisms, allowing scheduling strategies to better adapt to the complex collaboration needs between computing units in generative AI tasks.

[0045] The resource fluctuation entropy analysis module can separate the real-time load feature vectors of each computing unit from the resource state feature tensor, and calculate the GPU core's memory usage entropy, the CPU thread's instruction pipeline entropy, and the memory block's data access entropy. This is then aggregated to generate a heterogeneous resource entropy matrix, enabling a quantitative characterization of the load fluctuation characteristics of heterogeneous resources. The entropy matrix clearly reveals the dynamic changes in the load of each computing unit, allowing for timely identification of computing units with abnormal load fluctuations. This proactively detects potential resource contention risks, prevents abnormal computational task execution due to sudden load spikes, and ensures the stability of generative AI tasks.

[0046] The scheduling decision engine module performs joint analysis based on a multi-dimensional scheduling graph and a heterogeneous resource entropy matrix. This enables precise identification of resource contention characteristics at bottleneck nodes in computing tasks, generating a dynamic scheduling instruction set that includes GPU kernel scheduling priority vectors, CPU thread preemption strategy sequences, and memory block locking flags. This joint analysis-based decision-making approach fully integrates the topological relationships and load fluctuation characteristics of computing nodes, allowing scheduling instructions to be precisely adapted to the actual needs of different computing units. It achieves collaborative dynamic scheduling of GPU, CPU, and memory resources, avoiding the problem of independent allocation of heterogeneous resources. It can flexibly adjust resource allocation strategies according to the real-time computing needs of generative AI tasks, improving the utilization efficiency of heterogeneous computing resources on PC terminals. This ensures efficient and smooth operation of generative AI tasks and enhances the user experience of generative AI applications on PC terminals. Attached Figure Description

[0047] Figure 1This is a timing diagram of the generative AI heterogeneous computing resource dynamic scheduling system for PC terminals described in this invention.

[0048] Figure 2 A flowchart illustrating the working principle of the scheduling map generation module;

[0049] Figure 3 A flowchart illustrating the working principle of the resource fluctuation entropy analysis module;

[0050] Figure 4 A flowchart illustrating the working principle of the scheduling decision engine module;

[0051] Figure 5 This is a flowchart illustrating the working principle of the elastic expansion module. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Please see Figure 1 This invention provides a dynamic scheduling system for generative AI heterogeneous computing resources on a PC terminal, the system comprising:

[0054] The resource status acquisition module captures real-time runtime status parameters in the PC terminal, such as GPU shader core utilization, CPU thread instruction cache hit rate, and memory block page fault rate. These parameters undergo heterogeneous data format normalization to generate a resource status feature tensor. The scheduling graph generation module parses the computation node connection topology of this resource status feature tensor, extracting data transfer bandwidth between GPU cores, dependency chains between CPU threads, and shared memory block mapping relationships. This constructs a multi-dimensional scheduling graph including computation node connection weights and cross-device interaction latency. The resource fluctuation entropy analysis module separates the real-time load feature vectors of each computation unit from the resource status feature tensor, calculates the GPU core memory occupancy entropy, CPU thread instruction pipeline entropy, and memory block data access entropy, and aggregates them to generate a heterogeneous resource entropy matrix. The scheduling decision engine module performs joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy matrix to identify resource contention characteristics of bottleneck nodes in computational tasks. This generates a dynamic scheduling instruction set containing GPU core scheduling priority vectors, CPU thread preemption strategy sequences, and memory block locking identifiers. This system enables efficient resource scheduling for generative AI tasks in heterogeneous computing environments.

[0055] Example 1: See Figure 2This document details the operational processes of the resource status acquisition module and the scheduling graph generation module. Running in a PC terminal environment, the system addresses the heterogeneous computing needs of generative AI tasks by collaborating with the operating system kernel through underlying hardware interfaces to achieve real-time capture of resource status and construction of a scheduling graph. The heterogeneous parameter capture unit of the resource status acquisition module directly accesses hardware performance counters and system call interfaces to synchronously collect data on GPU kernel shader core utilization, CPU thread instruction cache hit rate, and memory block page fault rate. Shader core utilization data is obtained from the performance monitoring interface provided by the GPU driver, sampled at millisecond frequencies to capture the actual workload of the computing unit. Instruction cache hit rate is obtained through the CPU performance monitoring unit, recording hit and miss events at each level of cache access. Page fault rate is collected through the event counter of the memory management unit, statistically analyzing the frequency of page faults. These parameters are temporarily stored in the kernel buffer in raw binary data form.

[0056] The data fusion unit processes the aforementioned multi-source heterogeneous data, converting it into a time-series data stream with a unified timestamp. Time synchronization is based on a high-precision clock source, employing a subset of network time protocols to achieve clock alignment between devices, eliminating timing deviations caused by acquisition delays. Shader core utilization, instruction cache hit rate, and page fault rate data are resampled onto a common timeline, forming a sequence of time-stamped data points. Each data point contains a triplet of device identifier, timestamp, and parameter value. The data stream is stored in a circular buffer structure to ensure data continuity during real-time processing.

[0057] The tensor generation unit extracts spatial features from the temporal data stream. The 3D convolutional kernel is designed as a filter bank with learnable parameters, its size configured to cover three dimensions: device type, time window, and performance metrics. The convolution operation slides along the time axis, calculating the correlation between adjacent time points and capturing the mutual influence of resource states across device types. In the dimensional structure of the output resource state feature tensor, the device type dimension distinguishes between three types of hardware units: GPU, CPU, and memory; the time window dimension retains historical data from the most recent 10 sampling periods; and the performance metric dimension contains normalized utilization, hit rate, and error rate values. This tensor is stored in floating-point tensor format and serves as the basic data structure for subsequent system processing.

[0058] The topology parsing unit of the scheduling graph generation module analyzes the intrinsic relationships between resource state feature tensors. By parsing cross-device synchronization semaphores between the GPU kernel and CPU threads, it identifies the cooperative relationships between devices. Synchronization semaphores are derived from the operating system kernel's synchronization primitive logs, including events such as mutexes, semaphores, and barrier operations. The direct memory access channel from memory blocks to the GPU kernel is marked by monitoring PCIe bus transactions, recording the source address, destination address, and data volume information of DMA transfers. These topological relationships are initially constructed in the form of a graph structure, where vertices represent computing nodes and edges represent interaction relationships.

[0059] The weight calculation unit quantizes the topological relationships. The fluctuation coefficient of data transfer bandwidth between GPU cores is calculated by statistically analyzing the degree of variation in bandwidth values, using the ratio of the standard deviation to the mean within a sliding window as a metric. The waiting latency of inter-CPU thread dependency chains is extracted from the thread scheduler's wait queue, and the average thread blocking time is calculated. The conflict probability of shared memory block mapping relationships is determined by monitoring memory access conflict events and statistically analyzing the ratio of the number of conflicts to the total number of accesses per unit time. These quantified values ​​are all normalized and converted into weight values ​​between 0 and 1.

[0060] The graph construction unit integrates the above quantification results into a multi-dimensional scheduling graph. Computational nodes serve as vertices, with vertex attributes including device type identifier, resource capacity, and current load status. Interactions are represented as edges, with edge weights containing three attributes: fluctuation coefficient, waiting latency, and conflict probability. The graph is represented using an attribute graph model, supporting various graph algorithm operations. Graph data is stored in adjacency list format, while simultaneously maintaining vertex and edge attribute tables for easy querying and updating. This multi-dimensional scheduling graph dynamically reflects the real-time status and interaction relationships of computing resources within the system, providing a structured data foundation for subsequent resource scheduling decisions.

[0061] Example 2: See Figure 3 This involves the specific operation process of the resource fluctuation entropy analysis module, which is responsible for extracting key performance sequences from the resource state characteristic tensor, calculating entropy indexes that characterize system uncertainty, and organizing them into a structured entropy matrix.

[0062] The feature separation unit receives a resource status feature tensor from the resource status acquisition module. This tensor contains data along three dimensions: device type, time window, and performance metrics. The processing first performs a slicing operation along the device type dimension, separating independent data subsets for GPU, CPU, and memory devices. For the GPU device subset, the memory read / write throughput sequence is extracted. This sequence is obtained by parsing the performance counter data of the GPU memory controller, recording the data transfer volume of memory read / write operations within each time window. The processing of the CPU device subset focuses on the instruction execution cycle sequence. This data originates from the instruction execution clock cycle count recorded by the CPU performance monitoring unit, reflecting the efficiency changes of instruction execution within different time windows. The data processing of the memory device subset targets the cache line contention count sequence, using the memory controller's conflict detection mechanism to count the number of cache line access conflicts occurring within each time window. All sequence data are kept aligned with the same time window as the original tensor to ensure timing consistency.

[0063] The entropy calculation unit performs information entropy analysis on the extracted sequence data. The entropy calculation for the memory read / write throughput sequence is based on the distribution characteristics of the throughput values. The throughput range is divided into several discrete intervals, and the frequency of data points within each interval is statistically analyzed. The information entropy value is calculated based on the frequency distribution. The instruction execution cycle sequence processing considers the fluctuation characteristics of cycle time, classifying cycle time values ​​according to execution efficiency. The distribution of the probability of occurrence at different levels is analyzed, thereby deriving a measure of the uncertainty in instruction execution. The entropy calculation for the cache line contention count sequence focuses on the regularity of contention events, analyzing the changing patterns of the number of contentions per unit time. The entropy value is calculated by statistically analyzing the probability distribution of different contention intensity intervals. All entropy calculation processes employ a sliding window mechanism to maintain synchronization with real-time data acquisition.

[0064] The matrix generation unit organizes the calculated entropy values ​​into a structured matrix according to device type. The rows of the matrix correspond to different device instances, including identification information for multiple GPU cores, CPU threads, and memory blocks. The columns contain three entropy value types: memory usage entropy, instruction pipeline entropy, and data access entropy. Each matrix element stores the calculation result for a specific device instance on a specific entropy value type, using floating-point format to preserve calculation precision. The matrix data structure is stored as a two-dimensional array, while maintaining a device instance metadata index to support fast query and update operations. The matrix update cycle is synchronized with the data processing cycle of the feature separation unit to ensure the real-time performance of the entropy metrics.

[0065] The entire implementation process adopts a pipelined architecture, with feature separation, entropy calculation, and matrix generation operating in parallel across three stages. Data is transferred between stages via a shared memory buffer to reduce data copying overhead. The entropy calculation algorithm is optimized for real-time requirements, employing an incremental calculation method to avoid full recalculation each time. The matrix data structure design supports dynamic expansion to adapt to changes in the number of device instances in the system. All calculation processes are executed in kernel mode, directly accessing hardware performance data to ensure data accuracy and timeliness. The output of this module is a heterogeneous resource entropy matrix, providing a quantitative basis for subsequent scheduling decisions. The matrix data is transmitted to the scheduling decision engine module via the system bus, participating in the resource contention feature identification and dynamic scheduling instruction generation process. The matrix data is also persistently stored in the system log for subsequent performance analysis and scheduling strategy optimization reference. The entire implementation process emphasizes a balance between computational efficiency and resource overhead, ensuring the accuracy of entropy calculation while keeping computational complexity within the system's tolerance range.

[0066] Example 3: See Figure 4 This involves the specific operational flow of the scheduling decision engine module. This module performs joint analysis based on a multi-dimensional scheduling graph and a heterogeneous resource entropy matrix to identify resource contention characteristics in computational tasks and generate a dynamic scheduling instruction set. The entire implementation process includes three core stages: bottleneck identification, strategy generation, and instruction set construction. Intelligent resource scheduling decisions are achieved through the collaborative work of algorithm analysis and system calls.

[0067] The bottleneck identification unit receives a multi-dimensional scheduling graph from the scheduling graph generation module and a heterogeneous resource entropy matrix from the resource fluctuation entropy analysis module. The processing first analyzes the connection weights in the multi-dimensional scheduling graph, detecting computing node connections that exceed dynamic thresholds. The connection weight thresholds are adaptively adjusted based on the system's historical operating status, determined by monitoring the normal fluctuation range in historical performance data. For each connection exceeding the threshold, the unit records its associated computing node identifier and connection type. Subsequently, the unit accesses the heterogeneous resource entropy matrix, extracting entropy data corresponding to these computing nodes, including memory occupancy entropy, instruction pipeline entropy, and data access entropy. The unit calculates the gradient of these entropy values ​​over time, identifying resource contention characteristics by analyzing gradient change patterns. The entropy gradient is calculated using the following formula:

[0068]

[0069] in: This represents the entropy gradient of device instance i at time t. This represents the entropy value of device instance i at time t. This represents the entropy value of device instance i at the previous time point. This indicates the sampling time interval. A gradient value exceeding a preset range indicates a sudden change in resource status; combined with abnormal connection weights, this accurately identifies resource contention. The unit records the identification results as a resource contention feature descriptor, containing information such as contention type, intensity level, and involved devices.

[0070] The policy generation unit generates specific scheduling policies based on resource contention feature descriptors. For GPU kernels exhibiting resource contention characteristics, the unit generates a scheduling priority vector. The vector generation is based on the current value and historical trend of memory occupancy entropy, combined with the kernel's computational task characteristics. Priority values ​​are calculated through weighted averages, considering factors such as entropy magnitude, rate of change, and task criticality. Vector elements are indexed by kernel identifiers, supporting dynamic updates and adjustments. For associated CPU threads, the unit generates a thread preemption policy sequence. The sequence contains a series of thread operation instructions, such as suspend, resume, and priority adjustment. Policy formulation is based on instruction pipeline entropy analysis, considering thread dependencies and execution efficiency requirements. Each operation in the sequence is timestamped and has an execution condition to ensure scheduling accuracy. For conflicting memory blocks, the unit generates memory lock identifiers. These identifiers specify the memory block's access control policy, including access permission settings, cache policy adjustments, and data migration flags. Identifier generation is based on data access entropy and conflict probability data, aiming to reduce memory access conflicts and improve data locality.

[0071] The instruction set building unit integrates scheduling priority vectors, thread preemption policy sequences, and memory lock flags to generate a dynamically scheduled instruction set executable by the system. The instruction set adopts a layered structure, comprising three main categories: device control instructions, thread management instructions, and memory operation instructions. Device control instructions are responsible for setting the scheduling priority of the GPU kernel and allocating computational tasks; the instruction format includes information such as the target device identifier, priority value, and execution time. Thread management instructions handle the suspension, resumption, and priority adjustment operations of CPU threads; the instruction structure includes thread identifier, operation type, and parameter settings. Memory operation instructions implement memory block locking, unlocking, and data migration functions; the instruction content includes the memory address range, operation type, and access policy. All instructions include timestamps and execution condition judgment logic, supporting both condition-triggered and timed execution modes. The instruction set is distributed to the hardware layer and operating system kernel through system call interfaces and executed by the corresponding device drivers and system services.

[0072] The entire implementation adopts an event-driven architecture, triggering the scheduling decision process when resource status changes. The system maintains a decision context database, storing the current resource status, historical decision records, and execution result feedback. The decision algorithm employs a combination of heuristic rules and machine learning, dynamically adjusting decision parameters based on the system's operating status. After instruction set generation, it undergoes legality checks and conflict detection to ensure the safety and validity of scheduling instructions. Anomalies during execution are captured in real-time by a monitoring mechanism, triggering a re-execution of the decision process or instruction adjustment.

[0073] The module's output dynamic scheduling instruction set is transmitted to the scheduling policy execution module via the system bus to guide specific resource scheduling operations. Simultaneously, the instruction set generation log and execution results are recorded in the system performance database for subsequent optimization and improvement of the decision-making algorithm. The entire implementation process emphasizes a balance between real-time performance and accuracy, controlling computational overhead while ensuring decision quality, and ensuring timely system scheduling responses.

[0074] Example 4: This example involves the collaborative operation of the scheduling policy execution module and the resource reallocation module. The system receives a dynamic scheduling instruction set from the scheduling decision engine module, parses and executes these instructions to achieve real-time adjustment of computing resources, monitors system performance during execution, and triggers necessary resource reallocation operations. The scheduling policy execution module first parses the GPU kernel scheduling priority vector in the dynamic scheduling instruction set. Each element in the vector corresponds to a unique identifier for a GPU kernel and its corresponding priority value. The system accesses the kernel's command queue through the GPU driver interface and reorders the computation instructions according to the priority values. Instructions from high-priority kernels are promoted to the front of the queue, gaining priority access to execution resources. Instructions from low-priority kernels are deferred and can be temporarily suspended if necessary. The instruction reordering operation maintains the dependencies between computational tasks and avoids data inconsistency issues.

[0075] The module executes the thread preemption policy sequence through the operating system kernel's scheduler interface. Each entry in the sequence contains the target thread identifier, operation type, and execution parameters. Suspension operations are implemented by changing the thread state to blocked, pausing the thread's execution without releasing its occupied resources. Resume operations reset the thread state to ready, allowing it to re-enter the scheduling queue. Priority adjustment operations modify the thread's scheduling priority value, affecting the operating system scheduler's decision logic. All operations are logged in an audit log for subsequent analysis and troubleshooting.

[0076] The handling of memory lock flags is accomplished by the memory management unit. These flags specify the address range and access control policy of the target memory block. The module sets access permissions for memory pages via system calls, restricting access operations by specific processes or threads. For memory regions requiring isolation, the module modifies page table entries, setting read-only or inaccessible flags. When an unauthorized access attempt is detected, the memory controller generates a protection exception, which is then recorded by an exception handler, who executes a predefined processing procedure.

[0077] The performance monitoring unit of the resource reallocation module continuously collects key system performance indicators. The monitored data includes the GPU kernel instruction queue depth, the CPU thread wake-up queue length, and the backlog of memory block access requests. The instruction queue depth is read from the GPU command processor's status register, reflecting the backlog of unexecuted instructions. The wake-up queue length is obtained through the thread scheduler's statistics interface, indicating the number of threads in a waiting state. The backlog of access requests is monitored through the memory controller's performance counters, displaying the number of incomplete memory operation requests (see Table 1).

[0078] Table 1: Performance monitoring indicators and threshold settings.

[0079]

[0080] The migration decision unit triggers resource reallocation operations based on a comparison of monitoring data with thresholds. When the instruction queue depth continuously exceeds the full load threshold, the unit initiates GPU computing task migration. Migration target selection is based on the current load status of each GPU core, prioritizing cores with more idle resources. When the wake-up queue length exceeds the latency threshold, the unit reallocates CPU thread affinity, optimizing scheduling efficiency by modifying the binding relationship between threads and processor cores. When the backlog of access requests reaches the conflict threshold, the unit initiates a memory block copy operation, copying frequently accessed data to free storage areas to reduce access conflicts.

[0081] The load migration unit performs the specific resource reallocation operations. GPU computing tasks are migrated via the PCIe bus, using a DMA engine for fast memory-to-memory copying. The migration process maintains the execution state of the computing tasks, including register contents and command queue status. CPU thread affinity adjustment is implemented through the operating system scheduler interface, modifying the mapping between threads and processor cores. Memory block copying operations are performed by the memory controller, updating page table mappings and cache consistency information after copying. All migration operations ensure data consistency and integrity, preventing data errors or loss during the migration process.

[0082] The entire implementation process employs an asynchronous processing mechanism, with monitoring and migration operations executed in parallel. The system maintains a migration status database, recording ongoing migration operations and resource status changes. Migration decisions consider overall system load balancing to avoid new resource contention caused by migration operations. Anomalies during execution are handled through retry mechanisms and rollback strategies to ensure system reliability. Migration effectiveness is evaluated through continuous performance monitoring, with dynamic adjustments made as necessary.

[0083] Example 5: See Figure 5 This system involves the collaborative workflow of the elastic scaling module and the resource supply module. Targeting the computational characteristics of generative AI tasks, the system predicts resource demands by analyzing the characteristics of each task execution stage and dynamically adjusts the supply of computing resources to achieve elastic allocation and recycling of resources. The task parsing unit of the elastic scaling module continuously monitors the execution process of generative AI tasks. The unit obtains task execution logs and performance profiling data through the task profiling interface to identify different computational stages of the task. The matrix multiplication stage is characterized by a large number of parallel computation operations and high floating-point operation intensity, identified by monitoring the GPU's floating-point instruction execution rate and the frequency of matrix operation function calls. The backpropagation stage is characterized by frequent gradient calculations and weight update operations, identified by tracking the execution time and memory access patterns of the backpropagation operator. The gradient update stage involves parameter adjustment and synchronization operations, distinguished by analyzing the execution flow and communication overhead of the optimizer function. The time consumption ratio of each stage is calculated by statistically analyzing the percentage of each stage's execution time to the total task time; the stage division uses a pattern recognition algorithm based on computational characteristics.

[0084] The demand forecasting unit predicts resource requirements based on the current resource consumption rate and the computational complexity of the next stage. The current resource consumption rate is obtained by real-time monitoring of GPU core utilization, CPU thread throughput, and memory bandwidth utilization; these metrics are collected in time-series format, and their changing trends are calculated. The assessment of the computational complexity of the next stage is based on task characteristic parameters, including the dimension of matrix operations, the number of iterations, and data type precision. The prediction of GPU core demand increments comprehensively considers the current GPU utilization level and the floating-point operations required in the next stage, estimating the change in the number of cores needed to meet performance targets using a linear regression model. The CPU thread expansion factor is calculated based on the current thread load rate and the task parallelism requirements of the next stage, using a load balancing algorithm to determine the adjustment ratio of the thread pool size. The derivation of the memory block expansion coefficient is based on the current memory utilization rate and the data storage requirements of the next stage, predicting the required increase in storage capacity through memory usage trend analysis. The forecasting process uses a sliding window mechanism, periodically updating forecast parameters to adapt to dynamic changes during task execution.

[0085] The sequence generation unit encodes the predicted resource demand parameters into a structured sequence of resource scaling requirements. The encoding format uses a standardized data serialization protocol, including resource type identifiers, operation type fields, and quantity value fields. GPU kernel demand increments are encoded as instruction blocks containing device type, operation instructions, and quantity changes. Operation instructions include instance creation, destruction, or configuration adjustments. CPU thread expansion factors are converted into thread pool adjustment instructions, specifying the increase or decrease in the number of threads and adjustment time constraints. Memory block expansion coefficients are encoded as memory management instructions, including capacity adjustment values, allocation strategies, and priority settings. Each instruction item in the sequence includes a timestamp and execution condition description, supporting both immediate execution and timed triggering modes. The entire sequence is verified using checksums to ensure data integrity and consistency.

[0086] The resource supply module monitors the resource expansion and contraction demand sequence in real time and executes corresponding resource adjustment operations. For GPU kernel computing instance management, the module dynamically starts and stops computing instances through the GPU driver programming interface. The instance startup process includes allocating GPU memory resources, initializing the execution context, and registering device functions. The instance stopping process involves execution context cleanup, GPU memory resource release, and device state reset. Adjusting the CPU thread pool size is implemented through the thread pool management interface. The thread creation process includes allocating stack space, setting scheduling attributes, and registering callback functions. The thread destruction process includes waiting for execution completion, releasing resources, and unregistering information. Adjusting the memory block pre-allocation strategy is completed through the memory management interface, including reserving physical memory pages, setting mapping relationships, and configuring caching strategies. All resource operations are logged in detail, including operation time, resource identifier, and execution result.

[0087] The entire implementation process adopts an event-driven architecture, with resource demand forecasting and resource supply adjustment executed asynchronously. The system maintains a resource status database, recording current resource allocation and historical usage records. Resource adjustment operations follow a gradual approach to avoid frequent and drastic resource fluctuations. Resource dependencies are considered during operation execution to ensure the rationality of the adjustment sequence. An exception handling mechanism monitors the execution results of resource operations, retrying or rolling back failed operations.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic scheduling system for generative AI heterogeneous computing resources on a PC terminal, characterized in that, include: The resource status acquisition module is used to capture the running status parameters of the GPU kernel, CPU thread and memory block in the PC terminal in real time, and perform heterogeneous data format normalization processing on the running status parameters to generate resource status feature tensors. The scheduling graph generation module is used to parse the computing node connection topology of the resource state feature tensor, extract the data transmission bandwidth between GPU cores, the dependency chain between CPU threads and the memory block shared area mapping relationship, and construct a multi-dimensional scheduling graph that includes computing node connection weights and cross-device interaction latency. The resource fluctuation entropy analysis module is used to separate the real-time load feature vector of each computing unit in the resource state feature tensor, calculate the GPU core's memory occupancy entropy, the CPU thread's instruction pipeline entropy and the memory block's data access entropy based on the corresponding real-time load feature vector, and summarize them to generate a heterogeneous resource entropy value matrix. The scheduling decision engine module is used to perform joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy matrix to identify the resource contention characteristics of bottleneck nodes in computing tasks and generate a dynamic scheduling instruction set that includes GPU kernel scheduling priority vectors, CPU thread preemption strategy sequences and memory block locking identifiers.

2. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 1, characterized in that, The resource status acquisition module specifically includes: The heterogeneous parameter capture unit synchronously collects the shader core utilization of the GPU kernel, the instruction cache hit rate of the CPU thread, and the page fault rate of the memory block. The data fusion unit converts the shader core utilization, instruction cache hit rate, and page error rate into a time-series data stream with a unified timestamp. The tensor generation unit extracts the spatial correlation features of the time-series data stream through a three-dimensional convolution kernel and outputs a resource status feature tensor with dimensions of [device type × time window × performance index].

3. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 1, characterized in that, The scheduling map generation module specifically includes: The topology parsing unit identifies cross-device synchronization semaphores between the GPU kernel and CPU thread in the resource state feature tensor and marks the direct memory access channel from the memory block to the GPU kernel. The weight calculation unit quantifies the fluctuation coefficient of data transfer bandwidth between GPU cores, the waiting latency of dependency chains between CPU threads, and the probability of conflict in the mapping relationship of shared memory blocks. The graph construction unit uses the cross-device interaction latency as the weight of the connection edge and the computing node as the vertex to construct a multi-dimensional scheduling graph containing vertex attributes and edge weights.

4. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 1, characterized in that, The resource fluctuation entropy analysis module specifically includes: The feature separation unit extracts the GPU kernel's video memory read / write throughput sequence, the CPU thread's instruction execution cycle sequence, and the memory block's cache line contention count sequence from the resource state feature tensor. The entropy calculation unit calculates the video memory occupancy entropy based on the video memory read / write throughput sequence, calculates the instruction pipeline entropy based on the instruction execution cycle sequence, and calculates the data access entropy based on the cache line contention count sequence. The matrix generation unit organizes the video memory occupancy entropy, instruction pipeline entropy, and data access entropy into a heterogeneous resource entropy matrix according to device type.

5. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 1, characterized in that, The scheduling decision engine module specifically includes: The bottleneck identification unit detects computing nodes in the multidimensional scheduling graph whose connection weights exceed a preset threshold, and locates resource competition characteristics by combining the abrupt gradient of the corresponding entropy value in the heterogeneous resource entropy matrix. The policy generation unit generates scheduling priority vectors for GPU kernels with resource contention characteristics, generates thread preemption policy sequences for associated CPU threads, and generates memory lock identifiers for conflicting memory blocks. The instruction set construction unit integrates the scheduling priority vector, thread preemption strategy sequence, and memory lock flag to generate a dynamic scheduling instruction set.

6. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 5, characterized in that, Also includes: The scheduling policy execution module is used to parse the GPU kernel scheduling priority vector of the dynamic scheduling instruction set, perform computation instruction reordering operations on the target GPU kernel, and suspend the specified CPU thread according to the thread preemption policy sequence. Isolate conflicting memory block access requests based on the memory lock identifier; The resource reallocation module is used to monitor the GPU kernel instruction throughput, CPU thread wake-up latency, and memory block access conflict rate after the execution of the scheduling policy. When the instruction throughput is lower than the performance threshold or the wake-up latency exceeds the tolerance window, the computing resource load migration operation is triggered.

7. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 6, characterized in that, The resource reallocation module specifically includes: The performance monitoring unit collects the instruction queue depth of the GPU kernel, the wait-to-wake queue length of the CPU thread, and the backlog of memory block access requests. The migration decision unit migrates GPU computing tasks to idle kernels when the instruction queue depth continuously exceeds the full load threshold; reallocates CPU thread affinity when the wake-up queue length exceeds the latency threshold; and copies memory blocks to free storage areas when the access request backlog reaches the conflict threshold. The load migration unit performs GPU computing task migration operations based on the PCIe bus, adjusts CPU thread affinity through the operating system scheduler, and completes memory block copying operations using the memory controller.

8. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 1, characterized in that, Also includes: The elastic scaling module is used to analyze the computational stage characteristics of generative AI tasks, predict the number of GPU cores, CPU thread scale, and memory block capacity required for the next stage, and generate a resource scaling demand sequence. The resource supply module is used to respond to the resource expansion and contraction demand sequence, dynamically start and stop the computing instances of the GPU kernel, allocate the CPU thread pool size on demand, and adjust the memory block pre-allocation strategy.

9. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in claim 8, characterized in that, The elastic expansion module specifically includes: The task parsing unit identifies the time consumption ratios of the matrix multiplication stage, backpropagation stage, and gradient update stage in generative AI tasks. The demand forecasting unit outputs the GPU core demand increment, CPU thread expansion factor, and memory block expansion coefficient based on the current resource consumption rate and the computational complexity of the next stage. The sequence generation unit encodes the GPU kernel demand increment, CPU thread expansion factor, and memory block expansion coefficient into a resource expansion / shrinkage demand sequence.

10. A method for dynamic scheduling of heterogeneous computing resources for generative AI on a PC terminal, characterized in that, The system includes all modules and method flows of the generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in any one of claims 1 to 9.

Images