Generative AI heterogeneous computing resource dynamic scheduling method and system of PC terminal

Abstract

The invention relates to the technical field of PC (Personal Computer) terminal AI (Artificial Intelligence) computing, and discloses a method and a system for dynamically scheduling generative AI heterogeneous computing resources of a PC terminal. The system comprises a resource state acquisition module, a scheduling graph generation module, a resource fluctuation entropy analysis module and a scheduling decision engine module. The resource state acquisition module captures running state parameters of a GPU kernel, a CPU thread and a memory block in real time, and generates a resource state feature tensor through normalization processing; the scheduling atlas generation module analyzes and computes the node connection topology, extracts the correlation between the devices, and constructs a multi-dimensional scheduling atlas; the resource fluctuation entropy analysis module separates the load feature vectors, calculates the entropy of each calculation unit and generates a heterogeneous resource entropy matrix; and the scheduling decision engine module jointly analyzes the atlas and the matrix, identifies bottleneck node resource competition characteristics, generates a dynamic scheduling instruction set, adapts to generative AI task requirements, and ensures efficient and stable operation of the task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of PC terminal AI computing, in particular to a PC terminal generative AI heterogeneous computing resource dynamic scheduling method and system. BACKGROUND

[0002] With the rapid penetration of generative AI technology in the field of PC terminals, various AI applications have shown significant features of dynamic and heterogeneous demand for computing resources. Current PC terminals are generally equipped with heterogeneous computing architectures composed of GPU, CPU and multi-level memory. However, the existing resource scheduling mechanism cannot adapt to the complex needs of generative AI tasks.

[0003] Traditional scheduling systems mostly use single-dimensional resource state collection methods, only focusing on basic parameters such as CPU usage and GPU memory occupation, lacking comprehensive capture of data interaction status between heterogeneous devices, resulting in inability to accurately reflect the actual running state of computing nodes. In the scheduling decision-making process, static priority allocation strategies are often relied on, ignoring the dynamic changing dependency and data transmission needs between computing units in generative AI tasks, such as GPU kernel data transmission bandwidth fluctuations, CPU thread instruction dependency chain changes, etc., resulting in a deviation between scheduling instructions and actual computing needs.

[0004] The existing system lacks effective analysis means for resource load fluctuation characteristics, and cannot quantitatively evaluate the dynamic changes of GPU kernel memory occupation, the stability of CPU thread instruction flow, and the density of memory block data access, making it difficult to predict resource competition risks in advance when facing frequent load peaks in generative AI tasks, and thus causing computing task stalls, data transmission delays and other problems. In addition, the traditional scheduling mechanism fails to build an effective cross-device collaborative scheduling framework, with resource allocation between GPU, CPU and memory being independent of each other, unable to dynamically adapt heterogeneous resources according to real-time computing needs of tasks, greatly limiting the running efficiency and response speed of PC terminal generative AI tasks. SUMMARY

[0005] The purpose of the present application is to provide a PC terminal generative AI heterogeneous computing resource dynamic scheduling method and system to solve the problems raised in the background.

[0006] To achieve the above purpose, the present application provides a PC terminal generative AI heterogeneous computing resource dynamic scheduling system, which comprises: A resource state acquisition module for real-time capturing of running state parameters of GPU kernels, CPU threads and memory blocks in a PC terminal, performing heterogeneous data format normalization processing on the running state parameters, and generating a resource state feature tensor; The scheduling graph generation module is configured to parse a computing node connection topology of the resource state feature tensor, extract GPU inter-kernel data transmission bandwidth, CPU inter-thread dependency relationship chain and memory block shared area mapping relationship, and construct a multi-dimensional scheduling graph containing computing node connection weight and cross-device interaction delay. The resource fluctuation entropy analysis module is configured to separate real-time load feature vectors of each computing unit in the resource state feature tensor, calculate GPU kernel memory occupancy entropy, CPU thread instruction pipeline entropy and memory block data access entropy according to the corresponding real-time load feature vectors, and generate a heterogeneous resource entropy value matrix. The scheduling decision engine module is configured to perform joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy value matrix, identify resource competition characteristics of a computing task bottleneck node, and generate a dynamic scheduling instruction set containing GPU kernel scheduling priority vector, CPU thread preemption strategy sequence and memory block locking identifier.

[0007] Preferably, the resource state acquisition module specifically comprises: The heterogeneous parameter capture unit synchronously acquires GPU kernel shader core utilization rate, CPU thread instruction cache hit rate and memory block page error rate. The data fusion unit converts the shader core utilization rate, instruction cache hit rate and page error rate into time series data streams of a unified timestamp. The tensor generation unit extracts spatial correlation characteristics of the time series data streams through a three-dimensional convolution kernel, and outputs a resource state feature tensor with dimensions of [device type x time window x performance indicator].

[0008] Preferably, the scheduling graph generation module specifically comprises: The topology analysis unit identifies cross-device synchronization semaphores of GPU kernels and CPU threads in the resource state feature tensor, and labels direct memory access channels of memory blocks to GPU kernels. The weight calculation unit quantifies fluctuation coefficients of GPU inter-kernel data transmission bandwidth, waiting delay of CPU inter-thread dependency relationship chain and conflict probability of memory block shared area mapping relationship. The graph construction unit constructs a multi-dimensional scheduling graph containing vertex attributes and edge weights by taking the cross-device interaction delay as a connection edge weight and taking the computing node as a vertex.

[0009] Preferably, the resource fluctuation entropy analysis module specifically comprises: The feature separation unit extracts GPU kernel memory read-write throughput sequences, CPU thread instruction execution period sequences and memory block cache line contention count sequences from the resource state feature tensor. an entropy value calculation unit configured to calculate a memory occupancy entropy based on the memory read-write throughput sequence, calculate an instruction pipeline entropy based on the instruction execution period sequence, and calculate a data access entropy based on the cache line contention count sequence; a matrix generation unit configured to organize the memory occupancy entropy, the instruction pipeline entropy, and the data access entropy into a heterogeneous resource entropy value matrix according to device types.

[0010] Preferably, the scheduling decision engine module specifically includes: a bottleneck identification unit configured to detect a calculation node with a connection weight exceeding a preset threshold in the multi-dimensional scheduling graph, and locate resource contention features based on a mutation gradient of a corresponding entropy value in the heterogeneous resource entropy value matrix; a strategy generation unit configured to generate a scheduling priority vector for a GPU kernel with resource contention features, generate a thread preemption strategy sequence for an associated CPU thread, and generate a memory locking identifier for a conflict memory block; an instruction set construction unit configured to integrate the scheduling priority vector, the thread preemption strategy sequence, and the memory locking identifier to generate a dynamic scheduling instruction set.

[0011] Preferably, the system further includes: a scheduling strategy execution module configured to parse the GPU kernel scheduling priority vector of the dynamic scheduling instruction set, perform a calculation instruction reordering operation on a target GPU kernel, suspend a specified CPU thread according to the thread preemption strategy sequence, and isolate a conflict memory block access request according to the memory locking identifier; a resource reallocation module configured to monitor a GPU kernel instruction throughput, a CPU thread wake-up latency, and a memory block access conflict rate after the execution of the scheduling strategy, and trigger a computing resource load migration operation when the instruction throughput is lower than a performance threshold or the wake-up latency exceeds a tolerance window.

[0012] Preferably, the resource reallocation module specifically includes: a performance monitoring unit configured to collect an instruction queue depth of a GPU kernel, a waiting wake-up queue length of a CPU thread, and an access request accumulation amount of a memory block; a migration decision unit configured to migrate a GPU computing task to an idle kernel when the instruction queue depth continuously exceeds a full load threshold, reallocate a CPU thread affinity when the waiting wake-up queue length exceeds a latency threshold, and copy a memory block to an idle storage area when the access request accumulation amount reaches a conflict threshold; a load migration unit configured to perform a GPU computing task migration operation based on a PCIe bus, adjust a CPU thread affinity through an operating system scheduler, and complete a memory block copying operation using a memory controller.

[0013] Preferably, the system further includes: An elastic scaling module is configured to analyze the characteristics of the computing stage of a generative AI task, predict the number of GPU kernels, the size of CPU threads and the capacity of memory blocks required in the next stage, and generate a resource scaling demand sequence. A resource supply module is configured to dynamically start and stop the computing instances of GPU kernels, allocate the size of CPU thread pools on demand, and adjust the pre-allocation strategy of memory blocks in response to the resource scaling demand sequence.

[0014] Preferably, the elastic scaling module specifically comprises: A task analysis unit is configured to identify the time consumption proportion of the matrix multiplication stage, the back propagation stage and the gradient update stage in the generative AI task. A demand prediction unit is configured to output the GPU kernel demand increment, the CPU thread expansion factor and the memory block expansion coefficient based on the current stage resource consumption rate and the next stage computing complexity. A sequence generation unit is configured to encode the GPU kernel demand increment, the CPU thread expansion factor and the memory block expansion coefficient into a resource scaling demand sequence.

[0015] Preferably, the present application further comprises a generative AI heterogeneous computing resource dynamic scheduling method for a PC terminal, which comprises all the modules and method processes of the above-mentioned generative AI heterogeneous computing resource dynamic scheduling system for a PC terminal.

[0016] Compared with the prior art, the present application has the following advantages: By constructing a multi-dimensional resource state acquisition mechanism, the running state parameters of GPU kernels, CPU threads and memory blocks in the PC terminal can be captured in real time, and a resource state feature tensor is generated through heterogeneous data format normalization processing, breaking the limitations of single-dimensional data acquisition of traditional scheduling systems and realizing comprehensive and accurate perception of the running state of heterogeneous computing resources, so that the scheduling decision can be based on more complete resource state information.

[0017] The scheduling atlas generation module extracts the data transmission bandwidth between GPU kernels, the dependency relationship chain between CPU threads and the memory block sharing area mapping relationship by analyzing the computing node connection topology of the resource state feature tensor, constructs a multi-dimensional scheduling atlas containing the connection weight of the computing nodes and the cross-device interaction delay, clearly presents the association relationship and data interaction characteristics between the heterogeneous computing units, provides an intuitive and comprehensive topology reference for subsequent scheduling decisions, effectively solves the problem of ignoring the dynamic dependency relationship between devices in the traditional scheduling mechanism, and makes the scheduling strategy better adapt to the complex cooperation requirements between the computing units in the generative AI task.

[0018] The resource fluctuation entropy analysis module can separate the real-time load feature vector of each computing unit from the resource state feature tensor, and calculate the memory occupation entropy of the GPU kernel, the instruction flow entropy of the CPU thread and the data access entropy of the memory block, and generate a heterogeneous resource entropy value matrix, so as to realize the quantitative representation of the load fluctuation characteristics of the heterogeneous resources. Through the entropy value matrix, the dynamic change law of the load of each computing unit can be clearly mastered, the computing unit with abnormal load fluctuation can be identified in time, the potential resource competition risk can be perceived in advance, the abnormal operation of the computing task caused by the load peak burst can be avoided, and the stability of the generative AI task operation is ensured.

[0019] The scheduling decision engine module performs joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy value matrix, can accurately identify the resource competition characteristics of the computing task bottleneck node, and generate a dynamic scheduling instruction set containing the GPU kernel scheduling priority vector, the CPU thread preemption strategy sequence and the memory block locking identifier. This joint analysis decision mode fully integrates the computing node topology relationship and the load fluctuation characteristics, so that the scheduling instruction can accurately adapt to the actual needs of different computing units, realizes the cooperative dynamic scheduling of GPU, CPU and memory resources, avoids the problem of independent allocation of heterogeneous resources, can flexibly adjust the resource allocation strategy according to the real-time computing demand of the generative AI task, improves the utilization efficiency of the heterogeneous computing resources of the PC terminal, ensures that the generative AI task can be efficiently and smoothly run, and improves the user experience of the PC terminal generative AI application. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The timing diagram of the PC terminal generative AI heterogeneous computing resource dynamic scheduling system described in the application; Figure 2 The flowchart of the working principle of the scheduling graph generation module; Figure 3 The flowchart of the working principle of the resource fluctuation entropy analysis module; Figure 4 The flowchart of the working principle of the scheduling decision engine module; Figure 5 The flowchart of the working principle of the elastic scaling module. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0022] Please refer to Figure 1The application provides a PC terminal generative AI heterogeneous computing resource dynamic scheduling system, and the system comprises: The resource state acquisition module captures the running state parameters of the PC terminal in real time, such as the utilization rate of the shader core of the GPU kernel, the instruction cache hit rate of the CPU thread, and the page error rate of the memory block, and performs heterogeneous data format normalization processing on these parameters to generate a resource state feature tensor. The scheduling graph generation module analyzes the calculation node connection topology of the resource state feature tensor, extracts the data transmission bandwidth between GPU kernels, the dependency relationship chain between CPU threads, and the memory block sharing area mapping relationship, and constructs a multi-dimensional scheduling graph containing the calculation node connection weight and the cross-device interaction delay. The resource fluctuation entropy analysis module separates the real-time load feature vector of each calculation unit from the resource state feature tensor, calculates the GPU kernel memory occupation entropy, the CPU thread instruction flow entropy, and the memory block data access entropy, and generates a heterogeneous resource entropy value matrix. The scheduling decision engine module performs joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy value matrix, identifies the resource competition characteristics of the calculation task bottleneck node, and generates a dynamic scheduling instruction set containing the GPU kernel scheduling priority vector, the CPU thread preemption strategy sequence, and the memory block locking identifier. The system realizes efficient resource scheduling of generative AI tasks in a heterogeneous computing environment.

[0023] Embodiment 1: refer to Figure 2 , which relates to the specific operation process of the resource state acquisition module and the scheduling graph generation module. The system runs in the PC terminal environment, and for the heterogeneous computing needs of the generative AI task, it realizes real-time capture of resource state and construction of scheduling graph through the cooperation of the underlying hardware interface and the operating system kernel. The heterogeneous parameter capture unit of the resource state acquisition module synchronously collects the utilization rate of the shader core of the GPU kernel, the instruction cache hit rate of the CPU thread, and the page error rate of the memory block by directly accessing the hardware performance counter and the system call interface. The utilization rate of the shader core is derived from the performance monitoring interface provided by the GPU driver, and is sampled at a millisecond level to capture the actual workload of the calculation unit. The instruction cache hit rate is obtained through the performance monitoring unit of the CPU to record the hit and miss events of cache access at all levels. The page error rate is collected through the event counter of the memory management unit to count the frequency of page fault exceptions. These parameters are temporarily stored in the kernel buffer in the form of raw binary data.

[0024] The data fusion unit processes the above-mentioned multi-source heterogeneous data and converts it into a uniform timestamp time series data stream. Time synchronization is based on a high-precision clock source, and a subset of the network time protocol is used to align the clocks between devices, eliminating the timing deviation caused by collection delay. The utilization rate of the shader core, the instruction cache hit rate and the page error rate data are resampled onto a common time axis to form a sequence of time-labeled data points. Each data point contains a device identifier, a timestamp and a parameter value triple, and the data stream is stored in a ring buffer structure to ensure data continuity during real-time processing.

[0025] The tensor generation unit extracts spatial features from the time series data stream. The three-dimensional convolution kernel is designed as a set of learnable filters, and its size is configured to cover three dimensions of device type, time window and performance indicator. Convolution operation slides along the time axis to calculate the correlation between adjacent time points, while capturing the mutual influence of resource states across device types. In the dimension structure of the output resource state feature tensor, the device type dimension distinguishes between GPU, CPU and memory; the time window dimension retains the historical data of the last 10 sampling periods; and the performance indicator dimension accommodates the normalized utilization rate, hit rate and error rate values. The tensor is stored in a floating-point tensor format as the basic data structure for subsequent processing of the system.

[0026] The topology analysis unit of the scheduling atlas generation module analyzes the internal correlation of the resource state feature tensor. By analyzing the cross-device synchronization semaphores between GPU kernels and CPU threads, the cooperation relationship between devices is identified. The synchronization semaphores are derived from the synchronization primitive logs of the operating system kernel, including mutex, semaphore and barrier operations. The direct memory access channel from the memory block to the GPU kernel is labeled by monitoring the PCIe bus transactions, recording the source address, target address and data volume information of DMA transmission. These topological relationships are preliminarily constructed in the form of a graph structure, where the vertices represent computing nodes and the edges represent interaction relationships.

[0027] The weight calculation unit quantitatively processes the topological relationships. The fluctuation coefficient of the data transmission bandwidth between GPU kernels is calculated by statistical variation of bandwidth values, using the ratio of standard deviation to mean value in the sliding window as a measure. The waiting delay of the dependency chain between CPU threads is extracted from the waiting queue of the thread scheduler, and the average value of thread blocking time is calculated. The conflict probability of memory block shared area mapping relationship is monitored by monitoring memory access conflict events, and the ratio of conflict times to total access times per unit time is calculated. These quantitative values are normalized to convert them to weight values between 0 and 1.

[0028] The atlas construction unit integrates the above quantitative results into a multi-dimensional scheduling atlas. The computing nodes are the vertices, and the vertex attributes include device type identification, resource capacity, and current load state. The interaction relationship is the connection edge, and the edge weight includes three attributes: fluctuation coefficient, waiting time delay, and conflict probability. The atlas uses an attribute graph model to represent, supporting multiple graph algorithm operations. The atlas data is stored in the form of an adjacency list, while maintaining a vertex attribute table and an edge attribute table, facilitating fast query and update. The multi-dimensional scheduling atlas dynamically reflects the real-time state and interaction relationship of the computing resources in the system, providing a structured data foundation for subsequent resource scheduling decisions.

[0029] Embodiment 2: refer to Figure 3 , which involves the specific operation process of the resource fluctuation entropy analysis module. This module is responsible for extracting key performance sequences from the resource state feature tensor, calculating entropy value indicators representing system uncertainty, and organizing them into a structured entropy value matrix.

[0030] The feature separation unit receives the resource state feature tensor from the resource state acquisition module, which contains data in three dimensions: device type, time window, and performance indicator. The processing process first performs slicing along the device type dimension to separate independent data subsets for GPU, CPU, and memory devices. For the GPU device subset, the memory read-write throughput sequence is extracted. This sequence is obtained by analyzing the performance counter data of the GPU memory controller and records the data transfer volume of memory read-write operations in each time window. The processing of the CPU device subset focuses on the instruction execution cycle sequence, which is derived from the number of instruction execution clock cycles recorded by the CPU performance monitoring unit, reflecting the efficiency changes of instruction execution in different time windows. The data processing of the memory device subset targets the cache line contention count sequence, which is obtained by counting the number of cache line access conflicts in each time window through the conflict detection mechanism of the memory controller. All sequence data maintains the same time window alignment as the original tensor, ensuring time consistency.

[0031] The entropy value calculation unit performs information entropy analysis on the extracted sequence data. The entropy value calculation of the memory read-write throughput sequence is based on the distribution characteristics of the throughput value, which divides the throughput range into several discrete intervals, counts the frequency of data points in each interval, and calculates the information entropy value according to the frequency distribution. The processing of the instruction execution cycle sequence considers the fluctuation characteristics of the cycle time, classifies the cycle time values according to execution efficiency, analyzes the distribution of probability of different levels, and then derives the uncertainty measure of instruction execution. The entropy value calculation of the cache line contention count sequence focuses on the regularity of contention events, analyzes the change pattern of contention frequency in unit time, and calculates the entropy value by statistical probability distribution of different contention intensity intervals. All entropy value calculation processes use a sliding window mechanism to maintain synchronous updating with real-time data acquisition.

[0032] The matrix generation unit organizes the calculated entropy values of each type according to the device type into a structured matrix. The row dimension of the matrix corresponds to different device instances, including the identification information of multiple GPU kernels, CPU threads, and memory blocks. The column dimension includes three entropy value types: memory occupancy entropy, instruction flow entropy, and data access entropy. Each matrix element stores the calculation result of a specific device instance on a specific entropy value type, using a floating-point number format to preserve the calculation accuracy. The matrix data structure uses a two-dimensional array form for storage, while maintaining the device instance metadata index to support fast query and update operations. The matrix update period is synchronized with the data processing period of the feature separation unit to ensure the real-time nature of the entropy value indicators.

[0033] The entire implementation process adopts a pipeline architecture, with the three stages of feature separation, entropy value calculation, and matrix generation operating in parallel. Data is transferred between stages through shared memory buffers, reducing data duplication overhead. The entropy value calculation algorithm is optimized for real-time requirements, using incremental calculation to avoid full re-computation each time. The matrix data structure design supports dynamic expansion to adapt to changes in the number of device instances in the system. All calculation processes are executed in the kernel state, directly accessing hardware performance data to ensure data accuracy and timeliness. The output of this module is the heterogeneous resource entropy value matrix, which provides quantitative basis for subsequent scheduling decisions. The matrix data is transmitted to the scheduling decision engine module through the system bus, participating in resource competition feature recognition and dynamic scheduling instruction generation processes. Matrix data is also persistently stored in the system log for subsequent performance analysis and scheduling strategy optimization reference. The entire implementation process focuses on balancing calculation efficiency and resource overhead, ensuring entropy value calculation accuracy while controlling calculation complexity within the system's bearable range.

[0034] Embodiment 3: Refer to Figure 4 , which involves the specific operation process of the scheduling decision engine module. This module performs joint analysis based on the multi-dimensional scheduling graph and the heterogeneous resource entropy value matrix, identifies resource competition features in the computing task, and generates a set of dynamic scheduling instructions. The entire implementation process includes three core links: bottleneck identification, strategy generation, and instruction set construction, which realize intelligent decision-making for resource scheduling through the collaborative work of algorithm analysis and system call.

[0035] The bottleneck recognition unit receives the multi-dimensional scheduling graph from the scheduling graph generation module and the heterogeneous resource entropy value matrix from the resource fluctuation entropy analysis module. The processing process first analyzes the connection weight in the multi-dimensional scheduling graph, detects those calculation node connections that exceed the dynamic threshold. The threshold of the connection weight is adaptively adjusted according to the historical running state of the system, which is determined by monitoring the normal fluctuation range in the historical performance data. For each connection exceeding the threshold, the unit records the associated calculation node identifier and the connection type. Subsequently, the unit accesses the heterogeneous resource entropy value matrix, extracts the entropy value data corresponding to these calculation nodes, including the memory occupancy entropy, instruction pipeline entropy and data access entropy. The unit calculates the change gradient of these entropy values in the time dimension, and identifies the resource competition characteristics by analyzing the gradient change pattern. The calculation of the entropy value gradient uses the following formula:

[0036] Wherein: represents the entropy value gradient of device instance i at time t, represents the entropy value of device instance i at time t, represents the entropy value of device instance i at the previous time point, represents the sampling time interval. The gradient value exceeding the preset range indicates that the resource state has mutated, combined with the connection weight anomaly, the resource competition phenomenon can be accurately identified. The unit records the identification result as a resource competition characteristic descriptor, including competition type, intensity level and involved device information.

[0037] The strategy generation unit generates specific scheduling strategies according to the resource competition characteristic descriptor. For GPU kernels with resource competition characteristics, the unit generates a scheduling priority vector. The generation of the vector is based on the current value and historical trend of the memory occupancy entropy, combined with the calculation task characteristics of the kernel. The priority value is obtained by weighted calculation, considering factors such as entropy value, change rate and task criticality. The vector elements are indexed by kernel identifier, supporting dynamic update and adjustment. For associated CPU threads, the unit generates a thread preemption strategy sequence. The sequence contains a series of thread operation instructions, such as suspension, recovery, priority adjustment, etc. The strategy is based on the analysis results of the instruction pipeline entropy, considering the dependency relationship between threads and the execution efficiency requirements. Each operation in the sequence has a timestamp and an execution condition, ensuring the accuracy of the scheduling. For conflicting memory blocks, the unit generates a memory lock identifier. The identifier specifies the access control strategy of the memory block, including access permission setting, cache strategy adjustment and data migration flag, etc. The generation of the identifier is based on the data access entropy and conflict probability data, aiming to reduce memory access conflicts and improve data locality.

[0038] The instruction set construction unit integrates the scheduling priority vector, thread preemption strategy sequence and memory lock identification to generate a dynamic scheduling instruction set executable by the system. The instruction set adopts a hierarchical structure design and includes three categories of device control instructions, thread management instructions and memory operation instructions. The device control instructions are responsible for the scheduling priority setting and calculation task allocation of the GPU kernel, and the instruction format includes target device identification, priority value and execution time, etc. The thread management instructions handle the suspension, recovery and priority adjustment operations of the CPU thread, and the instruction structure includes thread identification, operation type and parameter setting. The memory operation instructions implement the locking, unlocking and data migration functions of the memory block, and the instruction content includes memory address range, operation type and access strategy. All instructions are provided with a timestamp and execution condition judgment logic, supporting two modes of conditional triggering and timing execution. The instruction set is delivered to the hardware layer and the operating system kernel through a system call interface and is executed by the corresponding device driver and system service.

[0039] The entire implementation process adopts an event-driven architecture, triggering the scheduling decision-making process when the resource state changes. The system maintains a decision context database to store the current resource state, historical decision records and execution result feedback. The decision algorithm adopts a combination of heuristic rules and machine learning methods to dynamically adjust the decision parameters according to the system running state. After the instruction set is generated, it is subjected to legality check and conflict detection to ensure the safety and effectiveness of the scheduling instructions. Abnormal situations during execution are captured in real time through a monitoring mechanism, triggering the re-execution or instruction adjustment of the decision-making process.

[0040] The output dynamic scheduling instruction set of the module is transmitted to the scheduling strategy execution module through the system bus to guide specific resource scheduling operations. At the same time, the generation log and execution result of the instruction set are recorded in the system performance database for subsequent optimization and improvement of the decision algorithm. The entire implementation process focuses on the balance between real-time performance and accuracy, controlling the calculation overhead on the premise of ensuring the decision-making quality to ensure timely response of the system scheduling.

[0041] Embodiment 4: Involving the collaborative operation of the scheduling policy execution module and the resource reallocation module, the system receives a dynamic scheduling instruction set from the scheduling decision engine module, realizes the real-time adjustment of computing resources by analyzing and executing these instructions, and monitors system performance during execution, triggering necessary resource reallocation operations. The scheduling policy execution module first parses the GPU kernel scheduling priority vector in the dynamic scheduling instruction set. Each element in the vector corresponds to a unique identifier of a GPU kernel and the corresponding priority value. The system accesses the kernel's command queue through the GPU driver interface and reorders the computing instructions according to the priority value. The instructions of high-priority kernels are promoted to the front of the queue to obtain execution resources preferentially. The instructions of low-priority kernels are delayed for processing and can be temporarily suspended for execution if necessary. The instruction reordering operation maintains the dependency relationship of the computing task and avoids data inconsistency problems.

[0042] For the execution of the thread preemption policy sequence, the module operates through the scheduler interface of the operating system kernel. Each entry in the sequence contains the target thread identifier, the operation type, and the execution parameters. The suspend operation is implemented by modifying the thread state to blocked, suspending the execution of the thread but not releasing the resources it occupies. The resume operation resets the thread state to ready, allowing it to re-enter the scheduling queue. The priority adjustment operation modifies the scheduling priority value of the thread, affecting the decision logic of the operating system scheduler. All operations are recorded in the audit log for subsequent analysis and troubleshooting.

[0043] The processing of memory lock identification is completed by the memory management unit, which specifies the address range of the target memory block and the access control policy. The module sets the access permissions of the memory page through system calls, limiting the access operations of specific processes or threads. For memory regions that need to be isolated, the module modifies the page table entry to set the read-only or inaccessible flags. When a violation access attempt is detected, the memory controller generates a protection exception, and the exception handling program records the event and executes the predefined processing flow.

[0044] The performance monitoring unit of the resource reallocation module continuously collects key performance indicators of the system, including the instruction queue depth of the GPU kernel, the length of the wait-wakeup queue of the CPU thread, and the access request accumulation amount of the memory block. The instruction queue depth is read through the state register of the GPU command processor, reflecting the backlog of unexecuted instructions. The wait-wakeup queue length is obtained through the statistical interface of the thread scheduler, indicating the number of threads in the waiting state. The access request accumulation amount is monitored through the performance counter of the memory controller, showing the number of outstanding memory operation requests, as shown in Table 1.

[0045] Table 1: Performance monitoring indicators and threshold settings.

[0046]

[0047] The migration decision unit triggers resource reallocation operations according to the monitoring data and threshold comparison results. When the instruction queue depth continuously exceeds the full load threshold, the unit initiates GPU computation task migration. The migration target selection is based on the current load status of each GPU core, with a preference for cores with more idle resources. When the length of the wait-to-wake-up queue breaks through the time delay threshold, the unit reallocates CPU thread affinity by modifying the binding relationship between threads and processor cores to optimize scheduling efficiency. When the access request accumulation reaches the conflict threshold, the unit initiates memory block copy operations to copy frequently accessed data to free storage areas, reducing access conflicts.

[0048] The load migration unit performs specific resource reallocation operations, and GPU computation task migration performs data transmission through the PCIe bus, using the DMA engine to achieve fast memory-to-memory copying. The migration process maintains the execution state of the computation task, including register content and command queue state. CPU thread affinity adjustment is achieved through the operating system scheduler interface, modifying the mapping relationship between threads and processor cores. Memory block copy operations are performed by the memory controller, and after the copy is completed, the page table mapping and cache coherence information are updated. All migration operations guarantee data consistency and integrity, avoiding data errors or loss during the migration process.

[0049] The entire implementation process adopts an asynchronous processing mechanism, and monitoring and migration operations are executed in parallel. The system maintains a migration state database to record ongoing migration operations and resource state changes. The migration decision considers overall system load balancing to avoid new resource competition caused by migration operations. Abnormal situations during execution are handled through a retry mechanism and rollback strategy to ensure system reliability. The migration effect is evaluated through continuous performance monitoring, and dynamic adjustments are made as necessary.

[0050] Example 5: see Figure 5, involving the cooperative workflow of the elastic scaling module and the resource supply module. The system, aiming at the computing characteristics of generative AI tasks, predicts resource demand by analyzing task execution phase characteristics and dynamically adjusts computing resource supply, realizing elastic allocation and recovery of resources. The task analysis unit of the elastic scaling module continuously monitors the execution process of generative AI tasks. The unit obtains task execution logs and performance profiling data through the task profiling interface, and identifies different computing phases of the task. The characteristics of the matrix multiplication phase are a large number of parallel computing operations and high floating-point operation intensity, which are identified by monitoring the floating-point instruction execution rate of the GPU and the matrix operation function call frequency. The computing characteristics of the back propagation phase are frequent gradient calculation and weight update operations, which are judged by tracking the execution time of the back propagation operator and the memory access mode. The gradient update phase involves parameter adjustment and synchronization operations, which are distinguished by analyzing the execution process of the optimizer function and communication overhead. The time proportion of each phase is calculated by calculating the percentage of the execution time of each phase in the total task time, and the phase division adopts a pattern recognition algorithm based on computing characteristics.

[0051] The demand prediction unit predicts resource demand based on the current phase resource consumption rate and the next phase computing complexity. The current phase resource consumption rate is obtained by real-time monitoring of the GPU kernel computing utilization, CPU thread processing throughput, and memory bandwidth usage. These indicators are collected in time series form and the trend is calculated. The next phase computing complexity is evaluated based on task characteristic parameters, including matrix operation dimension size, iteration number, and data type accuracy. The prediction of GPU kernel demand increment considers the current GPU utilization level and the floating-point operation amount required in the next phase, and estimates the number of kernels required to meet the performance target through a linear regression model. The calculation of the CPU thread expansion factor is based on the current thread load rate and the task parallelism requirement of the next phase, and a load balancing algorithm is used to determine the adjustment proportion of the thread pool size. The derivation of the memory block expansion coefficient is based on the current memory usage and the data storage demand of the next phase, and the required storage capacity is predicted by analyzing the memory occupation trend. The prediction process uses a sliding window mechanism, which updates the prediction parameters regularly to adapt to the dynamic changes in the task execution process.

[0052] The sequence generation unit encodes the predicted resource demand parameters into a structured resource scaling sequence. The encoding format adopts a standardized data serialization protocol, containing a resource type identifier, an operation type field, and a quantity value field. The GPU kernel demand increment is encoded as an instruction block containing a device type, an operation instruction, and a quantity change, with the operation instruction including instance creation, destruction, or configuration adjustment. The CPU thread expansion factor is converted into a thread pool adjustment instruction, specifying the increase or decrease magnitude of thread quantity and the adjustment time constraint. The memory block scaling coefficient is encoded as a memory management instruction, containing a capacity adjustment value, an allocation strategy, and a priority setting. Each instruction item in the sequence is accompanied by a timestamp and an execution condition description, supporting both immediate execution and timed triggering modes. The entire sequence is verified by a checksum to ensure data integrity and consistency.

[0053] The resource supply module monitors the resource scaling demand sequence in real time and performs corresponding resource adjustment operations. For GPU kernel computing instance management, the module dynamically starts and stops computing instances through the GPU driver programming interface. The instance start process includes allocating video memory resources, initializing the execution context, and registering device functions. The instance stop process involves execution context cleanup, video memory resource release, and device state reset. The adjustment of the CPU thread pool size is achieved through the thread pool management interface, and the thread creation process includes allocating stack space, setting scheduling attributes, and registering callback functions. The thread destruction process includes waiting for execution to complete, releasing resources, and unregistering registration information. The pre-allocation strategy adjustment of the memory block is completed through the memory management interface, including reserving physical memory pages, setting mapping relationships, and configuring cache strategies. All resource operations are logged in detail, including operation time, resource identification, and execution results.

[0054] The entire implementation process adopts an event-driven architecture, and resource demand prediction and resource supply adjustment are executed asynchronously. The system maintains a resource state database, recording the current resource allocation and historical usage records. Resource adjustment operations follow a gradual principle to avoid frequent and drastic resource fluctuations. The resource dependency relationship is considered during operation execution to ensure the rationality of the adjustment operation sequence. The exception handling mechanism monitors the execution results of resource operations and retries or rolls back failed operations.

[0055] It should be noted that, in this text, relationship terms such as first and second are only used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device.

[0056] While embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A generative AI heterogeneous computing resource dynamic scheduling system for PC terminals, characterized by: include: The resource status acquisition module is used to capture the operating status parameters of the GPU core, CPU thread and memory block in the PC terminal in real time, perform heterogeneous data format normalization processing on the operating status parameters, and generate resource status feature tensors; A scheduling graph generation module is used to parse the computing node connection topology of the resource state feature tensor, extract the data transmission bandwidth between GPU cores, the dependency chain between CPU threads, and the mapping relationship between memory block shared areas, and construct a multi-dimensional scheduling graph that includes computing node connection weights and cross-device interaction delays; The resource fluctuation entropy analysis module is used to separate the real-time load feature vector of each computing unit from the resource state feature tensor, calculate the GPU core memory usage entropy, the CPU thread instruction flow entropy and the memory block data access entropy based on the corresponding real-time load feature vector, and summarize them to generate a heterogeneous resource entropy value matrix; The scheduling decision engine module is used to perform joint analysis based on the multi-dimensional scheduling map and the heterogeneous resource entropy matrix, identify the resource competition characteristics of the bottleneck node of the computing task, and generate a dynamic scheduling instruction set including the GPU kernel scheduling priority vector, the CPU thread preemption strategy sequence and the memory block locking identifier.

2. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 1, characterized in that: The resource status acquisition module specifically includes: Heterogeneous parameter capture unit, which synchronously collects the shader core utilization of the GPU core, the instruction cache hit rate of the CPU thread, and the page fault rate of the memory block; a data fusion unit, converting the shader core utilization, instruction cache hit rate, and page fault rate into a time series data stream with a unified timestamp; The tensor generation unit extracts the spatial correlation features of the time series data stream through a three-dimensional convolution kernel, and outputs a resource status feature tensor with a dimension of [device type × time window × performance index].

3. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 1, characterized in that: The scheduling graph generation module specifically includes: a topology parsing unit, which identifies cross-device synchronization signals between the GPU kernel and the CPU thread in the resource state feature tensor, and marks direct memory access channels from memory blocks to the GPU kernel; The weight calculation unit quantifies the fluctuation coefficient of the data transmission bandwidth between GPU cores, the waiting delay of the dependency chain between CPU threads, and the conflict probability of the mapping relationship of the shared memory block area; The graph construction unit uses the cross-device interaction delay as the connection edge weight and the computing node as the vertex to construct a multi-dimensional scheduling graph including vertex attributes and edge weights.

4. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 1, characterized in that: The resource fluctuation entropy analysis module specifically includes: a feature separation unit, which extracts a video memory read and write throughput sequence of a GPU core, an instruction execution cycle sequence of a CPU thread, and a cache line contention count sequence of a memory block from the resource state feature tensor; an entropy value calculation unit, which calculates video memory occupancy entropy based on the video memory read and write throughput sequence, calculates instruction pipeline entropy based on the instruction execution cycle sequence, and calculates data access entropy based on the cache line contention count sequence; The matrix generating unit organizes the video memory occupancy entropy, instruction pipeline entropy and data access entropy into a heterogeneous resource entropy value matrix according to device type.

5. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 1, characterized in that: The scheduling decision engine module specifically includes: A bottleneck identification unit detects computing nodes whose connection weights exceed a preset threshold in the multidimensional scheduling graph, and locates resource competition characteristics based on the mutation gradient of the corresponding entropy value in the heterogeneous resource entropy matrix; A policy generation unit generates a scheduling priority vector for GPU cores with resource contention characteristics, a thread preemption policy sequence for associated CPU threads, and a memory lock identifier for conflicting memory blocks; The instruction set construction unit integrates the scheduling priority vector, the thread preemption strategy sequence and the memory lock flag to generate a dynamic scheduling instruction set.

6. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 5, characterized in that: Also includes: a scheduling policy execution module, configured to parse the GPU kernel scheduling priority vector of the dynamic scheduling instruction set, perform a computing instruction reordering operation on the target GPU kernel, and suspend a specified CPU thread according to the thread preemption policy sequence; Isolating conflicting memory block access requests according to the memory lock identifier; The resource reallocation module is used to monitor the GPU core instruction throughput, CPU thread wake-up latency, and memory block access conflict rate after executing the scheduling strategy. When the instruction throughput is lower than the performance threshold or the wake-up latency exceeds the tolerance window, the computing resource load migration operation is triggered.

7. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 6, characterized in that: The resource reallocation module specifically includes: Performance monitoring unit, which collects the instruction queue depth of the GPU core, the length of the CPU thread's waiting wakeup queue, and the amount of access request accumulation of the memory block; A migration decision unit is configured to migrate GPU computing tasks to idle cores when the instruction queue depth continuously exceeds a full load threshold; reallocate CPU thread affinity when the waiting wake-up queue length exceeds a latency threshold; and copy memory blocks to an idle storage area when the access request accumulation reaches a conflict threshold. The load migration unit performs GPU computing task migration operations based on the PCIe bus, adjusts CPU thread affinity through the operating system scheduler, and uses the memory controller to complete memory block copy operations.

8. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 1, characterized in that: Also includes: The elastic scaling module is used to analyze the computing phase characteristics of generative AI tasks, predict the number of GPU cores, CPU thread scale, and memory block capacity required for the next phase, and generate a sequence of resource expansion and contraction requirements; The resource supply module is used to respond to the resource expansion and contraction demand sequence, dynamically start and stop the computing instance of the GPU kernel, allocate the CPU thread pool size on demand, and adjust the pre-allocation strategy of the memory block.

9. The generative AI heterogeneous computing resource dynamic scheduling system for PC terminals according to claim 8, characterized in that: The elastic expansion module specifically includes: The task parsing unit identifies the time consumption ratio of the matrix multiplication phase, backpropagation phase, and gradient update phase in the generative AI task; The demand forecasting unit outputs the GPU core demand increment, CPU thread expansion factor, and memory block expansion coefficient based on the resource consumption rate of the current stage and the computational complexity of the next stage; The sequence generation unit encodes the GPU core requirement increment, the CPU thread expansion factor, and the memory block expansion coefficient into a resource expansion and contraction requirement sequence.

10. A generative AI heterogeneous computing resource dynamic scheduling method for PC terminals, characterized in that: Includes all modules and method flows of the generative AI heterogeneous computing resource dynamic scheduling system for PC terminals as described in any one of claims 1 to 9.

Images