Self-adaptive scheduling compensation system and method for GPU (Graphics Processing Unit) architecture

Abstract

The invention discloses a GPU (Graphics Processing Unit) architecture-oriented adaptive scheduling compensation system and method, and relates to the technical field of crossing of computer hardware and artificial intelligence. Comprising a GPU-operator dynamic matching module, a reinforcement learning scheduling center, a prediction-feedback compensation module and an adaptive optimization knowledge base, and a'perception-decision-execution-feedback-optimization 'closed-loop collaborative mechanism is formed through real-time data interaction; according to the method, features are extracted through static analysis and dynamic tracking fusion, a dynamic matching matrix is constructed through weighted cosine similarity, and deep adaptation of operators and GPU hardware is achieved; a deviation and differentiation compensation rule is predicted through an LSTM model, the performance loss is reduced, and the precision is guaranteed; through transfer learning and classification index multiplexing historical optimization experience, the cross-architecture adaptation period is shortened, the problems of low adaptation accuracy, scheduling staticization, compensation lag and low cross-architecture adaptation efficiency in the heterogeneous GPU environment are effectively solved, and the method is suitable for efficient and stable deployment of the AI model on the multi-architecture GPU.

Description

Technical Field

[0001] This invention belongs to the technical field of computer hardware and artificial intelligence, specifically an adaptive scheduling compensation system and method for GPU architecture. Background Technology

[0002] With the rapid development of artificial intelligence technology, the computing power demand of deep learning models is growing exponentially, and graphics processing units (GPUs) have become the core hardware supporting AI model training and inference due to their high parallel computing capabilities.

[0003] However, GPU architecture exhibits significant heterogeneity in practical applications: on the one hand, different manufacturers have different GPU hardware designs, such as the number of computing cores, memory bandwidth, and instruction set architecture; on the other hand, different generations of products from the same manufacturer also have significant differences in computing power characteristics and storage tiers. This heterogeneity leads to huge differences in the execution efficiency, resource consumption, and robustness of AI model operators on different GPUs, posing challenges to cross-architecture deployment and optimization.

[0004] Existing GPU scheduling and optimization schemes have the following limitations: 1. Lack of dynamic perception capability in hardware adaptation: Traditional solutions are mostly based on the static parameters of the GPU, such as peak computing power and memory capacity, to adapt operators, ignoring the dynamic characteristics of the GPU under different loads, such as computing power fluctuations and changes in memory access latency. This results in low matching degree between operators and hardware and low resource utilization.

[0005] 2. Static scheduling strategies lack adaptability: Existing scheduling strategies mostly rely on manually preset rules, such as fixed priorities and equal resource allocation. They cannot be dynamically adjusted according to the key features of operators and the real-time status of the GPU, which easily leads to the imbalance problem of "insufficient resources for large operators and idle resources for small operators".

[0006] 3. Lagging compensation mechanism and limited robustness: When GPU execution deviates, existing solutions mostly adopt passive correction, lacking the ability to predict potential deviations, resulting in significant performance loss, especially in meeting the needs of high-precision scenarios.

[0007] 4. Low efficiency in cross-architecture adaptation: For new GPU architectures, existing solutions need to redesign scheduling rules and compensation strategies, rely on expert experience for manual optimization, have long adaptation cycles and high costs, and are difficult to cope with the current situation of rapid iteration of GPU architecture.

[0008] Therefore, how to achieve deep matching between the dynamic features of GPU hardware and the key features of operators, and build an adaptive system with dynamic scheduling, prediction compensation and rapid cross-architecture capabilities, has become a key issue in improving the execution efficiency and robustness of AI models in heterogeneous GPU environments. Summary of the Invention

[0009] To overcome the shortcomings of existing technologies and solve the aforementioned technical problems, this invention proposes an adaptive scheduling compensation system and method for GPU architecture.

[0010] The technical solution adopted by this invention to solve its technical problem is to propose an adaptive scheduling compensation system for GPU architecture, characterized by comprising: The GPU-operator dynamic matching module is used to collect the hardware dynamic features of the target GPU in real time, extract the key features of the operators to be executed in the AI ​​model, and construct a dynamic matching matrix based on the hardware dynamic features and the key features of the operators to achieve deep adaptation. The adaptation accuracy is improved by the comprehensive coverage and real-time updates of the feature dimensions. The reinforcement learning scheduling hub takes a dynamic matching matrix as input and combines real-time GPU resource status and historical compensation effect data to dynamically generate operator scheduling strategies. Through collaborative decision-making of multi-dimensional inputs, it achieves accurate matching of resource allocation with operator characteristics and hardware status. The prediction-feedback compensation module predicts potential execution deviations based on the LSTM model and receives real-time status feedback, triggering a "pre-compensation-dynamic correction" linkage mechanism to form compensation rules. Through the dual protection of prediction and feedback, the performance loss caused by deviations is reduced. An adaptive optimization knowledge base stores hardware dynamic features, key operator features, dynamic matching matrices, operator scheduling strategies, compensation rules, and historical compensation effect data. Through transfer learning, historical optimization experience is mapped to new GPU architectures to achieve strategy iteration and shorten the cross-architecture adaptation cycle. The GPU-operator dynamic matching module, reinforcement learning scheduling center, prediction-feedback compensation module, and adaptive optimization knowledge base form a closed-loop collaborative mechanism through real-time data interaction, and achieve continuous improvement in the system's adaptive capability through efficient linkage between modules.

[0011] Furthermore, the key features of the operator include computational complexity, data access pattern, accuracy requirements, parallelism potential, and inter-operator dependencies. The GPU-operator dynamic matching module uses a fusion approach of "static analysis + dynamic tracking" to extract key features of the operator. Static analysis of the operator's computation graph obtains static features such as computational complexity and dependencies, while dynamic tracking of the memory access trajectory during operator execution obtains dynamic features such as data access pattern and parallelism potential. The complementary fusion of static and dynamic features ensures the comprehensiveness of feature extraction, thereby improving the quantization accuracy of the dynamic matching matrix.

[0012] Furthermore, the dynamic matching matrix is ​​generated by calculating the weighted cosine similarity between the key features of the operator and the dynamic features of the hardware. The weights are dynamically adjusted as the GPU load fluctuates. Through the adaptive adjustment of the weights, the matching matrix can respond to changes in the GPU's operating state in real time, ensuring the dynamic and accurate quantification of the operator's and hardware's compatibility.

[0013] Furthermore, the operator scheduling strategy includes a collaborative optimization scheme for operator execution priority, resource allocation ratio, and batch processing granularity, which is matched with the key features of the operators. The reinforcement learning scheduling center prioritizes the allocation of multiple computing core resources to operators with high parallelism potential. The adjustment range of batch processing granularity is positively correlated with the computational complexity. Through the resource allocation strategy guided by operator characteristics, resource idleness and insufficiency are avoided, thereby improving the utilization rate of GPU resources.

[0014] Furthermore, in the "pre-compensation-dynamic correction" linkage mechanism, the pre-compensation stage predicts the memory usage-related deviations based on the LSTM model and pre-allocates spare memory to avoid predictable deviations from the source; the dynamic correction stage selects a combination compensation scheme based on the deviation level and key features of the operator to form compensation rules, and ensures the pertinence and effectiveness of the compensation operation through hierarchical compensation and operator characteristic adaptation.

[0015] Furthermore, the compensation rules are as follows: for operators with high precision requirements, resource reallocation is prioritized to avoid precision loss; for operators with high computational complexity, subtasks are split in advance during the pre-compensation stage to avoid execution timeouts; for operators with low precision requirements, a "precision fine-tuning + resource preservation" strategy is adopted to balance execution efficiency and precision requirements, and the operation requirements of operators with different characteristics are met through differentiated compensation rules.

[0016] Furthermore, the historical optimization experience refers to the historical operator scheduling strategies, compensation rules, and corresponding execution effect data stored in the adaptive optimization knowledge base; transfer learning adopts a feature mapping network to perform nonlinear mapping between the source GPU architecture features and the target GPU features, and realizes efficient reuse of optimization experience through cross-architecture transfer at the feature level, reducing the manual cost of adapting to new architectures.

[0017] An adaptive scheduling compensation method for GPU architecture includes the following steps: Step S1: Hardware-operator dynamic matching: The GPU-operator dynamic matching module collects dynamic features of GPU hardware, extracts key features of operators, constructs a dynamic matching matrix and calculates the fitness score. Through feature collection and matching quantification, it provides accurate basis for subsequent scheduling decisions. Step S2: Reinforcement Learning Collaborative Scheduling: The reinforcement learning scheduling hub generates operator scheduling strategies based on fitness scores, real-time resource status, and historical compensation effect data. Through multi-source data-driven scheduling decisions, the adaptability of the strategies to the actual scenario is improved. Step S3: Prediction-Feedback Linkage Compensation: The prediction-feedback compensation module predicts potential deviations and performs pre-compensation. Combined with real-time status feedback, it triggers a combined compensation scheme to form compensation rules. By combining prediction prevention with real-time correction, operational risks are reduced. Step S4: Cross-architecture knowledge iteration: The adaptive optimization knowledge base stores the above data and maps historical optimization experience to the new GPU architecture through transfer learning to realize policy iteration. Through knowledge reuse and iterative optimization, the efficiency of cross-architecture deployment is improved. By forming a closed-loop linkage through steps S1 to S4, the AI ​​model can achieve efficient and stable operation in a heterogeneous GPU environment through collaborative optimization of the entire process.

[0018] Furthermore, in step S1, the key features of the operator include computational complexity, data access pattern, accuracy requirements, parallelism potential, and inter-operator dependencies. The parallelism potential is quantified by the number of separable dimensions and the independence of subtasks. The quantitative indicators provide a clear basis for the formulation of parallel scheduling strategies and improve the operability of the scheduling logic.

[0019] Furthermore, in step S3, the switching threshold of the combined compensation scheme is calibrated in real time based on the adaptability of the dynamic matching matrix to ensure the timeliness and rationality of the compensation strategy switching; in step S4, the adaptive optimization knowledge base establishes a classification index for the key features of the operator according to the computational complexity and parallelism potential, and prioritizes matching the historical optimization experience of the same type of operator during transfer learning, thereby improving the matching efficiency of historical experience through the classification index and further shortening the adaptation cycle of the new architecture.

[0020] The beneficial effects of this invention are as follows: 1. The adaptive scheduling compensation system and method for GPU architecture described in this invention extracts features by fusing static analysis and dynamic tracking in the GPU-operator dynamic matching module, calculates the dynamic matching matrix using weighted cosine similarity, and adaptively adjusts the weights according to the load. This achieves deep adaptation between the key features of the operators and the dynamic features of the GPU hardware, thereby improving resource utilization and operator execution adaptability in heterogeneous GPU environments. 2. The adaptive scheduling compensation system and method for GPU architecture described in this invention achieves early avoidance of potential deviations and accurate correction of sudden deviations through the LSTM model deviation prediction, pre-compensation and dynamic correction linkage mechanism of the prediction-feedback compensation module and the operator characteristic-oriented differentiated compensation rules, thereby reducing the performance loss caused by execution deviations and ensuring the high-precision and stable operation of AI models. 3. The adaptive scheduling compensation system and method for GPU architecture described in this invention achieves efficient reuse of historical optimization experience across GPU architectures by adaptively optimizing the classification index storage of the knowledge base and the transfer learning feature mapping network, thereby shortening the adaptation cycle of new architectures and reducing the manual cost of heterogeneous GPU deployment optimization. Attached Figure Description

[0021] The invention will now be further described with reference to the accompanying drawings.

[0022] Figure 1 This is a system block diagram of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0023] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, this invention provides an adaptive scheduling compensation system for GPU architecture, comprising: GPU-operator dynamic matching module: The core functions are hardware dynamic feature acquisition and operator key feature extraction, and a dynamic matching matrix is ​​constructed based on the two types of features to achieve deep adaptation between hardware and operators; Among them, hardware dynamic features are real-time characteristic data of the GPU during actual operation, rather than static parameters; key operator features include computational complexity, data access patterns, precision requirements, parallelism potential, and inter-operator dependencies. The extraction process adopts a fusion approach of "static analysis + dynamic tracking": static features such as computational complexity and inter-operator dependencies are obtained by statically analyzing the computation graph of the operator, while dynamic features such as data access patterns and parallelism potential are obtained by dynamically tracking the memory access trajectory during operator execution, ensuring the comprehensiveness and real-time performance of feature extraction; the dynamic matching matrix is ​​generated by calculating the weighted cosine similarity between the key operator features and the hardware dynamic features, and its weights can be dynamically adjusted according to GPU load fluctuations to accurately quantify the degree of adaptation between the operator and the current GPU hardware state.

[0025] The reinforcement learning scheduling hub uses the dynamic matching matrix output by the GPU-operator dynamic matching module as its core input. It also combines real-time GPU resource status (such as core utilization and remaining memory) with historical compensation effect data stored in the adaptive optimization knowledge base to dynamically generate operator scheduling strategies. These strategies include collaborative optimization schemes for operator execution priority, resource allocation ratios, and batch processing granularity, ensuring adaptability to key operator characteristics: for operators with high parallelism potential, priority is given to allocating more core resources; the adjustment range of batch processing granularity is positively correlated with the operator's computational complexity, ensuring that resource allocation matches operator characteristics and hardware status, avoiding resource idleness or insufficiency.

[0026] The prediction-feedback compensation module integrates prediction and feedback functions. Based on the LSTM model, it predicts potential deviations (such as memory usage-related deviations) during GPU execution and triggers a "pre-compensation-dynamic correction" linkage mechanism to form compensation rules. In the pre-compensation stage, it pre-allocates resources such as spare memory based on the prediction results to avoid predictable deviations at the source. In the dynamic correction stage, it receives real-time feedback on the GPU execution status and selects a combined compensation scheme based on the deviation level (such as slight latency, precision fluctuation, severe overflow, etc.) and key characteristics of the operator (such as precision requirements and computational complexity). Specifically, it prioritizes resource reallocation strategies for operators with high precision requirements to avoid precision loss; it pre-divides subtasks for operators with high computational complexity in the pre-compensation stage to avoid execution timeouts; and it adopts a "precision fine-tuning + resource preservation" strategy for operators with low precision requirements to balance efficiency and precision.

[0027] Adaptive Optimization Knowledge Base: This base stores the hardware dynamic features, key operator features, and dynamic matching matrix output by the GPU-operator dynamic matching module; the operator scheduling strategy generated by the reinforcement learning scheduling center; the compensation rules formed by the prediction-feedback compensation module; and historical compensation effect data generated during GPU execution. Historical optimization experience is defined as the historical operator scheduling strategies, compensation rules, and corresponding execution effect data stored in the knowledge base. Through transfer learning technology, the knowledge base uses a feature mapping network to non-linearly map the features of the source GPU architecture to the features of the target GPU architecture, rapidly transferring validated optimization experience to the new GPU architecture. This enables rapid cross-architecture iteration of operator scheduling strategies and compensation rules without requiring optimization from scratch for the new architecture.

[0028] The closed-loop collaborative mechanism, consisting of the GPU-operator dynamic matching module, reinforcement learning scheduling center, prediction-feedback compensation module, and adaptive optimization knowledge base, forms a closed-loop collaboration through real-time data interaction: the features and matching matrix output by the matching module provide decision-making basis for the scheduling center; the state feedback after the execution of the scheduling strategy drives the compensation module to trigger corresponding compensation operations; the compensation effect data and various core data are stored in the knowledge base; and the knowledge base provides optimization support for the matching module and scheduling center through transfer learning and iterative optimization, forming a complete closed loop of "perception-decision-execution-feedback-optimization" to ensure that the system continuously adapts to changes in GPU state and operator characteristics.

[0029] like Figure 2 As shown, based on the aforementioned adaptive scheduling compensation system for GPU architecture, this invention also provides an adaptive scheduling compensation method for GPU architecture, comprising the following steps: Step S1: Hardware-operator dynamic matching is performed by the GPU-operator dynamic matching module. It collects the dynamic hardware features of the GPU and extracts the key features of the operators to be executed in the AI ​​model (including computational complexity, data access pattern, accuracy requirements, parallelism potential, and inter-operator dependencies). The parallelism potential is quantified by the number of separable dimensions of the operator and the independence of subtasks. Based on the above two types of features, a dynamic matching matrix is ​​constructed, and the fit score is calculated to quantify the degree of matching between the operator and the hardware.

[0030] Step S2: Reinforcement learning collaborative scheduling is executed by the reinforcement learning scheduling center. It takes the fitness score output from step S1, the real-time resource status of the GPU, and the historical compensation effect data stored in the adaptive optimization knowledge base as input. It dynamically adjusts the weight of the reinforcement learning reward function based on the parallelism potential and computational complexity in the key features of the operator, and generates an operator scheduling strategy (including a collaborative optimization scheme for operator execution priority, resource allocation ratio, and batch processing granularity).

[0031] Step S3: Prediction-feedback linkage compensation is performed by the prediction-feedback compensation module. Based on the LSTM model, it predicts potential deviations during GPU execution and performs pre-compensation operations. The pre-compensation strategy is related to the operator parallelism potential. At the same time, it monitors the GPU execution status in real time and triggers a combined compensation scheme based on the feedback deviation level and the accuracy requirements and computational complexity in the key features of the operator, forming corresponding compensation rules. The switching threshold of the combined compensation scheme is calibrated in real time based on the fitness of the dynamic matching matrix.

[0032] Step S4: Cross-architecture knowledge iteration is performed by an adaptive optimization knowledge base, which stores the hardware dynamic features, key operator features, dynamic matching matrix, operator scheduling strategy, compensation rules, and historical compensation effect data generated in steps S1 to S3. Through transfer learning, the historical optimization experience in the knowledge base is mapped to the new GPU architecture, realizing rapid iteration of operator scheduling strategy and compensation rules. Among them, the knowledge base establishes a classification index for key operator features according to computational complexity and parallelism potential. During transfer learning, historical optimization experience of operators of the same category is prioritized to improve the adaptation efficiency under the new architecture.

[0033] Through the closed-loop linkage of steps S1 to S4, the AI ​​model achieves adaptive scheduling and compensation optimization under the heterogeneous GPU architecture.

[0034] To further clarify and explicitly explain the technical solution, implementation logic, and technical effects of the present invention, and to enable those skilled in the art to more intuitively understand and implement the present invention, detailed embodiments are provided below in conjunction with typical application scenarios and specific technical details. The selection of these embodiments does not limit the scope of protection of the present invention, but is only used to exemplarily verify the feasibility and superiority of the technical solution of the present invention.

[0035] This embodiment uses a heterogeneous GPU cluster (including NVIDIA A100, AMD MI250, and domestic Huawei Ascend 910 GPUs) as the deployment environment and Transformer model inference tasks as the application object. It elaborates on the specific implementation process of the adaptive scheduling compensation system and method for GPU architecture of the present invention. The core objective of this embodiment is to achieve adaptive scheduling, bias compensation, and rapid cross-architecture adaptation of the Transformer model on three types of heterogeneous GPUs, and to intuitively verify the technical effects of the system and method in improving resource utilization, reducing performance loss, and shortening the cross-architecture adaptation cycle.

[0036] A specific implementation of an adaptive scheduling compensation system for GPU architecture: Specific implementation of the GPU-operator dynamic matching module: 1. Hardware dynamic feature acquisition: This module collects dynamic hardware features through a unified adapter interface for GPUs from multiple vendors: for NVIDIA A100, it uses the NVML (NVIDIA Management Library) interface to collect real-time data on compute core utilization, memory access latency, L2 cache hit rate, and compute power fluctuation; for AMD MI250, it uses the ADL (AMD Display Library) interface to collect similar dynamic features; and for Huawei Ascend 910, it calls the AscendCL interface provided by Huawei to obtain hardware operating status data. The collection frequency is synchronized with the operator execution cycle to ensure that the feature data reflects the real-time operating status of the GPU. The collected dynamic hardware features include, but are not limited to: real-time compute core performance, memory read / write bandwidth, cache hit rate, compute power fluctuation coefficient related to core temperature, and memory access latency distribution characteristics.

[0037] 2. Key feature extraction of operators: The key features of the Transformer model's operators are extracted using a fusion approach of "static analysis + dynamic tracking": Static analysis: The computation graph of the Transformer model is parsed using the ONNX parser to extract the computational complexity (based on FLOPs quantization), inter-operator dependencies (based on node connection relationships in the computation graph), and accuracy requirements (FP32 / FP16 / BF16 accuracy identifiers defined in the model) of each operator (matrix multiplication, self-attention mechanism, activation function, etc.). Dynamic tracing: When the GPU executes the operator, the memory access trajectory (continuous access / random access identifier) ​​and parallelism potential (based on the number of dimensions that the operator can be split and the independence of subtasks) of the operator are tracked in real time through GPU performance counters.

[0038] The extracted key features of the operators and the dynamic features of the hardware are stored as feature vectors, each with a dimension of 128, to ensure the consistency of dimensions in subsequent matching calculations.

[0039] 3. Dynamic matching matrix construction: We use weighted cosine similarity to calculate the matching degree between the operator's key feature vector and the hardware's dynamic feature vector, and construct a dynamic matching matrix: Computational logic: Let the key feature vector of the operator be X=[x1,x2,...,x...]. 128 The hardware dynamic feature vector is Y=[y1,y2,...,y]. 128 Weighted cosine similarity ,in For the first Weights of dimensional features; Weight Adjustment: The initial weight allocation is preset (computation complexity weight 0.35, data access mode weight 0.25, precision requirement weight 0.2, parallelism potential weight 0.15, dependency weight 0.05). When the GPU load fluctuation exceeds the preset threshold (judged by the computing power fluctuation coefficient in the hardware dynamic features), the weights are automatically adjusted: when the load increases, the weights of data access mode and parallelism potential are increased, and the weights of dependency are decreased; when the load stabilizes, the initial weight allocation is restored.

[0040] The dynamic matching matrix is ​​an N×M matrix (N is the number of operators in the Transformer model, and M is the dimension of the current GPU's hardware dynamic features), and the matrix elements are the matching scores (0-1 range) between the corresponding operators and hardware features.

[0041] The specific implementation of the reinforcement learning scheduling center: 1. Input data preprocessing: The system receives the dynamic matching matrix (matching score), GPU real-time resource status (remaining computing cores, remaining video memory, and cache utilization) output by the GPU-operator dynamic matching module, and historical compensation effect data (execution latency and accuracy retention corresponding to historical operator scheduling strategies) from the adaptive optimization knowledge base. It then standardizes these three types of data into feature vectors in the 0-1 range, which are used as input to the reinforcement learning algorithm.

[0042] 2. Reinforcement Learning Algorithm Selection and Training: The Proximal Policy Optimization (PPO) algorithm is used as the core scheduling algorithm to construct a scheduling policy generation model: State space: average matching degree of dynamic matching matrix, GPU real-time resource utilization, operator queue length; Action space: operator execution priority (levels 1-5), number of computation cores allocated (1-32), batch processing granularity (16-1024); Reward function: R = α·resource utilization rate + β·execution efficiency - γ·precision loss rate, where α, β, and γ are dynamic weights that are adjusted according to the average matching degree of the dynamic matching matrix (α and β weights are increased when the matching degree is high, and γ weight is increased when the matching degree is low). Model training: Pre-training is performed based on historical data in the adaptive optimization knowledge base, and the model parameters are continuously fine-tuned through real-time feedback data during actual operation to ensure the adaptability of the scheduling strategy.

[0043] 3. Operator scheduling strategy generation and execution: Based on the output of the reinforcement learning model, generate scheduling strategies for each operator of the Transformer model: For matrix multiplication operators with high parallelism potential scores (≥0.8), allocate 16-32 computing cores, and adjust the batch processing granularity proportionally to the computational complexity (the batch processing granularity doubles for each level increase in complexity). For self-attention mechanism operators with strong dependencies (score ≥ 0.7), priority is given to allocating continuous computational core resources to ensure execution continuity; The scheduling policy is sent to the GPU for execution through the GPU driver interface (such as CUDA, HIP), and the execution status of the policy is monitored in real time.

[0044] Specific implementation of the prediction-feedback compensation module: 1. Construction of LSTM bias prediction model: A 3-layer LSTM network (input layer dimension 64, hidden layer dimension 128, output layer dimension 8) is constructed. Using GPU hardware dynamic feature time series data (feature sequences from the past 10 acquisition cycles) and operator scheduling policy parameters as input, it predicts the possible execution deviations (peak memory usage, execution latency, and accuracy fluctuations) that may occur in the next 3 cycles. Training data: Historical deviation data (cases of memory overflow, timeout, and accuracy loss) and corresponding feature sequences stored in the adaptive optimization knowledge base; Predicted output: Deviation type (memory-dependent, latency-dependent, precision-dependent), probability of deviation occurrence, and severity level of deviation (level 1-3).

[0045] 2. Implementation of the "pre-compensation - dynamic correction" linkage mechanism: Pre-compensation phase: When the LSTM model predicts that the peak memory usage exceeds 80% of the GPU memory capacity (the probability of deviation is ≥0.6), 10%-15% of spare memory is automatically pre-allocated (reserved through memory partitioning management technology), and the batch processing granularity of high computational complexity operators is adjusted to reduce the memory usage per transaction. Dynamic correction phase: Execution status feedback is collected in real time via GPU performance counters. If a deviation is detected (such as execution latency exceeding a preset threshold or accuracy fluctuation exceeding the allowable range), a combined compensation scheme is triggered based on the deviation level and key operator features. Level 1 Deviation (Slight Delay): Only adjusts the execution priority of the corresponding operator, increasing the resource preemption weight; Level 2 deviation (delay exceeds preset threshold / slight loss of accuracy): perform resource reallocation (add computing cores) for operators with high accuracy requirements (accuracy requirement ≥ 0.9), and perform accuracy fine-tuning (FP16 and BF16 switching) for operators with low accuracy requirements (≤ 0.6). Level 3 Deviation (Memory Overflow / Severe Precision Loss): Split high computational complexity operators into subtasks (split according to the number of separable dimensions), while fully scheduling spare memory resources, interrupting the current execution and rescheduling according to the new strategy.

[0046] 3. Compensation rule generation and storage: The compensation operations under different deviation scenarios are solidified into compensation rules, such as "high precision requirements + memory usage deviation → resource reallocation + spare memory scheduling" and "high computational complexity + latency deviation → subtask splitting + batch processing granularity adjustment". The compensation rules and execution effects are synchronously stored in the adaptive optimization knowledge base.

[0047] The specific implementation of the adaptive optimization knowledge base: 1. Data storage structure design: A distributed database (such as MongoDB) is used to store various core data, and a three-level index is built according to "GPU architecture - model type - operator type": First-level index: GPU architecture (NVIDIA A100 / AMD MI250 / Huawei Ascend 910); Second-level index: Model type (Transformer / CNN, etc.); Third-level index: Operator type (matrix multiplication / convolution / activation function, etc.); The stored data includes: hardware dynamic feature time series data, operator key feature data, dynamic matching matrix, operator scheduling strategy, compensation rules, and historical compensation effect data (execution latency, resource utilization, and accuracy retention).

[0048] 2. Transfer learning implementation: A feature mapping network with a residual connection structure is used to achieve optimization experience transfer across GPU architectures: Source architecture data preprocessing: Extract historical optimization experience data (scheduling strategies, compensation rules) under the NVIDIA A100 architecture and convert it into feature vectors; Feature mapping: The feature mapping network performs a non-linear mapping between the feature vectors of the source architecture and the hardware dynamic feature vectors of the target architecture (such as Huawei Ascend 910) to align the feature distribution; Target architecture strategy initialization: Based on the mapped feature vector, the optimization experience with the highest similarity is matched from the knowledge base as the initial scheduling strategy and compensation rule of Huawei Ascend 910 architecture, without the need for training from scratch; Iterative optimization: During the actual operation of the target architecture, the strategies and rules are fine-tuned based on real-time feedback data, and the knowledge base is updated.

[0049] The specific operation of the closed-loop coordination mechanism: The GPU-operator dynamic matching module collects hardware dynamic features and operator features every 10ms, updates the dynamic matching matrix, and sends it to the reinforcement learning scheduling center. The scheduling center generates a scheduling policy every 20ms and issues it for execution, while simultaneously receiving deviation feedback from the prediction-feedback compensation module. The compensation module monitors the execution status in real time, and after triggering a compensation operation, sends the compensation rules and effect data to the adaptive optimization knowledge base. The knowledge base updates its data every 50ms, and optimizes the feature extraction weights of the matching module and the reinforcement learning model parameters of the scheduling center through transfer learning, forming a real-time closed loop of "perception-decision-execution-feedback-optimization," iteratively improving the system's adaptive capability.

[0050] A specific implementation of an adaptive scheduling compensation method for GPU architecture: Step S1: Hardware-operator dynamic matching: 1. Start the Transformer model inference task, initialize the GPU-operator dynamic matching module and load the driver interface of the target GPU (such as AMD MI250); 2. Collect the hardware dynamic characteristics of the AMD MI250 (calculate core utilization, memory access latency, etc.) and generate a 128-dimensional feature vector; 3. Analyze the computation graph of the Transformer model, extract key features (computational complexity, accuracy requirements, etc.) of operators such as matrix multiplication and self-attention mechanism, and generate 128-dimensional operator feature vectors by dynamically tracking and supplementing data access patterns and parallelism potential features. 4. Use weighted cosine similarity to calculate the matching degree of the two types of feature vectors, construct an N×M dynamic matching matrix (N=number of Transformer model operators, M=128), and calculate the fitness score (0-1 interval) for each operator. 5. Send the dynamic matching matrix and fitness score to the reinforcement learning scheduling center and store them in the adaptive optimization knowledge base.

[0051] Step S2: Reinforcement Learning Cooperative Scheduling 1. Receive the adaptation score, real-time resource status of AMD MI250 (16 computing cores remaining, 20GB of video memory remaining), and historical compensation effect data from the knowledge base output in step S1; 2. Dynamically adjust the weights of the reinforcement learning reward function (if the current matching degree is high, α=0.4, β=0.4, γ=0.2); 3. Generate scheduling strategy using PPO algorithm: For matrix multiplication operator with a parallelism potential score of 0.9, allocate 16 computing cores with a batch processing granularity of 512; for self-attention operator with a dependency score of 0.8, allocate 8 consecutive computing cores with an execution priority level of 3. 4. The scheduling policy is sent to the AMDMI250 for execution via the HIP interface and stored in the knowledge base.

[0052] Step S3: Prediction-Feedback Linkage Compensation: 1. The LSTM model loads historical bias data to train weights, inputs current hardware dynamic feature time series data and scheduling strategy parameters, and predicts that the peak memory usage may be higher than normal in the next 3 cycles (probability 0.7). 2. Trigger pre-compensation operation, reserve 15% spare video memory (3GB), and adjust the batch processing granularity of matrix multiplication operator to 256; 3. Real-time monitoring of execution status; if the execution delay of the self-attention operator exceeds the preset threshold (secondary deviation) and the accuracy requirement score of the operator is 0.9; 4. Trigger the combined compensation scheme: Perform resource reallocation on this operator (add 4 computing cores) while maintaining the same accuracy; 5. Generate the compensation rule "High precision requirement + secondary delay deviation → resource reallocation", and send the rule and execution effect (latency reduction, precision maintenance) to the knowledge base.

[0053] Step S4: Cross-architecture knowledge iteration: 1. The knowledge base stores the hardware dynamic features, key operator features, dynamic matching matrix, scheduling strategy, compensation rules and effect data of steps S1-S3, and creates an index according to "AMDMI250-Transformer-Self-Attention Operator"; 2. When the task is switched to Huawei Ascend 910 GPU, the knowledge base starts transfer learning to extract the optimization experience feature vector under the AMD MI250 architecture; 3. The source architecture features are non-linearly mapped to the hardware features of Huawei Ascend 910 through a feature mapping network, and the scheduling strategy and compensation rule with the highest similarity are matched as the initial strategy. 4. Execute the initial strategy on Huawei Ascend 910, fine-tune it based on real-time feedback data (such as adjusting the number of computing cores allocated), and update the optimization experience in the knowledge base; 5. Complete cross-architecture adaptation, output the final optimized scheduling strategy and compensation rules, and support the efficient operation of the Transformer model on Huawei Ascend 910.

[0054] Through the actual deployment and testing of this embodiment, the technical effects of the present invention can be intuitively verified: 1. Improved resource utilization: Compared to traditional static scheduling schemes, this invention improves the GPU resource utilization of NVIDIA A100, AMD MI250, and Huawei Ascend 910 by 32%, 28%, and 35%, respectively, through dynamic matching and feature-oriented scheduling. 2. Reduced performance loss: The predictive-feedback linkage compensation mechanism keeps the performance loss caused by execution deviation on the three types of GPUs within 5%, with an accuracy retention rate of over 99.2%, which is significantly better than the traditional passive compensation scheme. 3. Improved cross-architecture adaptation efficiency: Through transfer learning and classification indexing, the adaptation cycle of Huawei Ascend 910 has been shortened from 72 hours in the traditional solution to 28 hours, improving adaptation efficiency by 61% and significantly reducing manual optimization costs.

[0055]

[0056] This embodiment is only an illustrative example. Those skilled in the art can adjust the specific implementation details of the module (such as feature acquisition frequency, algorithm parameters, compensation rule triggering conditions, etc.) according to actual application scenarios (such as different GPU architectures, different AI model types). However, all modifications and extensions based on the core technical solution of this invention fall within the protection scope of this invention.

[0057] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An adaptive scheduling compensation system for GPU architecture, characterized in that, include: The GPU-operator dynamic matching module is used to collect the hardware dynamic features of the target GPU in real time, extract the key features of the operators to be executed in the AI ​​model, and construct a dynamic matching matrix based on the hardware dynamic features and the key features of the operators to achieve deep adaptation. The reinforcement learning scheduling hub takes a dynamic matching matrix as input and combines real-time GPU resource status and historical compensation effect data to dynamically generate operator scheduling strategies. The prediction-feedback compensation module predicts potential execution deviations based on the LSTM model and receives real-time status feedback, triggering a "pre-compensation-dynamic correction" linkage mechanism to form compensation rules. An adaptive optimization knowledge base stores the hardware dynamic features, the operator key features, the dynamic matching matrix, the operator scheduling strategy, the compensation rules, and the historical compensation effect data. Through transfer learning, historical optimization experience is mapped to the new GPU architecture to achieve strategy iteration. The GPU-operator dynamic matching module, the reinforcement learning scheduling center, the prediction-feedback compensation module, and the adaptive optimization knowledge base form a closed-loop collaborative mechanism through data interaction.

2. The adaptive scheduling compensation system for GPU architecture according to claim 1, characterized in that, The key features of the operators include computational complexity, data access patterns, accuracy requirements, parallelism potential, and inter-operator dependencies. The GPU-operator dynamic matching module uses a fusion of static analysis and dynamic tracking to extract the key features of the operators. Static analysis obtains static features of computational complexity and dependencies, while dynamic tracking obtains dynamic features of data access patterns and parallelism potential.

3. The adaptive scheduling compensation system for GPU architecture according to claim 1, characterized in that, The dynamic matching matrix is ​​generated by calculating the weighted cosine similarity between the key features of the operator and the dynamic features of the hardware, and the weights are dynamically adjusted according to the fluctuation of GPU load.

4. The adaptive scheduling compensation system for GPU architecture according to claim 1, characterized in that, The operator scheduling strategy includes a collaborative optimization scheme for operator execution priority, resource allocation ratio, and batch processing granularity, and is matched with the key characteristics of the operators; The reinforcement learning scheduling center prioritizes allocating multiple computational core resources to operators with high parallelism potential, and the adjustment range of batch processing granularity is positively correlated with computational complexity.

5. The adaptive scheduling compensation system for GPU architecture according to claim 1, characterized in that, In the aforementioned "pre-compensation-dynamic correction" linkage mechanism, the pre-compensation stage predicts the memory usage-related deviations based on the LSTM model and pre-allocates spare memory; the dynamic correction stage selects a combination compensation scheme based on the deviation level and key features of the operator to form compensation rules.

6. The adaptive scheduling compensation system for GPU architecture according to claim 5, characterized in that, The compensation rules include prioritizing resource reallocation for high-precision demand operators; splitting high computational complexity operators into subtasks in advance during the pre-compensation phase; and employing a strategy of precision fine-tuning plus resource preservation for low-precision demand operators.

7. The adaptive scheduling compensation system for GPU architecture according to claim 1, characterized in that, The historical optimization experience refers to the historical operator scheduling strategies, compensation rules, and corresponding execution effect data stored in the adaptive optimization knowledge base; transfer learning uses a feature mapping network to perform non-linear mapping between the source GPU architecture features and the target GPU features.

8. An adaptive scheduling compensation method for GPU architecture, applicable to the adaptive scheduling compensation system for GPU architecture as described in any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Hardware-operator dynamic matching: The GPU-operator dynamic matching module collects dynamic features of GPU hardware, extracts key features of operators, constructs a dynamic matching matrix, and calculates the fitness score; Step S2: Reinforcement learning collaborative scheduling: The reinforcement learning scheduling center generates operator scheduling strategies based on fitness scores, real-time resource status, and historical compensation effect data; Step S3: Prediction-Feedback Linkage Compensation: The prediction-feedback compensation module predicts potential deviations and performs pre-compensation, and combines real-time status feedback to trigger a combined compensation scheme to form compensation rules; Step S4: Cross-architecture knowledge iteration: The adaptive optimization knowledge base stores the above data and maps historical optimization experience to the new GPU architecture through transfer learning to achieve policy iteration; a closed-loop linkage is formed through steps S1 to S4.

9. The adaptive scheduling compensation method for GPU architecture according to claim 8, characterized in that, In step S1, the key features of the operator include computational complexity, data access pattern, accuracy requirements, parallelism potential, and inter-operator dependencies; the parallelism potential is quantified by the number of separable dimensions and the independence of subtasks.

10. The adaptive scheduling compensation method for GPU architecture according to claim 8, characterized in that, In step S3, the switching threshold of the combined compensation scheme is calibrated in real time based on the fitness of the dynamic matching matrix; in step S4, the adaptive optimization knowledge base establishes a classification index for the key features of the operators according to computational complexity and parallelism potential, and prioritizes matching the historical optimization experience of operators of the same category during transfer learning.

Images