Heterogeneous computing resource collaborative acceleration method and system for AIGC

Abstract

The present application provides a method and system for collaborative acceleration of heterogeneous computing power resources for AIGC, which relates to the field of data processing technology. The method includes: inputting an activated AIGC model, determining the actual execution task subgraph, and performing model parsing of the task subgraph; layering the task subgraph using structural semantics and functional semantics; extracting operator features under dual-granularity constraints based on the operator sharding processing results and layered labels; performing computing power state collection of heterogeneous computing power resources, using multi-dimensional feature sets and computing power state feature sets as matching features, performing matching optimization between heterogeneous computing power resources and operators, configuring scheduling response plans; and performing collaborative acceleration management. This application solves the technical problem in the prior art of low operator scheduling efficiency during the execution of the AIGC model due to the strong dynamic nature of the computing power state of heterogeneous computing power resources. By realizing dynamic scheduling of heterogeneous computing power resources and model operators, the utilization rate of heterogeneous computing power resources is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and system for collaborative acceleration of heterogeneous computing resources for AIGC. Background Art

[0002] With the rapid development of AIGC, the demand for computing resources is also increasing. AIGC models typically involve a large number of computing tasks that need to be executed on different types of computing resources, such as CPUs, GPUs, and NPUs. Currently, task scheduling for AIGC models often uses a static resource allocation strategy. This strategy divides tasks into several stages based on the model's pre-set topology and selects fixed heterogeneous hardware resources for scheduling based on estimated computing power consumption. However, the paths actually activated and executed within the AIGC model under different generation requests are highly dynamic, and the specific computational content and order of each execution may vary. Due to the coarse granularity of operator scheduling, it is difficult to meet the performance requirements of heterogeneous resources. Furthermore, heterogeneous computing resources are inherently dynamic, and the load, memory usage, and operating status of different computing units fluctuate frequently over time. Existing scheduling methods typically dynamically reallocate resources based on task demand, resulting in some resources being idle while others experiencing performance bottlenecks. This leads to inefficient operator scheduling and uneven resource utilization.

[0003] In summary, the existing technology has a technical problem of low operator scheduling efficiency during the execution of the AIGC model due to the strong dynamics of the computing power status of heterogeneous computing resources. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for collaborative acceleration of heterogeneous computing resources for AIGC, so as to solve the technical problem in the prior art that the computing state of heterogeneous computing resources is highly dynamic, resulting in low operator scheduling efficiency during the execution of the AIGC model.

[0005] In view of the above problems, this application provides a method and system for collaborative acceleration of heterogeneous computing resources for AIGC.

[0006] In the first aspect, the present application provides a collaborative acceleration method for heterogeneous computing power resources for AIGC, which is implemented by a collaborative acceleration system for heterogeneous computing power resources for AIGC, wherein the collaborative acceleration method for heterogeneous computing power resources for AIGC includes: after the AIGC model is input and activated, the AIGC model is actually executed. Task subgraph judgment is performed on the AIGC model, model analysis of the task subgraph is performed, and operator graphs are extracted; the structural semantics and functional semantics of the AIGC model are used to layer task subgraphs based on operator graphs, and layer labels are established; operator sharding is performed under the layer labels, and operator features are extracted under dual-granularity constraints based on the operator sharding processing results and the layer labels to establish a multidimensional feature set; computing power status collection of heterogeneous computing power resources is performed, computing power status feature sets are extracted, and matching optimization between heterogeneous computing power resources and operators is performed using the multidimensional feature set and the computing power status feature set as matching features, and a scheduling response plan for heterogeneous computing power resources is configured according to the matching optimization results; and collaborative acceleration management is performed according to the scheduling response plan.

[0007] Optionally, optimization constraints are extracted based on the hierarchical labels and the operator shard processing results, and the optimization constraints include operator execution order dependency, same-layer shard and same-domain execution constraints, same-shard operator affinity constraints, and modality priority allocation constraints; after dividing the optimization constraints into mandatory constraints and soft constraints, the mandatory constraints and the soft constraints are used as additional features, and matching optimization is performed based on the matching features and the additional features.

[0008] Optionally, under the condition that the mandatory constraints are satisfied, an initial solution set is established; the initial solution set is scored for computing power adaptation to establish a first evaluation target; the initial solution set is scored for execution time to establish a second evaluation target; the initial solution set is analyzed for energy consumption to establish a third evaluation target; after normalizing the first evaluation target, the second evaluation target, and the third evaluation target, a fitness score of the solution is generated; the soft constraints are used to perform soft constraint penalty analysis on the initial solution set to establish a penalty decision direction; optimization iterative management is performed based on the fitness score and the penalty decision direction to complete matching optimization.

[0009] Optionally, a candidate solution is established for each solution in the initial solution set, wherein the candidate solutions include guided candidate solutions and random candidate solutions, wherein the guided candidate solutions are constructed based on the fitness score and the penalty decision direction, and the random candidate solutions are constructed based on the fitness score and a random function; a quantity threshold constraint for solution set expansion is established, and the quantity threshold constraint is used to perform threshold-triggered screening of candidate solutions; and the threshold-triggered screening result is used to complete the iterative update of the initial solution set.

[0010] Optionally, each solution in the initial solution set is used as the starting point of the path, and a path family is established according to the iterative optimization result; path exploration coverage analysis of the solution set is performed at preset periodic nodes to establish exploration constraints; the health index of the path is calculated using the path family to generate a health index calculation result; if the health index calculation result determines that the path family is degraded, the corresponding path family taboo operation is executed; and the optimization iterative management is completed according to the exploration constraints of the periodic nodes and the path family taboo operation.

[0011] Optionally, a context load feature is established based on the predicted generation length and input modality type of the task subgraph; a cross-layer context dependency label is extracted using the context load feature and the operator sharding processing result; the initial solution set is compensated and optimized based on the cross-layer context dependency label, and the solution set is updated using the compensated optimization result.

[0012] Optionally, the hardware resources and supporting modal information of the heterogeneous computing resources are obtained to establish static capability characteristics; the real-time computing power occupancy information of the heterogeneous computing resources is obtained, and dynamic load characteristics are established based on the real-time computing power occupancy information; and the computing power status feature set is extracted using the static capability characteristics and the dynamic load characteristics.

[0013] Optionally, a dynamic backtracking window of heterogeneous computing power resources is established; the dynamic backtracking window is used to perform stability analysis of the heterogeneous computing power resources and establish dynamic compensation features; and the dynamic compensation features are added as additional features to the computing power status feature set.

[0014] Optionally, a redundant detection window is configured, and when executing the scheduling response plan, the redundant detection window is used to monitor the actual response of heterogeneous computing resources; if the actual response monitoring meets the preset abnormal value, a redundant replacement instruction is triggered; and redundant compensation of heterogeneous computing resources is performed according to the redundant replacement instruction.

[0015] In the second aspect, the present application also provides a heterogeneous computing power resource collaborative acceleration system for AIGC, which is used to execute the heterogeneous computing power resource collaborative acceleration method for AIGC as described in the first aspect, wherein the heterogeneous computing power resource collaborative acceleration system for AIGC includes: a task subgraph determination module, which is used to determine the actual execution task subgraph of the AIGC model after the AIGC model is input and activated, perform model parsing of the task subgraph, and extract the operator graph; a task subgraph stratification module, which is used to use the structural semantics and functional semantics of the AIGC model to perform task subgraph stratification based on the operator graph, and build Establish hierarchical labels; an operator feature extraction module, which is used to perform operator sharding processing under hierarchical labels, perform operator feature extraction under dual-granularity constraints based on the operator sharding processing results and the hierarchical labels, and establish a multidimensional feature set; a matching optimization module, which is used to execute computing power status collection of heterogeneous computing power resources, extract computing power status feature sets, use the multidimensional feature sets and the computing power status feature sets as matching features, execute matching optimization between heterogeneous computing power resources and operators, and configure the scheduling response plan of heterogeneous computing power resources according to the matching optimization results; a collaborative acceleration management module, which is used to perform collaborative acceleration management according to the scheduling response plan.

[0016] One or more technical solutions provided in this application have at least the following beneficial effects:

[0017] After the AIGC model is activated, the AIGC model is evaluated for actual execution of task subgraphs, model parsing of the task subgraphs is performed, and operator graphs are extracted. The AIGC model's structural and functional semantics are used to stratify the task subgraphs based on the operator graphs, establishing hierarchical labels. Operator sharding is performed under the hierarchical labels, and operator features are extracted under dual-granularity constraints based on the operator sharding results and the hierarchical labels to establish a multidimensional feature set. The computing power status of heterogeneous computing resources is collected to extract a computing power status feature set. The multidimensional feature set and the computing power status feature set are used as matching features to optimize the matching of heterogeneous computing power resources with operators. A scheduling response plan for the heterogeneous computing power resources is configured based on the matching optimization results. Collaborative acceleration management is performed based on the scheduling response plan. In other words, by performing task subgraph evaluation and operator graph extraction, the AIGC model's execution path is parsed in real time. Based on the matching optimization of the multidimensional feature set and the computing power status feature set, dynamic scheduling of heterogeneous computing power resources and model operators is achieved, improving the utilization of heterogeneous computing power resources and thereby enhancing the execution efficiency of the AIGC model.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, which can be implemented in accordance with the contents of the description, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically listed below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in this application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and a person of ordinary skill in the art can obtain other drawings based on the provided drawings without creative work.

[0020] Figure 1 This is a flowchart of the AIGC-oriented heterogeneous computing resource collaborative acceleration method of this application.

[0021] Figure 2 This is a structural diagram of the heterogeneous computing resource collaborative acceleration system for AIGC in this application.

[0022] Explanation of the accompanying symbols: task subgraph determination module 11, task subgraph stratification module 12, operator feature extraction module 13, matching optimization module 14, collaborative acceleration management module 15. DETAILED DESCRIPTION

[0023] This application provides a method and system for collaborative acceleration of heterogeneous computing resources for AIGC, addressing the existing technical problem of low operator scheduling efficiency during the execution of the AIGC model due to the highly dynamic computing state of heterogeneous computing resources. By performing task subgraph determination and operator graph extraction, the execution path of the AIGC model is analyzed in real time. Based on the matching optimization of multi-dimensional feature sets and computing state feature sets, dynamic scheduling of heterogeneous computing resources and model operators is achieved, improving the utilization rate of heterogeneous computing resources and thus the execution efficiency of the AIGC model.

[0024] Below, the technical solutions in this application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, rather than all the embodiments of this application. It should be understood that this application is not limited to the example embodiments described herein. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. It should also be noted that, for the convenience of description, only the parts related to this application, rather than all of them, are shown in the accompanying drawings.

[0025] For example, see the attached Figure 1 The present application provides a method for collaborative acceleration of heterogeneous computing resources for AIGC, wherein the method is executed by a system for collaborative acceleration of heterogeneous computing resources for AIGC. The method specifically includes the following steps:

[0026] After the AIGC model is input and activated, the AIGC model is subjected to actual execution task subgraph determination, the model of the execution task subgraph is parsed, and the operator graph is extracted.

[0027] Specifically, when a user submits a generation request to the AIGC model, it is activated based on the input content, and the model enters the working state from the idle state. The AIGC model is typically composed of many layers and modules, forming a large computational graph. Not all layers and modules are actually executed after each input activation. The input generation request is analyzed to determine which specific layers and modules in the model will actually be called and calculated under this specific input, thereby identifying the subgraph that actually executes the task. Determining the subgraph that actually executes the task refers to identifying the portion of the structure actually used in the current inference or generation task within the entire model structure, forming a subgraph.

[0028] After determining the task subgraph to be executed, the task subgraph is model parsed, deeply analyzing its structure. This includes identifying the individual computing nodes (i.e., operators) in the subgraph, their connections (data flows), the input and output data types and dimensions of each operator, and the required computing resources (such as memory and computing power). The parsed operators and their connections are organized to form an operator graph. An operator graph is a graph structure consisting of the operators actually used in the current task and their connections. It is typically a directed acyclic graph, with each node representing an operator and each edge representing the data dependencies between operators.

[0029] By determining the task subgraph actually executed, invalid calculations are avoided, and the dynamic execution path of the AIGC model under different task requests is accurately identified and analyzed, avoiding full scheduling of the entire model structure, thereby reducing invalid resource calls and redundant calculations, and improving the information acquisition efficiency and resource mapping accuracy in the early stage of scheduling.

[0030] The structural semantics and functional semantics of the AIGC model are used to perform task subgraph stratification based on the operator graph and establish hierarchical labels.

[0031] Specifically, structural semantics refers to the connection relationships and topological structure between operators or modules in the AIGC model. It describes which operators the model consists of, how these operators are connected to each other (for example, how data flows from the output of one operator to the input of another), and their hierarchy or order in the computational graph. Structural relationships determine the execution order and dependencies of computational tasks. Functional semantics refers to the specific computational tasks performed by each operator or module in the AIGC model and is used to understand the role of each operator in the model.

[0032] Based on the extracted operator graph representing the currently executed task, task subgraphs are layered according to the structural and functional semantics of the AIGC model to identify the different hierarchical structures of the model. For example, consecutive convolution and pooling operators are typically used to gradually extract and reduce features and can be classified as a feature extraction layer. Based on the computational characteristics of the operators, for example, some operators may be primarily compute-intensive (such as large matrix multiplications) while others may be memory-intensive (such as processing large feature maps) or require specific hardware acceleration (such as certain types of tensor operations), the operator graph is divided into several logical layers. For each layer, a corresponding hierarchical label is established, such as input preprocessing layer, feature extraction layer, etc. The hierarchical labeling marks and categorizes the different hierarchical structures in the operator graph, dividing operators into multiple layers based on their function and structure. This labeling not only identifies the functional characteristics of the operator but also contains hardware adaptation information related to the operator type.

[0033] For example, let's assume the AIGC model uses a Transformer architecture, consisting of multiple self-attention layers, a feedforward network layer, and a final Softmax layer. Based on the text generation task input by the user, the subgraph actually executed involves the self-attention layers from layers 2 to 6 and the final Softmax output layer. Based on the model's connectivity, the self-attention layers from layers 2 to 6 sequentially process the input embedding vectors to generate intermediate representations, while the Softmax layer generates a probability distribution based on the final representations, determining the final output. Self-attention layers are typically compute-intensive, calculating the associations between each position in the input sequence and every other position; whereas the Softmax layer is memory-intensive, assigning probabilities to all elements in the entire sequence. When layering the task subgraph, the compute-intensive layers (self-attention layers) are labeled as layer 1 and assigned to GPU resources with higher performance; the memory-intensive layers (Softmax layers) are labeled as layer 2 and assigned to TPU resources with higher memory bandwidth.

[0034] By layering task subgraphs based on structural semantics and functional semantics and assigning hierarchical labels to each layer, we can gain a deep understanding of the computational characteristics of each operator, thereby fine-tuning resource allocation based on its computational intensity and hardware requirements, and avoiding resource bottlenecks during operator execution.

[0035] Operator sharding is performed under hierarchical labels, and operator features are extracted under dual-granularity constraints based on the operator sharding results and hierarchical labels to establish a multidimensional feature set.

[0036] Specifically, based on the already layered task subgraphs, operators within each layer are further refined. Specifically, within the already-divided layers, the operators that make up that layer are further divided or grouped, a process known as operator sharding. The set of operators within a logical layer is divided into smaller units that can be independently scheduled or allocated to different resources based on a strategy (such as operator type, computational load, memory requirements, and data dependencies). Operator sharding is performed under hierarchical labels, meaning the sharding strategy takes into account the functionality or characteristics represented by the layer's own labels. For example, in a parallel feature extraction layer (containing operators A, B, and C), sharding is performed based on the data dependencies or computational patterns between operators. Operators A and B are both convolution operators that receive the same batch of input image feature tensors and can execute in parallel. Operator C is an activation function (such as ReLU) that relies on the concatenation of the outputs of A and B and can only be initiated after A and B complete. Therefore, A and B are divided into one operator slice (slice 1), and C is divided into another operator slice (slice 2). The operator sharding processing result is obtained through the sharding operation, that is, each logical layer is divided into individual operator slices. Each operator slice contains one or more operators. These operator slices are logically easier to manage and schedule.

[0037] Operator feature extraction is performed under dual-granularity constraints based on the operator sharding results and hierarchical labels. This means that when extracting operator features, two levels of information constraints are considered simultaneously. The first is the global hierarchical semantic constraint provided by the hierarchical labels, or coarse-grained, meaning that when extracting operator features, the approximate functions or characteristics represented by the labels of the entire logical layer to which the operator belongs must be considered. The second is the fine-grained computational behavior characteristics provided by the operator sharding results, meaning that when extracting operator features, the internal structure of the specific operator slice to which the operator belongs and the specific relationships between operators must be considered. Dual-granularity constraints simultaneously utilize information at these two different granularities to constrain the operator feature extraction process, ensuring that the extracted features reflect both the macroscopic characteristics of the operator's layer and its microscopic characteristics and interactions within the operator slice to which it belongs.

[0038] All operators (or operator slices) are subjected to operator feature extraction under dual-granularity constraints, and all the obtained feature vectors are combined together to form a set containing multiple dimensions (each dimension represents a feature attribute), resulting in a multi-dimensional feature set that comprehensively describes the characteristics of all operator slices in the current task subgraph. For example, for operator slice 1 (including A and B), it is necessary to consider both the parallel feature extraction layer (first granularity constraint) and the specific operations of A and B within slice 1 (second granularity constraint). The extracted operator features include: operator type (convolution), computational effort (A and B combined are about 50 GFLOPs), memory usage (about 2GB), input tensor size 128MB, output tensor size 256MB, whether it is suitable for parallel operation (1) (yes), layer label (parallel feature extraction layer), slice number (1), data dependency (no dependency), scheduling priority (high), etc.; the feature representation of slice 1 is a vector [Conv, 50, 2.0, 128, 256, 1, parallel feature extraction layer, 1, 0, High]. For operator slice 2 (including C), its features are extracted, including: operator type (activation function ReLU), computational complexity (about 30GFLOPs), memory usage (about 1.5GB), input tensor size 256MB, output tensor size 256MB, suitability for parallelism 0 (no, dependent), layer label (parallel feature extraction layer), slice number (2), data dependency (dependent on slice 1), scheduling priority (medium), etc.; slice 2 is [ReLU, 30, 1.5, 256, 256, 0, parallel feature extraction layer, 2, 1, Mid].

[0039] By tightly integrating the two key operations of operator sharding and feature extraction, and employing a dual-granularity constraint mechanism, the extracted operator features incorporate both macro-level information and micro-level information within the operator slice, providing more comprehensive and accurate information. Compared to feature extraction based solely on a single operator or hierarchical labels, this multidimensional feature set under dual-granularity constraints can more accurately reflect the actual operational requirements of the operator slices, identifying which operator slices are compute-intensive, memory-intensive, suitable for parallelization, and have specific data dependencies.

[0040] Perform computing power status collection of heterogeneous computing power resources, extract computing power status feature set, use the multi-dimensional feature set and the computing power status feature set as matching features, perform matching optimization between heterogeneous computing power resources and operators, and configure the scheduling response plan of heterogeneous computing power resources according to the matching optimization results.

[0041] Furthermore, the present application also includes the following steps: extracting optimization constraints based on the hierarchical labels and the operator shard processing results, the optimization constraints including operator execution order dependency, same-layer shard and same-domain execution constraints, same-shard operator affinity constraints, and modality priority allocation constraints; after dividing the optimization constraints into mandatory constraints and soft constraints, the mandatory constraints and the soft constraints are used as additional features, and matching optimization is performed based on the matching features and the additional features.

[0042] Specifically, heterogeneous computing resources are different types of computing resources (such as CPUs, GPUs, TPUs, and FPGAs), with varying computing power, memory capacity, network bandwidth, and other characteristics, making them suitable for different types of operators. The computing power status of heterogeneous computing resources is collected in real time, including hardware configuration (such as the number of GPU cores and video memory size), current load (such as 76% GPU utilization and 8GB memory usage), and supported AIGC task types (such as text generation and multimodal tasks). This constitutes a computing power status feature set. The computing power status feature set is a structured representation of the collection results, typically including static capability features (such as memory capacity, number of cores, and supported operation types) and dynamic load features (such as GPU utilization, memory usage, and the number of currently queued tasks), as well as additional dynamic compensation features.

[0043] Multidimensional feature sets and computing power status feature sets are used as matching features, providing key information for matching operators (or operator slices) with heterogeneous computing resources and evaluating the compatibility between operators and computing resources. Based on the hierarchical labels and operator sharding processing results, optimization constraints are extracted: a set of rules or restrictions that must be followed when optimizing operator-resource matching. This ensures that the matching results not only consider efficiency but also meet model execution correctness and specific requirements. Optimization constraints include operator execution order dependency, same-layer shard and same-domain execution constraints, same-shard operator affinity constraints, and modality priority allocation constraints. Operator execution order dependency refers to the execution order relationship between operators in the model computation graph. For example, if the input of operator B must come from the output of operator A, then operator A must execute before operator B. The same-layer shard and same-domain execution constraint means that operator shards belonging to the same logical layer tend to be assigned to the same type of computing resources (i.e., domains) for execution. The same-slice operator affinity constraint means that operators within the same operator shard tend to be assigned to compute units that are physically close or share certain resources. This reduces data transmission delays between operators within the slice and improves execution efficiency. The modality priority allocation constraint requires that when processing multimodal AIGC models (such as simultaneous processing of text and images), specific modalities (such as image processing operators) be preferentially assigned to compute resources that can efficiently process that modality (such as GPUs with dedicated image processing units).

[0044] Optimization constraints ensure that operators are executed in the correct order, that operator slices at the same layer are placed on devices of the same type as much as possible, that operators within the same slice are placed as close together as possible, and that the processing requirements of specific modalities are prioritized. Optimization constraints are divided into mandatory constraints (which must be followed) and soft constraints (which are desirable). Mandatory constraints are rules that must be strictly followed; violating them will result in execution errors (for example, operator execution order dependencies). Soft constraints are rules that are desired to be met as much as possible; violating them may affect performance or efficiency but does not necessarily result in errors (for example, the same-layer, same-domain execution constraint for shards); cross-domain execution can sometimes work, but may be less efficient).

[0045] Based on matching features and additional features, the algorithm performs an optimization operation. By calculating the fitness of each solution (i.e., the quality of the resource scheduling solution), it generates a scheduling response plan, determining the optimal deployment of each operator slice on heterogeneous computing resources. Using matching features (describing key attributes of tasks and resources) and additional features (describing required rules and optimization preferences) as input, it explores various possible resource allocation combinations, evaluates how each combination best satisfies soft constraints while satisfying all mandatory constraints, and ultimately selects an optimal allocation solution based on pre-defined objectives (such as minimizing total execution time, minimizing total energy consumption, and maximizing computing power adaptation). Based on the matching optimization results, a scheduling response plan is generated: a specific, detailed set of instructions or plan that specifies which operators or operator slices in the AIGC model should be assigned to specific heterogeneous computing resources for execution. By efficiently matching the dynamic state of computing resources with task requirements, it significantly improves resource scheduling accuracy and task execution efficiency.

[0046] Furthermore, the present application further comprises the following steps:

[0047] Under the condition that the mandatory constraints are satisfied, an initial solution set is established; a computing power adaptation score is performed on the initial solution set to establish a first evaluation target; an execution time score is performed on the initial solution set to establish a second evaluation target; an energy consumption analysis is performed on the initial solution set to establish a third evaluation target; after normalizing the first evaluation target, the second evaluation target, and the third evaluation target, a fitness score of the solution is generated; the soft constraints are used to perform soft constraint penalty analysis on the initial solution set to establish a penalty decision direction; and optimization iterative management is performed based on the fitness score and the penalty decision direction to complete matching optimization.

[0048] Specifically, under mandatory constraints, a preliminary set of resource matching solutions is generated based on the existing operator sharding information, layer labels, and static hardware resource capabilities to establish an initial solution set. This initial solution set includes multiple solutions, each representing a possible operator-hardware resource matching method. All solutions in the initial solution set meet mandatory constraints, but they have varying strengths and weaknesses.

[0049] For each solution (i.e., a resource allocation method) in the initial solution set, a computing power adaptation score is performed to assess the degree to which the computing power resources allocated to each operator (or operator slice) match the actual computing requirements of the operator. For example, if a small operator requiring extensive parallel computing is assigned to a powerful GPU, the computing power is too high and the adaptation is poor; on the other hand, if an operator requiring a large amount of memory is assigned to a GPU with insufficient video memory, the adaptation is extremely poor. A higher score indicates a closer match between the resources and the operator's requirements. The computing power adaptation score considers the degree to which the hardware resources' computing power, memory bandwidth, storage capacity, and other factors match the operator's computing requirements (such as FLOPs and memory access patterns).

[0050] For each solution (i.e., a resource allocation method) in the initial solution set, an execution time score is calculated. This score evaluates the total time required to complete the entire task subgraph. This score takes into account the estimated execution time of each operator on its allocated resources and the dependencies between operators (serial or parallel). The execution time of the entire task is the sum of the execution times of all operators. The shorter the total execution time, the better.

[0051] For each solution (i.e., a resource allocation method) in the initial solution set, perform an energy consumption analysis. Consider the estimated energy consumption of each operator on its allocated resources, as well as the energy consumption characteristics of the resources themselves (for example, GPUs consume more power than CPUs under high load, but may be more efficient). The total energy consumption estimated for each solution in the initial solution set is evaluated. The lower the total energy consumption, the better.

[0052] The first, second, and third evaluation objectives are evaluation criteria based on the computing power adaptation score, execution time score, and energy consumption analysis, respectively. They represent three different dimensions of the optimization process: resource utilization efficiency, task completion speed, and energy consumption. Because the dimensions and numerical ranges of the three metrics—computing power adaptation score, execution time (seconds), and energy consumption (joules)—are significantly different, directly comparing their raw values ​​is meaningless. Therefore, the first, second, and third evaluation objectives are normalized. This process of converting these metrics of different dimensions to the same numerical range (typically 0 to 1) allows for comparison and weighting. For example, the solution with the shortest execution time is normalized to 1, and the longest to 0; the solution with the lowest energy consumption is normalized to 1, and the solution with the highest computing power adaptation score is normalized to 1, and the solution with the lowest computing power adaptation score is normalized to 0. Based on these normalized evaluation objectives, a comprehensive fitness score is generated for each solution, reflecting its overall performance in terms of resource utilization, task execution time, energy efficiency, and other aspects. Typically, the three normalized evaluation metrics (computing power adaptability, execution time, and energy consumption) are weighted and summed to create a comprehensive score. This score represents the overall quality of the initial solution after considering resource matching, execution speed, and energy consumption. A higher fitness score indicates a better solution.

[0053] Based on the soft constraints, each solution in the initial solution set is analyzed to see if it violates the soft constraints (e.g., prioritizing certain resources). For solutions that violate the soft constraints, an adjustment direction is determined, indicating how the solution should be modified in subsequent optimization iterations to better satisfy the soft constraints. For example, if many solutions are found to violate the same-shard operator affinity constraint, the penalty decision direction will be to allocate operators on the same shard to the same resources as much as possible. During the optimization process, the solution set is continuously adjusted through multiple iterations based on the fitness score and the penalty decision direction to approach the optimal solution.

[0054] For example, an initial solution set contains three solutions (S1, S2, and S3) for allocating operators slice 1 (50 GFLOPs, 1 GB), slice 2 (100 GFLOPs, 2 GB), and slice 3 (200 GFLOPs, 4 GB) to GPU0 (8 GB of free memory, 30% load) and GPU1 (6 GB of free memory, 50% load). Solution S1 is 1->GPU0, slice 2->GPU1, and slice 3->GPU0. Computational fit score: Slice 3 has too much computing power on GPU0, with a score of 0.7; slices 1 and 2 have acceptable fit, with a score of 0.8. Execution time score: Assuming a total execution time of 120 seconds. Energy consumption analysis: Total energy consumption is 300 joules. Soft constraints: Slices 1 and 3 are on the same GPU, satisfying affinity constraints. The modal processing operator (slice 3) is assigned to GPU0 (preferred), satisfying modal priority. The penalty is very low. Solution S2: Slice 1 -> GPU1, Slice 2 -> GPU1, Slice 3 -> GPU0. Computational Fit Score: Slice 3 on GPU0 has a score of 0.7; Slices 1 and 2 are memory-constrained on GPU1, so the score is 0.6. Execution Time Score: Assume a total execution time of 130 seconds (because GPU1 is under memory pressure, affecting Slice 2's speed). Energy Consumption Analysis: Total energy consumption is 310 joules. Soft Constraint: Slices 1 and 3 are not on the same GPU, violating the affinity constraint with a penalty of +5. Modal priority is satisfied. Solution S3: Slice 1 -> GPU0, Slice 2 -> GPU0, Slice 3 -> GPU1. Computational Fit Score: Slice 3 has just enough memory on GPU1 with a score of 0.75; Slices 1 and 2 are on GPU0 with a score of 0.8. Execution Time Score: Assume a total execution time of 115 seconds (GPU0 has high parallelism). Energy Consumption Analysis: Total energy consumption is 290 joules. Soft Constraint: Slices 1 and 3 are not on the same GPU, violating the affinity constraint with a penalty of +5. Modal priority may not be satisfied. Among the three solutions, the best fit score was 0.8, the worst was 0.6; the best time was 115 seconds, the worst was 130 seconds; the best energy consumption was 290 joules, and the worst was 310 joules. After normalization, the following results were obtained: the first evaluation target of solution S1 was 0.8, which was normalized to 1.0, the second evaluation target was 120, which was normalized to 0.33, and the third evaluation target was 300, which was normalized to 0.5; the first evaluation target of solution S2 was (0.7+0.6) / 2=0.65, which was normalized to 0.25, the second evaluation target was 130, which was normalized to 0, and the third evaluation target was 310, which was normalized to 1.0; the first evaluation target of solution S1 was (0.8+0.8+0.75) / 3=0.78, which was normalized to 0.9, the second evaluation target was 115, which was normalized to 1.0, and the third evaluation target was 290, which was normalized to 0. If the weight of the first evaluation target is 0.3, the weight of the second evaluation target is 0.5, and the weight of the third evaluation target is 0.2, the fitness of scheme S1 is 0.565, the fitness of S2 is 0.275, and the fitness of S3 is 0.77.Soft constraint penalty analysis: S1 has a low penalty, while S2 and S3 have the same penalty (+5). The penalty decision direction is to assign slices 1 and 3 to the same GPU. The current optimal solution is S3 (fitness 0.77). Based on the penalty decision direction, we attempt to modify S3 so that slices 1 and 3 are both on GPU 0. The resulting new solution, S4, is slice 1 -> GPU 0, slice 2 -> GPU 1, and slice 3 -> GPU 0. The calculated fitness of S4 is 0.75 (because slice 2 may spend a little longer on GPU 1) and the penalty is 0. Although the fitness is slightly reduced, the soft constraints are met. Based on comprehensive judgment (which may require adjusting weights or introducing a penalty term in the fitness calculation), S4 may become the new optimal solution. Iterations continue until a stable optimal solution is found.

[0055] By establishing a multi-objective, constrained optimization framework, we elevate the resource allocation problem from simple heuristic rules to an intelligent decision-making level closer to mathematical optimality. By introducing fitness scores, we quantitatively evaluate the pros and cons of different solutions, avoiding the biases inherent in subjective judgment or single-dimensional optimization. By considering soft constraints and performing penalty analysis, we achieve high performance while also addressing important non-functional requirements (such as resource affinity and modal processing priority), making the final scheduling solution more robust and reasonable.

[0056] Furthermore, the present application also includes the following steps: establishing candidate solutions for each solution in the initial solution set, the candidate solutions including guided candidate solutions and random candidate solutions, the guided candidate solutions being constructed based on the fitness score and the penalty decision direction, and the random candidate solutions being constructed based on the fitness score and a random function; establishing a quantity threshold constraint for solution set expansion, and using the quantity threshold constraint to perform threshold-triggered screening of candidate solutions; and using the threshold-triggered screening results to complete the iterative update of the initial solution set.

[0057] Specifically, for each solution in the initial solution set, several candidate solutions are generated, including guided candidate solutions and random candidate solutions. Each solution is used as a benchmark, and optimization modifications are made based on its current fitness score and the penalty for violating soft constraints. For example, operators with the longest execution time may be assigned to devices with lower loads, or operator slices may be migrated to nodes with higher affinity. These guided candidate solutions are constructed. Based on the fitness score of each solution, perturbations within a certain range are introduced (for example, using a Gaussian distribution to perturb resource allocation weights). While maintaining the original score distribution roughly unchanged, new resource allocation methods are added to construct random candidate solutions. Guided candidate solutions are generated based on the original solution by adjusting resource allocation or operator sorting based on the fitness score and the penalty for violating soft constraints. These solutions are guided by optimization. Random candidate solutions are generated based on random perturbations while preserving the characteristics of the fitness distribution. They are used to increase the diversity of the solution space and prevent local optima.

[0058] The solution set expansion threshold constraint sets the maximum number of candidate solutions allowed. It controls the solution set size, avoids excessive computational effort, explosion of the search space, and prevents unbounded solution growth. When the number of candidate solutions exceeds the threshold, a screening process is triggered. The solution set is sorted according to fitness score, prioritizing solutions with higher scores and eliminating those with lower scores to ensure that the solution set size remains below the threshold. For example, if the threshold constraint is set to 100, and the initial solution set contains 20 solutions, each generating 6 candidate solutions, for a total of 120 candidate solutions, exceeding the threshold, the top 100 candidate solutions are retained, while the remaining low-scoring solutions are discarded, ensuring that resources are prioritized for solutions with high potential. In subsequent iterations, new candidate solutions are generated for each of the 100 solutions, and the screening process is repeated until the optimal solution is found.

[0059] The screened high-quality candidate solutions are incorporated into the initial solution set, replacing some existing solutions or forming a new solution set for the next round of optimization iterations. The updated solution set not only includes higher-scoring scheduling strategies but also retains diversity, facilitating escape from local optima. By introducing guided and random candidate solutions, a more targeted and comprehensive exploration of the solution space is conducted, increasing the likelihood of finding the global optimal solution. By setting a threshold for quantity and performing threshold-triggered screening, the solution set size is effectively controlled, preventing uncontrolled expansion, thereby improving the computational efficiency and manageability of the optimization process.

[0060] Furthermore, the present application also includes the following steps: taking each solution in the initial solution set as the starting point of the path, establishing a path family according to the iterative optimization result; performing path exploration coverage analysis of the solution set at preset periodic nodes, and establishing exploration constraints; using the path family to calculate the health index of the path, and generating a health index calculation result; if the health index calculation result determines that the path family is degraded, executing the corresponding path family taboo operation; completing the optimization iterative management according to the exploration constraints of the periodic nodes and the path family taboo operation.

[0061] Specifically, each solution in the initial solution set is used as a path starting point, the starting point for the optimization path search. A path family is established based on the iterative optimization results. A path family is the set of solutions formed through a series of iterative optimization steps starting from the path starting point. Each path represents the evolution of a scheduling direction, while a path family represents the parallel exploration of multiple directions throughout the optimization process. Each solution is used as the starting point for the optimization path. For each solution, multiple rounds of candidate solutions are generated through a guided / random method. Their evolutionary trajectory is recorded to form a path family.

[0062] Periodic nodes are evaluation nodes set at iteration times. Every N iterations (e.g., every five iterations) triggers a path evaluation and adjustment, periodically reviewing the path's validity and whether it has fallen into a local optimum. At pre-set periodic nodes (e.g., every five iterations), a path exploration coverage analysis of the solution set is performed. This involves calculating the distribution of the path family in the solution space, including the solution space span of each path, the Euclidean distance between paths, and the fitness distribution of each path's endpoints. This evaluation assesses the breadth and depth of the path family's exploration of the solution space and establishes exploration constraints. Exploration constraints are a set of rules or restrictions established based on the path exploration coverage analysis to guide the expansion and optimization of the path family and ensure that it effectively explores the solution space. For example, if paths are overly concentrated, the path diversity weight is increased to encourage new paths to migrate to the boundary areas. If paths are too numerous in certain areas, the fitness values ​​of densely populated areas are penalized to encourage them to exit.

[0063] Path families are used to calculate path health indicators, generating health indicator results that quantify each path's current optimization potential. The health of each path is comprehensively assessed using factors such as fitness improvement rate (e.g., whether fitness has continued to grow over the past M rounds), local volatility (e.g., whether fitness is oscillating rather than converging), and structural diversity (e.g., whether candidate solutions within the path are gradually converging). A health threshold is set; if the calculated health indicator falls below the threshold, the path is considered degraded. In other words, when a path fails to improve in fitness over several consecutive iterations or repeatedly oscillates around a local solution, its optimization capability is reduced and it is considered degraded. When a path family becomes trapped in a local optimal solution during iteration, or when the diversity of the path family decreases, it becomes impossible to effectively explore the solution space.

[0064] For paths identified as degenerate, corresponding path family taboo operations are performed, including freeze operations (pausing subsequent iterations of the path to free up resources for other paths), perturbation operations (reconstructing the path's initial point and introducing high-intensity perturbations for regeneration), and blockade operations (completely removing it from the path family to prevent it from occupying the solution space). Path family taboo operations impose penalties or blockade strategies on degenerate paths, such as restricting further iterations, forcibly disrupting its solution, removing it from the path family, increasing randomness, introducing new candidate solutions, and adjusting iteration parameters, to prevent the algorithm from falling into local optimality.

[0065] Based on path taboo decisions and exploration constraints, the path family structure is adjusted to form a new path evolution direction, and the subsequent iteration strategy is updated until the global optimal solution is found. For example, for multimodal dialogue generation (comprising image and text input, outputting natural language paragraphs), the solution space dimension is 120 (operator × node combinations), the initial number of paths is 16 (corresponding to 16 initial solutions), each path is iterated for 10 rounds, and the periodic evaluation node is triggered every 5 rounds. Some data for path health scores are as follows: Path 3 has a fitness improvement rate of 0.2 over the past five rounds, high local volatility, and low structural diversity, resulting in a health score of 0.28 (degraded); Path 6 has a fitness improvement rate of 0.5 over the past five rounds, medium local volatility, and medium structural diversity, resulting in a health score of 0.61; Path 8 has a fitness improvement rate of 0.03 over the past five rounds, high local volatility, and low structural diversity, resulting in a health score of 0.11 (severely degraded). Perturbation reconstruction was performed on path 3, and the initial solution generation rules were adjusted. Freeze and blockade were performed on path 8, and the path was eliminated. Resources were reallocated to other paths, such as path 7 and path 9. The solution space diversity index was improved from 0.57 to 0.71.

[0066] By establishing a path family and performing path exploration coverage analysis, we comprehensively evaluate the exploration status of the path family in the solution space, ensure that the path family can effectively explore different areas and explore these areas in depth, promptly discover and deal with degradation problems that may occur in the path family during the iteration process, and maintain the exploration ability and diversity of the path family.

[0067] Furthermore, the present application also includes the following steps: establishing context load features based on the predicted generation length and input modality type of the task subgraph; extracting cross-layer context dependency labels using the context load features and operator sharding processing results; performing compensation optimization on the initial solution set based on the cross-layer context dependency labels, and updating the solution set using the compensation optimization results.

[0068] Specifically, after receiving the current AIGC task, the predicted generation length and input modality type are obtained. The predicted generation length is the length of the content that the AIGC model expects to generate in the current inference task, such as the number of tokens in the generated text and the resolution range of the generated image. This directly affects the task's computational effort and context maintenance cost. The input modality type is the type of input data entering the model, such as text, image, audio, video, or a multimodal combination (such as text-image dialogue). The processing paths and operator complexity corresponding to different modalities vary significantly.

[0069] Based on the predicted generation length and input modality type, context load features are established to characterize the context maintenance load of the current AIGC task on computing resources, including cache tensor size, context window length, and number of modal intersections. The following context load metrics are established based on the generation length and modality type: the context window length (token) is equal to or slightly higher than the generation length (e.g., L+64), where L+64 refers to the actual generation length L required by the AIGC model plus a buffer length of 64, which is the additional space reserved for the context window; the number of activation cache tensors is estimated by generation length × number of layers × number of heads; the number of modal intersections is typically ≥ 2 in image-text tasks (e.g., image-to-text attention, text-to-image alignment); the state transfer depth is the range of layers spanned by the activation subgraph; and the state persistence requirement is whether persistent caching is required.

[0070] Based on the context load characteristics and operator sharding processing results, cross-layer context dependency labels are extracted to reflect whether there are intermediate states, cached tensors, or control flow dependencies between different operator slices, such as strong dependency, weak dependency, or no dependency. For example, if the intermediate tensor output by operator A is reused by operators B and C, and A and B / C belong to different layers, then it is a strong dependency; if operators D and E share structured state through jump connections, then it is a weak dependency; if operators F and G do not affect each other and are on completely independent paths, then there is no dependency.

[0071] Based on the cross-layer context dependency labels, the initial solution set is subjected to compensation optimization to obtain a compensated optimization result. Since the initial solution set may not fully consider cross-layer dependencies, compensation optimization aims to adjust resource allocation to better meet these dependencies, thereby improving the efficiency and stability of task execution. For example, for strongly dependent chip pairs, priority is given to scheduling them to the same node to avoid high-latency cross-node state transfer; for weakly dependent chip pairs, they are scheduled to co-domain resources (such as the same GPU group) to maintain a certain degree of decoupling within the latency tolerance range; for non-dependent chip pairs, priority is given to scheduling them to idle or energy-efficient resources to maximize parallelism.

[0072] Based on the compensation optimization results, the initial solution set is updated, replacing the original solution with the optimized, more reasonable scheduling solution. This ensures that resource allocation better aligns with the actual task requirements and cross-layer dependencies. By establishing contextual load features and extracting cross-layer contextual dependency labels, the dependencies between task subgraphs at different layers are accurately captured, enabling more efficient resource allocation and task scheduling.

[0073] Furthermore, the present application also includes the following steps: obtaining the hardware resources and supporting modal information of heterogeneous computing resources to establish static capability characteristics; obtaining the real-time computing power occupancy information of heterogeneous computing resources, and establishing dynamic load characteristics based on the real-time computing power occupancy information; and extracting a computing power status feature set using the static capability characteristics and the dynamic load characteristics.

[0074] Furthermore, the present application also includes the following steps: establishing a dynamic backtracking window for heterogeneous computing resources; using the dynamic backtracking window to perform stability analysis of heterogeneous computing resources and establish dynamic compensation features; and adding the dynamic compensation features as additional features to the computing power status feature set.

[0075] Specifically, obtain information about the hardware resources and supported modalities of heterogeneous computing resources. Hardware resources include hardware component information, such as GPU core count, memory capacity, number of compute units, and bandwidth, among other hardware capability characteristics. Supported modalities refer to the different computing task types or data modalities (e.g., text, images, and video) that the hardware resources can support. Certain hardware may be more suitable for processing specific tasks, such as GPUs typically excel at deep learning tasks, while TPUs are optimized for neural network inference. Use system monitoring tools to obtain the static hardware capabilities of each computing resource, such as the number of GPU cores (e.g., the A100 has 6912 CUDA cores), memory (e.g., 16GB of video memory), and compute bandwidth (e.g., 900GB / s). Determine which modalities each hardware resource can efficiently execute. For example, some high-end GPUs support deep learning model training and inference, while some TPUs are optimized for neural network inference and matrix operations. Static capability characteristics are fixed hardware parameters and performance metrics of heterogeneous computing resources, such as memory capacity, number of compute units, and supported task modalities. These characteristics generally do not change over time.

[0076] By monitoring hardware usage in real time, we can obtain dynamic load information for each computing resource and obtain real-time computing power utilization information, including but not limited to GPU or TPU utilization, memory usage, bandwidth usage, and task queue information, reflecting the actual load and remaining computing capacity of the hardware resources. Real-time computing power utilization information is the current actual load of the hardware resources, including dynamic information such as the current computing resource utilization rate, memory usage, bandwidth usage, and the number of running tasks. It is usually obtained through real-time monitoring or resource management tools. Based on the real-time monitored computing power utilization information, dynamic load characteristics are generated to describe the current resource load status, such as GPU utilization, memory usage, and task queue length.

[0077] By combining static capacity characteristics with dynamic load characteristics, we can form a computing power status feature set for each computing resource, which comprehensively describes the current state of each heterogeneous computing power resource. The computing power status feature set includes both the inherent potential of the resource and its current availability.

[0078] A dynamic backtracking window is configured for each heterogeneous computing resource (such as a GPU or TPU), typically covering a period of time (e.g., load data for the past 5 or 10 minutes). Each dynamic backtracking window records the resource's load during that period, including CPU / GPU utilization, memory usage, and bandwidth usage. Dynamic backtracking windows can be used to analyze performance fluctuations and load changes of a hardware resource (such as a GPU or TPU) within a specific timeframe as needed.

[0079] Based on the data within the dynamic backtracking window, a stability analysis of heterogeneous computing resources is performed, analyzing the computing resource performance over that period, including metrics such as load fluctuation range, maximum load duration, and idle time. Resources with high stability are prioritized during scheduling. After performing a stability analysis on the data within the dynamic backtracking window, dynamic compensation features are extracted, including information such as resource fluctuation amplitude, load change trends, historical optimal load, and latency. These features reflect the recent instability, performance fluctuations, or potential risks exhibited by the computing resources. The dynamic compensation features are added as additional information to the computing power status feature set. The updated computing power status feature set not only includes the resource's static capabilities (such as the number of cores, memory size, and supported modes) and dynamic load (such as current utilization and memory pressure), but also includes information reflecting its recent stability.

[0080] Collaborative acceleration management is performed according to the scheduling response plan.

[0081] Furthermore, the present application also includes the following steps: configuring a redundant detection window, and using the redundant detection window to monitor the real response of heterogeneous computing resources when executing the scheduling response plan; if the real response monitoring meets the preset abnormal value, triggering a redundant replacement instruction; and performing redundant compensation of heterogeneous computing resources according to the redundant replacement instruction.

[0082] Specifically, according to the scheduling response plan, collaborative acceleration management of heterogeneous computing resources is initiated. Target heterogeneous computing resource nodes are activated sequentially according to the scheduling response plan, initiating the execution process of the corresponding operators. Because tasks may have dependencies (for example, operator C depends on the outputs of A and B), collaborative acceleration requires the introduction of a task dependency graph scheduler, which dynamically coordinates the start timing and communication strategies of each operator based on the operator execution order and data dependencies. During the collaborative process, the load status of each computing node is monitored in real time. If the load on a node surges (for example, GPU2 utilization exceeds 90% or the temperature exceeds 85°C), a fault-tolerant buffering strategy is used to transfer task B to the backup node GPU3, and the output of A is synchronized to GPU3 through a data mirroring mechanism. Precisely and coordinately controlling task execution on each heterogeneous computing resource according to the scheduling response plan involves more than simply assigning tasks to the corresponding resources; it involves a series of sophisticated management operations.

[0083] During the operation of heterogeneous computing resources, a real-time monitoring period window (e.g., 500ms to 2s) is set for each resource node. This fixed-length redundant detection time window is used to periodically collect and analyze resource operating status (such as execution latency, abnormal output, temperature, current, and voltage) to determine whether it conforms to the normal value range specified in the scheduling response plan. Using this redundant detection window, the actual status of heterogeneous computing resources is sampled and analyzed based on preset monitoring indicators (such as execution time, resource utilization, and error signals). The collected real-time data is compared with the expected values ​​for normal operation or a historical baseline.

[0084] Preset exception values ​​are a series of thresholds or criteria predefined during configuration that define normal operation and abnormal conditions. When the actual response data monitored exceeds the preset exception values, an abnormality is determined. For example, a preset GPU memory usage exceeding 90% for more than 3 seconds is considered an abnormality, or an operator execution time that is more than 50% slower than expected is considered an abnormality. When a resource abnormality is detected and an instruction is triggered, a redundant replacement instruction is generated, indicating that the task originally assigned to the abnormal resource needs to be transferred to the reserved, available redundant resources (or other healthy resources) in the system for execution.

[0085] Following redundancy replacement instructions, tasks are offloaded from the faulty resource, necessary state is saved (such as caching intermediate results), and then tasks are reassigned and scheduled to new, healthy resources for continued execution. This compensates for interruptions or performance degradation caused by resource anomalies and ensures the continuous and stable completion of the entire AIGC task. By configuring a redundancy detection window, the robustness and reliability of the entire heterogeneous computing resource collaborative acceleration system are improved.

[0086] In summary, the AIGC-oriented heterogeneous computing resource collaborative acceleration method provided in this application has the following beneficial effects:

[0087] After the AIGC model is activated, the AIGC model is evaluated for actual execution of task subgraphs, model parsing of the task subgraphs is performed, and operator graphs are extracted. The AIGC model's structural and functional semantics are used to stratify the task subgraphs based on the operator graphs, establishing hierarchical labels. Operator sharding is performed under the hierarchical labels, and operator features are extracted under dual-granularity constraints based on the operator sharding results and the hierarchical labels to establish a multidimensional feature set. The computing power status of heterogeneous computing resources is collected to extract a computing power status feature set. The multidimensional feature set and the computing power status feature set are used as matching features to optimize the matching of heterogeneous computing power resources with operators. A scheduling response plan for the heterogeneous computing power resources is configured based on the matching optimization results. Collaborative acceleration management is performed based on the scheduling response plan. In other words, by performing task subgraph evaluation and operator graph extraction, the AIGC model's execution path is parsed in real time. Based on the matching optimization of the multidimensional feature set and the computing power status feature set, dynamic scheduling of heterogeneous computing power resources and model operators is achieved, improving the utilization of heterogeneous computing power resources and thereby enhancing the execution efficiency of the AIGC model.

[0088] In the second embodiment, based on the same inventive concept as the AIGC-oriented heterogeneous computing power resource collaborative acceleration method in the aforementioned first embodiment, this application also provides an AIGC-oriented heterogeneous computing power resource collaborative acceleration system, please refer to the attached Figure 2 The AIGC-oriented heterogeneous computing resource collaborative acceleration system includes:

[0089] The task subgraph determination module 11 is used to determine the actual execution task subgraph of the AIGC model after the AIGC model is input and activated, perform model analysis of the task subgraph, and extract the operator graph; the task subgraph stratification module 12 is used to use the structural semantics and functional semantics of the AIGC model to stratify the task subgraph based on the operator graph and establish a stratification label; the operator feature extraction module 13 is used to perform operator sharding processing under the stratification label, perform operator feature extraction under double-granularity constraints based on the operator sharding processing results and the stratification label, and establish a multidimensional feature set; the matching optimization module 14 is used to perform computing power status collection of heterogeneous computing power resources, extract computing power status feature sets, use the multidimensional feature set and the computing power status feature set as matching features, perform matching optimization between heterogeneous computing power resources and operators, and configure the scheduling response plan of the heterogeneous computing power resources according to the matching optimization results; the collaborative acceleration management module 15 is used to perform collaborative acceleration management according to the scheduling response plan.

[0090] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing resource collaborative acceleration system is also used to: extract optimization constraints based on the hierarchical labels and the operator shard processing results, and the optimization constraints include operator execution order dependency, same-layer shard and same-domain execution constraints, same-shard operator affinity constraints, and modality priority allocation constraints; after dividing the optimization constraints into mandatory constraints and soft constraints, the mandatory constraints and the soft constraints are used as additional features, and matching optimization is performed according to the matching features and the additional features.

[0091] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing power resource collaborative acceleration system is also used to: establish an initial solution set under the conditions of satisfying the mandatory constraints; perform computing power adaptation scoring on the initial solution set to establish a first evaluation target; perform execution time scoring on the initial solution set to establish a second evaluation target; perform energy consumption analysis on the initial solution set to establish a third evaluation target; generate a fitness score for the solution after normalizing the first, second, and third evaluation targets; perform soft constraint penalty analysis on the initial solution set using the soft constraints to establish a penalty decision direction; perform iterative optimization management based on the fitness score and the penalty decision direction to complete matching optimization.

[0092] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing resource collaborative acceleration system is also used to: establish candidate solutions for each solution in the initial solution set, the candidate solutions include guided candidate solutions and random candidate solutions, the guided candidate solutions are constructed based on the fitness score and the penalty decision direction, and the random candidate solutions are constructed based on the fitness score and the random function; establish a quantity threshold constraint for solution set expansion, and use the quantity threshold constraint to perform threshold-triggered screening of candidate solutions; and use the threshold-triggered screening results to complete the iterative update of the initial solution set.

[0093] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing resource collaborative acceleration system is also used to: use each solution in the initial solution set as the path starting point, and establish a path family according to the iterative optimization result; perform path exploration coverage analysis of the solution set at preset periodic nodes to establish exploration constraints; use the path family to calculate the health index of the path and generate a health index calculation result; if the health index calculation result determines that the path family is degraded, execute the corresponding path family taboo operation; and complete the optimization iterative management according to the exploration constraints of the periodic nodes and the path family taboo operation.

[0094] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing resource collaborative acceleration system is also used to: establish context load features based on the predicted generation length and input modality type of the task subgraph; extract cross-layer context dependency labels using the context load features and operator sharding processing results; perform compensation optimization on the initial solution set based on the cross-layer context dependency labels, and update the solution set using the compensation optimization results.

[0095] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing power resource collaborative acceleration system is also used to: obtain the hardware resources and supporting modal information of the heterogeneous computing power resources to establish static capability characteristics; obtain the real-time computing power occupancy information of the heterogeneous computing power resources, and establish dynamic load characteristics based on the real-time computing power occupancy information; and extract the computing power status feature set using the static capability characteristics and the dynamic load characteristics.

[0096] Furthermore, the matching optimization module 14 in the AIGC-oriented heterogeneous computing power resource collaborative acceleration system is also used to: establish a dynamic backtracking window for heterogeneous computing power resources; use the dynamic backtracking window to perform stability analysis of heterogeneous computing power resources and establish dynamic compensation features; and add the dynamic compensation features as additional features to the computing power status feature set.

[0097] Furthermore, the collaborative acceleration management module 15 in the AIGC-oriented heterogeneous computing power resource collaborative acceleration system is also used to: configure a redundant detection window, and use the redundant detection window to monitor the real response of heterogeneous computing power resources when executing the scheduling response plan; if the real response monitoring meets the preset abnormal value, a redundant replacement instruction is triggered; and redundant compensation of heterogeneous computing power resources is performed according to the redundant replacement instruction.

[0098] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. Figure 1 The AIGC-oriented heterogeneous computing power resource collaborative acceleration method and specific examples in Example 1 are also applicable to the AIGC-oriented heterogeneous computing power resource collaborative acceleration system in this embodiment. Through the above detailed description of the AIGC-oriented heterogeneous computing power resource collaborative acceleration method, those skilled in the art can clearly understand the AIGC-oriented heterogeneous computing power resource collaborative acceleration system in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0099] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

[0100] Obviously, for those skilled in the art, several improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. The AIGC-oriented heterogeneous computing resource collaborative acceleration method is characterized by: include: After the AIGC model is input and activated, the AIGC model is actually executed on the task subgraph, the model of the task subgraph is parsed, and the operator graph is extracted; The structural semantics and functional semantics of the AIGC model are used to perform task subgraph stratification based on the operator graph and establish hierarchical labels; Operator sharding is performed under hierarchical labels. Operator feature extraction under dual-granularity constraints is performed based on the operator sharding results and hierarchical labels to establish a multidimensional feature set. Performing computing power status collection of heterogeneous computing power resources, extracting a computing power status feature set, performing optimal matching between heterogeneous computing power resources and operators using the multi-dimensional feature set and the computing power status feature set as matching features, and configuring a scheduling response plan for the heterogeneous computing power resources based on the optimal matching result; Performing collaborative acceleration management according to the dispatch response plan; The performing of optimal matching between heterogeneous computing resources and operators using the multi-dimensional feature set and the computing power status feature set as matching features includes: Extracting optimization constraints based on the layered labels and the operator sharding processing results, the optimization constraints include operator execution order dependency, same-layer sharding and same-domain execution constraints, same-shard operator affinity constraints, and modality priority allocation constraints; After dividing the optimization constraints into mandatory constraints and soft constraints, the mandatory constraints and the soft constraints are used as additional features, and matching optimization is performed according to the matching features and the additional features; The performing matching optimization according to the matching features and the additional features includes: Under the conditions of satisfying the mandatory constraints, an initial solution set is established; Performing a computing power adaptation score on the initial solution set to establish a first evaluation target; Score the execution time of the initial solution set to establish a second evaluation target; Performing energy consumption analysis on the initial solution set to establish a third evaluation target; After performing normalization processing on the first evaluation objective, the second evaluation objective, and the third evaluation objective, a fitness score of the solution is generated; Performing soft constraint penalty analysis on the initial solution set using the soft constraints to establish a penalty decision direction; Performing optimization iterative management according to the fitness score and the penalty decision direction to complete matching optimization; The optimization iterative management is performed according to the fitness score and the penalty decision direction to complete the matching optimization, including: Establishing a candidate solution for each solution in the initial solution set, the candidate solutions including guided candidate solutions and random candidate solutions, the guided candidate solutions being constructed based on the fitness score and the penalty decision direction, and the random candidate solutions being constructed based on the fitness score and a random function; Establishing a quantitative threshold constraint for solution set expansion, and using the quantitative threshold constraint to perform threshold-triggered screening of candidate solutions; The threshold is used to trigger the screening results to complete the iterative update of the initial solution set.

2. The AIGC-oriented heterogeneous computing resource collaborative acceleration method according to claim 1, characterized in that: The performing optimization iterative management according to the fitness score and the penalty decision direction further includes: Taking each solution in the initial solution set as a path starting point, a path family is established according to the iterative optimization result; Perform path exploration and coverage analysis of the solution set at the preset cycle nodes and establish exploration constraints; Calculate the health index of the path using the path family and generate the health index calculation result; If the calculation result of the health index determines that the path family is degraded, performing the corresponding path family taboo operation; The optimization iteration management is completed according to the exploration constraints of the periodic nodes and the taboo operations of the path family.

3. The AIGC-oriented heterogeneous computing resource collaborative acceleration method according to claim 1, characterized in that: The performing optimization iterative management according to the fitness score and the penalty decision direction further includes: Establishing a context load feature based on the predicted generation length of the task subgraph and the input modality type; Extracting cross-layer context dependency labels using the context load features and operator sharding processing results; The initial solution set is compensated and optimized based on the cross-layer context dependency label, and the solution set is updated using the compensated optimization result.

4. The AIGC-oriented heterogeneous computing resource collaborative acceleration method according to claim 1, characterized in that: The execution of computing power status collection of heterogeneous computing power resources and extraction of computing power status feature sets includes: Obtain the hardware resources and supported modality information of heterogeneous computing resources and establish static capability characteristics; Obtaining real-time computing power usage information of heterogeneous computing resources, and establishing dynamic load characteristics based on the real-time computing power usage information; The static capacity characteristics and the dynamic load characteristics are used to extract a computing power status feature set.

5. The AIGC-oriented heterogeneous computing resource collaborative acceleration method according to claim 4, characterized in that: The extracting of the computing power state feature set by using the static capability feature and the dynamic load feature includes: Establish a dynamic backtracking window for heterogeneous computing resources; Utilizing the dynamic backtracking window to perform stability analysis on heterogeneous computing resources and establish dynamic compensation features; The dynamic compensation feature is added as an additional feature to the computing power status feature set.

6. The AIGC-oriented heterogeneous computing resource collaborative acceleration method according to claim 1, characterized in that: The performing collaborative acceleration management according to the scheduling response scheme includes: Configuring a redundancy detection window, and utilizing the redundancy detection window to monitor the true response of heterogeneous computing resources when executing a scheduling response plan; If the actual response monitoring meets the preset abnormal value, the redundant replacement instruction is triggered; Redundancy compensation of heterogeneous computing resources is performed according to the redundant replacement instruction.

7. The AIGC-oriented heterogeneous computing resource collaborative acceleration system is characterized by: The steps for implementing the AIGC-oriented heterogeneous computing power resource collaborative acceleration method according to any one of claims 1 to 6, wherein the AIGC-oriented heterogeneous computing power resource collaborative acceleration system comprises: A task subgraph determination module is used to determine the actual execution task subgraph of the AIGC model after the AIGC model is input and activated, perform model analysis of the task subgraph, and extract the operator graph; The task subgraph stratification module is used to use the structural semantics and functional semantics of the AIGC model to stratify task subgraphs based on operator graphs and establish stratification labels; The operator feature extraction module is used to perform operator sharding processing under hierarchical labels, and extract operator features under dual-granularity constraints based on the operator sharding processing results and hierarchical labels to establish a multidimensional feature set; A matching optimization module is used to collect computing power status of heterogeneous computing power resources, extract a computing power status feature set, use the multi-dimensional feature set and the computing power status feature set as matching features, perform matching optimization between heterogeneous computing power resources and operators, and configure a scheduling response plan for the heterogeneous computing power resources based on the matching optimization results; The collaborative acceleration management module is used to perform collaborative acceleration management according to the scheduling response plan.

Images