Operator-level performance attribution method and system for artificial intelligence large models

Abstract

This invention discloses a method and system for operator-level performance attribution of a large-scale artificial intelligence model. The method includes: generating performance profiles of each operator instance on source and target hardware, and measuring the end-to-end true performance values ​​of the large-scale artificial intelligence model on the source and target hardware; constructing a counterfactual alliance; constructing a surrogate function based on the performance profiles, and performing endpoint calibration on the surrogate function using the end-to-end true performance values ​​to obtain a calibrated surrogate function; randomly sorting all operator instances multiple times, adding operator instances one by one to the counterfactual alliance in the sorted order each time, calculating the change in the end-to-end performance estimate of the counterfactual alliance before and after each addition of an operator instance based on the calibrated surrogate function, and using this change as the marginal contribution of the added operator instance; and statistically processing all marginal contributions of each operator instance to obtain the performance contribution estimate of each operator instance. This invention can effectively achieve operator-level performance attribution.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a method and system for attributing operator-level performance of large artificial intelligence models. Background Technology

[0002] With the widespread application of large-scale AI models, such as large language models, large-scale vision models, and multimodal large models, in various industries including computer vision and natural language processing, model inference performance has become a key bottleneck in industrial deployment. The diversity of hardware platforms and the heterogeneity of compilation backends mean that the inference performance of the same model can vary significantly across different hardware or backends. Therefore, performance diagnostics and operator-level attribution analysis are core technical requirements for guiding model adaptation, operator optimization, and hardware selection.

[0003] Existing performance diagnostic solutions can be summarized into the following three categories: The first approach is the end-to-end benchmarking scheme. This is currently the most widely used performance evaluation path. Its core is to run complete model inference on both the source and target hardware, recording macro-level metrics such as end-to-end latency, throughput, and energy consumption. The difference between these metrics is then compared to determine migration performance changes. This scheme can evaluate the model's speed on the target platform, and the measurement results are reliable. However, this scheme has inherent drawbacks: it lacks operator-level interpretation capabilities; end-to-end metrics cannot determine fine-grained issues such as performance differences caused by specific operators, making it difficult to establish executable optimization priorities; and its diagnostic efficiency is low, requiring developers to repeatedly modify the model or switch backends and re-execute complete benchmarks to locate performance bottlenecks, resulting in long iteration cycles and high costs.

[0004] The second approach involves operator micro-benchmarking and dynamic performance analysis tools. This approach measures the execution time of operator kernels (such as matrix multiplication, convolution, attention, and regularization modules) under different input shapes individually, or outputs the timeline percentage of each operator in the model through the framework's built-in dynamic performance analysis tools. This approach provides operator-level execution time statistics, helping developers identify hot operators. However, this approach has the following inherent drawbacks: isolated measurements ignore context; micro-benchmarking treats operators in isolation, and the measurement results cannot reflect the operator's critical path position in the computation graph; the direct summation of single operator latencies usually does not equal the end-to-end model latency; the percentages from dynamic performance analysis tools lack counterfactual semantics; the time percentages output by dynamic performance analysis tools are observational statistics of the operator under the current implementation, rather than causal or counterfactual contributions indicating how much end-to-end performance would change if the operator were replaced with the target hardware implementation, and cannot guide optimization decisions in cross-hardware migration scenarios; and it cannot evaluate non-executable combinations. In cross-hardware migration scenarios, arbitrary subsets of operators often cannot be directly spliced ​​and executed between two physical platforms. Micro-benchmark schemes cannot provide measurement for the counterfactual state that "some operators are implemented on the target side while the rest remain implemented on the source side".

[0005] The third approach is an attribution scheme based on the ordinary Monte Carlo Shapley value. This approach treats model performance as a cooperative game among operators. It constructs a coalition by randomly sampling operators and incrementally assigning them, then calculates the expected performance change of each operator before and after joining the coalition, using this as a measure of the operator's contribution. Theoretically, this approach provides a fair framework for contribution sharing. However, it has inherent drawbacks: exponential complexity or high query costs, and the need to evaluate all operators for an accurate Shapley value. There are several operator subsets. While Monte Carlo sampling can reduce the number of queries, the cost of obtaining each query through the execution of the full model is too high to be deployed in real-world scenarios. Ordinary permutations have large variances, and ordinary random permutation sampling is affected by the position of the operator in the permutation. The marginal contribution of the same operator differs significantly depending on whether it is at the beginning or end of the permutation, leading to unstable estimation and unreliable sorting under limited budget. Physical splicing is not feasible. In heterogeneous hardware scenarios, executing any subset of operators on the source hardware and the rest on the target hardware and splicing them into an end-to-end result of a complete model is usually not physically feasible.

[0006] The existing technology has the following five shortcomings: End-to-end benchmarks lack operator-level causal and counterfactual interpretations, making it impossible to formulate executable optimization priorities. The impact of a particular operator on end-to-end performance cannot be analyzed.

[0007] The direct summation of single operator delays does not consider the critical path of the computation graph, parallel execution, operator fusion, memory contention, and runtime scheduling overhead. This contributes to and significantly deviates from the true end-to-end migration difference, making the attribution results unreliable.

[0008] Accurate Shapley values ​​require exponential federation queries; physical execution federations are typically unconstructable across hardware scenarios; ordinary Monte Carlo sampling has large variance; and sorting is unstable under limited budgets.

[0009] Existing technologies lack a full-process reconciliation mechanism for migration differences from source hardware to target hardware. The attribution results and the measured end-to-end differences may have unexplained residuals, reducing the reliability of the engineering.

[0010] The root cause of these shortcomings lies in the need to strike a balance between expressiveness and computability in performance attribution. Accurate attribution requires considering all interactions between operators, but this necessitates exponential computation or physically non-executable union construction; surrogate approximations can reduce computational costs but may introduce systematic biases. Existing solutions fail to provide an operator-level attribution method that preserves counterfactual substitution semantics, completes in polynomial time, and is auditable. Furthermore, physical constraints in heterogeneous hardware scenarios—such as the inability to cross-platform operator implementations, the nonlinear impact of computation graph topology on execution time, and the dynamic changes in runtime overhead—further complicate accurate attribution.

[0011] The background section described above is merely a description made by the inventor based on his understanding, and the above content should not be regarded as evidence of prior art disclosed before the filing date of this application. Summary of the Invention

[0012] This invention provides an operator-level performance attribution method for large-scale artificial intelligence models, addressing the technical problems of existing end-to-end testing failing to provide operator-level performance explanations and single-operator micro-benchmarks ignoring computation graph topology and runtime context, leading to inaccurate attribution. It also solves the technical problems of requiring exponential consortium queries for accurate attribution, large variance in ordinary random sampling, and the impracticality of physical stitching in heterogeneous scenarios. Furthermore, it provides a mechanism for reconciling attribution results with actual end-to-end differences. The method includes: Performance measurements are performed on each operator instance in the computation graph of the large AI model on both the source and target hardware. Performance profiles of each operator instance are generated on the source and target hardware. The end-to-end true performance values ​​of the large AI model on the source hardware and the end-to-end true performance values ​​on the target hardware are measured. Construct a counterfactual alliance, which contains any number of operator instances, wherein the operator instances within the counterfactual alliance adopt the performance profile of the target hardware, and the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. A proxy function is constructed based on the performance profile, and the end-to-end real performance value is used to perform endpoint calibration on the proxy function to obtain the calibrated proxy function. All operator instances are randomly sorted multiple times. Each time, operator instances are added to the counterfactual coalition one by one according to the sorting order. The change in the end-to-end performance estimate of the counterfactual coalition before and after each addition of an operator instance is calculated based on the calibrated surrogate function and used as the marginal contribution of the added operator instances. Statistical processing is performed on all marginal contributions of each operator instance to obtain an estimated performance contribution value for each operator instance.

[0013] Another aspect of this invention provides an operator-level performance attribution system for large-scale artificial intelligence models, addressing the technical problems of existing end-to-end testing failing to provide operator-level performance explanations and single-operator micro-benchmarks ignoring computation graph topology and runtime context, leading to inaccurate attribution. It also solves the technical problems of requiring exponential consortium queries for accurate attribution, large variance in ordinary random sampling, and the impracticality of physical stitching in heterogeneous scenarios. Furthermore, it provides a mechanism for reconciling attribution results with actual end-to-end differences. The system includes: The dual-end profiling module is used to perform performance measurements on each operator instance in the computation graph of the large artificial intelligence model on both the source and target hardware, generate performance profiles of each operator instance on the source and target hardware, and measure the end-to-end true performance value of the large artificial intelligence model on the source hardware and the end-to-end true performance value on the target hardware. The counterfactual alliance generation module is used to construct a counterfactual alliance, which contains any number of operator instances. The operator instances within the counterfactual alliance adopt the performance profile of the target hardware, while the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. The proxy function construction module is used to construct a proxy function based on the performance profile, perform endpoint calibration on the proxy function using the end-to-end real performance value, and obtain the calibrated proxy function. The marginal contribution calculation module is used to randomly sort all operator instances multiple times. Each time, operator instances are added to the counterfactual alliance one by one according to the sorting order. Based on the calibrated surrogate function, the change in the end-to-end performance estimate of the counterfactual alliance before and after each addition of an operator instance is calculated and used as the marginal contribution of the added operator instance. The performance contribution estimation module is used to perform statistical processing on all marginal contributions of each operator instance to obtain the performance contribution estimate of each operator instance.

[0014] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the operator-level performance attribution method of the above-mentioned large-scale artificial intelligence model.

[0015] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the operator-level performance attribution method for the aforementioned large-scale artificial intelligence model.

[0016] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the operator-level performance attribution method of the aforementioned large-scale artificial intelligence model.

[0017] In this embodiment of the invention, performance measurements are performed on each operator instance on both the source and target hardware to generate a performance profile for each operator instance on both hardware. Based on the substitution semantics of using the target hardware performance profile for operator instances within the counterfactual coalition and using the source hardware performance profile for operator instances outside the coalition, a surrogate function is used to calculate the change in the end-to-end performance estimate of each operator instance before and after joining the counterfactual coalition as a marginal contribution. Finally, statistical processing is used to obtain the performance contribution estimate of each operator instance, thereby attributing the end-to-end performance difference to each operator instance. This solves the technical problem that end-to-end benchmarking schemes cannot provide operator-level explanations, enabling developers to obtain specific quantitative values ​​of the impact of each operator instance on end-to-end performance and form executable optimization priorities. By constructing surrogate functions based on performance profiles to estimate the end-to-end performance of counterfactual federations, these surrogate functions implicitly reflect the impact of contextual factors such as the critical path position of each operator instance in the computation graph, parallel execution relationships, operator fusion, memory contention, and runtime scheduling overhead on end-to-end performance. This solves the technical problem of unreliable attribution results caused by isolated measurements in single-operator micro-benchmarks and ignoring the computation graph context, making the attribution results more consistent with the actual end-to-end migration difference. By constructing surrogate functions to replace real hardware execution to estimate the end-to-end performance of counterfactual federations, and by performing multiple random sorts of all operator instances and adding them one by one in the sorted order to calculate the marginal contribution, the computational complexity is reduced from exponential federation enumeration to polynomial-level surrogate function queries, solving the technical problem that accurate Shapley values ​​require exponential federation queries. Simultaneously, through the substitution semantics of counterfactual federations, counterfactual state estimation at the performance profile level is achieved by "some operators implemented on the target hardware, and the rest on the source hardware," eliminating the need to physically concatenate operator instances from different hardware for execution, thus solving the technical problem that physical concatenation is infeasible in heterogeneous hardware scenarios. By measuring the end-to-end true performance values ​​of the large AI model on the source hardware and the target hardware, and using these end-to-end true performance values ​​to perform endpoint calibration on the proxy function, the output values ​​of the proxy function at the empty federation and the full federation are precisely aligned with the end-to-end true performance values ​​of the source hardware and the target hardware, respectively. This ensures a verifiable reconciliation relationship between the sum of the estimated performance contributions of all operator instances calculated by the proxy function and the end-to-end true performance difference from the source hardware to the target hardware. This solves the technical problem of the lack of a full-process reconciliation mechanism for migration differences from the source to the target in existing technologies, and improves the engineering credibility of the attribution results.By performing multiple random sorts on all operator instances, and adding each operator instance to the counterfactual coalition in the sorting order and calculating its marginal contribution each time, multiple marginal contribution samples are obtained for each operator instance under different arrangement positions. By statistically processing multiple marginal contributions, the performance contribution estimate is obtained, which effectively reduces the estimation bias and variance caused by the operator's position being earlier or later in a single random arrangement. This solves the technical problem of significant differences in marginal contribution of the same operator when it is arranged earlier or later in ordinary Monte Carlo sampling and the instability of attribution ranking under finite sampling budget, thus obtaining a more stable and reliable operator contribution ranking under the same sampling budget. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of the operator-level performance attribution method for large artificial intelligence models in this embodiment of the invention; Figure 2 This is a flowchart illustrating the model diagram normalization in an embodiment of the present invention; Figure 3 This is a flowchart of the confidence analysis in an embodiment of the present invention; Figure 4 This is a structural diagram of the operator-level performance attribution system for a large artificial intelligence model in an embodiment of the present invention; Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0020] Figure 1 This is a flowchart of the operator-level performance attribution method for large artificial intelligence models in this embodiment of the invention, as shown below. Figure 1 As shown, the method includes: Step 101: Perform performance measurements on each operator instance in the computation graph of the large AI model on the source hardware and target hardware respectively, generate performance profiles of each operator instance on the source hardware and target hardware, and measure the end-to-end true performance value of the large AI model on the source hardware and the end-to-end true performance value on the target hardware. Step 102: Construct a counterfactual alliance, which contains any number of operator instances, wherein the operator instances within the counterfactual alliance adopt the performance profile of the target hardware, and the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. Step 103: Construct a proxy function based on the performance profile, and perform endpoint calibration on the proxy function using the end-to-end real performance values ​​to obtain the calibrated proxy function; Step 104: Randomly sort all operator instances multiple times, and add operator instances one by one to the counterfactual coalition in the sorting order each time. Calculate the change in the end-to-end performance estimate of the counterfactual coalition before and after each addition of an operator instance based on the calibrated surrogate function, and use it as the marginal contribution of the added operator instance. Step 105: Perform statistical processing on all marginal contributions of each operator instance to obtain the performance contribution estimate of each operator instance.

[0021] Depend on Figure 1As shown in the process, in this embodiment of the invention, performance measurements are performed on each operator instance on both the source and target hardware to generate a performance profile for each operator instance on both hardware. Based on the substitution semantics of using the target hardware performance profile for operator instances within the counterfactual coalition and using the source hardware performance profile for operator instances outside the coalition, a surrogate function is used to calculate the change in the end-to-end performance estimate of each operator instance before and after joining the counterfactual coalition as a marginal contribution. Finally, the performance contribution estimate of each operator instance is obtained through statistical processing, thereby attributing the end-to-end performance difference to each operator instance. This solves the technical problem that end-to-end benchmarking schemes cannot provide operator-level explanations, enabling developers to obtain specific quantitative values ​​of the impact of each operator instance on end-to-end performance and form executable optimization priorities. By constructing surrogate functions based on performance profiles to estimate the end-to-end performance of counterfactual federations, these surrogate functions implicitly reflect the impact of contextual factors such as the critical path position of each operator instance in the computation graph, parallel execution relationships, operator fusion, memory contention, and runtime scheduling overhead on end-to-end performance. This solves the technical problem of unreliable attribution results caused by isolated measurements in single-operator micro-benchmarks and ignoring the computation graph context, making the attribution results more consistent with the actual end-to-end migration difference. By constructing surrogate functions to replace real hardware execution to estimate the end-to-end performance of counterfactual federations, and by performing multiple random sorts of all operator instances and adding them one by one in the sorted order to calculate the marginal contribution, the computational complexity is reduced from exponential federation enumeration to polynomial-level surrogate function queries, solving the technical problem that accurate Shapley values ​​require exponential federation queries. Simultaneously, through the substitution semantics of counterfactual federations, counterfactual state estimation at the performance profile level is achieved by "some operators implemented on the target hardware, and the rest on the source hardware," eliminating the need to physically concatenate operator instances from different hardware for execution, thus solving the technical problem that physical concatenation is infeasible in heterogeneous hardware scenarios. By measuring the end-to-end true performance values ​​of the large AI model on the source hardware and the target hardware, and using these end-to-end true performance values ​​to perform endpoint calibration on the proxy function, the output values ​​of the proxy function at the empty federation and the full federation are precisely aligned with the end-to-end true performance values ​​of the source hardware and the target hardware, respectively. This ensures a verifiable reconciliation relationship between the sum of the estimated performance contributions of all operator instances calculated by the proxy function and the end-to-end true performance difference from the source hardware to the target hardware. This solves the technical problem of the lack of a full-process reconciliation mechanism for migration differences from the source to the target in existing technologies, and improves the engineering credibility of the attribution results.By performing multiple random sorts on all operator instances, and adding each operator instance to the counterfactual coalition in the sorting order and calculating its marginal contribution each time, multiple marginal contribution samples are obtained for each operator instance under different arrangement positions. By statistically processing multiple marginal contributions, the performance contribution estimate is obtained, which effectively reduces the estimation bias and variance caused by the operator's position being earlier or later in a single random arrangement. This solves the technical problem of significant differences in marginal contribution of the same operator when it is arranged earlier or later in ordinary Monte Carlo sampling and the instability of attribution ranking under finite sampling budget, thus obtaining a more stable and reliable operator contribution ranking under the same sampling budget.

[0022] In step 101, performance measurements are performed on each operator instance in the computation graph of the large artificial intelligence model on the source hardware and the target hardware respectively, generating performance profiles of each operator instance on the source hardware and the target hardware, and measuring the end-to-end true performance value of the large artificial intelligence model on the source hardware and the end-to-end true performance value on the target hardware. In this embodiment of the invention, a large-scale artificial intelligence model It can come from various open-source software or other convertible intermediate representations.

[0023] Figure 2 This is a flowchart of model diagram normalization in an embodiment of the present invention. In one embodiment, the method further includes: Step 201: Before performing performance measurements on each operator instance in the computation graph of the large AI model on the source hardware and target hardware respectively, the large AI model is converted into an intermediate representation format and loaded. If the conversion is successful, the intermediate representation format is loaded to obtain the computation graph; if the conversion fails, the native graph representation of the large AI model is obtained through the native graph extraction interface of the corresponding framework of the large AI model to obtain the computation graph. Step 202: Expand the composite operator in the computation graph into multiple operator instances and split the fusion operator into independent operator instances so that the computation graph maintains a directed acyclic structure; Step 203: Generate a unique identifier for each operator instance and extract the metadata of each operator instance. The metadata includes predecessor-successor relationships, input tensor shape, output tensor shape, and data type.

[0024] In the above embodiments, large-scale artificial intelligence models such as PyTorch, TensorFlow, and ONNX are received. Standardized calculation graph Source hardware and backend Target hardware and backend The performance metrics to be attributed, such as latency, throughput, and energy consumption, and the sampling budget, K.

[0025] First, attempt to convert the large AI model to a unified intermediate representation format, such as ONNX. If the AI ​​model is in PyTorch format, call `torch.onnx.export` and specify the dynamic axis and input sample; if the AI ​​model is in TensorFlow format, convert it using `tf2onnx`; if it is already in ONNX, load it directly. If the conversion fails, fall back to the framework's native graph extraction interface to obtain the native graph representation of the AI ​​model to obtain the computation graph, and record the conversion method for report traceability.

[0026] Composite operators in the computation graph, such as LayerNorm and MultiHeadAttention, are expanded into sequences of atomic operator instances, such as ReduceMean, Sub, Mul, Add, Softmax, and MatMul, ensuring controllable attribution granularity. Simultaneously, known fusion patterns are inversely fused, splitting them into independent operator instances for individual evaluation of each instance's contribution. The expanded computation graph maintains a directed acyclic structure; if loops or control flow exist, symbolic expansion or expansion with a fixed number of iterations is performed based on a fixed input shape.

[0027] For each expanded operator instance, a globally unique identifier is generated, and the following metadata is extracted: original operator name, normalized operator family, input tensor shape, output tensor shape, data type, memory layout, operator parameters, predecessor and successor relationships (including predecessor node list and successor node list), topological sorting index, theoretical FLOPs, and theoretical memory flow. All metadata is stored in an immutable record table R, which serves as the basis for subsequent profiling measurements.

[0028] After obtaining the operator instance, an initial end-to-end real measurement test can be performed: complete model inference is executed on both the source and target hardware, with a warm-up of W times (e.g., ...). ), and then repeat the measurement M times (e.g. Record the delay for each instance and calculate the average delay. , And standard deviation. If the source or target hardware measurement fails, the exception is captured and explicitly marked, the attribution process is terminated, and an error report is output. If both the source and target hardware measurements pass, the migration difference is calculated: .

[0029] In step 102, a counterfactual alliance is constructed, which contains any number of operator instances, wherein the operator instances within the counterfactual alliance adopt the performance profile of the target hardware, and the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. This step involves performing micro-benchmark measurements on both the actual execution shape and the typical shape of each operator instance to generate a source-side profile. Target profile .

[0030] In one embodiment, performance measurements are performed on each operator instance in the computation graph of the large artificial intelligence model on both source and target hardware to generate performance profiles of each operator instance on the source and target hardware, including: For each operator instance, a set of shapes to be tested is generated. The set of shapes to be tested contains multiple shapes. The shapes include the actual shape values ​​of the input tensor shape and the output tensor shape of the operator instance in end-to-end execution, as well as typical shapes supplemented according to the operator family to which the operator instance belongs. For each shape in the set of shapes to be tested, query whether the operator instance has an available implementation under the shape and data type on the source hardware and the target hardware respectively. The operator instances with available implementations, the corresponding shapes, and the corresponding source hardware are formed into test items for the source hardware. At the same time, the operator instances with available implementations, the corresponding shapes, and the corresponding target hardware are formed into test items for the target hardware. For each test item, a performance measurement is performed and the measurement data is recorded. The performance test includes multiple warm-ups and multiple repeated measurements. Based on the measurement data of all test items for each operator instance on the source and target hardware, a performance profile of each operator instance on the source and target hardware is generated.

[0031] Generation of the shape set to be tested: for each operator instance It collects the actual shape values ​​of the input tensor shape and the output tensor shape during end-to-end execution. At the same time, based on its operator family, it supplements a set of typical shapes to cover the typical shapes of possible extrapolation scenarios, forming the test shape set.

[0032] It can be used to implement detection and shape filtering: for each shape in the set of shapes to be tested in China Shape checks are performed on both the source and target hardware to check if the operator instance has an available implementation for that shape and data type. If the target hardware explicitly reports that it is not supported, the corresponding performance profile is marked as unsupported, and specific error information is recorded. This shape will not be used for subsequent proxy fitting.

[0033] Warm-up and Synchronization Timing: For each operator instance, shape, and hardware (including source and target hardware) triple that passes the available implementation detection, as a test item (including test items for source and target hardware), perform the following measurement sequence: Warm-up Next, without recording data, only allowing the cache and hardware scheduler to reach a stable state; calling hardware synchronization functions such as cuda.synchronize() to ensure all pending kernels complete; repeating measurements. Each measurement is synchronized before and after, and a timestamp is recorded. If a measurement times out based on the timestamp, it is considered abnormal and removed, but the number of removals and measurement data must be recorded.

[0034] In one embodiment, the measurement data includes the mean execution delay and the variance of execution delay; Based on the measurement data of all test items for each operator instance on both the source and target hardware, a performance profile of each operator instance on both the source and target hardware is generated, including: Based on the mean and variance of the execution latency of each operator instance in all test items on the source hardware, a performance profile of the operator instance on the source hardware is generated. Based on the mean and variance of the execution latency of each operator instance on the target hardware for all test items, a performance profile of the operator instance on the target hardware is generated. For each operator instance, check whether the effective shape coverage on the source and target hardware meets the preset requirements, and delete the performance profiles that do not meet the preset requirements. The effective shape is the shape that can be implemented in the set of shapes to be tested.

[0035] Online statistics and profile storage: The Welford online algorithm is used to calculate the average execution latency. Execution delay variance Simultaneously, it records environmental information such as the number of valid samples, software version, hardware frequency, and temperature during measurement, ultimately generating a performance profile, including: Performance profile of the source hardware: These are the mean execution latency, variance of execution latency, number of test items, range, and extrapolation marker in the performance profile of the source hardware. Performance profile of the target hardware: These are the mean execution latency, variance of execution latency, number of test items, range, and extrapolation markers in the performance profile of the target hardware.

[0036] Among them, the range and This indicates whether the shape is interpolated within the support domain or extrapolated (marked when the extrapolation distance exceeds a threshold). and If true, it will trigger subsequent supplementary testing.

[0037] Image quality verification: For each operator instance, check whether the effective shape coverage of the performance images of its source hardware and target hardware meets the preset requirements. If it does not meet the requirements, mark it as non-attributable and issue a warning to the user.

[0038] In one embodiment, measuring the end-to-end real performance of the large AI model on the source hardware and the end-to-end real performance on the target hardware includes: The end-to-end inference of the large AI model is executed again on the source hardware, and the end-to-end latency is measured as the end-to-end true performance value of the source hardware. For each operator instance, the sum of the average execution latency of the source hardware in the performance profile is calculated, and the difference between the end-to-end real performance value of the source hardware and the sum of the average execution latency of the source hardware is calculated as the first difference of the operator instance. The end-to-end inference of the AI ​​model is executed again on the target hardware, and the end-to-end latency is measured as the end-to-end true performance value of the target hardware. For each operator instance, the sum of the average execution latency of the target hardware in the performance profile is calculated, and the difference between the end-to-end real performance value of the target hardware and the sum of the average execution latency of the target hardware is calculated as the second difference of the operator instance. If the first or second difference exceeds the first ratio, adjust the number of preheating times and the number of repeated measurements, and then remeasure the end-to-end real performance value of the AI ​​big model on the source hardware and target hardware.

[0039] In the above embodiment, after the profile is constructed, end-to-end inference is performed again to lock the final migration difference. Unlike the initial end-to-end real-world measurement, this measurement will perform a consistency check with the sum of the average execution latency of operator instances in the performance profile: if the sum of the average execution latency of a single operator instance exceeds a certain proportion compared to the end-to-end real-world performance value, it indicates significant runtime overhead or measurement error. The number of warm-up cycles and repeated measurements will be automatically adjusted before a remeasurement, and the difference will be attributed to runtime overhead. .

[0040] Ultimately, the end-to-end real-world performance of the large-scale AI storage model on the source hardware is as follows:

[0041] in, These are the number of repeated measurements and the source hardware, respectively. and backend .

[0042] End-to-end real-world performance of large-scale AI storage models on target hardware:

[0043] in, Do not specify the number of repeated measurements, target hardware, or backend.

[0044] The migration difference is: Used for subsequent residual verification.

[0045] In step 103, a proxy function is constructed based on the performance profile, and the proxy function is calibrated at the endpoints using the end-to-end real performance values ​​to obtain the calibrated proxy function. This step builds a system that can quickly evaluate any counterfactual coalition. proxy function Its core is to integrate computation graph structure, parallelism, runtime overhead and operator profile.

[0046] In one embodiment, constructing a proxy function based on the performance profile includes: For the constructed counterfactual federation, determine the execution time of each operator instance in the counterfactual federation; Dynamic programming is performed in reverse topological order based on the predecessor and successor relationships between the operator instances described in the counterfactual alliance, and the intra-graph execution latency of the computation graph is obtained based on the execution time of each operator instance. The runtime overhead of the air confederation is determined based on the end-to-end true performance value of the source hardware at the air confederation and the in-graph execution latency at the air confederation. The runtime overhead of the full confederation is determined based on the end-to-end true performance value of the target hardware at the full confederation and the in-graph execution latency at the full confederation. The air confederation is a counterfactual confederation that does not contain any operator instances, and the full confederation is a counterfactual confederation that contains all operator instances of the large artificial intelligence model. Based on the runtime overhead at the empty alliance and the runtime overhead at the full alliance, parameter fitting is performed to determine the functional relationship between runtime overhead and the number of operator instances in the counterfactual alliance that adopt the target-side performance profile; A proxy function is determined, the input of which is the counterfactual union, and the output of which is an end-to-end performance estimate, which is the sum of the in-graph execution latency and the runtime overhead determined according to the functional relationship.

[0047] In one embodiment, for the constructed counterfactual coalition, determining the execution time of each operator instance in the counterfactual coalition includes: If an operator instance belongs to the counterfactual alliance, the average execution latency in the performance profile of the target hardware is used as the execution time of the operator instance. If the operator instance does not belong to the counterfactual union, the average execution latency in the performance profile of the source hardware is used as the execution time of the operator instance.

[0048] Critical path calculation: for the constructed counterfactual coalition Define each operator instance The execution time is Its value is: If Then, take the average execution latency of the target hardware performance profile under the corresponding shape. Otherwise, take the average execution latency of the source hardware performance profile under the corresponding shape. Next, starting with the output operator instances of the computation graph, dynamic programming is performed in reverse topological order to calculate the earliest completion time for each operator instance. :

[0049] in express All subsequent operator instances. Finally, output the operator instances. This refers to the in-graph execution latency without considering runtime overhead.

[0050] In one embodiment, dynamic programming is performed according to the predecessor-successor relationships between the operator instances in the counterfactual alliance in reverse topological order, and the intra-graph execution latency of the computation graph is obtained based on the execution duration of each operator instance, including: Based on the predecessor-successor relationship between each operator instance, all predefined operator groups and all parallel structures in the computation graph are determined, and each parallel structure includes multiple parallel branches. For each operator group, if the target hardware supports the fusion mode of the operator group, then the operator group is regarded as a fusion group, and the execution time of the fusion group is determined to be the single execution time of the fusion core of the fusion group. For each parallel structure, the completion time of each parallel branch of the parallel structure is determined as the earliest completion time of the terminal operator instance of the parallel branch, and the maximum value of the completion time of all parallel branches in the parallel structure is taken as the execution time of the parallel structure. All fusion groups, parallel structures, and individual operator instances in the computation graph are treated as execution units. Based on the execution time of each execution unit and the earliest completion time of all subsequent execution units, the earliest completion time of each execution unit is calculated in reverse topological order. The earliest completion time of the output execution unit with no subsequent execution unit in the computation graph is taken as the execution delay within the graph.

[0051] Merging groups and parallel branch correction: For each operator group, if the target hardware supports the fusion mode of the operator group, then the operator group is regarded as a fusion group, and the execution time of the fusion group is determined to be the single execution time of the fusion core of the fusion group; For each parallel structure, the completion time of each parallel branch of the parallel structure is determined to be the earliest completion time of the terminal operator instance of the parallel branch, and the maximum value of the completion time of all parallel branches in the parallel structure is taken as the execution time of the parallel structure.

[0052] In addition to fusion and parallel branch correction, a memory bandwidth contention correction term can be added to detect operator groups that simultaneously access shared memory bandwidth and increase bandwidth contention latency correction.

[0053] Runtime overhead modeling: runtime overhead This includes framework scheduling, memory copying, data type conversion, and kernel startup synchronization. A segmented fitting approach is used: first, end-to-end real performance values ​​are utilized. and Back-calculation of runtime overhead:

[0054] Then, assuming The number of operator instances in the counterfactual alliance The parameters are fitted to determine whether the relationship is linear or piecewise linear. Additional data transformation costs are incurred based on... The number of operator instances involving cross-precision operations plus a penalty term.

[0055] Surrogate function calibration: The fitted function is then calibrated. and The calculations are combined to obtain the complete proxy function. The fitting parameters are retained, and the fitting residuals are recorded.

[0056] In one embodiment, endpoint calibration of the proxy function is performed using the end-to-end real performance values ​​to obtain a calibrated proxy function, including: The affine transformation parameters are determined based on the end-to-end true performance values ​​of the source hardware in the air-alliance and the end-to-end true performance values ​​of the target hardware in the entire alliance. The output value of the surrogate function is transformed using the affine transformation parameters so that the output value of the calibrated surrogate function at the air-alliance is equal to the end-to-end true performance value of the source hardware, and the output value at the entire alliance is equal to the end-to-end true performance value of the target hardware. Check whether the input tensor shape and output tensor shape of each operator instance are within the range of the corresponding performance profile; If the proportion of operator instances outside the range exceeds a preset ratio, a new performance profile will be generated for the operator instances outside the range.

[0057] Force endpoint matching: Set directly and Affine transformation can be used. ,in , The affine transformation parameters are determined by the end-to-end true performance values ​​of the source hardware in the air-coalition and the end-to-end true performance values ​​of the target hardware across the entire coalition. However, the original records are... And affine transformation parameters, to ensure traceability.

[0058] Range consistency check: Traverse all operator instances and check whether their input shapes fall within the range of the performance profile. If the proportion of operator instances outside the range exceeds a preset ratio, add test items for the out-of-range operator instances and regenerate the performance profile.

[0059] In one embodiment, after obtaining the calibrated surrogate function, the method further includes: Randomly select at least one counterfactual alliance; For each extracted counterfactual coalition, the end-to-end true performance value of each operator instance in the counterfactual coalition is measured. Specifically, the operator instances belonging to the counterfactual coalition are measured independently on the target hardware to obtain a first measurement value, and the operator instances not belonging to the counterfactual coalition are measured independently on the source hardware to obtain a second measurement value. The first measurement value and the second measurement value are summed to obtain an approximate end-to-end performance value of the counterfactual coalition. The calibrated proxy function is compared with the output value of the counterfactual alliance to obtain the relative error by comparing it with the end-to-end performance approximation. If the average relative error exceeds a preset threshold, the calibrated surrogate function is recalibrated using the end-to-end real performance values ​​of each operator instance on the source and target hardware.

[0060] Random sampling verification: A counterfactual alliance ( (But no more than 50). These counterfactual coalesces can be run through actual hardware. For the feasible parts, a virtual splicing method is used, measuring only the independent execution and accumulation of the operator instances within the counterfactual coalesce. However, it should be noted that this is only a rough check, or more robustly, an approximation can be made by modifying the model on a single backend. If the average relative error between the surrogate estimate and the true value exceeds a certain threshold in the sampling, a full retesting process is triggered, i.e., the calibrated surrogate function is recalibrated.

[0061] In step 104, all operator instances are randomly sorted multiple times. Each time, operator instances are added to the counterfactual alliance one by one according to the sorting order. The change in the end-to-end performance estimate of the counterfactual alliance before and after each addition of an operator instance is calculated based on the calibrated surrogate function and used as the marginal contribution of the added operator instance. This step is the core attribution calculation stage. It utilizes the calibrated surrogate function. and sampling budget For each operator, a Monte Carlo estimate of its marginal contribution is calculated.

[0062] In one embodiment, all operator instances are randomly sorted multiple times, and each time an operator instance is added to the counterfactual federation in the sorted order. The change in the end-to-end performance estimate of the counterfactual federation before and after each addition of an operator instance is calculated based on the calibrated surrogate function and used as the marginal contribution of the added operator instance, including: Perform multiple random sorts on all operator instances, generating a sorted sequence each time, and simultaneously constructing the reverse sequence of the sorted sequence. Treat each sorted sequence and its inversion as independent sequences, and perform the following operations on each: For each independent sequence, the empty union is taken as the current counterfactual union, and operator instances are added to the current counterfactual union one by one in the order of the independent sequence; The calibrated proxy function is invoked to calculate the end-to-end performance estimate of the current counterfactual coalition; The difference between the end-to-end performance estimates of the operator instance before and after it is added to the current counterfactual coalition is taken as the marginal contribution of the added operator instance.

[0063] Permutation generation and inversion construction: Repeat the following process Next, among them The number is even to ensure that each random permutation appears in reverse order: from A uniformly random sample of length is taken from the middle. sorted sequence And directly construct its reverse sequence The two permutations share the cost of generating a random number once. For the same operator, in When in the middle or near the front, The middle group will inevitably be at the bottom, and the mean of its two marginal contributions can offset part of the variance caused by the size of the alliance.

[0064] Increasing alliances and marginal contribution calculation: for each independent sequence Initialize the empty alliance The output value of the calibrated proxy function (i.e., the end-to-end performance estimate). ;according to The operators are traversed in sequence. Calculate the output value of the calibrated surrogate function after the addition (i.e., the end-to-end performance estimate). Marginal contribution Add to operator Contribution list ;renew , Marginal contribution list The updates are incremental, eliminating the need to store all permutation results. For ultra-large computational graphs with more than 2000 operators, a hierarchical sampling strategy can be adopted: first, the graph is layered by operator family or subgraph module, and the permutation budget is evenly distributed between layers. Within each layer, the permutation budget is weighted according to the number of operators, ensuring that each operator has sufficient contribution estimation samples.

[0065] In one embodiment, statistical processing is performed on all marginal contributions of each operator instance to obtain an estimated performance contribution value for each operator instance, including: The mean of all marginal contributions for each operator instance is used as the performance contribution estimate for each operator instance.

[0066] Statistical calculation: After the traversal is complete, for each operator instance... Total A marginal contribution, that is For each permutation, two samples are provided. Calculate: mean. Standard deviation Standard error 95% confidence interval .

[0067] Verification of efficiency properties: Due to the efficiency properties of the Shapley value This holds true under the calibrated proxy function. Because a calibrated proxy function is used here, it may... and There is a slight deviation, therefore the calculation As a residual. If If the threshold is exceeded, a supplementary test will be triggered.

[0068] In step 105, statistical processing is performed on all marginal contributions of each operator instance to obtain the performance contribution estimate of each operator instance.

[0069] Figure 3 This is a flowchart of confidence analysis in an embodiment of the present invention. In one embodiment, after obtaining the performance contribution estimate of each operator instance, the method further includes: Step 301: Calculate the standard error of all marginal contributions for each operator instance; Step 302: Construct a confidence interval for each operator instance based on the performance contribution estimate and the standard error; Step 303: For each operator instance, if the confidence interval of the operator instance contains a zero value, then the operator instance is marked as having an uncertain contribution direction; if the performance profile of the operator instance has an extrapolation mark, then the operator instance is marked as requiring supplementary measurement; if the standard error of the operator instance exceeds a predetermined proportion of the performance contribution estimate, then the operator instance is marked as having high variance; the uncertain contribution direction and high variance are used to represent the confidence level of the corresponding operator instance.

[0070] In the above embodiments, the uncertainty in the contribution direction and high variance indicate to the user that the attribution results of the operator instance are unreliable. The predetermined proportion of the performance contribution estimate can be set to 50%.

[0071] In one embodiment, the method further includes: The retest task will be triggered when any of the following conditions are met: There are operator instances that are marked as needing supplementary measurements and whose absolute value of the performance contribution estimate ranks in the top preset order. The type of the supplementary measurement task is marked as shape extrapolation. If the deviation between the output value of the calibrated proxy function at the calibration endpoint and the corresponding end-to-end true performance value exceeds the deviation threshold, the type of the supplementary test task is marked as the endpoint residual. The residual between the sum of the performance contribution estimates of all operator instances and the migration difference exceeds the residual threshold. The migration difference is the difference between the end-to-end true performance value of the AI ​​big model on the source hardware and the end-to-end true performance value on the target hardware. The type of the supplementary test task is marked as the efficiency residual. If the counterfactual alliance fails to perform a sampling review, the type of the retest task is marked as sampling review. The sampling review steps are as follows: input the counterfactual alliance into the calibrated surrogate function. If the deviation between the obtained end-to-end performance estimate and the end-to-end true performance value of the counterfactual alliance exceeds a preset threshold, the sampling review fails.

[0072] In the above embodiments, when the type of the supplementary measurement task is endpoint residual, it can be represented as follows: Or similar; when the type of supplementary testing task is efficiency residual, it can be expressed as The absolute value of the performance contribution estimate ranks in the top 20% of the preset order.

[0073] In one embodiment, the method further includes: If the type of supplementary testing task is shape extrapolation, update the performance profile of the operator instance on the target hardware; If the type of supplementary testing task is endpoint residual, then remeasure the end-to-end true performance value of the AI ​​large model on the source hardware and the end-to-end true performance value on the target hardware, and recalibrate the surrogate function. If the type of retest task is efficiency residual or sampling review, then randomly sample multiple additional counterfactual coalitions to reconstruct the surrogate function.

[0074] The retest task is submitted to the hardware executor. Upon successful execution, the profile is written back, and the incremental recalculation following the retest is triggered. After recalculation, the residuals are checked again. If they are still unacceptable, a warning is issued and the result is marked as unreliable.

[0075] The final report is generated, which sorts operator instances in descending order of absolute contribution and outputs a unique identifier for each operator instance. , 95% confidence interval, performance profile, predecessor and successor relationships and their critical paths, supplementary testing history, range and confidence level marking.

[0076] The report also includes a summary section: , , Total number of queries ( Sub-proxy queries + retest counts) and speedup ratio (relative to) (Enumeration). The report format supports both JSON structured output and PDF visual report. The JSON format facilitates downstream toolchain integration and contains complete attribution records and measurement references for each operator; the PDF format includes contribution ranking plots, critical path highlighting plots, and residual audit summaries for easy manual review. When the efficiency residual is unqualified, a red warning icon is displayed at the beginning of the report, clearly informing the user that the confidence of the current attribution result is insufficient and that additional measurements are required for recalculation.

[0077] In this embodiment of the invention, performance indicators Besides latency, it can be replaced with throughput, energy consumption, peak memory, cost, or a multi-metric weighted function. The surrogate function can employ longest-path dynamic programming, queuing networks, discrete-event simulation, graph neural networks, or a hybrid of both. Dual sampling can be combined with hierarchical sampling, importance sampling, quasi-Monte Carlo, or adaptive stopping rules. Calibration of the surrogate function can employ affine, piecewise linear, Bayesian calibration, or constrained regression. Real-world testing can also be triggered by a local task queue, experiment orchestrator, or manual confirmation; all thresholds can be configured according to hardware category. Operator instances can be atomic kernels, fusion operators, subgraphs, or pipeline stages, as long as they have stable identities and explicit substitution semantics.

[0078] This invention also provides an operator-level performance attribution system for large-scale artificial intelligence models, as described in the following embodiments. Since the principle behind this system's problem-solving is similar to that of the operator-level performance attribution method for large-scale artificial intelligence models, the implementation of this system can refer to the implementation of the operator-level performance attribution method for large-scale artificial intelligence models; repeated details will not be elaborated further.

[0079] Figure 4 This is a structural diagram of the operator-level performance attribution system for a large artificial intelligence model in an embodiment of the present invention, as shown below. Figure 4 As shown, the system includes: The dual-end profiling module 401 is used to perform performance measurements on each operator instance in the computation graph of the large artificial intelligence model on the source hardware and the target hardware respectively, generate performance profiles of each operator instance on the source hardware and the target hardware, and measure the end-to-end true performance value of the large artificial intelligence model on the source hardware and the end-to-end true performance value on the target hardware. The counterfactual alliance generation module 402 is used to construct a counterfactual alliance, which contains any number of operator instances, wherein the operator instances within the counterfactual alliance adopt the performance profile of the target hardware, and the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. The proxy function construction module 403 is used to construct a proxy function based on the performance profile, perform endpoint calibration on the proxy function using the end-to-end real performance value, and obtain a calibrated proxy function. The marginal contribution calculation module 404 is used to randomly sort all operator instances multiple times, and add operator instances one by one to the counterfactual alliance in the sorting order each time. Based on the calibrated surrogate function, it calculates the change in the end-to-end performance estimate of the counterfactual alliance before and after each addition of an operator instance, and uses it as the marginal contribution of the added operator instance. The performance contribution estimation module 405 is used to perform statistical processing on all marginal contributions of each operator instance to obtain the performance contribution estimate of each operator instance.

[0080] The system proposed in this embodiment is recommended to be deployed on computing nodes equipped with high-performance GPU servers. The dual-end profiling module can be dynamically scheduled to the corresponding physical device based on the source and target hardware types. All modules are encapsulated as independent executable components and exchange data through standardized interfaces to ensure the system's scalability and maintainability. The hardware adapter uniformly performs capability detection, warm-up, synchronization timing, anomaly capture, reset, and raw trace export.

[0081] In one embodiment, the system further includes a model graph normalization module, used for: Before performing performance measurements on each operator instance in the computation graph of the large AI model on the source and target hardware respectively, the large AI model is converted into an intermediate representation format and loaded. If the conversion is successful, the intermediate representation format is loaded to obtain the computation graph; if the conversion fails, the native graph representation of the large AI model is obtained through the native graph extraction interface of the corresponding framework of the large AI model to obtain the computation graph. The composite operator in the computation graph is expanded into multiple operator instances, and the fusion operator is split into independent operator instances, so that the computation graph maintains a directed acyclic structure. A unique identifier is generated for each operator instance, and the metadata of each operator instance is extracted. The metadata includes predecessor and successor relationships, input tensor shape, output tensor shape, and data type.

[0082] In one embodiment, the dual-end image module is used for: For each operator instance, a set of shapes to be tested is generated. The set of shapes to be tested contains multiple shapes. The shapes include the actual shape values ​​of the input tensor shape and the output tensor shape of the operator instance in end-to-end execution, as well as typical shapes supplemented according to the operator family to which the operator instance belongs. For each shape in the set of shapes to be tested, query whether the operator instance has an available implementation under the shape and data type on the source hardware and the target hardware respectively. The operator instances with available implementations, the corresponding shapes, and the corresponding source hardware are formed into test items for the source hardware. At the same time, the operator instances with available implementations, the corresponding shapes, and the corresponding target hardware are formed into test items for the target hardware. For each test item, a performance measurement is performed and the measurement data is recorded. The performance test includes multiple warm-ups and multiple repeated measurements. Based on the measurement data of all test items for each operator instance on the source and target hardware, a performance profile of each operator instance on the source and target hardware is generated.

[0083] In one embodiment, the measurement data includes the mean execution delay and the variance of execution delay; The dual-end portrait module is used for: Based on the mean and variance of the execution latency of each operator instance in all test items on the source hardware, a performance profile of the operator instance on the source hardware is generated. Based on the mean and variance of the execution latency of each operator instance on the target hardware for all test items, a performance profile of the operator instance on the target hardware is generated. For each operator instance, check whether the effective shape coverage on the source and target hardware meets the preset requirements, and delete the performance profiles that do not meet the preset requirements. The effective shape is the shape that can be implemented in the set of shapes to be tested.

[0084] In one embodiment, the dual-end image module is used for: The end-to-end inference of the large AI model is executed again on the source hardware, and the end-to-end latency is measured as the end-to-end true performance value of the source hardware. For each operator instance, the sum of the average execution latency of the source hardware in the performance profile is calculated, and the difference between the end-to-end real performance value of the source hardware and the sum of the average execution latency of the source hardware is calculated as the first difference of the operator instance. The end-to-end inference of the AI ​​model is executed again on the target hardware, and the end-to-end latency is measured as the end-to-end true performance value of the target hardware. For each operator instance, the sum of the average execution latency of the target hardware in the performance profile is calculated, and the difference between the end-to-end real performance value of the target hardware and the sum of the average execution latency of the target hardware is calculated as the second difference of the operator instance. If the first or second difference exceeds the first ratio, adjust the number of preheating times and the number of repeated measurements, and then remeasure the end-to-end real performance value of the AI ​​big model on the source hardware and target hardware.

[0085] In one embodiment, the proxy function construction module is used to: Constructing proxy functions based on the performance profile includes: For the constructed counterfactual federation, determine the execution time of each operator instance in the counterfactual federation; Dynamic programming is performed in reverse topological order based on the predecessor and successor relationships between the operator instances described in the counterfactual alliance, and the intra-graph execution latency of the computation graph is obtained based on the execution time of each operator instance. The runtime overhead of the air confederation is determined based on the end-to-end true performance value of the source hardware at the air confederation and the in-graph execution latency at the air confederation. The runtime overhead of the full confederation is determined based on the end-to-end true performance value of the target hardware at the full confederation and the in-graph execution latency at the full confederation. The air confederation is a counterfactual confederation that does not contain any operator instances, and the full confederation is a counterfactual confederation that contains all operator instances of the large artificial intelligence model. Based on the runtime overhead at the empty alliance and the runtime overhead at the full alliance, parameter fitting is performed to determine the functional relationship between runtime overhead and the number of operator instances in the counterfactual alliance that adopt the target-side performance profile; A proxy function is determined, the input of which is the counterfactual union, and the output of which is an end-to-end performance estimate, which is the sum of the in-graph execution latency and the runtime overhead determined according to the functional relationship.

[0086] In one embodiment, the proxy function construction module is used to: If an operator instance belongs to the counterfactual alliance, the average execution latency in the performance profile of the target hardware is used as the execution time of the operator instance. If the operator instance does not belong to the counterfactual union, the average execution latency in the performance profile of the source hardware is used as the execution time of the operator instance.

[0087] In one embodiment, the proxy function construction module is used to: Based on the predecessor-successor relationship between each operator instance, all predefined operator groups and all parallel structures in the computation graph are determined, and each parallel structure includes multiple parallel branches. For each operator group, if the target hardware supports the fusion mode of the operator group, then the operator group is regarded as a fusion group, and the execution time of the fusion group is determined to be the single execution time of the fusion core of the fusion group. For each parallel structure, the completion time of each parallel branch of the parallel structure is determined as the earliest completion time of the terminal operator instance of the parallel branch, and the maximum value of the completion time of all parallel branches in the parallel structure is taken as the execution time of the parallel structure. All fusion groups, parallel structures, and individual operator instances in the computation graph are treated as execution units. Based on the execution time of each execution unit and the earliest completion time of all subsequent execution units, the earliest completion time of each execution unit is calculated in reverse topological order. The earliest completion time of the output execution unit with no subsequent execution unit in the computation graph is taken as the execution delay within the graph.

[0088] In one embodiment, the proxy function construction module is used to: The affine transformation parameters are determined based on the end-to-end true performance values ​​of the source hardware in the air-alliance and the end-to-end true performance values ​​of the target hardware in the entire alliance. The output value of the surrogate function is transformed using the affine transformation parameters so that the output value of the calibrated surrogate function at the air-alliance is equal to the end-to-end true performance value of the source hardware, and the output value at the entire alliance is equal to the end-to-end true performance value of the target hardware. Check whether the input tensor shape and output tensor shape of each operator instance are within the range of the corresponding performance profile; If the proportion of operator instances outside the range exceeds a preset ratio, a new performance profile will be generated for the operator instances outside the range.

[0089] In one embodiment, the proxy function construction module is used to: Randomly select at least one counterfactual alliance; For each extracted counterfactual coalition, the end-to-end true performance value of each operator instance in the counterfactual coalition is measured. Specifically, the operator instances belonging to the counterfactual coalition are measured independently on the target hardware to obtain a first measurement value, and the operator instances not belonging to the counterfactual coalition are measured independently on the source hardware to obtain a second measurement value. The first measurement value and the second measurement value are summed to obtain an approximate end-to-end performance value of the counterfactual coalition. The calibrated proxy function is compared with the output value of the counterfactual alliance to obtain the relative error by comparing it with the end-to-end performance approximation. If the average relative error exceeds a preset threshold, the calibrated surrogate function is recalibrated using the end-to-end real performance values ​​of each operator instance on the source and target hardware.

[0090] In one embodiment, the marginal contribution calculation module is used for: Perform multiple random sorts on all operator instances, generating a sorted sequence each time, and simultaneously constructing the reverse sequence of the sorted sequence. Treat each sorted sequence and its inversion as independent sequences, and perform the following operations on each: For each independent sequence, the empty union is taken as the current counterfactual union, and operator instances are added to the current counterfactual union one by one in the order of the independent sequence; The calibrated proxy function is invoked to calculate the end-to-end performance estimate of the current counterfactual coalition; The difference between the end-to-end performance estimates of the operator instance before and after it is added to the current counterfactual coalition is taken as the marginal contribution of the added operator instance.

[0091] In one embodiment, the performance contribution estimation module is used for: The mean of all marginal contributions for each operator instance is used as the performance contribution estimate for each operator instance.

[0092] In one embodiment, the system further includes a calibration and retest module, used for: After obtaining the performance contribution estimate for each operator instance, calculate the standard error of all marginal contributions for each operator instance; Confidence intervals for each operator instance are constructed based on the performance contribution estimate and the standard error; For each operator instance, if the confidence interval of the operator instance contains zero, the operator instance is marked as having an uncertain contribution direction; if the performance profile of the operator instance has an extrapolation label, the operator instance is marked as requiring supplementary measurement; if the standard error of the operator instance exceeds a predetermined proportion of the performance contribution estimate, the operator instance is marked as having high variance; the uncertainty in contribution direction and high variance are used to represent the confidence level of the corresponding operator instance.

[0093] In one embodiment, the calibration and supplementary testing module is used for: The retest task will be triggered when any of the following conditions are met: There are operator instances that are marked as needing supplementary measurements and whose absolute value of the performance contribution estimate ranks in the top preset order. The type of the supplementary measurement task is marked as shape extrapolation. If the deviation between the output value of the calibrated proxy function at the calibration endpoint and the corresponding end-to-end true performance value exceeds the deviation threshold, the type of the supplementary test task is marked as the endpoint residual. The residual between the sum of the performance contribution estimates of all operator instances and the migration difference exceeds the residual threshold. The migration difference is the difference between the end-to-end true performance value of the AI ​​big model on the source hardware and the end-to-end true performance value on the target hardware. The type of the supplementary test task is marked as the efficiency residual. If the counterfactual alliance fails to perform a sampling review, the type of the retest task is marked as sampling review. The sampling review steps are as follows: input the counterfactual alliance into the calibrated surrogate function. If the deviation between the obtained end-to-end performance estimate and the end-to-end true performance value of the counterfactual alliance exceeds a preset threshold, the sampling review fails.

[0094] In one embodiment, the calibration and supplementary testing module is used for: If the type of supplementary testing task is shape extrapolation, update the performance profile of the operator instance on the target hardware; If the type of supplementary testing task is endpoint residual, then remeasure the end-to-end true performance value of the AI ​​large model on the source hardware and the end-to-end true performance value on the target hardware, and recalibrate the surrogate function. If the type of retest task is efficiency residual or sampling review, then randomly sample multiple additional counterfactual coalitions to reconstruct the surrogate function.

[0095] This invention also provides a computer device. Figure 5 This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, it implements an operator-level performance attribution method for a large artificial intelligence model.

[0096] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the operator-level performance attribution method for the aforementioned large-scale artificial intelligence model.

[0097] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the operator-level performance attribution method of the aforementioned large-scale artificial intelligence model.

[0098] The effects achieved by the method and system proposed in the embodiments of the present invention are as follows: By performing performance measurements on each operator instance on both source and target hardware, a performance profile of each operator instance is generated on both hardware. Based on the substitution semantics of using the target hardware performance profile for operators within the counterfactual coalition and using the source hardware performance profile for operators outside the coalition, a surrogate function is used to calculate the change in end-to-end performance estimate before and after joining the counterfactual coalition as the marginal contribution. Finally, the performance contribution estimate of each operator instance is obtained through statistical processing, thereby attributing the end-to-end performance difference to each operator instance. This solves the technical problem that end-to-end benchmarking schemes cannot provide operator-level explanations, enabling developers to obtain specific quantitative values ​​of the impact of each operator instance on end-to-end performance and form executable optimization priorities. By constructing surrogate functions based on performance profiles to estimate the end-to-end performance of counterfactual federations, these surrogate functions implicitly reflect the impact of contextual factors such as the critical path position of each operator instance in the computation graph, parallel execution relationships, operator fusion, memory contention, and runtime scheduling overhead on end-to-end performance. This solves the technical problem of unreliable attribution results caused by isolated measurements in single-operator micro-benchmarks and ignoring the computation graph context, making the attribution results more consistent with the actual end-to-end migration difference. By constructing surrogate functions to replace real hardware execution to estimate the end-to-end performance of counterfactual federations, and by performing multiple random sorts of all operator instances and adding them one by one in the sorted order to calculate the marginal contribution, the computational complexity is reduced from exponential federation enumeration to polynomial-level surrogate function queries, solving the technical problem that accurate Shapley values ​​require exponential federation queries. Simultaneously, through the substitution semantics of counterfactual federations, counterfactual state estimation at the performance profile level is achieved by "some operators implemented on the target hardware, and the rest on the source hardware," eliminating the need to physically concatenate operator instances from different hardware for execution, thus solving the technical problem that physical concatenation is infeasible in heterogeneous hardware scenarios. By measuring the end-to-end true performance values ​​of the large AI model on the source hardware and the target hardware, and using these end-to-end true performance values ​​to perform endpoint calibration on the proxy function, the output values ​​of the proxy function at the empty federation and the full federation are precisely aligned with the end-to-end true performance values ​​of the source hardware and the target hardware, respectively. This ensures a verifiable reconciliation relationship between the sum of the estimated performance contributions of all operator instances calculated by the proxy function and the end-to-end true performance difference from the source hardware to the target hardware. This solves the technical problem of the lack of a full-process reconciliation mechanism for migration differences from the source to the target in existing technologies, and improves the engineering credibility of the attribution results.By performing multiple random sorts on all operator instances, and adding each operator instance to the counterfactual coalition in the sorting order and calculating its marginal contribution each time, multiple marginal contribution samples are obtained for each operator instance under different arrangement positions. By statistically processing multiple marginal contributions, the performance contribution estimate is obtained, which effectively reduces the estimation bias and variance caused by the operator's position being earlier or later in a single random arrangement. This solves the technical problem of significant differences in marginal contribution of the same operator when it is arranged earlier or later in ordinary Monte Carlo sampling and the instability of attribution ranking under finite sampling budget, thus obtaining a more stable and reliable operator contribution ranking under the same sampling budget.

[0099] Before performing performance measurements on each operator instance in the computation graph of the large AI model on both the source and target hardware, the large AI model is converted into an intermediate representation format and loaded to obtain the computation graph, or the native graph representation is obtained through a native graph extraction interface to obtain the computation graph. This achieves unified graph parsing across frameworks, enabling the invention to be compatible with models from different sources. By expanding composite operators in the computation graph into multiple operator instances and splitting fusion operators into independent operator instances, the computation graph maintains a directed acyclic structure, ensuring the controllability of attribution granularity and the feasibility of subsequent dynamic programming. By generating a unique identifier for each operator instance and extracting metadata, each operator instance can be uniquely traced, laying the foundation for accurate positioning of subsequent performance profiling and attribution results.

[0100] By generating a set of test shapes containing real and typical shapes for each operator instance, available implementations are queried on both the source and target hardware to form test items. Performance measurements are performed on each test item and the measurement data is recorded. Based on the measurement data of all test items, a performance profile of each operator instance on the source and target hardware is generated, so that the performance profile can cover the real shape of the operator instance in actual inference and possible extrapolation scenarios, ensuring the data completeness of subsequent proxy function construction and endpoint calibration.

[0101] By obtaining the mean and variance of execution latency from the measurement data, a performance profile of the source hardware is generated based on the mean and variance of the execution latency of all test items. Similarly, a performance profile of the target hardware is generated based on the mean and variance of all test items. This ensures that the performance profile includes both central trend and dispersion information, providing data support for the subsequent uncertainty quantification of the surrogate function. By checking whether the effective shape coverage of each operator instance meets the preset requirements and deleting performance profiles that do not meet the requirements, it is ensured that all operator instances participating in the attribution have sufficient measurement data support, avoiding unreliable attribution due to insufficient data.

[0102] By performing end-to-end inference and measuring end-to-end latency again after generating the performance profile, the sum of the average execution latency in the performance profile is compared with the actual end-to-end performance value. When the deviation exceeds the proportion, the number of warm-up times and repeated measurements are adjusted and remeasured, ensuring the accuracy and repeatability of the actual end-to-end performance value. When the sum of the execution latency of a single operator is inconsistent with the actual end-to-end performance value, the difference is attributed to runtime overhead, providing a physical basis for modeling runtime overhead items in the proxy function.

[0103] By determining the execution time of each operator instance in the counterfactual federation, dynamic programming is performed according to the predecessor-successor relationship in reverse topological order to obtain the in-graph execution latency. The runtime overhead is determined by the end-to-end real performance values ​​at the empty federation and the full federation, and the functional relationship between the overhead and the number of operators on the target side is fitted. A surrogate function is constructed with the counterfactual federation as input and the sum of the in-graph execution latency and runtime overhead as output. This allows the surrogate function to estimate the end-to-end performance of any counterfactual federation in polynomial time without performing real hardware measurements or physical splicing for each federation. This solves the technical problems of requiring exponential federation queries for accurate Shapley values ​​and the infeasibility of physical splicing in heterogeneous scenarios.

[0104] By taking the average execution latency from the target hardware performance profile for operator instances belonging to the counterfactual coalition and taking the average execution latency from the source hardware performance profile for operator instances not belonging to the counterfactual coalition, the specific implementation of counterfactual substitution semantics in the proxy function is clarified, ensuring that the proxy function has a clear causal explanation for the performance estimation of any counterfactual coalition.

[0105] By determining the operator group and parallel structure based on the predecessor-successor relationship, the operator group is used as the fusion group for the fusion mode supported by the target hardware, and the single execution time of its fusion core is determined. For the parallel structure, the maximum value of the completion time of each parallel branch is taken as the execution time of the parallel structure. The fusion group, parallel structure and individual operator instances are used as execution units for dynamic planning, so that the in-graph execution delay calculation of the proxy function can reflect the actual operator fusion optimization and parallel execution characteristics of the target hardware. This avoids the estimation bias caused by simply adding the delays of multiple operators in the fusion group or repeatedly accumulating the delays of parallel branches, and further improves the accuracy of the proxy function.

[0106] By determining the affine transformation parameters based on the end-to-end true performance values ​​of the source hardware in the air-coalition and the target hardware in the full-coalition, and performing endpoint calibration on the surrogate function, the calibrated surrogate function is made to be precisely consistent with the true measurement values ​​at both the air-coalition and full-coalition, ensuring the unbiasedness of the surrogate function at both ends. By checking whether the shape of each operator instance is within the range of the performance profile, and regenerating the performance profile when it exceeds the scale, the effective coverage of the surrogate function in its input space is ensured, avoiding unreliable estimation in the extrapolation state.

[0107] By randomly sampling counterfactual federations and performing real hardware measurements after surrogate function calibration, operator instances belonging to the federation are measured independently on the target hardware, and operator instances not belonging to the federation are measured independently on the source hardware, and the approximate values ​​are obtained by summing them. The relative error is compared with the output value of the surrogate function. When the error exceeds the threshold, the surrogate function is recalibrated. This achieves sampling verification of the accuracy of the surrogate function in the intermediate federation region, further ensuring the reliability of the surrogate function in the entire federation space.

[0108] By randomly sorting all operator instances multiple times, generating a sorted sequence and simultaneously constructing its inversion sequence each time, and treating each sorted sequence and its inversion sequence as independent sequences, operator instances are added sequentially starting from an empty coalition. The difference between the end-to-end performance estimates is calculated as the marginal contribution by calling the calibrated surrogate function. This makes the positions of the same operator instance in the sorted and inversion sequences completely opposite, and its two marginal contributions can offset the estimation bias caused by coalition size and arrangement position, reducing the estimation variance under finite sampling budget and improving the stability of attribution sorting.

[0109] By using the mean of all marginal contributions of each operator instance as a performance contribution estimate, a quantitative assessment of the contribution of each operator instance is achieved, enabling developers to obtain specific numerical values ​​of the end-to-end performance impact of each operator instance and form an executable optimization priority ranking.

[0110] After obtaining the performance contribution estimate, the standard error and confidence interval are calculated. Operator instances with confidence intervals containing zero values ​​are marked as having uncertain contribution directions, those with extrapolation marks are marked as requiring supplementary measurements, and those with standard errors exceeding a predetermined proportion are marked as having high variance. This provides a multi-dimensional confidence assessment of the attribution results for each operator instance, enabling users to identify attribution results with insufficient reliability and treat them with caution.

[0111] By triggering supplementary testing tasks when there are operator instances that require supplementary measurement and rank among the top contributors, when endpoint deviation exceeds a threshold, when efficiency residual exceeds a threshold, or when sampling verification fails, a multi-layered quality control defense line is constructed. When the surrogate model evidence is insufficient, real hardware measurement is proactively triggered instead of forcibly giving a definitive conclusion, thus solving the technical problem of existing technologies lacking explicit rejection and supplementary testing mechanisms.

[0112] By performing actions such as updating the performance profile of the operator instance, re-measuring the end-to-end true performance value and recalibrating the surrogate function, and randomly sampling additional counterfactual coalitions to rebuild the surrogate function according to the type of supplementary testing, the supplementary testing task can take corresponding corrective measures for different triggering reasons. The supplementary testing results are written back to the profile and trigger incremental recalculation. If the result is still unqualified after recalculation, a warning is issued and the result is marked as unreliable, thus forming a complete evidence traceability closed loop.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for attributing operator-level performance of a large-scale artificial intelligence model, characterized in that, include: Performance measurements are performed on each operator instance in the computation graph of the large AI model on both the source and target hardware. Performance profiles of each operator instance are generated on the source and target hardware. The end-to-end true performance values ​​of the large AI model on the source hardware and the end-to-end true performance values ​​on the target hardware are measured. Construct a counterfactual alliance, which contains any number of operator instances, wherein the operator instances within the counterfactual alliance adopt the performance profile of the target hardware, and the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. A proxy function is constructed based on the performance profile, and the end-to-end real performance value is used to perform endpoint calibration on the proxy function to obtain the calibrated proxy function. All operator instances are randomly sorted multiple times. Each time, operator instances are added to the counterfactual coalition one by one according to the sorting order. The change in the end-to-end performance estimate of the counterfactual coalition before and after each addition of an operator instance is calculated based on the calibrated surrogate function and used as the marginal contribution of the added operator instances. Statistical processing is performed on all marginal contributions of each operator instance to obtain an estimated performance contribution value for each operator instance.

2. The method as described in claim 1, characterized in that, Also includes: Before performing performance measurements on each operator instance in the computation graph of the large AI model on the source and target hardware respectively, the large AI model is converted into an intermediate representation format and loaded. If the conversion is successful, the intermediate representation format is loaded to obtain the computation graph; if the conversion fails, the native graph representation of the large AI model is obtained through the native graph extraction interface of the corresponding framework of the large AI model to obtain the computation graph. The composite operator in the computation graph is expanded into multiple operator instances, and the fusion operator is split into independent operator instances, so that the computation graph maintains a directed acyclic structure. A unique identifier is generated for each operator instance, and the metadata of each operator instance is extracted. The metadata includes predecessor and successor relationships, input tensor shape, output tensor shape, and data type.

3. The method as described in claim 2, characterized in that, Performance measurements are performed on each operator instance in the computation graph of the large-scale artificial intelligence model on both the source and target hardware, generating performance profiles for each operator instance on both hardware, including: For each operator instance, a set of shapes to be tested is generated. The set of shapes to be tested contains multiple shapes. The shapes include the actual shape values ​​of the input tensor shape and the output tensor shape of the operator instance in end-to-end execution, as well as typical shapes supplemented according to the operator family to which the operator instance belongs. For each shape in the set of shapes to be tested, query whether the operator instance has an available implementation under the shape and data type on the source hardware and the target hardware respectively. The operator instances with available implementations, the corresponding shapes, and the corresponding source hardware are formed into test items for the source hardware. At the same time, the operator instances with available implementations, the corresponding shapes, and the corresponding target hardware are formed into test items for the target hardware. For each test item, a performance measurement is performed and the measurement data is recorded. The performance test includes multiple warm-ups and multiple repeated measurements. Based on the measurement data of all test items for each operator instance on the source and target hardware, a performance profile of each operator instance on the source and target hardware is generated.

4. The method as described in claim 3, characterized in that, The measurement data includes the mean execution delay and the variance of execution delay; Based on the measurement data of all test items for each operator instance on both the source and target hardware, a performance profile of each operator instance on both the source and target hardware is generated, including: Based on the mean and variance of the execution latency of each operator instance in all test items on the source hardware, a performance profile of the operator instance on the source hardware is generated. Based on the mean and variance of the execution latency of each operator instance on the target hardware for all test items, a performance profile of the operator instance on the target hardware is generated. For each operator instance, check whether the effective shape coverage on the source and target hardware meets the preset requirements, and delete the performance profiles that do not meet the preset requirements. The effective shape is the shape that can be implemented in the set of shapes to be tested.

5. The method as described in claim 1, characterized in that, Measuring the end-to-end real performance of the AI ​​large model on the source hardware and the end-to-end real performance on the target hardware includes: The end-to-end inference of the large AI model is executed again on the source hardware, and the end-to-end latency is measured as the end-to-end true performance value of the source hardware. For each operator instance, the sum of the average execution latency of the source hardware in the performance profile is calculated, and the difference between the end-to-end real performance value of the source hardware and the sum of the average execution latency of the source hardware is calculated as the first difference of the operator instance. The end-to-end inference of the AI ​​model is executed again on the target hardware, and the end-to-end latency is measured as the end-to-end true performance value of the target hardware. For each operator instance, the sum of the average execution latency of the target hardware in the performance profile is calculated, and the difference between the end-to-end real performance value of the target hardware and the sum of the average execution latency of the target hardware is calculated as the second difference of the operator instance. If the first or second difference exceeds the first ratio, adjust the number of preheating times and the number of repeated measurements, and then remeasure the end-to-end real performance value of the AI ​​big model on the source hardware and target hardware.

6. The method as described in claim 2, characterized in that, Constructing proxy functions based on the performance profile includes: Constructing proxy functions based on the performance profile includes: For the constructed counterfactual federation, determine the execution time of each operator instance in the counterfactual federation; Dynamic programming is performed in reverse topological order based on the predecessor and successor relationships between the operator instances described in the counterfactual alliance, and the intra-graph execution latency of the computation graph is obtained based on the execution time of each operator instance. The runtime overhead of the air confederation is determined based on the end-to-end true performance value of the source hardware at the air confederation and the in-graph execution latency at the air confederation. The runtime overhead of the full confederation is determined based on the end-to-end true performance value of the target hardware at the full confederation and the in-graph execution latency at the full confederation. The air confederation is a counterfactual confederation that does not contain any operator instances, and the full confederation is a counterfactual confederation that contains all operator instances of the large artificial intelligence model. Based on the runtime overhead at the empty alliance and the runtime overhead at the full alliance, parameter fitting is performed to determine the functional relationship between runtime overhead and the number of operator instances in the counterfactual alliance that adopt the target-side performance profile; A proxy function is determined, the input of which is the counterfactual union, and the output of which is an end-to-end performance estimate, which is the sum of the in-graph execution latency and the runtime overhead determined according to the functional relationship.

7. The method as described in claim 6, characterized in that, For the constructed counterfactual federation, determine the execution time of each operator instance in the counterfactual federation, including: If an operator instance belongs to the counterfactual alliance, the average execution latency in the performance profile of the target hardware is used as the execution time of the operator instance. If the operator instance does not belong to the counterfactual union, the average execution latency in the performance profile of the source hardware is used as the execution time of the operator instance.

8. The method as described in claim 6, characterized in that, Dynamic programming is performed in reverse topological order based on the predecessor-successor relationships between the operator instances described in the counterfactual alliance, and the intra-graph execution latency of the computation graph is obtained based on the execution time of each operator instance, including: Based on the predecessor-successor relationship between each operator instance, all predefined operator groups and all parallel structures in the computation graph are determined, and each parallel structure includes multiple parallel branches. For each operator group, if the target hardware supports the fusion mode of the operator group, then the operator group is regarded as a fusion group, and the execution time of the fusion group is determined to be the single execution time of the fusion core of the fusion group. For each parallel structure, the completion time of each parallel branch of the parallel structure is determined as the earliest completion time of the terminal operator instance of the parallel branch, and the maximum value of the completion time of all parallel branches in the parallel structure is taken as the execution time of the parallel structure. All fusion groups, parallel structures, and individual operator instances in the computation graph are treated as execution units. Based on the execution time of each execution unit and the earliest completion time of all subsequent execution units, the earliest completion time of each execution unit is calculated in reverse topological order. The earliest completion time of the output execution unit with no subsequent execution unit in the computation graph is taken as the execution delay within the graph.

9. The method as described in claim 1, characterized in that, The proxy function is calibrated using the end-to-end real performance values ​​to obtain a calibrated proxy function, including: The affine transformation parameters are determined based on the end-to-end true performance values ​​of the source hardware in the air-alliance and the end-to-end true performance values ​​of the target hardware in the entire alliance. The output value of the surrogate function is transformed using the affine transformation parameters so that the output value of the calibrated surrogate function at the air-alliance is equal to the end-to-end true performance value of the source hardware, and the output value at the entire alliance is equal to the end-to-end true performance value of the target hardware. Check whether the input tensor shape and output tensor shape of each operator instance are within the range of the corresponding performance profile; If the proportion of operator instances outside the range exceeds a preset ratio, a new performance profile will be generated for the operator instances outside the range.

10. The method as described in claim 1, characterized in that, After obtaining the calibrated proxy function, the following is also included: Randomly select at least one counterfactual alliance; For each extracted counterfactual coalition, the end-to-end true performance value of each operator instance in the counterfactual coalition is measured. Specifically, the operator instances belonging to the counterfactual coalition are measured independently on the target hardware to obtain a first measurement value, and the operator instances not belonging to the counterfactual coalition are measured independently on the source hardware to obtain a second measurement value. The first measurement value and the second measurement value are summed to obtain an approximate end-to-end performance value of the counterfactual coalition. The calibrated proxy function is compared with the output value of the counterfactual alliance to obtain the relative error by comparing it with the end-to-end performance approximation. If the average relative error exceeds a preset threshold, the calibrated surrogate function is recalibrated using the end-to-end real performance values ​​of each operator instance on the source and target hardware.

11. The method as described in claim 1, characterized in that, All operator instances are randomly sorted multiple times. Each time, operator instances are added to the counterfactual federation in the sorted order. The change in the end-to-end performance estimate of the counterfactual federation before and after each addition of an operator instance is calculated based on the calibrated surrogate function and used as the marginal contribution of the added operator instances, including: Perform multiple random sorts on all operator instances, generating a sorted sequence each time, and simultaneously constructing the reverse sequence of the sorted sequence. Treat each sorted sequence and its inversion as independent sequences, and perform the following operations on each: For each independent sequence, the empty union is taken as the current counterfactual union, and operator instances are added to the current counterfactual union one by one in the order of the independent sequence; The calibrated proxy function is invoked to calculate the end-to-end performance estimate of the current counterfactual coalition; The difference between the end-to-end performance estimates of the operator instance before and after it is added to the current counterfactual coalition is taken as the marginal contribution of the added operator instance.

12. The method as described in claim 1, characterized in that, Statistical processing is performed on all marginal contributions of each operator instance to obtain an estimated performance contribution value for each operator instance, including: The mean of all marginal contributions for each operator instance is used as the performance contribution estimate for each operator instance.

13. The method as described in claim 1, characterized in that, After obtaining the performance contribution estimate for each operator instance, the following is also included: Calculate the standard error of all marginal contributions for each operator instance; Confidence intervals for each operator instance are constructed based on the performance contribution estimate and the standard error; For each operator instance, if the confidence interval of the operator instance contains zero, the operator instance is marked as having an uncertain contribution direction; if the performance profile of the operator instance has an extrapolation label, the operator instance is marked as requiring supplementary measurement; if the standard error of the operator instance exceeds a predetermined proportion of the performance contribution estimate, the operator instance is marked as having high variance; the uncertainty in contribution direction and high variance are used to represent the confidence level of the corresponding operator instance.

14. The method as described in claim 1, characterized in that, The method further includes: The retest task will be triggered when any of the following conditions are met: There are operator instances that are marked as needing supplementary measurements and whose absolute value of the performance contribution estimate ranks in the top preset order. The type of the supplementary measurement task is marked as shape extrapolation. If the deviation between the output value of the calibrated proxy function at the calibration endpoint and the corresponding end-to-end true performance value exceeds the deviation threshold, the type of the supplementary test task is marked as the endpoint residual. The residual between the sum of the performance contribution estimates of all operator instances and the migration difference exceeds the residual threshold. The migration difference is the difference between the end-to-end true performance value of the AI ​​big model on the source hardware and the end-to-end true performance value on the target hardware. The type of the supplementary test task is marked as the efficiency residual. If the counterfactual alliance fails to perform a sampling review, the type of the retest task is marked as sampling review. The sampling review steps are as follows: input the counterfactual alliance into the calibrated surrogate function. If the deviation between the obtained end-to-end performance estimate and the end-to-end true performance value of the counterfactual alliance exceeds a preset threshold, the sampling review fails.

15. The method as described in claim 14, characterized in that, The method further includes: If the type of supplementary testing task is shape extrapolation, update the performance profile of the operator instance on the target hardware; If the type of supplementary testing task is endpoint residual, then remeasure the end-to-end true performance value of the AI ​​large model on the source hardware and the end-to-end true performance value on the target hardware, and recalibrate the surrogate function. If the type of retest task is efficiency residual or sampling review, then randomly sample multiple additional counterfactual coalitions to reconstruct the surrogate function.

16. An operator-level performance attribution system for a large-scale artificial intelligence model, characterized in that, include: The dual-end profiling module is used to perform performance measurements on each operator instance in the computation graph of the large artificial intelligence model on both the source and target hardware, generate performance profiles of each operator instance on the source and target hardware, and measure the end-to-end true performance value of the large artificial intelligence model on the source hardware and the end-to-end true performance value on the target hardware. The counterfactual alliance generation module is used to construct a counterfactual alliance, which contains any number of operator instances. The operator instances within the counterfactual alliance adopt the performance profile of the target hardware, while the operator instances outside the counterfactual alliance adopt the performance profile of the source hardware. The proxy function construction module is used to construct a proxy function based on the performance profile, perform endpoint calibration on the proxy function using the end-to-end real performance value, and obtain the calibrated proxy function. The marginal contribution calculation module is used to randomly sort all operator instances multiple times. Each time, operator instances are added to the counterfactual alliance one by one according to the sorting order. Based on the calibrated surrogate function, the change in the end-to-end performance estimate of the counterfactual alliance before and after each addition of an operator instance is calculated and used as the marginal contribution of the added operator instance. The performance contribution estimation module is used to perform statistical processing on all marginal contributions of each operator instance to obtain the performance contribution estimate of each operator instance.

17. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 15.

19. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 15.

Images