A portable execution trajectory generation method, system and computer storage medium

Abstract

The application belongs to the technical field of artificial intelligence, and discloses a portable execution trajectory generation method, system and computer storage medium, which comprises the following steps: constructing a componentized abstract framework; using the componentized abstract framework to run a training task on a target hardware to generate a general execution trajectory; constructing a task dynamic competition relationship graph and extracting competition features in a trajectory collection process; using the task dynamic competition relationship graph and the extracted competition features to realize a component model conforming to a standardized interface for a hardware target expected to be evaluated in a simulator to obtain hardware model parameters; constructing a logical queuing network in the simulator based on the general execution trajectory, using a hierarchical graph neural network model to predict a service node queue state; and combining the hardware model parameters and a prediction result of the service node queue state to dynamically deduce the complete performance of the training task on the target hardware. The Trace data can realize unprecedented portability.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of artificial intelligence technology, and in particular to a portable execution trajectory generation method, system, and computer storage medium. Background Technology

[0002] With the construction of domestic computing centers and the rapid development of artificial intelligence (AI) and embodied intelligence, large language models undoubtedly play a fundamental role. Therefore, to meet the configuration requirements of large language models (thousands or even tens of thousands of CPUs), higher demands are placed on the computing power, network, energy consumption, and heat dissipation of computing centers. Improving the resource utilization of computing centers while ensuring energy efficiency is a complex systems engineering project. First, the hardware of the computing center must be compatible, meaning the computing hardware must match its supporting network topology and cooling devices to maximize computing performance. Second, the training tasks of large models must be standardized. For large model training tasks submitted by numerous users, the allocation of training resources must be accurately estimated in advance to ensure effective allocation, improve the success rate of multi-task execution, and reduce ineffective resource loss. Finally, for the training task of a single large model, the computing center must have a clear understanding of the training process to avoid wasting global resources due to local bottlenecks. Furthermore, based on the analysis of historical training tasks, it should understand the patterns of large model training to better serve the efficient completion of each large model training task.

[0003] With the explosive growth in demand for computing power in artificial intelligence, the coexistence of heterogeneous computing architectures (such as GPUs, NPUs, and ASICs) has become the norm. Training large language models typically requires deployment, optimization, and performance evaluation on multiple hardware platforms. Execution traces, as key data recording the sequence and timing information of operations such as computation, communication, and memory access during training, are crucial for performance analysis, bottleneck identification, and system optimization.

[0004] Application content The purpose of this application is to provide a portable execution trace generation method, system, and computer storage medium to solve the problems of existing trace technology being heavily dependent on specific hardware, unable to be reused across platforms, and difficult to simulate resource contention dynamics.

[0005] The embodiments of this application adopt the following technical solutions: This application provides a portable execution trajectory generation method, the portable execution trajectory generation method comprising: Build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; Using the aforementioned component-based abstract framework, training tasks are run on the target hardware to generate a general execution trajectory; During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted; In the simulator, the task dynamic competition relationship graph and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and the hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. By combining the hardware model parameters and the prediction results of the service node queue status, the complete performance of the training task on the target hardware is dynamically simulated.

[0006] This application embodiment also provides a portable execution trajectory generation system, the portable execution trajectory generation system comprising: Build modules to construct a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; The generation module utilizes the component-based abstract framework to run training tasks on the target hardware and generate a general execution trajectory. The extraction module constructs a dynamic competition relationship graph of tasks during the trajectory collection process and extracts competition features; The interface module, in the simulator, utilizes the task dynamic competition relationship graph and the extracted competition features to implement a component model conforming to a standardized interface for the hardware target to be evaluated, thereby obtaining hardware model parameters; The prediction module, based on the general execution trajectory, constructs a logical queuing network in the simulator and uses a hierarchical graph neural network model to predict the queue status of service nodes. The simulation module, combining the hardware model parameters and the prediction results of the service node queue status, dynamically simulates the complete performance of the training task on the target hardware.

[0007] This application also provides a computer storage medium, including a program for use in conjunction with an electronic device, the program being executed by a processor to complete the following steps: Build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; Using the aforementioned component-based abstract framework, training tasks are run on the target hardware to generate a general execution trajectory; During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted; In the simulator, the task dynamic competition relationship graph and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and the hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. By combining the hardware model parameters and the prediction results of the service node queue status, the complete performance of the training task on the target hardware is dynamically simulated.

[0008] Based on the portable execution trajectory generation method, system, and computer storage medium in the embodiments of this application, a training task is run on the target hardware using a constructed component-based abstract framework to generate a general execution trajectory. During the trajectory collection process, a task dynamic competition relationship graph is constructed and competition features are extracted. In the simulator, the task dynamic competition relationship graph and extracted competition features are used to implement a component model conforming to a standardized interface for the hardware target to be evaluated, and hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue state of service nodes. Combining the hardware model parameters and the prediction results of the queue state of service nodes, the complete performance of the training task on the target hardware is dynamically deduced.

[0009] In this way, through parameter normalization and transformation intermediate layers, the generated general Trace is completely stripped of hardware performance parameters, becoming a pure behavior description script that can run on any hardware model that supports the interface, achieving unprecedented portability and completely realizing the platform independence of Trace.

[0010] Furthermore, the modular, pluggable design simplifies supporting new hardware. Only three standard component interfaces need to be implemented, without modifying the core recording and simulation engine. This significantly lowers the barrier to entry and expansion costs, resulting in a highly scalable and user-friendly system. It can be used not only for performance evaluation of existing hardware but also for exploring the design space of future hardware architectures. Engineers can use general-purpose traces to simulate the performance of large-scale model training before hardware manufacturing, guiding architecture design and possessing significant engineering and economic value. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of the embodiments of this specification and form part of the embodiments of this specification, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a portable execution trajectory generation method provided in an embodiment of this specification; Figure 2 This is a schematic diagram of the component-based framework corresponding to a portable execution trajectory generation method provided in the embodiments of this specification; Figure 3This is a flowchart illustrating the behavior recording of parameter normalization in a portable execution trajectory generation method provided in an embodiment of this specification. Figure 4 A flowchart illustrating the logical queuing network modeling in a portable execution trajectory generation method provided in this specification embodiment; Figure 5 This is a schematic diagram of the structure of a portable execution trajectory generation system provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the structure of a computer storage medium corresponding to a portable execution trajectory generation method provided in an embodiment of this specification. Detailed Implementation

[0012] The existing mainstream trace acquisition and simulation technologies have the following fundamental limitations: On the one hand, there is strong hardware binding and platform dependence: existing tools (such as NVIDIA NSight and the original implementation of Chakra) generate traces that are deeply coupled to the performance characteristics of specific hardware platforms. The trace records the absolute execution time under a specific GPU model, specific memory bandwidth, and specific network topology. Such traces cannot be directly used for performance prediction on other hardware platforms because they contain information from both "behavioral logic" and "hardware performance," resulting in "one trace per machine" and extremely poor portability.

[0013] On the other hand, it's about "result recording" rather than "behavioral recording": Traditional traces are essentially records of execution results. They record "on the A100, a certain MatMul operation took 1ms." But for a simulator, what's truly needed is behavioral logic: "This is a MatMul operation with input shape [M, N, K]." Execution time should be dynamically derived by the simulator based on the target hardware's computing power. Existing methods lack the ability to "translate" hardware-dependent performance results into hardware-independent behavioral descriptions.

[0014] Finally, resource contention behavior is difficult to replicate across platforms: dynamic behaviors such as queuing and contention during training (e.g., multiple communication streams competing for bandwidth, multiple computational tasks queuing for the SM) pose a challenge for performance modeling. Traditional methods either ignore these behaviors or record queuing latency under specific hardware configurations in the Trace. The latter is also not transferable because queuing behavior depends on the resource capacity and scheduling strategy of the target hardware.

[0015] Therefore, embodiments of this application provide a portable execution trajectory generation method, system, and computer storage medium. By utilizing a constructed component-based abstract framework, a training task is run on the target hardware to generate a general execution trajectory. During the trajectory collection process, a dynamic competition relationship graph of the task is constructed, and competition features are extracted. In the simulator, the dynamic competition relationship graph of the task and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue state of the service node. Combining the hardware model parameters and the prediction results of the queue state of the service node, the complete performance of the training task on the target hardware is dynamically deduced.

[0016] In this way, through parameter normalization and transformation intermediate layers, the generated general Trace is completely stripped of hardware performance parameters, becoming a pure behavior description script that can run on any hardware model that supports the interface, achieving unprecedented portability and completely realizing the platform independence of Trace.

[0017] Furthermore, the modular, pluggable design simplifies supporting new hardware. Only three standard component interfaces need to be implemented, without modifying the core recording and simulation engine. This significantly lowers the barrier to entry and expansion costs, resulting in a highly scalable and user-friendly system. It can be used not only for performance evaluation of existing hardware but also for exploring the design space of future hardware architectures. Engineers can use general-purpose traces to simulate the performance of large-scale model training before hardware manufacturing, guiding architecture design and possessing significant engineering and economic value.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0020] Please see Figure 1 This is a flowchart illustrating a portable execution trajectory generation method provided in an embodiment of this specification.

[0021] In the embodiments of this specification, the portable execution trajectory generation method may specifically include the following steps: S101: Construct a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; S103: Using the component-based abstract framework, run the training task on the target hardware to generate a general execution trajectory; S105: During trajectory collection, construct a dynamic competition relationship graph for the tasks and extract competition features; S107: In the simulator, using the task dynamic competition relationship graph and the extracted competition features, a component model conforming to a standardized interface is implemented for the hardware target to be evaluated, and hardware model parameters are obtained. S109: Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. S111: Combining the hardware model parameters and the prediction results of the service node queue status, dynamically deduce the complete performance of the training task on the target hardware.

[0022] In the embodiments of this specification, by defining a hardware-independent general Trace format and providing a complete framework from generation (recording) to consumption (simulation), a Trace can be replayed with high fidelity on any hardware model that conforms to the interface specification, thereby achieving "record once, simulate everywhere".

[0023] The overall approach of the embodiments in this specification is decoupling, translation, and reconstruction. Specifically, this involves decoupling hardware implementation from behavioral logic; translating hardware-related performance data into hardware-independent logic quantities; and reconstructing the resource contention model during the simulation phase to adapt it to the target hardware.

[0024] As an application embodiment of this specification, step S101, which involves constructing a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links, may specifically include: Define standardized performance interfaces, including compute unit interface specifications, memory level interface specifications, and network link interface specifications; Design a pluggable assembly mechanism; Interface verification and compatibility testing.

[0025] In the embodiments of this specification, the construction of a component-based abstract framework is the cornerstone of the entire technical solution. Its core objective is to decouple hardware-related performance implementation from hardware-independent behavioral logic, thereby achieving "record once, simulate everywhere".

[0026] The compute unit interface specification requires that the compute unit interface abstract the compute behavior of all types of processing cores (GPU, NPU, CPU core, etc.).

[0027] The memory hierarchy interface specification requires the abstraction of various storage media and access modes.

[0028] In the network link interface specification, the network link interface needs to abstract various interconnection topologies and communication protocols.

[0029] Furthermore, after defining the standardized performance interface, the method also includes: The conversion intermediate layer is designed to eliminate the influence of specific hardware.

[0030] In the embodiments described in this specification, the transformation intermediate layer is also known as the "reference implementation" intermediate layer. The "reference implementation" is a hardware-independent behavioral logic model whose core function is translation rather than simulation. That is, it modifies existing profiler programs, performing analysis, transformation, and recording of actual behavior and dependencies based on existing programs that only record execution data. The purpose of this "reference implementation" is to set a benchmark for generating the "general script." This benchmark itself is also a performance model; it does not pursue absolute accuracy but rather correct behavior and logical completeness.

[0031] Therefore, this "reference implementation" acts as a "translator" or "compiler," shifting the focus of performance analysis and optimization of existing profiler tools to the acquisition of general execution traces.

[0032] Specifically, this can include: referencing the implementation architecture and integrating it into PyTorch Profiler.

[0033] The design of a pluggable assembly mechanism may specifically include: a component registration and discovery mechanism, and a dynamic assembly manager.

[0034] Interface verification and compatibility testing can specifically include: automated test suites and reference implementation benchmark tests.

[0035] In the embodiments described in this specification, the hardware implementation and behavioral logic are completely separated, and the general trace does not contain any hardware-specific information, achieving complete decoupling; the interfaces of the three major components are standardized, and any hardware that conforms to the interface specifications can be plugged and played, achieving standardization; the hardware-independent "reference implementation" layer ensures the accurate translation of behavioral logic; runtime assembly and hot-swapping support flexible hardware simulation scenarios, achieving dynamic configurability; the test suite ensures the compliance and compatibility of third-party implemented interfaces, achieving automated verification.

[0036] This framework provides infrastructure-level support for cross-hardware performance analysis and optimization of large language model training tasks, truly achieving the goal of "record once, simulate everywhere".

[0037] Furthermore, as an application embodiment of this specification, step S103, utilizing the component-based abstract framework to run the training task on the target hardware and generate a general execution trajectory, may specifically include: Intercept computational operations, extract operator types and tensor shapes, and query the transformation intermediate layer to obtain the theoretical computational quantity; The communication operations are captured, and the data volume, participating nodes, and algorithm types are extracted to query the transformation intermediate layer to obtain the logical data exchange volume and path dependency. The logical data exchange volume and related metadata are assembled and serialized according to a standard format to generate the general execution trajectory that does not contain any absolute timestamps.

[0038] In the embodiments described in this specification, computational and communication operations are intercepted and captured, different parameters are extracted, and corresponding query and transformation intermediate layers are used to perform parameter transformation, i.e., parameter normalization. The parameter normalization recording process is the core step in generating hardware-independent general execution traces. This process intercepts specific operations in the deep framework in real time and converts them into pure logical workloads through the "reference implementation" intermediate layer, generating a general trace that is completely independent of hardware performance parameters.

[0039] The interception of computation operations is achieved by registering a forward propagation hook for the module in PyTorch, which is automatically triggered to intercept the operation when the module is executed.

[0040] The communication operation is captured by intercepting the communication primitives, which are key functions of the distributed communication library that are rewritten to capture metadata when communication occurs.

[0041] Furthermore, the embodiments in this specification also include memory operation interception, which can be achieved through tensor lifecycle monitoring, by rewriting PyTorch's memory allocator and overloading tensor methods to monitor memory access.

[0042] After each operation is captured and intercepted, it immediately enters the translation pipeline, which is to query the intermediate translation layer and convert the extracted parameters into a general execution trajectory.

[0043] In addition, to ensure efficient and uninterrupted training, the embodiments in this specification use a circular buffer to temporarily store trajectory nodes and then generate a general execution trajectory.

[0044] The embodiments in this specification complete translation synchronously during operation execution without blocking the training process. A circular buffer ensures efficient memory utilization, achieving real-time, zero-overhead translation. Through the "Reference Implementation" intermediate layer, hardware-specific concepts such as NVIDIA GPU kernel execution time and Cube Core computation cycle are uniformly translated into logical quantities such as FLOPs and Bytes, achieving complete hardware decoupling. It not only records the operation itself but also accurately captures the relationships between operations by analyzing data flow and explicit synchronization primitives, providing an accurate dependency graph for subsequent simulations and maintaining dependency integrity. The "Reference Implementation" can be easily extended to support new hardware architectures or operation types by simply adding corresponding translation rules, achieving scalable translation rules. It adopts industry-standard formats such as Chakra, ensuring that the generated general trajectory can be directly used by various simulators and analysis tools, achieving a standardized output format.

[0045] Furthermore, as an application embodiment of this specification, for step S105, during the trajectory collection process, constructing a dynamic competition relationship graph for tasks may specifically include: Record task service requirements to form a task requirement record set; Identify logical service nodes to construct a service node mapping table; Based on the task requirement record set and the service node mapping table, a dynamic competition relationship graph for tasks is constructed.

[0046] In the embodiments of this specification, the core of realizing hardware-independent queuing behavior modeling is to construct a dynamic competition relationship graph of tasks and extract competition features during the general trajectory collection stage. This process captures the dynamic nature of resource competition through refined data collection and real-time analysis.

[0047] Specifically, a task can be an execution instance of a computational operation (such as MatMul) or a communication operation (such as AllReduce).

[0048] Service requirements can be specifically defined as the amount of logical work required to complete a task. Service requirements can specifically include: Computational task: Theoretical computational load C (unit: FLOPs); Communication task: Data exchange volume S (unit: Bytes).

[0049] A logical service node can be an abstract unit that provides services, for example: Compute node: A specific type of computing unit (such as GEMM_Core_A); Communication node: logical link or path (e.g., Link_A_to_B).

[0050] Furthermore, during the trajectory collection process, competing features are extracted, which may specifically include: Quantitative features are extracted from key decision points for each logical service node to obtain the competitive features.

[0051] In the embodiments described in this specification, the key decision point is when the task arrives or leaves, and quantitative features are used to describe the intensity of competition.

[0052] Specifically, the parameters of the competition characteristics may include: service node object, current time and historical statistics window.

[0053] Furthermore, after extracting the competing features, the method further includes: The extracted competitive features are correlated with the actual physical queue length to form training data pairs for prediction by the hierarchical graph neural network model, and a supervision signal correlation is performed. Optimize data collection strategies.

[0054] In the embodiments of this specification, in order to ensure the quality and efficiency of training data, an intelligent data collection strategy can be adopted to intelligently determine whether data needs to be collected at the current moment, avoid collecting a large amount of similar data, and pay attention to changing nodes in a timely manner.

[0055] As an application embodiment of this specification, for step S107, in the simulator, using the task dynamic competition relationship graph and the extracted competition features, a component model conforming to a standardized interface is implemented for the hardware target to be evaluated, and hardware model parameters are obtained, including: Perform hardware specification parsing and mapping on the obtained target hardware specifications; Implement standardized interfaces for computing units; Implement standardized interfaces for memory-level components; Implement standardized interfaces for network-level components; The standardized interfaces of the computing unit, the memory-level components, and the network-level components are integrated into a unified framework and then verified.

[0056] In the embodiments described in this specification, implementing a component model conforming to a standardized interface for the hardware target to be evaluated is the core step in achieving cross-hardware performance simulation. This process concretizes abstract logical components into performance models specific to the hardware, enabling the general-purpose Trace to perform accurate performance extrapolation on different hardware architectures. It also yields hardware model parameters, representing a conversion process from hardware specifications to an executable performance model.

[0057] The embodiments in this specification achieve plug-and-play functionality for different hardware models through clearly defined interfaces, i.e., a standardized pluggable architecture. Furthermore, the use of parameterized models based on measured data, rather than black-box fitting, ensures the interpretability and scalability of the models, achieving parameterized performance modeling. It considers not only peak performance but also models micro-characteristics such as bank conflicts, caching behavior, and network topology, achieving fine-grained hardware characteristic modeling. Multi-level verification from the component level to the system level ensures the accuracy of the models, implementing a layered verification mechanism.

[0058] As an application embodiment of this specification, for step S109, based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. Specifically, this may include: In the simulator, it is abstracted into a three-layer graph structure: resource layer, service node layer, and task layer; Perform multi-round intra-layer and cross-layer message passing; The representation vector of the service node after message passing is concatenated with the input target hardware parameters. The expected queue state of the service node is obtained by inputting the regression network prediction.

[0059] In this embodiment of the specification, the simulator is abstracted as a three-layer graph structure, from bottom to top as follows: Resource Layer: This layer consists of the most basic physical resource units, including computing units, memory channels, and network links. Each resource unit is a node, and its initial characteristics are the theoretical performance parameters of the resource, including the peak computing power Fmax of the computing unit and the nominal bandwidth B of the link.

[0060] Service Node Layer: This layer corresponds to the logical service nodes defined in step two, including a complete communication link path and a computing core group. Service nodes are connected to resource layer nodes through membership edges, indicating that the performance of a service node depends on its multiple subordinate resource units.

[0061] Task Layer: This layer consists of currently active tasks, including communication flows and computation jobs. Task nodes are connected to the service node layer via competition edges, indicating that the task is using or competing for the service node.

[0062] Furthermore, performing multi-round intra-layer and cross-layer message passing can specifically include: Aggregate load information between adjacent nodes within the resource layer; Aggregate congestion information among relevant nodes within the service node layer; Resource states are aggregated to the service nodes from the bottom up using attention-weighted aggregation; Competitive pressure propagates from top to bottom.

[0063] As an application embodiment of this specification, for step S111, combining the prediction results of the hardware model parameters and the service node queue state, dynamically extrapolating the complete performance of the training task on the target hardware, specifically may include: The operations in the general execution trajectory are replayed in a preset order; When the target operation reaches a logical service node, extract real-time competition features based on the current competition relationship graph. The real-time competition features are input into the hierarchical graph neural network model to obtain the predicted queue state; Query the target hardware model to obtain the performance parameters corresponding to the logical service node; Based on the performance parameters, the queuing delay of the target operation on the logical service node is dynamically calculated; The theoretical service time and queuing delay of computational or communication operations are summed to generate a simulated execution timeline on the target hardware.

[0064] This specification provides a portable execution trajectory generation method that utilizes a constructed component-based abstract framework to run training tasks on target hardware, generating a general execution trajectory. During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted. In a simulator, the dynamic competition relationship graph of tasks and the extracted competition features are used to implement a component model conforming to a standardized interface for the hardware target to be evaluated, obtaining hardware model parameters. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue state of service nodes. Combining the hardware model parameters and the prediction results of the queue state of service nodes, the complete performance of the training task on the target hardware is dynamically deduced.

[0065] In this way, through parameter normalization and transformation intermediate layers, the generated general Trace is completely stripped of hardware performance parameters, becoming a pure behavior description script that can run on any hardware model that supports the interface, achieving unprecedented portability and completely realizing the platform independence of Trace.

[0066] Furthermore, the modular, pluggable design simplifies supporting new hardware. Only three standard component interfaces need to be implemented, without modifying the core recording and simulation engine. This significantly lowers the barrier to entry and expansion costs, resulting in a highly scalable and user-friendly system. It can be used not only for performance evaluation of existing hardware but also for exploring the design space of future hardware architectures. Engineers can use general-purpose traces to simulate the performance of large-scale model training before hardware manufacturing, guiding architecture design and possessing significant engineering and economic value.

[0067] It should be noted that the above-described specific portable execution trajectory generation method is merely a specific application embodiment and does not limit the scope of the embodiments in this specification. Other specific embodiments may also be included, which will not be elaborated here.

[0068] Based on the same inventive concept, this specification also provides specific application examples of the above-described portable execution trajectory generation method.

[0069] The embodiments in this specification mainly include the following parts: Obtaining a pure trace: Using the Chakra framework and PyTorch, we obtain the execution data of the GPU and CPU, correlate the two sets of execution data by the time and sequence of each behavior, and output the system's execution trace.

[0070] The main nodes are established using various objects in the trace, and different nodes are connected according to their dependencies. At the same time, data such as workload, bandwidth, and duration are assigned to the weights of the nodes and the edges.

[0071] The embodiments in this specification address the production technology process of training large language models on high-computing clusters, which mainly involves three aspects of parameters: computation, bandwidth, and memory.

[0072] First, in terms of the overall framework design of the embodiments in this specification, the whole is divided into three abstract components: computing unit, memory level, and network link. Standardized performance interfaces are defined for each component, which serves as the basis for solving the problem of specific hardware binding.

[0073] Secondly, conventional methods for obtaining traces often record data such as the computation execution time of kernel A on GPU 0 during the training process. However, such data is performance data affected by hardware and cannot be used as a universal trace. This invention will develop an intermediate layer of "reference implementation" for behavior translation and recording.

[0074] Finally, there is also competition and queuing behavior caused by insufficient bandwidth and memory due to hardware limitations.

[0075] This specification's embodiments abandon the traditional approach of recording execution result data. Instead, it constructs and recreates the dynamic behavior of resource contention and performs this recreation on a preset hardware configuration. Having established the overall technical framework, the following sections will detail three parts: a component-based framework, parameter-normalized behavior recording, and logical queuing network modeling. These aim to overcome the influence of hardware configuration, hardware parameters, and hardware architecture—all factors inherent in hardware-bound systems—to obtain pure trace data.

[0076] This completes the modeling of the execution graph in this hardware configuration environment. However, regardless of the hardware used as the object of extraction, the final output trace will contain hardware-related parameters and effects. These hardware-bound effects or behaviors need to be removed to obtain the pure logical skeleton, which can then be applied to other different hardware testing environments. Therefore, this section aims to build a general method for obtaining traces by removing the effects of hardware bindings, building upon the conventional method for obtaining traces. The specific steps are as follows: The first step is to build a component-based framework, where the overall architecture is as follows: Figure 2 As shown. Figure 2 This is a schematic diagram of the component-based framework corresponding to a portable execution trajectory generation method provided in the embodiments of this specification.

[0077] like Figure 2 As shown, the open-source model, combined with the PyTorch framework, enters the reference implementation module, which generates the CPU HostTrace, CPU and GPU Kineto Trace respectively. It then enters the Trace synthesizer for timestamp alignment, outputting Trace Plus in the middle. This Trace converter then performs dependency retrieval and finally outputs the Chakra Trace. Finally, it inputs data to obtain the simulation model.

[0078] In the embodiments of this specification, firstly, three major components are constructed: computing unit, memory hierarchy, and network link, which are then abstracted into Compute_Unit, Storage_Unit, and Communication_Unit.

[0079] Secondly, a "reference implementation" is set up to remove the influence of specific hardware. This involves modifying the existing profiler program to analyze, transform, and record actual behavior and dependencies, going beyond simply recording execution data. The purpose of this "reference implementation" is to create a benchmark for generating the "general script." This benchmark itself is a performance model; it doesn't pursue absolute precision but rather correct behavior and logical integrity. Specifically, when running an AI training task on real hardware, the profiler tool doesn't directly record, for example, "This operator took 1ms on the A100." Instead, it queries the "reference implementation" performance model: "How much computation does a matmul operation with shape [1024, 1024] represent in your model?" The reference implementation will answer: "This represents 2.1e9 FLOPs." The final result recorded in the trace is not 1ms, but this 2.1e9 FLOPs.

[0080] Therefore, this "reference implementation" acts as a "translator" or "compiler," shifting the focus of performance analysis and optimization in existing profiler tools to obtaining general execution traces. The conversion rules for computation unit behavior, memory hierarchy behavior, and network link behavior are shown in Table 1 below: Table 1. Transformation rules for computational unit behavior, memory hierarchy behavior, and network link behavior.

[0081] Finally, in accordance with the framework's generality requirements, a "pluggable" component assembly mode is designed. As long as the implementation of a hardware model conforms to this interface, it can be "plugged" into the system like a USB drive without modifying the core code. That is, the attributes of each component are no longer fixed values ​​obtained from specific hardware execution processes, but rather editable attributes that can be assembled by specific models. For example, the previously mentioned "computation execution time of kernel A on GPU 0" can be described as "an operation of type GEMM and size M*N*K was performed on Compute_Unit," and the specific execution time of this operation can be calculated by inputting the equipment parameters of the specific production process. The defined standard interface, such as a Compute_Unit component, must be able to answer how much time is required for execute(kernel_type, problem_size). This "pluggable" design separates the system's core—the trajectory generation and playback engine—from the specific performance implementation of the hardware.

[0082] The second step is to record the behavior of parameter normalization.

[0083] This step follows the previous steps of completing the transformation of computational unit behavior, memory hierarchy behavior, and network link behavior. Then comes the specific execution process in Chakra ET, starting with the technical implementation of the "reference implementation" code.

[0084] Down Figure 3 Chakra currently embeds an observer class into the PyTorch code to monitor the trace execution process. Figure 3 This is a flowchart illustrating the behavior recording of parameter normalization in a portable execution trajectory generation method provided in an embodiment of this specification.

[0085] The embodiments in this specification are modified based on the Chakra observer technology. The "reference implementation" constructs a hardware-independent trajectory generation system by embedding a processing module into the Chakra observer framework. The specific technical content may include: Data processing module 1) Data Acquisition Management Layer: This layer is implemented through a reference-implemented observer module, which is responsible for coordinating the entire data acquisition process. Internally, this module maintains three dedicated calculators: a computation calculator, a memory calculator, and a communication calculator, each handling different types of operations. Additionally, this module is equipped with a circular buffer for temporarily storing the acquired trajectory data; the buffer capacity is configurable as C_buffer.

[0086] 2) Standardized Trajectory Generation Layer: This layer receives hardware-independent data from the metric calculation layer and generates execution trajectories conforming to the Chakra standard. For each operation record, a standardized node is generated containing the following fields: node identifier ID_node, operation name Name_op, operation type Type_op, and attribute set Attributes_set, where the attribute set Attributes_set contains hardware-independent metrics such as computational cost C, memory access volume M, and communication data volume D.

[0087] PyTorch framework integration solution 1) Runtime Interception Mechanism: This method achieves runtime operation interception by registering forward propagation hooks in the PyTorch framework. For each neural network module type (Module_type), a corresponding hook function (Hook_forward) is registered. When the module is executed, the hook function is automatically triggered, capturing the operation type (Op_type), input tensors (Inputs), output tensors (Outputs), and execution context (Context).

[0088] 2) Communication Operation Interception: For communication operations in distributed training, this solution monitors them by intercepting standard communication primitive functions. Specifically, it intercepts collective communication operations such as AllReduce, AllGather, and Broadcast, capturing key parameters such as the amount of communication data (Data_comm), the number of participating nodes (N_nodes), and the communication algorithm (Algo_comm).

[0089] 3) Configurable Management Interface: Provides a unified management interface, including the enable interface Enable_collection(), the disable interface Disable_collection(), and the track export interface Export_trace(Path_output). Users can flexibly control the start and stop of data collection and the output of track files through these interfaces.

[0090] Workflow and Data Processing 1) Real-time conversion process: When PyTorch executes the computation graph, this invention processes each operation according to the following process: ① Operation interception: Capture operation execution events through registered hook functions; ② Parameter extraction: Extract parameters such as tensor shape (Shape_tensor), data type (Type_data), and device information (Info_device) from the operation context; ③Logic quantity calculation: Input the parameters into the corresponding calculation module according to the operation type to obtain hardware-independent logic quantities C, M, D; ④ Trajectory Recording: Assemble logical quantities and related metadata into standardized trajectory nodes and store them in a buffer.

[0091] Track generation process: When a user calls the track export interface, the system performs the following steps: ① Read all accumulated operation records from the buffer; ② Generate a unique node identifier ID_node for each record; ③ Map operation types to the Chakra standard type system; ④ Construct an attribute set Attributes_set containing all calculated hardware-independent metrics; ⑤ Serialize to the Chakra standard format and output to the specified path Path_output.

[0092] The second step is the parameter normalization process, which includes three independent calculation modules. Each module is responsible for converting specific types of hardware-related performance data into hardware-independent logical quantities. The conversion execution method is as follows: (1) Calculation unit behavior: Map the operator Op(Params) to the theoretical computational quantity C. Wherein, the standard form of Op(Params) is tensor shape [M, N, K], and the theoretical computational quantity C = f(Op, Params) FLOPs; [f(Op, Params) is the name of the method to obtain the theoretical computational quantity, where Op is the operator type and Params is the parameter set].

[0093] (2) Memory-level behavior: Map the memory access operation Access(Tensor) to logical data transport volume V and pattern P. Wherein, V = g(Access, Tensor) Bytes; [g(Access, Tensor) is the name of the method for calculating the logical data transport volume of the system, where Access is the access type and Tensor is the tensor description].

[0094] (3) Network link behavior: Map the communication primitive Comm(Algo, Data) to the logical data exchange quantity D of the node and the dependency set Dep. Wherein, D = h_comm(Algo, Data) Bytes, [h_comm(Algo, Data) is the name of the method for calculating the logical data exchange quantity of the node, where Algo is the communication primitive type and Data is the data description].

[0095] The third step is to model the logical queuing network.

[0096] Logical queuing is common during large model training, typically caused by limitations in computing power, memory, or bandwidth resources. The aforementioned embodiments in this specification have already decoupled the tracing process of large model training from hardware configuration, parameters, and performance. Therefore, we do not need to focus on resource peak limitations here; the main focus is on extracting the dependencies between tasks and devices to ultimately reconstruct resource contention and queuing behavior. The overall approach is to disregard and not record the real-time length of the physical queue, but rather record the task arrival logic, service requirements, and dependencies.

[0097] Figure 4 This is a flowchart illustrating the logical queuing network modeling process in a portable execution trajectory generation method provided in this specification.

[0098] In the simulator, these behaviors are reenacted and reproduced using a logical queuing network decoupled from hardware parameters, such as... Figure 4 As shown, the specific steps can be divided into the following: Phase 1: General Trajectory Collection Phase. This phase involves instrumentation at the source code or framework level to generate and collect a hardware-independent general execution trajectory.

[0099] Step 1: Record the service requirements and service nodes of the task. Monitor the execution process within the PyTorch framework. When a communication flow or computation task is detected, record its service requirements. For network flows, the service requirement is the flow size S (in bytes); for computation tasks, the service requirement is the computational load C (in FLOPs). Simultaneously, record the set {N} of logical service nodes that the task must traverse. For network flows, a service node is a set of logical links along its transmission path; for computation tasks, a service node is the type of computational unit required.

[0100] Step two, constructing a dynamic competition graph: Instead of recording the absolute queuing time of tasks on specific hardware, a dynamic competition graph is constructed by analyzing the overlapping lifecycles of tasks. This graph records which tasks simultaneously compete for the same logical service node N at any given time point t. i Specifically, for each service node N i Maintain a set A of tasks that are active on this node at the same time. i (t). This set is dynamically updated as tasks begin and end.

[0101] Step 3: Extract competitive features as supervision signals: While collecting trajectories, key competitive features need to be extracted as supervision signals for subsequent training of the queue prediction model. For each service node N... i At each decision point, such as when the task arrives or departs, a feature vector F is extracted. i (t).

[0102] This vector includes, but is not limited to: the number of currently active tasks | A i (t)|, the total service demand ΣS (or ΣC) for these tasks, the mean and variance of the service demand distribution parameters, and the historical arrival rate. At this stage, only these characteristics and the actual physical queue length Q at this time are recorded. actual However, it is not used for real-time prediction. (F) i (t), Q actual The data pairs will be used as the training dataset.

[0103] The second stage is the performance simulation and deduction stage. In this stage, the performance of the application on the target hardware is deduced in the simulator using the general trajectory and trained model generated and collected in the first stage.

[0104] Step 4: Deploy the core queue prediction model based on hierarchical graph neural networks: In the simulator, a queue prediction model based on a hierarchical graph neural network is integrated for each type of service node. The core innovation of this model is that it does not treat the competitive relationship as a flat graph structure, but rather captures the cascading effects of resource competition more accurately by constructing a multi-level, fine-grained system model.

[0105] The deployment and workflow of this model are as follows: (1) Constructing a hierarchical graph structure First, the simulation system is abstracted into a three-layer graph structure, from bottom to top as follows: Resource Layer: This layer consists of the most basic physical resource units, including computing units, memory channels, and network links. Each resource unit is a node, and its initial characteristics are the theoretical performance parameters of that resource, including the peak computing power F of the computing unit. max The nominal bandwidth of the link, B.

[0106] Service Node Layer: This layer corresponds to the logical service nodes defined in step two, including a complete communication link path and a computing core group. Service nodes are connected to resource layer nodes through membership edges, indicating that the performance of a service node depends on its multiple subordinate resource units.

[0107] Task Layer: This layer consists of currently active tasks, including communication flows and computation jobs. Task nodes are connected to the service node layer via competition edges, indicating that the task is using or competing for the service node.

[0108] (2) Perform layered message passing After constructing the graph structure, this hierarchical GNN performs information propagation and node state updates through the following steps: ① Intra-layer message passing: Within each layer, nodes exchange information through their connections; ② Intra-resource layer transfer: Adjacent resource nodes, i.e., computing units connected on the same bus, aggregate information with each other and update their own load status characteristics; ③ Intra-service node transmission: Service nodes with dependencies, i.e., two paths sharing the same upstream link, aggregate information and update their congestion status characteristics; ④ Intra-task layer transmission: Task nodes are not directly connected to each other; their status is mainly updated through connections with service nodes. ⑤ Cross-layer message passing: This is the key to the model, allowing information to flow bidirectionally between different levels of abstraction; ⑥ Bottom-up propagation: The status of resource nodes, i.e., their current utilization rate, is aggregated to their respective service nodes.

[0109] In terms of network bandwidth, the final bandwidth of a service node is determined by the minimum effective bandwidth of all link resource nodes along its path. This process uses an attention aggregation mechanism, where the service node... Sj For its subordinate resource node R i Features The weighted summation is performed according to the following formula (1): Formula (1) in, The attention weight represents the resource node R. i For service node S j The importance of Softmax lies in its normalized exponential function, which ensures that the sum of all weights is 1, and LeakyReLU, which is the activation function of the linear rectified unit with leakage. Let W be the transpose of the learnable attention vector, || be the vector concatenation operation, and W be the learnable weight matrix used for feature transformation. For service node S j eigenvectors, For resource node R i eigenvectors.

[0110] ⑦ Top-down propagation: The global competition status of service nodes is propagated to task nodes and resource nodes, enabling lower-level resources to perceive the competitive pressure of upper-level tasks.

[0111] (3) Generate queue state prediction After L=3 rounds of the aforementioned layered message passing, each service node obtains a final embedding vector that incorporates the global system competition state. This vector is concatenated with the externally input target hardware configuration vector, and then fed into a two-layer fully connected network with a hidden layer dimension H=256 and an activation function of SiLU for regression prediction. The final output is the expected queue state Q of the service node. pred .

[0112] By introducing this hierarchical graph structure and its corresponding message passing mechanism, this model can fundamentally model the complex, cascading competition behavior caused by resource dependencies, thereby achieving more accurate and general predictions of queuing delays when simulating different target hardware.

[0113] Step 5: Dynamic queuing simulation based on model predictions; When the simulator replays the general trajectory, when a task arrives at a service node, the simulator will perform the following operation: update the dynamic competition graph of that node, i.e., set A. i (t), based on the updated competition situation, calculate the feature vector F in real time. i (t). F i(t) is input into the queue prediction core model to obtain the prediction queue state Q of the current node. pred According to Q pred In addition to the target hardware's performance parameters, such as link bandwidth B and computing unit computing power P, the queuing delay T of the task is dynamically calculated. queue = f(Q pred , B, P).

[0114] In this way, queuing delay is not a fixed value recorded during collection, but a generated value determined by the target hardware parameters and the learned dynamic competition model during simulation. This preserves the core, universal behavior of resource contention while delegating the determination of its numerical outcome to the simulator and target hardware configuration. Thus, the trace acquisition process, freed from the influence of specific hardware constraints, is complete.

[0115] This specification proposes a component-based abstract framework that divides the system into three core components: computing units, memory layers, and network links, and defines standardized performance interfaces for each component. Through a "pluggable" design, the hardware model is decoupled from the system core, supporting flexible assembly and replacement of different hardware implementations.

[0116] Secondly, a "reference implementation" intermediate layer is developed as a hardware-independent behavior translator. The existing Profiler tool is modified to intercept operations during execution and convert hardware-related performance data into hardware-independent logical quantities in real time, generating standardized general execution traces.

[0117] Then, a logical queuing network modeling method is proposed, which realizes the reconstruction and deduction of resource competition in two stages: General trajectory collection phase: Record the service requirements and logical service nodes of the task, construct a dynamic competition relationship graph, extract the competition feature vector and the actual queue length as supervision signals, and form a training dataset.

[0118] Performance simulation and deduction phase: Deploy a queue prediction model based on hierarchical graph neural network. Through a three-layer graph structure of resource layer, service node layer, and task layer, execute intra-layer and cross-layer message passing, and dynamically generate queuing delay in combination with target hardware parameters.

[0119] Meanwhile, a hierarchical graph neural network was designed and implemented as the core model for queue prediction. Through the attention mechanism, the resource layer state is aggregated to the service node, supporting cross-layer information flow from bottom to top and from top to bottom. This enables accurate modeling of the cascading effect of resource competition. Finally, the queue state is predicted through regression using a fully connected network, ensuring the accuracy and versatility of cross-hardware simulation.

[0120] Finally, a complete standardized trajectory data structure and system interface, as well as a configurable management interface, are defined to form an end-to-end hardware-unbound Trace solution that covers data acquisition, trajectory generation, and performance simulation.

[0121] This specification's embodiments overcome the shortcomings of traditional tracing methods, which heavily rely on specific hardware configurations and cannot be reused across platforms, by constructing a component-based framework and a "reference implementation" intermediate layer to transform hardware-dependent execution data into hardware-independent logical quantities. Through parameter normalization and behavior translation, a universal execution trajectory is generated, significantly improving the portability and applicability of tracing data.

[0122] Furthermore, by introducing logical queuing network modeling and hierarchical graph neural network prediction models, the challenge of reproducing resource contention behavior across hardware was solved. Through dynamic contention graph construction, hierarchical message passing, and queue state prediction, task queuing behavior was accurately reproduced in the simulator. This ensured that the performance projection results not only conformed to general behavioral logic but also adapted to the specific parameters of the target hardware, achieving high-fidelity cross-hardware performance prediction.

[0123] Secondly, by adopting a "pluggable" component design standard and standardized performance interfaces, the system core and hardware implementation are decoupled. This supports plug-and-play functionality for different hardware models without requiring modification of the core trajectory generation and playback engine, greatly improving the method's versatility, scalability, and engineering implementation efficiency. It provides a robust and reusable infrastructure for performance analysis and optimization of large model training tasks.

[0124] The specific implementation process of the embodiments in this specification can be referred to the various implementation steps corresponding to the above embodiments, and will not be repeated here.

[0125] Based on the same inventive concept, embodiments of this specification also provide a portable execution trajectory generation system. For example... Figure 5 The diagram shown is a structural schematic of a portable execution trajectory generation system provided in an embodiment of this specification.

[0126] Specifically, the portable execution trajectory generation system may include: Module 501 is used to build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links. The generation module 502 uses the componentized abstract framework to run training tasks on the target hardware and generate a general execution trajectory. The extraction module 503 constructs a dynamic competition relationship graph of tasks and extracts competition features during the trajectory collection process; Interface module 504, in the simulator, uses the task dynamic competition relationship graph and the extracted competition features to implement a component model that conforms to the standardized interface for the hardware target to be evaluated, and obtains hardware model parameters. The prediction module 505, based on the general execution trajectory, constructs a logical queuing network in the simulator and uses a hierarchical graph neural network model to predict the queue status of service nodes. The simulation module 506, combining the hardware model parameters and the prediction results of the service node queue status, dynamically simulates the complete performance of the training task on the target hardware.

[0127] based on Figure 5 The system described in this specification also provides some specific implementation schemes of the system, which will be described below.

[0128] Furthermore, a component-based abstract framework is constructed to define standardized performance interfaces for computing units, memory layers, and network links, including: Define standardized performance interfaces, including compute unit interface specifications, memory level interface specifications, and network link interface specifications; Design a pluggable assembly mechanism; Interface verification and compatibility testing.

[0129] Furthermore, after defining the standardized performance interface, the method also includes: The transformation intermediate layer is designed to remove the influence of specific hardware, so as to shift the focus of tool performance analysis and optimization to the acquisition of general execution traces.

[0130] Furthermore, utilizing the aforementioned component-based abstract framework, a training task is run on the target hardware to generate a general execution trajectory, including: Intercept computational operations, extract operator types and tensor shapes, and query the transformation intermediate layer to obtain the theoretical computational quantity; The communication operations are captured, and the data volume, participating nodes, and algorithm types are extracted to query the transformation intermediate layer to obtain the logical data exchange volume and path dependency. The logical data exchange volume and related metadata are assembled and serialized according to a standard format to generate the general execution trajectory that does not contain any absolute timestamps.

[0131] Furthermore, during the trajectory collection process, a dynamic competition relationship graph for tasks is constructed, including: Record task service requirements to form a task requirement record set; Identify logical service nodes to construct a service node mapping table; Based on the task requirement record set and the service node mapping table, a dynamic competition relationship graph for tasks is constructed.

[0132] Furthermore, during the trajectory collection process, competing features are extracted, which may specifically include: Quantitative features are extracted from key decision points for each logical service node to obtain the competitive features.

[0133] Furthermore, after extracting the competitive features, the system also includes: The extracted competitive features are correlated with the actual physical queue length to form training data pairs for prediction by the hierarchical graph neural network model.

[0134] Furthermore, based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue state of service nodes, including: In the simulator, it is abstracted into a three-layer graph structure: resource layer, service node layer, and task layer; Perform multi-round intra-layer and cross-layer message passing; The representation vector of the service node after message passing is concatenated with the input target hardware parameters. The expected queue state of the service node is obtained by inputting the regression network prediction.

[0135] Furthermore, perform multi-round intra-layer and cross-layer message passing, including: Aggregate load information between adjacent nodes within the resource layer; Aggregate congestion information among relevant nodes within the service node layer; Resource states are aggregated to the service nodes from the bottom up using attention-weighted aggregation; Competitive pressure propagates from top to bottom.

[0136] Furthermore, by combining the predicted results of the hardware model parameters and the service node queue status, the complete performance of the training task on the target hardware is dynamically extrapolated, including: The operations in the general execution trajectory are replayed in a preset order; When the target operation reaches a logical service node, extract real-time competition features based on the current competition relationship graph. The real-time competition features are input into the hierarchical graph neural network model to obtain the predicted queue state; Query the target hardware model to obtain the performance parameters corresponding to the logical service node; Based on the performance parameters, the queuing delay of the target operation on the logical service node is dynamically calculated; The theoretical service time and queuing delay of computational or communication operations are summed to generate a simulated execution timeline on the target hardware.

[0137] This specification provides a portable execution trajectory generation system that, by utilizing a constructed component-based abstract framework, runs a training task on target hardware to generate a general execution trajectory. During trajectory collection, a dynamic competition relationship graph of the task is constructed, and competition features are extracted. In a simulator, using the dynamic competition relationship graph of the task and the extracted competition features, a component model conforming to a standardized interface is implemented for the hardware target to be evaluated, and hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue state of service nodes. Combining the hardware model parameters and the prediction results of the queue state of service nodes, the complete performance of the training task on the target hardware is dynamically deduced.

[0138] In this way, through parameter normalization and transformation intermediate layers, the generated general Trace is completely stripped of hardware performance parameters, becoming a pure behavior description script that can run on any hardware model that supports the interface, achieving unprecedented portability and completely realizing the platform independence of Trace.

[0139] Furthermore, the modular, pluggable design simplifies supporting new hardware. Only three standard component interfaces need to be implemented, without modifying the core recording and simulation engine. This significantly lowers the barrier to entry and expansion costs, resulting in a highly scalable and user-friendly system. It can be used not only for performance evaluation of existing hardware but also for exploring the design space of future hardware architectures. Engineers can use general-purpose traces to simulate the performance of large-scale model training before hardware manufacturing, guiding architecture design and possessing significant engineering and economic value.

[0140] Based on the same inventive concept, embodiments of this specification also provide an electronic device, including at least one processor and a memory, wherein the memory stores a program and is configured to be executed by the at least one processor in the following steps: Build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; Using the aforementioned component-based abstract framework, training tasks are run on the target hardware to generate a general execution trajectory; During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted; In the simulator, the task dynamic competition relationship graph and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and the hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. By combining the hardware model parameters and the prediction results of the service node queue status, the complete performance of the training task on the target hardware is dynamically simulated.

[0141] Other functions of the processor can be found in the above embodiments, and will not be repeated here.

[0142] Based on the same inventive concept, embodiments of this specification also provide a computer-readable storage medium, including a program for use in conjunction with an electronic device, the program being executable by a processor to perform the following steps: Build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; Using the aforementioned component-based abstract framework, training tasks are run on the target hardware to generate a general execution trajectory; During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted; In the simulator, the task dynamic competition relationship graph and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and the hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. By combining the hardware model parameters and the prediction results of the service node queue status, the complete performance of the training task on the target hardware is dynamically simulated.

[0143] in Figure 6 As shown in the figure, this specification also provides a schematic diagram of the structure of a computer storage medium.

[0144] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0145] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0146] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0148] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0150] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0151] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0152] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0153] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0155] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0156] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of protection of the claims of this application.

Claims

1. A portable execution trajectory generation method, characterized in that, The portable execution trajectory generation method includes: Build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; Using the aforementioned component-based abstract framework, training tasks are run on the target hardware to generate a general execution trajectory; During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted; In the simulator, the task dynamic competition relationship graph and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and the hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. By combining the hardware model parameters and the prediction results of the service node queue status, the complete performance of the training task on the target hardware is dynamically simulated.

2. The method as described in claim 1, characterized in that, Construct a component-based abstract framework to define standardized performance interfaces for computation units, memory levels, and network links, including: Define standardized performance interfaces, including compute unit interface specifications, memory level interface specifications, and network link interface specifications; Design a pluggable assembly mechanism; Interface verification and compatibility testing.

3. The method as described in claim 2, characterized in that, After defining the standardized performance interface, the method further includes: The transformation intermediate layer is designed to remove the influence of specific hardware, so as to shift the focus of tool performance analysis and optimization to the acquisition of general execution traces.

4. The method as described in claim 3, characterized in that, Using the aforementioned component-based abstract framework, a training task is run on the target hardware to generate a general execution trajectory, including: Intercept computational operations, extract operator types and tensor shapes, and query the transformation intermediate layer to obtain the theoretical computational quantity; The communication operations are captured, and the data volume, participating nodes, and algorithm types are extracted to query the transformation intermediate layer to obtain the logical data exchange volume and path dependency. The logical data exchange volume and related metadata are assembled and serialized according to a standard format to generate the general execution trajectory that does not contain any absolute timestamps.

5. The method as described in claim 1, characterized in that, During trajectory collection, a dynamic competition graph of tasks is constructed, including: Record task service requirements to form a task requirement record set; Identify logical service nodes to construct a service node mapping table; Based on the task requirement record set and the service node mapping table, a dynamic competition relationship graph for tasks is constructed.

6. The method as described in claim 5, characterized in that, During trajectory collection, competing features are extracted, including: Quantitative features are extracted from key decision points for each logical service node to obtain the competitive features.

7. The method as described in claim 6, characterized in that, After extracting the competing features, the method further includes: The extracted competitive features are correlated with the actual physical queue length to form training data pairs for prediction by the hierarchical graph neural network model.

8. The method as described in claim 1, characterized in that, Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes, including: In the simulator, it is abstracted into a three-layer graph structure: resource layer, service node layer, and task layer; Perform multi-round intra-layer and cross-layer message passing; The representation vector of the service node after message passing is concatenated with the input target hardware parameters. The expected queue state of the service node is obtained by inputting the regression network prediction.

9. The method as described in claim 8, characterized in that, Perform multi-round intra-layer and cross-layer message passing, including: Aggregate load information between adjacent nodes within the resource layer; Aggregate congestion information among relevant nodes within the service node layer; Resource states are aggregated to the service nodes from the bottom up using attention-weighted aggregation; Competitive pressure propagates from top to bottom.

10. The method as described in claim 1, characterized in that, By combining the predicted results of the hardware model parameters and the service node queue status, the complete performance of the training task on the target hardware is dynamically extrapolated, including: The operations in the general execution trajectory are replayed in a preset order; When the target operation reaches a logical service node, extract real-time competition features based on the current competition relationship graph. The real-time competition features are input into the hierarchical graph neural network model to obtain the predicted queue state; Query the target hardware model to obtain the performance parameters corresponding to the logical service node; Based on the performance parameters, the queuing delay of the target operation on the logical service node is dynamically calculated; The theoretical service time and queuing delay of computational or communication operations are summed to generate a simulated execution timeline on the target hardware.

11. A portable execution trajectory generation system, characterized in that, The portable execution trajectory generation system includes: Build modules to construct a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; The generation module utilizes the component-based abstract framework to run training tasks on the target hardware and generate a general execution trajectory. The extraction module constructs a dynamic competition relationship graph of tasks during the trajectory collection process and extracts competition features; The interface module, in the simulator, utilizes the task dynamic competition relationship graph and the extracted competition features to implement a component model conforming to a standardized interface for the hardware target to be evaluated, thereby obtaining hardware model parameters; The prediction module, based on the general execution trajectory, constructs a logical queuing network in the simulator and uses a hierarchical graph neural network model to predict the queue status of service nodes. The simulation module, combining the hardware model parameters and the prediction results of the service node queue status, dynamically simulates the complete performance of the training task on the target hardware.

12. A computer storage medium comprising a program for use in conjunction with an electronic device, the program being executable by a processor to perform the following steps: Build a component-based abstract framework to define standardized performance interfaces for computing units, memory levels, and network links; Using the aforementioned component-based abstract framework, training tasks are run on the target hardware to generate a general execution trajectory; During trajectory collection, a dynamic competition relationship graph of tasks is constructed, and competition features are extracted; In the simulator, the task dynamic competition relationship graph and the extracted competition features are used to implement a component model that conforms to a standardized interface for the hardware target to be evaluated, and the hardware model parameters are obtained. Based on the general execution trajectory, a logical queuing network is constructed in the simulator, and a hierarchical graph neural network model is used to predict the queue status of service nodes. By combining the hardware model parameters and the prediction results of the service node queue status, the complete performance of the training task on the target hardware is dynamically simulated.

Images