Generation method and device for large model execution sequence load, method for large model simulation training and electronic equipment
By constructing a program control flow graph and using data flow analysis and branch selection algorithms to generate large model execution sequence loads, the problem of inaccurate simulation of complex conditional branches in existing technologies is solved, higher-fidelity simulation results are achieved, and costs and time are reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA ACADEMY OF INFORMATION & COMM
- Filing Date
- 2025-12-30
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies cannot accurately handle complex conditional branches when generating large models to execute sequence loads, resulting in inaccurate simulation results.
By constructing a program control flow graph, performing data flow analysis based on large model execution parameters, eliminating deterministic conditional branches, and using a branch selection algorithm to determine the target path of non-deterministic conditional branches, a high-fidelity large model execution sequence load is generated.
It improves the simulation fidelity under complex conditional branches, and the generated execution sequence load can more comprehensively reflect the key execution modes in the training process, reducing simulation costs and time overhead, and improving the accuracy and reliability of simulation results.
Smart Images
Figure CN121996412A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, such as a method and apparatus for generating execution sequence loads for large models, a method for large model simulation training, and electronic equipment. Background Technology
[0002] Large Language Models (LLMs), a revolutionary technological breakthrough in artificial intelligence, are profoundly reshaping human work and lifestyles with their powerful natural language understanding, generation, and multimodal interaction capabilities. The core driving force behind this breakthrough stems from large-scale parameter training based on massive amounts of data—model parameters typically reach tens or even trillions, requiring the use of large-scale high-performance computing resources (including GPU clusters and dedicated AI accelerators) to execute continuous training processes lasting weeks or even months. Such a massive, time-consuming, and costly training task makes the analysis and optimization of the training process a key bottleneck. Effectively understanding, predicting, and optimizing training performance is crucial for reducing training costs, improving resource utilization, and accelerating model iteration.
[0003] Large-scale model training simulation tools utilize software virtualization technology to perform full-stack simulation and prediction of the entire training process before deploying actual hardware resources, providing crucial decision support for training system designers. In a typical large-scale model training simulation system, the execution sequence load and the target hardware configuration together constitute the core input of the simulation module. Simulated execution yields key performance indicators such as training time and resource utilization. One of the mainstream methods for generating execution sequence loads in the industry is pre-execution generation technology. The basic principle of this method is to dynamically capture and record the actual executed operation instruction sequences by pre-executing the model training process within a training software framework (such as PyTorch or TensorFlow), transforming them into a load description usable by the simulation system.
[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art: When dealing with complex conditional branches, the pre-execution method generates an execution sequence load that lacks accuracy.
[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.
[0007] This disclosure provides a method and apparatus for generating execution sequence loads for large models, a method for large model simulation training, and an electronic device to improve the accuracy of the generated execution sequence loads when dealing with complex conditional branches.
[0008] In some embodiments, the method for generating a large model execution sequence load includes: constructing a program control flow graph corresponding to a large model training program; performing data flow analysis on the program control flow graph based on the large model execution parameters to eliminate deterministic conditional branches in the program control flow graph; after eliminating deterministic conditional branches in the program control flow graph, statically traversing the large model execution path to determine the target path of non-deterministic conditional branches in the program control flow graph and obtaining the execution path sequence of the program control flow graph; and generating a large model execution sequence load based on the execution path sequence of the program control flow graph.
[0009] Optionally, deterministic conditional branches include constant conditional branches and / or loop conditional branches; data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate deterministic conditional branches in the program control flow graph, including: when deterministic conditional branches include constant conditional branches, data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate constant conditional branches in the program control flow graph through constant propagation and constant folding; when deterministic conditional branches include loop conditional branches, data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate loop conditional branches in the program control flow graph by expanding loops in the program control flow graph with a constant number of iterations.
[0010] Optionally, data flow analysis is performed on the program control flow graph based on the large model execution parameters. Constant conditional branches in the program control flow graph are eliminated through constant propagation and constant folding. This includes: determining constant values in the large model training program based on the large model execution parameters; propagating the constant values to each node in the program control flow graph and compiling and calculating the expressions composed of constant values; and eliminating constant conditional branches in the program control flow graph based on the deterministic results obtained from the compilation and calculation.
[0011] Optionally, data flow analysis is performed on the program control flow graph based on the large model execution parameters. This is achieved by expanding loops with constant iteration counts in the program control flow graph to eliminate loop condition branches. This includes: identifying loop structures in the program control flow graph and determining the iteration count of the loop structure based on the large model execution parameters; copying the iteration count of the loop body of the loop structure and converting loop control dependencies into sequence dependencies to eliminate loop condition branches in the program control flow graph.
[0012] Optionally, statically traversing the execution path of the large model to determine the target path of the non-deterministic conditional branch in the program control flow graph includes: statically traversing the execution path of the large model to determine the non-deterministic conditional branch in the program control flow graph; and using a branch selection algorithm to determine the target path in the non-deterministic conditional branch.
[0013] Optionally, a branch selection algorithm is used to determine the target path in a non-deterministic conditional branch, including: calculating the dominance relationships of the program control flow graph; determining the critical merging node corresponding to the conditional branch node in the non-deterministic conditional branch based on the dominance relationships of the program control flow graph; wherein, the critical merging node is a necessary node from the conditional branch node to the program exit, and the critical merging node is the node closest to the conditional branch node; calculating the path weight of the execution path from the conditional branch node to the critical merging node; and selecting the target path according to the path weight.
[0014] Optionally, the path weight of the execution path from the conditional branch node to the key convergence node is calculated, including: determining the operation weight of each basic block on the execution path; wherein the operation weight is associated with the operation type and operation size of the basic block; and calculating the path weight based on the operation weight.
[0015] Optionally, selecting a target path based on path weights includes: selecting the execution path with the largest path weight as the target path when the weight difference among multiple path weights is greater than or equal to a preset weight difference threshold; and selecting the execution path containing the most computational operations as the target path when the weight difference among multiple path weights is less than a preset weight difference threshold.
[0016] In some embodiments, the method for large model simulation training includes: performing simulation training on the large model using the aforementioned large model execution sequence load.
[0017] In some embodiments, the apparatus for generating a large model execution sequence load includes: a construction module configured to construct a program control flow graph corresponding to a large model training program; an elimination module communicatively connected to the construction module configured to perform data flow analysis on the program control flow graph based on the large model execution parameters to eliminate deterministic conditional branches in the program control flow graph; a determination module communicatively connected to the elimination module configured to, after eliminating deterministic conditional branches in the program control flow graph, statically traverse the large model execution path to determine the target path of non-deterministic conditional branches in the program control flow graph and obtain the execution path sequence of the program control flow graph; and a generation module communicatively connected to the elimination module configured to generate a large model execution sequence load based on the execution path sequence of the program control flow graph.
[0018] In some embodiments, the electronic device includes: a processor, a memory, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the aforementioned method for generating execution sequence loads for large models, or the aforementioned method for simulation training of large models.
[0019] The method and apparatus for generating execution sequence loads for large models, the method for large model simulation training, and the electronic equipment provided in this disclosure can achieve the following technical effects: In this disclosed technical solution, a program control flow graph corresponding to a large model training program is constructed. Through data flow analysis based on the execution parameters of the large model, deterministic conditional branches dependent on fixed configurations in the program control flow graph are eliminated, thereby focusing on truly complex conditional branches that cannot be statically determined. For non-deterministic conditional branches, the target path of the non-deterministic conditional branch in the program control flow graph is determined during static traversal, ensuring that the generated execution path sequence can cover the key execution modes or worst-case scenarios in the training process. This allows for a more comprehensive and accurate reflection of the execution behavior under complex conditional branches in a static environment, improving the simulation fidelity of the execution sequence load. Thus, by combining compile-time static analysis and heuristic path selection, the accuracy of the generated large model execution sequence load when dealing with complex conditional branches is significantly improved.
[0020] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description
[0021] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a flowchart illustrating a method for generating large model execution sequence loads provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating another method for generating large model execution sequence loads provided in this disclosure embodiment; Figure 3 This is a flowchart illustrating another method for generating large model execution sequence loads provided in this disclosure embodiment; Figure 4 This is a flowchart illustrating a method for large model simulation training provided in an embodiment of this disclosure; Figure 5A This disclosure provides a large model training program and a schematic diagram of the program control flow graph corresponding to the large model training program. Figure 5B This disclosure provides a schematic diagram of an original program control flow graph and a simplified program control flow graph. Figure 6 This is a schematic diagram of a device for generating large model execution sequence loads provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0022] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.
[0023] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.
[0024] Unless otherwise stated, the term "multiple" means two or more. In embodiments of this disclosure, the character " / " indicates that the preceding and following objects are in an "OR" relationship. For example, A / B means: A or B. The term "and / or" describes an association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B. The term "correspondence" can refer to an association or binding relationship; A corresponding to B means that there is an association or binding relationship between A and B.
[0025] The following explains some key technical terms used in the embodiments of this disclosure: Large model training programs refer to the source code used to implement training algorithms for specific large models (such as Transformer, MOE, etc.). A large model training program fully describes the computational logic and control flow of the training task, including core processes such as model forward propagation, loss calculation, back propagation, and optimizer updates, as well as potentially advanced features such as parallel communication, gradient accumulation, and dynamic precision switching.
[0026] Large model execution parameters refer to the set of configuration parameters required to concretize and instantiate a large model training program. Large model execution parameters include model structure parameters (such as the number of layers, hidden layer dimension, and number of attention heads), training configuration parameters (such as global batch size, gradient accumulation steps, and optimizer type), and parallelism strategy parameters (such as data parallelism, tensor parallelism, and pipeline parallelism).
[0027] A program control flow graph is a directed graph representation that reflects the execution flow of a large model training program. Nodes in a program control flow graph are basic blocks (a piece of code executed sequentially), and edges represent control transfers (such as sequential execution, conditional branches, and loop jumps). The program control flow graph abstracts the specific computational details and focuses on depicting the path selection and flow structure of program execution.
[0028] The large model execution sequence refers to a complete record of all operations and their detailed execution contexts that are dynamically generated and arranged in strict chronological order when running a large model training program in a real hardware environment.
[0029] Large model execution sequence load refers to an abstraction of operation sequences that describes the key execution modes and performance characteristics of a training program, generated through static analysis or abstract modeling methods for the purpose of performance simulation. It is platform-independent.
[0030] Combination Figure 1 As shown, this disclosure provides a method for generating execution sequence loads for large models, including the following steps: S101, Construct the program control flow graph corresponding to the large model training program.
[0031] Figure 5A (Left) is a schematic diagram of a large model training procedure provided in an embodiment of this disclosure. Figure 5A (Right) is a schematic diagram of the program control flow graph corresponding to a large model training program provided in an embodiment of this disclosure. Combined with... Figure 5A As shown, the program control flow graph corresponding to the large model training program is constructed, including: capturing all instruction sequences of the large model training program; dividing the continuous instruction sequences into basic blocks; analyzing the control transfer relationship between basic blocks and establishing directed edges.
[0032] A basic block includes an entry point and an exit point, and there are no branching or jumping instructions within the basic block. Directed edges include sequential edges, conditional branch edges, loop edges, and jump edges. Sequential edges show the control transfer relationship from one basic block to the next; conditional branch edges show the control transfer relationship between true and false branches established based on conditional judgment results; loop edges show the control transfer relationship from the loop head to the loop body and from the loop body back to the loop head, based on the identification of loop structures; and jump edges show the control transfer relationship between break, continue, and return instructions.
[0033] The constructed program control flow graph abstracts the complex large-scale model training process into a graphical representation, explicitly showing all possible execution paths, providing a foundation for static traversal, and providing control dependency information for subsequent data flow analysis. By accurately constructing the program control flow graph, a solid foundation is laid for subsequent deterministic branch elimination, non-deterministic branch processing, and the generation of the final large-scale model execution sequence load, ensuring the structural accuracy and semantic integrity of the generated large-scale model execution sequence load.
[0034] S102, perform data flow analysis on the program control flow graph based on the large model execution parameters to eliminate deterministic conditional branches in the program control flow graph.
[0035] Optionally, deterministic conditional branches include constant conditional branches and / or loop conditional branches; data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate deterministic conditional branches in the program control flow graph, including: when deterministic conditional branches include constant conditional branches, data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate constant conditional branches in the program control flow graph through constant propagation and constant folding; when deterministic conditional branches include loop conditional branches, data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate loop conditional branches in the program control flow graph by expanding loops in the program control flow graph with a constant number of iterations.
[0036] Based on compiler optimization theory, this method utilizes the specific configuration information provided by the large model execution parameters to simulate the actual execution semantics of the program in a static environment through data flow analysis. For constant conditional branches, constant propagation is used to substitute parameter values into program variables, and then constant folding is used to calculate the expression result. This allows conditional decisions that depend on constants, such as `if (hidden_size>0)`, to be directly solved at compile time, eliminating redundant branches. For loop conditional branches, loop structures whose boundaries are defined by constant values (e.g., `num_layers=24`) are identified. Loop unrolling copies the loop body into continuous sequential basic blocks, thus converting dynamic control dependencies into static linear sequences. By pre-parsed and solidified the static dependencies of configuration parameters in the control flow graph, the complexity of the control flow is significantly simplified.
[0037] Data flow analysis eliminates many configuration-determined deterministic branches statically at an early stage, allowing subsequent steps to focus on truly complex dynamic branches that cannot be statically determined, thus avoiding unnecessary analysis overhead. Simultaneously, loop unrolling makes repetitive computation patterns explicit, enabling the simulation load to more accurately characterize the resource consumption and time accumulation effects of operations within loops. Overall, this reduces the complexity of the program control flow graph, providing a clearer and more stable analytical foundation for the selection of subsequent nondeterministic branches. This fundamentally enhances the structural accuracy and simulation fidelity of the final large model execution sequence load, and provides key technical support for reliable performance prediction independent of specific hardware environments.
[0038] In some possible implementations, data flow analysis is performed on the program control flow graph based on the large model execution parameters. Constant conditional branches in the program control flow graph are eliminated through constant propagation and constant folding. This includes: determining constant values in the large model training program based on the large model execution parameters; propagating the constant values to each node in the program control flow graph and compiling and calculating the expressions composed of constant values; and eliminating constant conditional branches in the program control flow graph based on the deterministic results obtained from the compilation and calculation.
[0039] The following is a specific example to illustrate this: The following is a snippet of the original training procedure for the large model: def transformer_forward(x, config): hidden_size = config.hidden_dim # Get from configuration attention_dim = hidden_size / / config.num_heads # Conditional branch 1: Check if the hidden layer dimension is valid if hidden_size>0: # Constant conditional branch x = layer_norm(x) # Conditional Branch 2: Select the computation path based on the number of attention heads if config.num_heads>= 8: # Constant conditional branch # Implemented using multi-head attention optimization attn_output = multi_head_attention_optimized(x, attention_dim) else: # Implement using the standard attn_output = multi_head_attention_standard(x, attention_dim) return attn_output The execution parameters for the large model are as follows: config = { "hidden_dim": 768, # Constant value "num_heads": 12# Constant value } The constant values in the large model training program are determined based on the large model execution parameters, namely hidden_dim=768 and num_heads=12, and propagated to the program, namely hidden_size=768 and attention_dim=64 (768÷12).
[0040] If constant values are propagated to each node in the program control flow graph, and expressions consisting of constant values are compiled and evaluated, then: # Post-propagation program semantics hidden_size = 768 attention_dim = 64# 768 / / 12 (calculated under collapsed condition) # Conditional branch 1: if 768>0 → always True if True: # Constant folding result x = layer_norm(x) # Conditional branch 2: if 12>= 8 → always True if True: # Constant folding result attn_output = multi_head_attention_optimized(x, 64) Therefore, branch 1: directly delete the conditional statement, retaining x = layer_norm(x); branch 2: delete the conditional statement and the else branch, retaining the optimized implementation path. The final simplified code is as follows: def transformer_forward(x, config): x = layer_norm(x) attn_output = multi_head_attention_optimized(x, 64) return attn_output Figure 5B (Left) is a schematic diagram of a raw program control flow graph provided in an embodiment of this disclosure. Figure 5B (Right) is a simplified program control flow diagram provided in an embodiment of this disclosure. Combined with... Figure 5B As shown, by precisely solving and eliminating constant conditional branches determined by configuration parameters at compile time, deviations caused by incomplete path coverage in the pre-execution method are avoided. By eliminating a large number of statically deterministic control flow branches, the program control flow graph is greatly simplified.
[0041] In some possible implementations, data flow analysis is performed on the program control flow graph based on the large model execution parameters. This is achieved by expanding loops with constant iteration counts in the program control flow graph to eliminate loop condition branches. This includes: identifying loop structures in the program control flow graph and determining the iteration count of the loop structure based on the large model execution parameters; copying the iteration count of the loop body of the loop structure; and converting loop control dependencies into sequence dependencies to eliminate loop condition branches in the program control flow graph.
[0042] Here, a loop structure is a strongly connected subgraph in the program control flow graph, consisting of a loop head node (the entry point that governs all nodes within the loop body), a loop body subgraph (the part that is repeatedly executed), and back edges returning from the loop body to the loop head. The core feature of a loop structure is that there are control flow edges within the loop body pointing to the loop head or earlier nodes, forming closed paths, allowing the program flow to potentially traverse nodes within the loop body multiple times.
[0043] The loop body is the core part of a loop structure that is repeatedly executed; it is the subgraph in the program control flow graph that is located after the loop head and before the back edge and is dominated by the loop. The loop body includes the actual computation and communication operations performed in each iteration, but does not contain the conditional branch nodes in the loop head that determine whether to continue iterating.
[0044] Based on loop unrolling techniques in compiler optimization, this study identifies loop structures in the program's control flow graph where the loop boundaries are constant due to large model execution parameters (such as the number of model layers and gradient accumulation steps) through data flow analysis. During the static analysis phase, based on the determined number of loop iterations, the loop body is copied a corresponding number of times in the control flow graph. A series of sequentially connected basic block copies replace the original loop header, loop body, and loop condition decision nodes, thereby transforming dynamic control flow edges with conditional jumps into static, linear sequential dependencies, fundamentally eliminating conditional branches caused by loop iterations.
[0045] By converting the main dynamic control structure in the large model training program—the loop—into a deterministic static sequence, the accuracy of generating the large model execution sequence load and the reliability of simulation results are significantly improved. The expanded large model training execution sequence allows the simulation to accurately calculate the cumulative computation and communication time of each iteration within the loop, avoiding the uncertainty in estimating loop control overhead. At the same time, the linearized control flow greatly simplifies the subsequent dependency analysis and path traversal complexity, laying a solid foundation for building a high-fidelity performance model for the simulation system. Ultimately, this helps to obtain more reliable performance predictions before hardware deployment, reducing system design risks and tuning costs.
[0046] S103, after eliminating the deterministic conditional branches in the program control flow graph, statically traverse the execution path of the large model to determine the target path of the non-deterministic conditional branches in the program control flow graph, and obtain the execution path sequence of the program control flow graph.
[0047] Optionally, statically traversing the execution path of the large model to determine the target path of the non-deterministic conditional branch in the program control flow graph includes: statically traversing the execution path of the large model to determine the non-deterministic conditional branch in the program control flow graph; and using a branch selection algorithm to determine the target path in the non-deterministic conditional branch.
[0048] By statically traversing the program control flow graph, when encountering non-deterministic conditional branches (i.e., branches whose conditions depend on runtime data) that cannot be eliminated by constant propagation or loop unrolling, a branch selection algorithm based on dominance relationship analysis and path weight evaluation is used to make a decision, determine the target path in the non-deterministic conditional branch, and thus statically resolve the non-deterministic branch into a definite execution direction, generating a complete linear execution sequence.
[0049] By defining branch scope through dominance relationships and intelligently selecting based on path weights, the generated large model execution sequence load prioritizes critical execution paths that may become performance bottlenecks (such as the most computationally or communication-intensive paths), thereby improving the accuracy and reference value of simulation results. Furthermore, this method is entirely performed in static analysis, without requiring actual training program execution, maintaining the platform independence and low cost of large model execution sequence load generation. It also systematically solves the problem of handling nondeterministic branches, enhancing the versatility and robustness of the large model execution sequence load generation method, and providing a reliable basis for early performance evaluation and optimization of the training system.
[0050] S104 generates a large model execution sequence load based on the execution path sequence of the program control flow graph.
[0051] In practical applications, based on the execution path sequence of the program control flow graph, a large model execution sequence load is generated. This includes: encoding the specific operations (such as matrix multiplication and communication synchronization) within each basic block of the execution path sequence into standardized operation units with performance semantics, based on the operation type, input / output tensor shape, and data dependencies of the specific operations; and based on the structure of the program control flow graph, preserving and explicitly defining the data dependencies and control dependencies between operations to generate a large model execution sequence load containing complete operation sequences, dependencies, and estimated resource requirements (such as computational load and communication load), which serves as the direct input to the simulator.
[0052] Large model execution sequence workloads accurately capture the execution semantics and dependencies of large model training programs, enabling simulators to perform efficient and accurate performance simulations independent of specific source code and hardware environments. The platform independence of large model execution sequence workloads allows the same workload to be reused for simulation comparisons across various hardware configurations, greatly improving the flexibility and efficiency of simulation experiments. This provides a reliable and convenient performance evaluation basis for the co-design and optimization of training systems' hardware and software, significantly reducing the uncertainty and trial-and-error costs of system design.
[0053] The method for generating execution sequence loads for large models, as provided in this disclosure, constructs a program control flow graph corresponding to the large model training program. Through data flow analysis based on the large model execution parameters, deterministic conditional branches dependent on fixed configurations in the program control flow graph are eliminated, thereby focusing on truly complex conditional branches that cannot be statically determined. For non-deterministic conditional branches, the target path of the non-deterministic conditional branch in the program control flow graph is determined during static traversal, ensuring that the generated execution path sequence can cover key execution modes or worst-case scenarios during training. This allows for a more comprehensive and accurate reflection of execution behavior under complex conditional branches in a static environment, improving the simulation fidelity of the execution sequence load. Thus, by combining compile-time static analysis with heuristic path selection, the accuracy of the generated large model execution sequence loads when dealing with complex conditional branches is significantly improved.
[0054] In some embodiments, a branch selection algorithm is used to determine the target path in a non-deterministic conditional branch, including the following steps: S201, calculate the dominance relationships in the program control flow graph.
[0055] Optionally, calculating the dominance relationships in the program control flow graph includes: initializing a dominance set for each node in the program control flow graph, wherein the dominance set of the entry node includes itself, and the dominance set of other nodes is the universal set of all nodes; repeatedly applying the dominance equation Dom(n) = {n} ∪ (∩ Dom(p) for p in pred(n)) for iterative calculation, wherein the dominance set of each node n is the intersection of its own dominance set and the dominance sets of all its predecessor nodes, until the dominance sets of all nodes no longer change; constructing a dominance tree based on the final dominance set, wherein the direct dominator of each node is the node closest to it in the dominance set except itself, that is, the unique node that dominates the node but does not dominate any other dominators of the node.
[0056] In practical applications, the dominance front of each node can be efficiently calculated by post-order traversal of the dominance tree or by using specialized algorithms (such as the Lengauer-Tarjan algorithm). The dominance set can be represented by a bit vector to optimize set operation efficiency, and the dominance relationship of any node can be quickly queried using the dominance tree structure.
[0057] Based on the dominance relationships in the program control flow graph, key merging nodes corresponding to conditional branch nodes can be quickly located, accurately defining the scope of branch influence within the local subgraph from the branch point to the merging point, avoiding the complexity and unnecessary traversal of the entire path. Simultaneously, the dominance tree structure allows path weight calculation to focus on the code regions truly affected by branch decisions, ensuring the fairness and representativeness of path comparisons. This fundamentally guarantees the feasibility of the branch selection algorithm, enabling the execution sequence load of the statically generated large model to reasonably simulate branch behavior, enhancing simulation accuracy while maintaining the efficiency of static analysis.
[0058] S202, based on the dominance relationship of the program control flow graph, determine the key merging node corresponding to the conditional branch node in the non-deterministic conditional branch.
[0059] Among them, the critical merging node is the necessary node from the conditional branch node to the program exit, and the critical merging node is the node closest to the conditional branch node.
[0060] For a given nondeterministic conditional branch node, its corresponding key merging node is the direct ruler (IDom) of the conditional branch node in the dominance tree. By querying the pre-built dominance tree, the key merging node can be quickly and uniquely determined, thereby accurately defining the range of code regions affected by the branch decision.
[0061] By locating the "nearest necessary merging node," the branch path selection problem, which requires complex trade-offs, is strictly limited to the analysis within a finite program region between the conditional branch node and the key merging node. This avoids the problem of the evaluation range being too large or too small due to the path merging too early or too late. It ensures that the subsequent calculation and comparison of path weights are carried out within a controllable and semantically complete program segment, which not only improves the rationality of path selection but also significantly reduces computational complexity. This is a key prerequisite for ensuring that the branch selection algorithm can make reliable decisions in a static analysis environment.
[0062] S203, calculate the path weight of the execution path from the conditional branch node to the critical convergence node.
[0063] Optionally, the path weight of the execution path from the conditional branch node to the key convergence node is calculated, including: determining the operation weight of each basic block on the execution path; wherein the operation weight is associated with the operation type and operation size of the basic block; and calculating the path weight based on the operation weight.
[0064] In some possible implementations, the path weight is calculated according to the following formula:
[0065] in, For path weights, For the execution path, , For conditional branch nodes, As a key rendezvous point, For the first on the path A basic block, Representing basic blocks The operational weights, Representing basic blocks The number of operations included. Representing basic blocks The j-th operation in Indicates the operation type (e.g., GEMM, AllReduce, LayerNorm). Indicates operational scale parameters (such as tensor shape, communication data volume, etc.). It is a weight mapping function that maps operation type and scale to weight values.
[0066] The weight mapping function is calculated using the following formula:
[0067] in, The basic weighting coefficients for operation types. This is a complexity function of the operation size. Different operation types correspond to different weight mapping functions.
[0068] Transforming complex path performance comparisons into computable numerical comparisons provides an objective and consistent basis for branch selection.
[0069] S204, Select the target path based on path weight.
[0070] Optionally, selecting a target path based on path weights includes: selecting the execution path with the largest path weight as the target path when the weight difference among multiple path weights is greater than or equal to a preset weight difference threshold; and selecting the execution path containing the most computational operations as the target path when the weight difference among multiple path weights is less than a preset weight difference threshold.
[0071] When the weight differences between different execution paths are significant (exceeding a preset threshold), the principle of prioritizing the maximum weight is adopted, directly selecting the path with the highest estimated cost as the target path, thereby ensuring that the simulation can cover the most time-consuming critical scenarios. When the weights of multiple paths are similar (the difference is less than the threshold), an auxiliary decision factor is activated, and the path containing the most computational operations is selected instead. This is based on the assumption that computationally intensive paths are usually more representative of performance and have more optimization value in training scenarios.
[0072] On the one hand, the threshold mechanism avoids decision jitter caused by subtle differences in weights, ensuring that the weight index is relied upon only when the differences are significant. On the other hand, in "fuzzy" scenarios where weights are similar, the number of computational operations is introduced as a secondary indicator, prioritizing computationally intensive paths. This better aligns with the performance characteristics of large model training (computation is often the main bottleneck), making the generated large model execution sequence load more representative in key scenarios. While maintaining the algorithm's simplicity and efficiency, the interpretability of path selection results and the accuracy of simulation load are enhanced.
[0073] The branch selection algorithm of this disclosure firstly uses dominance relationships to accurately locate key merging nodes, limiting the influence of branches to locally controllable path segments and avoiding the complexity of global analysis. Secondly, it quantitatively compares candidate paths by calculating path weights and uses a difference threshold and multi-factor decision-making mechanism to select the most representative target path, thereby significantly improving the accuracy of the generated load in characterizing key execution behaviors during real training. Finally, the algorithm achieves reasonable prediction of nondeterministic branches in a completely static environment without actual operation, ensuring both the availability and platform independence of the load, and greatly enhancing the reliability of simulation results and the guiding value for system design optimization.
[0074] Combination Figure 3 As shown, the method for generating execution sequence loads for large models includes the following steps: S301, Construct the program control flow graph corresponding to the large model training program.
[0075] S302, when deterministic conditional branches include constant conditional branches, performs data flow analysis on the program control flow graph based on the large model execution parameters, and eliminates constant conditional branches in the program control flow graph through constant propagation and constant folding.
[0076] S303, when deterministic conditional branches include loop conditional branches, perform data flow analysis on the program control flow graph based on the large model execution parameters, and eliminate loop conditional branches in the program control flow graph by expanding loops with constant loop counts.
[0077] S304, statically traverses the execution path of the large model to determine the non-deterministic conditional branches in the program control flow graph.
[0078] S305, calculates the dominance relationships in the program control flow graph.
[0079] S306, Based on the dominance relationship of the program control flow graph, determine the key merging node corresponding to the conditional branch node in the non-deterministic conditional branch; wherein, the key merging node is the necessary node from the conditional branch node to the program exit, and the key merging node is the node closest to the conditional branch node.
[0080] S307, calculate the path weight of the execution path from the conditional branch node to the critical convergence node.
[0081] S308, select the target path based on path weight.
[0082] S309, obtain the execution path sequence of the program control flow graph.
[0083] S310 generates a large model execution sequence load based on the execution path sequence of the program control flow graph.
[0084] By employing systematic static program analysis techniques, the dynamic execution behavior of a large model training program is transformed into a deterministic load description. First, data flow analysis is performed on the program control flow graph based on given execution parameters. Static branches determined by constants are eliminated through constant propagation and folding, and loops with fixed iteration counts are transformed into linear sequences through loop unrolling, thereby maximizing the simplification of control flow. Subsequently, for non-deterministic conditional branches that still exist after optimization and depend on runtime data, the dominance relationship analysis in the compiler domain is used to locate the branch's influence range. By calculating and comparing the path weights of each path within this influence range, heuristic rules are used to determine the execution path to be used in the simulation. Finally, this complete execution path sequence derived statically is encoded into a platform-independent large model execution sequence load containing abstract information such as operations, dependencies, and resource requirements.
[0085] The technical solution of this disclosure firstly eliminates static uncertainty through compilation optimization and adopts heuristic selection based on dominance analysis and path weight for dynamic branches, so that the generated load can more comprehensively represent key or typical execution modes and improve the fidelity of simulation prediction; secondly, the entire generation process is completely static, and there is no need to run the model on actual hardware, which realizes the platform independence of the generation of large model execution sequence load and greatly reduces the time and economic cost of obtaining large model execution sequence load.
[0086] Combination Figure 4 As shown, the method for large model simulation training includes the following steps: S401, Construct the program control flow graph corresponding to the large model training program.
[0087] S402 performs data flow analysis on the program control flow graph based on the execution parameters of the large model, eliminating deterministic conditional branches in the program control flow graph.
[0088] S403, after eliminating deterministic conditional branches in the program control flow graph, statically traverse the execution path of the large model to determine the target path of non-deterministic conditional branches in the program control flow graph and obtain the execution path sequence of the program control flow graph.
[0089] S404 generates a large model execution sequence load based on the execution path sequence of the program control flow graph.
[0090] S405 uses a large model execution sequence load to simulate and train a large model.
[0091] The generated large model execution sequence load is used as input to the simulator. Combined with the configuration parameters of the target hardware system (such as computing unit performance, memory bandwidth, network topology, and communication latency), the simulator dynamically simulates the specific execution of the training process through discrete event simulation or performance model-based simulation methods. Based on the operation sequence, dependencies, and resource requirements described in the large model execution sequence load, the simulator schedules each computation or communication operation in chronological order, simulating the execution time, resource consumption, and interlocking waits of operations on a virtual timeline. It also considers hardware parallelism and resource contention, thereby constructing a complete training timeline. Finally, through aggregation and statistics, key performance indicators such as overall training time, resource utilization, and communication overhead are obtained, enabling low-cost, high-efficiency performance prediction and bottleneck analysis of the large model training process.
[0092] The method for large-scale model simulation training provided in this disclosure firstly eliminates deterministic conditional branches in advance through data flow analysis based on the large model's execution parameters, and then uses a branch selection algorithm to predict reasonable paths for the remaining non-deterministic branches. This significantly improves the accuracy of the generated execution sequence load, enabling it to more comprehensively cover key execution modes and worst-case scenarios during training, overcoming the biases caused by single-run and runtime dependencies in traditional pre-execution methods. Secondly, this method generates the execution sequence load entirely in a static environment, eliminating the need for actual use of expensive hardware resources for trial runs, greatly reducing costs and time overhead, and ensuring the platform independence of the execution sequence load, allowing the same execution sequence load to be flexibly applied to simulation tests on different hardware configurations. Finally, simulation training based on this high-fidelity load can provide highly reliable performance predictions for parallel strategy optimization, hardware scale planning, and bottleneck diagnosis, thereby systematically improving the design efficiency and resource utilization of the large-scale model training system. It should be noted that this technical solution is positioned as a performance optimization method for the core components of a large-scale model training simulation system. Its fundamental value lies in the substantial improvement of the simulation system's "internal performance." Generating high-fidelity large-scale model execution sequence loads through static program analysis directly and inevitably reflects a substantial improvement in the internal performance and functionality of the "large-scale model training simulation system," thus solving the core technical bottleneck of low load accuracy and strong platform dependence caused by traditional pre-execution methods' reliance on specific hardware and runtime data. Therefore, the direct technical effect of this solution is not simply an innovation in algorithmic logic, but rather provides the upstream simulation system with more accurate, more universal, and more efficient key input data, thereby directly improving the reliability, computational efficiency, and resource planning capabilities of the simulation system for performance prediction, meeting the patent examination criteria for "improvement of the internal performance of a computer system."
[0093] Combination Figure 6 As shown, this disclosure provides a device 600 for generating large model execution sequence loads, including a construction module 610, an elimination module 620, a determination module 630, and a generation module 640. The construction module 610 is configured to construct a program control flow graph corresponding to a large model training program. The elimination module 620 is communicatively connected to the construction module 610 and is configured to perform data flow analysis on the program control flow graph based on the large model execution parameters, eliminating deterministic conditional branches in the program control flow graph. The determination module 630 is communicatively connected to the elimination module 620 and is configured to, after eliminating deterministic conditional branches in the program control flow graph, statically traverse the large model execution path to determine the target path of non-deterministic conditional branches in the program control flow graph and obtain the execution path sequence of the program control flow graph. The generation module 640 is communicatively connected to the determination module 630 and is configured to generate a large model execution sequence load based on the execution path sequence of the program control flow graph.
[0094] The apparatus for generating execution sequence loads for large models, as provided in this disclosure, constructs a program control flow graph corresponding to a large model training program. Through data flow analysis based on the large model execution parameters, deterministic conditional branches dependent on fixed configurations in the program control flow graph are eliminated, thereby focusing on truly complex conditional branches that cannot be statically determined. For non-deterministic conditional branches, the target path of the non-deterministic conditional branch in the program control flow graph is determined during static traversal, ensuring that the generated execution path sequence can cover key execution modes or worst-case scenarios during training. This allows for a more comprehensive and accurate reflection of execution behavior under complex conditional branches in a static environment, improving the simulation fidelity of the execution sequence load. Thus, by combining compile-time static analysis with heuristic path selection, the accuracy of the generated large model execution sequence loads when dealing with complex conditional branches is significantly improved.
[0095] Combination Figure 7 As shown, this embodiment of the disclosure provides an electronic device (e.g., a computer, controller, etc.) 700, which includes a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the bus 704.
[0096] The memory 701 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).
[0097] The memory 701 can store programs. When the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the various steps of the method for generating large model execution sequence load or the method for large model simulation training according to the embodiments of this application.
[0098] Processor 702 is a circuit with signal processing capabilities. In one implementation, processor 702 can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, processor 702 can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, processor 702 can be a hardware circuit implemented as an ASIC or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a type of ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), or deep learning processing unit (DPU). The processor 702 is used to execute related programs to implement the functions required by the units in the apparatus for generating large model execution sequence loads according to the embodiments of this application, or to execute the method for generating large model execution sequence loads and the method for large model simulation training according to the method embodiments of this application.
[0099] As can be seen, each module in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types.
[0100] Furthermore, the modules in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these modules are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or for implementing the functions of the modules of the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.
[0101] The communication interface 703 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the electronic device 700 and other devices or communication networks. For example, data can be acquired through the communication interface 703.
[0102] Bus 704 may include a pathway for transmitting information between various components of electronic device 700 (e.g., memory 701, processor 702, communication interface 703).
[0103] It should be noted that, although Figure 7 The illustrated electronic device 700 only shows the memory, processor, and communication interface. However, those skilled in the art should understand that in specific implementations, the electronic device 700 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the electronic device 700 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that the electronic device 700 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 7 All the devices shown.
[0104] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of any of the above methods.
[0105] This application also provides a computer program product containing instructions. When the computer program product is run on a computer or processor, it causes the computer or processor to perform one or more steps of any of the methods described above.
[0106] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.
[0107] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.
[0108] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or substituted for parts and features of other embodiments. The scope of the embodiments of this disclosure includes the entire scope of the claims and all available equivalents of the claims. While the terms “first,” “second,” etc., may be used in this application to describe elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be called a second element without changing the meaning of the description, and similarly, a second element may be called a first element, provided that all occurrences of “first element” are consistently renamed and all occurrences of “second element” are consistently renamed. First and second elements are both elements, but may not be the same element. Moreover, the terminology used in this application is only for describing embodiments and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. Similarly, the term “and / or” as used herein means including one or more of the associated listed any and all possible combinations. Additionally, when used herein, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase “comprising an…” does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.
[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0110] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
Claims
1. A method for generating execution sequence loads for large models, characterized in that, include: Construct the program control flow graph corresponding to the large model training program; Data flow analysis is performed on the program control flow graph based on the large model execution parameters to eliminate deterministic conditional branches in the program control flow graph; After eliminating deterministic conditional branches in the program control flow graph, statically traverse the execution path of the large model to determine the target path of non-deterministic conditional branches in the program control flow graph and obtain the execution path sequence of the program control flow graph. Based on the execution path sequence of the program control flow graph, a large model execution sequence load is generated.
2. The generation method according to claim 1, characterized in that, Deterministic conditional branches include constant conditional branches and / or loop conditional branches; Data flow analysis of the program control flow graph is performed based on the execution parameters of the large model to eliminate deterministic conditional branches in the program control flow graph, including: In cases where deterministic conditional branches include constant conditional branches, data flow analysis is performed on the program control flow graph based on the large model execution parameters. Constant conditional branches in the program control flow graph are eliminated through constant propagation and constant folding. In cases where deterministic conditional branches include loop conditional branches, data flow analysis is performed on the program control flow graph based on the large model execution parameters. By expanding loops with a constant number of iterations in the program control flow graph, loop conditional branches in the program control flow graph are eliminated.
3. The generation method according to claim 2, characterized in that, Data flow analysis of the program control flow graph is performed based on the execution parameters of the large model. Constant propagation and constant folding are used to eliminate constant conditional branches in the program control flow graph, including: Determine the constant values in the large model training procedure based on the large model execution parameters; The constant values are propagated to each node in the program control flow graph, and the expressions composed of constant values are compiled and evaluated. Based on the deterministic results obtained from compilation calculations, constant conditional branches in the program control flow graph are eliminated; Data flow analysis of the program control flow graph is performed based on the execution parameters of the large model. This involves expanding loops with constant iteration counts in the program control flow graph to eliminate loop condition branches, including: Identify loop structures in the program control flow graph and determine the number of iterations of the loop structure based on the large model execution parameters; The loop body of the loop structure is copied with the number of iterations, and loop control dependencies are converted to sequence dependencies to eliminate loop condition branches in the program control flow graph.
4. The generation method according to any one of claims 1 to 3, characterized in that, Static traversal of the large model execution path determines the target path of non-deterministic conditional branches in the program control flow graph, including: Static traversal of the large model execution path determines the non-deterministic conditional branches in the program control flow graph; Using a branch selection algorithm, the target path in a non-deterministic conditional branch is determined.
5. The generation method according to claim 4, characterized in that, Using branch selection algorithms, the target path in non-deterministic conditional branches is determined, including: Calculate the dominance relationships in the control flow graph of the program; Based on the dominance relationship of the program control flow graph, the key merging node corresponding to the conditional branch node in the non-deterministic conditional branch is determined; wherein, the key merging node is the necessary node from the conditional branch node to the program exit, and the key merging node is the node closest to the conditional branch node. Calculate the path weight of the execution path from the conditional branch node to the critical convergence node; Select the target path based on path weight.
6. The generation method according to claim 5, characterized in that, Calculate the path weight of the execution path from the conditional branch node to the critical merge node, including: Determine the operation weight of each basic block on the execution path; where the operation weight is related to the operation type and operation size of the basic block; Path weights are calculated based on operation weights.
7. The generation method according to claim 5, characterized in that, Selecting the target path based on path weight includes: If the weight difference of multiple paths is greater than or equal to a preset weight difference threshold, the execution path with the largest path weight is selected as the target path. If the weight difference among multiple paths is less than a preset weight difference threshold, the execution path containing the most computational operations is selected as the target path.
8. A method for large-scale model simulation training, characterized in that, include: The large model is simulated and trained using the large model execution sequence load as described in any one of claims 1 to 7.
9. An apparatus for generating large model execution sequence loads, comprising: The construction module is configured to construct the program control flow graph corresponding to the large model training program; The elimination module, which communicates with the construction module, is configured to perform data flow analysis on the program control flow graph based on the large model execution parameters, and eliminate deterministic conditional branches in the program control flow graph. The determination module, which communicates with the elimination module, is configured to eliminate deterministic conditional branches in the program control flow graph. It then statically traverses the execution path of the large model to determine the target path of non-deterministic conditional branches in the program control flow graph and obtains the execution path sequence of the program control flow graph. The generation module, which communicates with the determination module, is configured to generate a large model execution sequence load based on the program control flow graph execution path sequence.
10. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory, wherein the processor, when executing the computer program, performs: The method for generating large model execution sequence loads as described in any one of claims 1 to 7; or, The method for large model simulation training as described in claim 8.