Graph interrupt optimization method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to a graph interrupt optimization method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: capturing a calculation process of a target model, and generating a calculation graph of the target model; for graph interruptions occurring in the generation process of the calculation graph, identifying interruption generation reasons of the graph interruptions; eliminating the graph interrupt according to the interrupt generation reason; in response to the graph interrupt elimination, the computational graph is updated.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a graph interruption optimization method, a graph interruption optimization device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (such as ChatGPT) have demonstrated excellent performance in natural language processing, machine translation, text generation and other fields. However, the complexity and scale of large language models also bring huge training costs and resource requirements. Large language models usually require expensive multi-card multi-node GPU (Graphics Processing Unit) clusters for training, which not only increases hardware costs, but also puts higher requirements on training efficiency. When training large language models, the open source systems in related technologies usually have low machine utilization, resulting in waste of resources.

[0003] To address these challenges, the related technology introduces TorchCompile into the Megatron-LM framework, and graph optimization technology can further improve the performance of distributed training. Although TorchCompile provides significant performance improvements, in practical applications, TorchDynamo will create graph breaks, which limits the performance improvement of TorchCompile in the Megatron-LM framework. Summary of the invention

[0004] The present disclosure provides a technical solution for graph interruption optimization.

[0005] According to one aspect of the present disclosure, a graph interruption optimization method is provided, comprising:

[0006] Capturing the computation process of the target model and generating a computation graph of the target model;

[0007] For a graph interruption that occurs during the generation of the computation graph, identifying a cause of the graph interruption;

[0008] Eliminate the graph interruption according to the cause of the interruption;

[0009] In response to the graph interruption being eliminated, updating the computation graph.

[0010] In a possible implementation, eliminating the graph interruption according to the cause of the interruption includes:

[0011] In response to the interruption being caused by an application programming interface collective communication operation not supporting capture into a computation graph, the collective communication operation is replaced with an equivalent collective communication operation supporting capture into a computation graph.

[0012] In a possible implementation, the collective communication operation does not support being captured into the computation graph and includes:

[0013] The collective communication operation is not supported by the static graph capture tool, or the collective communication operation is incompatible with the static graph execution mode.

[0014] In a possible implementation, eliminating the graph interruption according to the cause of the interruption includes:

[0015] In response to the interruption being caused by a custom operation, obtaining description information of the custom operation, wherein the description information of the custom operation includes at least part of an operation type, an input / output format, and an execution logic of the custom operation;

[0016] According to the description information of the custom operation, the custom operation is registered in the deep learning framework.

[0017] In a possible implementation, eliminating the graph interruption according to the cause of the interruption includes:

[0018] In response to the interrupt generation cause being an assertion check, the assertion check is temporarily removed or disabled, and before the input data participates in automatic differentiation calculation, the memory layout of the input data is converted to a continuous one.

[0019] In a possible implementation, eliminating the graph interruption according to the cause of the interruption includes:

[0020] In response to the interruption being caused by the original code containing preset context environment setting information, the original code is replaced by an equivalent code that does not contain the preset context environment setting information.

[0021] In one possible implementation,

[0022] After identifying the interruption cause of the graph interruption and before eliminating the graph interruption according to the interruption cause, the method further includes: storing the interruption cause in a preset set, wherein the preset set is used to automatically remove repeated interruption causes;

[0023] Eliminating the graph interruption according to the interruption cause includes: for any interruption cause in the preset set, eliminating all graph interruptions corresponding to the interruption cause in batches according to the interruption cause.

[0024] In a possible implementation, eliminating the graph interruption according to the cause of the interruption includes:

[0025] In response to the interruption generation cause belonging to a preset interruption generation cause set, the diagram interruption is eliminated according to the interruption generation cause.

[0026] According to one aspect of the present disclosure, a graph interruption optimization device is provided, comprising:

[0027] A generation module, used to capture the calculation process of the target model and generate a calculation graph of the target model;

[0028] An identification module, used for identifying a cause of a graph interruption that occurs during the generation process of the computation graph;

[0029] A eliminating module, used for eliminating the graph interruption according to the cause of the interruption;

[0030] An updating module is used to update the computational graph in response to elimination of the graph interruption.

[0031] In a possible implementation, the elimination module is used to:

[0032] In response to the interruption being caused by an application programming interface collective communication operation not supporting capture into a computation graph, the collective communication operation is replaced with an equivalent collective communication operation supporting capture into a computation graph.

[0033] In a possible implementation, the collective communication operation does not support being captured into the computation graph and includes:

[0034] The collective communication operation is not supported by the static graph capture tool, or the collective communication operation is incompatible with the static graph execution mode.

[0035] In a possible implementation, the elimination module is used to:

[0036] In response to the interruption being caused by a custom operation, obtaining description information of the custom operation, wherein the description information of the custom operation includes at least part of an operation type, an input / output format, and an execution logic of the custom operation;

[0037] According to the description information of the custom operation, the custom operation is registered in the deep learning framework.

[0038] In a possible implementation, the elimination module is used to:

[0039] In response to the interrupt generation cause being an assertion check, the assertion check is temporarily removed or disabled, and before the input data participates in automatic differentiation calculation, the memory layout of the input data is converted to a continuous one.

[0040] In a possible implementation, the elimination module is used to:

[0041] In response to the interruption being caused by the original code containing preset context environment setting information, the original code is replaced by an equivalent code that does not contain the preset context environment setting information.

[0042] In one possible implementation,

[0043] The device further comprises: a storage module, configured to store the interruption generation cause in a preset set, wherein the preset set is used to automatically remove repeated interruption generation causes;

[0044] The elimination module is used for: for any interruption generation cause in the preset set, according to the interruption generation cause, batch eliminating all graph interruptions corresponding to the interruption generation cause.

[0045] In a possible implementation, the elimination module is used to:

[0046] In response to the interruption generation cause belonging to a preset interruption generation cause set, the diagram interruption is eliminated according to the interruption generation cause.

[0047] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0048] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0049] According to one aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0050] In an embodiment of the present disclosure, the calculation process of the target model is captured to generate a calculation graph of the target model. For graph interruptions that occur during the generation of the calculation graph, the cause of the interruption is identified, the graph interruption is eliminated according to the cause of the interruption, and the calculation graph is updated in response to the elimination of the graph interruption. Thus, by eliminating the graph interruption, more operators can be included in the calculation graph, that is, the number of subgraphs of the calculation graph can be reduced, thereby further improving the graph optimization capability. By adopting the embodiment of the present disclosure, a more continuous and complete calculation graph can be generated. A continuous calculation graph can improve execution efficiency, reduce delays and overhead during training, and thus improve the performance of model training. In addition, the embodiment of the present disclosure can reduce the additional calculations and memory usage caused by graph interruptions, so that computing resources are more efficiently utilized. In addition, the embodiment of the present disclosure is scalable and can adapt to different hardware platforms and training environments.

[0051] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0052] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0054] Figure 1 A flowchart of a graph interruption optimization method provided by an embodiment of the present disclosure is shown.

[0055] Figure 2a and Figure 2b The end-to-end training performance and convergence after graph interruption optimization using the graph interruption optimization method provided by the embodiment of the present disclosure are shown.

[0056] Figure 3 A block diagram of a graph interruption optimization device provided by an embodiment of the present disclosure is shown.

[0057] Figure 4 A block diagram of an electronic device 1900 provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0059] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0060] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0061] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0062] Related technologies In order to improve the training efficiency of large language models, an open source deep learning optimization library called Megatron-LM was proposed. This framework accelerates the training process of large language models through mixed precision, data parallelism, tensor parallelism and pipeline parallelism (i.e. 3D (3 Dimensions) parallelism), recomputation, pre-training such as ZeRO, and other methods. However, the runtime stage of the Megatron-LM framework inherits the eager mode of PyTorch by default, and its training throughput and memory usage efficiency are not optimal.

[0063] PyTorch 2.0 introduces a static graph mode, TorchCompile, which includes three components: TorchDynamo, AOTAutograd, and TorchInductor. TorchDynamo is responsible for converting Python code into a static graph, AOTAutograd handles automatic differentiation, and TorchInductor is used for distributed training. Introducing TorchCompile into the Megatron-LM framework can further improve the performance of distributed training based on graph optimization technology.

[0064] Although TorchCompile provides significant performance improvements, in practice, TorchDynamo cannot capture all functions into a computational graph. When encountering unsupported operators, TorchDynamo creates a graph break, splits the computational graph into several subgraphs it can support, and returns the Python interpreter to execute the operators it cannot handle. This graph break limits the performance improvement of TorchCompile in the Megatron-LM framework.

[0065] Graph interruption refers to the inability to include certain operations in the automatically generated computational graph during deep learning model training, especially when using graph compilation technology (such as TorchDynamo of PyTorch), because some operations do not support automatic differentiation or graph optimization. In this case, the computational graph will be interrupted and some parts need to be executed through the Python interpreter instead of in the optimized computational graph. Since each subgraph needs to be executed separately, if the computational graph contains more subgraphs, it will lead to reduced execution efficiency and increase latency and overhead during training. In addition, the increase in the number of subgraphs may also lead to additional calculations and memory usage, affecting the effective use of resources, and thus reducing the performance of model training.

[0066] The inventor of this application found that the related technology has certain limitations when dealing with graph interruptions. TorchDynamo has problems in dealing with collective communication, custom fusion operators, Python assertions, etc., which leads to frequent graph interruptions. These problems not only affect the training efficiency, but also increase the workload of developers during debugging and optimization.

[0067] In order to overcome these limitations, a method is needed to optimize and reduce the number of graph interruptions generated by TorchDynamo in the Megatron-LM framework. The disclosed embodiment provides a graph interruption optimization method, which captures the calculation process of the target model, generates the calculation graph of the target model, identifies the interruption cause of the graph interruption that occurs during the generation process of the calculation graph, eliminates the graph interruption according to the interruption cause, and updates the calculation graph in response to the elimination of the graph interruption, thereby eliminating the graph interruption, so that more operators can be included in the calculation graph, that is, the number of subgraphs of the calculation graph can be reduced, thereby further improving the graph optimization capability. By adopting the disclosed embodiment, a more continuous and complete calculation graph can be generated. A continuous calculation graph can improve execution efficiency, reduce delays and overhead during training, and thus improve the performance of model training. In addition, the disclosed embodiment can reduce the additional calculation and memory usage caused by graph interruption, so that computing resources are more effectively utilized. In addition, the disclosed embodiment is scalable and can adapt to different hardware platforms and training environments.

[0068] The graph interruption optimization method provided by the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0069] Figure 1 A flow chart of a graph interruption optimization method provided by an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the graph interruption optimization method may be a graph interruption optimization apparatus. For example, the graph interruption optimization method may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device or a wearable device, etc. In some possible implementations, the graph interruption optimization method may be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the graph interruption optimization method includes steps S11 to S14.

[0070] In step S11, the calculation process of the target model is captured and a calculation graph of the target model is generated.

[0071] In step S12, for a graph interruption occurring during the generation process of the computation graph, the cause of the graph interruption is identified.

[0072] In step S13, the graph interruption is eliminated according to the cause of the interruption.

[0073] In step S14, in response to the graph interruption being eliminated, the computation graph is updated.

[0074] The graph interruption optimization method provided in the embodiment of the present disclosure does not rely on a specific deep learning framework, but focuses on the construction and optimization of the computational graph. Therefore, the graph interruption optimization method provided in the embodiment of the present disclosure has cross-framework versatility. The graph interruption optimization method provided in the embodiment of the present disclosure can be applied to a variety of deep learning frameworks, such as Megatron-LM, PyTorch, TensorFlow, etc., which are not limited here. Most deep learning frameworks support the construction of computational graphs and face the problem of graph interruption. The graph interruption optimization method provided in the embodiment of the present disclosure improves the continuity and integrity of the computational graph by identifying and eliminating graph interruptions, which is beneficial to all deep learning frameworks that use computational graphs.

[0075] The graph interruption optimization method provided in the embodiment of the present disclosure can be applied to a variety of training chips. Among them, the training chip can represent a processor for machine learning and deep learning model training. Since the cause of the graph interruption is independent of the chip type, as long as the user code on the framework side is the same, the cause of the graph interruption is the same. Therefore, the graph interruption optimization method provided in the embodiment of the present disclosure can be applied to a variety of training chips such as CUDA devices, MTT GPUs (such as MUSA devices).

[0076] In the disclosed embodiment, the target model may be any model for which a computational graph is to be generated. In one possible implementation, the target model may be a deep learning model. In one example, the target model may be a large language model in the field of deep learning that needs to be trained and optimized.

[0077] The computation process of the target model may refer to a series of operations performed by the target model when performing forward propagation or forward and backward propagation. These operations may include but are not limited to matrix multiplication, addition, activation functions, etc. Capturing the computation process of the target model may mean recording or tracking all computation steps during the execution of the target model. This can be achieved programmatically, such as using the tracing function of the deep learning framework.

[0078] A computational graph is a data structure that can be used to represent all operations and data dependencies in a computational process. A computational graph can be composed of nodes and edges. Among them, a node can represent an operation, and an edge can represent a data flow, such as a tensor. In the disclosed embodiment, a specific tool or framework can be used to convert the computational process of the target model into the form of a computational graph. This computational graph can be used for subsequent analysis, optimization, and execution.

[0079] In one possible implementation, a static graph capture tool can be used to capture the calculation process of the target model and generate a calculation graph of the target model. The static graph capture tool can be used to convert dynamic operations in the code into static calculation graphs. For example, on the Megatron-LM framework, TorchDynamo in TorchCompile can be used to capture the calculation graph. Of course, other AI compiler backends can also be used to capture calculation graphs, which are not limited here.

[0080] In the disclosed embodiment, in the process of generating the computation graph of the target model, all graph interruption information can be captured. Specific tools or methods can be used to identify graph interruptions and record graph interruption information, such as the cause of the interruption, the location of the graph interruption, etc. The purpose of capturing graph interruption information is to better understand and analyze the construction process of the computation graph, so as to find a method to optimize the computation graph, reduce graph interruptions, and improve computational efficiency.

[0081] In one possible implementation, a specific API (Application Programming Interface) or tool provided by a deep learning framework can be used to capture graph interruption information. For example, in PyTorch, the torch._dynamo.explain interface can be used to obtain graph interruption information. The torch._dynamo.explain interface is an interface provided by PyTorch that can be used to explain and debug the graph optimization process in TorchDynamo.

[0082] In a possible implementation manner, the cause of the graph interruption and the location of the graph interruption may be determined according to information provided by the stack trace.

[0083] In a possible implementation, a ParseDynamoExplain class can be implemented based on the torch._dynamo.explain interface. The ParseDynamoExplain class is a custom class that can be used to parse the graph interruption information output by the torch._dynamo.explain interface.

[0084] In a possible implementation, the break_reasons attribute of the ParseDynamoExplain class can be used to obtain the interruption reason of each graph interruption.

[0085] In a possible implementation, it can be used to deduplicate the information of the graph interruption to avoid the graph interruption with the same interruption cause being recorded multiple times, and the deduplicated graph interruption information can be stored in a file.

[0086] In a possible implementation, the graph interruption information may be saved to a file through the dump function of the ParseDynamoExplain class.

[0087] In a possible implementation, in distributed training, the information of graph interruptions generated by each training node (rank) can be exported to separate files.

[0088] In a possible implementation, after identifying the interruption cause of the graph interruption and before eliminating the graph interruption according to the interruption cause, the method further includes: storing the interruption cause in a preset set, wherein the preset set is used to automatically remove duplicate interruption causes; and eliminating the graph interruption according to the interruption cause includes: for any interruption cause in the preset set, batch eliminating all graph interruptions corresponding to the interruption cause according to the interruption cause.

[0089] As an example of this implementation, the break_reasons attribute of the ParseDynamoExplain class can be used to store the break reasons of all graph breaks in a preset collection (eg, a set in Python).

[0090] In this implementation, the preset set has a deduplication mechanism, that is, the preset set can automatically filter out repeated interruption causes. Therefore, the preset set can be used to record all different interruption causes.

[0091] In this implementation, for each interruption cause in the preset set, corresponding measures can be taken to eliminate these interruptions.

[0092] In this implementation, the interruption causes are stored in a preset set, wherein the preset set is used to automatically remove duplicate interruption causes, and for any interruption cause in the preset set, all graph interruptions corresponding to the interruption cause are batch eliminated according to the interruption cause, thereby enabling batch processing of graph interruptions with the same interruption cause, thereby improving the efficiency of graph interruption optimization.

[0093] In a possible implementation, at least part of the following may be obtained: the number of subgraphs of the computation graph, the number of graph interruptions, and the number of operations of the target model.

[0094] As an example of this implementation, the number of subgraphs of the computation graph may be obtained. For example, the number of subgraphs of the computation graph may be obtained through the graph_count attribute of the ParseDynamoExplain class.

[0095] As an example of this implementation, the number of graph breaks may be obtained. For example, the number of graph breaks may be obtained through the graph_break_count attribute of the ParseDynamoExplain class.

[0096] As an example of this implementation, the number of operations of the target model may be obtained. For example, the number of operations of the target model may be obtained through the op_count attribute of the ParseDynamoExplain class.

[0097] In this implementation, by obtaining the number of subgraphs and graph interruptions of the computation graph, developers can evaluate the continuity and integrity of the computation graph. Fewer subgraphs and graph interruptions usually mean that the computation graph is more continuous, which helps improve execution efficiency. By obtaining the number of operations of the target model, developers can better understand the structure and potential computational load of the target model.

[0098] In a possible implementation, if the constructor is configured to output guard functions and compile times, when the instance of the ParseDynamoExplain class performs related operations, the information of all used guard functions and the compile times of these operations can also be recorded and output.

[0099] In the disclosed embodiments, the interruption cause of the graph interruption may be that the collective communication operation does not support capture into the computational graph, there are custom operations, assertion checks, random number seed synchronization issues, the original code contains preset context environment setting information, there are no view tensor (Viewless Tensor) memory operations, etc., which are not limited here. For example, the interruption cause of the graph interruption on the CUDA device may be that the collective communication operation does not support capture into the computational graph, there are custom operations, assertion checks, or random number seed synchronization issues. For another example, the interruption cause of the graph interruption on the MUSA device may be that the original code contains preset context environment setting information or there are no view tensor memory operations.

[0100] In a possible implementation, eliminating the graph interruption according to the interruption cause includes: in response to the interruption cause belonging to a preset interruption cause set, eliminating the graph interruption according to the interruption cause.

[0101] The preset interruption generation cause set may include at least one interruption generation cause, and the graph interruption corresponding to the interruption generation cause in the preset interruption generation cause set is a graph interruption that can be eliminated.

[0102] For example, the interruption causes in the preset interruption cause set include: collective communication operations do not support capture into the computational graph, there are custom operations, assertion checks, and the original code contains preset context environment setting information.

[0103] In this implementation, the identified interruption cause can be matched with a preset interruption cause set to determine whether it can be eliminated. For a successfully matched graph interrupt, corresponding elimination measures are taken according to the interruption cause.

[0104] In this implementation, a set of preset interrupt cause causes provides a clear set of guidelines, simplifying the development and debugging process.

[0105] In one possible implementation, eliminating the graph interruption according to the cause of the interruption includes: in response to the cause of the interruption being that the collective communication operation does not support capture into the computational graph, replacing the collective communication operation with an equivalent collective communication operation that supports capture into the computational graph.

[0106] In deep learning, collective communication is an operation that synchronizes data between multiple computing nodes and is often used in distributed training. For example, operations such as all_reduce, all_gather, and reduce_scatter can help average gradients or synchronize model parameters between different GPUs or computing nodes.

[0107] TorchDynamo is a static graph capture tool that converts operations in Python code into a static computational graph to optimize execution efficiency. However, TorchDynamo may encounter problems when processing certain collective communication primitives, resulting in the inability to capture these operations into the computational graph, resulting in graph interruptions.

[0108] For example, the Megatron-LM framework uses a lot of collective communications such as all_reduce, all_gather, and reduce_scatter to implement tensor parallelism. TorchDynamo cannot capture such operations into the computational graph, resulting in a large number of graph interrupts.

[0109] For example, the collective communication operations that do not support capture into the computation graph include torch.distributed.all_reduce, torch.distributed._all_gather_base, and torch.distributed._reduce_scatter_base. These collective communication operations that do not support capture into the computation graph can be replaced with equivalent collective communication operations that support capture into the computation graph. For example, these collective communication operations that do not support capture into the computation graph can be replaced with equivalent APIs provided by torch.distributed._functional_collectives (referred to as distf). Among them, _functional_collective can represent a library of collective communication operations that support capture into the computation graph. For example, torch.distributed.all_reduce(…) can be replaced with distf.all_reduce(…), torch.distributed._all_gather_base(…) can be replaced with distf.all_gather_tensor(…), and torch.distributed._reduce_scatter_base(…) can be replaced with distf.reduce_scatter_tensor(…).

[0110] In an example, torch.distributed.all_reduce, torch.distributed._all_gather_base, and torch.distributed._reduce_scatter_base can be globally searched in Megatron-LM and corresponding replacement operations can be performed so that all collective communication operations can be correctly captured by TorchDynamo and included in the computational graph.

[0111] In this implementation, by replacing the collective communication operations that are not supported to be captured in the computational graph, the graph interruptions that occur during the construction of the computational graph are reduced, thereby generating a more continuous and complete computational graph. The replacement operation makes the code of the deep learning framework (such as the Megatron-LM framework) more compatible with static graph capture tools (such as the TorchDynamo tool), simplifying the migration and use process on the new tool.

[0112] In a possible implementation, the collective communication operation does not support being captured into a computational graph, including: the collective communication operation is not supported by a static graph capture tool, or the collective communication operation is incompatible with a static graph execution mode.

[0113] As an example of this implementation, the collective communication operation does not support being captured into the computational graph, including: the collective communication operation is not supported by the static graph capture tool. The collective communication operation is not supported by the static graph capture tool, which may mean that some collective communication operations are not designed with compatibility with static graph capture tools (such as TorchDynamo) in mind, so the static graph capture tool cannot recognize and process these collective communication operations, resulting in the inability to include the relevant operations in the generated computational graph.

[0114] In this example, in response to the interruption being caused by the collective communication operation not being supported by the static graph capture tool, the collective communication operation may be replaced with an equivalent collective communication operation that supports capture into the computation graph.

[0115] As an example of this implementation, the collective communication operation does not support capture into the computational graph, including: the collective communication operation is incompatible with the static graph execution mode. The collective communication operation does not support capture into the computational graph, which may mean that even if some collective communication operations can be technically identified, their execution method may not match the requirements of the static graph execution mode, such as requiring dynamic execution or relying on certain specific environments at runtime, which makes them unsuitable for execution in a static graph.

[0116] In this example, in response to the interruption being caused by the collective communication operation being incompatible with the static graph execution mode, the collective communication operation may be replaced with an equivalent collective communication operation that supports capture into the computation graph.

[0117] By adopting this implementation, collective communication operations can be correctly captured into the computational graph and run efficiently in the static graph execution mode. This helps reduce graph interruptions and improves the performance and resource utilization of model training.

[0118] In one possible implementation, eliminating the graph interruption according to the cause of the interruption includes: in response to the cause of the interruption being the existence of a custom operation, obtaining description information of the custom operation, wherein the description information of the custom operation includes at least part of the operation type, input and output formats, and execution logic of the custom operation; and registering the custom operation in a deep learning framework according to the description information of the custom operation.

[0119] For example, in a deep learning framework, custom operations (such as apex.FusedRMSNormAffineFunction) may use custom CUDA kernel functions to implement their forward and backward propagation, and bind them to the Python side through pybind. When TorchDynamo tries to capture these custom operations, since they are bound through pybind, TorchDynamo cannot track the CUDA kernel function implementation inside these custom operations, resulting in graph interruptions at these points.

[0120] In this implementation, in response to the interruption being caused by the existence of a custom operation, the description information of the custom operation can be obtained. The description information of the custom operation can include at least part of the operation type, input and output format, and execution logic of the custom operation. For example, the operation type of the custom operation can be a matrix operation, a convolution operation, or another type of operation. The input and output format of the custom operation can refer to what format of data is required for the input of the custom operation, and what format of data will be generated by the output. The execution logic of the custom operation can refer to the specific calculation process or algorithm of the custom operation. After obtaining the description information of the custom operation, the custom operation can be registered in the deep learning framework according to the description information of the custom operation. For example, the custom operation can be registered as a torch.op. The torch.op is a mechanism for registering custom operations in the deep learning framework, allowing developers to expose custom implemented operations to the automatic differentiation system and the static graph capture tool. After registering fused_layer_norm_cuda.rms_forward_affine as torch.op, when TorchDynamo captures this operation, it will be regarded as an indivisible minimum unit and its internal implementation will not be further tracked.

[0121] In this implementation, by responding to the fact that the cause of the interruption is the existence of a custom operation, the custom operation is registered in the deep learning framework, thereby solving the problem that the static graph capture tool cannot capture the internal implementation of the custom operation, thereby eliminating the graph interruption and improving the efficiency and performance of deep learning model training.

[0122] In one possible implementation, eliminating the graph interruption according to the cause of the interruption includes: in response to the cause of the interruption being an assertion check, temporarily removing or disabling the assertion check, and converting the memory layout of the input data to continuous before the input data participates in automatic differentiation calculation.

[0123] For example, in Megatron-LM, when using the sequence parallel feature, the input data of the model is split (scattered) on the sequence dimension in the forward propagation phase and aggregated through the all_gather operation in the backward propagation phase. In the implementation of the _gather_along_first_dim function, the all_gather_tensor operation of functional_collective is used. In the implementation of the PyTorch community, the assertion tensor.is_contiguous() assertion is used to check whether the input data is stored continuously in memory. If the input data is not continuous, it will trigger an assertion failure, resulting in a graph interruption.

[0124] In this implementation, in response to the interruption being caused by assertion checking, the assertion checking may be temporarily removed or disabled. As an example of this implementation, the assertion checking statement may be temporarily surrounded by comment symbols (such as # in Python) in the code so that the compiler or interpreter ignores the assertion checking when executing the code, thereby avoiding graph interruption caused by assertion failure.

[0125] In addition, in this implementation, in response to the interrupt being caused by an assertion check, the memory layout of the input data can be converted to continuous before the input data participates in the automatic differentiation calculation. As an example of this implementation, while temporarily removing or disabling the assertion check, in order to ensure that the input data can correctly participate in the automatic differentiation calculation, the memory layout of the input data can be converted to continuous using the contiguous() method before the input data enters the automatic differentiation system. The contiguous() method returns a copy of the tensor with a continuous layout, ensuring that the data is tightly arranged in memory, meeting the requirements of operations such as all_gather.

[0126] By adopting this implementation, graph interruptions caused by assertion checks can be eliminated, so that the computational graph remains continuous at these points, allowing static graph capture tools such as TorchDynamo to smoothly capture and optimize the computational graph. This implementation allows the construction of the computational graph to be avoided while maintaining sequential parallel functionality due to assertion checks.

[0127] In one possible implementation, eliminating the graph interruption according to the cause of the interruption includes: in response to the cause of the interruption being that the original code contains preset context environment setting information, replacing the original code with an equivalent code that does not contain the preset context environment setting information.

[0128] For example, in PyTorch, some operations may require specific context environment setup information to execute correctly. For example, torch.nn.functional.scaled_dot_product is a flashattention API implemented by the PyTorch community. When using this interface, users usually need to set the flash kernel version through a specific context. For example, you can use the context manager of torch.backends.cuda.sdp_kernel to specify which flash kernel version to use. However, when static graph capture tools (such as TorchDynamo) try to capture code that uses the above context, graph interrupts will occur at these points because the static graph capture tools cannot handle the context manager.

[0129] In this implementation, in response to the interruption being caused by the original code containing preset context environment setting information, the original code can be replaced by an equivalent code that does not contain the preset context environment setting information, thereby eliminating the graph interruption caused by the context environment setting information, making the computational graph more complete, and improving the efficiency and performance of model training.

[0130] In one possible implementation, the cause of the interruption may be a random number seed synchronization problem. Large language models often include dropout, etc. In deep learning models, dropout is a regularization technique that prevents overfitting by randomly dropping (zeroing) some neurons in the network. The randomness of the dropout layer comes from the random number seed. In large-scale model training, in order to fully utilize the computing power of multiple GPUs, tensor parallel technology can be used to distribute different parts of the model to different GPUs. When using tensor parallel technology, it is necessary to ensure that the dropout layer on each GPU generates the same random number seed to ensure the consistency of the calculation results on different GPUs. In the Megatron-LM framework, the tensor_parallel.get_cuda_rng_tracker().fork() method can be used to synchronize random number seeds on different GPUs, and a context manager (such as Python's context manager) can be used to ensure that the consistency of random number generation is maintained in different calculation branches. However, static graph capture tools (such as TorchDynamo) cannot handle the context for synchronizing random number seeds, and cannot include this part of the logic in the static graph, resulting in graph interruption.

[0131] In one possible implementation, the interruption may be caused by viewless tensor memory operations. In deep learning training, the gradient information related to these parameters is usually cleared after the parameters are updated. The default approach of PyTorch is to directly release the memory of these gradients so that they can be re-applied the next time the gradients are calculated, which can avoid memory leaks. However, this frequent memory release and application operation will incur additional overhead), affecting training efficiency. Megatron-LM uses a different approach to handle gradient clearing when implementing pipeline parallelism. It does not actually release the memory of the gradient tensor, but sets the data attribute of the gradient tensor to a scalar, thereby avoiding repeated memory allocation and release operations and reducing the overhead of memory management. This tensor that modifies the data attribute but retains other attributes is called a viewless tensor. However, if there is a direct operation on the tensor memory in the code, such as out.data = inp.data, the static graph capture tool (such as TorchDynamo) will not be able to handle this operation because it violates the immutability principle of the static graph, resulting in graph interruption at these points.

[0132] In the embodiment of the present disclosure, after the graph interruption is eliminated, the computation graph may be updated. The graph interruption elimination may mean that all graph interruptions that can be eliminated are eliminated, for example, all graph interruptions whose interruption causes belong to a preset interruption cause set are eliminated. Updating the computation graph may mean regenerating the computation graph.

[0133] The graph interruption optimization method provided in the embodiments of the present disclosure can be applied to technical fields such as GPU, AI, distributed training, large language models, Megatron-LM, TorchCompile, etc., without limitation herein. For example, if a user of Megatron-LM+TorchDynamo finds a graph interruption in his or her code, the graph interruption optimization method provided in the embodiments of the present disclosure can be used to eliminate the graph interruption, thereby improving the end-to-end training performance of the model.

[0134] The graph interruption optimization method provided by the embodiment of the present disclosure is described below through a specific application scenario. The application scenario can be applied to the Megatron-LM framework, and in the application scenario, the target model can be a large language model.

[0135] You can use TorchDynamo in TorchCompile on the Megatron-LM framework to capture the calculation process of the target model and generate the calculation graph of the target model. In the process of generating the calculation graph of the target model, you can use the torch._dynamo.explain interface to obtain all the information of graph interruptions. The information of graph interruptions may include the cause of the interruption of the graph, the location of the interruption of the graph, etc. The cause of the interruption of the graph may be that the collective communication operation does not support capture into the calculation graph, there are custom operations, assertion checks, random number seed synchronization problems, the original code contains preset context environment setting information, there is no view tensor memory operation, etc.

[0136] A ParseDynamoExplain class can be implemented based on the torch._dynamo.explain interface. The ParseDynamoExplain class can be used to parse the graph interruption information output by the torch._dynamo.explain interface. The break_reasons attribute of the ParseDynamoExplain class can be used to obtain the interruption cause of each graph interruption, and the interruption causes of all graph interruptions can be stored in a preset collection (such as a set in Python). The preset collection has a built-in deduplication mechanism, that is, the preset collection can automatically filter out duplicate interruption causes.

[0137] The interruption causes in the preset interruption cause set may include: collective communication operations do not support capture into the computational graph, custom operations exist, assertion checks, and the original code contains preset context environment setting information. The identified interruption causes can be matched with the preset interruption cause set to determine whether they can be eliminated. For graph interrupts that are successfully matched, corresponding elimination measures are taken according to the interruption causes.

[0138] For example, if the interruption is caused by the fact that the collective communication operation does not support capture into the computational graph, the collective communication operation can be replaced with an equivalent collective communication operation that supports capture into the computational graph; if the interruption is caused by the existence of a custom operation, the custom operation can be registered in the deep learning framework; if the interruption is caused by an assertion check, the assertion check can be temporarily removed or disabled, and the memory layout of the input data can be converted to continuous before the input data participates in the automatic differentiation calculation; if the interruption is caused by the original code containing preset context environment setting information, the original code can be replaced with an equivalent code that does not contain the preset context environment setting information.

[0139] After all graph interruptions whose interruption causes belong to a preset interruption cause set are eliminated, the computational graph of the target model may be regenerated.

[0140] Figure 2a and Figure 2b The end-to-end training performance and convergence after the graph interruption optimization method provided by the embodiment of the present disclosure is used to perform graph interruption optimization are shown. Figure 2a and Figure 2b In , the target model is the large language model LLaMA2-7B. Figure 2a In , the horizontal axis represents the number of steps (step), and the vertical axis represents the value of the loss function. Figure 2b In the figure, the horizontal axis represents the number of samples, and the vertical axis represents the value of the loss function. By adopting the graph interruption optimization method provided by the embodiment of the present disclosure, more samples can be trained in one step (ie, one step), thereby improving the training performance of the target model.

[0141] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0142] In addition, the present disclosure also provides a graph interruption optimization device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any graph interruption optimization method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records in the method part and will not be repeated here.

[0143] Figure 3 FIG. 1 is a block diagram of a graph interruption optimization device provided by an embodiment of the present disclosure. Figure 3 As shown, the graph interruption optimization device comprises:

[0144] A generation module 31, used to capture the calculation process of the target model and generate a calculation graph of the target model;

[0145] An identification module 32, for identifying a cause of a graph interruption that occurs during the generation of the computation graph;

[0146] The elimination module 33 is used to eliminate the graph interruption according to the cause of the interruption;

[0147] The updating module 34 is used to update the computation graph in response to the elimination of the graph interruption.

[0148] In a possible implementation, the elimination module 33 is used to:

[0149] In response to the interruption being caused by an application programming interface collective communication operation not supporting capture into a computation graph, the collective communication operation is replaced with an equivalent collective communication operation supporting capture into a computation graph.

[0150] In a possible implementation, the collective communication operation does not support being captured into the computation graph and includes:

[0151] The collective communication operation is not supported by the static graph capture tool, or the collective communication operation is incompatible with the static graph execution mode.

[0152] In a possible implementation, the elimination module 33 is used to:

[0153] In response to the interruption being caused by a custom operation, obtaining description information of the custom operation, wherein the description information of the custom operation includes at least part of an operation type, an input / output format, and an execution logic of the custom operation;

[0154] According to the description information of the custom operation, the custom operation is registered in the deep learning framework.

[0155] In a possible implementation, the elimination module 33 is used to:

[0156] In response to the interrupt generation cause being an assertion check, the assertion check is temporarily removed or disabled, and before the input data participates in automatic differentiation calculation, the memory layout of the input data is converted to a continuous one.

[0157] In a possible implementation, the elimination module 33 is used to:

[0158] In response to the interruption being caused by the original code containing preset context environment setting information, the original code is replaced by an equivalent code that does not contain the preset context environment setting information.

[0159] In one possible implementation,

[0160] The device further comprises: a storage module, configured to store the interruption generation cause in a preset set, wherein the preset set is used to automatically remove repeated interruption generation causes;

[0161] The elimination module 33 is used for: for any interruption generation cause in the preset set, according to the interruption generation cause, batch eliminating all graph interruptions corresponding to the interruption generation cause.

[0162] In a possible implementation, the elimination module 33 is used to:

[0163] In response to the interruption generation cause belonging to a preset interruption generation cause set, the diagram interruption is eliminated according to the interruption generation cause.

[0164] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0165] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

[0166] The embodiment of the present disclosure further provides a computer program, including a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above method.

[0167] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0168] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0169] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0170] Figure 4 1 is a block diagram of an electronic device 1900 provided in an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal. Figure 4 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0171] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (MacOS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0172] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0173] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0174] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0175] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0176] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0177] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0178] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0179] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0180] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0181] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0182] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0183] If the technical solution of the embodiments of the present disclosure involves personal information, the product using the technical solution of the embodiments of the present disclosure has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of the embodiments of the present disclosure involves sensitive personal information, the product using the technical solution of the embodiments of the present disclosure has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to collect his or her personal information; or on the device for processing personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0184] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A graph interruption optimization method, characterized in that: include: Capturing the computation process of the target model and generating a computation graph of the target model; For a graph interruption that occurs during the generation of the computation graph, identifying a cause of the graph interruption; Eliminate the graph interruption according to the cause of the interruption; In response to the graph interruption being eliminated, updating the computation graph.

2. The method according to claim 1, characterized in that The step of eliminating the graph interruption according to the cause of the interruption includes: In response to the interruption being caused by an application programming interface collective communication operation not supporting capture into a computation graph, the collective communication operation is replaced with an equivalent collective communication operation supporting capture into a computation graph.

3. The method according to claim 2, characterized in that The collective communication operations that are not supported in the computation graph include: The collective communication operation is not supported by the static graph capture tool, or the collective communication operation is incompatible with the static graph execution mode.

4. The method according to claim 1, characterized in that: The step of eliminating the graph interruption according to the cause of the interruption includes: In response to the interruption being caused by a custom operation, obtaining description information of the custom operation, wherein the description information of the custom operation includes at least part of an operation type, an input / output format, and an execution logic of the custom operation; According to the description information of the custom operation, the custom operation is registered in the deep learning framework.

5. The method according to claim 1, characterized in that The step of eliminating the graph interruption according to the cause of the interruption includes: In response to the interrupt generation cause being an assertion check, the assertion check is temporarily removed or disabled, and before the input data participates in automatic differentiation calculation, the memory layout of the input data is converted to a continuous one.

6. The method according to claim 1, characterized in that The step of eliminating the graph interruption according to the cause of the interruption includes: In response to the interruption being caused by the original code containing preset context environment setting information, the original code is replaced by an equivalent code that does not contain the preset context environment setting information.

7. The method according to any one of claims 1 to 6, characterized in that After identifying the interruption cause of the graph interruption and before eliminating the graph interruption according to the interruption cause, the method further includes: storing the interruption cause in a preset set, wherein the preset set is used to automatically remove repeated interruption causes; Eliminating the graph interruption according to the interruption cause includes: for any interruption cause in the preset set, eliminating all graph interruptions corresponding to the interruption cause in batches according to the interruption cause.

8. The method according to any one of claims 1 to 6, characterized in that: The step of eliminating the graph interruption according to the cause of the interruption includes: In response to the interruption cause belonging to a preset interruption cause set, the diagram interruption is eliminated according to the interruption cause.

9. A graph interruption optimization device, characterized in that: include: A generation module, used to capture the calculation process of the target model and generate a calculation graph of the target model; An identification module, used for identifying a cause of a graph interruption that occurs during the generation of the computation graph; A eliminating module, used for eliminating the graph interruption according to the cause of the interruption; An updating module is used to update the computational graph in response to elimination of the graph interruption.

10. An electronic device, characterized in that: include: one or more processors; a memory for storing executable instructions; The one or more processors are configured to call the executable instructions stored in the memory to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code is executed in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 8.

Images