Calculation graph generation method and device of neural network model, equipment and medium

Abstract

The embodiment of the invention provides a computational graph generation method and device of a neural network model, equipment and a medium, and relates to the technical field of deep learning. The computational graph generation method of the neural network model comprises the following steps: wrapping each to-be-tracked function in the neural network model, so that each to-be-tracked function is converted into a leaf function, and a target neural network model is obtained; and traversing each function in the target neural network model to perform tracking processing, and generating a computational graph of the neural network model. According to the embodiment of the invention, all functions in the neural network model can be quickly tracked to completely and accurately generate the computational graph, and the computational graph generation efficiency of the neural network model is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method, device, equipment and medium for generating a computational graph of a neural network model. Background Art

[0002] PyTorch FX is a powerful deep learning framework for processing neural network models at the intermediate representation (IR) level. PyTorch FX allows users to capture the behavior of neural network models as computational graphs. The fx.wrap function in PyTorch FX is a function wrapper. The fx.wrap function allows users to convert a neural network model into a functional representation using the FX intermediate representation (IR). By using the fx.wrap function, users can analyze, modify, or optimize the behavior of a neural network model at a more fine-grained level and generate a computational graph. Users can also transform and optimize the computational graph of a neural network model, thereby achieving tasks such as introspection of the neural network model, graph rewriting, and deployment optimization.

[0003] However, the fx.wrap function in native PyTorch FX has the following limitations:

[0004] 1. Limited flexibility: The fx.wrap function lacks the ability to handle different types of inputs, support multiple models or architectures, and adapt to different usage scenarios.

[0005] 2. Performance issues: The current implementation has performance issues, such as low computational efficiency and poor execution, which may affect the overall efficiency of the encapsulated model.

[0006] 3. Insufficient error handling and debugging support: The fx.wrap function does not provide enough error information or debugging information, making it difficult to accurately locate and solve problems when they occur.

[0007] It can be seen that it is difficult to quickly track all functions in the neural network model and generate the computational graph completely and accurately based on the native PyTorch FX processing neural network model, and the computational graph generation efficiency of the neural network model is low. Summary of the invention

[0008] The purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for generating a computational graph of a neural network model, so as to achieve the technical effect of quickly tracking all functions in the neural network model to completely and accurately generate a computational graph, thereby improving the efficiency of generating the computational graph of the neural network model.

[0009] In a first aspect, an embodiment of the present invention provides a method for generating a computational graph of a neural network model, comprising:

[0010] Each to-be-tracked function in the neural network model is packaged to convert each to-be-tracked function into a leaf function, thereby obtaining a target neural network model;

[0011] Traverse each function in the target neural network model for tracing processing to generate a computational graph of the neural network model.

[0012] In the above implementation process, by converting each function to be tracked in the neural network model into a leaf function respectively, the target neural network model is obtained, all functions in the target neural network model are traversed for tracking processing, and the calculation graph of the neural network model is generated. It can ensure that the leaf functions after the conversion of each function to be tracked are retained to create the calculation graph nodes, and the information of each function to be tracked is recorded in the calculation graph, thereby achieving the technical effect of quickly tracking all functions in the neural network model, completely and accurately generating the calculation graph, and improving the efficiency of the calculation graph generation efficiency of the neural network model.

[0013] Furthermore, traversing each function in the target neural network model for tracking processing to generate a computational graph of the neural network model specifically includes:

[0014] Traversing each function in the target neural network model, and determining whether the current function is a leaf function after transformation of any of the functions to be tracked;

[0015] When the current function is a leaf function converted from any of the functions to be tracked, the current function is captured, and a node for calling the current function to be tracked is created in the initial calculation graph; wherein the current function to be tracked is the function to be tracked corresponding to the current function;

[0016] When all functions in the target neural network model are traversed, the latest initial calculation graph is used as the calculation graph of the neural network model.

[0017] In the above implementation process, each function in the target neural network model is traversed for tracking processing. When traversing to a leaf function transformed from any function to be tracked, the current function is captured, and a node that calls the corresponding function to be tracked is created in the initial calculation graph. This can support identifying all leaf functions transformed from the functions to be tracked as nodes, generating a calculation graph completely and accurately, and further improving the calculation graph generation efficiency of the neural network model.

[0018] Furthermore, when the current function is a leaf function converted from any of the functions to be traced, the current function is captured, and a node for calling the current function to be traced is created in the initial calculation graph, specifically including:

[0019] When the current function is a leaf function corresponding to any of the functions to be traced, the current function is captured, and whether there is a Proxy class instance in the input parameters of the current function is queried;

[0020] If there is a Proxy class instance in the input parameters of the current function, the Proxy class instance is used to create a node in the initial calculation graph that calls the current function to be traced;

[0021] If there is no Proxy class instance in the input parameters of the current function, a Tracer class instance is used to create a node in the initial calculation graph that calls the current function to be traced.

[0022] In the above implementation process, in the process of tracing the target neural network model, when traversing to a leaf function transformed from any function to be traced and there is a Proxy class instance in its input parameter, the Proxy class instance is used to create a node in the initial calculation graph that calls the current function to be traced; when traversing to a leaf function transformed from any function to be traced but there is no Proxy class instance in its input parameter, the Tracer class instance is used to create a node in the initial calculation graph that calls the current function to be traced. This allows nodes to be created quickly in the initial calculation graph, further improving the calculation graph generation efficiency of the neural network model.

[0023] Furthermore, the traversing each function in the target neural network model for tracing processing to generate a computational graph of the neural network model specifically includes:

[0024] When the current function cannot be captured, the current function is added to the global variable of the current function to be traced.

[0025] In the above implementation process, by adding the current function to the global variable of the current function to be tracked when the current function cannot be captured, it can be ensured that the leaf functions transformed from all the functions to be tracked in the target neural network model are effectively captured.

[0026] Furthermore, the process of respectively wrapping each function to be tracked in the neural network model so that each function to be tracked is converted into a leaf function to obtain a target neural network model specifically includes:

[0027] The function to be traced is wrapped with a pre-designed decorator to obtain a leaf function after the function to be traced is transformed; or,

[0028] The wrapping function is called to wrap the function to be tracked, and obtain a leaf function after the function to be tracked is transformed.

[0029] In the above implementation process, by using a pre-designed decorator or directly calling the wrapping function to wrap the tracking function, it is possible to adapt to a variety of usage scenarios to effectively wrap the tracking function and further improve the efficiency of the computational graph generation of the neural network model.

[0030] Furthermore, the use of a pre-designed decorator to wrap the function to be traced to obtain a leaf function after the function to be traced is transformed specifically includes:

[0031] When the function to be traced is a user-defined function, the decorator is used to wrap the user-defined function to obtain a leaf function after the user-defined function is transformed;

[0032] When the function to be traced is a built-in function, the decorator is used to obtain the module object of the built-in function, to wrap the built-in function, and obtain a leaf function after the built-in function is converted;

[0033] When the function to be traced is a third-party library function, the decorator is used to obtain the module object of the third-party library function, to wrap the third-party library function, and to obtain a leaf function after the third-party library function is converted.

[0034] In the above implementation process, by using pre-designed decorators to perform different wrapping processing on different types of functions to be traced, it can ensure that each function to be traced is effectively converted into a leaf function, further improving the efficiency of the computational graph generation of the neural network model.

[0035] Furthermore, before the process of respectively wrapping each to-be-tracked function in the neural network model to convert each to-be-tracked function into a leaf function and obtaining the target neural network model, the process further includes:

[0036] In response to a function tracing request initiated by a user for the neural network model, all the functions to be traced selected by the user from all the functions of the neural network model are determined.

[0037] In the above implementation process, by responding to the function tracing request initiated by the user for the neural network model, all the functions to be traced selected by the user from all the functions of the neural network model are determined, thereby meeting the user's tracing needs for the neural network model.

[0038] In a second aspect, an embodiment of the present invention provides a computational graph generating device for a neural network model, comprising:

[0039] A function wrapping module is used to wrap each to-be-tracked function in the neural network model, so that each to-be-tracked function is converted into a leaf function to obtain a target neural network model;

[0040] The computational graph generation module is used to traverse each function in the target neural network model for tracking processing and generate a computational graph of the neural network model.

[0041] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the memory is coupled to the processor, and the processor implements the computational graph generation method of the neural network model as described above when executing the computer program.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the computational graph generation method of the neural network model as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A schematic diagram of a flow chart of a method for generating a computational graph of a neural network model provided in the first embodiment of the present invention;

[0045] Figure 2 A data flow diagram of a method for generating a computational graph of a neural network model exemplified in the first embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the structure of a computational graph generating device for a neural network model provided in a second embodiment of the present invention;

[0047] Figure 4 A schematic structural diagram of an electronic device provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0049] It should be noted that in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. At the same time, the step numbers in the text are only for the convenience of explaining the embodiments of the present invention and do not serve to limit the order of execution of the steps. The method provided in the embodiment of the present invention can be executed by a related terminal device, and the following description will be taken as an example of a processor as the execution subject.

[0050] In deep learning, a computational graph is a data structure used to represent the computational process of a neural network. It is usually composed of nodes and edges. Nodes represent variables and edges represent functions. The computational graph can help users better understand and debug neural networks.

[0051] At present, neural network models are often first converted into computational graphs, and then executable programs are generated through compilers to run the executable programs to complete various tasks such as target detection and target classification. In order to ensure that the executable programs are correct and usable, the neural network model must be completely and accurately converted into a computational graph.

[0052] Based on this, an embodiment of the present invention provides a method for generating a computational graph of a neural network model. The method for generating a computational graph of a neural network model replaces each function to be traced in the neural network model with a leaf function obtained after wrapping the function to be traced. In the process of tracing the neural network model, the leaf functions after the transformation of each function to be traced can be retained to create a computational graph node. All functions in the neural network model can be quickly traced to completely and accurately generate a computational graph, thereby improving the computational graph generation efficiency of the neural network model.

[0053] Please see Figure 1 , Figure 1 A flowchart of a method for generating a computational graph of a neural network model provided in the first embodiment of the present invention. The first embodiment of the present invention provides a method for generating a computational graph of a neural network model, comprising steps S101 to S102:

[0054] S101, respectively wrapping each to-be-tracked function in the neural network model, so that each to-be-tracked function is converted into a leaf function, and a target neural network model is obtained;

[0055] S102, traverse each function in the target neural network model for tracking processing, and generate a calculation graph of the neural network model.

[0056] As an exemplary embodiment, all functions to be tracked in the neural network model are obtained, and for each function to be tracked, the function to be tracked is packaged to convert the function to be tracked into a leaf function, thereby obtaining a target neural network model.

[0057] When the function to be traced is packaged, it will be converted into a new function, namely the "leaf function". A leaf function is a function that does not allocate stack space, does not call other functions, does not store non-volatile registers, and does not handle exceptions. When the leaf function converted from the function to be traced generates a computation graph, it represents the call of the function to be traced and becomes a node in the computation graph. Specifically, the leaf function converted from the function to be traced will record its input and output during execution and generate a corresponding computation graph node. This node can retain the call information of the function to be traced, including the input parameters, output results, and other relevant context information of the function to be traced.

[0058] Traverse each function in the target neural network model for tracing. When traversing to a leaf function transformed from any function to be traced, the leaf function after the function to be traced can be regarded as a side effect function and will not be deleted, allowing nodes to be created in the initial calculation graph, thereby generating a calculation graph for the neural network model.

[0059] The embodiment of the present invention converts each function to be tracked in the neural network model into a leaf function respectively to obtain a target neural network model, traverses all functions in the target neural network model for tracking processing, and generates a calculation graph of the neural network model. It can ensure that the leaf functions after the conversion of each function to be tracked are retained to create calculation graph nodes, and the information of each function to be tracked is recorded in the calculation graph, thereby achieving the technical effect of quickly tracking all functions in the neural network model, completely and accurately generating a calculation graph, and improving the efficiency of generating the calculation graph of the neural network model.

[0060] In an optional embodiment, before the process of respectively wrapping each function to be traced in the neural network model to convert each function to be traced into a leaf function to obtain the target neural network model, it also includes: responding to a function tracing request initiated by a user for the neural network model, and determining all functions to be traced selected by the user from all functions of the neural network model.

[0061] As an example, the user can freely select at least one function as a function to be traced from all functions of the neural network model according to his or her own tracing requirements for the neural network model, and initiate a function tracing request.

[0062] When receiving a function tracing request initiated by the user for the neural network model, respond to the function tracing request initiated by the user for the neural network model, and determine all functions to be traced selected by the user from all functions of the neural network model, such as custom functions, built-in functions, third-party library functions, etc.

[0063] The embodiment of the present invention responds to a function tracing request initiated by a user for a neural network model and determines all functions to be traced selected by the user from all functions of the neural network model, thereby meeting the user's tracing needs for the neural network model.

[0064] In an optional embodiment, each function to be tracked in the neural network model is wrapped separately to convert each function to be tracked into a leaf function to obtain a target neural network model, which specifically includes: using a pre-designed decorator to wrap the function to be tracked to obtain a leaf function after the function to be tracked is converted; or, calling a wrapping function to wrap the function to be tracked to obtain a leaf function after the function to be tracked is converted.

[0065] As an example, based on PyTorch FX, a decorator can be pre-designed. The interface of the decorator, such as custom_wrap, is a high-order function with parameters. The pre-designed decorator interface is called to wrap the function to be tracked, so that the function to be tracked is converted into a leaf function, and the leaf function after the conversion of the function to be tracked is obtained.

[0066] Alternatively, based on PyTorch FX, you can also directly call a wrapping function, such as a wrap function, to wrap the function to be tracked, so that the function to be tracked is converted into a leaf function, and the leaf function after the conversion of the function to be tracked is obtained.

[0067] The embodiment of the present invention uses a pre-designed decorator or directly calls the wrapping function to wrap the traced function, which can adapt to various usage scenarios and effectively wrap the traced function, thereby further improving the efficiency of computing graph generation of the neural network model.

[0068] In an optional embodiment, the method of using a pre-designed decorator to wrap the function to be tracked to obtain a leaf function after the function to be tracked is specifically as follows: when the function to be tracked is a custom function, the decorator is used to wrap the custom function to obtain a leaf function after the custom function is converted; when the function to be tracked is a built-in function, the decorator is used to obtain the module object of the built-in function to wrap the built-in function to obtain a leaf function after the built-in function is converted; when the function to be tracked is a third-party library function, the decorator is used to obtain the module object of the third-party library function to wrap the third-party library function to obtain a leaf function after the third-party library function is converted.

[0069] As an exemplary example, for each function to be traced, the function type of the function to be traced is obtained to determine whether the function to be traced is a user-defined function, a built-in function, or a third-party library function.

[0070] When the function to be tracked is a custom function, the decorator can be directly used to wrap the custom function, so that the custom function is converted into a leaf function, and the leaf function after the custom function is converted is obtained.

[0071] For example, for a custom function custom_function, the decorator interface custom_wrap is called to wrap the custom function custom_function. The processing methods include:

[0072] Method 1:

[0073] “#custom wrap method 1

[0074] @custom_wrap()

[0075] def custom_function():

[0076] …”.

[0077] Method 2:

[0078] “#custom wrap method 2

[0079] def custom_function(): ...

[0081] custom_wrap()(custom_function)".

[0082] Method 3:

[0083] “#custom wrap method 3

[0084] def custom_function(): ...

[0086] custom_wrap()('custom_function')".

[0087] Assuming that the user defines the subgraph segmentation position identifier, obtains the custom function set_break, calls the decorator interface custom_wrap, and wraps the custom function set_break, the processing process is as follows:

[0088] @custom_wrap()

[0089] def set_break(): ...

[0091] just a flag for the positon of the subgraph segmentation ...

[0093] pass".

[0094] Assume that the user has two files, one is model.py, which is used to define the neural network model, and the other is utils.py, which is used to define the utility function. Based on PyTorch FX, the ways to call the decorator interface custom_wrap include:

[0095] Method 1: Import the custom function set_break directly in the model.py file through the "from...import..." instruction, and call the decorator interface custom_wrap through the custom function set_break in the neural network model.

[0096] Method 2: Import the library package through the "import trace_utils as m" instruction in the model.py file, and call the decorator interface custom_wrap through the "m.set_break()" instruction in the neural network model.

[0097] Method 3: Import the custom function set_break through the "from...import..." instruction in the util.py file and define the module pre_process. Import the module pre_process through the "import util" instruction in the model.py file, and call the decorator interface custom_wrap through the "util.pre_process()" instruction in the neural network model. Or import the module pre_process through the "from util import pre_process" instruction, and call the decorator interface custom_wrap through the "pre_process()" instruction in the neural network model.

[0098] For method 3, customize a new module pre_process in the file util.py. The definition process of the module pre_process is as follows:

[0099] “Python

[0100] class pre_preocess(torch.nn.Module):

[0101] def forward(self,*args,**kwargs): ...

[0103] set_break()

[0104] …”.

[0105] By defining the module pre_process, users can capture "Module.__call__" and do monkey patch before execution to add the wrapped function to be tracked to the global variable.

[0106] Method 4: In the util.py file, import the library package through the "import trace_utils as m" instruction, call it through the "m.set_break()" instruction, and define the function pre_process. In the model.py file, import the function pre_process through the "importutil" instruction, and call the decorator interface custom_wrap through the "util.pre_process()" instruction in the neural network model, or import the function pre_process through the "from util import pre_process" instruction, and call the decorator interface custom_wrap through the "pre_process()" instruction in the neural network model.

[0107] When the function to be traced is a built-in function, it is necessary to first obtain the module object of the built-in function as a parameter, and wrap the built-in function according to the module object of the built-in function to convert the built-in function into a leaf function, thereby obtaining the leaf function after the conversion of the built-in function.

[0108] For example, for a built-in function, such as the print function, the module object builtins of the built-in function print is passed in as a parameter, and the built-in function print is packaged according to the module object builtins of the built-in function print. The processing methods include:

[0109] Method 1:

[0110] “import builtins

[0111] #custom wrap method 1

[0112] custom_wrap(builtins)(print)”.

[0113] Method 2:

[0114] “import builtins

[0115] #custom wrap method 2

[0116] custom_wrap(builtins)('print')".

[0117] When the function to be tracked is a third-party library function, it is necessary to first obtain the module object of the third-party library function as a parameter, package the third-party library function according to the module object of the third-party library function, convert the third-party library function into a leaf function, and obtain the leaf function converted from the third-party library function.

[0118] For example, for a third-party library function, such as the set_trace function for setting a tracing breakpoint, the module object pdb of the third-party library function set_trace is passed in as a parameter, and the third-party library function set_trace is packaged according to the module object pdb of the third-party library function set_trace. The processing methods include:

[0119] Method 1:

[0120] "import pdb

[0121] from pdb import set_trace

[0122] #custom wrap method 1

[0123] custom_wrap(pdb)(set_trace)".

[0124] Method 2:

[0125] "import pdb

[0126] from pdb import set_trace

[0127] #custom wrap method 2

[0128] custom_wrap(pdb)('set_trace')".

[0129] The embodiment of the present invention uses a pre-designed decorator to perform different wrapping processing on different types of functions to be traced, which can ensure that each function to be traced is effectively converted into a leaf function, thereby further improving the efficiency of generating the computational graph of the neural network model.

[0130] In an optional embodiment, the traversal of each function in the target neural network model is performed for tracking processing to generate a computational graph of the neural network model, specifically including: traversing each function in the target neural network model to determine whether the current function is a leaf function transformed from any function to be tracked; when the current function is a leaf function transformed from any function to be tracked, capturing the current function and creating a node in the initial computational graph that calls the current function to be tracked; wherein the current function to be tracked is the function to be tracked corresponding to the current function; when all functions in the target neural network model are traversed, the latest initial computational graph is used as the computational graph of the neural network model.

[0131] As an example, after obtaining the target neural network model, each function in the target neural network model is traversed to determine whether the current function is a leaf function after transformation of any function to be tracked.

[0132] If the current function is a leaf function transformed from any function to be tracked, the function to be tracked corresponding to the current function is used as the current function to be tracked, and a node for calling the current function to be tracked is created in the initial calculation graph. The node records the information of calling the current function to be tracked.

[0133] If the current function is not a leaf function transformed from any function to be tracked, continue to traverse the next function of the target neural network model.

[0134] When all functions in the target neural network model are traversed, in the target neural network model, the leaf functions transformed from each function to be tracked are replaced with the corresponding functions to be tracked to obtain the original neural network model, and the latest initial calculation graph obtained at this time is used as the calculation graph of the neural network model.

[0135] The embodiment of the present invention performs tracing processing by traversing each function in the target neural network model. When traversing to a leaf function transformed from any function to be traced, the current function is captured, and a node that calls the corresponding function to be traced is created in the initial calculation graph. This can support identifying all leaf functions transformed from the functions to be traced as nodes, and completely and accurately generate a calculation graph, thereby further improving the calculation graph generation efficiency of the neural network model.

[0136] In an optional embodiment, when the current function is a leaf function converted from any function to be traced, the current function is captured, and a node that calls the current function to be traced is created in the initial computation graph, specifically including: when the current function is a leaf function corresponding to any function to be traced, the current function is captured, and it is queried whether there is a Proxy class instance in the input parameters of the current function; if there is a Proxy class instance in the input parameters of the current function, the Proxy class instance is used to create a node that calls the current function to be traced in the initial computation graph; if there is no Proxy class instance in the input parameters of the current function, the Tracer class instance is used to create a node that calls the current function to be traced in the initial computation graph.

[0137] As an example, when the current function is a leaf function converted from any function to be tracked, it is queried whether there is a Proxy class instance in the input parameters of the current function. Proxy refers to a proxy server, which is an important server security function that mainly works at the session layer of the Open System Interconnection (OSI) model, thereby playing the role of a firewall.

[0138] If there is a Proxy class instance in the input parameters of the current function, the Proxy class instance is used, such as creating a node in the initial calculation graph that calls the current function to be traced.

[0139] If there is no Proxy class instance in the input parameters of the current function, the Tracer class instance is used, for example, the tracer creates a node in the initial calculation graph that calls the current function to be traced. Tracer is a component used for call tracing.

[0140] The embodiment of the present invention can quickly create nodes in the initial calculation graph by using the Proxy class instance to create a node that calls the current function to be traced in the initial calculation graph when traversing to a leaf function converted from any function to be traced and its input parameters contain a Proxy class instance; and by using the Tracer class instance to create a node that calls the current function to be traced in the initial calculation graph when traversing to a leaf function converted from any function to be traced but its input parameters contain no Proxy class instance. This can further improve the calculation graph generation efficiency of the neural network model by allowing nodes to be created in the initial calculation graph quickly.

[0141] In an optional embodiment, each function in the target neural network model is traversed for tracking processing to generate a computational graph of the neural network model, and specifically includes: when the current function cannot be captured, the current function is added to the global variable of the current function to be tracked.

[0142] As an example, considering that in the process of traversing each function in the target neural network model for tracing, it may not be possible to capture the leaf function after each function to be traced is converted, but the Module.__call__ method can be captured. Therefore, before execution, monkey patch (code injection) is performed to add each leaf function after each function to be traced is converted to its global variable. This means that in some cases, there may be certain limitations on wrapping specific functions. To address this limitation, the leaf function can be replaced by defining a module instead of a function.

[0143] The embodiment of the present invention can ensure that all leaf functions converted from the functions to be tracked in the target neural network model are effectively captured by adding the current function to the global variable of the current function to be tracked when the current function cannot be captured.

[0144] In an optional embodiment, when the current function is a leaf function converted from any function to be traced, the current function is captured, and after a node that calls the current function to be traced is created in the initial calculation graph, it also includes: when a node that calls the current function to be traced is successfully created in the initial calculation graph, a prompt message indicating that the node has been successfully created is returned.

[0145] As an example, when the current function is a leaf function converted from any function to be traced, the function to be traced corresponding to the current function is used as the current function to be traced, a node calling the current function to be traced is created in the initial calculation graph, and it is monitored in real time whether the node calling the current function to be traced is successfully created in the initial calculation graph.

[0146] If it is detected that a node that calls the current function to be traced is successfully created in the initial calculation graph, a prompt message of node creation success is returned to prompt the user that the node is successfully created; if it is detected that a node that calls the current function to be traced is not successfully created in the initial calculation graph, a prompt message of node creation failure is returned to prompt the user that the node creation failed.

[0147] For example, when the current function is a leaf function corresponding to any function to be traced, the function to be traced corresponding to the current function is used as the current function to be traced. The current function accepts a false input Proxy, inserts a node that records the information of calling the current function to be traced into the initial calculation graph, returns a Proxy, and prompts the user for the node creation result.

[0148] The embodiment of the present invention can promptly feedback the node creation result to the user by returning a prompt message indicating that the node is successfully created when a node that calls the current function to be traced is successfully created in the initial calculation graph, thereby facilitating the user to take countermeasures based on the calculation graph generation situation and further improving the calculation graph generation efficiency of the neural network model.

[0149] As an example, in order to more clearly illustrate a method for generating a computational graph of a neural network model provided in the first embodiment of the present invention, the method for generating a computational graph of a neural network model is applied. The data flow diagram of the method for generating a computational graph of a neural network model is as follows: Figure 2 shown.

[0150] exist Figure 2 In the figure, A represents the function to be traced, B represents the improved PyTorch FX, which is configured with a pre-designed decorator or wrapper function, and C and D both represent leaf functions obtained by wrapping the function to be traced.

[0151] In the process of applying the computational graph generation method of the neural network model, the functionality of the native fx.wrap function in PyTorch FX is expanded and can be used for:

[0152] 1. Wrap any custom function as a leaf function and identify it as a node during fx tracing.

[0153] For functions without input from other nodes in the computation graph, the original torch can only run it simply and cannot recognize it as a node.

[0154] 2. Wrap any built-in function as a leaf function and identify it as a node during fx tracing, and wrap any third-party library function as a leaf function and identify it as a node during fx tracing.

[0155] However, torch.fx.wrap can only wrap the functions of the entire module, and the interface is not easy to use and the efficiency is not high.

[0156] 3. For the wrapped function, you can call it in two ways: "import module; module.wrapped_func()" and "from module import wrapped_func; wrapped_func()".

[0157] However, torch.fx.wrap only supports the call method of “import module; module.wrapped_func()”.

[0158] 4. For the wrapped function, even if the output is not used, it will not be regarded as a dead node, but can be regarded as a side effect function and will not be deleted.

[0159] However, torch.fx.wrap does not support side-effect functions, and output nodes that are not used will be deleted as dead nodes.

[0160] Please see Figure 3, Figure 3 A flowchart of a method for generating a computational graph of a neural network model provided in the second embodiment of the present invention. The second embodiment of the present invention provides a computational graph generating device for a neural network model, comprising: a function wrapping module 21, for wrapping each function to be tracked in the neural network model, so that each function to be tracked is converted into a leaf function to obtain a target neural network model; a computational graph generating module 22, for traversing each function in the target neural network model for tracking processing, and generating a computational graph of the neural network model.

[0161] In an optional embodiment, the function wrapping module 21 is also used to wrap each function to be traced in the neural network model respectively, so that each function to be traced is converted into a leaf function, and before obtaining the target neural network model, it responds to the function tracing request initiated by the user for the neural network model, and determines all functions to be traced selected by the user from all functions of the neural network model.

[0162] In an optional embodiment, each function to be tracked in the neural network model is wrapped separately to convert each function to be tracked into a leaf function to obtain a target neural network model, which specifically includes: using a pre-designed decorator to wrap the function to be tracked to obtain a leaf function after the function to be tracked is converted; or, calling a wrapping function to wrap the function to be tracked to obtain a leaf function after the function to be tracked is converted.

[0163] In an optional embodiment, the method of using a pre-designed decorator to wrap the function to be tracked to obtain a leaf function after the function to be tracked is specifically as follows: when the function to be tracked is a custom function, the decorator is used to wrap the custom function to obtain a leaf function after the custom function is converted; when the function to be tracked is a built-in function, the decorator is used to obtain the module object of the built-in function to wrap the built-in function to obtain a leaf function after the built-in function is converted; when the function to be tracked is a third-party library function, the decorator is used to obtain the module object of the third-party library function to wrap the third-party library function to obtain a leaf function after the third-party library function is converted.

[0164] In an optional embodiment, the traversal of each function in the target neural network model is performed for tracking processing to generate a computational graph of the neural network model, specifically including: traversing each function in the target neural network model to determine whether the current function is a leaf function transformed from any function to be tracked; when the current function is a leaf function transformed from any function to be tracked, capturing the current function and creating a node in the initial computational graph that calls the current function to be tracked; wherein the current function to be tracked is the function to be tracked corresponding to the current function; when all functions in the target neural network model are traversed, the latest initial computational graph is used as the computational graph of the neural network model.

[0165] In an optional embodiment, when the current function is a leaf function converted from any function to be traced, the current function is captured, and a node that calls the current function to be traced is created in the initial computation graph, specifically including: when the current function is a leaf function converted from any function to be traced, the current function is captured, and it is queried whether there is a Proxy class instance in the input parameters of the current function; if there is a Proxy class instance in the input parameters of the current function, the Proxy class instance is used to create a node that calls the current function to be traced in the initial computation graph; if there is no Proxy class instance in the input parameters of the current function, the Tracer class instance is used to create a node that calls the current function to be traced in the initial computation graph.

[0166] In an optional embodiment, the calculation graph generation module 22 is also used to add the current function to the global variables of the current function to be traced when the current function cannot be captured.

[0167] In an optional embodiment, the computational graph generation module 22 is also used to capture the current function when the current function is a leaf function converted from any function to be traced, and after creating a node that calls the current function to be traced in the initial computational graph, it also includes: when a node that calls the current function to be traced is successfully created in the initial computational graph, returning a prompt message that the node has been successfully created.

[0168] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0169] Please see Figure 3 , Figure 3 The third embodiment of the present invention provides a structural diagram of an electronic device. The third embodiment of the present invention provides an electronic device 30, including a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301; the memory 302 is coupled to the processor 301, and when the processor 301 executes the computer program, the computational graph generation method of the neural network model as described in the first embodiment of the present invention is implemented, and the same beneficial effects can be achieved.

[0170] Among them, when the processor 301 reads the computer program from the memory 302 through the bus 303 and executes the computer program, the method of any embodiment included in the method for generating a computational graph of a neural network model as described in the first embodiment of the present invention can be implemented.

[0171] Processor 301 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 301 can be a microprocessor.

[0172] The memory 302 may be used to store instructions executed by the processor 301 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present invention. The processor 301 of the disclosed embodiment may be used to execute instructions in the memory 302 to implement the computational graph generation method of the neural network model as described in the first embodiment of the present invention. The memory 302 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0173] The fourth embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the calculation graph generation method of the neural network model as described in the first embodiment of the present invention, and can achieve the same beneficial effects as the first embodiment.

[0174] In summary, the embodiments of the present invention provide a method, device, equipment and medium for generating a computational graph of a neural network model, wherein the method for generating a computational graph of a neural network model comprises: respectively wrapping each function to be tracked in the neural network model, so that each function to be tracked is converted into a leaf function to obtain a target neural network model; traversing each function in the target neural network model for tracking processing to generate a computational graph of the neural network model. The embodiments of the present invention respectively convert each function to be tracked in the neural network model into a leaf function to obtain a target neural network model, traverse all functions in the target neural network model for tracking processing to generate a computational graph of the neural network model, thereby ensuring that the leaf functions after the conversion of each function to be tracked are retained to create computational graph nodes, and recording the information of each function to be tracked in the computational graph, thereby achieving the technical effect of quickly tracking all functions in the neural network model to completely and accurately generate a computational graph, and improving the efficiency of generating a computational graph of the neural network model.

[0175] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0176] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0177] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0178] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for generating a computational graph of a neural network model, characterized in that: include: Each to-be-tracked function in the neural network model is packaged to convert each to-be-tracked function into a leaf function, thereby obtaining a target neural network model; Traverse each function in the target neural network model for tracing processing to generate a computational graph of the neural network model.

2. The method for generating a computational graph of a neural network model according to claim 1, characterized in that: The traversing each function in the target neural network model for tracking processing to generate a calculation graph of the neural network model specifically includes: Traversing each function in the target neural network model, and determining whether the current function is a leaf function after transformation of any of the functions to be tracked; When the current function is a leaf function converted from any of the functions to be tracked, the current function is captured, and a node for calling the current function to be tracked is created in the initial calculation graph; wherein the current function to be tracked is the function to be tracked corresponding to the current function; When all functions in the target neural network model are traversed, the latest initial calculation graph is used as the calculation graph of the neural network model.

3. The method for generating a computational graph of a neural network model according to claim 2, wherein: When the current function is a leaf function converted from any of the functions to be traced, capturing the current function and creating a node for calling the current function to be traced in the initial calculation graph specifically includes: When the current function is a leaf function converted from any of the functions to be tracked, the current function is captured, and whether there is a Proxy class instance in the input parameters of the current function is queried; If there is a Proxy class instance in the input parameters of the current function, the Proxy class instance is used to create a node in the initial calculation graph that calls the current function to be traced; If there is no Proxy class instance in the input parameters of the current function, a Tracer class instance is used to create a node in the initial calculation graph that calls the current function to be traced.

4. The method for generating a computational graph of a neural network model according to claim 2, wherein: The traversing each function in the target neural network model for tracking processing to generate a calculation graph of the neural network model specifically includes: When the current function cannot be captured, the current function is added to the global variable of the current function to be traced.

5. The method for generating a computational graph of a neural network model according to claim 1, wherein: The process of respectively wrapping each to-be-tracked function in the neural network model so that each to-be-tracked function is converted into a leaf function to obtain a target neural network model specifically includes: The function to be traced is wrapped with a pre-designed decorator to obtain a leaf function after the function to be traced is transformed; or, The wrapping function is called to wrap the function to be tracked, and obtain a leaf function after the function to be tracked is transformed.

6. The method for generating a computational graph of a neural network model according to claim 5, characterized in that: The method of using a pre-designed decorator to wrap the function to be tracked to obtain a leaf function after the function to be tracked is transformed, specifically including: When the function to be traced is a user-defined function, the decorator is used to wrap the user-defined function to obtain a leaf function after the user-defined function is transformed; When the function to be traced is a built-in function, the decorator is used to obtain the module object of the built-in function, to wrap the built-in function, and obtain a leaf function after the built-in function is converted; When the function to be traced is a third-party library function, the decorator is used to obtain the module object of the third-party library function, to wrap the third-party library function, and to obtain a leaf function after the third-party library function is converted.

7. The method for generating a computational graph of a neural network model according to claim 1, wherein: Before the process of respectively wrapping each to-be-tracked function in the neural network model to convert each to-be-tracked function into a leaf function and obtaining the target neural network model, the process further includes: In response to a function tracing request initiated by a user for the neural network model, all the functions to be traced selected by the user from all the functions of the neural network model are determined.

8. A computational graph generation device for a neural network model, characterized in that: include: A function wrapping module is used to wrap each to-be-tracked function in the neural network model, so that each to-be-tracked function is converted into a leaf function to obtain a target neural network model; The computational graph generation module is used to traverse each function in the target neural network model for tracking processing and generate a computational graph of the neural network model.

9. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the memory is coupled to the processor, and when the processor executes the computer program, it implements the computational graph generation method of the neural network model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the computational graph generation method of the neural network model according to any one of claims 1 to 7.

Images