Automatic generation method and device from deep learning program to computational graph

Abstract

The embodiment of the invention discloses an automatic generation method and device from a deep learning program to a computational graph, and the method comprises the steps: receiving deep learning programs written in different languages, and carrying out the preprocessing; performing deep analysis and processing on the input deep learning program by using LLM; according to an LLM processing result, a calculation graph description program is generated in combination with a preset calculation node library, and a final calculation graph is generated after the program is executed, so that the accuracy and integrity of the calculation graph are ensured; wherein the computational node library comprises an API (Application Program Interface) defined by each operator, and each operator interface realizes that specific nodes and edges are added into the computational graph; the method has the beneficial effects that an innovative and efficient solution is provided for generation of the computational graph of the deep learning program by utilizing the advantages of the LLM.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and more particularly to a method and apparatus for automatically generating computational graphs from deep learning programs. Background Art

[0002] Currently, the closest existing technologies for the technical problem of generating computational graphs from deep learning programs are mainly based on specific compiler methods. These methods first perform code detection and then map the results of the code detection based on specific rules and algorithms to generate computational graphs.

[0003] Constraint-based matching, also known as pattern-based matching, searches for constraints and patterns in the code that match a set of previously defined constraints. For example, in the literature, SMR uses constraint-based matching to match and rewrite idioms, converting them into a Machine Learning Intermediate Representation (MLIR) dialect. In this approach, the constraints are hard-coded into the compiler and cannot be modified by the user. Constraint-based matching is also very brittle and often fails to match complex code structures.

[0004] That is, the prior art has the following three disadvantages.

[0005] Disadvantage 1: The fragility of MLIR's C / C++ language front-end and its limited support for advanced features.

[0006] MLIR's C / C++ language front-end exhibits significant fragility when processing complex deep learning programs. Its support for advanced C language features is incomplete, leading to problems when processing code that utilizes these features. For example, using C structures, advanced math functions, inline assembly language, and generic programming using templates can result in compilation errors, impacting subsequent computational graph generation and performance optimization. This incomplete support limits its widespread application in deep learning programs, which often rely on these advanced features to implement complex model architectures and efficient data processing.

[0007] Disadvantage 2: Limitations of traditional matching methods.

[0008] Traditional matching methods, such as SMR, rely primarily on a library of predefined operator templates for matching. This reliance on templates severely limits their matching scope and can only match functions, not code snippets. In deep learning programs, much important logic may be distributed across code snippets, not just within a single function. Therefore, methods that only match functions miss a significant amount of important information.

[0009] Furthermore, the predefined operator template library is relatively narrow in scope, covering only a limited number of code patterns and failing to meet the diverse code structure and logic requirements of deep learning programs. For example, the existing predefined template library may not contain the corresponding templates for some custom operators and new code patterns, making it impossible to match and optimize them.

[0010] Furthermore, this matching method, based on a library of predefined operator templates, is poorly adaptable to new code patterns and programming paradigms. As the field of deep learning continues to evolve, new code patterns and programming paradigms are constantly emerging, such as new neural network architectures and optimization techniques that introduce new code structures. Traditional matching methods are unable to automatically adapt to these new changes, limiting their effectiveness in this evolving technological landscape.

[0011] Disadvantage 3: High requirements for maintenance and updates.

[0012] Existing technologies typically require professional compiler developers to maintain and update them. This involves not only maintaining the C / C++ front-end of MLIR, MLIR, and even the LLVM-level logic, but also updating the template libraries and rules for various matching methods. For methods like SMR, when matching updates are needed, professional compiler developers are required to deeply modify and debug the code. This not only increases labor costs but also requires developers to have a deep understanding of the internal structure of the entire compiler and the matching rules of the CDG (Control Dependency Graph) and DDG (Data Dependency Graph).

[0013] The complexity and high requirements for maintenance and updates make it difficult for these technologies to quickly adapt to new needs and technological developments. For example, when new deep learning algorithms and code structures emerge, updating existing technologies to support these new code patterns takes a long time and has a high professional threshold. As a result, the speed of upgrading existing technologies cannot keep up with the innovation speed of deep learning programs, thus affecting their effectiveness and efficiency in practical applications. Summary of the Invention

[0014] In view of the technical deficiencies of the prior art, the purpose of the embodiments of the present invention is to provide a method and device for automatically generating computational graphs from deep learning programs.

[0015] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a method for automatically generating a computational graph from a deep learning program, the method comprising:

[0016] Receive deep learning programs written in different languages and perform preprocessing;

[0017] Use LLM to deeply analyze and process the input deep learning program;

[0018] Based on the results of LLM processing, a computational graph description program is generated in combination with a preset computational node library, and the final computational graph is generated after executing the program to ensure its accuracy and completeness; wherein, the computational node library includes the API interface defined by each operator, and each operator interface realizes the addition of specific nodes and edges to the computational graph.

[0019] As a specific implementation of this application, the preprocessing specifically includes:

[0020] Extract the mapping relationship between function names and function bodies in the program and generate a dictionary structure to provide basic data for subsequent in-depth analysis.

[0021] As a specific implementation of the present application, the in-depth analysis and processing includes performance analysis, operator identification and code segment matching;

[0022] The performance analysis includes:

[0023] Use the gprof tool to analyze the performance of the input program and generate a performance analysis file;

[0024] Then analyze the performance analysis file and extract a list of performance bottleneck functions whose cumulative time percentage exceeds a set threshold;

[0025] The operator identification and code segment matching include:

[0026] For each performance bottleneck function, call LLM to perform code detection to identify operators;

[0027] LLM determines whether the function is an operator based on the function body content; if so, it further calls LLM to obtain a list of all upper-level functions that call the operator; if not, it directly treats it as a function to be processed and forms a list of functions to be processed.

[0028] As a specific implementation of the present application, the generation of the final computation graph in combination with the preset computation node library specifically includes:

[0029] For the upper-level function list of the operator, the API interface of the corresponding operator is called through LLM to generate the calculation graph description code;

[0030] For the function list to be processed, the LLM is called to generate the corresponding computational graph description code in combination with the computing node library. The generated computational graph description code is saved in the specified file in units of functions.

[0031] When generating a computational graph to describe a program, LLM automatically infers and marks undetermined constant values based on the semantics and logical structure of the function body and records them as assumed values.

[0032] For the assumed values of the markers, call LLM to insert print statements in the function body to output the real values of these constants;

[0033] Recompile and run the modified program to obtain the actual value of the constant and update the constant value in the computational graph description program. Finally, the computational graph is checked to ensure its accuracy and completeness.

[0034] The updated computational graph description program is then integrated and executed to generate the final computational graph.

[0035] In a second aspect, an embodiment of the present invention provides a method for automatically generating a computational graph from a deep learning program, the method comprising:

[0036] Receive a deep learning program; wherein the deep learning program is written based on Python;

[0037] Call LLM to insert the code for exporting ONNX graph into the program file;

[0038] Execute the modified program to generate the corresponding ONNX graph;

[0039] Load the generated ONNX graph, parse the relationship between operators and nodes, combine the preset computing node library, map the information in the ONNX graph to operators, obtain the computational graph description code, and execute the computational graph description code to generate the final computational graph.

[0040] As a specific implementation of this application, LLM will select the best location to insert the code for exporting the ONNX graph based on the program structure and framework characteristics; wherein, the best location is immediately after the model is loaded and before the model reasoning is performed.

[0041] In a third aspect, an embodiment of the present invention further provides an apparatus for automatically generating a computational graph from a deep learning program, the apparatus comprising:

[0042] The preprocessing module is used to receive deep learning programs written in different languages and perform preprocessing;

[0043] The analysis and detection module is used to use LLM to deeply analyze and process the input deep learning program;

[0044] The computational graph generation module is used to generate a computational graph description program based on the results of LLM processing and a preset computational node library, and to generate the final computational graph after executing the program to ensure its accuracy and completeness; wherein, the computational node library includes the API interface defined by each operator, and each operator interface realizes the addition of specific nodes and edges to the computational graph.

[0045] In a fourth aspect, an embodiment of the present invention further provides an apparatus for automatically generating a computational graph from a deep learning program, the apparatus comprising:

[0046] A receiving module, configured to receive a deep learning program; wherein the deep learning program is written in Python;

[0047] Insert module, used to call LLM to insert the code for exporting ONNX graph in the program file;

[0048] Processing module for:

[0049] Execute the modified program to generate the corresponding ONNX graph;

[0050] Load the generated ONNX graph, parse the relationship between operators and nodes, combine the preset computing node library, map the information in the ONNX graph to operators, obtain the computational graph description code, and execute the computational graph description code to generate the final computational graph.

[0051] The technical solution provided by the embodiments of the present invention abandons traditional constraint-matching-based code detection and utilizes LLM for code matching, which brings greater versatility and fewer restrictions. It can handle more diverse and complex code structures and overcome the limitations of traditional methods (such as SMR) in template matching. LLM's powerful text understanding and pattern recognition capabilities enable it to better adapt to different deep learning program structures. Whether it is a newly emerged or customized code pattern, it can more effectively match it, significantly expanding the matching range.

[0052] At the same time, unlike traditional matching methods that rely on predefined operator template libraries and can only match functions, this invention uses the powerful capabilities of LLM to directly identify and match code segments without relying on predefined operator template libraries. By understanding the overall semantics of the code segments, it can mine more important information without missing the key logic distributed in the code segments of deep learning programs, greatly improving the completeness and accuracy of matching.

[0053] In addition, with the help of LLM, when new deep learning algorithms and code structures appear, LLM can automatically adapt to new changes without the need for manual updates of template libraries and rules. The LLM API used can be switched in a specified manner, which greatly reduces the workload of maintenance personnel and significantly reduces maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the specific implementation or the description of the prior art.

[0055] Figure 1 This is a flowchart of a method for automatically generating a computational graph from a deep learning program, provided by an embodiment of the present invention;

[0056] Figure 2 This is a processing flow chart of a C language deep learning program provided by an embodiment of the present invention;

[0057] Figure 3 This is a processing flow chart of a Python language deep learning program provided by an embodiment of the present invention;

[0058] Figure 4 This is a principle block diagram of an automatic generation device from a deep learning program to a computational graph provided by an embodiment of the present invention;

[0059] Figure 5 This is a principle block diagram of another device for automatically generating a computational graph from a deep learning program, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0062] Throughout this specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment," "in an embodiment," "an example," or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations in one or more embodiments or examples.

[0063] SMR: Source Matching and Rewriting. From the paper "Sourcematching and rewriting for MLIR using string-based automata".

[0064] MLIR: Multi-Level Intermediate Representation, multi-level intermediate representation.

[0065] LLVM: Low-Level Virtual Machine. An open-source compiler infrastructure project widely used to build programming language front-ends, optimizers, and back-ends.

[0066] CDG: Control-Dependency Graph, control dependency graph.

[0067] DDG: Data-Dependency Graph, data dependency graph.

[0068] LLM: Large Language Model, large language model.

[0069] Llama2: Large Language Model Meta AI 2, is the second-generation open source large language model developed by Meta.

[0070] 3DGS: 3D Gaussian Splatting. It is an important technology in the fields of 3D rendering and computer graphics.

[0071] SCI: Self-Calibrated Illumination Learning Framework. From the paper "Toward Fast, Flexible, and Robust Low-Light Image Enhancement."

[0072] gprof: GNU Profiler, a program performance analysis tool widely used in Unix and Unix-like systems (such as Linux).

[0073] DSE: Design Space Exploration, design space search.

[0074] PyTorch: An open-source deep learning framework developed by Facebook's AI research team.

[0075] TensorFlow: An open-source deep learning framework developed by Google.

[0076] ONNX: Open Neural Network Exchange, open neural network exchange.

[0077] GPT-4: A large multimodal language model developed by OpenAI.

[0078] Check mechanism: usually refers to a mechanism for checking, verifying or verifying to ensure the correctness, integrity and security of a system, program, data or process.

[0079] Compute node library: defines the compute nodes that match the computation graph. Each compute node is used to add specific nodes and edges to the computation graph.

[0080] Task Graph: Task graph, also known as computation graph. In this article, computation graph and task graph represent the same concept.

[0081] It should be noted that, unless otherwise specified, the technical terms in this embodiment have the common meanings understood in the relevant technical field.

[0082] Example 1, please refer to Figure 1 , an embodiment of the present invention provides a method for automatically generating a computational graph from a deep learning program, the method comprising:

[0083] S101, receiving deep learning programs written in different languages and performing preprocessing;

[0084] S102, using LLM to deeply analyze and process the input deep learning program;

[0085] S103: Generate a computational graph description program based on the results of the LLM processing and in combination with a preset computational node library, and execute the program to generate a final computational graph to ensure its accuracy and completeness; wherein, the computational node library includes an API interface defined by each operator, and each operator interface implements the addition of specific nodes and edges to the computational graph.

[0086] This program uses a variety of deep learning programs, including neural network programs with different architectures, such as Llama2 and 3DGS in C and SCI in Python, to cover different languages and complexity levels.

[0087] When implementing, refer to Figure 2Taking a deep learning program written in C language (such as Llama2, 3DGS, etc.) as an example, the preprocessing specifically includes:

[0088] Extract the mapping relationship between function names and function bodies in the program and generate a dictionary structure to provide basic data for subsequent in-depth analysis.

[0089] That is, a dictionary structure {function_name:function_body} is generated to provide basic data for subsequent performance analysis and code segment matching.

[0090] The in-depth analysis and processing includes performance analysis, operator identification and code segment matching;

[0091] The performance analysis includes:

[0092] Use the gprof tool to analyze the performance of the input program and generate a performance analysis file;

[0093] Then analyze the performance analysis file and extract the list of performance bottleneck functions whose cumulative time percentage exceeds the set threshold; wherein the set threshold is selected as 99%; that is, Figure 2 Obtain the top n performance bottleneck functions as described in Note 1. These n performance bottleneck functions satisfy the requirement that the sum of their time proportions is greater than 0.99. The functions in the performance bottleneck function list are the focus of subsequent optimization and computational graph generation.

[0094] The operator identification and code segment matching include:

[0095] For each performance bottleneck function, call LLM to perform code detection to identify operators;

[0096] LLM determines whether the function (i.e., the current function) is an operator based on the function body content; if so, it further calls LLM to obtain a list of all upper-level functions that call the operator and iterates; if not, it directly treats it as a function to be processed and forms a list of functions to be processed.

[0097] The process of generating the final computation graph by combining the preset computation node library specifically includes:

[0098] For the upper-level function list of an operator, the LLM calls the API interface of the corresponding operator to generate a calculation graph description program. It should be noted that in this embodiment, the calculation graph description code and the calculation graph description program have the same meaning.

[0099] For the function list to be processed, the LLM is called in conjunction with the compute node library to generate the corresponding computational graph description code. The generated computational graph description code is saved in a specified file on a function-by-function basis. Specifically, the compute node library is used in conjunction with the LLM when generating the computational graph description code. It appears as known information in the LLM input prompt, letting the LLM know which compute node API interfaces are available for call. These API interfaces and other auxiliary information are ultimately combined to form the final computational graph description code.

[0100] When generating a computational graph to describe a program, LLM automatically infers and marks undetermined constant values (such as model dimensions or Tensor shapes) based on the semantics and logical structure of the function body, and records them as assumed values; that is, they are marked with the #Assumed value label;

[0101] For the assumed values of the markers, call LLM to insert print statements in the function body to output the real values of these constants;

[0102] Recompile and run the modified program to obtain the actual value of the constant and update the constant value in the computational graph description program. Finally, the computational graph is checked to ensure its accuracy and completeness.

[0103] The updated computational graph description program is then integrated and executed to generate the final computational graph; since this computational graph reflects the performance bottlenecks and logical structure of the program, it can be used for subsequent design space exploration (DSE) and hardware optimization.

[0104] The key technical point of this invention is that by introducing a large language model (LLM), it innovates the method of generating computational graphs from deep learning programs, solving several key problems in the existing technology. The following are the key differences and improvements of this invention over existing solutions:

[0105] Key point 1: LLM-based code segment identification and matching

[0106] Existing technologies (such as SMR) rely heavily on predefined operator template libraries and can only match functions, failing to effectively handle complex logic within code segments. This invention leverages the powerful semantic understanding capabilities of LLM to directly identify and match code segments without relying on predefined template libraries.

[0107] The improved present invention achieves greater breadth, completeness, and adaptability. In terms of breadth, LLM can identify and process a variety of code patterns, including custom operators and emerging programming paradigms, significantly expanding the scope of matching. In terms of completeness, by understanding the overall semantics of code segments, it avoids the problem of traditional methods missing important logic, improving the completeness and accuracy of matching. In terms of adaptability, LLM can automatically adapt to new code structures and programming paradigms, eliminating the need for manual updates to template libraries and reducing maintenance costs.

[0108] Key point 2: The system is simple and easy to use, maintenance costs are reduced, and update speed is increased

[0109] Existing technologies require professional compiler developers to maintain and update, involving complex updates to the MLIR front-end, LLVM layer logic, and template libraries. This invention significantly reduces maintenance costs and update difficulty through automated processing of LLM.

[0110] This improved system offers the advantages of ease of use, low maintenance, and rapid updates. Regarding low maintenance, LLM automatically adapts to new code patterns and programming paradigms, eliminating the need for manual updates to template libraries and rules, thus reducing maintenance costs. Regarding rapid updates, the system can quickly adapt and generate corresponding computational graphs as new deep learning algorithms and code structures emerge, significantly improving update speed. Furthermore, through modular design and automated scripting, the system lowers the barrier to entry, enabling even non-professionals to effectively use the system.

[0111] The key technical aspect of this invention lies in the automated and multilingual translation of deep learning programs into computational graphs through the introduction of LLM. Compared to existing technologies, this invention offers significant advantages in terms of matching scope, computational graph quality, maintenance costs, and update speed, providing an innovative and efficient solution for generating computational graphs for deep learning programs.

[0112] It should be noted that this solution can be directly applied to the template-based automatic generation technology of dedicated accelerators. As the front-end module of this technology, its input is the deep learning program and its output is the corresponding computational graph; the output computational graph is connected to the subsequent DSE module, RTL generation module and simulator module.

[0113] This solution abandons traditional constraint-matching-based code detection and leverages LLM for code matching, offering greater versatility and fewer restrictions. It can handle more diverse and complex code structures, overcoming the limitations of traditional methods (such as SMR) in template matching. LLM's powerful text understanding and pattern recognition capabilities make it more adaptable to diverse deep learning program structures, enabling more effective matching of both newly emerged and custom code patterns, significantly expanding the matching scope.

[0114] At the same time, unlike traditional matching methods that rely on predefined operator template libraries and can only match functions, this invention uses the powerful capabilities of LLM to directly identify and match code segments without relying on predefined operator template libraries. By understanding the overall semantics of the code segments, it can mine more important information without missing the key logic distributed in the code segments of deep learning programs, greatly improving the completeness and accuracy of matching.

[0115] In addition, with the help of LLM, when new deep learning algorithms and code structures appear, LLM can automatically adapt to new changes without the need for manual updates of template libraries and rules. The LLM API used can be switched in a specified manner, which greatly reduces the workload of maintenance personnel and significantly reduces maintenance costs.

[0116] Example 2, reference Figure 3 , an embodiment of the present invention also provides a method for automatically generating a computational graph from a deep learning program, the method comprising:

[0117] Receive a deep learning program; wherein the deep learning program is written based on Python;

[0118] Call LLM to insert the code for exporting ONNX graph into the program file;

[0119] Execute the modified program to generate the corresponding ONNX graph;

[0120] Load the generated ONNX graph, parse the relationship between operators and nodes, combine the preset computing node library, map the information in the ONNX graph to operators, obtain the computational graph description code, and execute the computational graph description code to generate the final computational graph.

[0121] The combination of the computing node library and the steps here is the same as that of the aforementioned C language and will not be repeated here.

[0122] Among them, LLM will select the best location to insert the code for exporting the ONNX graph based on the program structure and framework characteristics (such as PyTorch or TensorFlow); the best location is immediately after the model is loaded and before performing model inference.

[0123] The technical solution of the present invention starts with the input of a deep learning program and covers multiple processing steps to finally generate the required computational graph. First, a deep learning program written in different languages (such as C or Python) is received. After the program is input, the LLM automatically inserts the ONNX graph export code for Python programs to ensure that the generated ONNX graph accurately reflects the program logic.

[0124] For C language, performance analysis is performed using the gprof tool to identify performance bottleneck functions within the program. LLM then leverages its text understanding and pattern recognition capabilities to directly identify and match code segments without relying on predefined operator template libraries. This means it can identify various code patterns within the program, both standard and custom, and more accurately match code segments, avoiding the problem of traditional methods that only match functions and miss important information.

[0125] After completing performance analysis and code segment matching, the LLM generates the corresponding computational graph description code, and uses a check mechanism to ensure accuracy and completeness. The check mechanism generates the corresponding C code calculation logic from the computational graph and compares it with the output of the source program for verification. If the check passes, the next step is carried out. If it fails, the error message is prompted to the LLM for correction. If the error still fails after multiple corrections, the process is rolled back.

[0126] Based on the same inventive concept, Figure 4 , an embodiment of the present invention further provides an automatic generation device from a deep learning program to a computational graph, the device comprising:

[0127] The preprocessing module is used to receive deep learning programs written in different languages and perform preprocessing;

[0128] The analysis and detection module is used to use LLM to deeply analyze and process the input deep learning program;

[0129] The computational graph generation module is used to generate a computational graph description program based on the results of LLM processing and a preset computational node library, and to generate the final computational graph after executing the program to ensure its accuracy and completeness; wherein, the computational node library includes the API interface defined by each operator, and each operator interface realizes the addition of specific nodes and edges to the computational graph.

[0130] It should be noted that for a more specific description of the workflow of the device embodiment, please refer to the method embodiment corresponding to the aforementioned embodiment 1, which will not be repeated here, and can have the following beneficial effects:

[0131] 1. The matching rate has been greatly improved

[0132] Through LLM-based code detection, the present invention overcomes the limitations of traditional methods, which rely on predefined operator template libraries and can only match functions, and can directly identify and match code segments. LLM's powerful text understanding and pattern recognition capabilities enable it to better understand the overall semantics of the code, thereby identifying a wide range of code patterns, including emerging and customized ones, and avoiding the omission of important information that traditional methods often fail to capture.

[0133] When dealing with complex code structures and new code patterns, the proposed method significantly improves the matching rate compared to traditional methods. This means that during the computation graph generation process, key information in the program can be obtained more comprehensively and accurately, laying a solid foundation for subsequent computation graph generation.

[0134] 2. Maintenance costs are significantly reduced

[0135] Existing technologies require professional compiler developers to maintain and update the complex MLIR front-end, LLVM hierarchical logic, and template libraries, resulting in high maintenance costs and difficulty in updating. However, the present invention leverages LLM, which automatically adapts to new deep learning algorithms and code structures as they emerge, eliminating the need for manual updates to template libraries and rules. The LLM API allows for specific switching. LLM is also rapidly evolving, and its code detection capabilities are constantly improving.

[0136] From the perspective of actual development and maintenance, after adopting the technical solution of the present invention, the workload of maintenance personnel is greatly reduced; at the same time, the update speed is significantly improved, which can quickly adapt to new needs and technological developments, avoiding the impact of untimely technological updates on effectiveness and efficiency in actual applications.

[0137] Reference Figure 5 , an embodiment of the present invention further provides an automatic generation device from a deep learning program to a computational graph, the device comprising:

[0138] A receiving module, configured to receive a deep learning program; wherein the deep learning program is written in Python;

[0139] Insert module, used to call LLM to insert the code for exporting ONNX graph in the program file;

[0140] Processing module for:

[0141] Execute the modified program to generate the corresponding ONNX graph;

[0142] Load the generated ONNX graph, parse the relationship between operators and nodes, combine the preset computing node library, map the information in the ONNX graph to operators, obtain the computational graph description code, and execute the computational graph description code to generate the final computational graph.

[0143] Among them, LLM will select the best location to insert the code for exporting the ONNX graph based on the program structure and framework characteristics; the best location is immediately after the model is loaded and before the model reasoning is performed; that is, it can be inserted at any code point after the model is loaded and before the model reasoning is performed; this will be input to LLM as part of the prompt.

[0144] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for automatically generating computational graphs from deep learning programs, characterized in that: The method comprises: Receive deep learning programs written in different languages and perform preprocessing; Use LLM to deeply analyze and process the input deep learning program; Based on the results of LLM processing, a computational graph description program is generated in combination with a preset computational node library, and the final computational graph is generated after executing the program to ensure its accuracy and completeness; wherein, the computational node library includes the API interface defined by each operator, and each operator interface realizes the addition of specific nodes and edges to the computational graph.

2. The method for automatically generating a computational graph from a deep learning program according to claim 1, wherein: The pretreatment specifically includes: Extract the mapping relationship between function names and function bodies in the program and generate a dictionary structure to provide basic data for subsequent in-depth analysis.

3. The method for automatically generating a computational graph from a deep learning program according to claim 1, wherein: The in-depth analysis and processing includes performance analysis, operator identification and code segment matching; The performance analysis includes: Use the gprof tool to analyze the performance of the input program and generate a performance analysis file; Then analyze the performance analysis file and extract a list of performance bottleneck functions whose cumulative time percentage exceeds a set threshold; The operator identification and code segment matching include: For each performance bottleneck function, call LLM to perform code detection to identify operators; LLM determines whether the function is an operator based on the function body content; if so, it further calls LLM to obtain a list of all upper-level functions that call the operator; if not, it directly treats it as a function to be processed and forms a list of functions to be processed.

4. The method for automatically generating a computational graph from a deep learning program according to claim 3, wherein: The process of generating the final computation graph by combining the preset computation node library specifically includes: For the upper-level function list of the operator, the API interface of the corresponding operator is called through LLM to generate the calculation graph description code; For the function list to be processed, the LLM is called to generate the corresponding computational graph description code in combination with the computing node library. The generated computational graph description code is saved in the specified file in units of functions. When generating a computational graph to describe a program, LLM automatically infers and marks undetermined constant values based on the semantics and logical structure of the function body and records them as assumed values. For the assumed values of the markers, call LLM to insert print statements in the function body to output the real values of these constants; Recompile and run the modified program to obtain the actual value of the constant and update the constant value in the computational graph description program. Finally, the computational graph is checked to ensure its accuracy and completeness. The updated computational graph description program is then integrated and executed to generate the final computational graph.

5. A method for automatically generating computational graphs from deep learning programs, characterized in that: The method comprises: Receive a deep learning program; wherein the deep learning program is written based on Python; Call LLM to insert the code for exporting ONNX graph into the program file; Execute the modified program to generate the corresponding ONNX graph; Load the generated ONNX graph, parse the relationship between operators and nodes, combine the preset computing node library, map the information in the ONNX graph to operators, obtain the computational graph description code, and execute the computational graph description code to generate the final computational graph.

6. The method for automatically generating a computational graph from a deep learning program according to claim 5, wherein: LLM selects the best location to insert the code for exporting the ONNX graph based on the program structure and framework characteristics; the best location is immediately after the model is loaded and before performing model inference.

7. An automatic generation device from deep learning program to computational graph, characterized in that The device comprises: The preprocessing module is used to receive deep learning programs written in different languages and perform preprocessing; The analysis and detection module is used to use LLM to deeply analyze and process the input deep learning program; The computational graph generation module is used to generate a computational graph description program based on the results of LLM processing and a preset computational node library, and to generate the final computational graph after executing the program to ensure its accuracy and completeness; wherein, the computational node library includes the API interface defined by each operator, and each operator interface realizes the addition of specific nodes and edges to the computational graph.

8. An automatic generation device from deep learning program to computational graph, characterized in that The device comprises: A receiving module, configured to receive a deep learning program; wherein the deep learning program is written in Python; Insert module, used to call LLM to insert the code for exporting ONNX graph in the program file; Processing module for: Execute the modified program to generate the corresponding ONNX graph; Load the generated ONNX graph, parse the relationship between operators and nodes, combine the preset computing node library, map the information in the ONNX graph to operators, obtain the computational graph description code, and execute the computational graph description code to generate the final computational graph.

Images