Computational graph optimization method and device, electronic device, and computer-readable medium

Abstract

This disclosure provides a computation graph optimization method, apparatus, electronic device, and computer-readable medium. The method includes: acquiring a first computation graph to be optimized; detecting the first computation graph using a pre-trained graph neural network to determine a first subgraph to be optimized within the first computation graph; and optimizing the first computation graph based on the first subgraph to obtain an optimized second computation graph. Embodiments of this disclosure can improve the optimization efficiency of computation graphs, thereby improving the compilation efficiency of computation graphs.

Description

Technical Field

[0001] This disclosure relates to the field of computational graph technology, and in particular to a computational graph optimization method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the continuous development of artificial intelligence technology, more and more neural networks are being applied to production practices. These neural networks are often described using computation graphs during programming, deployment, and execution. As an intermediate representation (IR), computation graphs are also used in other fields such as high-performance computing. Therefore, optimization of computation graphs (such as compiler optimization and execution optimization) has received increasing attention. Summary of the Invention

[0003] This disclosure provides a computational graph optimization method and apparatus, a processing core, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, this disclosure provides a computation graph optimization method, which includes: obtaining a first computation graph to be optimized; detecting the first computation graph through a pre-trained graph neural network to determine a first subgraph to be optimized in the first computation graph; and optimizing the first computation graph based on the first subgraph to obtain an optimized second computation graph.

[0005] Secondly, this disclosure provides a computation graph optimization apparatus, which includes: an acquisition module for acquiring a first computation graph to be optimized; a detection module for detecting the first computation graph through a pre-trained graph neural network to determine a first subgraph to be optimized in the first computation graph; and an optimization module for optimizing the first computation graph based on the first subgraph to obtain an optimized second computation graph.

[0006] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the computational graph optimization method described above.

[0007] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor / processing core, implements the above-described computational graph optimization method.

[0008] The embodiments provided in this disclosure use graph neural networks (GNNs) to detect computation graphs, which can detect fuzzy structures or operators that are difficult to define with uniform and explicit rules and are to be optimized, thereby improving the success rate of detection, thus improving the optimization efficiency of computation graphs, and further improving the compilation efficiency of computation graphs.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0011] Figure 1 This is a schematic diagram of the process for optimizing the computation graph in related technologies;

[0012] Figure 2 A flowchart of a computational graph optimization method provided in this embodiment of the disclosure;

[0013] Figure 3 A flowchart illustrating some steps of a computational graph optimization method provided in this embodiment of the disclosure;

[0014] Figure 4 A flowchart illustrating some steps of a computational graph optimization method provided in this embodiment of the disclosure;

[0015] Figure 5 A flowchart illustrating some steps of a computational graph optimization method provided in this embodiment of the disclosure;

[0016] Figure 6 A flowchart illustrating some steps of a computational graph optimization method provided in this embodiment of the disclosure;

[0017] Figure 7 A schematic diagram of a computational graph optimization method provided in an embodiment of this disclosure;

[0018] Figure 8 A schematic diagram of a computational graph optimization method provided in an embodiment of this disclosure;

[0019] Figure 9 A flowchart illustrating some steps of a computational graph optimization method provided in this embodiment of the disclosure;

[0020] Figure 10 A block diagram of a computational graph optimization apparatus provided in an embodiment of this disclosure;

[0021] Figure 11 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0024] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0026] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0027] In some related technologies, a series of optimization processes are written for specific operators or structures. These optimization processes are generally called passes. Passes are used to complete the transformation, analysis, or optimization of the compilation object (such as a computation graph). Figure 1The execution of a pass is the process by which the compiler transforms, analyzes, and optimizes the compiled object. By executing the passes corresponding to the computation graph (i.e., pass1, pass2, pass3, and pass4 in the graph), the compilation result corresponding to the computation graph can be obtained.

[0028] Template matching can be used to find specific operators or structures on a compilation object (such as a computation graph). This involves representing specific operators or structures using explicit rules, then matching them on the compilation object using the corresponding rules. Once an operator or structure that matches the rules is found, the computation graph is optimized using the corresponding compilation process or optimization method (such as pass).

[0029] However, in practical applications, some specific operators or structures are often fuzzy and difficult to define with unified and clear rules. For example, branch structures that may cause a significant increase in intermediate hardware storage or unbalanced structures that cause a decrease in computational utilization. During compilation, it is often necessary to identify these operators or structures and call targeted optimization methods for optimization. Using template matching methods makes it difficult to identify these operators or structures, resulting in low identification accuracy, which affects the optimization efficiency of the computation graph and further affects the compilation efficiency of the computation graph.

[0030] The computation graph optimization method according to the embodiments of this disclosure uses graph neural networks (GNNs) to detect computation graphs, which can detect fuzzy structures or operators that are difficult to define with uniform and explicit rules, thereby improving the success rate of detecting fuzzy structures or operators that are difficult to define with uniform and explicit rules, thus improving the optimization efficiency of computation graphs and further improving the compilation efficiency of computation graphs.

[0031] The computational graph optimization method according to embodiments of this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0032] Figure 2 A flowchart illustrating a computational graph optimization method provided in this embodiment of the disclosure. (Refer to...) Figure 2 The method includes:

[0033] In step S21, the first computational graph to be optimized is obtained;

[0034] In step S22, the first computation graph is detected by a pre-trained graph neural network to determine the first subgraph to be optimized in the first computation graph;

[0035] In step S23, the first computation graph is optimized based on the first subgraph to obtain the optimized second computation graph.

[0036] For example, in step S21, the first computation graph may be an uncompiled and unoptimized computation graph, or it may be an uncompiled computation graph optimized by optimization methods in related technologies. This disclosure does not limit the specific method of obtaining the first computation graph.

[0037] In some possible implementations, the first computation graph can be an intermediate representation corresponding to the neural network during programming, deployment, and execution. The neural network is used to perform processing tasks, including any one of image processing, speech processing, text processing, and video processing tasks. This disclosure does not limit the type of task performed by the neural network.

[0038] In step S22, the graph neural network used to detect the first computation graph is a pre-trained graph neural network.

[0039] In some possible implementations, a graph neural network is trained using a computation graph containing specific operators or structures and the specific locations of those operators or structures within the computation graph as positive samples, making the trained graph neural network sensitive to those specific operators or structures.

[0040] Detecting the first computation graph using a graph neural network can be achieved by inputting the first computation graph into a trained graph neural network, which then scans and detects the first computation graph. The output of the graph neural network can be whether the first computation graph contains a specific operator or structure, and the specific location of the specific operator or structure in the first computation graph. The specific operator or structure in the first computation graph is the first subgraph to be optimized in the first computation graph.

[0041] In some possible implementations, the output of a graph neural network may include an identifier of a first subgraph corresponding to a specific operator or structure. This identifier may be an index of the specific operator or structure, through which the optimizer or optimization result corresponding to the specific operator or structure can be determined. For example, the output of a graph neural network may be directly an index of the optimizer corresponding to a specific operator or structure, through which the optimizer corresponding to the specific operator or structure can be found in an optimization library composed of multiple optimizers.

[0042] In some possible implementations, multiple graph neural networks can be used to detect the first computation graph, each sensitive to different specific operators or structures; alternatively, a single graph neural network can be used to detect the first computation graph, and this graph neural network is a general graph neural network that is sensitive to multiple specific operators or structures.

[0043] In some possible implementations, there can be multiple first subgraphs, and all graph neural networks detect the first subgraph.

[0044] In step S23, when the graph neural network detects the first subgraph in the first computation graph, the first computation graph is optimized based on the first subgraph. The first subgraph can be extracted, and the optimizer corresponding to the first subgraph can be found through the identifier of the first subgraph. The optimizer is used to optimize the first subgraph, and the optimization result is merged into the first computation graph to obtain the second computation graph.

[0045] According to embodiments of this disclosure, using graph neural networks to detect computation graphs can detect fuzzy structures or operators that are difficult to define with uniform and explicit rules and are to be optimized, thereby improving the success rate of detection, thus improving the optimization efficiency of computation graphs, and further improving the compilation efficiency of computation graphs.

[0046] The computational graph optimization method according to embodiments of this disclosure will now be described in detail.

[0047] As previously mentioned, multiple graph neural networks can be used to detect the first computation graph, such as a graph neural network comprising multiple first graph neural networks from a network library, each sensitive to one or more different specific operators or structures. Alternatively, a single graph neural network sensitive to multiple specific operators or structures can be used to detect the first computation graph to determine the first subgraph.

[0048] Figure 3 The flowchart shows the steps of using multiple first graph neural networks from the network library to detect the first computation graph, referencing... Figure 3 Step S22 may include steps S31 and S32.

[0049] In step S31, the multiple first graph neural networks in the network library are divided into multiple detection groups, and each detection group includes at least one graph neural network;

[0050] In step S32, each detection group is used to detect the first computation graph and determine at least one first subgraph in the first computation graph.

[0051] For example, in step S31, multiple first graph neural networks in the network library can be divided into multiple detection groups according to certain rules. The number of first graph neural networks included in each detection group can be the same or different, but each detection group should include at least one first graph neural network.

[0052] In special cases, each detection group contains only one first-graph neural network. In other words, the number of detection groups is equal to the number of first-graph neural networks in the network library.

[0053] In step S32, multiple detection groups are used to detect the first computation graph. Multiple computing units can be used to perform parallel operations. Each computing unit runs a first graph neural network. The parallel operation of multiple computing units is equivalent to multiple first graph neural networks performing parallel detection on the first computation graph to determine whether the first computation graph includes a specific operator or structure that the first graph neural network of the detection group is sensitive to, i.e., the first subgraph.

[0054] In some possible implementations, after a detection group completes the detection of the first computation graph and determines the first subgraph in the first computation graph, it can optimize the first computation graph based on the determined first subgraph. The optimized computation graph is then used as the input of the first graph neural network of the next detection group, and so on, until the last detection group completes the detection. Based on the detection results of the last detection group, the computation graph is optimized, and the result is the second computation graph.

[0055] In other words, the input to the first graph neural network of each detection group is consistent, that is, the computational graph optimized based on the output of the first graph neural network of the previous detection (the input of the first detection group is the first computational graph), while the input to the first graph neural network of different detection groups is different.

[0056] In some possible implementations, after all detection groups have been completed, the first computation graph can be optimized based on the first subgraph determined by all detection groups to obtain the second computation graph.

[0057] Since each first graph neural network is only sensitive to one or more specific operators or structures, it is simpler in structure, easier to train, and more accurate than general graph neural networks. At the same time, the parallel operation of multiple computing units makes up for the disadvantage of using multiple first graph neural networks for detection, which leads to excessively long detection time, thus shortening the total detection time of all first graph neural networks and improving detection efficiency.

[0058] Figure 4 A flowchart illustrating the steps of using a graph neural network sensitive to multiple specific operators or structures to detect the first computation graph, referencing... Figure 4Step S22 may include step S41.

[0059] In step S41, the first computation graph is detected by the second graph neural network to determine at least one first subgraph in the first computation graph.

[0060] The second graph neural network can be a more general graph neural network that is sensitive to multiple specific operators or structures. After the first computation graph is input into the second graph neural network, the second graph neural network can detect all specific operators or structures in the first computation graph and output them in the form of the first subgraph.

[0061] Because the second graph neural network is sensitive to multiple specific operators or structures, its structure is inevitably more complex than that of the first graph neural network, and its training process is also inevitably more complex. The time required for a single detection will also be longer. However, when using the second graph neural network for detection, only one detection is needed. That is, it only needs to read the first computation graph once. Therefore, it will occupy less memory. Although the time for a single detection is longer than that of the first graph neural network, since it only needs one detection to determine all the first subgraphs, the total time for using the second graph neural network to detect the first computation graph may be less than the total time for using multiple first graph neural networks to detect the first computation graph.

[0062] Of course, the optimization process is the same whether the first computation graph is optimized after the first graph neural network in a detection group has completed detection and the optimization result is used as the input for the next detection group, or the first computation graph is optimized based on the results of all detection groups after the first graph neural networks in all detection groups have completed detection, or the first computation graph is detected using the second graph neural network and optimized based on the detection results, or the first computation graph is optimized based on the first subgraph.

[0063] Figure 5 This is a flowchart illustrating the steps of optimizing the first computation graph based on the first subgraph, referencing... Figure 5 Step S23 may include steps S51, S52 and S53.

[0064] In step S51, the optimizer corresponding to the first subgraph is determined based on the identifier of the first subgraph;

[0065] In step S52, the optimizer corresponding to the first subgraph is invoked to optimize the first subgraph and obtain the second subgraph;

[0066] In step S53, the corresponding first subgraph in the first computation graph is replaced with the second subgraph to obtain the second computation graph.

[0067] The identifier of the first subgraph can be an index of a specific operator or structure. The index of a specific operator or structure can be used to determine the optimizer or optimization result corresponding to the specific operator or structure. For example, the output of the neural network can be directly the index of the optimizer corresponding to a specific operator or structure. The optimizer corresponding to the specific operator or structure can be found in the optimization library through this index.

[0068] An optimization library can consist of multiple optimizers, each of which can be used to optimize a specific operator or structure. Since the first subgraph is the specific operator or structure that exists in the first computation graph, the optimizer can be used to optimize the first subgraph.

[0069] In some possible implementations, there may be more than one first subgraph. For each first subgraph, in step S51, the optimizer corresponding to the first subgraph can be determined from the optimization library based on the identifier of the first subgraph. In step S52, the optimizer corresponding to the first subgraph is called from the optimization library to optimize the first subgraph. The optimization result is the second subgraph corresponding to the first subgraph. In step S53, the corresponding first subgraph in the first computation graph is replaced by the second subgraph corresponding to the first subgraph.

[0070] Perform the above steps on all first subgraphs, and the resulting computation graph is the second computation graph.

[0071] During the optimization process, there may be overlap between different first subgraphs. For example, first subgraph A and first subgraph B share operator C. After the optimizer corresponding to first subgraph A optimizes first subgraph A, it merges operator C with other operators for optimization. After using the second subgraph corresponding to first subgraph A to replace first subgraph A to optimize the first computation graph, the optimized computation graph no longer has first subgraph B, so it is impossible to use the second subgraph corresponding to first subgraph B to optimize the computation graph.

[0072] In some possible implementations, when multiple first subgraphs are determined, the first computation graph is optimized according to each first subgraph and its corresponding pre-set priority to obtain an optimized second computation graph.

[0073] Figure 6 This is a flowchart illustrating the steps of optimizing the first computation graph based on each first subgraph and its corresponding pre-defined priority. (Refer to...) Figure 6 Optimizing the first computation graph according to each first subgraph and the pre-set priority corresponding to the first subgraph may include steps S61 and S62.

[0074] In step S61, when multiple first subgraphs intersect, the priorities of the multiple first subgraphs are determined respectively;

[0075] In step S62, the first computation graph is optimized according to the priority order of the multiple first subgraphs.

[0076] For example, the first subgraph is optimized using the optimizer corresponding to the first subgraph in descending order of priority, resulting in the second subgraph. The second subgraph is then used to replace the first subgraph. If, when running a certain first subgraph, it is found that it has "disappeared" from the optimized first computation graph in the previous optimization process, then the first subgraph is not used to optimize the first computation graph.

[0077] The intersection between multiple first subgraphs refers to the fact that multiple first subgraphs share operators or structures.

[0078] If first subgraph A and first subgraph B share operator C, after the optimizer corresponding to first subgraph A optimizes first subgraph A, it merges operator C with other operators for optimization. Then, after replacing first subgraph A with the second subgraph corresponding to first subgraph A to optimize the first computation graph, the optimized computation graph no longer contains first subgraph B. Since the priority of first subgraph A is higher than that of first subgraph B in the pre-defined priority settings, the optimization of the first computation graph using first subgraph B is abandoned.

[0079] Meanwhile, the first subgraph B and the first subgraph D also share operator E. After the optimizer corresponding to the first subgraph B optimizes the first subgraph B, operator E will be merged with other operators for optimization. However, since the first subgraph B and the first subgraph A have an intersection, the first subgraph B is abandoned for optimization of the first computation graph. Therefore, the "conflict" between the first subgraph B and the first subgraph D "disappears", and the first subgraph D can be used to optimize the first computation graph.

[0080] Figure 7 and Figure 8 This is a schematic diagram of the overall flow of the computational graph optimization method according to an embodiment of the present disclosure.

[0081] Reference Figure 7 and Figure 8 The first computation graph is input into the graph neural network. The graph neural network detects the first computation graph, identifies specific structures or operators in the first computation graph, and calls the optimizer corresponding to the structure or operator from the optimization library to optimize the specific operator or structure, thereby obtaining the optimization result corresponding to the specific operator or structure.

[0082] The graph neural network used for detection in the first computation graph can be as follows: Figure 7 As shown, this is a collection of first-graph neural networks from a network library, which can also be represented as follows: Figure 8The diagram shown is a general graph neural network, also known as the second graph neural network.

[0083] As mentioned earlier, the optimization library consists of multiple optimizers, and the corresponding optimizer can be obtained from the optimization library by identifying a specific structure or operator.

[0084] After obtaining the optimization result corresponding to a specific operator or structure, the optimization result can be used to replace the specific operator or structure in the first computation graph to obtain the optimization result corresponding to the first computation graph, i.e., the second computation graph.

[0085] In some possible implementations, after obtaining the second computation graph, the second computation graph is compiled, and the compilation result is loaded and used.

[0086] Figure 9 This is a flowchart illustrating the steps involved in compiling the second computation graph and loading the compilation results. (Refer to...) Figure 9 The second computation graph is compiled, and the compilation result is loaded and used, including steps S91 and S92.

[0087] In step S91, the second computation graph is compiled to obtain the compilation result of the second computation graph.

[0088] In step S92, the compilation results are loaded into multiple processing cores of the many-core system so that the multiple processing cores can execute the processing tasks corresponding to the second computation graph.

[0089] In other words, the first computation graph can be an intermediate representation corresponding to the task loaded in the many-core system. For example, the first computation graph is an intermediate representation in graph form constructed based on the neural network loaded in the many-core system. The processing cores of the many-core system cannot directly run the computation graph. Therefore, after optimizing the first computation graph to obtain the second computation graph, the second computation graph needs to be compiled, and the compilation result is loaded into multiple processing cores of the many-core system so that the processing cores can run the compilation result, that is, execute the task corresponding to the neural network. This task includes any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks.

[0090] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0091] In addition, this disclosure also provides a computational graph optimization apparatus, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the computational graph optimization methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0092] Figure 10 This is a block diagram of a computational graph optimization apparatus provided in an embodiment of the present disclosure.

[0093] Reference Figure 10 This disclosure provides a computational graph optimization apparatus, which includes:

[0094] The acquisition module is used to acquire the first computational graph to be optimized;

[0095] The detection module is used to detect the first computation graph through a pre-trained graph neural network and determine the first subgraph to be optimized in the first computation graph;

[0096] The optimization module is used to optimize the first computation graph based on the first subgraph to obtain the optimized second computation graph.

[0097] Figure 11 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0098] Reference Figure 11 This disclosure provides an electronic device, which includes: at least one processor 1101; at least one memory 1102; and one or more I / O interfaces 1103 connected between the processor 1101 and the memory 1102; wherein the memory 1102 stores one or more computer programs that can be executed by the at least one processor 1101, and the one or more computer programs are executed by the at least one processor 1101 to enable the at least one processor 1101 to perform the above-described computational graph optimization method.

[0099] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the above-described computational graph optimization method. The computer-readable storage medium may be volatile or non-volatile.

[0100] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described computational graph optimization method.

[0101] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0102] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0103] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0104] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0105] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0106] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0107] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0108] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0110] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A computational graph optimization method, characterized in that, Applied to a compiler, the method includes: Obtain a first computational graph to be optimized; the first computational graph is an intermediate representation in graph form constructed based on a neural network, the neural network being used to perform processing tasks, the processing tasks including any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks; The first computation graph is detected by a pre-trained graph neural network to determine the first subgraph to be optimized in the first computation graph. The first subgraph is difficult to define with a unified and clear rule, and the first subgraph includes at least a branch structure that affects the intermediate storage of the hardware and an unbalanced structure that affects the utilization of the computation. The graph neural network is trained using the computation graph containing the first subgraph and the specific position of the first subgraph in the computation graph as positive samples. Based on the first subgraph, the first computation graph is optimized to obtain the optimized second computation graph; The second computation graph is compiled to obtain the compilation result of the second computation graph; The compilation results are loaded into multiple processing cores of the many-core system so that the multiple processing cores execute processing tasks corresponding to the second computation graph.

2. The method according to claim 1, characterized in that, The graph neural network includes multiple first graph neural networks in the network library; The step of detecting the first computation graph using a pre-trained graph neural network to determine the first subgraph to be optimized in the first computation graph includes: The multiple first graph neural networks in the network library are divided into multiple detection groups, and each detection group includes at least one graph neural network. Each detection group is used to detect the first computation graph and determine at least one first subgraph in the first computation graph.

3. The method according to claim 1, characterized in that, The graph neural network includes a second graph neural network; The step of detecting the first computation graph using a pre-trained graph neural network to determine the first subgraph to be optimized in the first computation graph includes: The first computation graph is detected by the second graph neural network to determine at least one first subgraph in the first computation graph.

4. The method according to claim 2 or 3, characterized in that, The step of optimizing the first computation graph based on the first subgraph to obtain an optimized second computation graph includes: Given multiple first subgraphs, the first computation graph is optimized according to each first subgraph and its corresponding pre-set priority to obtain an optimized second computation graph.

5. The method according to claim 4, characterized in that, The optimization of the first computation graph based on each first subgraph and its corresponding pre-set priority includes: When multiple first subgraphs intersect, determine the priority of each first subgraph. The first computation graph is optimized according to the priority order of multiple first subgraphs.

6. The method according to claim 1, characterized in that, The step of optimizing the first computation graph based on the first subgraph to obtain an optimized second computation graph includes: Based on the identifier of the first subgraph, determine the optimizer corresponding to the first subgraph; Invoke the optimizer corresponding to the first subgraph to optimize the first subgraph and obtain the second subgraph; The second computation graph is obtained by replacing the corresponding first subgraph in the first computation graph with the second subgraph.

7. A computational graph optimization device, characterized in that, Applied to a compiler, the apparatus includes: The acquisition module is used to acquire a first computational graph to be optimized; the first computational graph is an intermediate representation in graph form constructed based on a neural network, the neural network being used to perform processing tasks, the processing tasks including any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks; The detection module is used to detect the first computation graph through a pre-trained graph neural network to determine the first subgraph to be optimized in the first computation graph; the first subgraph is difficult to define with a unified and clear rule, and the first subgraph includes at least a branch structure that affects the intermediate storage of the hardware and an unbalanced structure that affects the utilization of the computation. The graph neural network is trained using the computation graph containing the first subgraph and the specific position of the first subgraph in the computation graph as positive samples. An optimization module is used to optimize the first computation graph based on the first subgraph to obtain an optimized second computation graph. The computational graph optimization device is also used for: The second computation graph is compiled to obtain the compilation result of the second computation graph; The compilation results are loaded into multiple processing cores of the many-core system so that the multiple processing cores execute processing tasks corresponding to the second computation graph.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the optimization method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the optimization method as described in any one of claims 1-6.

Images