Neural network optimization method, computer equipment and storage medium

Abstract

The invention relates to the field of model optimization, and provides a neural network optimization method, computer equipment and a storage medium, and the method comprises the steps: obtaining a directed acyclic graph corresponding to a neural network; identifying a global node in the directed acyclic graph, taking the global node as a first sub-graph, and removing the global node from the directed acyclic graph to obtain at least two connected regions; matching the preset structure in the connected region, taking the preset structure as a second sub-graph, and removing the second sub-graph from the connected region to obtain isolated nodes; and determining a second sub-graph corresponding to the isolated node, determining a target sub-graph corresponding to the directed acyclic graph according to the isolated node, the first sub-graph and the second sub-graph, and executing calculation corresponding to the neural network based on the target sub-graph. The time delay of carrying data in the off-chip cache and the on-chip cache in the calculation process is reduced, and the calculation efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of model optimization, and more particularly to a method for optimizing neural networks, a computer device, and a storage medium. Background Technology

[0002] Chips used to run neural networks typically consist of multiple computing cores, each with its own on-chip cache and a shared off-chip cache. Since on-chip caches are usually small, much data needs to be moved from the off-chip cache before computation, and then the output is moved out to the off-chip cache after computation; if subsequent computations require the output, it is moved back from the off-chip cache. This frequent data movement leads to significant memory access overhead, resulting in unnecessary execution latency and power consumption. Optimizing the computation process of neural networks to reduce the overhead and latency of data movement between on-chip and off-chip caches, and improving the computational efficiency of neural networks, has become a pressing issue. Summary of the Invention

[0003] The main objective of this application is to provide an optimization method, computer device, and storage medium for neural networks, aiming to improve the computational efficiency of neural networks and reduce computational latency.

[0004] In a first aspect, this application provides a method for optimizing a neural network, the method comprising the following steps: Obtain the directed acyclic graph corresponding to the neural network, wherein the directed acyclic graph is used to represent the data flow in the neural network, including operation nodes and connection relationships; The global nodes in the directed acyclic graph are identified, and the global nodes are used as the first subgraph. The global nodes are removed from the directed acyclic graph to obtain at least two connected regions. The global nodes are the operation nodes in the directed acyclic graph that obtain input data from the off-chip cache and store the output data in the off-chip cache. In the connected region, a preset structure is matched, the preset structure is used as a second subgraph, and the second subgraph is removed from the connected region to obtain isolated nodes. The preset structure includes at least one preset operation node and a preset connection relationship between the preset operation nodes. A second subgraph corresponding to the isolated node is determined. Based on the isolated node, the first subgraph, and the second subgraph, a target subgraph corresponding to the directed acyclic graph is determined. The computation corresponding to the neural network is then performed based on the target subgraph.

[0005] Secondly, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the neural network optimization method described above.

[0006] Thirdly, this application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the neural network optimization method described above.

[0007] This application provides a method for optimizing a neural network, a computer device, and a storage medium. The method involves obtaining a directed acyclic graph (DAG) corresponding to the neural network, where the DAG represents the data flow within the neural network, including operation nodes and connection relationships. Global nodes in the DAG are identified, designated as a first subgraph, and removed from the DAG to obtain at least two connected regions. The global nodes are operation nodes in the DAG that retrieve input data from an off-chip cache and store output data in the off-chip cache. A preset structure is matched within the connected regions, designated as a second subgraph, and removed from the connected regions to obtain isolated nodes. The preset structure includes at least one preset operation node and preset connection relationships between these preset operation nodes. The second subgraph corresponding to the isolated node is determined. Based on the isolated node, the first subgraph, and the second subgraph, a target subgraph corresponding to the DAG is determined, and the computation corresponding to the neural network is performed based on the target subgraph. By dividing the directed acyclic graph (DAG) corresponding to the neural network into subgraphs based on global nodes, the latency consumed by moving data between off-chip and on-chip caches during neural network computation is reduced; by dividing the DAG corresponding to the neural network into subgraphs based on a pre-defined structure, the rationality of data dependencies in the subgraphs is improved; thereby improving the computational efficiency of the neural network. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A flowchart illustrating a neural network optimization method provided in an embodiment of this application; Figure 2 A schematic diagram of a directed acyclic graph provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the transfer of data between the operating node and the on-chip cache, provided for related technologies; Figure 4 A schematic diagram illustrating the transfer of data between an operating node and an on-chip cache, provided as an embodiment of this application; Figure 5 A schematic diagram of global nodes and connected regions in a directed acyclic graph provided in an embodiment of this application; Figure 6 A schematic diagram of connected regions and module subgraphs in a directed acyclic graph provided in an embodiment of this application; Figure 7 A flowchart illustrating a sub-step of step S103 provided in an embodiment of this application; Figure 8 A schematic diagram of a second isolated node and a candidate subgraph provided in an embodiment of this application; Figure 9 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0012] This application provides a method for optimizing a neural network, a computer device, and a storage medium.

[0013] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0014] Please refer to Figure 1 , Figure 1This is a flowchart illustrating a neural network optimization method provided in an embodiment of this application. This neural network optimization method can be used in a terminal or server to divide the directed acyclic graph of the neural network into multiple target subgraphs. The terminal can be an electronic device such as a mobile phone, tablet, laptop, desktop computer, personal digital assistant, or wearable device; the server can be a standalone server, a server cluster, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0015] like Figure 1 As shown, the optimization method of the neural network includes steps S101 to S104.

[0016] Step S101: Obtain the directed acyclic graph corresponding to the neural network. The directed acyclic graph is used to represent the data flow in the neural network, including operation nodes and connection relationships.

[0017] Please refer to Figure 2 , Figure 2 This is a schematic diagram of a directed acyclic graph provided in an embodiment of this application.

[0018] like Figure 2 As shown, neural networks can be represented by a Directed Acyclic Graph (DAG), which maps the various operations (such as convolution, activation functions, addition, etc.) and data flow relationships in the neural network to the nodes and edges of the graph. Figure 2 Each ring in the diagram represents an operation node, or an operator (Op). Figure 2 The edges with arrows indicate the connection between the operation nodes, and the arrows indicate the direction of data flow.

[0019] Understandably, visualizing the data flow in a neural network using a directed acyclic graph (DAG) can intuitively demonstrate the structure of the neural network and the relationship between data flow, which is beneficial for efficient gradient calculation and facilitates automatic differentiation of the neural network. It also makes it easy to perform operations such as pruning and grouping on the neural network based on the DAG to optimize model performance.

[0020] Step S102: Identify the global nodes in the directed acyclic graph, take the global nodes as the first subgraph, and remove the global nodes from the directed acyclic graph to obtain at least two connected regions. The global nodes are the operation nodes in the directed acyclic graph that obtain input data from the off-chip cache and store the output data in the off-chip cache.

[0021] For example, the inputs and outputs of a global node are both on an off-chip cache, so no on-chip cache planning is required when executing a global node. The off-chip cache can be, for example, Double Data Rate SDRAM (DDR).

[0022] Please refer to Figure 3 , 4 , Figure 3 A schematic diagram illustrating the transfer of data between the operating node and the on-chip cache, provided for related technologies; Figure 4 This is a schematic diagram illustrating the transfer of data between an operating node and an on-chip cache, provided as an embodiment of this application.

[0023] For example, Op_1, Op_2, Op_3, and Op_4 represent different operators. In related technologies, such as... Figure 3 As shown, each operator needs to output its result to DDR, and the next operator then moves the result of the previous operator from DDR as input to the on-chip cache. This frequent data movement between the off-chip and on-chip caches causes latency. Figure 4 As shown, Op_2 is identified as a global node, and the operators before Op_2 and the operators after Op_2 are merged into two subgraphs respectively. For example, Op_3 and Op_4 are merged into one subgraph. Op_4 can directly obtain the output of Op_3 without having to move the output of Op_3 from DDR to the on-chip cache. This reduces the latency required to move data between the off-chip cache and the on-chip cache, thereby improving the computational efficiency of the neural network.

[0024] For example, a preset node belonging to the global node can be pre-defined, so as to identify the preset node in the operation node, in order to identify the global node in the directed acyclic graph. The preset node can be, for example, the Load operation node, the Store operation node, etc., without limitation.

[0025] Please refer to Figure 5 , Figure 5 This is a schematic diagram of global nodes and connected regions in a directed acyclic graph provided in an embodiment of this application.

[0026] like Figure 5As shown, the red ring is identified as the operation node. After the global node is extracted, two connected regions are obtained: connected region 1 and connected region 2.

[0027] Step S103: Match the preset structure in the connected region, take the preset structure as the second subgraph, and remove the second subgraph from the connected region to obtain isolated nodes. The preset structure includes at least one preset operation node and a preset connection relationship between the preset operation nodes.

[0028] For example, neural networks contain modules with specific structures, such as attention modules and MLP modules. These modules consist of multiple operators, and the operators within the same module often have data dependencies. Assigning operators within the same module to different subgraphs may lead to incorrect data dependencies. Therefore, this application identifies module subgraphs by matching preset structures in connected regions, such as identifying the structures corresponding to attention modules and MLP modules in connected regions, thus avoiding assigning operators within the same module to different subgraphs.

[0029] Please refer to Figure 6 , Figure 6 This is a schematic diagram of connected regions and module subgraphs in a directed acyclic graph provided in an embodiment of this application.

[0030] For example, Figure 6 The yellow and green nodes in the graph represent module subgraphs with different structures. Figure 6 The orange nodes in the graph are the isolated nodes remaining after removing the module subgraph.

[0031] For example, a predefined structure can be searched in a connected region using graph search algorithms, such as Depth-First Search (DFS) and Breadth-First Search (BFS). The predefined structure defines the types of each predefined operation node that makes up the predefined structure, as well as the predefined connection relationships between these predefined operation nodes.

[0032] Please see Figure 7 , Figure 7 This is a flowchart illustrating a sub-step of step S103 provided in an embodiment of this application.

[0033] like Figure 7 As shown, in some embodiments, step S103 matches a preset structure in the connected region, uses the preset structure as a second subgraph, and removes the second subgraph from the connected region to obtain isolated nodes, including: Step S1031: Determine the complexity of at least one of the preset structures, wherein the complexity is positively correlated with the number of preset operation nodes and the number of preset connection relationships in the preset structure; Step S1032: Match a first preset structure in the connected region according to the complexity, use the first preset structure as a second subgraph, and remove the second subgraph from the connected region to obtain the first connected region; Step S1033: Match a second preset structure in the first connected region according to the complexity, take the second preset structure as the second subgraph, and remove the second subgraph from the first connected region to obtain the second connected region; Step S1034: After completing the matching of each of the preset structures, the remaining operation nodes in the second connected region are determined as the isolated nodes; The complexity of the first preset structure is greater than that of the second preset structure.

[0034] For example, matching a preset structure in a connected region requires traversing the connected region. To reduce the computational cost of traversing the connected region, the preset structures with higher complexity can be identified first, followed by the preset structures with lower complexity, until all preset structures have been traversed.

[0035] For example, the complexity of a preset structure can be determined based on the number of operation nodes and connections in the preset structure. Specifically, the preset structures are sorted from most to least number of operation nodes, and for preset structures with the same number of operation nodes, they are sorted from most to least number of connections, thus obtaining a sorting of complexity from high to low. Of course, this is not limited to this method and is not restricted here.

[0036] For example, the complexity of the attention module is greater than that of the MLP module. First, the attention module is matched in the connected region, and all matched attention modules are removed. Then, the MLP module is matched in the connected region after the attention modules are removed.

[0037] Step S104: Determine the second subgraph corresponding to the isolated node, determine the target subgraph corresponding to the directed acyclic graph based on the isolated node, the first subgraph and the second subgraph, and perform the computation corresponding to the neural network based on the target subgraph.

[0038] For example, after removing the preset structure, there are still isolated nodes in the directed acyclic graph that have weak data dependencies with other operation nodes. These isolated nodes can form a target subgraph with the connected second subgraph. Therefore, the second subgraph corresponding to the isolated node is determined.

[0039] In some implementations, determining the second subgraph corresponding to the isolated node, and determining the target subgraph corresponding to the directed acyclic graph based on the isolated node, the first subgraph, and the second subgraph, includes: Obtain the connection relationship between the isolated node and each of the second subgraphs; If there is a connection between the isolated node and a second subgraph, the isolated node is identified as the first isolated node, the second subgraph is identified as the target second subgraph corresponding to the first isolated node, and the first isolated node is added to the target second subgraph to obtain a candidate subgraph. If there is a connection between the isolated node and at least two of the second subgraphs, the isolated node is identified as a second isolated node. Determine the target second subgraph corresponding to the second isolated node from at least two second subgraphs, add the second isolated node to the target second subgraph to obtain a candidate subgraph, and use the candidate subgraph and the first subgraph as the target subgraph.

[0040] For example, if an isolated node is connected to only one second subgraph, then that second subgraph is designated as the target second subgraph, so that the isolated node and the target second subgraph form the target subgraph. However, if an isolated node is connected to two or more second subgraphs, then a target second subgraph needs to be randomly selected from the two or more second subgraphs. For instance, a target second subgraph can be randomly selected from the second subgraphs, so that the isolated node and the target second subgraph form a target candidate subgraph.

[0041] For example, each candidate subgraph and the first subgraph are treated as independent groups in the directed acyclic graph, i.e., the target subgraph.

[0042] In some implementations, determining the target second subgraph corresponding to the second isolated node from at least two second subgraphs includes: Obtain at least two second subgraphs that are connected to the second isolated node, and determine the complexity and / or inflation coefficient of each of the at least two second subgraphs. The target second subgraph is determined from at least two second subgraphs based on the complexity and / or the inflation coefficient.

[0043] Please refer to Figure 8 , Figure 8 This is a schematic diagram of a second isolated node and a candidate subgraph provided in an embodiment of this application.

[0044] like Figure 8As shown, Op_5 is a second isolated node that is connected to both the second subgraph A and the second subgraph B. In related technologies, Op_5 will randomly form candidate subgraphs with the second subgraph A and the second subgraph B, for example, with the second subgraph A before Op_5. However, when the second subgraph A itself is already quite complex, adding Op_5 to the second subgraph A will cause an unreasonable increase in the complexity of the second subgraph A, resulting in a complexity imbalance with other subgraphs.

[0045] In the neural network optimization method provided in this application embodiment, a target second subgraph is determined by identifying a second isolated node based on the complexity and / or inflation coefficient of the second subgraph. For example, the second isolated node will be combined with a second subgraph with lower complexity to form a candidate subgraph, avoiding an imbalance in complexity between different target subgraphs; alternatively, the candidate subgraph with a lower inflation coefficient can be determined as the target second subgraph.

[0046] For example, the dilation coefficient is used to reflect the degree to which the output size increases relative to the input size. The larger the dilation coefficient, the more spatial detail and resolution the candidate subgraph adds when processing data, thus reflecting the more complex the structure of the candidate subgraph.

[0047] For example, in order to avoid the addition of a second isolated node further increasing the complexity or expansion coefficient of the second subgraph, resulting in a large difference in complexity or expansion coefficient between different target subgraphs, the second subgraph with smaller complexity and / or expansion coefficient is preferentially identified as the target second subgraph.

[0048] In some implementations, obtaining at least two second subgraphs connected to the second isolated node, and determining the complexity and / or bloat coefficient of each of the at least two second subgraphs, includes: The complexity is determined based on the number of operation nodes and the number of connection relationships in the second subgraph; and / or, Obtain the first size of the input feature map and the second size of the output feature map of the second sub-map, and determine the expansion coefficient based on the ratio of the second size to the first size.

[0049] For example, the complexity of the second subgraph is proportional to the number of operation nodes and the number of connections, respectively. Specifically, the second subgraphs are sorted from most to least number of operation nodes, and for second subgraphs with the same number of operation nodes, they are sorted from most to least number of connections, thus obtaining a sorting of complexity from high to low.

[0050] For example, the dilation coefficient is used to reflect the degree to which the output size increases relative to the input size; therefore, the dilation coefficient is obtained by comparing the first size of the input feature map and the second size of the output feature map. Assume the size of the input feature map is... The size of the output feature map is Then the coefficient of expansion is .

[0051] In some embodiments, the method further includes: The memory requirements of the operation nodes in each of the target subgraphs are accumulated based on the depth-first topological order to obtain the cumulative memory. If the accumulated memory is greater than the preset memory, the target subgraph is split into a first target subgraph and a second target subgraph at the operation node corresponding to the current item.

[0052] For example, to avoid insufficient on-chip cache memory to support the operation of the target subgraph (e.g., insufficient memory to simultaneously store the outputs of each operation node within the target subgraph), the cumulative memory of the operation nodes in the target subgraph is calculated. If the final cumulative memory is less than the preset memory, it means that the target subgraph can run on the on-chip cache. Conversely, if the cumulative memory at a certain operation node is greater than the preset memory, it means that the on-chip cache is insufficient to support the operation of the target subgraph. At that operation node, the target subgraph is split into a first target subgraph and a second target subgraph. The first target subgraph includes all operation nodes before the current operation node in the target subgraph, and the second target subgraph includes the current operation node and all subsequent operation nodes in the target subgraph, thereby avoiding insufficient on-chip cache to support the operation of the target subgraph.

[0053] The preset memory size can be set according to actual needs. For example, it can be the size of the on-chip cache, or smaller than the size of the on-chip cache. There is no limitation here.

[0054] In some embodiments, the method further includes: Obtain the first execution delay of the computational cost of executing the neural network based on the target subgraph, and the second execution delay of the computational cost of executing the neural network based on the connected region; If the second execution delay is less than the first execution delay, the first subgraph and the connected region are determined as the target subgraph.

[0055] For example, due to the complexity of neural network computation, involving the arrangement of transport and computation instructions, the execution latency of the target subgraph after partitioning may not be less than the execution latency of the first subgraph and connected regions in step S102. To avoid the execution latency of the target subgraph being greater than that of the first subgraph and connected regions, the first and second execution latencies are compared, and the one with the smaller latency is selected as the group partitioning method during computation, thereby reducing the overall execution latency of the neural network and improving computational efficiency.

[0056] In some implementations, performing the computation corresponding to the neural network based on the target subgraph includes: The operation data of each operation node in the target subgraph is obtained from the off-chip cache, and the operation data includes at least one of input data and weight data; The operation nodes in the target subgraph are run on the on-chip cache, and the output data of the target subgraph is stored in the off-chip cache.

[0057] Understandably, after dividing the directed acyclic graph into target subgraphs, during computation, it is only necessary to move the operation data required by each target subgraph to the on-chip cache before the execution of each target subgraph, and then move the output generated by the execution of the target subgraph to the off-chip cache after the execution of the target subgraph is completed. This reduces the latency caused by data movement during neural network computation and improves the computational efficiency of the neural network.

[0058] For example, the above-described method and apparatus can be implemented as a computer program, which can be used in, for example... Figure 9 It runs on the computer device shown.

[0059] Please see Figure 9 , Figure 9 This is a schematic block diagram illustrating the structure of a computer device provided in an embodiment of this application. The computer device may be a server or a terminal.

[0060] like Figure 9 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and internal memory.

[0061] The storage medium can store the operating system and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any optimization method for a neural network.

[0062] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0063] Internal memory provides an environment for the execution of computer programs stored in the storage medium. When these computer programs are executed by the processor, the processor can perform any optimization method for a neural network.

[0064] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0065] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0066] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Obtain the directed acyclic graph corresponding to the neural network, wherein the directed acyclic graph is used to represent the data flow in the neural network, including operation nodes and connection relationships; The global nodes in the directed acyclic graph are identified, and the global nodes are used as the first subgraph. The global nodes are removed from the directed acyclic graph to obtain at least two connected regions. The global nodes are the operation nodes in the directed acyclic graph that obtain input data from the off-chip cache and store the output data in the off-chip cache. In the connected region, a preset structure is matched, the preset structure is used as a second subgraph, and the second subgraph is removed from the connected region to obtain isolated nodes. The preset structure includes at least one preset operation node and a preset connection relationship between the preset operation nodes. A second subgraph corresponding to the isolated node is determined. Based on the isolated node, the first subgraph, and the second subgraph, a target subgraph corresponding to the directed acyclic graph is determined. The computation corresponding to the neural network is then performed based on the target subgraph.

[0067] In one embodiment, the step of matching a preset structure in the connected region, using the preset structure as a second subgraph, and removing the second subgraph from the connected region to obtain isolated nodes includes: Determine the complexity of at least one of the preset structures, wherein the complexity is positively correlated with the number of preset operation nodes and the number of preset connection relationships in the preset structure; Based on the complexity, a first preset structure is matched in the connected region, the first preset structure is used as a second subgraph, and the second subgraph is removed from the connected region to obtain the first connected region. Based on the complexity, a second preset structure is matched in the first connected region, the second preset structure is used as the second subgraph, and the second subgraph is removed from the first connected region to obtain the second connected region. After completing the matching of each of the preset structures, the remaining operation nodes in the second connected region are determined as the isolated nodes; The complexity of the first preset structure is greater than that of the second preset structure.

[0068] In one embodiment, determining the second subgraph corresponding to the isolated node, and determining the target subgraph corresponding to the directed acyclic graph based on the isolated node, the first subgraph, and the second subgraph, includes: Obtain the connection relationship between the isolated node and each of the second subgraphs; If there is a connection between the isolated node and a second subgraph, the isolated node is identified as the first isolated node, the second subgraph is identified as the target second subgraph corresponding to the first isolated node, and the first isolated node is added to the target second subgraph to obtain a candidate subgraph. If there is a connection between the isolated node and at least two of the second subgraphs, the isolated node is identified as a second isolated node. Determine the target second subgraph corresponding to the second isolated node from at least two second subgraphs, add the second isolated node to the target second subgraph to obtain a candidate subgraph, and use the candidate subgraph and the first subgraph as the target subgraph.

[0069] In one embodiment, determining the target second subgraph corresponding to the second isolated node from at least two second subgraphs includes: Obtain at least two second subgraphs that are connected to the second isolated node, and determine the complexity and / or inflation coefficient of each of the at least two second subgraphs. The target second subgraph is determined from at least two second subgraphs based on the complexity and / or the inflation coefficient.

[0070] In one embodiment, obtaining at least two second subgraphs connected to the second isolated node, and determining the complexity and / or inflation coefficient of each of the at least two second subgraphs, includes: The complexity is determined based on the number of operation nodes and the number of connection relationships in the second subgraph; and / or, Obtain the first size of the input feature map and the second size of the output feature map of the second sub-map, and determine the expansion coefficient based on the ratio of the second size to the first size.

[0071] In one embodiment, the method further includes: The memory requirements of the operation nodes in each of the target subgraphs are accumulated based on the depth-first topological order to obtain the cumulative memory. If the accumulated memory is greater than the preset memory, the target subgraph is split into a first target subgraph and a second target subgraph at the operation node corresponding to the current item.

[0072] In one embodiment, the method further includes: Obtain the first execution delay of the computational cost of executing the neural network based on the target subgraph, and the second execution delay of the computational cost of executing the neural network based on the connected region; If the second execution delay is less than the first execution delay, the first subgraph and the connected region are determined as the target subgraph.

[0073] In one embodiment, performing the computation corresponding to the neural network based on the target subgraph includes: The operation data of each operation node in the target subgraph is obtained from the off-chip cache, and the operation data includes at least one of input data and weight data; The operation nodes in the target subgraph are run on the on-chip cache, and the output data of the target subgraph is stored in the off-chip cache.

[0074] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described neural network optimization method can be referred to the corresponding process in the aforementioned neural network optimization method embodiments, and will not be repeated here.

[0075] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to various embodiments of the neural network optimization method of this application.

[0076] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0077] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0078] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0079] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of optimizing a neural network, the method comprising: The method comprises: obtaining a directed acyclic graph corresponding to a neural network, the directed acyclic graph being used to represent data flow in the neural network, comprising operation nodes and connection relationships; identifying a global node in the directed acyclic graph, taking the global node as a first subgraph, and removing the global node from the directed acyclic graph to obtain at least two connected regions, wherein the global node is an operation node that obtains input data from an off-chip cache and stores output data to the off-chip cache in the directed acyclic graph; matching a preset structure in the connected region, taking the preset structure as a second subgraph, and removing the second subgraph from the connected region to obtain an isolated node, wherein the preset structure comprises at least one preset operation node and a preset connection relationship between the preset operation nodes; determining a second subgraph corresponding to the isolated node, determining a target subgraph corresponding to the directed acyclic graph according to the isolated node, the first subgraph and the second subgraph, and executing calculation corresponding to the neural network based on the target subgraph.

2. The method of optimizing a neural network of claim 1, wherein, The matching of the preset structure in the connected region, taking the preset structure as a second subgraph, and removing the second subgraph from the connected region to obtain an isolated node comprises: determining the complexity of at least one preset structure, wherein the complexity is positively correlated with the number of preset operation nodes and the number of preset connection relationships in the preset structure; matching a first preset structure in the connected region according to the complexity, taking the first preset structure as a second subgraph, and removing the second subgraph from the connected region to obtain a first connected region; matching a second preset structure in the first connected region according to the complexity, taking the second preset structure as a second subgraph, and removing the second subgraph from the first connected region to obtain a second connected region; after completing the matching of each preset structure, determining the operation nodes remaining in the second connected region as the isolated node; wherein the complexity of the first preset structure is greater than the complexity of the second preset structure.

3. The method of claim 1, wherein, The determination of the second subgraph corresponding to the isolated node, the determination of the target subgraph corresponding to the directed acyclic graph according to the isolated node, the first subgraph and the second subgraph comprises: obtaining the connection relationship between the isolated node and each second subgraph; in the case that there is a connection relationship between the isolated node and a second subgraph, determining the isolated node as a first isolated node, determining the second subgraph as a target second subgraph corresponding to the first isolated node, and adding the first isolated node to the target second subgraph to obtain a candidate subgraph; in the case that there is a connection relationship between the isolated node and at least two second subgraphs, determining the isolated node as a second isolated node; determining a target second subgraph corresponding to the second isolated node from at least two second subgraphs, adding the second isolated node to the target second subgraph to obtain a candidate subgraph, and taking the candidate subgraph and the first subgraph as the target subgraph.

4. The method of optimizing a neural network of claim 3, wherein, The determining the target second subgraph corresponding to the second isolated node from the at least two second subgraphs comprises: obtaining at least two second subgraphs connected with the second isolated node, and determining complexity and / or an inflation coefficient of each of the at least two second subgraphs respectively; determining the target second subgraph from the at least two second subgraphs according to the complexity and / or the inflation coefficient.

5. The method of optimizing a neural network of claim 4, wherein, The obtaining at least two second subgraphs connected with the second isolated node, and determining complexity and / or an inflation coefficient of each of the at least two second subgraphs respectively comprises: determining the complexity according to the number of operation nodes and the number of connection relationships in the second subgraph; and / or obtaining a first size of an input feature map and a second size of an output feature map of the second subgraph, and determining the inflation coefficient according to a ratio of the second size to the first size.

6. The method of Claim 1, wherein, The method further comprises: accumulating memory requirements of operation nodes in each of the target subgraphs based on a depth-first topological order to obtain cumulative memory; in a case where the cumulative memory is greater than a preset memory, splitting the target subgraph into a first target subgraph and a second target subgraph at the operation node corresponding to the current item.

7. The method of optimizing a neural network of claim 1, wherein, The method further comprises: obtaining a first execution latency corresponding to computation consumption of the neural network based on the target subgraph, and a second execution latency corresponding to computation consumption of the neural network based on the connected region; in a case where the second execution latency is less than the first execution latency, determining the first subgraph and the connected region as the target subgraph.

8. The method of optimizing a neural network according to any one of claims 1-7, wherein, The computation based on the target subgraph executing the neural network comprises: obtaining operation data of each operation node in the target subgraph from the off-chip cache, the operation data comprising at least one of input data and weight data; running each of the operation nodes in the target subgraph on the on-chip cache, and storing output data of the target subgraph into the off-chip cache.

9. A computer device, comprising: The computer device comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the optimization method of the neural network according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein when the computer program is executed by the processor, the steps of the optimization method of the neural network according to any one of claims 1 to 8 are implemented.

Images