Resource-constrained terminal-oriented depth model computational graph peak memory optimization method

Abstract

The embodiment of the invention relates to the technical field of peak value memory optimization, in particular to a resource-constrained terminal-oriented depth model computational graph peak value memory optimization method, which comprises the following steps of: obtaining a computational graph of a depth model, and if a network architecture of the depth model is an irregular network architecture, obtaining a resource-constrained terminal; searching by using an optimal topological sorting search algorithm oriented to irregular network architecture; when the searching step i is started, current state information is obtained, the current state information comprises a scheduling sequence Si and memory occupation mu i up to the current and peak memory occupation # imgabs0 # up to the current, and an unscheduled node ui is selected for scheduling; after the ui is scheduled, the output of the ui is activated to form Si + 1 and mui + 1; updating the peak memory occupation to obtain # imgabs1 #, releasing the node of which the out-degree is changed into 0 before ui, and updating mu i + 1; and entering the next searching step until an optimal solution meeting all constraint conditions is found, and obtaining and executing an optimal execution sequence. According to the method, the optimization of the use of the peak memory during the running of the depth model is realized.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of peak memory optimization technology, and in particular to a method for peak memory optimization of deep model computation graphs for resource-constrained terminals. Background Art

[0002] With the rapid development of AIOT (Artificial Intelligence of Things) technology, deep models are increasingly becoming "marginalized". However, the success of deep models is often at the cost of very high computational complexity. In edge computing scenarios, the memory capacity of terminal devices is limited and often cannot meet the computational requirements of deep models. In order to overcome this challenge, some model compression technologies have emerged. Methods such as pruning, quantization, and knowledge distillation can reduce the size of the model and reduce the peak memory by reducing the number of model parameters, reducing the complexity of the model, or extracting key information from the model, thereby adapting to the computing and storage capabilities of terminal devices. These technologies have achieved remarkable results in the past few years and have been widely used in practical applications.

[0003] However, with the continuous development of model compression technology, its optimization space has gradually become smaller, and it is necessary to find deeper optimization methods. In recent years, computational graph optimization has become a hot research direction. By modifying the computational graph, the computational amount and peak memory of the model can be effectively reduced. Some technologies such as operator fusion and operator rearrangement have been proposed to solve the problem of limited computing and storage resources on terminal devices. However, how to automatically determine the optimal operator rearrangement scheme is an NP-hard problem, which requires efficient search algorithms and heuristic methods to achieve. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a peak memory optimization method for deep model computation graphs for resource-constrained terminals, design an optimal execution order search algorithm based on dynamic programming that conforms to the characteristics of the network architecture, fully tap the huge potential of operator rearrangement technology in reducing the peak memory occupancy during deep model runtime, and achieve the optimization of peak memory usage during deep model runtime.

[0005] In order to achieve the above-mentioned purpose, an embodiment of the present application proposes a method for optimizing the peak memory of a deep model calculation graph for a resource-constrained terminal, comprising the following steps: obtaining a calculation graph of a deep model, and if the network architecture of the deep model is an irregular network architecture, searching using an optimal topological sorting search algorithm for an irregular network architecture; at the beginning of the search step i, obtaining the current state information, including the scheduling sequence S so far; i , for the current signature Z i The memory usage of μ i, and the peak memory usage so far For every unique zero-indegree set z i , select an unscheduled node u i Scheduling; Among them, the zero-indegree set z i Represents the set of nodes that can be scheduled currently; when u i After being dispatched, u i The output is activated and added to S i S i+1 , and is assigned to μ i In order to form μ i+1 ; Based on μ i+1 and Update the peak memory usage to get Check u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 ; At the end of search step i, save S i+1 , μ i+1 and Go to the next search step until the optimal solution that satisfies all constraints is found, and the optimal execution order is obtained and executed.

[0006] In order to achieve the above-mentioned purpose, the embodiment of the present application also proposes a deep model calculation graph peak memory optimization system for resource-constrained terminals, the system comprising: a calculation graph determination module, used to obtain the calculation graph of the deep model, if the network architecture of the deep model is an irregular network architecture, then the search is performed using the optimal topological sorting search algorithm for the irregular network architecture; a search initialization module, used to obtain the current state information at the beginning of the search step i, including the scheduling sequence S so far i , for the current signature Z i The memory usage of μ i , and the peak memory usage so far The scheduling module is used for each unique zero-indegree set z i , select an unscheduled node u i Scheduling; Among them, the zero-indegree set z i Represents the set of nodes that can be scheduled currently; update module, used when u i After being dispatched, u i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i In order to form μ i+1 , based on μ i+1 and Update the peak memory usage to get Release module for checking u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 ; Solving module, used to save S at the end of search step i i+1 , μ i+1 and Enter the next search step until the optimal solution that meets all constraints is found; the optimization execution module is used to obtain the optimal execution order based on the optimal solution and execute it.

[0007] In order to achieve the above-mentioned objectives, an embodiment of the present application also proposes a server, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described above.

[0008] In order to achieve the above-mentioned objectives, an embodiment of the present application also proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the peak memory optimization method of the deep model calculation graph for resource-constrained terminals as described above.

[0009] The peak memory optimization method for deep model calculation graphs for resource-constrained terminals proposed in the embodiments of the present application, for deep models with irregular network architectures and deep models with multi-branch network architectures, respectively designs the best execution order search algorithm based on dynamic programming that meets their respective characteristics, to fully tap the huge potential of operator rearrangement technology in reducing the peak memory usage of deep models during operation. In the process of searching and optimizing, the calculation graph is used as an effective means of compiling layer optimization without changing the model architecture, which can effectively maintain the mathematical integrity of the neural network, and keep the accuracy of the model as unaffected as possible while keeping the peak memory low. This optimization method can be orthogonally combined with model lightweight technologies at other algorithm layers and memory layers, and shows a more complete advantage in edge intelligence scenarios. The optimal topological sorting search algorithm for irregular network architectures enumerates all execution plans recursively, and the optimal topological sorting search algorithm for multi-branch network architectures enumerates all execution plans through a backtracking algorithm, with very low time complexity. In summary, this method quickly, highly and flexibly achieves the optimization of peak memory usage when deep models are running.

[0010] In addition, there are n nodes in the computational graph G of the deep model. When searching using the optimal topological sorting search algorithm for irregular network architecture, it is necessary to iterate n-1 steps. First, the memory table M0 is initialized. S0 is empty, μ0=0, z0=zero-indegree(s0,G); For search step i, in forming S i+1 After that, based on S i+1 Right i Update to form z i+1 ; At the end of search step i, save S i+1 , μ i+1 , and z i+1 , get the memory table M i+1 ; After completing all n-1 steps of iteration, the memory table M containing the optimal execution order is obtained n .

[0011] In addition, i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i In order to form μ i+1 , achieved through the following formula: S i+1 =S i ·append(u i ); μ i+1 =μ i +∏(u i ·shape); append(·) is a list adding function, which is used to add an element to the end of the list, u i ·Shape means calling u i Memory usage based on S i+1 Right i Update to form z i+1 , which is achieved by the following formula: i+1 =zero-indegree(S i+1 ,G).

[0012] In addition, based on μ i+1 and Update the peak memory usage to get Includes: Compare μ i+1 and The size of i+1 Greater than Then μ i+1 As If μ i+1 Less than or equal to Then As

[0013] In addition, if the network architecture of the deep model is a multi-branch network architecture, the optimal topological sorting search algorithm for the multi-branch network architecture is used for search; first, based on the calculation graph of the deep model, the tensors at all branches are determined, each tensor is divided into constants and variables, and all constants are excluded; each branch is traversed, and for any branch, all possible execution orders are enumerated, and the peak memory usage of each execution order is calculated respectively, and the execution order with the smallest peak memory usage is selected as the optimal execution order; after traversing all branches, the optimal execution order of the entire deep model is obtained and executed.

[0014] In addition, when searching using the optimal topological sorting search algorithm for a multi-branch network architecture, it is stipulated that the same operator will not be executed multiple times in the plan. The provision is implemented by checking whether the current tensor is a predecessor of other tensors to ensure the uniqueness of the operator in the scheduling plan; in the process of searching using the optimal topological sorting search algorithm for a multi-branch network architecture, the calculated results are recorded and cached to avoid repeated calculation of the memory usage of the same tensor.

[0015] In addition, after selecting the execution order with the smallest peak memory usage as the optimal execution order, it also includes: judging whether the optimal execution order is consistent with the default execution order; if the optimal execution order is consistent with the default execution order, directly executing the default execution order; if the optimal execution order is inconsistent with the default execution order, calling the operator reordering method to rearrange the operators according to the optimal execution order to execute the optimal execution order. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of a method for optimizing peak memory of a deep model calculation graph for a resource-constrained terminal provided in one embodiment of the present application;

[0018] Figure 2 It is a pseudo code diagram of an optimal topological sorting search algorithm for irregular network architecture provided in one embodiment of the present application;

[0019] Figure 3 It is a pseudo code diagram of an optimal topological sorting search algorithm for a multi-branch network architecture provided in one embodiment of the present application;

[0020] Figure 4It is a structural schematic diagram of a deep model calculation graph peak memory optimization system for resource-constrained terminals provided in another embodiment of the present application;

[0021] Figure 5 It is a schematic diagram of the structure of a server provided in another embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0023] In order to solve the NP-hard problem of how to automatically determine the optimal operator rearrangement scheme, an embodiment of the present application proposes a peak memory optimization method for a deep model calculation graph for a resource-constrained terminal, which is applied to a server. The implementation details of the peak memory optimization method for a deep model calculation graph for a resource-constrained terminal proposed in this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this solution.

[0024] The specific process of the peak memory optimization method for deep model calculation graphs for resource-constrained terminals proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0025] Step 101, obtain the calculation graph of the deep model. If the network architecture of the deep model is an irregular network architecture, use the optimal topological sorting search algorithm for irregular network architecture to search.

[0026] In the specific implementation, the server obtains the deep model that needs to be optimized and the calculation graph of the deep model, and determines the network architecture of the deep model through the calculation graph of the deep model. If the network architecture of the deep model is an irregular network architecture, the optimal topological sorting search algorithm for irregular network architecture is used for search.

[0027] The server starts with the computational graph and optimizes the graph topology to reduce the memory allocation of deep learning tasks while maintaining accuracy and reasoning efficiency as much as possible to meet the memory resource constraints of smart IoT terminal devices. For deep models with irregular network architectures, due to their own characteristics, the server attempts to add a subgraph replacement strategy to achieve the possibility of orthogonal optimization of computational graph optimization technology.

[0028] Unlike networks with regular topology, running deep models with irregular network architectures will produce different ranges of memory usage depending on the plan. The first step in applying dynamic programming to new problems is to accurately describe the structure of the optimal solution, that is, assume that s * is the global optimal solution, s * Can be decomposed into optimal sub-solutions on n nodes Next, we need to clearly identify the sub-problems The optimal solution and the previous level problem To achieve this goal, we adopt a topological sorting method that directly analyzes the recursive relationship. However, this method may involve an extensive scan of the entire search space when it is inefficient, and the recursive method needs to be improved.

[0029] In essence, the topological sorting algorithm is a cyclic process, the core of which is to continuously identify the set of nodes that are "ready", which is called the zero-in degree set. z represents the set of nodes for which all incoming edges and their corresponding predecessor nodes (i.e. nodes with zero in-degree) have been processed. In graph theory, these nodes can be safely scheduled in the next step. For the redundant zero-in-degree sets z that may be generated during the recursive topological sorting process, we optimize the calculation by merging these redundant sets and giving them unique identifiers, thereby avoiding unnecessary repeated calculations. Therefore, the design goal of the method for irregular network architecture is to find the scheduling of nodes from the search space S to minimize the peak memory usage.

[0030] Step 102, when the search step i starts, obtain the current state information, including the scheduling sequence S so far i , for the current signature Z i The memory usage of μ i , and the peak memory usage so far

[0031] In the specific implementation, there are n nodes in the computational graph G of the deep model with an irregular network architecture. When searching using the optimal topological sorting search algorithm for irregular network architecture, it is necessary to iterate n-1 steps to enumerate all possible orderings of the nodes.

[0032] First, initialize the memory table M0, which is specifically expressed by the formula:

[0033] Among them, S0 is empty, μ0=0, z0=zero-indegree(s0,G).

[0034] At the beginning of search step i, obtain the current state information (that is, memory table M i ), including the scheduling sequence S so far i , for the current signature Z i The memory usage of μ i , and the peak memory usage so far

[0035] Step 103, for each unique zero-indegree set z i , select an unscheduled node u i Scheduling is performed, where the zero-indegree set z i Represents the set of nodes that can be scheduled currently.

[0036] In the specific implementation, for every unique zero-indegree set z i , the server needs to select an unscheduled node u i Among them, the zero-indegree set z i Represents the set of nodes that can be scheduled currently.

[0037] Step 104, when u i After being dispatched, u i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i In order to form μ i+1 .

[0038] In the specific implementation, when u i After being dispatched, u i The output is activated and added to S i S i+1 , S i+1 It can be expressed by the formula: S i+1 =S i ·append(u i ), append(·) is a list adding function, which is used to add an element to the end of the list. i The output is activated and assigned to μ i In order to form μ i+1 , μ i+1 =μ i +∏(u i·shape), shape means calling u i Memory usage.

[0039] In addition, the server forms S i+1 After that, it is also necessary to base on S i+1 Right i Update to form z i+1 , z i+1 The formation of can be expressed by the formula: i+1 =zero-indegree(S i+1 ,G).

[0040] Step 105, based on μ i+1 and Update the peak memory usage to get

[0041] In the specific implementation, the server obtains μ i+1 After that, we can compare μ i+1 and The size of μ i+1 Greater than Then μ i+1 As If μ i+1 Less than or equal to Then As Will Update to μ i+1 and The larger value between and ensures that you are tracking the peak memory usage so far.

[0042] Step 106, check u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 .

[0043] In the specific implementation, the server obtains After that, the out-degree needs to be updated and the memory released. i , some located in u i The out-degree of the previous node may decrease because u i Now it has become one of their successors. When the out-degree of a node is reduced to 0, this node is no longer depended on by subsequent nodes and can be safely released from memory. Therefore, the server needs to check u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 , to reflect the reduced memory footprint.

[0044] Release the node whose out-degree becomes 0 from the memory and update μi+1 , achieved through the following formula:

[0045] μ i+1 =u i+1 -∏(p i shape);

[0046] Among them, p i Indicates the node whose degree becomes 0, p i ·Shape means calling p i The memory used.

[0047] Step 107, at the end of the search step i, save S i+1 , μ i+1 and Go to the next search step until the optimal solution that satisfies all constraints is found, and the optimal execution order is obtained and executed.

[0048] In the specific implementation, at the end of search step i, save S i+1 , μ i+1 , and z i+1 , get the memory table M i+1 , and then enter the next search step until the optimal solution that satisfies all constraints is found and the optimal execution order is obtained. That is, after completing all n-1 steps of iteration, the memory table M containing the optimal execution order is obtained. n , and finally perform the optimal execution order to achieve peak memory optimization.

[0049] In an example, the pseudo code of the optimal topological sorting search algorithm for irregular network architecture is as follows Figure 2 shown.

[0050] The following demonstrates the effectiveness of the optimal topological sorting search algorithm for irregular network architecture. i , in the corresponding scheduling set S i Only one memory pair with peak value (S i ,z i ) is sufficient. Then, by i By adding subsequent nodes on top of S, we can build S in each search step. i+1 .

[0051] First, consider the base case. When i=0, the optimal S0 is an empty sequence, so its peak memory consumption μ0 must be 0. Next, we consider the case of i≥1. Assume that there is a (suboptimal) scheduling sequence v i constitutes * part of , and there exists a u i ∈zi Make the scheduling reach the optimal peak memory μ * In this case, if we use the optimal u i Replace v i , then the new peak memory μ peak will be min[μ i +∏(v i ·shape),μ i +∏(u i ·shape)]. And μ i+1 is through μ i Subtract all predecessor operators pi∈{u i ·pends∩zero-out degree(s i+1 ,G)} is calculated based on the memory occupied.

[0052] Then, by recursively applying the above logic to the remaining search steps, we can ensure that due to the existence of the min operator, the algorithm will find a peak memory consumption μ * ′≤μ * However, this contradicts our original assumption that S is optimal, and thus the peak memory consumption of S is μ * is already the smallest. Therefore, we conclude that this algorithm can obtain the operator schedule with the best peak memory consumption.

[0053] In one example, if the network architecture of the deep model is a multi-branch network architecture, the server uses the optimal topological sorting search algorithm for the multi-branch network architecture to search. The algorithm recursively enumerates the execution plan and tries to cancel each operator that generates tensors to calculate the minimum peak memory usage. First, based on the calculation graph of the deep model, the tensors at all branches are determined, and each tensor is divided into constants and variables, and all constants are excluded, because constants only increase the peak memory burden and do not affect the execution order; then traverse each branch, for any branch, enumerate all possible execution orders, and calculate the peak memory usage of each execution order respectively, and select the execution order with the smallest peak memory usage as the optimal execution order; after traversing all branches, the optimal execution order of the entire deep model can be obtained and executed.

[0054] When searching using the optimal topological sorting search algorithm for multi-branch network architectures, it is stipulated that the same operator will not be executed multiple times in the plan. The stipulation is implemented by checking whether the current tensor is a predecessor of other tensors to ensure the uniqueness of the operator in the scheduling plan. During the search process using the optimal topological sorting search algorithm for multi-branch network architectures, the calculated results are recorded and cached to avoid repeated calculation of the memory usage of the same tensor.

[0055] After selecting the execution order with the smallest peak memory usage as the optimal execution order, the server needs to determine whether the optimal execution order is consistent with the default execution order; if the optimal execution order is consistent with the default execution order, the default execution order is directly executed; if the optimal execution order is inconsistent with the default execution order, the operator reordering method is called to rearrange the operators according to the optimal execution order to execute the optimal execution order and optimize the peak memory usage.

[0056] In an example, the pseudo code of the optimal topological sorting search algorithm for a multi-branch network architecture is as follows: Figure 3 shown.

[0057] In this embodiment, for the deep model of irregular network architecture and the deep model of multi-branch network architecture, the best execution order search algorithm based on dynamic programming that meets their respective characteristics is designed to fully tap the huge potential of operator rearrangement technology in reducing the peak memory usage when the deep model is running. In the process of searching and optimizing, the calculation graph is used as an effective means of compiling layer optimization without changing the model architecture, which can effectively maintain the mathematical integrity of the neural network and keep the accuracy of the model as unaffected as possible while reducing the peak memory. This optimization method can be orthogonally combined with model lightweight technologies of other algorithm layers and memory layers, showing a more complete advantage in edge intelligence scenarios. The optimal topological sorting search algorithm for irregular network architecture enumerates all execution plans recursively, and the optimal topological sorting search algorithm for multi-branch network architecture enumerates all execution plans through a backtracking algorithm, and the time complexity is very low. In summary, this method quickly, highly and flexibly achieves the optimization of peak memory usage when the deep model is running.

[0058] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0059] Another embodiment of the present application proposes a deep model computation graph peak memory optimization system for resource-constrained terminals. The details of the deep model computation graph peak memory optimization system for resource-constrained terminals proposed in this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this example.

[0060] Figure 4It is a schematic diagram of a deep model computation graph peak memory optimization system for resource-constrained terminals proposed in this embodiment, including: a computation graph determination module 201, a search initialization module 202, a scheduling module 203, an update module 204, a release module 205, a solution module 206 and an optimization execution module 207.

[0061] The computation graph determination module 201 is used to obtain the computation graph of the deep model. If the network architecture of the deep model is an irregular network architecture, the search is performed using the optimal topological sorting search algorithm for the irregular network architecture.

[0062] The search initialization module 202 is used to obtain the current state information, including the scheduling sequence S so far, at the beginning of the search step i. i , for the current signature Z i The memory usage of μ i , and the peak memory usage so far

[0063] The scheduling module 203 is used to schedule each unique zero-indegree set z i , select an unscheduled node u i Scheduling is performed, where the zero-indegree set z i Represents the set of nodes that can be scheduled currently.

[0064] Update module 204, used when u i After being dispatched, u i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i To form v i+1 , based on v i+1 and Update the peak memory usage to get

[0065] Release module 205, used to check u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 .

[0066] The solution module 206 is used to save S at the end of the search step i. i+1 , μ i+1 and Go to the next search step until the optimal solution that satisfies all constraints is found.

[0067] The optimized execution module 207 is used to obtain the optimal execution sequence based on the optimal solution and execute it.

[0068] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.

[0069] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0070] Another embodiment of the present application provides a server, such as Figure 5 As shown, it includes: at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can execute the deep model computation graph peak memory optimization method for resource-constrained terminals described in the above-mentioned method embodiments.

[0071] Among them, the memory and the processor are connected in a bus manner, and the bus includes any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and will not be further described in this article. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0072] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0073] Another embodiment of the present application proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the deep model calculation graph peak memory optimization method for resource-constrained terminals described in the above method embodiments.

[0074] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0075] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for optimizing peak memory of deep model computation graphs for resource-constrained terminals, characterized in that: include: Obtain the computational graph of the deep model. If the network architecture of the deep model is an irregular network architecture, use the optimal topological sorting search algorithm for irregular network architecture to search; At the beginning of search step i, obtain the current state information, including the scheduling sequence S so far i , for the current signature Z i The memory usage of μ i , and the peak memory usage so far For every unique zero-indegree set z i , select an unscheduled node u i Scheduling; Among them, the zero-indegree set z i Represents the set of nodes that can be scheduled currently; When u i After being dispatched, u i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i In order to form μ i+1 ; Based on μ i+1 and Update the peak memory usage to get Check u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 ; At the end of search step i, save S i+1 , μ i+1 and Go to the next search step until the optimal solution that satisfies all constraints is found, and the optimal execution order is obtained and executed.

2. The method for optimizing the peak memory of a deep model computation graph for resource-constrained terminals according to claim 1, characterized in that: There are n nodes in the computational graph G of the deep model. When searching using the optimal topological sorting search algorithm for irregular network architecture, it takes n-1 steps to iterate. First, the memory table Mo is initialized. S0 is empty, μ0=0, z0=zero-indegree(s0,G); For search step i, in forming S i+1 After that, based on S i+1 Right i Update to form Z i+1 ; At the end of search step i, save S i+1 , μ i+1 , and z i+1 , get the memory table M i+1 ; After completing all n-1 iterations, the optimal execution order memory table M is obtained. n .

3. The method for optimizing the peak memory of a deep model computation graph for resource-constrained terminals according to claim 2, characterized in that: u i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i In order to form μ i+1 , achieved through the following formula: S i+1 =S i ·append(u i ); m i+1 =μ i +P(u i ·shape); Among them, append(·) is a list addition function, which is used to add an element to the end of the list, u i ·Shape means calling u i Memory usage; Based on S i+1 Right i Update to form z i+1 , achieved through the following formula: z i+1 =zero-indegree(S i+1 ,G)。 4. The method for optimizing peak memory of deep model computation graph for resource-constrained terminals according to claim 3, characterized in that: Based on μ i+1 and Update the peak memory usage to get include: Compare μ i+1 and size; If μ i+1 Greater than Then μ i+1 As If μ i+1 Less than or equal to Then As 5. The method for optimizing the peak memory of a deep model computation graph for resource-constrained terminals according to claim 1, characterized in that: If the network architecture of the deep model is a multi-branch network architecture, the optimal topological sorting search algorithm for the multi-branch network architecture is used for search; First, based on the computational graph of the deep model, determine the tensors at all branches, divide each tensor into constants and variables, and exclude all constants; Traverse each branch, for any branch, enumerate all possible execution orders, calculate the peak memory usage of each execution order, and select the execution order with the smallest peak memory usage as the optimal execution order; After traversing all branches, the optimal execution order of the entire deep model is obtained and executed.

6. The method for optimizing peak memory of deep model computation graph for resource-constrained terminals according to claim 5, characterized in that: When searching using the optimal topological sorting search algorithm for a multi-branch network architecture, it is stipulated that the same operator will not be executed multiple times in the plan, and the provision is implemented by checking whether the current tensor is a predecessor of other tensors to ensure the uniqueness of the operator in the scheduling plan; During the search process using the optimal topological sorting search algorithm for multi-branch network architectures, the calculated results are recorded and cached to avoid repeated calculations of the memory usage of the same tensor.

7. The method for optimizing peak memory of deep model computation graph for resource-constrained terminals according to claim 6, characterized in that: After selecting the execution order with the smallest peak memory usage as the optimal execution order, it also includes: Determine whether the optimal execution order is consistent with the default execution order; If the optimal execution order is consistent with the default execution order, the default execution order will be executed directly; If the optimal execution order is inconsistent with the default execution order, the operator reordering method is called to rearrange the operators according to the optimal execution order to execute the optimal execution order.

8. A deep model computation graph peak memory optimization system for resource-constrained terminals, characterized in that: include: A computational graph determination module is used to obtain the computational graph of the deep model. If the network architecture of the deep model is an irregular network architecture, the optimal topological sorting search algorithm for the irregular network architecture is used for searching; The search initialization module is used to obtain the current state information, including the scheduling sequence S so far, at the beginning of the search step i. i , for the current signature Z i The memory usage of μ i , and the peak memory usage so far The scheduling module is used for each unique zero-indegree set z i , select an unscheduled node u i Scheduling is performed, where the zero-indegree set z i Represents the set of nodes that can be scheduled currently; Update module, used when u i After being dispatched, u i The output is activated and added to S i S i+1 , and simultaneously assigned to μ i In order to form μ i+1 , based on μ i+1 and Update the peak memory usage to get Release module for checking u i The out-degree of each node before, release the node whose out-degree becomes 0 from the memory, and update μ i+1 ; The solver module is used to save S at the end of the search step i. i+1 , μ i+1 and Go to the next search step until the optimal solution that satisfies all constraints is found; The optimized execution module is used to obtain the optimal execution order based on the optimal solution and execute it.

9. A server, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep model computation graph peak memory optimization method for resource-constrained terminals as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described in any one of claims 1 to 7 is implemented.

Images