Peak memory optimization method for deep model computation graph for resource-constrained terminals

Abstract

The embodiments of this application relate to the field of peak memory optimization technology, and particularly to a method for optimizing peak memory in a deep model computation graph for resource-constrained terminals, comprising: obtaining the computation graph of the deep model; if the network architecture of the deep model is an irregular network architecture, performing a search using an optimal topology sorting search algorithm for irregular network architectures; at the start of search step i, obtaining current state information, including the scheduling sequence S so far. i Memory usage μ i And, based on the peak memory usage to date, select an unscheduled node u. i Perform scheduling; when u i After being scheduled, u i The output is activated to form S i+1 and μ i+1 Update peak memory usage to get u i Nodes whose out-degree previously became 0 are released and μ is updated. i+1 The process continues until the optimal solution satisfying all constraints is found, resulting in the optimal execution order, which is then executed. This method optimizes peak memory usage during the runtime of deep learning models.

Description

Technical Field

[0001] The embodiments of this application relate to the field of peak memory optimization technology, and in particular to a method for peak memory optimization of deep model computation graphs for resource-constrained terminals. Background Technology

[0002] With the rapid development of AIoT (Artificial Intelligence of Things) technology, deep learning models are increasingly becoming "edge-oriented." However, the success of deep learning models often comes at the cost of extremely high computational demands. In edge computing scenarios, the memory capacity of terminal devices is limited, often insufficient to meet the computational requirements of deep learning models. To overcome this challenge, some model compression techniques have emerged. Methods such as pruning, quantization, and knowledge distillation can reduce the model size and peak memory usage by decreasing the number of parameters, reducing model complexity, or extracting key information from the model, thus adapting to the computing and storage capabilities of terminal devices. These techniques have achieved significant results in the past few years and have been widely applied in practical applications.

[0003] However, with the continuous development of model compression technology, its optimization space is gradually shrinking, necessitating the search for deeper optimization methods. In recent years, computation graph optimization has become a popular research direction. By modifying the computation graph, the computational load and peak memory usage of the model can be effectively reduced. Techniques such as operator fusion and operator rearrangement have been proposed to address the problem of limited computing and storage resources on terminal devices. However, automatically determining the optimal operator rearrangement scheme is an NP-hard problem, requiring efficient search algorithms and heuristic methods. Summary of the Invention

[0004] The main objective of the embodiments of this application is to propose a peak memory optimization method for deep model computation graphs for resource-constrained terminals. It designs a dynamic programming-based optimal execution order search algorithm that conforms to the characteristics of network architecture, fully explores the huge potential of operator rearrangement technology in reducing the peak memory usage during deep model runtime, and achieves the optimization of peak memory usage during deep model runtime.

[0005] To achieve the above objectives, embodiments of this application propose a method for optimizing peak memory in the computation graph of a deep model for resource-constrained terminals, comprising the following steps: obtaining the computation graph of the deep model; if the network architecture of the deep model is an irregular network architecture, then performing a search using an optimal topology sorting search algorithm for irregular network architectures; at the start of search step i, obtaining the current state information, including the scheduling sequence S up to date. i For the current signature Z i memory usage μ iand peak memory usage to date. For every unique set of zero-in-degree z i Select an unscheduled node u i Scheduling is performed; where the zero-in-degree set z i This represents the set of nodes that can currently be scheduled; when u i After being scheduled, u i The output is activated and added to S i China and Israel form S i+1 Simultaneously allocated to μ i μ is formed in the middle i+1 Based on μ i+1 and Update peak memory usage to obtain Check u i The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and μ is updated. i+1 At the end of search step i, save S. i+1 μ i+1 and Proceed to the next search step until the optimal solution that satisfies all constraints is found, obtain the optimal execution order, and execute.

[0006] To achieve the above objectives, embodiments of this application also propose a peak memory optimization system for deep model computation graphs on resource-constrained terminals. This system includes: a computation graph determination module, used to obtain the computation graph of the deep model; if the network architecture of the deep model is an irregular network architecture, then a search algorithm for optimal topology sorting for irregular network architectures is used for searching; and a search initialization module, used to obtain current state information, including the scheduling sequence S up to now, at the start of search step i. i For the current signature Z i memory usage μ i and peak memory usage to date. The scheduling module is used for each unique zero-in-degree set z i Select an unscheduled node u i Scheduling is performed; where the zero-in-degree set z i Represents the set of nodes that can currently be scheduled; the update module is used to update the set of nodes when u... i After being scheduled, u i The output is activated and added to S i China and Israel form S i+1 Simultaneously allocated to μ i μ is formed in the middle i+1 Based on μ i+1 and Update peak memory usage to obtain Release module, used to check u i The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and μ is updated. i+1 The solver module is used to save S at the end of search step i. i+1 μ i+1 and Proceed to the next search step until the optimal solution that satisfies all constraints is found; the optimization execution module is used to obtain the optimal execution order based on the optimal solution and execute it.

[0007] To achieve the above objectives, embodiments of this application also propose a server, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described above.

[0008] To achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described above.

[0009] The embodiments of this application propose a method for optimizing peak memory usage of deep learning computation graphs for resource-constrained terminals. For deep learning models with irregular network architectures and those with multi-branch network architectures, dynamic programming-based optimal execution order search algorithms are designed to fully leverage the significant potential of operator rearrangement techniques in reducing peak memory consumption during deep learning model runtime. During the search and optimization process, the computation graph is used as an effective means of compiler-level optimization without altering the model architecture, effectively maintaining the mathematical integrity of the neural network and preserving model accuracy as much as possible while minimizing peak memory usage. This optimization method can be orthogonally combined with other algorithm-level and memory-level lightweighting techniques, demonstrating more comprehensive advantages in edge intelligence scenarios. The optimal topology sorting search algorithm for irregular network architectures enumerates all execution plans recursively, while the optimal topology sorting search algorithm for multi-branch network architectures enumerates all execution plans through backtracking, resulting in very low time complexity. In summary, this method achieves fast, high-quality, and flexible optimization of peak memory usage during deep learning model runtime.

[0010] Furthermore, the computational graph G of the deep model has n nodes. When using the optimal topology sorting search algorithm for irregular network architectures, n-1 iterations are required. First, the memory table M0 is initialized. S0 is empty, μ0 = 0. z0 = zero-indegree(s0,G); For search step i, in forming S i+1 Afterwards, based on S i+1 For z i Update to form z i+1 At the end of search step i, save S. i+1 μ i+1 , and z i+1 , obtain memory table M i+1 After completing all n-1 iterations, a memory table M containing the optimal execution order is obtained. n .

[0011] In addition, u i The output is activated and added to S i China and Israel form S i+1 Simultaneously allocated to μ i μ is formed in the middle i+1 This is achieved through the following formula: S i+1 =S i ·append(u i );μ i+1 =μ i +∏(u i ·shape); where append(·) is the append function, used to add an element to the end of the list, u i • shape indicates calling u i Memory usage; based on S i+1 For z i Update to form z i+1 This is achieved through the following formula: z i+1 =zero-indegree(S) i+1 ,G).

[0012] In addition, based on μ i+1 and Update peak memory usage to obtain Includes: comparing μ i+1 and The size; if μ i+1 Greater than Then μ i+1 As If μ i+1 Less than or equal to Then As

[0013] In addition, if the deep model's network architecture is a multi-branch network architecture, the optimal topology sorting search algorithm for multi-branch network architectures is used for the search. First, based on the computation graph of the deep model, the tensors at all branches are determined, and each tensor is divided into constants and variables, with all constants excluded. Each branch is traversed, and for any branch, all possible execution orders are enumerated, and the peak memory usage of each execution order is calculated. The execution order with the smallest peak memory usage is selected as the optimal execution order. After traversing all branches, the optimal execution order of the entire deep model is obtained and executed.

[0014] Furthermore, when using the optimal topology sorting search algorithm for multi-branch network architecture, it is stipulated that the same operator will not be executed multiple times in the plan. This stipulation is implemented by checking whether the current tensor is the predecessor of other tensors to ensure the uniqueness of the operator in the scheduling plan. During the search process using the optimal topology sorting search algorithm for multi-branch network architecture, the calculated results are recorded and cached to avoid repeated calculations of the memory usage of the same tensor.

[0015] In addition, after selecting the execution order with the lowest peak memory usage as the optimal execution order, the process also includes: determining whether the optimal execution order is consistent with the default execution order; if the optimal execution order is consistent with the default execution order, then the default execution order is executed directly; if the optimal execution order is inconsistent with the default execution order, then the operator rearrangement method is called to rearrange the operators according to the optimal execution order in order to execute the optimal execution order. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a method for optimizing peak memory in a deep model computation graph for resource-constrained terminals, provided in one embodiment of this application;

[0018] Figure 2 This is a pseudocode diagram of an optimal topology sorting search algorithm for irregular network architectures provided in one embodiment of this application;

[0019] Figure 3 This is a pseudocode diagram of an optimal topology sorting search algorithm for multi-branch network architecture provided in one embodiment of this application;

[0020] Figure 4This is a schematic diagram of the structure of a peak memory optimization system for deep model computation graphs for resource-constrained terminals provided in another embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the server structure provided in another embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0023] To address the NP-hard problem of automatically determining the optimal operator rearrangement scheme, one embodiment of this application proposes a peak memory optimization method for deep model computation graphs on resource-constrained terminals, applied to servers. The implementation details of this embodiment's peak memory optimization method for deep model computation graphs on resource-constrained terminals are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution.

[0024] The specific process of the peak memory optimization method for deep model computation graphs for resource-constrained terminals proposed in this embodiment can be described as follows: Figure 1 As shown, it includes:

[0025] Step 101: Obtain the computation graph of the deep model. If the network architecture of the deep model is an irregular network architecture, then use the optimal topology sorting search algorithm for irregular network architecture to perform the search.

[0026] In the specific implementation, the server obtains the deep model that needs to be optimized and its computation graph. The network architecture of the deep model is determined by the computation graph. If the network architecture of the deep model is an irregular network architecture, the optimal topology sorting search algorithm for irregular network architecture is used for searching.

[0027] Starting with the computation graph, the server optimizes the graph topology to reduce memory allocation for deep learning tasks while maintaining accuracy and inference efficiency as much as possible, thus meeting the memory resource constraints of smart IoT terminal devices. For deep models with irregular network architectures, due to their inherent characteristics, the server attempts to incorporate a subgraph substitution strategy to realize the possibility of orthogonal optimization using computation graph optimization techniques.

[0028] Unlike networks with regular topologies, deep models running irregular network architectures will incur varying memory footprints depending on the plan. The first step in applying dynamic programming to new problems is to accurately describe the structure of the optimal solution, i.e., assuming s * It is the globally optimal solution, s * It can be decomposed into optimal sub-solutions on n nodes. The combination of these. Next, we need to clearly identify the sub-problems. The optimal solution and the previous level problem To address this, we employ a topological sorting method that directly analyzes the recursive relationships. However, this method can be inefficient and involves extensive scanning of the entire search space, necessitating improvements to the recursive approach.

[0029] Essentially, topology sorting algorithms are a cyclical process. Their core lies in continuously identifying the set of nodes that are "ready" to proceed, called the zero-in-degree set. Here, z represents the set of nodes whose incoming edges and their corresponding predecessors (nodes with an in-degree of zero) have been processed. In graph theory, these nodes are those that can be safely scheduled in the next step. To address the potentially redundant zero-in-degree sets z generated during recursive topology sorting, we optimize computation by merging these redundant sets and assigning them unique identifiers, thus avoiding unnecessary redundant computation. Therefore, the design goal of methods for irregular network architectures is to find the scheduling of nodes from the search space S to minimize peak memory usage.

[0030] Step 102: At the start of the search step i, obtain the current state information, including the scheduling sequence S so far. i For the current signature Z i memory usage μ i and peak memory usage to date.

[0031] In the specific implementation, the computation graph G of the deep model with an irregular network architecture has a total of n nodes. When using the optimal topology sorting search algorithm for irregular network architecture, it is necessary to iterate n-1 steps to enumerate all possible sortings of the nodes.

[0032] First, initialize the memory table M0, specifically expressed by the formula:

[0033] Where S0 is empty, μ0 = 0, z0 = zero-indegree(s0,G).

[0034] At the start of search step i, obtain the current state information (i.e., memory table M). i ), including the scheduling sequence S to date. i For the current signature Z i memory usage μ i and peak memory usage to date.

[0035] Step 103, for each unique set z with zero in-degree i Select an unscheduled node u i Scheduling is performed, where the zero-in-degree set z i This represents the set of nodes that can currently be scheduled.

[0036] In the specific implementation, for each unique zero-in-degree set z i The server needs to select an unscheduled node u i Perform scheduling. Where the zero-in-degree set z... i This represents the set of nodes that can currently be scheduled.

[0037] Step 104, when u i After being scheduled, u i The output is activated and added to S i China and Israel form S i+1 Simultaneously allocated to μ i μ is formed in the middle i+1 .

[0038] In the specific implementation, when u i After being scheduled, u i The output is activated and added to S i China and Israel form S i+1 S i+1 This can be expressed by the formula: S i+1 =S i ·append(u i `),` and `append(·)` are functions for adding elements to a list, used to add an element to the end of the list. Also, `u` i The output is activated and assigned to μ. i μ is formed in the middle i+1 μ i+1 =μ i +∏(u i·shape), where shape represents the call to u i Memory usage.

[0039] In addition, the server forms S i+1 After that, it is also necessary to base it on S i+1 For z i Update to form z i+1 , z i+1 The formation of z can be expressed by the formula: i+1 =zero-indegree(S) i+1 ,G).

[0040] Step 105, based on μ i+1 and Update peak memory usage to obtain

[0041] In the specific implementation, the server obtains μ i+1 Then, μ can be compared. i+1 and The size of μ i+1 Greater than Then μ i+1 As If μ i+1 Less than or equal to Then As Will Updated to μ i+1 and A larger value between these values ​​ensures that peak memory usage to date has been tracked.

[0042] Step 106, check u i The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and μ is updated. i+1 .

[0043] In the specific implementation, the server obtains Afterwards, the out-degree needs to be updated and memory released, because u was scheduled. i Some located in u i The out-degree of previous nodes may decrease because u i Now it becomes a successor to them. When a node's out-degree decreases to 0, this node is no longer depended on by subsequent nodes and can therefore be safely released from memory. Therefore, the server needs to check u i The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and μ is updated. i+1 This is to reflect the reduced memory usage.

[0044] Release nodes whose out-degree becomes 0 from memory and update μ.i+1 This can be achieved through the following formula:

[0045] μ i+1 =μ i+1 -∏(p i ·shape);

[0046] Where, p i p represents the node whose out-degree becomes 0. i • shape indicates calling p i The amount of memory occupied.

[0047] Step 107: At the end of search step i, save S. i+1 μ i+1 and Proceed to the next search step until the optimal solution that satisfies all constraints is found, obtain the optimal execution order, and execute.

[0048] In the specific implementation, S is saved at the end of search step i. i+1 μ i+1 , and z i+1 , obtain memory table M i+1 Then, the search continues to the next step until the optimal solution that satisfies all constraints is found, thus obtaining the optimal execution order. In other words, after completing all n-1 iterations, a memory table M containing the optimal execution order is obtained. n Finally, the optimal execution order is executed to achieve peak memory optimization.

[0049] In one example, the pseudocode for the optimal topology sorting search algorithm for irregular network architectures is as follows: Figure 2 As shown.

[0050] The effectiveness of the optimal topology sorting search algorithm for irregular network architectures is demonstrated below. For each zero-in-degree set z i In the corresponding scheduling set S i In the middle, only one peak memory pair (S) is retained. i ,z i The scheduling of ) is sufficient. Subsequently, through S i By adding subsequent nodes to the top, we can build S in each search step. i+1 .

[0051] First, consider the basic case: when i = 0, the optimal S0 is an empty sequence, therefore its peak memory consumption μ0 must be 0. Next, consider the case where i ≥ 1, assuming there exists a (suboptimal) scheduling sequence v. i It constitutes s * A part of, and there exists a u i ∈zi This allows scheduling to reach optimal peak memory μ. * In this case, if the optimal u is used... i Replace v i So, the new peak memory μ peak It will be min[μ] i +∏(v i ·shape),μ i +∏(u i ·shape). And μ i+1 This is achieved by using μ i Subtract all predecessor operators pi∈{u i ·pends∩zero-outdegree(s i+1 The calculation is based on the memory occupied by ,G)}.

[0052] Then, by recursively applying the above logic to the remaining search steps, it can be ensured that, due to the presence of the min operator, the algorithm will find a peak memory consumption μ. * ′≤μ * The alternative sequence S′. However, this contradicts our original assumption that S has optimality, thus leading to the conclusion that the peak memory consumption μ of S is... * This is already the minimum. Therefore, we conclude that this algorithm can achieve operator scheduling with optimal peak memory consumption.

[0053] In one example, if the deep learning model's network architecture is a multi-branch network architecture, the server uses an optimal topology sorting search algorithm for multi-branch network architectures. This algorithm recursively enumerates the execution plan and attempts to cancel each operator that generates tensors to calculate the minimum peak memory usage. First, based on the deep learning model's computation graph, the tensors at all branches are determined. Each tensor is divided into constants and variables, and all constants are excluded because constants only increase peak memory burden and do not affect the execution order. Then, each branch is traversed. For any branch, all possible execution orders are enumerated, and the peak memory usage of each execution order is calculated. The execution order with the minimum peak memory usage is selected as the optimal execution order. After traversing all branches, the optimal execution order for the entire deep learning model is obtained and executed.

[0054] When using the optimal topology sorting search algorithm for multi-branch network architectures, it is stipulated that the same operator will not be executed multiple times in the schedule. This stipulation is implemented by checking whether the current tensor is a predecessor of other tensors, thus ensuring the uniqueness of the operator in the scheduling plan. During the search process using the optimal topology sorting search algorithm for multi-branch network architectures, the calculated results are recorded and cached to avoid duplicate calculations of the memory usage of the same tensor.

[0055] After selecting the execution order with the lowest peak memory usage as the optimal execution order, the server needs to determine whether the optimal execution order is consistent with the default execution order. If the optimal execution order is consistent with the default execution order, the default execution order is executed directly. If the optimal execution order is inconsistent with the default execution order, the operator rearrangement method is called to rearrange the operators according to the optimal execution order in order to execute the optimal execution order and achieve the optimization of peak memory usage.

[0056] In one example, the pseudocode for the optimal topology ranking search algorithm for multi-branch network architectures is as follows: Figure 3 As shown.

[0057] In this embodiment, optimal execution order search algorithms based on dynamic programming are designed for deep models with irregular network architectures and deep models with multi-branch network architectures, respectively, to fully explore the huge potential of operator rearrangement technology in reducing peak memory usage during deep model runtime. During the search and optimization process, the computation graph is used as an effective means of compiler layer optimization without altering the model architecture, effectively maintaining the mathematical integrity of the neural network and minimizing the impact on model accuracy while keeping peak memory usage low. This optimization method can be orthogonally combined with other algorithm layer and memory layer model lightweighting techniques, demonstrating more comprehensive advantages in edge intelligence scenarios. The optimal topology sorting search algorithm for irregular network architectures enumerates all execution plans recursively, while the optimal topology sorting search algorithm for multi-branch network architectures enumerates all execution plans through backtracking, resulting in very low time complexity. In summary, this method achieves fast, high-quality, and flexible optimization of peak memory usage during deep model runtime.

[0058] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0059] Another embodiment of this application proposes a peak memory optimization system for deep model computation graphs for resource-constrained terminals. The details of the peak memory optimization system for deep model computation graphs for resource-constrained terminals proposed in this embodiment are described below. The following content is only for the convenience of understanding and is not necessary for implementing this example.

[0060] Figure 4This is a schematic diagram of a peak memory optimization system for deep model computation graphs for resource-constrained terminals proposed in this embodiment, including: a computation graph determination module 201, a search initialization module 202, a scheduling module 203, an update module 204, a release module 205, a solution module 206, and an optimization execution module 207.

[0061] The computation graph determination module 201 is used to obtain the computation graph of the deep model. If the network architecture of the deep model is an irregular network architecture, the optimal topology sorting search algorithm for irregular network architecture is used for searching.

[0062] The search initialization module 202 is used to obtain the current state information, including the scheduling sequence S so far, at the start of search step i. i For the current signature Z i memory usage μ i and peak memory usage to date.

[0063] Scheduling module 203 is used for each unique zero-in-degree set z i Select an unscheduled node u i Scheduling is performed, where the zero-in-degree set z i This represents the set of nodes that can currently be scheduled.

[0064] Update module 204, used when u i After being scheduled, u i The output is activated and added to S i China and Israel form S i+1 Simultaneously allocated to μ i μ is formed in the middle i+1 Based on μ i+1 and Update peak memory usage to obtain

[0065] Release module 205, used to check u i The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and μ is updated. i+1 .

[0066] Solver module 206 is used to save S at the end of search step i. i+1 μ i+1 and Proceed to the next search step until the optimal solution that satisfies all constraints is found.

[0067] The optimization execution module 207 is used to obtain the optimal execution order based on the optimal solution and execute it.

[0068] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0069] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0070] Another embodiment of this application proposes a server, such as Figure 5 As shown, it includes: at least one processor 301; and a memory 302 communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions executable by the at least one processor 301, the instructions being executed by the at least one processor 301 to enable the at least one processor 301 to execute the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described in the above method embodiments.

[0071] The memory and processor are connected via a bus, which includes any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0072] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0073] Another embodiment of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the peak memory optimization method for deep model computation graphs for resource-constrained terminals described in the above method embodiments.

[0074] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, ROM (Read-Only Memory), RAM (Random Access Memory), a magnetic disk, or an optical disk.

[0075] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for optimizing peak memory usage in deep model computation graphs for resource-constrained terminals, characterized in that, include: Obtain the computational graph of the deep model. If the network architecture of the deep model is an irregular network architecture, then use the optimal topology sorting search algorithm for irregular network architecture to perform the search. In the search steps At the start, obtain the current state information, including the scheduling sequence so far. For the current signature memory usage and peak memory usage to date. ; For each unique set of zero-in-degree Select an unscheduled node Scheduling is performed; among which, the zero-in-degree set This represents the set of nodes that can currently be scheduled. when After being dispatched, The output is activated and added to China and Israel form At the same time, allocated to China and Israel form ; based on and Update peak memory usage to obtain ; examine The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and the process is updated. ; In the search steps Save when finished , and Proceed to the next search step until the optimal solution that satisfies all constraints is found, obtain the optimal execution order, and execute. Computational graph of deep models The CCP When searching for nodes using the optimal topology sorting search algorithm for irregular network architectures, iterative steps are required. The first step is to initialize the memory table. , , Empty , , ; For search steps In formation Afterwards, based on right Update and form ; In the search steps Save when finished , , and , obtain memory table ; After completing all After step iterations, a memo table containing the optimal execution order is obtained. .

2. The method for optimizing peak memory usage in deep model computation graphs for resource-constrained terminals according to claim 1, characterized in that, The output is activated and added to China and Israel form At the same time, allocated to China and Israel form This can be achieved through the following formula: ； ； in, The `addList` function is used to add an element to the end of a list. Indicates a call Memory usage; based on right Update and form This can be achieved through the following formula: 。 3. The method for optimizing peak memory usage in deep model computation graphs for resource-constrained terminals according to claim 2, characterized in that, based on and Update peak memory usage to obtain ,include: Compare and Size; like Greater than Then As ; like Less than or equal to Then As .

4. The method for optimizing peak memory usage in deep model computation graphs for resource-constrained terminals according to claim 1, characterized in that, If the network architecture of the deep model is a multi-branch network architecture, then the optimal topology sorting search algorithm for multi-branch network architecture is used for the search. First, based on the computational graph of the deep model, the tensors at all branches are determined, each tensor is divided into constants and variables, and all constants are excluded. Iterate through each branch. For any branch, enumerate all possible execution orders and calculate the peak memory usage of each execution order. Select the execution order with the smallest peak memory usage as the optimal execution order. After traversing all branches, the optimal execution order of the entire depth model is obtained and executed.

5. The method for optimizing peak memory usage in deep model computation graphs for resource-constrained terminals according to claim 4, characterized in that, When using the optimal topology sorting search algorithm for multi-branch network architecture, it is stipulated that the same operator will not be executed multiple times in the plan. This stipulation is implemented by checking whether the current tensor is the predecessor of other tensors to ensure the uniqueness of the operator in the scheduling plan. During the search process using the optimal topology sorting search algorithm for multi-branch network architecture, the calculated results are recorded and cached to avoid repeated calculations of the memory usage of the same tensor.

6. The method for optimizing peak memory usage in deep model computation graphs for resource-constrained terminals according to claim 5, characterized in that, After selecting the execution order with the lowest peak memory usage as the optimal execution order, the following steps are also included: Determine whether the optimal execution order is consistent with the default execution order; If the optimal execution order is the same as the default execution order, then the default execution order will be executed directly. If the optimal execution order is inconsistent with the default execution order, the operator rearrangement method is called to rearrange the operators according to the optimal execution order in order to execute the optimal execution order.

7. A peak memory optimization system for deep model computation graphs for resource-constrained terminals, characterized in that, include: The computation graph determination module is used to obtain the computation graph of the deep model. If the network architecture of the deep model is an irregular network architecture, the optimal topology sorting search algorithm for irregular network architecture is used for searching. The search initialization module is used in the search steps. At the start, obtain the current state information, including the scheduling sequence so far. For the current signature memory usage and peak memory usage to date. ; The scheduling module is used for each unique zero-in-degree set. Select an unscheduled node Scheduling is performed, where the zero-in-degree set This represents the set of nodes that can currently be scheduled. Update module, used when After being dispatched, The output is activated and added to China and Israel form At the same time, allocated to China and Israel form ,based on and Update peak memory usage to obtain ; Release module, used for checking The out-degrees of the previous nodes are determined, nodes with an out-degree of 0 are released from memory, and the process is updated. ; The solver module is used in the search step. Save when finished , and Then proceed to the next search step until the optimal solution that satisfies all constraints is found; The optimization execution module is used to obtain the optimal execution order based on the optimal solution and then execute it. Computational graph of deep models The CCP When searching for nodes using the optimal topology sorting search algorithm for irregular network architectures, iterative steps are required. The first step is to initialize the memory table. , , Empty , , ; For search steps In formation Afterwards, based on right Update and form ; In the search steps Save when finished , , and , obtain memory table ; After completing all After step iterations, a memo table containing the optimal execution order is obtained. .

8. A server, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the peak memory optimization method for deep model computation graphs for resource-constrained terminals as described in any one of claims 1 to 6.

Images