A method for optimizing deployment of a deep learning model based on a storage-computing integrated system
By representing deep learning models as computational graphs, dividing them into constants and variables, and optimizing the execution order of variables, the problem of limited computing and storage resources on terminal devices is solved, achieving minimal memory usage and efficient deployment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2023-11-15
- Publication Date
- 2026-07-24
AI Technical Summary
Mobile and wearable devices have limited computing and storage resources, making it impossible to effectively deploy large-scale deep learning models, resulting in excessive memory consumption and affecting processing accuracy.
By representing the model as a computation graph, dividing it into constants and variables, optimizing the execution order of variables, using operator rearrangement techniques to determine the optimal execution order, and deploying it, memory usage is reduced.
Without affecting model accuracy, memory usage has been reduced, making it compatible with resource-constrained terminal devices and reducing memory overhead.
Smart Images
Figure CN117436546B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computation graph optimization design technology, and in particular to a method for optimizing the deployment of a deep learning model terminal based on in-memory computing. Background Technology
[0002] In recent years, the widespread use of mobile and wearable devices, along with the rapid development of deep learning technology, has brought ubiquitous opportunities for intelligent applications and services. In the context of the Internet of Things (IoT), deploying deep learning models (hereinafter referred to as models) on resource-constrained intelligent IoT terminal devices has become a trend.
[0003] However, as people pursue higher accuracy, the scale of models and sample data continues to increase, putting enormous pressure on computing power and memory. For example, to classify a 224px × 224px resolution color image, the 8-layer AlexNet model has 610 million network parameters, requires 729 million floating-point operations, and occupies 240MB of memory. The VGG-16 model, on the other hand, has 144 million network parameters, requires 15 billion floating-point operations, and occupies 528MB of memory. Even with the development of system-on-a-chip (SoC) and the emergence of AI chips, terminal devices have gradually acquired stronger computing and storage capabilities, but they still cannot match the powerful computing and storage capabilities of traditional CPU and GPU platforms, failing to meet the storage and operational needs of models.
[0004] To address this issue, research and industrial fields have conducted a series of explorations and studies on model optimization methods based on terminal devices. In these studies, researchers discovered that trained models can be compressed to adapt their size to the computing power and memory of terminal devices. There are many methods for model compression, such as pruning redundant parts, replacing the original large model with a smaller, more compact one, or reducing the storage types of parameters in the model. However, as compression technology matures, the optimization space for model compression is shrinking, making it increasingly difficult to overcome the limitations of computing and storage resources. Summary of the Invention
[0005] The purpose of this application is to provide a method for optimizing the deployment of a deep learning model on a terminal based on in-memory computing. This method can optimize the deployment of the model on the terminal according to the optimal execution order of the model and its operators, thereby minimizing memory usage and reducing memory overhead without affecting the model's processing accuracy.
[0006] To address the aforementioned technical problems, embodiments of this application provide a method for optimizing the deployment of a deep learning model terminal based on in-memory computing, comprising the following steps: parsing the trained model to obtain a computation graph corresponding to the model; determining all branches of the model in the computation graph, traversing each branch, and dividing all tensors at the current branch into constants and variables; wherein, the constants are tensors calculated by operators not belonging to the computation graph, and the variables are tensors calculated by operators belonging to the computation graph; calculating the peak memory usage corresponding to the operators generating each variable in each candidate execution order at the current branch, and determining the total memory usage corresponding to each candidate execution order at the current branch based on the peak memory usage, and determining the candidate execution order with the smallest total memory usage as the optimal execution order at the current branch; and deploying the model on a preset terminal for inference based on the optimal execution order corresponding to each branch.
[0007] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the above-described in-memory computing-based deep learning model terminal optimization deployment method.
[0008] Embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for optimizing the deployment of a deep learning model terminal based on in-memory computing.
[0009] The embodiments of this application provide a method for optimizing the deployment of deep learning models on terminals based on in-memory computing. The model file is represented as a computation graph. Operator rearrangement techniques are used at the computation graph level to address the limitations of terminal computing and storage resources. For each branch, tensors are first divided into constants and variables. Constants do not affect the execution order and cannot be optimized, while variables are the objects to be optimized. For multiple candidate execution orders at a branch, the total memory usage of each candidate execution order at the branch is calculated by generating the peak memory usage corresponding to the operators of each variable. The candidate execution order with the smallest total memory usage is determined as the optimal execution order. The model is then deployed according to the optimal execution order corresponding to each branch. The optimal execution order minimizes the memory usage during model runtime, effectively adapting to terminals with limited memory resources. It achieves minimal memory usage and reduces memory overhead without affecting the model's processing accuracy.
[0010] In some optional embodiments, the step of deploying the model on a preset terminal for inference based on the optimal execution order corresponding to each branch includes: for each branch, recording the position of each pruning operator in the optimal execution order corresponding to the branch; traversing each pruning operator and determining whether the predecessor operator of the current pruning operator is an element-wise operator; if the predecessor operator of the current pruning operator is an element-wise operator, swapping the positions of the current pruning operator and its predecessor operator, and updating the optimal execution order until the current pruning operator has no swappable objects; after all pruning operators have no swappable objects, obtaining the final updated optimal execution order corresponding to the branch; and deploying the model on a preset terminal for inference based on the final updated optimal execution order corresponding to each branch. In addition to performing topological rearrangement for computation graph branches, this application also performs rearrangement for pruning operators. The final updated optimal execution order obtained by the rearrangement not only significantly reduces memory usage but also reduces computational load, enabling the model to be well adapted to terminals with limited computing resources.
[0011] In some optional embodiments, the current clipping operator has no interchangeable objects, including: the predecessor operator of the current clipping operator is a clipping operator, or there are no other operators before the current clipping operator.
[0012] In some optional embodiments, calculating the peak memory usage corresponding to the operator generating each variable in each candidate execution order at the current branch includes: traversing each candidate execution order, traversing each variable according to the current candidate execution order, calculating the sum of the memory usage of the input tensor of the operator generating the current variable, other tensors that need to be cached before the input tensor, and the current variable, and using the sum of the memory usage as the peak memory usage of the operator generating the current variable. Adding the memory usage of an operator's input tensor to the memory usage of its output tensor, plus the memory usage of other tensors that need to be cached before that operator, as the peak memory usage of that operator, can effectively measure the memory usage required by each candidate execution order, thereby intuitively and clearly determining the optimal execution order.
[0013] In some optional embodiments, before dividing all tensors at the current branch into constants and variables, the method further includes: calculating the memory footprint of each tensor at the branch and caching the memory footprint of each tensor. Caching the memory footprint of each tensor in advance can avoid the additional overhead caused by repeated calculations.
[0014] In some optional embodiments, after dividing all tensors at the current branch into constants and variables, the method further includes: if all tensors at the current branch are constants, then the default execution order at the current branch is directly determined as the optimal execution order at the current branch. Considering that there is no optimization space at branches with all constants, directly determining the default execution order as the optimal execution order can improve optimization efficiency and save reordering resources.
[0015] In some optional embodiments, the step of parsing the trained model to obtain the computation graph corresponding to the model includes: converting the model trained on a preset training platform into a model file of a preset format using the API interface provided by the training platform, and parsing the model file to obtain the computation graph corresponding to the model.
[0016] In some optional embodiments, the preset training platform is the TensorFlow training platform, and the preset format is the TFLite format. Attached Figure Description
[0017] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.
[0018] Figure 1 This is a flowchart of an embodiment of the in-memory computing-based deep learning model terminal optimization deployment method provided in this application;
[0019] Figure 2 This is a schematic diagram of topological rearrangement for computation graph branches provided in one embodiment of this application;
[0020] Figure 3 This is a schematic diagram of a branch and tensor of the model provided in one embodiment of this application;
[0021] Figure 4 This is a flowchart of an embodiment of the present application, which describes how the model is deployed on a preset terminal for inference based on the optimal execution order corresponding to each branch.
[0022] Figure 5 This is a schematic diagram of rearrangement for clipping operators provided in one embodiment of this application;
[0023] Figure 6 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.
[0025] One embodiment of this application relates to a method for optimizing the deployment of a deep learning model on a terminal based on in-memory computing, applied to an electronic device. The electronic device can be a terminal or a server; this embodiment and the following embodiments will use a server as an example for illustration. The implementation details of the method for optimizing the deployment of a deep learning model on a terminal based on in-memory computing in this embodiment will be described in detail below. The following content is only for ease of understanding and is not essential for implementing this solution.
[0026] The specific process of the in-memory computing-based deep learning model terminal optimization deployment method in this embodiment can be described as follows: Figure 1 As shown, it includes:
[0027] Step 101: Analyze the trained model to obtain the computation graph corresponding to the model.
[0028] Specifically, the server first needs to transform the model into a computation graph, and then use operator rearrangement techniques in the computation graph dimension to solve the problem of limited computing and storage resources on the terminal. That is, the trained model is parsed to obtain the computation graph corresponding to the model.
[0029] In some examples, the server converts a model trained on a pre-defined training platform into a model file of a pre-defined format using the API (Application Programming Interface) provided by that platform, and then parses the model file to obtain the computational graph corresponding to the model. Both the pre-defined training platform and the pre-defined format can be selected and set by those skilled in the art according to actual needs.
[0030] In some examples, the default training platform is the TensorFlow training platform, and the default format is TFLite format. That is, the server will convert the model trained on the TensorFlow training platform into a TFLite format model file using the API interface provided by the TensorFlow training platform.
[0031] Step 102: Determine all branches of the model in the computation graph, traverse each branch, and divide all tensors at the current branch into constants and variables.
[0032] In practical implementation, the optimization space in the computation graph lies in the execution order at the branches. First, it's necessary to determine all the branches of the model in the computation graph. For example, a branch of the model... Figure 2 As shown, it contains five operators: operator 1, operator 2, operator 3, operator 4, and operator 5. Figure 2 (Tensors are not displayed in the graph). The server traverses each branch and divides all tensors at the current branch into constants and variables based on their attributes in the computation graph. Constants are tensors calculated by operators that do not belong to the computation graph, while variables are tensors calculated by operators that belong to the computation graph.
[0033] It's important to note that constants only increase memory usage and do not affect the execution order. In other words, changes in the execution order do not affect the memory usage of constants; therefore, constants are not an optimization target. By dividing all tensors at each branch into constants and variables, we can filter out these constants that have no room for optimization and directly optimize the variables, making efficient use of optimization resources.
[0034] In some cases, before the server divides all tensors at the current branch into constants and variables, it also needs to calculate the memory usage of each tensor at that branch and cache the memory usage of each tensor. This is because when executing in different orders, the memory usage of a series of identical tensors may be calculated repeatedly. Therefore, caching the memory usage of each tensor in advance can avoid the additional overhead caused by repeated calculations.
[0035] Step 103: Calculate the peak memory usage of the operators that generate each variable in each candidate execution order at the current branch, and determine the total memory usage of each candidate execution order at the current branch based on the peak memory usage. The candidate execution order with the smallest total memory usage is determined as the optimal execution order at the current branch.
[0036] Before an operator performs an operation, a buffer needs to be allocated in memory to store the operator's input and output tensors. Once the operator finishes execution, if the input tensors are not needed in subsequent operators, the memory they occupied can be reclaimed, while the output tensor will continue to participate in the computation as the input tensor of other operators. When performing operations according to the topological order of the computation graph, the tensors stored in memory are closely related to the execution order, which also affects the memory usage during the execution process.
[0037] In practical implementation, there are multiple candidate execution orders at a branch, such as... Figure 2The branch shown contains two candidate execution orders: "Operator 1, Operator 2, Operator 4, Operator 3, Operator 5" and "Operator 1, Operator 3, Operator 2, Operator 3, Operator 5". The server needs to calculate the peak memory usage corresponding to the operators that generate each variable in each candidate execution order at the current branch, and determine the total memory usage corresponding to each candidate execution order at the current branch based on the peak memory usage corresponding to the operators that generate each variable. From all candidate execution orders, the candidate execution order with the smallest total memory usage is selected as the optimal execution order at the current branch.
[0038] In some cases, when the server calculates the peak memory usage of the operators for each variable in each candidate execution order at the current branch, it needs to traverse each candidate execution order, traverse each variable according to the current candidate execution order, calculate the input tensor of the operator that generates the current variable, other tensors that need to be cached before this input tensor, and the memory usage of the current variable, and use this memory usage as the peak memory usage of the operator that generates the current variable.
[0039] For example, a branch of the model and its tensor, such as Figure 3 As shown, this branch includes five operators: operator 1, operator 2, operator 3, operator 4, and operator 5. The input tensor of operator 1 is X0, which is a constant. The output tensor of operator 1 is X1, which is also the input tensor of operators 2 and 3. The output tensor of operator 2 is X2, which is also the input tensor of operator 4. The output tensor of operator 4 is X3, and the output tensor of operator 3 is X4. X3 and X4 also serve as the input tensors of operator 5, and the output tensor of operator 5 is X5.
[0040] For the first candidate execution order, namely "operator 1, operator 2, operator 4, operator 3, operator 5", operator 1 reaches its peak memory usage when outputting X1, with a peak memory usage of X0 + X1. At this point, X0 has been used up and can be removed, no longer consuming memory. The peak memory usage of operator 2 is X1 + X2. Since operator 3 has not yet been executed, X1 still needs to be cached. Next, operator 4 is executed, with a peak memory usage of X1 + X2 + X3. At this point, X2 has been used up and can be removed, no longer consuming memory. Next, operator 3 is executed, with a peak memory usage of X1 + X3 + X4. At this point, X1 has been used up and can be removed, no longer consuming memory. Finally, operator 5 is executed, with a peak memory usage of X3 + X4 + X5. Thus, the peak memory usage for each operator is shown in Table 1.
[0041] Table 1: Peak memory usage for each operator in the first candidate execution order
[0042] Peak memory usage X0+X1 X1+X2 X1+X2+X3 X1+X3+X4 X3+X4+X5
[0043] For the second candidate execution order, namely "operator 1, operator 3, operator 2, operator 4, operator 5", the peak memory usage of each operator is shown in Table 2 according to the same calculation method.
[0044] Table 2: Peak memory usage for each operator in the second candidate execution order
[0045] Peak memory usage X0+X1 X1+X4 X1+X4+X2 X4+X2+X3 X3+X4+X5
[0046] Based on this, the total memory usage corresponding to the first candidate execution order is calculated as: X0 + 4X1 + 2X2 + 3X3 + 2X4 + X5, and the total memory usage corresponding to the second candidate execution order is calculated as: X0 + 3X1 + 2X2 + 2X3 + 4X4 + X5. The server compares the two. If X0 + 4X1 + 2X2 + 3X3 + 2X4 + X5 is greater than X0 + 3X1 + 2X2 + 2X3 + 4X4 + X5, then the second candidate execution order is taken as the optimal execution order at the current branch. Otherwise, the first candidate execution order is taken as the optimal execution order at the current branch.
[0047] like Figure 2 As shown, X0+4X1+2X2+3X3+2X4+X5 is greater than X0+3X1+2X2+2X3+4X4+X5, so the server takes "operator 1, operator 3, operator 2, operator 4, operator 5" as the optimal execution order at the current branch.
[0048] In some cases, if all tensors at the current branch are constants, the default execution order at the current branch is directly determined as the optimal execution order. Considering that there is no optimization space at branches with all constants, directly determining the default execution order as the optimal execution order can improve optimization efficiency and save reordering resources.
[0049] Step 104: Based on the optimal execution order corresponding to each branch, deploy the model on a preset terminal for inference.
[0050] In practical implementation, after determining the optimal execution order for each branch, the server can deploy the model on a preset terminal for inference according to the optimal execution order for each branch. This deployment minimizes the memory consumption of the model during runtime, enabling the model to adapt well to terminals with limited memory resources.
[0051] In this embodiment, the model file is represented as a computation graph. Operator rearrangement techniques are used at the computation graph level to address the limited computing and storage resources of the terminal. For each branch, tensors are first divided into constants and variables. Constants do not affect the execution order and cannot be optimized, while variables are the objects to be optimized. For multiple candidate execution orders at a branch, the total memory usage of each candidate execution order at the branch is calculated by generating the peak memory usage corresponding to the operators of each variable. The candidate execution order with the smallest total memory usage is determined as the optimal execution order. The model is then deployed according to the optimal execution order corresponding to each branch. The optimal execution order minimizes the memory usage during model runtime, effectively adapting to terminals with limited memory resources. It achieves minimal memory usage and reduces memory overhead without affecting the model's processing accuracy.
[0052] In some embodiments, in addition to performing topological rearrangement of the model based on computation graph branches, the server can also perform rearrangement of the model based on pruning operators. That is, the server deploys the model on a preset terminal for inference based on the optimal execution order corresponding to each branch, which can be achieved through methods such as... Figure 4 The steps shown are implemented as follows:
[0053] Step 201: For each branch, record the position of each trimming operator in the optimal execution order corresponding to the branch.
[0054] In the specific implementation, for each branch, the server takes the optimal execution order corresponding to that branch as input and rearranges it according to the pruning operators. First, it records the position of the pruning operators. Figure 5 The figure shows the position of a pruning operator in the optimal execution order.
[0055] Step 202: Traverse each clipping operator and determine whether the predecessor operator of the current clipping operator is an element-wise operator.
[0056] Step 203: If the predecessor operator of the current pruning operator is an element-wise operator, then swap the positions of the current pruning operator and its predecessor operator, and update the optimal execution order until the current pruning operator has no swappable objects. After all pruning operators have no swappable objects, the final updated optimal execution order corresponding to the branch is obtained.
[0057] In the specific implementation, the server traverses each trimming operator in the optimal execution order at a branch, determining whether the predecessor operator of the current trimming operator is an element-wise operator. If the predecessor operator of the current trimming operator is not an element-wise operator, there is no need to adjust the position of the current trimming operator in the optimal execution order. If the predecessor operator of the current trimming operator is an element-wise operator, the server swaps the positions of the current trimming operator and its predecessor operator, updates the optimal execution order, and then continues to determine whether the predecessor operator of the current trimming operator is an element-wise operator, until there are no more swappable objects for the current trimming operator. After all trimming operators have no more swappable objects, the final updated optimal execution order corresponding to that branch can be obtained.
[0058] In some cases, the current clipping operator has no commutative objects, including when the predecessor operator of the current clipping operator is a clipping operator, or when there are no other operators preceding the current clipping operator.
[0059] Step 204: Based on the optimal execution order after the final update at each branch, deploy the model on a preset terminal for inference.
[0060] In its implementation, the server performs a pruning-oriented reordering of the model. The resulting optimal execution order significantly reduces both memory usage and computational load. Based on this optimal execution order for each branch, the server deploys the model on a pre-defined terminal for inference. This allows the model to be well-suited for terminals with limited computing resources.
[0061] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.
[0062] Another embodiment of this application relates to an electronic device, such as... Figure 6 As shown, it includes: at least one processor 301; and a memory 302 communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions executable by the at least one processor 301, the instructions being executed by the at least one processor 301 to enable the at least one processor 301 to execute the in-memory computing-based deep learning model terminal optimization deployment method in the above embodiments.
[0063] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.
[0064] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.
[0065] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.
[0066] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0067] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.
Claims
1. A method for optimizing the deployment of deep learning models on terminals based on in-memory computing, characterized in that, include: The trained model is analyzed to obtain the computational graph corresponding to the model. In the computation graph, all branches of the model are determined, and each branch is traversed. All tensors at the current branch are divided into constants and variables. The constants are tensors calculated by operators that do not belong to the computation graph, and the variables are tensors calculated by operators that belong to the computation graph. Calculate the peak memory usage of the operators that generate each variable in each candidate execution order at the current branch, and determine the total memory usage of each candidate execution order at the current branch based on the peak memory usage. Then, determine the candidate execution order with the smallest total memory usage as the optimal execution order at the current branch. Based on the optimal execution order corresponding to each branch, the model is deployed on a preset terminal for inference.
2. The method for optimizing the deployment of deep learning model terminals based on in-memory computing as described in claim 1, characterized in that, The step of deploying the model on a preset terminal for inference based on the optimal execution order corresponding to each of the branches includes: For each branch, record the position of each trimming operator in the optimal execution order corresponding to that branch; Iterate through each clipping operator and determine whether the predecessor operator of the current clipping operator is an element-wise operator; If the predecessor operator of the current pruning operator is an element-wise operator, then swap the positions of the current pruning operator and its predecessor operator, and update the optimal execution order until the current pruning operator has no swappable objects. After all the pruning operators have no swappable objects, the final updated optimal execution order corresponding to the branch is obtained. Based on the optimal execution order after the final update at each of the branches, the model is deployed on a preset terminal for inference.
3. The method for optimizing the deployment of deep learning model terminals based on in-memory computing as described in claim 2, characterized in that, The current clipping operator has no commutable objects, including: The preceding operator of the current clipping operator is a clipping operator, or there are no other operators before the current clipping operator.
4. The method for optimizing the deployment of deep learning model terminals based on in-memory computing according to any one of claims 1 to 2, characterized in that, The step of calculating the peak memory usage corresponding to the operator that generates each variable in each candidate execution order at the current branch includes: Traverse each of the candidate execution orders, traverse each of the variables according to the current candidate execution order, calculate the sum of the input tensor of the operator that generates the current variable, the other tensors that need to be cached before the input tensor, and the memory usage of the current variable, and use the sum of the memory usage as the peak memory usage of the operator that generates the current variable.
5. The method for optimizing the deployment of deep learning model terminals based on in-memory computing as described in claim 4, characterized in that, Before dividing all tensors at the current branch into constants and variables, the following is also included: Calculate the memory usage of each tensor at the branch and cache the memory usage of each tensor.
6. The method for optimizing the deployment of deep learning model terminals based on in-memory computing according to any one of claims 1 to 2, characterized in that, After dividing all tensors at the current branch into constants and variables, the following is also included: If all tensors at the current branch are constants, then the default execution order at the current branch is directly determined as the optimal execution order at the current branch.
7. The method for optimizing the deployment of deep learning model terminals based on in-memory computing according to any one of claims 1 to 2, characterized in that, The step of parsing the trained model to obtain the computational graph corresponding to the model includes: The model trained on the preset training platform is converted into a model file of a preset format using the API interface provided by the training platform, and the model file is parsed to obtain the computation graph corresponding to the model.
8. The method for optimizing the deployment of deep learning model terminals based on in-memory computing as described in claim 7, characterized in that, The preset training platform is the TensorFlow training platform, and the preset format is the TFLite format.
9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the in-memory computing-based deep learning model terminal optimization deployment method as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the terminal optimization deployment method for deep learning models based on in-memory computing as described in any one of claims 1 to 8.