Deep learning network optimization method, system and device
By dividing the calculation subgraphs and evaluating the performance values of nodes, and adjusting the neural network computing graph structure, the calculation efficiency problem caused by incompatibility with the network layer in the prior art is solved, and efficient data reading and writing under finite on-chip memory is achieved .
Patent Information
- Application Number
- CN202310622726.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-29
- Publication Date
- 2024-11-29
AI Technical Summary
In the prior art, the method of inter-layer fusion of neural networks is limited by the network structure, resulting in more network layers that are not suitable for fusion, which cannot improve computing efficiency, especially when the on-chip memory bandwidth is limited, resulting in low data reading and writing efficiency. .
By obtaining the original calculation diagram, it is divided into multiple computing sub-graphs that can perform on-chip memory fusion, evaluate the theoretical performance value of each node, determine whether on-chip memory fusion is performed, adjust the calculation diagram structure based on the fusion income value, add or delete nodes to Optimize computing efficiency.
It improves the computing efficiency of the entire computing graph, ensures the optimal execution method of each node, improves the data reading and writing efficiency, and is suitable for neural network applications with limited on-chip memory.
Smart Images

Figure CN119048876A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of video monitoring and analysis technology, and in particular to a deep learning network optimization method, system and device. Background Art
[0002] Neural networks are widely used in various fields, such as image recognition, object recognition, and vehicle recognition. In the process of neural network application, it will be arranged in the corresponding execution subject according to the specific application scenario. In the specific implementation process, the efficiency of direct reading and writing through on-chip memory is much faster than obtaining from off-chip memory, but the device memory of the execution subject is limited. For example, the bandwidth of the on-chip memory of the device is limited. When reading and writing data, if the amount of input data is larger than the on-chip memory, the data cannot be read directly from the on-chip memory, and the data needs to be stored outside the chip first, which leads to low computational efficiency of the neural network application.
[0003] In the prior art, optimization is often performed by fusing neural network layers. However, since this method is limited by the network structure, once there are network layers in the neural network structure that are not suitable for fusion, optimization cannot be performed. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a deep learning network optimization method, system and device to solve the problem of how to improve the computing efficiency when there are many network layers in the neural network that are not suitable for fusion. The specific technical solution is as follows:
[0005] In a first aspect of the present application, a deep learning network optimization method is provided, the method comprising:
[0006] Acquire an original computation graph, where the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory;
[0007] Evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion respectively; wherein the theoretical performance value is the time consumed from input to output of each node; and the theoretical performance value is negatively correlated with the execution efficiency;
[0008] Determine a node whose lower theoretical performance value between the first theoretical performance value and the second theoretical performance value satisfies a preset condition as a node to be optimized;
[0009] A first fusion strategy for each of the computational subgraphs is determined based on nodes other than the optimized node.
[0010] In a possible implementation, the method further includes:
[0011] Traversing all the computational subgraphs, and evaluating the fusion benefit values of the fusion between the computational subgraphs;
[0012] A second fusion strategy between the original computational graph structures is determined based on the fusion benefit values of the respective computational subgraphs.
[0013] In a possible implementation, the computation subgraph is obtained in the following manner:
[0014] According to the way in which each node supports input data, the original computation graph is divided into a plurality of computation subgraphs that can be fused with on-chip memory, wherein each computation subgraph supports input data in the same way.
[0015] In a possible implementation, the method further includes:
[0016] Obtaining hardware specification information of each computing subgraph; the hardware specification information includes: at least one of on-chip memory size, DDR memory size, bandwidth data, and NPU core number;
[0017] Based on the hardware specification information and the input data of each computing subgraph, it is determined whether to add a node identical to the node at a location where the topological positions of the nodes in each computing subgraph are identical.
[0018] In a possible implementation, the method further includes:
[0019] Merge the single operator nodes supporting DDR fusion in the original computation graph into one DDR node;
[0020] and / or,
[0021] Delete nodes in the original computation graph that cannot be fused with on-chip memory.
[0022] In a possible implementation, the original calculation graph is a radar detection calculation graph, and the radar detection calculation graph is used to determine a radar detection algorithm so that the radar device performs detection based on the radar detection algorithm.
[0023] In a second aspect of the present application, a deep learning network optimization system is provided, the system comprising: an algorithm generation module, a radar device, wherein:
[0024] The algorithm generation module is used to obtain a radar detection algorithm after optimization based on the method of the first aspect;
[0025] The radar device is used for performing detection based on the radar detection algorithm.
[0026] In a third aspect of the present application, a deep learning network optimization device is provided, the device comprising:
[0027] An acquisition module, used to acquire an original computation graph, wherein the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory;
[0028] A first evaluation module is used to evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion; wherein the theoretical performance value is the time consumed from input to output of each node; and the theoretical performance value is negatively correlated with the execution efficiency;
[0029] An optimization node determination module, used to determine a node whose lower theoretical performance value between the first theoretical performance value and the second theoretical performance value meets a preset condition as an optimized node;
[0030] A fusion strategy determination module is used to determine a first fusion strategy for each of the computational subgraphs based on nodes other than the optimized node.
[0031] In a possible implementation manner, the device further includes:
[0032] A second evaluation module is used to traverse all the computing subgraphs and evaluate the fusion benefit value of the fusion between the computing subgraphs;
[0033] The fusion strategy determination module is further used to determine a second fusion strategy between the original calculation graph structures based on the fusion benefit values of the respective calculation subgraphs;
[0034] The computational subgraph is obtained in the following manner:
[0035] According to the way each node supports input data, the original computation graph is divided into a plurality of computation subgraphs that can be fused with on-chip memory, wherein each computation subgraph supports input data in the same way;
[0036] The acquisition module is further used to acquire the hardware specification information of each computing subgraph; the hardware specification information includes: at least one of on-chip memory size, DDR memory size, bandwidth data, and NPU core number;
[0037] A replication module, used for determining whether to add a node identical to the node at the same topological position of the nodes in the respective computing subgraphs based on the hardware specification information and the input data of the respective computing subgraphs;
[0038] A preprocessing module, used for fusing the single operator nodes supporting DDR fusion in the original calculation graph into one DDR node;
[0039] and / or,
[0040] Deleting nodes in the original computation graph that cannot be fused with on-chip memory;
[0041] The original calculation graph is a radar detection calculation graph, and the radar detection calculation graph is used to determine a radar detection algorithm so that the radar device performs detection based on the radar detection algorithm.
[0042] In a fourth aspect of the present application, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0043] Memory, used to store computer programs;
[0044] The processor is used to implement the method steps described in the first aspect when executing the program stored in the memory.
[0045] In a fifth aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps described in the first aspect are implemented.
[0046] Beneficial effects of the embodiments of the present application:
[0047] A method, system and device for deep learning network optimization provided by an embodiment of the present application can determine the fusion strategy by using the computational subgraphs in the original computational graph, split the original computational graph into multiple computational subgraphs that can perform on-chip memory fusion, and respectively evaluate the theoretical performance values of each node in each computational subgraph with and without on-chip memory fusion, and then determine the first fusion strategy based on the theoretical performance values. In an embodiment of the present application, the situations of performing on-chip memory fusion and not performing on-chip fusion on each node are compared to determine whether each node needs to perform on-chip memory fusion, and then make each node as optimal as possible. The execution mode, and then determine the fusion strategy of each computational subgraph based on each node, so that each computational subgraph can be the optimal execution scheme as much as possible, thereby improving the computational efficiency of the entire original computational graph.
[0048] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.
[0050] Figure 1-1 A schematic diagram of on-chip memory fusion in a prior art provided in an embodiment of the present application;
[0051] Figure 1-2 A schematic diagram of on-chip memory fusion in deep learning network optimization provided in an embodiment of the present application;
[0052] Figure 2-1 A schematic diagram of a deep learning network optimization method provided in an embodiment of the present application;
[0053] Figure 2-2 A schematic diagram of a calculation process after data segmentation provided in an embodiment of the present application;
[0054] Figure 3 A schematic diagram of the overall process of a deep learning network optimization method provided in this application;
[0055] Figure 4 A schematic diagram of a graph optimization preprocessing process provided in an embodiment of the present application;
[0056] Figure 5-1 A schematic diagram of a fusion strategy generation process provided in an embodiment of the present application;
[0057] Figure 5-2 A schematic diagram of a fusion calculation graph flow chart after fusion based on a fusion strategy provided in an embodiment of the present application;
[0058] Figure 5-3 A schematic diagram of a non-human detection algorithm flow provided in an embodiment of the present application;
[0059] Figure 5-4 A schematic diagram of an optimized machine-non-human detection algorithm flow provided in an embodiment of the present application;
[0060] Figure 6 A schematic diagram of a deep learning network optimization device provided in an embodiment of the present application;
[0061] Figure 7 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.
[0063] Neural network calculations require a large data throughput. In the process of neural network applications, data reading and writing often affect the performance of the entire neural network. In the case of limited on-chip memory bandwidth, to solve the neural network performance problem, it is necessary to improve data reading and writing efficiency.
[0064] In the prior art, the method for optimizing data reading and writing mainly adopts the method of fusing deep learning network layers. However, this method is limited by the network structure. If there are many network layers that are not suitable for fusion in the network structure, the data reading and writing efficiency cannot be improved, which leads to low computing efficiency.
[0065] Operator fusion is divided into two categories: one is complete fusion in formulas, and the other is that users combine their own business needs to trigger and manually formulate rules to fuse multiple layers. In actual applications, the first type of formula fusion is a true fusion, which has the best way to achieve data throughput and computing acceleration, but there are fewer layers with such high fusion. The second type of fusion is often more common in applications. Therefore, how to internally accelerate the second type of fusion solution is an issue that needs to be considered.
[0066] Due to the limited size of on-chip memory, in the prior art, when performing on-chip memory fusion, only the operator data flow modification is involved, such as Figure 1-1 As shown in the figure, Data is input data, Op is node, and Output is output data. However, when on-chip memory fusion is performed in this way, the capacity of on-chip memory and off-chip DDR memory often differs by several orders of magnitude. Therefore, Figure 1-1 When on-chip memory fusion is performed in the manner shown, once the difference between the on-chip memory and the input data amount is too large, this will cause the on-chip memory of the node to be unable to bear the input data and the on-chip memory can only read input data of the memory size each time, thereby affecting the computing efficiency.
[0067] In order to solve the above problems, the inventors considered using a splitting operator to reduce the amount of data calculated in the on-chip memory each time. Figure 1-2 As shown, on-chip memory fusion is performed and operators are segmented, so as to achieve the purpose of on-chip memory data temporary storage, thereby improving computing efficiency.
[0068] An embodiment of the present application provides a deep learning network optimization method, which can determine the first fusion strategy within each computing subgraph structure in the original computing graph through the theoretical performance value of each node on the original computing graph, and each node in the first fusion strategy is the optimal fusion strategy, so fusing the original computing graph based on the first fusion strategy can improve the computing efficiency of the original computing graph.
[0069] In a possible embodiment, a deep learning network optimization is provided, such as Figure 2-1 As shown, the method includes:
[0070] S201. Obtain an original computation graph, where the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory.
[0071] S202. Evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value with on-chip memory fusion respectively; wherein the theoretical performance value is the time consumed by each node from input to output, and the theoretical performance value is negatively correlated with the execution efficiency.
[0072] S203, determining a node whose lower theoretical performance value between the first theoretical performance value and the second theoretical performance value meets a preset condition as a node to be optimized;
[0073] S204: Determine a first fusion strategy for each computational subgraph based on nodes other than the optimized node.
[0074] In an embodiment of the present application, a comparison is made between the situations where each node undergoes on-chip memory fusion and does not undergo on-chip fusion to determine whether each node needs on-chip memory fusion, thereby making each node as optimally executed as possible, and then determining a fusion strategy for each computational subgraph based on each node. This way, each computational subgraph can be made as optimally executed as possible, thereby improving the computational efficiency of the entire original computational graph.
[0075] S201-S204 are described in detail below:
[0076] In S201, the original computation graph is a computation graph corresponding to the deep learning network structure. The original computation graph includes at least a plurality of computation subgraphs that can be fused with on-chip memory, that is, it can be understood that the original computation graph includes both a plurality of computation subgraphs that can be fused with on-chip memory and computation subgraphs that cannot be fused with on-chip memory. The original computation graph may also only include a plurality of computation subgraphs that can be fused with on-chip memory. However, a deep learning network optimization method in an embodiment of the present application is not applicable to the case where the original computation graph only includes computation subgraphs that cannot be fused with on-chip memory.
[0077] In S202, for each node in each computing subgraph, the first theoretical value is the theoretical performance value obtained when each node does not perform on-chip memory fusion, and the second theoretical value is the theoretical performance value obtained when each node performs on-chip memory fusion. That is, the theoretical performance values obtained when each node performs on-chip memory fusion and when each node does not perform on-chip memory fusion are evaluated.
[0078] The theoretical performance value is the time consumed by each node from input to output. This time can be understood as the time period from inputting data to the node to obtaining the output result, that is, the time period spent by the node to perform calculations.
[0079] The shorter the time period that the node takes to perform the calculation, the higher the execution efficiency of the node. The execution efficiency of a node can be determined based on the lower of the first theoretical performance value and the second theoretical performance value. For example, the first theoretical performance value of node A is t1, the second theoretical performance value is t2, t1<t2, then the execution efficiency of node A can be determined based on the first theoretical performance value t1.
[0080] In S203, by comparing the size of the first theoretical performance value and the size of the second theoretical performance value of each node, it is determined whether to perform on-chip memory fusion on the node. Exemplarily, when the first theoretical performance value of a node is greater than the second theoretical performance value, it means that the time consumed for on-chip memory fusion is short, and the node performs on-chip memory fusion; when the first theoretical performance value of a node is less than the second theoretical performance value, it means that the time consumed for not performing on-chip memory fusion is short, and the node does not perform on-chip memory fusion. For each computational subgraph, it is determined in the same way whether each node in the computational subgraph performs on-chip memory fusion, and then the first fusion strategy in each computational substructure can be obtained. Due to the limited size of the on-chip memory space, it is necessary to make high-yield nodes perform on-chip memory fusion as much as possible in the future. Therefore, in order to further improve the computing efficiency, for inefficient nodes, the node can be deleted from the strategy that needs to be fused. The deletion in the embodiment of the present application does not delete the node and thus change the computational graph, but excludes it from the fusion strategy, that is, the node does not need to be fused.
[0081] In order to make the high-yield nodes perform on-chip memory fusion as much as possible, and then make the overall computing efficiency as much as possible, it is necessary to optimize the optimized nodes whose execution efficiency meets the preset conditions. That is, it is deleted from the preliminary order for determining the current fusion scheme, and the fusion scheme is determined from the nodes other than the optimized nodes. The deletion in the embodiment of the present application does not mean deleting the node from the calculation graph, but deleting the node from the preliminary nodes in determining the fusion strategy. That is, when determining the fusion strategy, the node will not be considered. The first theoretical performance value is a value set by those skilled in the art according to the actual application scenario. When the execution efficiency of the node meets the preset conditions, it means that the execution efficiency of the node is low. The fusion strategy will not select the node, so it needs to be deleted from the preliminary nodes.
[0082] There are two cases where the preset condition is met: one is to determine whether the execution efficiency of the node is less than the preset execution efficiency threshold. If it is less than, then the preset condition is met. The other is to sort the execution efficiency of each node in each computational subgraph in descending order, and take the preset number of nodes with the lowest execution efficiency as the nodes that meet the preset condition. In this embodiment, the preset execution efficiency and the preset number are set by those skilled in the art according to the actual application scenario.
[0083] In S204, the first fusion strategy includes all nodes in the computational subgraph that can be fused with on-chip memory. Exemplarily, assuming that the computational subgraph includes nodes 1, 2, 3, 4, and 5, and the execution efficiencies of nodes 1, 2, 3, 4, and 5 are all theoretical performance values for on-chip memory fusion, wherein the execution efficiencies of nodes 1 and 5 are lower than the preset execution efficiencies, then nodes 1 and 5 are deleted in the current fusion strategy, and the first fusion strategy includes: nodes 2, 3, and 4. The first fusion strategy may also include some nodes in the computational subgraph that can be fused with on-chip memory, according to actual conditions. For example, assuming that the computing subgraph includes node 1, node 2, node 3, node 4, and node 5, and the execution efficiency of node 1, node 2, node 3, node 4, and node 5 are all theoretical performance values for on-chip memory fusion, wherein the execution efficiency of node 1 and node 5 is lower than the preset execution efficiency, then node 1 and node 4 will be deleted in the current fusion strategy, but considering the actual application scenario, all nodes cannot be fused, then the first fusion strategy includes: node 2 and node 3.
[0084] In a possible embodiment, in order to make the operators of each divided computational subgraph run normally as much as possible and further improve the computational efficiency, the computational subgraph can be obtained in the following manner:
[0085] According to the way each node supports input data, the original computation graph is divided into multiple computation subgraphs that can be fused with on-chip memory, where each computation subgraph supports input data in the same way.
[0086] In this step, the way in which each node supports input data can be the node's support capability for on-chip memory scenarios, such as not supporting splitting, supporting batch axis splitting, supporting internal axis splitting, and supporting on-chip memory data flow; thus, the capabilities supported by the nodes within each split computing subgraph can be made the same as much as possible, so that the data input to the computing subgraph can run normally without additional processing as much as possible.
[0087] In order to further improve the computational efficiency of the original computational graph, a second fusion strategy between the various computational subgraph structures may be determined based on the whole. In a possible embodiment, the above method further includes:
[0088] S205: traverse all computation subgraphs and evaluate the fusion benefit value of the fusion between the computation subgraphs.
[0089] In this step, traversing all computational subgraphs means traversing all the fusion situations between computational subgraphs and evaluating the fusion benefit of each situation. The fusion benefit is the time period from operator input to output of the entire solution. The fusion benefit is negatively correlated with the length of the time period.
[0090] For example, it is assumed that computation graph 1 includes computation subgraph 1, computation subgraph 2, and computation subgraph Figure 3 , calculator Figure 4 , calculation subgraph 5, where calculation subgraph 2, calculation subgraph Figure 3 , calculator Figure 4 It can be fused, but computation subgraph 1 and computation subgraph 5 do not need to be fused. There are three fusion methods: fusion method 1: fusion of computation subgraphs 2, 3, and 4; fusion method 2: fusion of computation subgraphs 2 and 3; and fusion method 3: fusion of computation subgraphs 2 and 3. Figure 3 , 4 , it is necessary to traverse all computational subgraphs and evaluate the fusion benefits of the three fusion methods. Fusion benefit 1 is the time t1 from the input of computational graph 1 to the output of computational graph 5, fusion benefit 2 is the time t2 from the input of computational graph 1 to the output of computational graph 5, and fusion benefit 3 is the time t3 from the input of computational graph 1 to the output of computational graph 5. By comparing the size relationship of t1, t2, and t3, the final fusion strategy is determined.
[0091] S206. Determine a second fusion strategy between the original computation graph structures based on the fusion benefit values of each computation subgraph.
[0092] The solution corresponding to the minimum value is selected from each fusion benefit value as the second fusion strategy.
[0093] Exemplarily, still taking the aforementioned calculation graph 1 as an example, assuming that t2<t1<t3, the second fusion strategy between the structures of the final calculation graph 1 is determined according to fusion mode 2.
[0094] In a possible embodiment, since the size of the on-chip memory is limited, in order to avoid excessive data volume processed by the on-chip memory, resulting in low computing efficiency, after obtaining the first fusion strategy and the second fusion strategy, according to the size of the on-chip memory and the input data, add nodes identical to the nodes at the same topological position of the nodes in each computing subgraph. Distribute the input data to different branches for processing, thereby improving the computing efficiency of the on-chip memory. The method also includes:
[0095] S207: Obtain hardware specification information of each computing subgraph.
[0096] In this step, the hardware specification information of the computational subgraph includes: on-chip memory size, DDR memory size, bandwidth data, number of NPU (Neural Network Processing Unit) cores, etc. Among them, bandwidth data is the data transmission capacity, and the number of NPU cores is the processor performance value. The hardware specification information of the computational subgraph is used for subsequent performance evaluation and on-chip resource allocation.
[0097] S208: Based on the hardware specification information and the input data of the computation subgraphs, determine whether to add a node identical to the node at the same topological position of the nodes in each computation subgraph.
[0098] In this step, it is determined whether the on-chip memory size is smaller than the input data size of the computational subgraph. If not, no node needs to be added.
[0099] If it is less than, then calculate the ratio of input data to on-chip memory, and use the ratio to determine how many identical nodes to add in the same topological position of the node. Figure 1-2 As shown, the amount of input data on the two branches OP2-1→OP3-1→OP4-1 and OP2-2→OP3-2→OP4-2 is made less than or equal to the on-chip memory as much as possible.
[0100] For example, assuming that the input data size is 300KB, the on-chip memory is 160KB, and the ratio is 1.875, then add 1 node identical to the node in the same place as each node in the computational subgraph, and the input data is divided into 2 150KB portions and input into the on-chip memory respectively. Assuming that the input data size is 600KB, the on-chip memory is 100KB, and the ratio is 6, then add 5 nodes identical to the node in the same place as each node in the computational subgraph, and the input data is divided into 6 100KB portions and input into the on-chip memory respectively. The entire computational process of inputting data into the on-chip memory and then outputting data to the on-chip memory is actually divided into three parts: 1) copying the original data to be calculated on the DDR memory to the on-chip memory; 2) the on-chip memory calculates the data; 3) the calculated data is copied from the on-chip memory to the DDR memory. Due to the limited on-chip memory, such as Figure 2-2 As shown, still taking the aforementioned input data size of 300KB as an example, when the input data is divided into two parts, the calculation process of the first data block and the process of the second data block are distributedly arranged, and the three calculation processes of each data are performed alternately, thereby improving the on-chip memory utilization and allowing the network to operate normally and efficiently even when the input data is larger than the on-chip memory.
[0101] In a possible embodiment, when evaluating the theoretical performance value of each node in the original calculation graph, it is necessary to traverse the original calculation graph. In order to reduce the traversal time, the original calculation graph can be preprocessed, thereby improving the calculation efficiency.
[0102] The method further includes:
[0103] S209, merging the single operator nodes supporting DDR fusion in the original calculation graph into one DDR node;
[0104] and / or,
[0105] Delete the nodes in the original computation graph that cannot be fused with on-chip memory.
[0106] In this step, the node fusion or node deletion operation can be used to reduce the number of nodes in the original calculation graph and simplify the original calculation graph, thereby reducing the traversal time as much as possible during subsequent evaluation.
[0107] like Figure 3 As shown in FIG. 1 , it is a schematic diagram of the overall process of a deep learning network optimization method provided by the present application. The entire deep learning network optimization method is divided into three modules: graph optimization preprocessing, fusion strategy generation, and network layer optimization module. Figure 4As shown, it is a schematic diagram of a graph optimization preprocessing process provided by an embodiment of the present application. The specific graph optimization preprocessing is: graph fusion module → graph cropping module → graph segmentation module → obtain N sub-graph structures (obtain N sub-computing graph structures). Among them, the graph fusion module fuses DDR operators by setting fusion rules (optional, that is, manually setting fusion rules and forced fusion); the graph cropping module deletes useless nodes by setting cropping rules (optional, that is, manually setting cropping rules and forced cropping); the graph segmentation module identifies network layer attributes and splits the computing graph by setting segmentation rules (optional, that is, manually setting segmentation rules and forced segmentation). Figure 5-1 As shown in FIG. 1 , a schematic diagram of a fusion strategy generation process provided in an embodiment of the present application is shown. Figure 5-2 As shown, a schematic diagram of a fused computational graph flow based on a fusion strategy provided in an embodiment of the present application is provided, and the process includes: traversing all computational subgraphs, copying nodes according to the segmentation strategy, modifying shape information, modifying node data flow, setting key node data flow attributes, and obtaining an optimized computational graph. The specific process details have been described in the aforementioned steps and will not be elaborated on here.
[0108] In a possible embodiment, the original calculation graph is a radar detection calculation graph, which is used to determine a radar detection algorithm so that the radar device performs detection based on the radar detection algorithm, thereby making the detection efficiency as high as possible.
[0109] The present application provides a method for determining a radar detection algorithm, the method comprising:
[0110] S301, obtaining an original computation graph, wherein the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory, and the original computation graph is a radar detection computation graph.
[0111] S302. Evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion respectively. The theoretical performance value is the time consumed by each node from input to output. The time can be understood as the time period from inputting data to the node to obtaining the output result, that is, the time period spent by the node to perform the calculation.
[0112] S303: Determine a node whose lower theoretical performance value satisfies a preset condition between the first theoretical performance value and the second theoretical performance value as the node to be optimized.
[0113] S304: Determine the first fusion strategy for each computational subgraph based on nodes other than the optimized node
[0114] S305: traverse all computation subgraphs and evaluate the fusion benefit value of the fusion between the computation subgraphs.
[0115] S306: Determine a second fusion strategy between the original computation graph structures based on the fusion benefit values of each computation subgraph.
[0116] S307: Obtain hardware specification information of each computational subgraph.
[0117] S308: Based on the hardware specification information and the input data of each computing subgraph, determine whether to add a node identical to the node at the same topological position of the nodes in each computing subgraph.
[0118] The above S301-S308 are the same as the aforementioned S201-S208, and will not be elaborated here.
[0119] S309: Modify the original calculation graph based on the first fusion strategy and the second fusion strategy to obtain a radar detection algorithm.
[0120] In this step, the entire radar detection calculation graph is traversed, and the radar detection calculation graph is modified based on the first fusion strategy and the second fusion strategy. Then, the attributes of each data flow are recorded (DDR->on-chip memory, or on-chip memory->DDR), and finally the optimized radar detection algorithm is obtained.
[0121] In a possible embodiment, the original calculation graph may also be a face detection calculation graph, which is used to determine a face detection algorithm so that the face detection device performs detection based on the face detection algorithm, thereby making the detection efficiency as high as possible.
[0122] The present application provides a method for determining a face detection algorithm, the method comprising:
[0123] S401, obtaining an original computation graph, wherein the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory, and the original computation graph is a facial detection computation graph.
[0124] S402. Evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion respectively. The theoretical performance value is the time consumed by each node from input to output. The time can be understood as the time period from inputting data to the node to obtaining the output result, that is, the time period spent by the node to perform calculations.
[0125] S403: Determine a node whose lower theoretical performance value satisfies a preset condition between the first theoretical performance value and the second theoretical performance value as the node to be optimized.
[0126] S404: Determine a first fusion strategy for each computational subgraph based on nodes other than the optimized node
[0127] S405: traverse all computation subgraphs and evaluate the fusion benefit value of the fusion between the computation subgraphs.
[0128] S406: Determine a second fusion strategy between the original computation graph structures based on the fusion benefit values of each computation subgraph.
[0129] S407: Obtain hardware specification information of each computational subgraph.
[0130] S408: Based on the hardware specification information and the input data of each computation subgraph, determine whether to add a node identical to the node at the same topological position of the nodes in each computation subgraph. The above S401-S407 are the same as the above S301-S307, and will not be described in detail here.
[0131] S409: Modify the original calculation graph based on the first fusion strategy and the second fusion strategy to obtain a face detection algorithm.
[0132] In this step, the entire face detection calculation graph is traversed, and the face detection calculation graph is modified based on the first fusion strategy and the second fusion strategy, and then the attributes of each data flow are recorded (DDR->on-chip memory, or on-chip memory->DDR), and finally the optimized face detection algorithm is obtained.
[0133] In a possible embodiment, the original calculation graph can also be a structured machine-non-human (motor vehicle, non-motor vehicle, human) detection calculation graph, which is used to determine a structured machine-non-human detection algorithm, so that the structured machine-non-human detection device performs detection based on the structured machine-non-human detection algorithm, thereby making the detection efficiency as high as possible.
[0134] The embodiment of the present application provides a method for determining a structured machine non-human detection algorithm. In actual application scenarios, the structured machine non-human detection algorithm needs to be deployed on a computer. The following describes a specific application scenario of the network optimization method provided in the embodiment of the present application applied to the machine non-human detection algorithm. The method includes:
[0135] S501, obtaining the original calculation graph of the machine-not-human detection algorithm, wherein the original calculation graph includes a decoding module, a target detection module, a feature analysis module, a picture capture module, and a business processing module.
[0136] In this step, if Figure 5-3As shown in the figure, the whole non-human detection algorithm is divided into five major modules, namely: decoding module, target detection module, target tracking module, image capture module, and business processing module. The business processing module performs different processing units according to the image business type, such as human business processing unit, vehicle business processing unit, and non-motor vehicle business processing unit. These five major modules are composed of multiple computational subgraphs. And they include multiple computational subgraphs that can be fused with on-chip memory.
[0137] S502, respectively evaluate the first theoretical performance value of the decoding module, the target detection module, the target tracking module, the image capture module, and the business processing module without on-chip memory fusion and the second theoretical performance value of on-chip memory fusion. Among them, the theoretical performance value is the time consumed by each node from input to output; the theoretical performance value is the time consumed by each node from input to output, which can be understood as the time period from inputting data to the node to obtaining the output result, that is, the time period spent by the node to perform calculations, and the theoretical performance value is negatively correlated with the execution efficiency.
[0138] In this step, the first theoretical performance value and the second theoretical performance value of the decoding module, the target detection module, the target tracking module, the image capture module, and the business processing module with and without on-chip fusion are evaluated respectively.
[0139] S503: Determine a module whose lower theoretical performance value satisfies a preset condition between the first theoretical performance value and the second theoretical performance value as the optimized module.
[0140] In this step, by comparing the first theoretical performance value and the second theoretical performance value of each module, the module with the lower theoretical performance value is selected to determine the execution efficiency of the module. Exemplarily, assuming that the first theoretical performance values of the decoding module, the target detection module, the target tracking module, the picture capture module, and the business processing module are all less than the second theoretical performance value, it is necessary to further compare the first theoretical performance value of each module with the preset threshold value, and determine that the first theoretical performance value of the target detection module, the target tracking module, and the picture capture module is less than the preset threshold value, and the first theoretical performance value of the decoding module and the business processing module is greater than the preset threshold value, then determine that the decoding module and the business processing module are optimized modules.
[0141] S504: Determine a first fusion strategy for each module based on modules other than the optimized module.
[0142] In this step, still taking the above as an example, since the decoding module and the business processing module are the optimized modules, the first fusion strategy is determined based on the target detection module, the target tracking module, and the picture capture module, that is, the first fusion strategy is to fuse the target detection module, the target tracking module, and the picture capture module on-chip.
[0143] S505: traverse all modules and evaluate the fusion benefit value of the fusion between each module.
[0144] S506: Determine a second fusion strategy among modules based on the fusion benefit value.
[0145] In this step, the fusion benefits of the fusion between the decoding module, the target detection module, the target tracking module, the picture capture module, and the business processing module are determined. For example, it is assumed that case 1 is: the decoding module, the target detection module, the target tracking module, the picture capture module, and the business processing module are all fused, and the fusion benefit is t1. Case 2 is: the decoding module, the target detection module, the target tracking module, and the picture capture module are fused, and the business processing module is not fused, and the fusion benefit is t2. By analogy, the fusion benefits of all fusion situations are determined. It is assumed that the situation with the highest fusion benefit is: the target detection module, the target tracking module, and the picture capture module are fused, and the decoding module and the business processing module are not fused. Then the second fusion strategy is: the target detection module, the target tracking module, and the picture capture module are fused, and the decoding module and the business processing module are not fused.
[0146] S507 , obtaining the on-chip memory size, DDR memory size, bandwidth data, and NPU core number pre-allocated by the computer for each module.
[0147] In this step, since the above modules are to be deployed in a computer, and the resources of each computer are limited, the resources allocated to each module are also limited. Therefore, before deployment, it is necessary to obtain the on-chip memory size, DDR memory size, bandwidth data and NPU core number allocated by the computer to each module, so as to better deploy the optimized structured non-human detection algorithm in the computer later.
[0148] S508 . Based on the size of the on-chip memory and the size of the input data, determine whether to add a module identical to the module at the location where the topological positions of the modules are the same.
[0149] In this step, as described in S207 above, branches are added based on the size of the on-chip memory and the size of the input data in the hardware specification information. Taking the above as an example, Figure 5-4As shown, assuming that the input data size is 300KB and the on-chip memory is 160KB, since the target detection module, target tracking module, and picture capture module need to be integrated in the determined fusion strategy, and the on-chip memory is smaller than the input data, in order to make it run normally as much as possible, the input data is divided into two input data 1 and input data 2. The sizes of input data 1 and input data 2 are 150KB respectively, and the target detection module, target tracking module, and picture capture module are copied to obtain two branches: target detection module 1, target tracking module 1, picture capture module 1 and target detection module 2, target tracking module 2, picture capture module 2. In this way, the input data 1 and the input data 2 can be input into the two branches respectively, thereby improving the on-chip memory utilization, and even when the input data is larger than the on-chip memory, the network can run normally and efficiently.
[0150] S509: Modify the original calculation graph based on the first fusion strategy and the second fusion strategy to obtain a structured machine non-human detection algorithm.
[0151] In this step, the entire structured machine non-person detection calculation graph is traversed, and the structured machine non-person detection calculation graph is modified based on the first fusion strategy and the second fusion strategy, and then each data flow attribute is recorded (DDR->on-chip memory, or on-chip memory->DDR), and finally the optimized structured machine non-person detection algorithm is obtained.
[0152] After the above S509, the optimized structured machine non-human detection algorithm is deployed to the computer, so that the structured machine non-human detection can be implemented through the computer, and the hardware resources on the computer are fully utilized to improve the execution efficiency.
[0153] The present application also provides a deep learning network optimization device, such as Figure 6 As shown, the device comprises:
[0154] An acquisition module 601 is used to acquire an original computation graph, where the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory;
[0155] A first evaluation module 602 is used to evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion; wherein the theoretical performance value is the time consumed from input to output of each node; and the theoretical performance value is negatively correlated with the execution efficiency;
[0156] An optimization node determination module 603 is used to determine a node whose lower theoretical performance value between the first theoretical performance value and the second theoretical performance value meets a preset condition as an optimized node;
[0157] The fusion strategy determination module 604 is used to determine the first fusion strategy of each computing subgraph based on the nodes other than the optimized node.
[0158] In a possible implementation manner, the device further includes:
[0159] The second evaluation module 605 is used to traverse all the computation subgraphs and evaluate the fusion benefit value of the fusion between the computation subgraphs;
[0160] The fusion strategy determination module 604 is further used to determine a second fusion strategy between the original calculation graph structures based on the fusion benefit values of the respective calculation subgraphs;
[0161] The computational subgraph is obtained in the following manner:
[0162] According to the way each node supports input data, the original computation graph is divided into a plurality of computation subgraphs that can be fused with on-chip memory, wherein each computation subgraph supports input data in the same way;
[0163] The acquisition module 601 is further used to acquire the hardware specification information of each computing subgraph; the hardware specification information includes: at least one of on-chip memory size, DDR memory size, bandwidth data, and NPU core number;
[0164] A copy module 606, configured to determine whether to add a node identical to the node at a location where the topological position of the nodes in the respective computing subgraphs is identical based on the hardware specification information and the input data of the respective computing subgraphs;
[0165] A preprocessing module 607, configured to fuse the single operator nodes supporting DDR fusion in the original calculation graph into one DDR node;
[0166] and / or,
[0167] Deleting nodes in the original computation graph that cannot be fused with on-chip memory;
[0168] The original calculation graph is a radar detection calculation graph, and the radar detection calculation graph is used to determine a radar detection algorithm so that the radar device performs detection based on the radar detection algorithm.
[0169] The present application also provides an electronic device, such as Figure 7 As shown, including:
[0170] Memory 701, used for storing computer programs;
[0171] The processor 702 is used to execute the program stored in the memory 701, and implements the following steps:
[0172] Acquire an original computation graph, where the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory;
[0173] Respectively evaluating a first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and a second theoretical performance value of each node with on-chip memory fusion;
[0174] A first fusion strategy within each of the computing subgraph structures is determined based on the first theoretical performance value and the second theoretical performance value.
[0175] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 702, the communication interface, and the memory 701 communicate with each other via the communication bus.
[0176] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0177] The communication interface is used for communication between the above electronic device and other devices.
[0178] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0179] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0180] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned deep learning network optimization methods are implemented.
[0181] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any deep learning network optimization method in the above embodiments.
[0182] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0183] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0184] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0185] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims
1. A deep learning network optimization method, characterized in that: The method comprises: Acquire an original computation graph, where the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory; Evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion respectively; wherein the theoretical performance value is the time consumed from input to output of each node; and the theoretical performance value is negatively correlated with the execution efficiency; Determine a node whose lower theoretical performance value between the first theoretical performance value and the second theoretical performance value satisfies a preset condition as an optimized node; A first fusion strategy for each of the computational subgraphs is determined based on nodes other than the optimized node.
2. The method according to claim 1, characterized in that The method further comprises: Traversing all the computational subgraphs, and evaluating the fusion benefit values of the fusion between the computational subgraphs; A second fusion strategy between the original computational graph structures is determined based on the fusion benefit values of the respective computational subgraphs.
3. The method according to claim 1, characterized in that The computational subgraph is obtained in the following manner: According to the way in which each node supports input data, the original computation graph is divided into a plurality of computation subgraphs that can be fused with on-chip memory, wherein each computation subgraph supports input data in the same way.
4. The method according to claim 1, characterized in that: The method further comprises: Obtaining hardware specification information of each computing subgraph; the hardware specification information includes: at least one of on-chip memory size, DDR memory size, bandwidth data, and NPU core number; Based on the hardware specification information and the input data of each computing subgraph, it is determined whether to add a node identical to the node at a location where the topological positions of the nodes in each computing subgraph are identical.
5. The method according to claim 1, characterized in that The method further comprises: Merge the single operator nodes supporting DDR fusion in the original computation graph into one DDR node; and / or, Delete nodes in the original computation graph that cannot be fused with on-chip memory.
6. The method according to any one of claims 1 to 5, characterized in that: The original calculation graph is a radar detection calculation graph, and the radar detection calculation graph is used to determine a radar detection algorithm so that the radar device performs detection based on the radar detection algorithm.
7. A deep learning network optimization system, characterized in that: The system comprises: an algorithm generation module, a radar device, wherein: The algorithm generation module is used to obtain a radar detection algorithm after optimization based on the method described in any one of claims 1 to 6; The radar device is used for performing detection based on the radar detection algorithm.
8. A deep learning network optimization device, characterized in that: The device comprises: An acquisition module, used to acquire an original computation graph, wherein the original computation graph includes a plurality of computation subgraphs that can be fused with on-chip memory; A first evaluation module is used to evaluate the first theoretical performance value of each node in each computing subgraph without on-chip memory fusion and the second theoretical performance value of each node with on-chip memory fusion; wherein the theoretical performance value is the time consumed from input to output of each node; and the theoretical performance value is negatively correlated with the execution efficiency; An optimization node determination module, used to determine a node whose lower theoretical performance value between the first theoretical performance value and the second theoretical performance value meets a preset condition as an optimized node; A fusion strategy determination module is used to determine a first fusion strategy for each of the computational subgraphs based on nodes other than the optimized node.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Adaptive node fusion compiling optimization method based on heterogeneous platform
CN112527304A
Operator fusion method and device, storage medium and electronic equipment
CN114692823A
