DNN model-based division and reasoning acceleration method and system for vehicle-mounted edge intelligence

Abstract

The invention provides a DNN model-based division and reasoning acceleration method and system for vehicle-mounted edge intelligence. The method comprises the following steps: constructing a DAG graph based on a vehicle-mounted DNN model; constructing an auxiliary network flow graph of the DAG; adding a head node representing local calculation of a vehicle and a tail node representing calculation of an edge server or an auxiliary vehicle, and setting the weight of an edge as time delay overhead for segmenting the edge; obtaining different layering modes of an auxiliary network flow diagram of the DAG and corresponding maximum tolerance point sets meeting local time delay, memory and energy consumption limitation of the vehicle; correspondingly modifying the weight of the edge in the auxiliary network flow graph of the DAG based on each maximum tolerance point set, and calculating the maximum flow of each modified network flow auxiliary graph; and selecting a division mode corresponding to the maximum flow as a DNN model division mode. According to the method, the vehicle and road cloud cooperative reasoning is realized through a DNN model division mode, so that the DNN task cooperative reasoning completion time delay is reduced, and the vehicle safety and the user experience are improved.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving computing power network and multi-dimensional computing power collaboration, and specifically relates to a DNN model partitioning and reasoning acceleration method and system for vehicle-mounted edge intelligence. Background Art

[0002] In recent years, the rise of autonomous driving and smart travel has promoted the development of Internet of Vehicles and edge computing (MEC). At the same time, the vigorous development of the field of multimodal large language models (MLLMs) and the maturity of lightweight vehicle-mounted large model technology have enabled large models to be deployed at the edge to complete more complex tasks such as intelligent voice interaction, visual scene understanding, and common sense reasoning, which has promoted the development of smart travel. However, while providing high-quality services, smart travel also brings huge challenges to network, computing resources, and storage resources. Since most tasks are computationally intensive and latency-sensitive, when vehicles process these tasks, the remote transmission latency of cloud computing is too high, and the on-board resources of vehicles are limited, which makes it difficult to meet service requirements. At this time, it is necessary to add an edge server layer between the cloud data center and the user terminal to realize the Internet of Vehicles edge computing (MEC). Users offload tasks to edge computing, thereby reducing task transmission latency, alleviating the pressure on the cloud center, and significantly improving the system response speed and user experience. However, with the gradual increase in tasks and the deployment of a large number of computing nodes in the network, the network has gradually become congested, and it has become critical to achieve the coordinated scheduling of these nodes and the flexible deployment of applications. The introduction of computing power network solves this problem. The computing power network can uniformly manage and schedule resources scattered in the network to achieve optimal resource allocation, improve resource utilization, use limited resources to complete as many tasks as possible, and improve system throughput.

[0003] To achieve autonomous driving, vehicle terminals will collect a large amount of raw image and video data. After the introduction of edge computing in the Internet of Vehicles, the vehicle will upload data and service requests to the MEC server, which will be processed by the server and returned to the vehicle in the form of instructions or services. However, the huge amount of data generated by the vehicle will be completely processed by the edge server, which will put great pressure on bandwidth, spectrum, and computing resources. Deep Neural Network (DNN) is a key technology for processing image and video data. In order to alleviate the pressure on the edge, the DNN model can be deployed on the vehicle side, and the vehicle computing resources can be used for edge-to-edge collaborative DNN distributed reasoning to alleviate the pressure on edge computing.

[0004] In the research on DNN collaborative reasoning, the use of model partitioning as a way to optimize collaborative reasoning in IoT scenarios cannot be directly applied to mobile IoV scenarios. In the research on collaborative reasoning of IoV DNN tasks, many studies only consider chained DNN structures, which are often difficult to solve when faced with DNN models with complex structures. Moreover, most studies only consider the minimization of latency in actual IoV environments, while ignoring the additional constraints of local devices. How to balance the relationship between limited computing resources, communication resources, task completion latency, and system overhead to achieve low-latency, high-reliability DNN reasoning is an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to address the shortcomings of the above-mentioned background technology and to provide a DNN model partitioning and reasoning acceleration method and system for vehicle-mounted edge intelligence, taking into account the local memory limitations and energy consumption limitations of the vehicle, and ensuring that the local resources of the vehicle allow, while minimizing the completion delay of DNN task collaborative reasoning and improving the vehicle's safety and user experience.

[0006] The technical solution adopted by the present invention is: a DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence, comprising the following steps:

[0007] Build a DAG graph based on the vehicle-mounted DNN model;

[0008] Construct an auxiliary network flow graph of the DAG graph: add a head node representing the local calculation in the vehicle and a tail node representing the calculation in the edge server or auxiliary vehicle, and set the edge weight to the delay cost of splitting the edge;

[0009] Obtain different layering methods of the auxiliary network flow graph of the DAG graph and their corresponding maximum tolerance point sets that meet the vehicle's local latency, memory and energy consumption constraints;

[0010] Based on each maximum tolerance point set, modify the weights of the edges in the auxiliary network flow graph of the DAG graph, and calculate the maximum flow of each modified network flow auxiliary graph;

[0011] The maximum flow-maximum corresponding partitioning method is selected as the DNN model partitioning method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

[0012] In the above technical solution, the vehicle’s local latency, memory and energy consumption limitations include:

[0013] The total amount of parameter data calculated by the DNN model task locally in the vehicle must not exceed the actual available memory in the vehicle.

[0014] The energy consumption generated by the local calculation of the DNN model task in the vehicle must not exceed the maximum energy consumption that can be tolerated locally;

[0015] The total delay of the DNN model task must not exceed the local maximum tolerance delay, otherwise the DNN model task is considered to have failed.

[0016] In the above technical solution, in the process of constructing the auxiliary network flow graph of the DAG graph, for each node with an out-degree greater than 1, a copy node is added; the chain nodes with in-degree and out-degree of 1 are merged into the same node; it is stipulated that for nodes with an out-degree greater than 1, after adding the copy node, only the edge from the original node to the copy node is considered, and the edge from the copy node to the next node is not considered.

[0017] In the above technical solution, in the auxiliary network flow graph of the constructed DAG graph:

[0018] The edge from the head node to any node indicates that the node is in edge computing, and the weight is the latency of the node in edge server or auxiliary vehicle computing;

[0019] The edge from the head node to the first node indicates that the entire DNN model task is placed on the edge server or auxiliary vehicle calculation;

[0020] There is no edge from the head node to the replica node;

[0021] There is no edge from the head node to the tail node;

[0022] The edge from any node to its own replica node indicates that the node is calculated locally in the vehicle, and the output data of the node will be remotely transmitted to the edge server or auxiliary vehicle as the input of the subsequent layer of the DNN model; the weight of the edge is the sum of the transmission delay of the node output data and the waiting delay on the edge server;

[0023] The edge from any node to the tail node indicates that the node is calculated locally in the vehicle, and the weight is the delay of the local calculation of the node;

[0024] The weight of the edge between two adjacent non-head and non-tail nodes is the transmission delay of the output data from the previous node to the next node;

[0025] The edge cost of copying any node to the next node is infinite.

[0026] In the above technical solution, if the DNN model is a chain structure, in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on the maximum tolerance set, the weights of all edges from the nodes to the tail nodes that are not in the maximum tolerance point set are modified to infinity.

[0027] In the above technical solution, if the DNN model is a mesh structure, in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph, the Dinic algorithm is first used to calculate the partitioning result with the theoretically optimal delay;

[0028] For this division result, compare it with the elements in all the maximum tolerance point sets of the corresponding layering method:

[0029] If the partition result is a subset of the elements in all the maximum tolerance point sets of the corresponding layering method, the partition result is directly output as the DNN model partition method;

[0030] If the optimal partition result is not a subset of the maximum tolerance set, the following process is performed:

[0031] In the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on any maximum tolerance set, the weights of all edges from the nodes to the tail nodes that are not in the maximum tolerance point set are modified to be infinite.

[0032] In the above technical solution, the process of calculating the maximum flow of the auxiliary network flow graph modified according to the maximum tolerance set includes: using the FINDAUGPATH function to find the augmented path from the head node to the tail node of the sealing mode, calculating the flow that can be transmitted by the current path, and accumulating it into the maximum flow; this process continues until no new augmented path can be found, and outputting the maximum flow finally accumulated

[0033] The present invention provides a DNN-based model partitioning and reasoning acceleration system for vehicle-mounted edge intelligence, which is used to implement the DNN-based model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence described in the above technical solution.

[0034] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the DNN-based model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence as described in the above technical solution is implemented.

[0035] The present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence as described in the above technical solution by executing the computer instructions.

[0036] The present invention provides a vehicle, comprising:

[0037] one or more processors;

[0038] A memory for storing one or more programs;

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement the DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence as described in the above technical solution.

[0040] The beneficial effects of the present invention are: in a specific Internet of Vehicles scenario, the present invention fully considers the local memory limitations and energy consumption limitations of the vehicle, and ensures that the completion delay of DNN task collaborative reasoning is reduced as much as possible under the premise that the local resources of the vehicle allow, helping the vehicle to complete DNN tasks with low latency and high accuracy that cannot be completed by local computing resources, and accelerating the DNN reasoning speed, improving the safety and user experience of the vehicle. Unlike the previous DNN partitioning that is only applicable to chain structures, the present invention can be used for DNN models with complex non-chain structures. By modeling the DNN model into a DAG graph and constructing an auxiliary graph, the model partitioning problem is transformed into a problem of solving the maximum flow, and the Dinic algorithm is optimized, and the concept of the maximum tolerance set is introduced, so that the Dinic algorithm can solve the optimal model partitioning point while considering local constraints, thereby realizing vehicle-road-cloud collaborative accelerated reasoning.

[0041] Furthermore, the present invention defines the limit standards for the latency, memory and energy consumption of local vehicle computing, ensuring that DNN tasks will not exceed the local hardware limitations of the vehicle when running on the vehicle, and can ensure resource stability. Clear memory, energy consumption and latency limits can prevent the vehicle from being overloaded due to computing tasks, ensuring the stability of the system. At the same time, the computing efficiency is improved. Under the constraints, the system can reasonably arrange the task allocation of the DNN model, so that the vehicle and the edge server can work together efficiently. And enhance the reliability of the task, ensure the safety and reliability of the DNN task running locally in the vehicle, and reduce the risk of computing task failure.

[0042] Furthermore, when constructing an auxiliary network flow graph of a DAG graph, the present invention performs special processing on the in-degree and out-degree characteristics of different nodes: for nodes with an out-degree greater than 1, a copy node is added to reduce the segmentation complexity. Chain nodes with both in-degree and out-degree of 1 are merged into a single node to simplify the graph structure. Restrictions are added to only consider the edges from the original node to the copy node, and not the edges from the copy node to the subsequent node. The present invention reduces computational complexity, reduces the number of nodes and structural complexity of the DAG graph through the merging operations of the copy node and the chain node, and simplifies the calculation of the network flow. Among them, the replication of nodes with an out-degree greater than 1 increases the segmentation options, and can be more flexibly arranged for local or edge execution. The present invention further optimizes the efficiency of network flow solution, reduces unnecessary calculation paths, makes the structure of the graph more in line with the partitioning requirements, and accelerates the maximum flow solution process.

[0043] Furthermore, the present invention specifies in detail the edge weight setting method in the auxiliary network flow graph, so as to control the direction of calculation allocation. The setting of different edge weights ensures that tasks can be reasonably allocated to vehicles or edges for execution, avoiding direct or wrong path selection. At the same time, the rationality of the calculation is ensured, and specific paths are restricted by infinite weights to ensure that the flow of the calculation tasks conforms to the calculation dependencies of the DNN layer. In addition, the present invention improves the calculation performance, and the reasonable allocation of weights can balance the delay costs of each calculation task and improve the overall calculation efficiency of the task.

[0044] Furthermore, the present invention introduces the maximum tolerance set and adjusts the weight of the edge so that the Dinic algorithm can take into account local constraints in the process of solving the optimal solution, and improves the computing efficiency: nodes that are not suitable for local execution are forced to be divided to the edge, reducing the burden on local resources. It also reduces the search complexity, reduces unnecessary weight modification steps by limiting the weight, and simplifies the maximum flow calculation of the flow graph. The present invention further ensures that the division results meet local restrictions, controls task allocation through infinite weights, reduces the pressure of local computing, and meets the actual computing needs that meet the current conditions.

[0045] Furthermore, the present invention ensures optimality of latency, and through the Dinic algorithm and the theoretical optimal partitioning method, ensures that a partitioning method that meets the optimal latency is found in a complex mesh structure. The present invention can ensure the reasonable allocation of nodes in the mesh structure by dynamically adjusting weights to control resource allocation, so that latency and local resource constraints can be balanced. The present invention further improves the reliability of the global optimal partitioning: based on stratification and tolerance sets, dynamic adjustments are made to ensure that the partition obtained is the global optimal solution.

[0046] Furthermore, the present invention improves the efficiency of maximum flow calculation. The augmented path search method is used to calculate the maximum flow of the network flow graph, which effectively reduces the solution time and improves the calculation efficiency. The present invention can ensure the accuracy of the solution, and through the augmented path, ensure that the maximum flow calculated each time meets the local computing resource constraints. The present invention also improves the versatility of the method. The FINDAUGPATH algorithm is applicable to a variety of network flow scenarios, making the method more practical and able to quickly converge to the optimal solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0048] Figure 2 It is a schematic diagram of an application scenario of the present invention;

[0049] Figure 3 It is an application scenario and a flowchart of a specific embodiment;

[0050] Figure 4 It is a schematic diagram of the DNN model structure;

[0051] Figure 5 Schematic diagram of building auxiliary flow graph for DNN model. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments to facilitate a clear understanding of the present invention, but they do not constitute a limitation on the present invention.

[0053] like Figure 1 As shown, the present invention provides a DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence, comprising the following steps:

[0054] Build a DAG graph based on the vehicle-mounted DNN model;

[0055] Construct an auxiliary network flow graph of the DAG graph: add a head node representing the local calculation in the vehicle and a tail node representing the calculation in the edge server or auxiliary vehicle, and set the edge weight to the delay cost of splitting the edge;

[0056] Obtain different layering methods of the auxiliary network flow graph of the DAG graph and their corresponding maximum tolerance point sets that meet the vehicle's local latency, memory and energy consumption constraints;

[0057] Based on each maximum tolerance point set, modify the weights of the edges in the auxiliary network flow graph of the DAG graph, and calculate the maximum flow of each modified network flow auxiliary graph;

[0058] The maximum flow-maximum corresponding partitioning method is selected as the DNN model partitioning method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

[0059] The principle of the present invention is further explained below with reference to specific embodiments.

[0060] like Figure 2As shown, the application scenario of the present invention is an intelligent driving scenario. The system architecture of the intelligent driving scenario is divided into three layers: edge layer, roadside unit and vehicle layer. The core network of the edge layer is responsible for scheduling and allocating edge nodes (i.e. edge servers or auxiliary vehicles) as a global controller to assist vehicles in completing DNN tasks. Since the DNN tasks generated by local vehicles in the autonomous driving scenario have the characteristics of large computational complexity, delay sensitivity, and high reasoning accuracy requirements, the resources of the local vehicles themselves cannot meet the requirements of such tasks at all. Therefore, the core network needs to allocate edge nodes to assist in completing DNN tasks. The edge nodes include roadside units RSU and auxiliary computing vehicles to achieve end-to-end collaborative reasoning or edge-to-end collaborative reasoning. However, since the computing resources of the vehicle itself and the edge nodes are limited, the division of the DNN model and the unloading of tasks also require additional transmission delays. How to reasonably divide the DNN model while meeting these restrictions and balance the delays of computation and transmission to accelerate collaborative reasoning is the technical problem solved by the present invention.

[0061] like Figure 3 As shown in the figure, it is assumed that all DNN tasks generated by vehicles are of the same type and that the vehicles have certain computing capabilities. It is assumed that the vehicles have obtained the edge node information for collaborative auxiliary computing.

[0062] The process of collaborative reasoning optimization of in-vehicle DNN tasks based on model partitioning is as follows: (1) Generate DNN tasks locally in the vehicle. (2) Estimate the performance of model partitioning locally in the vehicle, select the appropriate split point, and divide the DNN model into two parts: local calculation and edge node calculation. (3) According to the partitioning results, calculate some DNN layer tasks locally. (4) Upload the data of the intermediate layer to the edge node via wireless transmission. (5) The edge node uses the intermediate data as input to calculate subsequent DNN tasks. (6) The edge node returns the calculated data to the vehicle user.

[0063] Since the completion time of this type of delay-sensitive DNN task is usually in milliseconds, this specific embodiment assumes that the vehicle always moves within the service range of the edge node and will not drive out of the boundary of the edge node service range, and the channel information remains unchanged during the processing of the task, and the return information of the edge node can be received smoothly.

[0064] The purpose of this specific embodiment is to minimize the task completion delay of the DNN task model while meeting the local vehicle memory limit, local vehicle energy consumption limit and edge node energy consumption limit. The above constraints propose a DNN task model, a delay model, a memory limit model, an energy consumption model, and propose an optimization goal.

[0065] In the DNN task model, the DNN task model mainly includes convolution layer I CL and fully connected layer I FCLThe amount of data input to the i-th layer can be expressed as The amount of computation required can be expressed in floating point operations per second (FLOPs). i express.

[0066] In the latency model, the latency of DNN collaborative reasoning includes local computing latency, transmission latency, waiting latency, edge computing latency, and computing result return latency. The local computing latency can be expressed as T local , the edge calculation delay can be expressed as T edge , the waiting delay is T wait , because in the cross-cutting model cutting mode, the input data is more and the output data is less, so the uplink transmission delay can be expressed as T trans , and the downlink return delay can be ignored. So far, the total delay of the DNN task collaborative reasoning process can be expressed as T total =T local +T trans +T wait +T edge The delay of DNN task completion must be less than the maximum tolerable delay that the vehicle user can accept, otherwise it is considered that the DNN task reasoning has failed, that is:

[0067]

[0068] In the memory-limited model, the total number of parameters of the DNN task model calculated locally must not exceed the actual available memory of the vehicle, that is:

[0069]

[0070] Where V local represents the set of DNN layers calculated locally after DNN partitioning, M local Indicates the maximum available memory in the vehicle. Dynamic load on the vehicle's local memory.

[0071] In the energy consumption model, the energy consumption of collaborative reasoning mainly consists of two parts: computing energy consumption and transmission energy consumption. The transmission energy consumption can be expressed as E trans , the energy consumption of local computation of DNN tasks can be expressed as E local , the energy consumption generated by the vehicle terminal when performing autonomous driving tasks shall not exceed the maximum energy consumption that can be tolerated by the local or edge end, that is:

[0072]

[0073] The optimization goal of the present invention is to select appropriate DNN model partitioning points to minimize the task completion delay while satisfying the local vehicle memory limit, local vehicle energy consumption limit and edge node energy consumption limit. The problem is stated as:

[0074] minT total

[0075] C1:

[0076] C2:

[0077] C3:

[0078] C1 indicates that the total amount of parameter data calculated by the DNN task in the local vehicle must not exceed the actual available memory of the local vehicle. C2 indicates that the energy consumption generated by the DNN task in the local vehicle must not exceed the maximum energy consumption that can be tolerated locally. C3 indicates that the total delay of task reasoning must not exceed the maximum tolerated delay of the local vehicle, otherwise the reasoning task will be considered as failed.

[0079] Based on the above optimization objectives and constraints, this specific embodiment transforms the DNN model partitioning problem under the constraints into a minimum cut problem for solving the DAG flow graph.

[0080] In the first step, each layer in the DNN model can be modeled as a directed acyclic graph (DAG) G=<V,E> . Define the node set V = {v1,v2,...,v n} represents each layer of the DNN task, E is the set of all edges, (v i ,v i+1 )∈E means there is an output v i , the input is v i+1 Each edge represents the output of the previous node as the input of the next node. The weight of the edge represents the cost of splitting the edge, that is, the latency overhead. The nodes in the DAG are divided into two parts after reasonable division: All nodes in the subgraph containing the input nodes are executed locally in the vehicle. The set of these nodes is defined as V local , all nodes in the subgraph containing the output node are calculated on the edge server or auxiliary vehicle, and the set of these nodes is defined as V edge , and V local ∪V edge =V.

[0081] The edge passed by the dividing line indicates that the output data of the previous node will be uploaded to the edge through computation offloading for the second half of the reasoning. The calculation result of the output node is then sent back to the vehicle user through the wireless link to complete the DNN collaborative reasoning task. It should be noted that if the partition node is before the input node, it means that all tasks are offloaded to the edge node for execution, and the vehicle does not perform any calculations locally, that is, At this time, all input data needs to be transferred to the edge node. If the partition node is after the output node, it means that the task is all executed locally by the vehicle, and the edge server does not need to participate.

[0082] The second step is to construct the auxiliary network flow graph of each layer of the directed acyclic graph in the DNN model to obtain the auxiliary network flow graph G′.

[0083] The problem of solving the minimum cost DAG graph segmentation is usually solved by constructing an auxiliary network flow graph and then converting it into a st-cut problem. This specific embodiment takes a simple nonlinear DAG containing four nodes as an example. Figure 4 As shown. The constructed auxiliary network flow graph G′ is as follows Figure 5 The construction steps are as follows:

[0084] In the first step, add the head node S and the tail node T, which represent the calculation on the local device and the calculation on the edge node respectively. In addition, for each node with an out-degree greater than 1, add a replica node v i The significance of adding a replica node is that a node with an out-degree greater than 1 indicates that its output data is used as input data for multiple points. During the collaborative computing of DNN tasks, the data is only transmitted once in wireless transmission and is used as input for multiple DNN task layers after reaching the edge node.

[0085] The second step is to add the weight of each edge. The weight of each edge represents the cost consumed when the dividing line cuts this edge. In the DNN model partitioning problem, it is specifically expressed as time overhead. The local computing delay, transmission delay and edge computing delay between each layer of the DNN are calculated as the weight of the edge.

[0086] Specifically, local computing latency refers to the time required to compute a certain layer on the local device of the vehicle. This latency is related to the computational complexity of the layer and the use of computing resources.

[0087] Transmission latency refers to the time it takes to transmit the output data of a certain layer to an edge computing device (such as an edge server or auxiliary vehicle). This latency depends on the data size and network bandwidth.

[0088] Edge computing latency refers to the time required to perform a certain layer of calculation on an edge computing device. This also depends on the processing power of the edge device and the complexity of the layer.

[0089] The meaning of each edge is as follows:

[0090] (1) Head node S to v i The edge represents the node v iIn edge computing, the weight is the latency of the node in edge computing. In particular, the edge from the head node S to the first node v1 means that the entire task is placed on the edge computing. There is no edge from the head node S to the replica node v i ′, because the edge from S to the original node v i The meanings are the same. There is no edge from the head node S to the tail node T.

[0091] (2) From node v i To its own replication node v i ′, indicating the node v i Compute locally, v i The output data of the node will be transmitted remotely to the edge server as the input of the subsequent layer. The weight of this edge is v i The sum of the transmission delay of the output data and the waiting delay at the edge server. When a node needs to be calculated locally, the edge from the node to the tail node needs to be cut to obtain the local calculation delay of the node.

[0092] (3)v i To the tail node T edge, indicating v i In local calculation, the weight is the latency of the node's local calculation. When a node needs to be calculated at the edge, cut the edge from the node to the head node to get the latency of the node's edge calculation.

[0093] (4) Copy node v i 'To the next node v i+1 The edge cost is infinite. This is because the edge represents a hybrid cut method, which places multiple subsequent nodes of the same layer on the local layer and the edge layer for execution. (5) From node v i Go to the next node v i+1 The weight of the edge is v i The transmission delay required for the output data to be transmitted to the edge. When it is necessary to divide between two adjacent nodes, the edge between the two nodes needs to be cut to obtain the transmission delay of the intermediate data.

[0094] After obtaining the auxiliary network flow graph F′, the model partitioning problem of minimizing the delay can be transformed into the st minimum cut problem: divide the auxiliary network flow graph F′ into two non-intersecting subsets V s ,V t , the head node S and the tail node T are in different subsets, and the total cut cost is the smallest. All model partitioning methods can be mapped to the cutting method in the DAG graph, and the minimum cut can be transformed into the maximum flow problem solution.

[0095] The partitioning of DAG flow graph has the following theorem: Assume that at node v i The previous division can reduce the overall delay. If v iThe in-degree is 1, and the out-degree is greater than or equal to 1, then in v i The performance of the previous partition is better than that of the i Then divide it. That is, the node is more suitable for computing on the edge node, because the computing power of the edge node is much greater than that of the local device.

[0096] Therefore, in order to simplify the model, before building the DAG flow graph, the chain nodes with in-degree and out-degree of 1 are merged into the same node. From the above theorem, it can be seen that such simplification will not affect the optimal result. It is also stipulated that for nodes with out-degree greater than 1, after adding the copy node, only the edge from the original node to the copy node is considered, and the edge from the copy node to the next node is not considered, that is, only cross-cutting without mixed cutting. For nodes with in-degree greater than 1, the transmission delay is calculated. Since the input data can be transmitted in parallel, direct addition will calculate unnecessary time. Therefore, for this type of node, the transmission delay needs to be redefined:

[0097]

[0098] in, represents the i-th transmission delay, represents the edge computing latency of the jth point, where j represents the node in the edge computing in the parallel path, and k represents the node in the local computing in the parallel path.

[0099] The third step is to solve the minimum cut problem of the DNN model under restricted conditions through the Dinic algorithm.

[0100] The Dinic algorithm is an algorithm for calculating the maximum flow in a network flow. The idea of ​​the algorithm is to first divide all nodes in the auxiliary network flow graph G′ into layers. The rule of layering is that the head node S is in the 0th layer, the tail node T is in the last layer, and the directed edges in the middle layer indicate that the nodes in the previous layer can reach the nodes in the next layer. The same auxiliary network flow graph G′ may have multiple layering labeling methods. Using the idea of ​​breadth-first traversal, the auxiliary network flow graph G′ is layered in turn to obtain several layering methods. In each layer, the maximum flow is obtained by continuously augmenting. The basic Dinic algorithm can only obtain the optimal partitioning method. Such a partitioning may cause the local memory or energy consumption to exceed the maximum tolerance limit. In the actual vehicle network autonomous driving scenario, such a partitioning may cause the local memory or energy consumption to exceed the maximum tolerance limit, and the Dinic algorithm cannot quickly give a suboptimal partitioning method.

[0101] The present invention proposes a DNN task model partitioning method based on the Dinic algorithm under constraints. The specific ideas are as follows:

[0102] When the auxiliary network flow graph G′ is layered, the layering method is put into a set S divThen, the energy consumption and memory of this layered method are calculated to obtain the corresponding maximum tolerance point set S that meets the local delay, memory and energy consumption constraints of the vehicle. max In chained DNN task models such as VGG16, S max Since the energy consumption of chain tasks in local computing is cumulative, there must be a unique dividing point, so that S max In mesh DNN models such as inceptionV3, there may be multiple maximum tolerance point sets S max .

[0103] For the chain DNN structure: the chain structure has a unique layering method, S max It is also unique. After obtaining the maximum tolerance set S that meets the energy consumption and memory constraints max After that, the processing method for the auxiliary network flow graph G′ is to modify all max The weight of the edge from the node to the tail node is infinite, because the energy consumption of local computing of these nodes is unacceptable to the local device, so by changing the local computing delay to infinite, these DNN layers are unloaded to the edge node for computing. After modifying the weight of the edge of the auxiliary network flow graph G′, the depth-first search method is used to calculate the maximum flow of the auxiliary network flow graph G′.

[0104] The segmentation method corresponding to the maximum flow is the optimal DNN model partitioning method. The partitioning method obtained at this time must meet the local energy consumption and memory constraints of the vehicle, because the partitioning method that does not meet the constraints has been avoided in advance.

[0105] For the mesh DNN structure: First, for each layer, the maximum tolerance set S that meets the energy consumption and memory constraints is obtained. max Finally, the Dinic algorithm is used to calculate the optimal delay partition result in theory. The optimal partition result is unique, which is the maximum value of the maximum flow corresponding to all layering methods.

[0106] For this division result, compare it with all S of the corresponding hierarchical method max The elements in are compared separately:

[0107] If the division result is the above S max It is a subset of an element in , indicating that the theoretically optimal partitioning method is allowed by the local energy consumption limit, so this partitioning method is the global optimal partitioning method, and the partitioning method is directly output as the optimal partitioning method of the DNN model.

[0108] If the division result is not the above S max If there is no subset of any element in , it means that this partitioning method exceeds the local limit and cannot be output as the optimal partitioning method. At this time, the following process needs to be performed:

[0109] For all S corresponding to all stratification methods max Any maximum tolerance set S max , modify the weights of all nodes to the tail node that are not in the maximum tolerance point set of the delay, memory and energy consumption constraints of the corresponding vehicle to be infinite; after modifying the weights of the edges of the auxiliary network flow graph G′, use the depth-first search method to calculate the maximum flow of the auxiliary network flow graph G′ modified in this way;

[0110] The division method corresponding to the maximum flow among all calculation results is selected as the DNN model division method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

[0111] At this point, it should be noted that if a partitioning method that meets the conditions is found, that is, according to a certain S max The optimal partitioning method of the DAG with the maximum tolerance point set in the modified weight is only the optimal solution under the maximum tolerance node set in this case, not the global optimal solution. It is necessary to continue traversing S max All elements, and then find the best way to divide them.

[0112] The model partitioning problem under the constraint condition is: divide the auxiliary network flow graph G′ into two non-intersecting subsets V s ,V t , the head node S and the tail node T are in different subsets, V s The sum of the node costs in must not exceed the maximum cost limit, so that the cutting weight is minimized.

[0113] Summarizing the above steps, in this specific embodiment, the algorithm solution process for the minimum cut of the auxiliary network flow graph G′ is as follows:

[0114] 1. Initialize variables:

[0115] Initialize the collection of storage layering, energy consumption, and memory limits:

[0116] S div : A collection of storage hierarchical methods used to record all possible partitioning schemes.

[0117] S max : Stores the maximum set of points that can be tolerated under local constraints in a hierarchical manner.

[0118] 2. Initialize the node hierarchy:

[0119] The node level array, initially empty.

[0120] 3. Use the MARKLEVEL function to mark the loop in layers:

[0121] MARKLEVEL is a hierarchical marking algorithm based on breadth-first search (BFS). Its purpose is to mark the level of each node to ensure that the subsequent augmenting path search is only performed from low to high levels. The level array records the level value of each node.

[0122] Cycle process:

[0123] Each call to MARKLEVEL may generate a new layering method (i.e., a new level marking of the nodes of the graph G′).

[0124] If MARKLEVEL succeeds (i.e. there is a feasible path from the source node src to the target node dst), add this hierarchical method to the set S div

[0125] Loop termination condition:

[0126] When MARKLEVEL returns False, it means that there are no more augmenting paths to find and the loop ends.

[0127] After each loop:

[0128] Compute the set of all nodes that can be executed locally under local constraints and add them to S max for subsequent decision making.

[0129] 4. Perform the maximum flow loop and augmenting path loop:

[0130] Based on each maximum tolerance set, the weights of the edges in the auxiliary network flow graph of the DAG graph are modified accordingly: the local execution delay of the nodes (tasks) that are not in the maximum tolerance set is set to infinity (∞), which means that these tasks must be executed on the edge devices.

[0131] Initialize flow variables: maxflow←0: The maximum flow is 0. curflow←INF: The current flow is set to infinity, indicating that there is no constraint initially.

[0132] For each modified auxiliary network flow graph, the augmenting path algorithm FINDAUGPATH is called to find the augmenting path starting from the source node src, update curflow, and add the found augmented flow to maxflow.

[0133] Loop termination condition:

[0134] When no augmenting path can be found, or curflow is 0, it means that the maximum flow maxflow under the current auxiliary network flow graph has been determined.

[0135] 5. Record the optimal partitioning scheme:

[0136] Select the partitioning method with the largest maximum flow as the optimal hierarchical partitioning scheme and obtain the minimum cut set V local ,V edge :

[0137] V local : A collection of tasks that are executed on the local device.

[0138] V edge : A collection of tasks executed on edge devices.

[0139] The time complexity of the Dinic algorithm designed in this specific embodiment is O(|V| 2 |E|).

[0140] The pseudo code of this specific embodiment is as follows:

[0141] Input: Parameters of each layer of the DNN task FLOPs i , uplink transmission rate R, maximum memory limit Maximum power consumption limit

[0142] Output: Task segmentation result V local ,V edge

[0143] Build the DAG graph G(V,E) of the DNN model

[0144] Calculate the local computing delay T of each layer local,i Transmission delay T trans,i , edge computing latency T edge,i

[0145] Construct DAG extended auxiliary network flow graph G′(V,E)

[0146] Initialize storage layer mode S div , the maximum tolerable point set S under local energy consumption and memory constraints max ,

[0147] Initialize node level level←DEAFULTED(int)

[0148] while MARKLEVEL(G,src,dst,level)do

[0149] Add this stratification level to the set S div

[0150] Calculate the maximum energy consumption tolerated locally, the set of maximum memory points, and add S max

[0151] end while

[0152]

[0153] for eachs max,j ∈S max do

[0154] Modify the T of the corresponding node local,i ←∞

[0155] Initialize maxflow←0,curflow←INF

[0156] for eachs div,k ∈S div do

[0157] maxflow←maxflow+NDAUGPATH(src,curflow)

[0158] end for

[0159]

[0160] end for

[0161] according to Get the minimum cut set V local ,V edge

[0162] returnV local ,V edge

[0163] The pseudo code of the MARKLEVEL breadth-first traversal algorithm for G(V,E) is as follows:

[0164] Input: graph G(V,E), head node src, tail node dst, array level of record levels

[0165] Output: Is there a new labeling method, and record it in level

[0166]

[0167]

[0168] The pseudo code of the FINDAUGPATH() depth-first search augmenting path algorithm is as follows:

[0169] Input: current node v u , the current remaining flow restflow

[0170] Output: New flow

[0171]

[0172] In order to analyze the performance of the DNN model partitioning algorithm under the restriction conditions proposed in this specific embodiment, Edge-Only (edge ​​computing only), End-Only (local computing only), Neur algorithm, DADs and Min-Energy algorithm are selected as comparison algorithms. YOLOV2 and VGG16 are used as representatives of chain DNN tasks, and AlexNet and InceptionV3 are used as representatives of DAG DNN tasks to evaluate the algorithm performance.

[0173] The present invention selects acceleration performance and throughput improvement as performance indicators for evaluating algorithms. Acceleration performance indicates the multiple of the acceleration of the inference speed, and throughput improvement indicates the multiple of the increase in the number of requests processed by the system per unit time.

[0174] Experimental results show that compared with local computing and edge computing, the DNN task reasoning acceleration strategy of model partitioning can reduce the total reasoning latency, speed up reasoning and improve system throughput. For chained DNN tasks, MPAUC has similar performance to classic Neur and DADs, but for DAG-structured DNN models, MPAUC can get reasoning results faster, and in specific scenarios with memory and energy consumption constraints, the MPAUC algorithm can solve the latency optimization problem under local device constraints, which is something Neur and DADs cannot do.

[0175] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence, characterized by: The following steps are involved: Build a DAG graph based on the vehicle-mounted DNN model; Construct an auxiliary network flow graph of the DAG graph: add a head node representing the local calculation in the vehicle and a tail node representing the calculation in the edge server or auxiliary vehicle, and set the edge weight to the delay cost of splitting the edge; Obtain different layering methods of the auxiliary network flow graph of the DAG graph and their corresponding maximum tolerance point sets that meet the vehicle's local latency, memory and energy consumption constraints; Based on each maximum tolerance point set, modify the weights of the edges in the auxiliary network flow graph of the DAG graph, and calculate the maximum flow of each modified network flow auxiliary graph; The maximum flow-maximum corresponding division method is selected as the DNN model division method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

2. The DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence according to claim 1 is characterized in that: The vehicle’s local latency, memory, and energy constraints include: The total amount of parameter data calculated locally by the DNN model task must not exceed the actual available memory in the vehicle. The energy consumption generated by the local calculation of the DNN model task in the vehicle must not exceed the maximum energy consumption that can be tolerated locally; The total delay of the DNN model task must not exceed the local maximum tolerance delay, otherwise the DNN model task is considered to have failed.

3. The DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence according to claim 1 is characterized in that: In the process of constructing the auxiliary network flow graph of the DAG graph, for each node with an out-degree greater than 1, add a copy node; merge the chain nodes with in-degree and out-degree of 1 into the same node; It is stipulated that for nodes with out-degree greater than 1, after adding the copy node, only the edge from the original node to the copy node is considered for division, and the edge from the copy node to the next node is not considered for division.

4. The DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence according to claim 3 is characterized in that: In the auxiliary network flow diagram of the constructed DAG graph: The edge from the head node to any node indicates that the node is in edge computing, and the weight is the latency of the node in edge server or auxiliary vehicle computing; The edge from the head node to the first node indicates that the entire DNN model task is placed on the edge server or auxiliary vehicle calculation; There is no edge from the head node to the replica node; There is no edge from the head node to the tail node; The edge from any node to its own replica node indicates that the node is calculated locally in the vehicle, and the output data of the node will be remotely transmitted to the edge server or auxiliary vehicle as the input of the subsequent layer of the DNN model; the weight of the edge is the sum of the transmission delay of the node output data and the waiting delay on the edge server; The edge from any node to the tail node indicates that the node is calculated locally in the vehicle, and the weight is the delay of the local calculation of the node; The weight of the edge between two adjacent non-head and non-tail nodes is the transmission delay of the output data from the previous node to the next node; The edge cost of copying any node to the next node is infinite.

5. The DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence according to claim 1 is characterized in that: If the DNN model is a chain structure, in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on the maximum tolerance set, the weights of all edges from the nodes that are not in the maximum tolerance point set to the tail node are modified to be infinite.

6. The DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence according to claim 1 is characterized in that: If the DNN model is a mesh structure, in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph, the Dinic algorithm is first used to calculate the partitioning result with the theoretically optimal delay; For this division result, compare it with the elements in all the maximum tolerance point sets of the corresponding layering method: If the partition result is a subset of the elements in all the maximum tolerance point sets of the corresponding layering method, the partition result is directly output as the DNN model partition method; If the optimal partition result is not a subset of the maximum tolerance set, the following process is performed: In the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on any maximum tolerance set, the weights of all edges from the nodes to the tail nodes that are not in the maximum tolerance point set are modified to be infinite.

7. The DNN-based model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence according to claim 5 or 6, characterized in that: The process of calculating the maximum flow of the auxiliary network flow graph modified according to the maximum tolerance set includes: using the FINDAUGPATH function to find the augmenting path from the head node to the tail node of the sealing method, calculating the flow that can be transmitted by the current path, and accumulating it into the maximum flow; this process continues until no new augmenting path can be found, and outputs the final accumulated maximum flow.

8. A DNN model partitioning and reasoning acceleration system for vehicle-mounted edge intelligence, characterized by: Used to implement the DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence as described in any one of claims 1-7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence as described in any one of claims 1 to 7 by executing the computer instructions.

Images