A DNN model partitioning and reasoning acceleration method and system for in-vehicle edge intelligence

Abstract

The present invention provides a DNN model partitioning and reasoning acceleration method and system for vehicle-mounted edge intelligence, the method comprising the following steps: constructing a DAG graph based on the vehicle-mounted DNN model; constructing an auxiliary network flow graph of the DAG graph: adding a head node representing local calculations in the vehicle and a tail node representing calculations in the edge server or auxiliary vehicle, and setting the weight of the edge to the delay overhead of splitting the edge; obtaining different layering methods of the auxiliary network flow graph of the DAG graph and its corresponding maximum tolerance point set that meets the vehicle's local delay, memory, and energy consumption constraints; modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on each maximum tolerance point set, and calculating the maximum flow of each modified network flow auxiliary graph; selecting the partitioning method corresponding to the maximum flow as the DNN model partitioning method. The present invention implements vehicle-road-cloud collaborative reasoning through the DNN model partitioning method to reduce the completion delay of DNN task collaborative reasoning and improve vehicle safety and user experience.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving computing power networks and multi-dimensional computing power collaboration, and specifically relates to a DNN model partitioning and inference acceleration method and system for on-board edge intelligence. Background Art

[0002] In recent years, the rise of autonomous driving and smart mobility has driven the development of connected vehicles (IoVs) and edge computing (MEC). The booming field of multimodal large language models (MLLMs) and the maturity of lightweight onboard large model technologies have enabled large models to be deployed at the edge to accomplish more complex tasks, such as intelligent voice interaction, visual scene understanding, and commonsense reasoning, driving the development of IoV. However, while providing high-quality services, IoV also poses significant challenges to network, computing, and storage resources. Because most tasks are computationally intensive and latency-sensitive, when vehicles process these tasks, the high transmission latency of cloud computing and the limited onboard resources make it difficult to meet service requirements. Therefore, it is necessary to add an edge server layer between cloud data centers and user terminals to implement IoV edge computing (MEC). Users offload tasks to edge computing, reducing transmission latency, alleviating pressure on cloud centers, and significantly improving system responsiveness and user experience. However, with the increasing number of tasks and the deployment of a large number of computing nodes in the network, the network is becoming increasingly congested. Coordinating these nodes and flexibly deploying applications are key. The introduction of computing power network solves this problem. The computing power network can uniformly manage and schedule resources scattered in the network to achieve optimal resource allocation, improve resource utilization, use limited resources to complete as many tasks as possible, and improve system throughput.

[0003] To achieve autonomous driving, vehicle terminals collect large amounts of raw image and video data. With the introduction of edge computing (MEC) in the connected vehicle (IoV), vehicles upload this data and service requests to MEC servers, which process the data and return it to the vehicle as instructions or services. However, relying solely on edge servers to process the massive amounts of data generated by vehicles places significant pressure on bandwidth, spectrum, and computing resources. Deep Neural Networks (DNNs) are a key technology for processing image and video data. To alleviate this pressure on the edge, DNN models can be deployed on the vehicle side, leveraging vehicle computing resources for edge-to-edge collaborative DNN distributed inference, alleviating edge computing pressure.

[0004] However, in the research on DNN collaborative reasoning, the use of model partitioning as a collaborative reasoning optimization method in IoT scenarios is not directly applicable to mobile IoV scenarios. Many studies on collaborative reasoning of IoV DNN tasks have only considered chained DNN structures, which often makes it difficult to solve complex DNN models. Furthermore, most studies in actual IoV environments only consider latency minimization, while ignoring the additional constraints of local devices. Balancing limited computing and communication resources with task completion latency and system overhead to achieve low-latency, high-reliability DNN reasoning is an urgent issue. Summary of the Invention

[0005] The purpose of the present invention is to address the shortcomings of the above-mentioned background technology and provide a DNN model partitioning and reasoning acceleration method and system for vehicle-mounted edge intelligence. Taking into account the local memory limitations and energy consumption limitations of the vehicle, and ensuring that the local resources of the vehicle allow, the completion delay of DNN task collaborative reasoning is reduced as much as possible, thereby improving the safety of the vehicle and user experience.

[0006] The technical solution adopted by the present invention is: a DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence, comprising the following steps:

[0007] Build a DAG graph based on the vehicle's DNN model;

[0008] Construct an auxiliary network flow graph for the DAG graph: add a head node representing the local calculation on the vehicle and a tail node representing the calculation on the edge server or auxiliary vehicle, and set the edge weight to the latency cost of splitting the edge;

[0009] Obtain different layering methods of the auxiliary network flow graph of the DAG graph and its corresponding maximum tolerance point set that meets the vehicle's local latency, memory and energy consumption constraints;

[0010] Based on each maximum tolerance point set, modify the weights of the edges in the auxiliary network flow graph of the DAG graph, and calculate the maximum flow of each modified network flow auxiliary graph;

[0011] The maximum flow-maximum corresponding partitioning method is selected as the DNN model partitioning method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

[0012] In the above technical solution, the vehicle's local latency, memory, and energy consumption constraints include:

[0013] The total amount of parameter data calculated locally on the vehicle for DNN model tasks must not exceed the actual available memory on the vehicle.

[0014] The energy consumption generated by the local calculation of the DNN model task in the vehicle must not exceed the maximum energy consumption that can be tolerated locally;

[0015] The total delay of the DNN model task must not exceed the local maximum tolerance delay, otherwise the DNN model task is considered failed.

[0016] In the above technical solution, in the process of constructing the auxiliary network flow graph of the DAG graph, for each node with an out-degree greater than 1, a copy node is added; the chain nodes with in-degree and out-degree of 1 are merged into the same node; it is stipulated that for nodes with an out-degree greater than 1, after adding the copy node, only the edge from the original node to the copy node is considered, and the edge from the copy node to the next node is not considered.

[0017] In the above technical solution, the auxiliary network flow diagram of the constructed DAG graph is:

[0018] The edge from the head node to any node indicates that the node is in edge computing, and the weight is the latency of the node in the edge server or auxiliary vehicle computing;

[0019] The edge from the head node to the first node indicates that the entire DNN model task is placed on the edge server or auxiliary vehicle computing;

[0020] There is no edge from the head node to the replica node;

[0021] There is no edge from the head node to the tail node;

[0022] An edge from any node to its own replica node indicates that the node is performing computation locally on the vehicle. The output data of this node will be remotely transmitted to an edge server or auxiliary vehicle as input to subsequent layers of the DNN model. The weight of this edge is the sum of the transmission delay of the node's output data and the waiting delay on the edge server.

[0023] The edge from any node to the tail node indicates that the node is calculated locally on the vehicle, and the weight is the delay of the local calculation of the node;

[0024] The weight of the edge between two adjacent non-head and non-tail nodes is the transmission delay of the output data from the previous node to the next node;

[0025] The cost of copying an edge from any node to the next node is infinite.

[0026] In the above technical solution, if the DNN model is a chain structure, in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on the maximum tolerance set, the weights of all edges from the nodes to the tail nodes that are not in the maximum tolerance point set are modified to infinity.

[0027] In the above technical solution, if the DNN model has a mesh structure, then in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph, the Dinic algorithm is first used to calculate the partitioning result with the theoretically optimal delay;

[0028] For this division result, compare it with the elements in all the maximum tolerance point sets of the corresponding hierarchical method:

[0029] If the partition result is a subset of the elements in all the maximum tolerance point sets of the corresponding layered method, the partition result is directly output as the DNN model partition method;

[0030] If the optimal partition result is not a subset of the maximum tolerance set, the following process is performed:

[0031] In the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on any maximum tolerance set, the weights of all edges from the nodes that are not in the maximum tolerance point set to the tail node are modified to be infinite.

[0032] In the above technical solution, the process of calculating the maximum flow of the auxiliary network flow graph modified according to the maximum tolerance set includes: using the FINDAUGPATH function to find the augmenting path from the head node to the tail node of the sealing method, calculating the flow that can be transmitted by the current path, and accumulating it into the maximum flow; this process continues until no new augmenting path can be found, and outputting the maximum flow finally accumulated.

[0033] The present invention provides a DNN model partitioning and reasoning acceleration system for in-vehicle edge intelligence, which is used to implement the DNN model partitioning and reasoning acceleration method for in-vehicle edge intelligence described in the above technical solution.

[0034] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for DNN-based model partitioning and inference acceleration for in-vehicle edge intelligence as described in the above technical solution is implemented.

[0035] The present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the DNN model partitioning and inference acceleration method for in-vehicle edge intelligence as described in the above technical solution.

[0036] The present invention provides a vehicle, comprising:

[0037] one or more processors;

[0038] a memory for storing one or more programs;

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement the DNN model partitioning and reasoning acceleration method for in-vehicle edge intelligence as described in the above technical solution.

[0040] The beneficial effects of the present invention are: in a specific Internet of Vehicles scenario, the present invention fully considers the local memory limitations and energy consumption limitations of the vehicle, and ensures that the completion delay of DNN task collaborative reasoning is reduced as much as possible under the premise that the local resources of the vehicle allow, helping the vehicle to complete DNN tasks with low latency and high accuracy that cannot be completed by local computing resources, and accelerating the DNN reasoning speed, improving the safety and user experience of the vehicle. Unlike the previous DNN partitioning that is only applicable to chain structures, the present invention can be used for DNN models with complex non-chain structures. By modeling the DNN model into a DAG graph and constructing an auxiliary graph, the model partitioning problem is converted into a problem of solving the maximum flow, and the Dinic algorithm is optimized, and the concept of the maximum tolerance set is introduced, so that the Dinic algorithm can solve the optimal model partitioning point while considering local constraints, thereby realizing vehicle-road-cloud collaborative accelerated reasoning.

[0041] Furthermore, the present invention defines limits on latency, memory, and energy consumption for local vehicle computing, ensuring that DNN tasks do not exceed the vehicle's local hardware limitations when running on the vehicle, thus guaranteeing resource stability. Clear limits on memory, energy consumption, and latency prevent the vehicle from being overloaded by computing tasks, ensuring system stability. This also improves computing efficiency. Under restricted conditions, the system can rationally arrange the task allocation of the DNN model, enabling the vehicle and edge server to work together efficiently. Furthermore, task reliability is enhanced, ensuring the safety and reliability of DNN tasks running locally in the vehicle and reducing the risk of computing task failure.

[0042] Furthermore, when constructing an auxiliary network flow graph of a DAG graph, the present invention performs special processing on the in-degree and out-degree characteristics of different nodes: for nodes with an out-degree greater than 1, a copy node is added to reduce the complexity of segmentation. Chain nodes with both in-degree and out-degree of 1 are merged into a single node to simplify the graph structure. Restrictions are added to only consider edges from the original node to the copy node, while edges from the copy node to subsequent nodes are not considered. The present invention reduces computational complexity, reduces the number of nodes and structural complexity of the DAG graph through the merging operations of copy nodes and chain nodes, and simplifies the calculation of network flows. Among them, the replication of nodes with an out-degree greater than 1 increases the splitting options, which can be more flexibly arranged for local or edge execution. The present invention further optimizes the efficiency of network flow solution, reduces unnecessary calculation paths, makes the structure of the graph more in line with the partitioning requirements, and accelerates the maximum flow solution process.

[0043] Furthermore, the present invention specifies in detail a method for setting edge weights in the auxiliary network flow graph, thereby controlling the direction of computational allocation. The setting of different edge weights ensures that tasks can be reasonably allocated to vehicles or edges for execution, avoiding direct or incorrect path selection. At the same time, computational rationality is ensured by limiting specific paths with infinite weights, ensuring that the flow of computational tasks conforms to the computational dependencies of the DNN layer. Furthermore, the present invention improves computational performance, and the reasonable allocation of weights can balance the latency costs of each computational task, improving the overall computational efficiency of the task.

[0044] Furthermore, the present invention introduces a maximum tolerance set and adjusts the weights of edges, allowing the Dinic algorithm to take local constraints into account when seeking the optimal solution, thereby improving computational efficiency. Nodes unsuitable for local execution are forcibly partitioned to the edges, reducing the burden on local resources. It also reduces search complexity, reduces unnecessary weight modification steps by limiting weights, and simplifies the maximum flow calculation of the flow graph. The present invention further ensures that the partitioning results comply with local constraints, controls task allocation through infinite weights, reduces local computational pressure, and meets the actual computational requirements that meet current conditions.

[0045] Furthermore, the present invention ensures optimal latency. Using the Dinic algorithm and theoretically optimal partitioning, it ensures that a latency-optimal partitioning solution is found within a complex mesh structure. By dynamically adjusting weights to control resource allocation, the present invention ensures the proper allocation of nodes within the mesh structure, balancing latency with local resource constraints. The present invention further improves the reliability of the globally optimal partitioning by dynamically adjusting the partitioning based on layering and tolerance sets to ensure that the resulting partitioning solution is the globally optimal solution.

[0046] Furthermore, the present invention improves the efficiency of maximum flow calculations. Using an augmenting path search method to calculate the maximum flow of a network flow graph effectively reduces solution time and improves computational efficiency. The present invention ensures solution accuracy by augmenting the path, ensuring that the maximum flow calculated each time complies with local computing resource constraints. The present invention also improves the versatility of the method, making the FINDAUGPATH algorithm applicable to a variety of network flow scenarios, making the method more practical and enabling rapid convergence to the optimal solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the method flow of the present invention;

[0048] Figure 2 Schematic diagram of an application scenario of the present invention;

[0049] Figure 3 Schematic diagram of application scenarios and flow charts of specific embodiments;

[0050] Figure 4 Schematic diagram of the DNN model structure;

[0051] Figure 5 Schematic diagram of building auxiliary flow graph for DNN model. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments to facilitate a clear understanding of the present invention, but they do not constitute a limitation to the present invention.

[0053] like Figure 1 As shown, the present invention provides a DNN model partitioning and reasoning acceleration method for vehicle-mounted edge intelligence, comprising the following steps:

[0054] Build a DAG graph based on the vehicle's DNN model;

[0055] Construct an auxiliary network flow graph for the DAG graph: add a head node representing the local calculation on the vehicle and a tail node representing the calculation on the edge server or auxiliary vehicle, and set the edge weight to the latency cost of splitting the edge;

[0056] Obtain different layering methods of the auxiliary network flow graph of the DAG graph and its corresponding maximum tolerance point set that meets the vehicle's local latency, memory and energy consumption constraints;

[0057] Based on each maximum tolerance point set, modify the weights of the edges in the auxiliary network flow graph of the DAG graph and calculate the maximum flow of each modified network flow auxiliary graph;

[0058] The maximum flow-maximum corresponding partitioning method is selected as the DNN model partitioning method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

[0059] The principle of the present invention is further explained below with reference to specific embodiments.

[0060] like Figure 2As shown, the application scenario of the present invention is an intelligent driving scenario. The system architecture of the intelligent driving scenario is divided into three layers: edge layer, roadside unit and vehicle layer. The core network of the edge layer acts as a global controller responsible for scheduling and allocating edge nodes (i.e., edge servers or auxiliary vehicles) to assist vehicles in completing DNN tasks. Since the DNN tasks generated by local vehicles in autonomous driving scenarios have the characteristics of large computational complexity, latency sensitivity, and high inference accuracy, the resources of local vehicles themselves cannot meet the requirements of such tasks at all. Therefore, the core network needs to allocate edge nodes to assist in completing DNN tasks. The edge nodes include roadside units RSU and auxiliary computing vehicles to achieve end-to-end collaborative reasoning or edge-to-end collaborative reasoning. However, since the computing resources of the vehicle itself and the edge nodes are limited, the division of the DNN model and the offloading of tasks also require additional transmission delays. How to reasonably divide the DNN model while meeting these constraints and balance the computational and transmission delays to accelerate collaborative reasoning is the technical problem solved by the present invention.

[0061] like Figure 3 As shown in the figure, it is assumed that all vehicles generate the same type of DNN tasks and have a certain computing capability. It is also assumed that the vehicles have obtained the edge node information for collaborative auxiliary computing.

[0062] The process of collaborative inference optimization for in-vehicle DNN tasks based on model partitioning is as follows: (1) DNN tasks are generated locally in the vehicle. (2) The performance of model partitioning is estimated locally in the vehicle, and an appropriate split point is selected to divide the DNN model into two parts: local computing and edge node computing. (3) Based on the partitioning results, tasks of some DNN layers are computed locally. (4) The data of the intermediate layer is uploaded to the edge node via wireless transmission. (5) The edge node uses the intermediate data as input to compute subsequent DNN tasks. (6) The edge node returns the computed data to the vehicle user.

[0063] Since the completion time of this type of delay-sensitive DNN task is usually in the millisecond level, this specific embodiment assumes that the vehicle always moves within the service range of the edge node and will not drive out of the boundary of the edge node's service range, and the channel information remains unchanged during the processing of the task, and the return information of the edge node can be successfully received.

[0064] The purpose of this specific embodiment is to minimize the task completion latency of the DNN task model while meeting local vehicle memory constraints, local vehicle energy consumption constraints, and edge node energy consumption constraints. The above constraints propose a DNN task model, a latency model, a memory constraint model, an energy consumption model, and an optimization goal.

[0065] In the DNN task model, the DNN task model mainly includes convolution layer I CL and fully connected layer I FCLThe task of, where the amount of data input to the i-th layer can be expressed as The amount of computation required can be expressed in floating point operations per second (FLOPs). i express.

[0066] In the delay model, the delay of DNN collaborative reasoning includes local computing delay, transmission delay, waiting delay, edge computing delay, and calculation result return delay. Among them, the local computing delay can be expressed as T local , the edge calculation delay can be expressed as T edge , the waiting delay is T wait ,Since in the cross-cutting model cutting mode, the input data is more and the output data is less, the uplink transmission delay can be expressed as T trans , and the downlink return delay can be ignored. So far, the total delay of the DNN task collaborative reasoning process can be expressed as T total =T local +T trans +T wait +T edge The delay of DNN task completion must be less than the maximum tolerable delay that the vehicle user can accept, otherwise it is considered that the DNN task reasoning has failed, that is:

[0067]

[0068] In the memory-limited model, the total number of parameters of the locally calculated DNN task model must not exceed the actual available memory of the vehicle, that is:

[0069]

[0070] Where V local represents the set of DNN layers calculated locally after DNN partitioning, M local Indicates the maximum available memory in the vehicle. Dynamic load on the vehicle's local memory.

[0071] In the energy consumption model, the energy consumption of collaborative reasoning mainly consists of two parts: computing energy consumption and transmission energy consumption. The transmission energy consumption can be expressed as E trans , the energy consumption of local computing of DNN tasks can be expressed as E local , the energy consumption generated by the vehicle terminal when performing autonomous driving tasks shall not exceed the maximum energy consumption that can be tolerated by the local or edge terminal, that is:

[0072]

[0073] The optimization goal of this invention is to select appropriate DNN model partition points to minimize the task completion delay while meeting the local vehicle memory limit, local vehicle energy consumption limit, and edge node energy consumption limit. The problem is stated as:

[0074] minT total

[0075] C1:

[0076] C2:

[0077] C3:

[0078] C1 indicates the total amount of parameter data required for DNN task calculations on the vehicle's local system, which must not exceed the vehicle's available memory. C2 indicates the energy consumption generated by DNN task calculations on the vehicle's local system must not exceed the maximum energy consumption tolerated by the local system. C3 indicates the total latency of task inference must not exceed the maximum tolerated latency, otherwise the inference task will be considered a failure.

[0079] Based on the above optimization objectives and constraints, this specific embodiment transforms the DNN model partitioning problem under the constraints into a minimum cut problem for solving the DAG flow graph.

[0080] The first step is to model each layer in the DNN model as a directed acyclic graph (DAG) G =<V,E> . Define the node set V = {v1,v2,...,v n} represents each layer of the DNN task, E is the set of all edges, (v i ,v i+1 )∈E means there is an output v i , the input is v i+1 Each edge represents the output of the previous node as the input of the next node. The weight of the edge represents the cost of splitting the edge, that is, the delay overhead. The nodes in the DAG are divided into two parts after reasonable division: All nodes in the subgraph containing the input nodes are executed locally in the vehicle. The set of these nodes is defined as V local , all nodes in the subgraph containing the output node are calculated on the edge server or auxiliary vehicle, and the set of these nodes is defined as V edge , and V local ∪V edge =V.

[0081] The edge passed by the dividing line indicates that the output data of the previous node will be uploaded to the edge through computation offloading for the second half of the reasoning. The calculation result of the output node is then sent back to the vehicle user through the wireless link to complete the DNN collaborative reasoning task. It should be noted that if the dividing node is before the input node, it means that all tasks are offloaded to the edge node for execution, and the vehicle does not perform any local calculations, that is, At this time, all input data needs to be transferred to the edge node. If the partition node is after the output node, it means that the task is all performed locally by the vehicle and does not require the participation of the edge server.

[0082] The second step is to construct the auxiliary network flow graph of each layer of the directed acyclic graph in the DNN model to obtain the auxiliary network flow graph G′.

[0083] Solving the minimum cost DAG graph segmentation problem usually constructs an auxiliary network flow graph and then converts it into an ST-cut problem. This specific embodiment takes a simple nonlinear DAG containing four nodes as an example. Figure 4 As shown. The auxiliary network flow graph G′ constructed is as follows Figure 5 As shown. The construction steps are as follows:

[0084] The first step is to add the head node S and the tail node T, which represent the calculation on the local device and the calculation on the edge node respectively. In addition, for each node with an out-degree greater than 1, add a replica node v i The significance of adding a replica node is that a node with an outdegree greater than 1 indicates that its output data serves as input data for multiple points. During the collaborative computing of DNN tasks, this data is only transmitted once in wireless transmission and serves as input for multiple DNN task layers after reaching the edge node.

[0085] The second step is to add the weight of each edge. The weight of each edge represents the cost consumed when the dividing line cuts this edge. In the DNN model partitioning problem, it is specifically expressed as time overhead. The local computing delay, transmission delay, and edge computing delay between each DNN layer are calculated as the edge weight.

[0086] Specifically, local computing latency refers to the time required to compute a certain layer on the vehicle's local device. This latency is related to the computational complexity and computing resource usage of that layer.

[0087] Transmission latency refers to the time it takes to transmit the output data of a certain layer to an edge computing device (such as an edge server or auxiliary vehicle). This latency depends on the data size and network bandwidth.

[0088] Edge computing latency refers to the time required to execute a certain layer of computation on an edge computing device. This also depends on the processing power of the edge device and the complexity of the layer.

[0089] The meaning of each edge is as follows:

[0090] (1) Head node S to v i The edge represents the node v iIn edge computing, the weight is the latency of the node in edge computing. In particular, the edge from the head node S to the first node v1 means that the entire task is placed on the edge computing. There is no edge from the head node S to the replica node v i ′, because the edge from S to the original node v i The meaning is the same. There is no edge from the head node S to the tail node T.

[0091] (2) From node v i To its own copy node v i ′, representing the node v i Compute locally, v i The output data of the node will be remotely transmitted to the edge server as the input of the subsequent layer. The weight of this edge is v i The sum of the transmission delay of the output data and the waiting delay at the edge server. When a node needs to be calculated locally, the edge from the node to the tail node needs to be cut to obtain the local calculation delay of the node.

[0092] (3)v i To the end node T edge, indicating v i In local computing, the weight is the node's local computing latency. When a node needs to perform edge computing, the edge from the node to the head node is cut to obtain the node's edge computing latency.

[0093] (4) Copy node v i ' to the next node v i+1 The edge cost is infinite. This is because the edge represents a hybrid cut method, which places multiple subsequent nodes of the same layer on the local and edge layers for execution. (5) From node v i Go to the next node v i+1 The weight of the edge is v i The transmission delay required to transmit the output data to the edge. When partitioning is required between two adjacent nodes, the edge between the two nodes needs to be cut to obtain the transmission delay of the intermediate data.

[0094] After obtaining the auxiliary network flow graph F′, the model partitioning problem of minimizing the delay can be transformed into the st minimum cut problem: divide the auxiliary network flow graph F′ into two non-intersecting subsets V s ,V t , the head node S and the tail node T are in different subsets, and the total cut cost is minimized. All model partitioning methods can be mapped to the cutting method in the DAG graph, and the minimum cut can be transformed into a maximum flow problem.

[0095] The following theorem is used to partition the DAG flow graph: Assume that at node v i The previous division can reduce the overall delay. If v iThe in-degree is 1 and the out-degree is greater than or equal to 1, then in v i The performance of the previous partition is better than that of the i Then the nodes are divided. That is, the node is more suitable for computing on the edge node, because the computing power of the edge node is much greater than that of the local device.

[0096] Therefore, in order to simplify the model, before building the DAG flow graph, the chain nodes with in-degree and out-degree of 1 are merged into the same node. From the above theorem, it can be seen that such a simplification method will not affect the optimal result. It is also stipulated that for nodes with out-degree greater than 1, after adding a copy node, only the edge from the original node to the copy node is considered, and the edge from the copy node to the next node is not considered, that is, only cross-cutting without mixed cutting. For nodes with in-degree greater than 1, when calculating the transmission delay, since the input data can be transmitted in parallel, directly adding them will calculate unnecessary time. Therefore, for this type of node, the transmission delay needs to be redefined:

[0097]

[0098] in, represents the i-th transmission delay, represents the edge computing latency of the jth point, where j represents the edge computing node in the parallel path, and k represents the local computing node in the parallel path.

[0099] The third step is to solve the minimum cut problem of the DNN model under the constraints through the Dinic algorithm.

[0100] The Dinic algorithm is an algorithm for calculating the maximum flow in a network flow. The algorithm's principle is to first divide all nodes in the auxiliary network flow graph G′ into layers. The layering rule is that the head node S is in the 0th layer, the tail node T is in the last layer, and the directed edges in the middle layer indicate that the nodes in the previous layer can reach the nodes in the next layer. The same auxiliary network flow graph G′ may have multiple layer labeling methods. Using a breadth-first traversal approach, the auxiliary network flow graph G′ is layered sequentially, resulting in several layering methods. In each layer, the maximum flow is found through continuous augmentation. The basic Dinic algorithm can only find the optimal partitioning method, which may cause local memory or energy consumption to exceed the maximum tolerable limit. In actual vehicle network autonomous driving scenarios, such a partitioning method may cause local memory or energy consumption to exceed the maximum tolerable limit, and the Dinic algorithm cannot quickly provide a suboptimal partitioning method.

[0101] This paper proposes a DNN task model partitioning method based on the Dinic algorithm under constraints. The specific ideas are as follows:

[0102] When the auxiliary network flow graph G′ is layered, the layering method is put into a set S divThen, the energy consumption and memory of this hierarchical method are calculated to obtain the corresponding maximum tolerance point set S that meets the local delay, memory and energy consumption constraints of the vehicle. max In chain DNN task models such as VGG16, S max Is unique. Since the energy consumption of chain tasks in local calculation is cumulative and increasing, there must be a unique dividing point so that S max In the mesh DNN model such as inceptionV3, there may be multiple maximum tolerance point sets S max .

[0103] For chain DNN structure: the chain structure has a unique layering method, S max It is also unique. After obtaining the maximum tolerance set S that meets the energy consumption and memory constraints max After that, the processing method for the auxiliary network flow graph G′ is to modify all max The weights of the edges from the nodes to the tail nodes are infinite. This is because the energy consumption of local computation at these nodes is unacceptable to the local devices. Therefore, by modifying the local computation latency to infinite, these DNN layers can be offloaded to edge nodes for computation. After modifying the weights of the edges in the auxiliary network flow graph G′, a depth-first search method is used to calculate the maximum flow in the auxiliary network flow graph G′.

[0104] The partitioning method corresponding to this maximum flow is the optimal DNN model partitioning method. The partitioning method obtained at this time must meet the local energy consumption and memory constraints of the vehicle, because partitioning methods that do not meet the constraints have been avoided in advance.

[0105] For the mesh DNN structure: First, for each layer, we get the maximum tolerance set S that meets the energy and memory constraints. max Finally, the Dinic algorithm is used to calculate the theoretical optimal delay partitioning result. The optimal partitioning result is unique, which is the maximum value of the maximum flow corresponding to all layering methods.

[0106] For the division result, compare it with all S of the corresponding layered method max The elements in are compared separately:

[0107] If the division result is the above S max It indicates that the theoretically optimal partitioning method is allowed by the local energy consumption limit, so this partitioning method is the global optimal partitioning method, and the partitioning method is directly output as the optimal partitioning method of the DNN model.

[0108] If the division result is not the above S max If there is no subset of any element in , it means that this partitioning method exceeds the local limit and cannot be output as the optimal partitioning method. In this case, the following process needs to be performed:

[0109] For all S corresponding to all layering methods max Any maximum tolerance set S max , modify the weights of all nodes to the tail node that are not in the maximum tolerance point set of the corresponding vehicle's local delay, memory and energy consumption constraints to infinity; after modifying the weights of the edges of the auxiliary network flow graph G′, use the depth-first search method to calculate the maximum flow of the auxiliary network flow graph G′ modified in this way;

[0110] The partitioning method corresponding to the maximum flow among all calculation results is selected as the DNN model partitioning method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

[0111] It should be noted that if a partitioning method that meets the conditions is found, that is, according to a certain S max The optimal partitioning method of the DAG with the maximum tolerance point set in the modified weight is only the optimal solution under the maximum tolerance node set in this case, not the global optimal solution. It is necessary to continue traversing S max All elements, and then find the best way to divide them.

[0112] The model partitioning problem under the constraint condition is: divide the auxiliary network flow graph G′ into two mutually disjoint subsets V s ,V t , the head node S and the tail node T are in different subsets, V s The sum of the node costs in must not exceed the maximum cost limit, so that the cutting weight is minimized.

[0113] Summarizing the above steps, the algorithm solution process for the minimum cut of the auxiliary network flow graph G′ in this specific embodiment is as follows:

[0114] 1. Initialize variables:

[0115] Initialize the collection of storage layering, energy consumption, and memory limits:

[0116] S div : A collection of storage hierarchical methods used to record all possible partitioning schemes.

[0117] S max : Stores the maximum set of points that can be tolerated under local constraints in a hierarchical manner.

[0118] 2. Initialize the node hierarchy:

[0119] The node level array, initially empty.

[0120] 3. Use the MARKLEVEL function to mark the loop in layers:

[0121] MARKLEVEL is a hierarchical marking algorithm based on breadth-first search (BFS). Its purpose is to mark each node with its level to ensure that subsequent augmenting path searches are performed only from lower levels to higher levels. The level array records the level value of each node.

[0122] Cycle process:

[0123] Each call to MARKLEVEL may generate a new hierarchical method (i.e., a new hierarchical marking of the nodes of the graph G′).

[0124] If MARKLEVEL succeeds (i.e. there is a feasible path from the source node src to the target node dst), this hierarchical method is added to the set S div

[0125] Loop termination condition:

[0126] When MARKLEVEL returns False, it means that there are no more augmenting paths to search and the loop ends.

[0127] After each loop:

[0128] Compute the set of all nodes that can be executed locally under local constraints and add them to S max for subsequent decision-making.

[0129] 4. Perform the maximum flow loop and augmenting path loop:

[0130] Based on each maximum tolerance set, the weights of the edges in the auxiliary network flow graph of the DAG graph are modified accordingly: the local execution delay of nodes (tasks) that are not in the maximum tolerance set is set to infinite (∞), which means that these tasks must be executed on edge devices.

[0131] Initialize flow variables: maxflow←0: The maximum flow is 0. curflow←INF: The current flow is set to infinity, indicating that there are no constraints initially.

[0132] For each modified auxiliary network flow graph, the augmenting path algorithm FINDAUGPATH is called to find the augmenting path starting from the source node src, update curflow, and add the found augmented flow to maxflow.

[0133] Loop termination condition:

[0134] When no augmenting path can be found, or curflow is 0, it means that the maximum flow maxflow under the current auxiliary network flow graph has been determined.

[0135] 5. Record the optimal partitioning scheme:

[0136] Select the partitioning method with the largest maximum flow as the optimal hierarchical partitioning scheme and obtain the minimum cut set V local ,V edge :

[0137] V local : A collection of tasks executed on the local device.

[0138] V edge : A collection of tasks executed on edge devices.

[0139] The time complexity of the Dinic algorithm designed in this specific embodiment is O(|V| 2 |E|).

[0140] The pseudo code of this specific embodiment is as follows:

[0141] Input: Parameters of each layer of the DNN task FLOPs i , uplink transmission rate R, maximum memory limit Maximum energy consumption limit

[0142] Output: Task segmentation result V local ,V edge

[0143] Build the DAG graph G(V,E) of the DNN model

[0144] Calculate the local computing delay T of each layer local,i Transmission delay T trans,i , edge computing latency T edge,i

[0145] Construct DAG extended auxiliary network flow graph G′(V,E)

[0146] Initialize storage layer mode S div , the maximum tolerance point set S under local energy consumption and memory constraints max ,

[0147] Initialize node level level←DEAFULTED(int)

[0148] while MARKLEVEL(G,src,dst,level)do

[0149] Add this hierarchical level to the set S div

[0150] Calculate the maximum energy consumption tolerated by the local, the set of maximum memory points, and add S max

[0151] end while

[0152]

[0153] for eachs max,j ∈S max do

[0154] Modify the T of the corresponding node local,i ←∞

[0155] Initialize maxflow←0,curflow←INF

[0156] for eachs div,k ∈S div do

[0157] maxflow←maxflow+NDAUGPATH(src,curflow)

[0158] end for

[0159]

[0160] end for

[0161] according to Get the minimum cut set V local ,V edge

[0162] returnV local ,V edge

[0163] The pseudo code of the MARKLEVEL breadth-first traversal algorithm for G(V,E) is as follows:

[0164] Input: graph G(V,E), head node src, tail node dst, array level of record levels

[0165] Output: Is there a new labeling method, and record it in level

[0166]

[0167]

[0168] The pseudo code of the FINDAUGPATH() depth-first search augmenting path algorithm is as follows:

[0169] Input: current node v u , the current remaining flow restflow

[0170] Output: New flow

[0171]

[0172] To analyze the performance of the DNN model partitioning algorithm under the constraints proposed in this specific embodiment, Edge-Only (edge ​​computing only), End-Only (local computing only), Neur algorithm, DADs, and Min-Energy algorithm were selected as comparison algorithms. YOLOV2 and VGG16 were used as representatives of chain-type DNN tasks, and AlexNet and InceptionV3 were used as representatives of DAG-type DNN tasks to evaluate the algorithm performance.

[0173] The present invention selects acceleration performance and throughput improvement as performance indicators for evaluating algorithms. Acceleration performance represents the multiple of the acceleration of inference speed, while throughput improvement represents the multiple of the increase in the number of requests processed per unit time.

[0174] Experimental results show that, compared to local computing and edge computing, model partitioning, a DNN task inference acceleration strategy, can reduce overall inference latency, speed up inference, and improve system throughput. For chained DNN tasks, MPAUC performs similarly to classic Neur and DADs. However, for DAG-structured DNN models, MPAUC can achieve faster inference results. Furthermore, in specific scenarios with memory and energy constraints, the MPAUC algorithm can address latency optimization issues within the constraints of local devices, something that Neur and DADs cannot achieve.

[0175] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence, characterized by: The following steps are involved: Build a DAG graph based on the vehicle's DNN model; Construct an auxiliary network flow graph for the DAG graph: add a head node representing the local calculation on the vehicle and a tail node representing the calculation on the edge server or auxiliary vehicle, and set the edge weight to the latency cost of splitting the edge; Obtain different layering methods of the auxiliary network flow graph of the DAG graph and its corresponding maximum tolerance point set that meets the vehicle's local latency, memory and energy consumption constraints; Based on each maximum tolerance point set, modify the weights of the edges in the auxiliary network flow graph of the DAG graph, and calculate the maximum flow of each modified network flow auxiliary graph; The partitioning method corresponding to the maximum flow is selected as the DNN model partitioning method, and the DNN model tasks performed by the local vehicle and the edge server or auxiliary vehicle are determined respectively.

2. The DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence according to claim 1 is characterized by: The vehicle's local latency, memory, and energy constraints include: The total amount of parameter data calculated locally on the vehicle for DNN model tasks must not exceed the actual available memory on the vehicle. The energy consumption generated by the local calculation of the DNN model task in the vehicle must not exceed the maximum energy consumption that can be tolerated locally; The total delay of the DNN model task must not exceed the local maximum tolerance delay, otherwise the DNN model task is considered failed.

3. The DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence according to claim 1 is characterized by: In the process of constructing the auxiliary network flow graph of the DAG graph, for each node with an out-degree greater than 1, add a copy node; merge the chain nodes with an in-degree and out-degree of 1 into the same node; It is stipulated that for nodes with out-degree greater than 1, after adding a copy node, only the edge from the original node to the copy node is considered for division, and the edge from the copy node to the next node is not considered for division.

4. The DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence according to claim 3 is characterized by: In the auxiliary network flow diagram of the constructed DAG graph: The edge from the head node to any node indicates that the node is in edge computing, and the weight is the latency of the node in the edge server or auxiliary vehicle computing; The edge from the head node to the first node indicates that the entire DNN model task is placed on the edge server or auxiliary vehicle computing; There is no edge from the head node to the replica node; There is no edge from the head node to the tail node; An edge from any node to its own replica node indicates that the node is performing computation locally on the vehicle. The output data of this node will be remotely transmitted to an edge server or auxiliary vehicle as input to subsequent layers of the DNN model. The weight of this edge is the sum of the transmission delay of the node's output data and the waiting delay on the edge server. The edge from any node to the tail node indicates that the node is calculated locally on the vehicle, and the weight is the delay of the local calculation of the node; The weight of the edge between two adjacent non-head and non-tail nodes is the transmission delay of the output data from the previous node to the next node; The cost of copying an edge from any node to the next node is infinite.

5. The DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence according to claim 1 is characterized by: If the DNN model is a chain structure, in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on the maximum tolerance set, the weights of all edges from nodes that are not in the maximum tolerance point set to the tail node are modified to be infinite.

6. The DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence according to claim 1 is characterized by: If the DNN model is a mesh structure, then in the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph, the Dinic algorithm is first used to calculate the partitioning result with the theoretically optimal delay; For this division result, compare it with the elements in all the maximum tolerance point sets of the corresponding hierarchical method: If the partition result is a subset of the elements in all the maximum tolerance point sets of the corresponding layered method, the partition result is directly output as the DNN model partition method; If the optimal partition result is not a subset of the maximum tolerance set, the following process is performed: In the process of modifying the weights of the edges in the auxiliary network flow graph of the DAG graph based on any maximum tolerance set, the weights of all edges from the nodes that are not in the maximum tolerance point set to the tail node are modified to be infinite.

7. The DNN-based model partitioning and inference acceleration method for in-vehicle edge intelligence according to claim 5 or 6, characterized in that: The process of calculating the maximum flow of the auxiliary network flow graph modified according to the maximum tolerance set includes: using the FINDAUGPATH function to find the augmenting path from the head node to the tail node in the hierarchical manner, calculating the flow that can be transmitted by the current path, and accumulating it into the maximum flow; this process continues until no new augmenting path can be found, and the final accumulated maximum flow is output.

8. A DNN model partitioning and inference acceleration system for in-vehicle edge intelligence, characterized by: Used to implement the DNN model partitioning and reasoning acceleration method for in-vehicle edge intelligence as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the DNN model partitioning and reasoning acceleration method for in-vehicle edge intelligence as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the DNN model partitioning and inference acceleration method for in-vehicle edge intelligence as described in any one of claims 1 to 7 by executing the computer instructions.

Images