A flexible collaborative reasoning method in a dynamic vehicle-to-everything (V2X) environment

Abstract

The application provides an elastic collaborative inference method in a dynamic vehicle networking environment. Based on an inference scene of an edge acceleration deep neural network in the dynamic vehicle networking environment, a collaborative inference delay model established according to a propagation characteristic, dynamic vehicle networking environment characteristics and edge node reliability are combined to solve an optimal distribution strategy of the edge nodes and an optimal distribution scheme of inference block segmentation and load division. Then, a client vehicle decomposes a model into inference blocks, and refines a work load of each inference block into smaller independent sub-tasks. The sub-tasks are unloaded to selected edge nodes for parallel processing. Meanwhile, the client vehicle locally calculates the same inference blocks. The client vehicle iteratively executes the above process for each inference block until the inference of the entire model is completed. By using the provided method, adaptive control of load segmentation, division and unloading in the dynamic vehicle networking environment is achieved, and low-delay and high-reliability services are provided for intelligent networked vehicle inference tasks.

Description

Technical Field

[0001] This invention relates to the field of edge computing for vehicle-to-everything (V2X) networks, and specifically to a flexible collaborative reasoning method in a dynamic V2X environment. Background Technology

[0002] The rapid development of Industrial Cyber-Physical Systems (ICPSs) is reshaping multiple fields such as smart factories, intelligent transportation systems, and intelligent connected vehicles. Among them, vehicle-to-everything (V2X) communication, as a typical application of ICPSs, is gradually becoming a focus. In V2X, vehicles equipped with various sensors, communication modules, and computing units interact with edge infrastructure and cloud centers through V2X communication to achieve information sharing and collaborative processing.

[0003] Meanwhile, the development of Deep Neural Networks (DNNs) has demonstrated outstanding performance in high-level feature extraction and content processing. Embedding DNN models in connected vehicle devices can support numerous intelligent applications, such as augmented reality navigation and autonomous driving. However, with the continuous growth in network model size and increasing computational demands, the efficient deployment of DNNs on resource-constrained in-vehicle devices faces significant challenges, especially given the growing emphasis on low latency and high reliability, which traditional methods can no longer meet.

[0004] To improve the execution efficiency of DNN tasks, cooperative inference (CI) is rapidly developing. Its main cooperative methods can be divided into two modes: model cooperation and data cooperation. Model cooperation optimizes resource utilization by splitting the DNN model into layers or blocks and offloading the split parts to edge nodes (such as intelligent connected vehicles or roadside devices) for phased execution. In contrast, data cooperation, based on the spatial independence of convolutional operations, divides the inference task into smaller loads and offloads them to multiple peer devices for parallel execution, thereby significantly accelerating the inference process.

[0005] While the aforementioned technologies offer substantial improvements in task execution efficiency, their deployment in real-world connected vehicle environments still faces numerous challenges: the dynamic nature of the vehicle environment, including high vehicle mobility, heterogeneous computing resources, and occasional edge node outages, leads to fluctuating service performance, potentially compromising inference efficiency and functional safety; moreover, most existing technologies are geared towards stable and reliable edge computing environments, resulting in one-off, fixed decision-making processes that are unsuitable for dynamic connected vehicle environments. Furthermore, existing technologies neglect offloading fault protection strategies, failing to quickly restore services in the event of offloading failures. Summary of the Invention

[0006] Based on this, the present invention provides an elastic collaborative inference method in a dynamic vehicle network environment, which adaptively controls the load segmentation, partitioning and offloading in the dynamic vehicle network to ensure the real-time performance and reliability of DNN inference tasks, and provides low-latency and high-reliability services for inference tasks of intelligent connected vehicles.

[0007] To achieve the above objectives, the present invention provides a flexible collaborative reasoning method in a dynamic vehicle-to-everything (V2X) environment, comprising the following steps:

[0008] S100. Based on the inference scenario of edge-accelerated DNN in dynamic vehicle networking environment, a collaborative inference latency model is established considering the forward propagation characteristics of DNN inference. Combining the characteristics of dynamic vehicle networking environment and edge node reliability, the optimal allocation strategy of edge nodes and the optimal allocation scheme of inference block partitioning and load division are solved.

[0009] S200. Based on the assigned edge nodes and offloading strategy, the client vehicle decomposes the model into inference blocks, and further refines the workload of each inference block into smaller, independent subtasks. These subtasks are offloaded to selected edge nodes for parallel processing. Simultaneously, the client vehicle computes the same inference blocks locally to quickly restore service in case of offloading failure. The specific process is as follows:

[0010] Task unloading path: The client vehicle partitions the input data or intermediate features and sends the input required for each inference block to the selected edge node. The edge node completes the calculation of each subtask in this block and returns the output to the client vehicle, repeating the same operation for each inference block.

[0011] Local computation path: The client vehicle simultaneously computes the same inference block locally. If the unloading path output is successfully received, local computation terminates and processing continues to the next inference block; otherwise, local computation will complete the subsequent inference block.

[0012] S300. The client vehicle iteratively executes step 2 for each inference block until the inference of the entire model is completed.

[0013] Furthermore, the following optimization method is adopted to minimize the average collaborative inference latency in the overall task:

[0014]

[0015] st

[0016]

[0017] Where 1,2,…,T represents the inference task sequence generated by the client vehicle, and A v B is the set of edge nodes to be assigned.m For inference block segmentation strategy, For the offloading strategy (including the allocated edge nodes, inference block partitioning, and inference load allocation scheme), CID t Let m be the collaborative inference latency for task t, and m be the DNN model deployed on the customer's vehicle. This is the k-th inference block of the model, starting from the i-th layer and ending at the j-th layer. The execution delay of this inference block under the task unloading path, The execution latency of this inference block under the local computation path of the customer vehicle v. L is the number of output rows of the last layer of the model. m This is the set of layer subscripts for the model.

[0018] The CID is obtained by the collaborative inference latency model.

[0019] Furthermore, the collaborative inference latency CID is calculated as follows:

[0020]

[0021] Collaborative inference latency is determined by the inference latency of all blocks. Sum calculation.

[0022] Furthermore, reasoning blocks Inference delay The calculation method is as follows:

[0023]

[0024] The inference latency of the inference block depends on the local computation path and the task unloading path, where I(·)∈{0,1} is used to determine whether the unloading process of the edge node is successful.

[0025] Furthermore, local computing latency The reasoning process is as follows:

[0026]

[0027] Where c v For the computing power of customer vehicle v, f k For reasoning blocks The computational overhead.

[0028] Furthermore, unloading latency The reasoning process is as follows:

[0029]

[0030] in, Upload latency for inference block input data To handle latency at edge nodes, The output delays at the end of the inference block are aggregated and calculated as follows:

[0031] The DNN network structure is modeled as a directed acyclic graph M = (L, E), with vertex l. i (1≤l i ≤|L|) represents an inference layer of a DNN, such as convolution, pooling, etc., and edge e i =(l i ,l j ) represents the data flow between inference layers.

[0032] For a specific DNN layer l i Its kernel size, padding, and stride are defined as k. i ,p i ,s i Define the input data size for each layer as follows: These represent the height, width, and number of channels of the input matrix, respectively. Similarly, the output data size is defined as... These represent the height, width, and number of channels of the output matrix, respectively.

[0033] Since each element in the output layer can be computed by sliding a kernel window over the input layer, the height of the output matrix is, based on the corresponding properties:

[0034]

[0035] This formula also applies to the width of the output matrix.

[0036] The computational cost f of the layer i Quantization can be achieved through floating-point operations (FLOPs):

[0037]

[0038] Data size between layers e i for:

[0039]

[0040] in Memory usage per unit of data.

[0041] Because the inference operations at each layer are not atomic, they can be divided into multiple smaller inference fragments and distributed to multiple edge nodes for parallel execution. Assuming the layers are partitioned along their height, a matrix row within a layer is considered the smallest unit of partitioning. Define layer l. i The inference fragment of the output matrix is Given the expected output matrix of an inference segment, based on the sliding window principle of convolution and pooling, the corresponding required input data range is: in:

[0042]

[0043] Define v as the customer vehicle, N v ={n1,n2,…,n |N|} represents the set of available edge nodes, and the set of nodes selected to participate in the unloading is... The computational capabilities of customer vehicle v and edge node n are c, respectively. v and c n The V2X transmission bandwidth is B.

[0044] Based on the selected set of nodes A v Suppose that a specific DNN model m∈M can be decomposed into a series of inference blocks containing multiple DNN layers. Define the set of decomposed inference blocks as:

[0045]

[0046] in, This represents the k-th inference block that starts at level i and ends at level j, |L m | indicates the last level index. Because blocks do not intersect, the start and end indices should typically satisfy certain conditions.

[0047] Based on the assigned set of edge nodes A v The DNN model is partitioned into |B m | inference blocks, where each inference block It is further divided into multiple reasoning segments, and the unloading strategy is defined as a dictionary.

[0048] Here, it is represented as edge node n being responsible for executing the inference block. Partial inference load and output matrix Send to customer vehicle v.

[0049] Assuming all available edge nodes share a spectrum with a total bandwidth of B for V2X transmission via Orthogonal Frequency Division Multiplexing (OFDM), and the client vehicle allocates bandwidth equally to each selected node, then the transmission rate between the client vehicle and the edge node can be expressed as:

[0050]

[0051] Where P represents transmission power, σ 2 G represents the received noise power. vnThe channel gain for vehicle v and edge node n is expressed as follows: For small-scale Rayleigh fading parameters, This represents the large-scale fading coefficient.

[0052] Furthermore, the upload latency of input data The calculation method is as follows:

[0053]

[0054] in, r represents the height of the input matrix of the inference block. vn It is the transmission rate between the customer vehicle v and the edge node n.

[0055] Related research indicates that, for specific layers and devices, communication and computational overhead are directly proportional to data size. Based on this, the latency of the edge node processing inference task blocks... The calculation method is as follows:

[0056]

[0057] Furthermore, the output result at the end of the inference block is aggregated with a delay. The calculation method is as follows:

[0058]

[0059] Furthermore, in optimizing the overall task, an online learning scheduling algorithm based on dynamic programming is used to determine the optimal allocation strategy for edge nodes and the best allocation scheme for inference block partitioning and load balancing.

[0060] Furthermore, the optimal allocation strategy for edge nodes and the best allocation scheme for inference block partitioning and load balancing are determined according to the following strategies.

[0061] For the assigned set of edge nodes A v The capability of the edge node n is defined as follows:

[0062]

[0063] Among them, c n For edge node computing power, The unloading success rate of edge node n observed by the client vehicle. It is a monotonically decreasing function in the range [1, +∞).

[0064] Within each inference block, taking into account the heterogeneity of the computing power of the selected edge nodes, the output matrix is ​​allocated in the following manner to balance the workload, ensuring that they return the corresponding outputs in approximately the same amount of time.

[0065]

[0066] If we define the success rate of an edge node n returning an assigned inference fragment within a specific time threshold as Pr(n), then we define an inference block. uninstallation success rate for:

[0067]

[0068] When the k-th inference block executes successfully on the unloading path, its latency is the sum of its execution latency on the unloading path and the expected latency of subsequent inference blocks. When the k-th inference block fails to execute on the unloading path, the k-th and subsequent inference blocks will be executed locally. The recursive formula for the expected latency is:

[0069]

[0070] Considering both successful and failed uninstallation scenarios, and using a binary tree of uninstallation success rates, the general expression for expected latency is:

[0071]

[0072] Let I(·) be an indicator function used to indicate whether the unloading process of an edge node is successful. Then, the unloading success rate of edge node n after update is defined as follows:

[0073]

[0074] Based on this, the optimal allocation strategy for edge nodes and the best allocation scheme for inference block partitioning and load balancing are determined according to the following steps:

[0075] S500. Initialization: Since the actual value of the uninstallation success rate Pr(n) cannot be known in advance, an estimated value is used. This represents the unloading success rate of the edge nodes observed by the client vehicles, initially set to...

[0076] S600. Dynamic Programming: Sort all available edge nodes in descending order based on θ(n), then sort from 1 to... Traverse the top_N edge nodes and iteratively select the top_N edge nodes according to the sorted queue. Then, determine the inference block partitioning scheme based on dynamic programming, and determine the load partitioning and allocation scheme within the inference block according to the computing power of the edge nodes. Finally, merge some inference blocks according to the offloading success rate of each edge node. In summary, we aim to obtain the offloading scheme with the lowest inference latency and the highest offloading success rate.

[0077] S700. Local Computation: For each inference block, the client vehicle simultaneously executes the task unloading path and the local computation path, and updates the unloading success rate of each edge node. If the result of the unloading path is received before the local computation is completed, the inference block is considered complete, the local computation terminates, and subsequent inference blocks continue to be executed using the initial unloading strategy. If the system dynamically causes the unloading path to fail, the local computation completes the block, and subsequent inference blocks are rescheduled as new tasks until completion.

[0078] Compared with existing technologies, the technical advantages of the proposed elastic collaborative reasoning method in a dynamic vehicle-to-everything (V2X) environment are at least reflected in:

[0079] Firstly, in a dynamic vehicular network environment, the high mobility of vehicles, heterogeneous computing resources, and occasional edge node failures may affect inference efficiency and functional safety. This invention introduces a dual-path parallel architecture of task offloading and local computation to interactively complete inference tasks and deliver the final result. While improving computational efficiency, it also enables rapid recovery in the event of offloading failure, ensuring system reliability.

[0080] Secondly, to fully utilize the idle resources of edge devices to reduce end-to-end inference latency and maximize service quality, this invention adaptively segments the DNN model and offloads the inference load to edge nodes for processing, minimizing overall offloading latency. Simultaneously, it adaptively merges some DNN inference blocks to reduce the number of offloading stages, thereby improving the success rate of offloading tasks and optimizing overall inference performance.

[0081] Thirdly, addressing the issue of unknown reliability at edge nodes, this invention integrates online learning methods into the scheduling process and utilizes a binary tree model to evaluate the offloading success rate of inference tasks. This method ensures real-time performance while maintaining reliability, effectively overcoming the shortcomings of existing technologies and providing low-latency, high-reliability services for inference tasks in intelligent connected vehicles. Attached Figure Description

[0082] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0083] Figure 1 This is a flowchart of the workflow of a flexible collaborative reasoning method in a dynamic vehicle networking environment.

[0084] Figure 2 This is a schematic diagram of a task scenario for an embodiment of the elastic collaborative reasoning method provided in a dynamic vehicle network environment;

[0085] Figure 3This is a schematic flowchart of an embodiment of a flexible collaborative reasoning method in a dynamic vehicle network environment.

[0086] Figure 4 This is a schematic diagram of the dynamic programming-based inference block segmentation and load partitioning mechanism used in the provided embodiments;

[0087] Figure 5 This is an example diagram of a potential unloading strategy after merging inference blocks in the provided embodiments;

[0088] Figure 6 This is a schematic diagram of the binary tree-based expected inference delay evaluation method in the provided embodiments;

[0089] Figure 7 This is a schematic diagram of inference rescheduling under unloading failure in the provided embodiment;

[0090] Figure 8 This is a flowchart illustrating the optimal allocation strategy for determining edge nodes and the optimal allocation scheme for inference block partitioning and load balancing. Detailed Implementation

[0091] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The descriptions of the exemplary embodiments are merely illustrative and are in no way intended to limit the present disclosure or its application or use. The present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

[0092] like Figure 1 As shown, this invention provides a flexible collaborative reasoning method in a dynamic vehicle-to-everything (V2X) environment, comprising the following steps:

[0093] S100: Based on the inference scenario of edge-accelerated deep neural networks in dynamic vehicle networking environment, combined with the collaborative inference latency model established by the propagation characteristics, dynamic vehicle networking environment characteristics and edge node reliability, the optimal allocation strategy of edge nodes and the optimal allocation scheme of inference block partitioning and load division are solved.

[0094] S200: Based on the allocated edge nodes and offloading strategy, the client vehicle decomposes the model into inference blocks and further refines the workload of each inference block into smaller independent subtasks; the subtasks are offloaded to selected edge nodes for parallel processing; at the same time, the client vehicle performs computation on the same inference blocks locally to quickly restore service in case of offloading failure.

[0095] S300: The client vehicle iteratively executes S200 for each inference block until the inference of the entire model is completed.

[0096] This invention proposes a resilient collaborative inference method for dynamic vehicular network environments, a resilient accelerated inference mechanism that balances inference efficiency and functional safety. This mechanism introduces a dual-path parallel architecture of task offloading and local computation, interactively completing inference tasks and delivering the final result. While improving computational efficiency, it enables rapid recovery in the event of offloading failure, ensuring system reliability. Furthermore, this invention adaptively segments the DNN model and offloads the inference load to edge nodes for processing, minimizing overall offloading latency. Simultaneously, it adaptively merges some DNN inference blocks to reduce the number of offloading stages, thereby improving the success rate of offloading tasks and optimizing overall inference performance.

[0097] To make the objectives, technical solutions, and advantages of this invention clearer, the elastic collaborative reasoning method in the dynamic vehicle networking environment proposed in this invention will be further described in detail below with reference to the accompanying drawings.

[0098] For example Figure 2 The illustrated vehicle-to-everything (V2X) environment demonstrates the present invention. In this context, the vehicle performing DNN inference tasks such as classification or detection is defined as the client vehicle, while other peer vehicles and infrastructure such as roadside units (RSUs) are defined as edge nodes. Generally, the client vehicle is defined as v, and the set of edge nodes is N. v ={n1,n2,…,n |N| The client vehicle adaptively segments the DNN model and offloads the inference load to edge nodes via V2X communication for processing. To ensure rapid recovery of the inference service in case of offloading failure, this invention integrates an additional execution path for active inference on the local computing platform. Finally, the entire model inference is completed through parallel execution of the task offloading path and the local computing path.

[0099] This embodiment employs the following DNN inference latency model: During implementation, the edge-accelerated DNN inference mechanism is basically as follows: Figure 3 As shown, the DNN network structure is modeled as a directed acyclic graph M = (L, E), with vertex l i (1≤l i ≤|L|) represents an inference layer of a DNN, such as convolution, pooling, etc., and edge e i =(l i ,l j The symbol () represents the data flow between inference layers. Related research indicates that for any complex DNN architecture, the feature extraction stage accounts for the majority of the total computation time. Furthermore, the client vehicle processes the output of the feature extraction stage locally to obtain the final results for classification, detection, or other downstream tasks. Therefore, this embodiment only considers the more general feature extraction stage in model inference.

[0100] For a specific DNN layer l i Its kernel size, padding, and stride are defined as k. i ,p i ,s i The input data size for each layer is These represent the height, width, and number of channels of the input matrix, respectively. Similarly, the output data size is defined as... These represent the height, width, and number of channels of the output matrix, respectively.

[0101] Since each element in the output layer can be computed by sliding a kernel window over the input layer, the height of the output matrix is, based on the corresponding properties:

[0102]

[0103] This formula also applies to the width of the output matrix.

[0104] Layer l i computational overhead f i Quantization can be achieved through floating-point operations (FLOPs):

[0105]

[0106] Data size between layers e i for:

[0107]

[0108] in Memory usage per unit of data.

[0109] During implementation, each layer's inference is divided into multiple smaller inference segments and distributed to multiple edge nodes for parallel execution. Assuming the layers are partitioned along their height, a matrix row within a layer is considered the smallest unit of partitioning. Layer l is defined. i The matrix output of a certain segment of reasoning is The required input data range is: Based on the sliding window principle of convolution and pooling, specifically:

[0110]

[0111] Define the set of edge nodes selected to participate in unloading as follows: The computational capabilities of customer vehicle v and edge node n are c, respectively. v and c n The available V2X transmission bandwidth is B.

[0112] In a dynamic vehicular network environment, the available edge nodes and their operational status may change over time. Based on a selected node set A v This embodiment assumes that a specific DNN model m∈M can be decomposed into a series of inference blocks containing multiple DNN layers, and defines the set of decomposed inference blocks as follows:

[0113] in This represents the k-th inference block that starts at level i and ends at level j, |L m | indicates the last level index. Because blocks do not intersect, the start and end indices should typically satisfy certain conditions.

[0114] Based on the assigned set of edge nodes A v The DNN model is partitioned into |B m | inference blocks, where each inference block It can be further divided into multiple reasoning segments, and the unloading strategy is defined as a dictionary. It is represented by edge node n, which is responsible for executing the inference block. Partial inference load and output matrix Send to customer vehicle v. Based on the unloading strategy, the required input matrix for all preceding layers in a given inference block can be determined. Each inference block is treated as a separate subtask and offloaded to an edge node for parallel inference. Specifically, an inference block executes in three phases along the offload path: the client vehicle sends the input matrix to the edge node (communication), the edge node performs parallel processing (computation), and the output is aggregated from the edge node to the client vehicle (communication).

[0115] Assuming all available edge nodes share a spectrum with a total bandwidth of B for V2X transmission via Orthogonal Frequency Division Multiplexing (OFDM), and the client vehicle allocates bandwidth equally to each selected node, the transmission rate r between the client vehicle and the edge node can be calculated using Shannon's formula. vn The calculation process in this embodiment is as follows, but is not limited thereto:

[0116]

[0117] Where P represents transmission power, σ 2 G represents the received noise power. vn The channel gain for vehicle v and edge node n is expressed as follows: For small-scale Rayleigh fading parameters, The large-scale fading coefficient reflects the effects of path propagation loss and log-normal shadowing. In this embodiment, the path loss model of the V2X link is 38.77 + 18.2log(f c)+16.7log(d vn ) indicates that f c Indicates the center carrier frequency (GHz), d vn This represents the Euclidean distance (m) between the client vehicle and the edge node. Based on the above definition, the upload latency of the input data... The calculation method is as follows:

[0118]

[0119] in, r represents the height of the input matrix of the inference block. vn It is the transmission rate between the customer vehicle v and the edge node n.

[0120] The customer vehicle slices the input to the inference load handled by the edge node and transmits it to the edge node. Once the edge node receives the complete input data, it independently generates the specified output matrix. Related research indicates that for specific layers and devices, communication and computational overhead are proportional to data size. Based on this, the latency of the edge node processing inference task blocks... The calculation method is as follows:

[0121]

[0122] After the edge nodes complete their calculations, they transmit the corresponding output matrix back to the customer vehicle for output aggregation and repartitioning for the next inference block. The latency of transmitting the inference block's output back to the customer vehicle is also considered. The calculation method is as follows:

[0123]

[0124] Only when A v A complete output matrix can only be concatenated at the end of the inference block when all edge nodes have successfully returned results. Based on this, since all edge nodes execute their assigned segments in parallel, the offloading delay of the block is caused by A. v The maximum processing time between mid-edge nodes determines the aggregated latency of the output at the end of the inference block. The calculation method is as follows:

[0125]

[0126] On the other hand, this embodiment performs proactive computation on the local path to accelerate service recovery under uncertain conditions. When a customer vehicle partitions an inference block and offloads it to an edge node for collaborative inference, the same inference block is processed in parallel on the local computing platform. Local computation latency The reasoning process is as follows:

[0127]

[0128] Where c v For the computing power of customer vehicle v, f k For reasoning blocks The computational overhead.

[0129] The inference latency of an inference block depends on the local computation path and the task unloading path, denoted by I(·)∈{0,1}, which represents whether the unloading process of an edge node was successful. Inference Block Inference delay The calculation method is as follows:

[0130]

[0131] Cooperative inference latency CID is represented as the inference latency of all blocks. sum:

[0132]

[0133] Based on the above model, this embodiment adopts the following... Figure 3 The method shown is used to optimize the allocation scheme for edge-accelerated inference tasks.

[0134] Assumption For the unloading strategy of customer vehicle v, where A v B is the set of edge nodes to be assigned. m For inference block segmentation strategy, This is represented as edge node n, responsible for executing the inference block. and output matrix Sending data to customer vehicle v. The proposed task offloading problem is then to minimize the average collaborative inference latency:

[0135]

[0136] Of the three calculation formulas mentioned above, the second formula indicates that the execution latency of the task unloading path should be lower than the execution latency of the local computing path processing the same block. The third formula indicates that the number of selected edge nodes should not exceed the number of output rows of the last layer of the model, because one row of the output matrix is ​​considered the smallest part of the DNN partition.

[0137] In this implementation, a dual-path parallel architecture of task offloading and local computation is used to simultaneously ensure service efficiency and functional safety. Firstly, to guarantee service efficiency, the customer vehicle adaptively segments the DNN model based on model characteristics and the heterogeneous resource characteristics of edge nodes, and offloads the inference load to edge nodes for processing, reducing overall offloading latency. Simultaneously, some DNN inference blocks are adaptively merged to reduce the number of offloading stages, thereby improving the success rate of offloading tasks and optimizing overall inference performance.

[0138] Considering that any dynamic changes in the system, such as transmission interruptions, service preemption, or edge node crashes, will affect the task unloading process, we use Pr(n)∈[0,1] to represent the probability that edge node n successfully returns the allocated partition within a specific time threshold. It is worth noting that Pr(n) is an independent and identically distributed random variable, and its value is unknown for the customer vehicle. Only when A... v A complete output matrix can only be concatenated at the end of the block when all edge nodes have returned successfully. Define the inference block. uninstallation success rate for:

[0139]

[0140] In this embodiment, a binary tree is used to evaluate the offloading success rate of each inference block edge node, and an online learning scheduling algorithm based on dynamic programming is used to determine the optimal matching strategy for edge nodes and the best allocation scheme for inference block partitioning and load balancing. Figure 8 As shown, the specific steps are as follows:

[0141] S500. Initialization: Since the actual value of the uninstallation success rate Pr(n) cannot be known in advance, an estimated value is used. This represents the unloading success rate of the edge nodes observed by the client vehicles, initially set to...

[0142] S600. Dynamic programming, including:

[0143] First, based on ability Sort all available edge nodes in descending order, then sort them from 1 to... Iterate through the top_N nodes, selecting the top_N edge nodes in an iteratively increasing manner according to the sorted queue, and then perform subsequent operations.

[0144] Then, disregarding the success rate of unloading from edge nodes, the inference block partitioning scheme S is determined based on dynamic programming, such as... Figure 4 As shown, from 1 to |L m | Iterate through j, then iterate through i from 1 to j-1, calculating the execution delay of the unloading path for various inference block splitting schemes. Based on the minimum unloading latency of the inference block Inference blocks are added sequentially to obtain an inference block partitioning scheme S. Then, based on the computing capabilities of the edge nodes, a load distribution scheme is determined within the inference blocks to balance the workload and ensure that they return the corresponding outputs at approximately the same time.

[0145]

[0146] Then, considering the unloading success rate of each edge node, the expected collaborative inference time E[CID(b)] is calculated. m,k ,b m,K Then merge some reasoning blocks, such as Figure 5 This demonstrates the potential unloading strategy after merging inference blocks. This embodiment uses a binary tree to calculate the unloading success rate of each edge node, such as... Figure 6 As shown, when the k-th inference block executes successfully on the unloading path, its latency is the sum of its execution latency on the unloading path and the expected latency of subsequent inference blocks; when the k-th inference block fails to execute on the unloading path, the k-th and subsequent inference blocks will be executed locally. The recursive formula for the expected latency is:

[0147]

[0148] Considering both successful and failed uninstallation scenarios, and using a binary tree of uninstallation success rates, the general expression for expected latency is:

[0149]

[0150] Finally, based on dynamic programming, the offloading strategy (optimal allocation strategy for edge nodes and optimal allocation scheme for inference block partitioning and load balancing) is obtained, and the optimal number of edge nodes is obtained.

[0151] S700. Local computing, including:

[0152] First, for each inference block, the client vehicle executes both the task unloading path and the local computation path simultaneously.

[0153] Then, update the unloading success rate for each edge node. Define I(·) as an indicator function to indicate whether the unloading process of an edge node is successful. Then, define the updated unloading success rate for edge node n as follows:

[0154]

[0155] Finally, as Figure 7 As shown, if the result of the unloading path is received before the local computation is completed, the inference block is considered complete, the local computation terminates, and subsequent inference blocks continue to be executed using the initial unloading strategy. If the system dynamically causes the unloading path to fail, the local computation completes the block, and subsequent inference blocks are re-executed as new tasks in steps 2) and 3) until completion.

[0156] In summary, the method in this embodiment takes into account the heterogeneity of edge node computing and communication capabilities, edge node reliability, and DNN inference forward propagation, balancing reliability and real-time performance. It effectively compensates for the shortcomings of existing technologies and can provide low-latency and high-reliability services for inference tasks of intelligent connected vehicles.

[0157] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An elastic collaborative inference method in a dynamic vehicular networking environment, characterized in that: S100: Based on the inference scene of edge-accelerated deep neural network in dynamic vehicular networking environment, combined with the collaborative inference delay model established based on forward propagation characteristics, dynamic vehicular networking environment characteristics and edge node reliability, the optimal allocation strategy of edge node and the best allocation scheme of inference block segmentation and load division are solved; S200: Based on the allocated edge node and unloading strategy, the client vehicle decomposes the model into inference blocks, and further refines the workload of each inference block into smaller independent subtasks; the subtasks are unloaded to the selected edge node for parallel processing; at the same time, the client vehicle locally calculates the same inference block to quickly recover the service when unloading fails; S300: The client vehicle iteratively executes S200 for each inference block until the inference of the entire model is completed; In optimizing the overall task, the following minimization problem representation is used to minimize the average collaborative inference delay: s.t. The recursive formula of the expected collaborative inference delay is: Considering the scenarios of successful and failed unloading, combined with the unloading success rate binary tree, the general expression of the expected delay is: Defining I(·) as an indicator function, which represents whether the unloading process of the edge node is successful, then the updated unloading success rate of the edge node n is defined as: α is the learning rate, the process of determining the optimal allocation strategy of edge node and the best allocation scheme of inference block segmentation and load division includes: | A v | h ∣Lm∣ | where 1, 2, …, T is the sequence of inference tasks generated by the client vehicle, A v is the set of allocated edge nodes, B m is the inference block partitioning strategy, is the offloading strategy, CID t is the collaborative inference latency of task t, m is the deep neural network model deployed on the client vehicle, is the kth inference block of the model starting from the ith layer and ending at the jth layer, is the execution latency of the inference block under the task offloading path, is the execution latency of the inference block under the client vehicle v local computing path, is the output row number of the last layer of the model, L m is the set of layer indices of the model; S700. Local calculation: for each inference block, the client vehicle simultaneously executes the task unloading path and the local calculation path, and updates the unloading success rate of each edge node; if the result of the unloading path is received before the local calculation is completed, the inference block is considered complete, the local calculation terminates, and the initial unloading strategy is followed to execute the subsequent inference block; if the unloading path fails due to system dynamics, the local calculation completes the block, and the subsequent inference block is rescheduled as a new task until completion.

2. The elastic collaborative inference method in a dynamic vehicular networking environment according to claim 1, characterized in that: The CID is obtained from the collaborative inference delay model, and the calculation method of the collaborative inference delay CID is:

3. The elastic collaborative inference method in a dynamic vehicular networking environment according to claim 2, characterized in that: S500. Initialization: Since the actual value of the offloading success rate Pr(n) cannot be known in advance, the estimated value is used denotes the offloading success rate of the edge node observed by the client vehicle, initialized to S600. Dynamic programming: sort all available edge nodes in descending order according to θ(n), then iterate top_N from 1 to min(|h |Lm| |,|N v |), incrementally select top_N edge nodes according to the sorted queue, then determine the inference block partitioning scheme based on dynamic programming, determine the load partitioning allocation scheme within the inference block according to the edge node computing capability, finally merge some inference blocks according to the offloading success rate of each edge node, and finally obtain the offloading scheme with the lowest inference delay and the highest offloading success rate; θ(n) is the capability of edge node n; N v is the set of available edge nodes; The inference delay of the inference block depends on the local calculation path and the task unloading path, where I(·)∈{0,1} is used to determine whether the unloading process of the edge node is successful.

4. The elastic collaborative inference method in a dynamic vehicular networking environment according to claim 3, characterized in that:

5. The elastic collaborative inference method in a dynamic vehicular networking environment according to claim 3, characterized in that: for inference blocks inference latency of the inference block.

6. The elastic collaborative inference method in a dynamic vehicular networking environment according to claim 5, characterized in that: Inference blocks Inference latency of the inference blocks The computation is:

7. The elastic collaborative inference method in a dynamic vehicular networking environment according to claim 4, characterized in that: In optimizing the overall task, the optimal allocation strategy of edge node and the best allocation scheme of inference block segmentation and load division are determined according to the following strategy: Local computation latency The inference process is: where c v is the computing power of the client vehicle v, f k is the computing overhead of the inference block . ​ Offload latency The inference process is as follows: wherein, uploading latency for inputting data to the inference block, edge node processing latency, aggregating latency for output results at the end of the inference block. ​ Upload latency of input data The calculation is as follows: wherein, denotes the input matrix height of the inference block, r vn is the transmission rate between the customer vehicle v and the edge node n; Latency of edge node processing inference task block The calculation is as follows: Output result aggregation latency at end of inference block The calculation is as follows: ​ ​ For the assigned set of edge nodes A v , define the capabilities of edge node n in A as: where c n Compute the edge node capability, is the offloading success rate of edge node n observed by the client vehicle, is a monotonically decreasing function in the range [1, +∞); Within each inference block, considering the heterogeneity of the selected edge node computing capabilities, the output matrices are allocated in a way to balance the workload and guarantee their return in the same time with the corresponding outputs; The success rate of the edge node n to return the assigned inference fragment within the time threshold is defined as Pr(n), then the success rate of the inference block offloading is defined as: ​ When the kth inference block executes successfully in the offloading path, the latency is the sum of its execution latency in the offloading path and the expected latency of the subsequent inference blocks; when the kth inference block fails to execute in the offloading path, the kth and subsequent inference blocks will execute locally.

Images