Internet of vehicles distributed speculation sampling architecture and method based on end-side cooperation

Abstract

The invention relates to a distributed speculation sampling architecture and method for the Internet of Vehicles based on end-side cooperation, a lightweight draft model is deployed at a vehicle end, a verification large model is deployed at an edge end, and when the edge end receives a reasoning request, an optimal vehicle is dynamically selected to execute a reasoning task according to the bandwidth, computing power, model precision and communication delay of each vehicle. The vehicle end adaptively adjusts a token uploading window according to the real-time bandwidth and precision requirements in the reasoning process and selectively uploads part of reasoning results based on importance indexes, and the edge end dynamically adjusts the verification proportion according to the bandwidth state and the precision constraint, executes verification and result fusion on the key token, and optimizes the uploading strategy of the vehicle end through a feedback mechanism. According to the method, the communication load and verification delay are remarkably reduced on the premise of ensuring the reasoning precision, large-model efficient distributed reasoning in a multi-vehicle cooperative environment is realized, and the method has the advantages of high real-time performance, low bandwidth occupation and high expandability.

Description

Technical Field

[0001] This invention relates to the fields of vehicle networking, edge computing, artificial intelligence inference and intelligent transportation systems, specifically to a distributed speculative sampling architecture for vehicle networking based on edge-end collaboration and its inference method, which can be applied to scenarios such as multi-vehicle collaborative inference, generative AI edge computing and real-time intelligent decision-making. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure, and these statements may constitute prior art. In the process of developing this invention, the inventors discovered at least the following problems in the prior art.

[0003] Currently, with the rapid development of intelligent driving and vehicle networking technologies, vehicles need to simultaneously complete complex tasks such as environmental perception, semantic understanding, path planning, and human-machine interaction during driving. These tasks often rely on artificial intelligence models with high computational complexity and large-scale parameters.

[0004] Traditional vehicle-to-everything (V2X) inference primarily employs two approaches: one involves the vehicle independently executing inference tasks, deploying models locally for real-time inference. This reduces communication latency for data uploads, but its limited computing power and storage capacity make it difficult to support large-scale deep learning model computations, resulting in insufficient inference accuracy. The other approach involves uploading raw perception data to edge servers or the cloud, where high-performance servers perform inference and return results. While this approach ensures model complexity and inference accuracy, in highly dynamic traffic environments, bandwidth constraints, fluctuating link quality, and high vehicle speeds can easily lead to data transmission congestion and inference delays, impacting real-time performance and security. As large-scale models are increasingly used in V2X scenarios, traditional single-end or centralized inference architectures are no longer sufficient to meet the dual demands of real-time performance and bandwidth.

[0005] Edge-end collaborative inference has become a research hotspot in recent years. Its basic idea is to dynamically divide tasks between the vehicle and the edge, with the vehicle executing the lightweight part and the edge executing the complex part to reduce latency. However, existing edge-end collaborative inference mechanisms still have the following shortcomings: First, the task division between the vehicle and the edge is usually static, lacking the ability to dynamically schedule tasks based on network conditions, vehicle computing power, and task complexity. Second, the communication process still requires uploading complete inference features or intermediate results, resulting in high bandwidth consumption. Third, the edge performs full verification of the uploaded results, ignoring the differences in importance between different tokens or intermediate features, leading to unnecessary verification overhead. Finally, existing methods lack a feedback-based closed-loop control mechanism; the vehicle cannot adaptively adjust its upload strategy based on edge feedback information, making it difficult to achieve the optimal balance between inference accuracy and communication latency in complex dynamic environments.

[0006] On the other hand, speculative sampling, as a mechanism to accelerate inference using large models, can significantly reduce generation time by generating candidate results using lightweight draft models and selectively validating them using large models. However, a mature end-edge collaborative implementation framework has not yet been formed in the distributed scenarios of connected vehicles. How to establish an efficient distributed speculative sampling mechanism between vehicles and edge nodes, and dynamically control the communication load and verification ratio while ensuring accuracy, has become a major technical challenge for realizing intelligent inference in connected vehicles.

[0007] Therefore, there is an urgent need to propose a distributed inference method that combines speculative sampling with an edge-end collaborative architecture, enabling vehicle-end and edge-end to achieve low-latency, high-precision joint inference under bandwidth-constrained and heterogeneous computing conditions, thereby meeting the real-time requirements of intelligent driving and vehicle-road cooperative applications. Summary of the Invention

[0008] In view of the above problems, the purpose of this invention is to solve some of the problems in the prior art, or at least alleviate these problems.

[0009] A distributed speculative sampling method for vehicle-to-everything (V2X) networks based on edge-end collaboration includes the following steps:

[0010] Inference request vehicle A sends an inference request to edge node E within its coverage area;

[0011] Edge node E receives the inference request and selects the optimal inference execution vehicle B to perform the inference task based on the vehicle status table.

[0012] After receiving the inference task, vehicle B runs a draft model locally to generate multiple tokens as candidate inference results and calculates the importance index I for each token. t ;

[0013] The inference execution vehicle B adaptively adjusts the token upload window based on real-time bandwidth status and task accuracy requirements, and based on importance index I. t Selectively upload key tokens and their trust information;

[0014] Edge node E performs adaptive verification on the token uploaded by inference execution vehicle B and merges them to generate the final inference result;

[0015] Edge node E feeds back the final inference result to the inference requesting vehicle A for local decision-making.

[0016] Edge node E selects the optimal inference execution vehicle B based on the vehicle state table to perform the inference task, including the following steps:

[0017] Edge node E broadcasts a status request packet to all active vehicles, and vehicle nodes periodically return status information. The status information includes vehicle ID, CPU usage (CPU_Usage) and GPU usage of the vehicle's computing unit, real-time bandwidth between the vehicle and the edge node, round-trip latency between the vehicle and the edge node, model inference accuracy of the vehicle's historical tasks, and percentage of remaining computing resources or computing power of the vehicle.

[0018] Based on the collected vehicle status information, edge node E comprehensively considers the vehicle's bandwidth, latency, computational load, historical accuracy, and remaining computing power to calculate the task execution cost function for each vehicle node:

[0019]

[0020] Where α1~α5 are parameters, B v L represents the vehicle's real-time bandwidth. v Acc represents the communication latency between the vehicle and the edge node. hist For historical accuracy parameters, Load v E represents the vehicle's current computing load. v Indicates the percentage of remaining computing resources for the vehicle;

[0021] Based on the calculated task execution cost function for each vehicle node, edge node E sorts all vehicles and selects the vehicle with the lowest cost as the primary task node, i.e., inference execution vehicle B. At the same time, edge node E will continue to update the state of the primary task execution vehicle and the second-best backup vehicle so that it can quickly switch when the primary execution vehicle cannot complete the task.

[0022] After selecting the primary task execution vehicle, edge node E sends the task scheduling results to inference execution vehicle B to execute the inference task.

[0023] Furthermore, when edge node E keeps updating the state of the main task node, if it detects that the inference execution vehicle B can no longer execute the inference task, it initiates the task migration mechanism, selects the vehicle of the backup node to take over the task, and synchronizes the current information to the vehicle of the backup node.

[0024] The inference execution vehicle B locally runs a draft model to generate multiple candidate inference result tokens and calculates the importance index I for each token. t This includes the following steps:

[0025] Inference execution vehicle B runs a lightweight draft model M1 locally to generate preliminary inference results, outputting a candidate token sequence T = {t1, t2, ..., t}. n} and calculate the probability distribution for each token:

[0026] P(ti ) = softmax(z i )

[0027] Among them, z i The logit vector output by the draft model reflects the model's relative preference for different candidate words;

[0028] Simultaneously, the inference execution vehicle B calculates the confidence level Conf(t) for each token. i )=max(P(t i Attention weights Att(t) i And generate an importance metric for each token based on the following formula:

[0029] I t =λ1·Conf(t) i )+λ2·Att(t i ).

[0030] Wherein, λ1 and λ2 are the confidence weight coefficient and attention weight coefficient, respectively, used to balance the influence of token confidence and attention features in importance assessment.

[0031] Furthermore, the inference execution vehicle B adaptively adjusts the token upload window based on real-time bandwidth status and task accuracy requirements, and based on the importance index I. t Selectively upload key tokens and their trust information, including the following steps:

[0032] Inference execution vehicle B based on real-time bandwidth status B v Time delay Δt and mission accuracy target Acc target Adaptively adjust the upload window w t ;

[0033] The adjustment strategy for the upload window is as follows:

[0034]

[0035] Where η1 and η2 are window adjustment coefficients; ε1 is the window contraction factor; ε2 is the window expansion factor; B min B is the minimum bandwidth threshold required for normal transmission. thr This is a bandwidth stability threshold used to determine whether the communication channel is in a high-quality state; T link T is the link hold-up time, which is the expected duration for which the vehicle and the edge node maintain a communication connection. min Minimum stable time threshold for the path; when T link A value less than this indicates that the connection is about to be interrupted; T stable This is the upper limit threshold for link stability, indicating that the current link is in a stable, high-quality communication state.

[0036] Inference execution vehicle B screen importance index I t ≥μ t The tokens are sorted in descending order of importance, and the top w are selected. t Each token is uploaded as a key token; among them, μ t Threshold for importance indicators;

[0037] When communication delays or network congestion are detected, inference execution vehicle B automatically reduces the upload window and prioritizes the transmission of high-confidence tokens; when bandwidth is sufficient, the window is appropriately expanded to improve verification coverage.

[0038] If a link switch is imminent due to link instability or vehicle movement, the inference execution vehicle B will urgently upload the remaining high-confidence tokens in advance.

[0039] Edge node E performs adaptive verification on the token uploaded by inference execution vehicle B and fuses the results to generate the final inference result, including the following steps:

[0040] After receiving the token data uploaded by the inference execution vehicle B, edge node E performs hierarchical verification on the token to compare consistency or check precision and determine whether to accept or reject the token result; the uploaded tokens are divided into high-confidence tokens, medium-confidence tokens and low-confidence tokens.

[0041] For high-confidence tokens (I t ≥θ H The process involves rapid consistency verification, comparing only the consistency between the draft inference results and the edge large model inference results; if they are consistent, the output is directly confirmed; if they are inconsistent, they are marked as "requires review".

[0042] For Zhongzhixin token(θ) L ≤I t <θ H The system verifies tokens by comparing their similarity to contextual matching; if the similarity is less than 0.8, it proceeds to the deep correction module.

[0043] For low-confidence tokens (I t <θ L ), then enter the full model inference path and use the large model M2 for deep inference correction;

[0044] Where, θ H θ represents the high confidence threshold. L Indicates a low confidence threshold;

[0045] Edge node E merges all verified tokens and calculates the final inference result.

[0046] Furthermore, in multi-vehicle scenarios, the edge node E can receive token sequences from different vehicles in parallel and synthesize them together through a weighted fusion mechanism to generate the final inference result.

[0047] The aforementioned distributed speculative sampling method for vehicle-to-everything (V2X) based on edge-to-edge collaboration also includes a feedback mechanism, comprising the following steps:

[0048] After the task is completed, edge node E simultaneously evaluates the verification accuracy and system latency, and calculates feedback parameters based on the real-time status, including the verification ratio κ, bandwidth utilization η, and latency compensation δ. These feedback parameters are sent back to the inference execution vehicle B in the form of control signals to guide it in adjusting the upload window size and token threshold in subsequent inference tasks, thereby achieving closed-loop adaptive control.

[0049] Edge node E is determined based on the actual inference accuracy Acc. real With target accuracy Acc target The difference is used to update the historical accuracy parameter Acc of vehicle B. hist The system feeds back model accuracy, average latency, and communication volume metrics to the cloud-based scheduling center for long-term optimization. If a specific vehicle performs stably across multiple tasks, it is designated as a priority scheduling node. If a vehicle frequently disconnects or fails to meet accuracy standards, its scheduling priority is reduced.

[0050] An architecture employing the aforementioned edge-cooperative distributed speculative sampling method for vehicle-to-everything (V2X) networks includes:

[0051] The edge task scheduling module is used to dynamically select the optimal vehicle to perform the inference task based on the bandwidth, computing power, model accuracy and communication latency of each vehicle when an inference request is received.

[0052] The vehicle-side draft inference module is used to run a lightweight draft model and perform local inference on the input request to generate candidate tokens;

[0053] The vehicle-side adaptive control module is used to adjust the token upload window and verification ratio of the vehicle-side in real time based on feedback signals from the edge terminal, so as to achieve the joint optimization of communication, computing and accuracy.

[0054] The edge verification module is used to deploy and verify the large verification model, verify some candidate tokens uploaded by the vehicle, and merge them to generate the final inference result.

[0055] Furthermore, based on the bandwidth status and latency changes between the vehicle and the edge node, the upload rate and data batch size are adaptively controlled to balance inference accuracy and transmission latency.

[0056] It also includes a feedback control module that updates the upload strategy and importance threshold on the vehicle side in real time, enabling the system to dynamically adapt to link changes in complex traffic environments and achieve low-latency, high-precision collaborative reasoning.

[0057] The task scheduling module maintains a vehicle status table, records the bandwidth, computing power, model accuracy, and average latency information of each vehicle, and dynamically selects the vehicle that meets the inference accuracy and has the lowest inference latency to perform the inference task based on the comprehensive latency prediction model.

[0058] When generating candidate tokens, the vehicle-side draft reasoning module calculates the importance index of each token. The index includes, but is not limited to, entropy value, confidence difference, or attention weight, and only uploads tokens whose importance is higher than a preset threshold.

[0059] The adaptive control module dynamically adjusts the upload window parameters based on the communication latency and accuracy status fed back from the edge end through a threshold triggering mechanism, and performs window updates in conjunction with the cache status of the vehicle end, thereby achieving closed-loop control between the edge and the end.

[0060] The edge verification module is based on network bandwidth B v Historical acceptance rate r v and task uncertainty U t Through the functional relationship κ=f(B) v ,r v U t The verification ratio κ is dynamically determined, and the upload and verification loads are automatically adjusted when bandwidth is insufficient or task complexity is high.

[0061] The present invention has the following beneficial effects:

[0062] This invention enables distributed collaborative reasoning between draft and validation models in the bandwidth-constrained and dynamically changing vehicular network environment. It significantly reduces communication data volume and validation latency while maintaining inference accuracy, and facilitates dynamic task scheduling among multiple vehicles. This invention features low bandwidth consumption, high robustness, and strong scalability, and can be widely applied in intelligent driving, vehicle-to-everything (V2X) communication, and edge computing scenarios. Attached Figure Description

[0063] Figure 1 This is the overall system architecture diagram of the present invention;

[0064] Figure 2 This is a system flowchart of the present invention. Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings. The embodiments of the present invention are only used to illustrate the present invention and not to limit the present invention. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the technical concept of the present invention should be included within the scope of the present invention.

[0066] Existing intelligent inference systems for connected vehicles typically employ either vehicle-side independent inference or edge-centralized processing. However, limitations in vehicle computing power, bandwidth, and communication latency hinder efficient, low-latency collaborative inference in heterogeneous multi-vehicle environments. Furthermore, fixed-ratio result uploading and verification mechanisms result in high communication overhead and poor real-time performance. To address this, the applicant has designed a distributed speculative sampling architecture and method for connected vehicles based on edge-end collaboration. This invention, on the one hand, fully utilizes the local computing power of vehicles by deploying lightweight draft models at the vehicle end to achieve rapid candidate inference generation; on the other hand, it deploys a high-precision verification model at the edge to verify and fuse uploaded key inference results, thereby significantly reducing communication load and response latency while maintaining inference accuracy. This invention solves the shortcomings of existing connected vehicle inference systems, such as uneven computational load, high communication bandwidth consumption, and large inference latency. It significantly reduces communication load and verification latency while ensuring inference accuracy, enabling efficient distributed inference of large models in multi-vehicle collaborative environments. It boasts advantages such as high real-time performance, low bandwidth consumption, and strong scalability.

[0067] like Figure 1 As shown, the distributed speculative sampling inference architecture for vehicle-to-everything (V2X) based on edge collaboration proposed in this invention includes a vehicle-side draft model, edge computing nodes, and a base station collaborative unit.

[0068] Let A be the inference request vehicle, B be the inference execution vehicle, and edge node E serve as the scheduling and verification node. A lightweight draft model M1 is deployed on the vehicle side, and a large verification model M2 is deployed on the edge side. Edge nodes can consist of roadside units (RSUs), 5G small base stations, or edge servers with computing capabilities. Vehicle nodes and edge nodes are interconnected via wireless communication links to form a dynamically collaborative distributed inference system. The services provided by this invention include various intelligent processing tasks such as scene semantic recognition, image or text generation, and driving assistance decision-making.

[0069] like Figure 2 As shown, the present invention consists of the following stages.

[0070] 1. Task scheduling phase.

[0071] When vehicle A needs to perform an intelligent inference task, it first sends an inference request to edge node E within its coverage area. Edge node E then calculates the task cost function C for each vehicle based on the bandwidth, computing power, communication latency, and model accuracy information of the currently connected vehicles. vThe vehicle with the lowest cost and best communication quality is selected as the inference execution vehicle B, which serves as the draft model execution node. Specifically:

[0072] 1) Inference Request: Vehicle A detects a task trigger signal (e.g., camera recognition request, navigation scene analysis, or voice dialogue command) and generates a task description message. The task message includes:

[0073] Task_ID: Task number;

[0074] Task_Type: Task category;

[0075] Input_Size: Size of the input data;

[0076] Deadline: Maximum allowable response latency;

[0077] Priority: Task priority;

[0078] QoS_Target: The threshold for required inference accuracy.

[0079] 2) Inference request vehicle A sends a task request to its covered edge node E via the C-V2X interface. After receiving the request, edge node E creates a new task entry in the task management table and begins collecting vehicle status information. C-V2X (Cellular Vehicle-to-Everything) is an abbreviation for cellular vehicle-to-everything communication technology.

[0080] 3) Edge node E broadcasts a status request packet to all active vehicles, and vehicle nodes periodically return status information, including:

[0081] Vehicle ID: A unique identifier for a vehicle used to identify the vehicle participating in the mission.

[0082] CPU_Usage: CPU utilization of the vehicle's computing unit, used to assess the vehicle's computing capabilities.

[0083] GPU_Usage: If the vehicle is equipped with a GPU, this provides the GPU usage rate so that edge nodes can evaluate the vehicle's computing performance.

[0084] B_v: Real-time bandwidth between the vehicle and the edge node, affecting the upload and download speed of tasks.

[0085] L_v: Round-trip time (RTT) between the vehicle and the edge node, used to evaluate communication latency.

[0086] Acc_hist: Model inference accuracy for vehicle history tasks, used to evaluate the performance of the vehicle-side model.

[0087] Energy_Remain: Percentage of the vehicle's remaining computing resources or computing power, used to measure the vehicle's current processing capacity.

[0088] Edge node E aggregates all collected vehicle state information into a vehicle state matrix V(t), where each row represents the current state information of a vehicle node, for use in subsequent task scheduling and resource allocation. The vehicle state matrix V(t) is:

[0089]

[0090] 4) After collecting all vehicle status information, edge node E will comprehensively consider the vehicle's bandwidth, latency, computational load, historical accuracy, and remaining computing power to calculate the task execution cost function for each vehicle node. Then, based on this cost function, all vehicles are sorted, and the vehicle with the lowest cost is selected as the main execution node (i.e., inference execution vehicle B). Here, α1~α5 are parameters, and B... v L represents the vehicle's real-time bandwidth. v Acc represents the communication latency between the vehicle and the edge node. hist For historical accuracy parameters, Load v E represents the vehicle's current computing load. v This indicates the proportion of remaining computing resources for the vehicle. Meanwhile, edge node E will continue to update the status of the primary executing vehicle and backup nodes C1 and C2, enabling rapid switching if the primary executing vehicle is unable to complete the task. The task scheduling results will be sent to the inference executing vehicle B, including parameters such as task identifier, upload window size, initial threshold, and accuracy target.

[0091] After receiving an external task request (such as image recognition or speech reasoning), inference execution vehicle B generates preliminary inference results and corresponding importance tokens using a local lightweight draft model, and sends some high-confidence tokens to the edge node via a wireless link. (See the draft inference stage for details)

[0092] 2. Draft reasoning stage.

[0093] After the vehicle B receives the inference task, a lightweight draft model M1 is run locally to generate preliminary inference results, outputting multiple candidate token sequences, and simultaneously calculating the importance index I for each token. t This metric, which can be calculated based on confidence, attention weight, or prediction entropy, measures the token's contribution to the final inference result. Specifically:

[0094] After receiving the task, vehicle B will begin executing its local draft model M1. Draft model M1 is a lightweight inference model that can run in real-time on the vehicle's CPU or GPU, quickly generating preliminary inference results for the task. During inference execution, vehicle B processes the input data to generate a candidate token sequence T = {t1, t2, ..., t...}. n} and calculate the probability distribution for each token:

[0095] P(t i ) = sofmtax(z i )

[0096] Among them, z i The logit vector output by the draft model reflects the model's relative preference for different candidate words.

[0097] Simultaneously, the inference execution vehicle B calculates the confidence level Conf(t) for each token. i )=max(P(t i Attention weights Att(t) i And generate an importance metric for each token based on the following formula:

[0098] I t =λ1·Conf(t) i )+λ2·Att(t i )

[0099] Wherein, λ1 and λ2 are the confidence weight coefficient and attention weight coefficient, respectively, used to balance the influence of token confidence and attention features in importance assessment.

[0100] All generated tokens and their importance metrics are stored in a local cache, Buffer_T, for later upload. Buffer_T is used to temporarily store each token generated by the vehicle-side draft model and its corresponding importance metrics.

[0101] 3. Selective upload and adaptive upload control stage.

[0102] During the inference process, the inference execution vehicle B determines the real-time bandwidth status based on the inference execution vehicle B. v Time delay Δt and mission accuracy target Acc target Adaptively adjust the upload window w tOnly key tokens with importance above a threshold θ and their confidence information are uploaded. When communication delays or network congestion are detected, inference execution vehicle B automatically reduces the upload window, prioritizing the transmission of high-confidence tokens; when bandwidth is sufficient, the window is appropriately expanded to improve verification coverage. To further conserve bandwidth, inference execution vehicle B can use token compression or aggregation to encapsulate the uploaded data.

[0103] Specifically:

[0104] 1) The upload window size of vehicle B during inference execution is based on bandwidth B. v Time delay Δt and target accuracy Acc target Multi-parameter joint optimization function w t =g(B v ,Δt,Acc tarfet It is calculated in real time and used to achieve a dynamic balance between upload rate and accuracy constraints.

[0105] The adjustment strategy for the upload window is as follows:

[0106]

[0107] Where ε1 and ε2 are window adjustment coefficients; ε1 is the window contraction factor, and ε2 is the window expansion factor; B min B is the minimum bandwidth threshold required for normal transmission. thr This is a bandwidth stability threshold used to determine whether the communication channel is in a high-quality state; T libk T is the link hold-up time, which is the expected duration for which the vehicle and the edge node maintain a communication connection. min Minimum stable time threshold for the path; when T link A value less than this indicates that the connection is about to be interrupted; T stable This is the upper limit threshold for link stability, indicating that the current link is in a stable and high-quality communication state.

[0108] 2) Vehicle B selects importance index I from the local cache Buffer_T. t ≥μ t The token, μ t The importance index thresholds are used, and the values ​​are sorted in descending order of importance. The top w values ​​are selected. t Vehicle B will upload each token. If the link is about to switch due to instability or vehicle movement, vehicle B will upload the remaining high-confidence tokens in advance.

[0109] 4. Edge node verification and fusion stage.

[0110] Verification Phase: Edge nodes possess large-scale model inference and resource scheduling capabilities. After receiving partial inference results uploaded by inference execution vehicle B, edge node E allocates computing resources according to the task scheduling module and performs verification operations on the edge large model. The large verification model M2 performs tiered verification on the received tokens: high-confidence tokens are processed with a fast pass strategy, performing only a lightweight comparison; low-confidence tokens undergo deep verification by calling the complete large model verification module, and the inference branches are corrected. After performing consistency comparison or precision verification on each token, a decision is made on whether to accept or reject the token result.

[0111] Fusion Phase: The verification results are integrated by the fusion module to generate the final inference result output.

[0112] Specifically:

[0113] 1) After receiving the token data uploaded by vehicle B, edge node E first performs hierarchical processing on the token:

[0114] High-confidence token (I) t ≥θ H The system performs rapid consistency verification by comparing the consistency between the draft inference results and the edge large model inference results. If they are consistent, the output is directly confirmed; if they are inconsistent, they are marked as "requires review".

[0115] Zhongzhixin token(θ) L ≤I t <θ H The system verifies tokens by comparing their similarity to context; if the similarity is less than 0.8, it proceeds to the deep correction module.

[0116] Low-confidence token (I t <θ L Then, it enters the full model inference path and uses the large model M2 for deep inference correction.

[0117] Where, θ H θ represents the high confidence threshold. L This indicates a low confidence threshold.

[0118] 2) Edge node E merges all verified tokens and calculates the final inference result. A weighted average method is used to weight the verification results at different levels, and the merged result will be output as the final inference result.

[0119] 5. Feedback phase.

[0120] After the inference is completed, the edge device will directly return the final inference result to the inference requesting vehicle A for local decision-making.

[0121] In addition, after the task is completed, edge node E simultaneously evaluates the verification accuracy and system latency, and calculates feedback parameters based on the real-time status, including the verification ratio κ, bandwidth utilization η, and latency compensation δ. These feedback parameters are sent back to the inference execution vehicle B in the form of control signals to guide it in adjusting the upload window size and token threshold in subsequent inference tasks, thereby achieving closed-loop adaptive control.

[0122] Furthermore, the edge node E is determined based on the actual inference accuracy Acc. real With target accuracy Acc target The difference is used to update the historical accuracy parameter Acc of vehicle B. hist The system feeds back model accuracy, average latency, and communication volume metrics to the cloud-based scheduling center for long-term optimization. If a specific vehicle performs consistently across multiple tasks, it is prioritized for scheduling; if a vehicle frequently experiences connection drops or fails to meet accuracy standards, its scheduling priority is reduced.

[0123] To enable seamless switching of the reasoning process and ensure the continuity of the task, this invention also includes a task migration mechanism.

[0124] When inference execution vehicle B is unable to continue due to degraded communication quality or task overload, edge node E can select another vehicle node to take over the inference task based on the real-time status table, ensuring system continuity and stability. All edge nodes maintain synchronized task status records and vehicle feature tables to support cross-node migration and multi-edge collaboration. Specifically:

[0125] 1) Inference execution: Vehicle B monitors its position, velocity, and orientation vector (Pos(t) = (x(t), y(t)), v(t), θ(t)) in real time using GPS, IMU (Inertial Measurement Unit), and vehicle speed sensor, updating its state information accordingly. Edge node E predicts the link maintenance time between vehicle B and the edge node based on vehicle B's position and motion state (velocity v(t) and orientation θ(t)).

[0126]

[0127] Among them, R cov Let be the communication coverage radius of the edge node, and d(t) represent the real-time distance between vehicle B and its current serving edge node at time t. If T link Less than the safety threshold T min If so, the edge node will trigger the task migration mechanism in advance.

[0128] 2) When the inference execution link of vehicle B is about to be interrupted, edge node E initiates the task migration mechanism, selects backup vehicle C1 to take over the task, and synchronizes information such as the current task status, token cache, and model status to vehicle C1. In this way, the inference process of the task can be seamlessly switched, ensuring the continuity of the task.

[0129] The present invention proposes an architecture for a distributed speculative sampling method for vehicle-to-everything (V2X) networks based on edge-end collaboration, comprising:

[0130] The edge task scheduling module is used to dynamically select the optimal vehicle to perform the inference task based on the bandwidth, computing power, model accuracy and communication latency of each vehicle when an inference request is received.

[0131] The vehicle-side draft inference module is used to run a lightweight draft model and perform local inference on the input request to generate candidate tokens;

[0132] The vehicle-side adaptive control module is used to adjust the token upload window and verification ratio of the vehicle-side in real time based on feedback signals from the edge terminal, so as to achieve the joint optimization of communication, computing and accuracy.

[0133] The edge verification module is used to deploy and verify the large verification model, verify some candidate tokens uploaded by the vehicle, and merge them to generate the final inference result.

[0134] The architecture also includes a vehicle-side communication optimization module, which adaptively controls the upload rate and data batch size based on the bandwidth status and latency changes between the vehicle and the edge node, in order to balance inference accuracy and transmission latency.

[0135] It also includes a feedback control module that updates the upload strategy and importance threshold on the vehicle side in real time, enabling the system to dynamically adapt to link changes in complex traffic environments and achieve low-latency, high-precision collaborative reasoning.

[0136] The task scheduling module maintains a vehicle status table, records the bandwidth, computing power, model accuracy, and average latency information of each vehicle, and dynamically selects the vehicle that meets the inference accuracy and has the lowest inference latency to perform the inference task based on the comprehensive latency prediction model.

[0137] When generating candidate tokens, the vehicle-side draft reasoning module calculates the importance index of each token. The index includes, but is not limited to, entropy value, confidence difference, or attention weight, and only uploads tokens whose importance is higher than a preset threshold.

[0138] The adaptive control module dynamically adjusts the upload window parameters based on the communication latency and accuracy status fed back from the edge end through a threshold triggering mechanism, and performs window updates in conjunction with the cache status of the vehicle end, thereby achieving closed-loop control between the edge and the end.

[0139] The edge verification module is based on network bandwidth B v Historical acceptance rate r v and task uncertainty U t Through the functional relationship κ=f(B) v ,r v U t The verification ratio κ is dynamically determined, and the upload and verification loads are automatically adjusted when bandwidth is insufficient or task complexity is high.

[0140] The system can be extended to multi-edge node collaborative scenarios, with edge nodes sharing verification load and scheduling information to achieve cross-regional vehicle-side collaborative inference.

[0141] This invention deploys a lightweight draft model on the vehicle side and a large verification model on the edge side. When the edge side receives an inference request, it dynamically selects the optimal vehicle to perform the inference task based on the bandwidth, computing power, model accuracy, and communication latency of each vehicle. During the inference process, the vehicle side adaptively adjusts the token upload window according to real-time bandwidth and accuracy requirements, and selectively uploads some inference results based on importance indicators. The edge side dynamically adjusts the verification ratio based on bandwidth status and accuracy constraints, performs verification and result fusion on key tokens, and optimizes the vehicle side's upload strategy through a feedback mechanism. This method can significantly reduce communication load and verification latency while ensuring inference accuracy, enabling efficient distributed inference of large models in a multi-vehicle collaborative environment. It has the advantages of high real-time performance, low bandwidth consumption, and strong scalability.

Claims

1. A distributed speculative sampling method for vehicle-to-everything (V2X) networks based on edge-end collaboration, characterized in that, Includes the following steps: Inference request vehicle A sends an inference request to edge node E within its coverage area; Edge node E receives the inference request and selects the optimal inference execution vehicle B to perform the inference task based on the vehicle status table. After receiving the inference task, vehicle B runs a draft model locally to generate multiple tokens as candidate inference results and calculates the importance index I for each token. t ; The inference execution vehicle B adaptively adjusts the token upload window based on real-time bandwidth status and task accuracy requirements, and based on importance index I. t Selectively upload key tokens and their trust information; Edge node E performs adaptive verification on the token uploaded by inference execution vehicle B and merges them to generate the final inference result; Edge node E feeds back the final inference result to the inference requesting vehicle A for local decision-making.

2. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 1, characterized in that, Edge node E selects the optimal inference execution vehicle B based on the vehicle state table to perform the inference task, including the following steps: Edge node E broadcasts a status request packet to all active vehicles, and vehicle nodes periodically return status information. The status information includes vehicle ID, CPU usage (CPU_Usage) and GPU usage of the vehicle's computing unit, real-time bandwidth between the vehicle and the edge node, round-trip latency between the vehicle and the edge node, model inference accuracy of the vehicle's historical tasks, and percentage of remaining computing resources or computing power of the vehicle. Based on the collected vehicle status information, edge node E comprehensively considers the vehicle's bandwidth, latency, computational load, historical accuracy, and remaining computing power to calculate the task execution cost function for each vehicle node: Where α1~α5 are parameters, B v L represents the vehicle's real-time bandwidth. v Acc represents the communication latency between the vehicle and the edge node. hist For historical accuracy parameters, Load v E represents the vehicle's current computing load. v Indicates the percentage of remaining computing resources for the vehicle; Based on the calculated task execution cost function for each vehicle node, edge node E sorts all vehicles and selects the vehicle with the lowest cost as the primary task node, i.e., inference execution vehicle B. At the same time, edge node E will continue to update the state of the primary task execution vehicle and the second-best backup vehicle so that it can quickly switch when the primary execution vehicle cannot complete the task. After selecting the primary task execution vehicle, edge node E sends the task scheduling results to inference execution vehicle B to execute the inference task.

3. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 2, characterized in that, When edge node E keeps updating the state of the main task node, if it detects that the inference execution vehicle B can no longer execute the inference task, it initiates the task migration mechanism, selects the vehicle of the backup node to take over the task, and synchronizes the current information to the vehicle of the backup node.

4. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 1, characterized in that, The inference execution vehicle B locally runs a draft model to generate multiple candidate inference result tokens and calculates the importance index I for each token. t This includes the following steps: Inference execution vehicle B runs a lightweight draft model M1 locally to generate preliminary inference results, outputting a candidate token sequence T = {t1, t2, ..., t}. n } and calculate the probability distribution for each token: P(t i )=softmax(z i ) Among them, z i The logit vector output by the draft model reflects the model's relative preference for different candidate words; Simultaneously, the inference execution vehicle B calculates the confidence level Conf(t) for each token. i )=max(P(t i Attention weights Att(t) i And generate an importance metric for each token according to the following formula: I t =λ1·Conf(t i )+λ2·Att(t i )。 Wherein, λ1 and λ2 are the confidence weight coefficient and attention weight coefficient, respectively, used to balance the influence of token confidence and attention features in importance assessment.

5. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 1 or 4, characterized in that, The inference execution vehicle B adaptively adjusts the token upload window based on real-time bandwidth status and task accuracy requirements, and based on importance index I. t Selectively upload key tokens and their trust information, including the following steps: Inference execution vehicle B based on real-time bandwidth status B v Time delay Δt and mission accuracy target Acc target Adaptively adjust the upload window w t ; The adjustment strategy for the upload window is as follows: Where ε1 and ε2 are window adjustment coefficients; ε1 is the window contraction factor, and ε2 is the window expansion factor; B min B is the minimum bandwidth threshold required for normal transmission. thr T is a bandwidth stability threshold used to determine whether the communication channel is in a high-quality state. link T is the link hold-up time, which is the expected duration for which the vehicle and the edge node maintain a communication connection. min Minimum stable time threshold for the path; when T link A value less than this indicates that the connection is about to be interrupted; T stable This is the upper limit threshold for link stability, indicating that the current link is in a stable, high-quality communication state; Inference execution vehicle B screen importance index I t ≥μ t The tokens are sorted in descending order of importance, and the top w are selected. t Each token is uploaded as a key token; among them, μ t Threshold for importance indicators; When communication delays or network congestion are detected, inference execution vehicle B automatically reduces the upload window and prioritizes the transmission of high-confidence tokens; when bandwidth is sufficient, the window is appropriately expanded to improve verification coverage. If a link switch is imminent due to link instability or vehicle movement, the inference execution vehicle B will urgently upload the remaining high-confidence tokens in advance.

6. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 1, characterized in that, Edge node E performs adaptive verification on the token uploaded by inference execution vehicle B and fuses the results to generate the final inference result, including the following steps: After receiving the token data uploaded by the inference execution vehicle B, edge node E performs hierarchical verification on the token to compare consistency or check precision and determine whether to accept or reject the token result; the uploaded tokens are divided into high-confidence tokens, medium-confidence tokens and low-confidence tokens. For high-confidence tokens (I t ≥θ H The process involves rapid consistency verification, comparing only the consistency between the draft inference results and the edge large model inference results; if they are consistent, the output is directly confirmed; if they are inconsistent, they are marked as "requires review". For Zhongzhixin token(θ) L ≤I t <θ H The system verifies tokens by comparing their similarity to contextual matching; if the similarity is less than 0.8, it proceeds to the deep correction module. For low-confidence tokens (I t <θ L ), then enter the full model inference path and use the large model M2 for deep inference correction; Where, θ H θ represents the high confidence threshold. L Indicates a low confidence threshold; Edge node E merges all verified tokens and calculates the final inference result.

7. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 6, characterized in that, In a multi-vehicle scenario, the edge node E can receive token sequences from different vehicles in parallel and synthesize them together through a weighted fusion mechanism to generate the final inference result.

8. The distributed speculative sampling method for vehicle-to-everything (V2X) networks based on end-edge collaboration according to claim 1, characterized in that, It also includes a feedback mechanism, comprising the following steps: After the task is completed, edge node E simultaneously evaluates the verification accuracy and system latency, and calculates feedback parameters based on the real-time status, including the verification ratio κ, bandwidth utilization η, and latency compensation δ. These feedback parameters are sent back to the inference execution vehicle B in the form of control signals to guide it in adjusting the upload window size and token threshold in subsequent inference tasks, thereby achieving closed-loop adaptive control. Edge node E is determined based on the actual inference accuracy Acc. real With target accuracy Acc target The difference is used to update the historical accuracy parameter Acc of vehicle B. hist The system feeds back model accuracy, average latency, and communication volume metrics to the cloud-based scheduling center for long-term optimization. If a specific vehicle performs stably across multiple tasks, it is designated as a priority scheduling node. If a vehicle frequently disconnects or fails to meet accuracy standards, its scheduling priority is reduced.

9. An architecture employing the end-edge collaborative distributed speculative sampling method for vehicle-to-everything (V2X) networks as described in any one of claims 1 to 8, characterized in that, include: The edge task scheduling module is used to dynamically select the optimal vehicle to perform the inference task based on the bandwidth, computing power, model accuracy and communication latency of each vehicle when an inference request is received. The vehicle-side draft inference module is used to run a lightweight draft model and perform local inference on the input request to generate candidate tokens; The vehicle-side adaptive control module is used to adjust the token upload window and verification ratio of the vehicle-side in real time based on feedback signals from the edge terminal, so as to achieve the joint optimization of communication, computing and accuracy. The edge verification module is used to deploy and verify the large verification model, verify some candidate tokens uploaded by the vehicle, and merge them to generate the final inference result.

10. The architecture according to claim 9, characterized in that, It also includes a vehicle-side communication optimization module, which adaptively controls the upload rate and data batch size based on the bandwidth status and latency changes between the vehicle and the edge node, in order to balance inference accuracy and transmission latency. It also includes a feedback control module that updates the upload strategy and importance threshold on the vehicle side in real time, enabling the system to dynamically adapt to link changes in complex traffic environments and achieve low-latency, high-precision collaborative reasoning. The task scheduling module maintains a vehicle status table, records the bandwidth, computing power, model accuracy, and average latency information of each vehicle, and dynamically selects the vehicle that meets the inference accuracy and has the lowest inference latency to perform the inference task based on the comprehensive latency prediction model. When generating candidate tokens, the vehicle-side draft reasoning module calculates the importance index of each token. The index includes, but is not limited to, entropy value, confidence difference, or attention weight, and only uploads tokens whose importance is higher than a preset threshold. The adaptive control module dynamically adjusts the upload window parameters based on the communication latency and accuracy status fed back from the edge end through a threshold triggering mechanism, and performs window updates in conjunction with the cache status of the vehicle end, thereby achieving closed-loop control between the edge and the end. The edge verification module is based on network bandwidth B v Historical acceptance rate r v and task uncertainty U t Through the functional relationship κ=f(B v ,r v U t The verification ratio κ is dynamically determined, and the upload and verification loads are automatically adjusted when bandwidth is insufficient or task complexity is high.

Images