An FPGA acceleration method and device for unmanned vehicle VLA planning and scheduling

Abstract

The application discloses an FPGA acceleration method and device for unmanned vehicle VLA planning and scheduling, and belongs to the technical field of unmanned vehicle edge intelligent sensing and hardware acceleration. A current period environment and a vehicle state token are acquired, a comprehensive risk value is calculated according to a vehicle speed obstacle, and a decoding time delay budget is determined; a scheduling strategy is generated according to this, a VLA decoding KV cache is divided into three layers of safety, heat and cold, and is respectively loaded on-chip or off-chip; a reconfigurable array is configured according to a resource ratio, multi-branch forward decoding is executed in parallel, a candidate action and a trajectory sequence are generated; the trajectory sequence is input into kinematics and collision constraints solidified in a pipeline, and illegal sequences are verified and removed in parallel; a passer is scored, an optimal output is selected, and a high hit rate cache block is reserved on-chip for a next period. The application solves the problems of high reasoning delay and real-time embedding of safety constraints, and realizes efficient acceleration and reliable decision-making.

Description

Technical Field

[0001] This invention relates to the field of edge intelligent perception and hardware acceleration technology for unmanned vehicles, and in particular to an FPGA acceleration method and device for VLA planning and scheduling of unmanned vehicles. Background Technology

[0002] With the development of autonomous driving technology, autonomous vehicles need to perceive surrounding targets, road structures, drivable areas, and the dynamic status of traffic participants in complex road environments in real time. Existing autonomous vehicles typically acquire environmental information from multiple sources, such as cameras, LiDAR, millimeter-wave radar, positioning and navigation equipment, high-precision maps, and vehicle status sensors, and complete environmental understanding through multimodal fusion perception methods.

[0003] In recent years, multimodal large-scale models and end-to-end perception models have been increasingly applied to perception tasks in autonomous vehicles. These models can uniformly model heterogeneous information such as visual images, point clouds, radar, maps, and vehicle status, and output perception results such as target recognition, road semantic understanding, and spatial occupancy prediction. However, these models typically require processing a large number of visual, spatial, and state terms during inference, involving cross-modal feature alignment, attention computation, and multi-layer network mapping, resulting in high computational load, frequent storage access, and high inference latency. Directly deploying them at the edge of autonomous vehicles is easily limited by onboard computing power, power consumption, storage bandwidth, and real-time requirements.

[0004] FPGAs, with their reconfigurability, low power consumption, strong parallel computing capabilities, and high pipeline efficiency, can be used to accelerate inference of edge perception models in autonomous vehicles. Existing FPGA-based perception inference acceleration solutions mostly target fixed network structures, fixed operators, or single-modal models, typically employing preset computation paths, fixed computational precision, and static resource allocation. These solutions struggle to adapt to the dynamic inference requirements of large multimodal perception models under varying road scenarios, sensor quality, and hardware load conditions. When changes occur in road environment complexity, perception quality of each modality, lexical importance, and FPGA on-chip resource status, existing solutions cannot dynamically adjust the modal data, number of lexical units, computational precision, cache reuse methods, and parallel execution methods involved in inference based on the current perception task. This can easily lead to redundant computation, increased memory access overhead, insufficient on-chip resource utilization, and decreased real-time performance of edge inference.

[0005] Therefore, existing technologies still suffer from problems such as high inference latency, high power consumption, excessive redundant computation, high storage access overhead, and insufficient hardware resource utilization efficiency in complex scenarios for multimodal perception at the edge of autonomous vehicles. There is an urgent need for an FPGA inference acceleration chip and method that can combine the perception scenario, the importance of modal lexical units, and the FPGA hardware status for adaptive inference control, so as to improve the real-time performance, energy efficiency, and deployment adaptability of multimodal environmental perception at the edge of autonomous vehicles. Summary of the Invention

[0006] The present invention aims to at least partially solve one of the technical problems in the related art.

[0007] To address this, this invention proposes an FPGA-accelerated method for VLA planning and scheduling of unmanned vehicles. This method acquires multi-source input data for the current planning cycle, including environmental perception terms and vehicle state terms. Based on vehicle speed and obstacle information, a comprehensive risk value is calculated, and a decoding delay budget is determined. A scheduling strategy is generated, including the number of candidate action branches and the allocation ratio of computational resources. According to this strategy, the key-value (KV) cache is divided into three layers: safe, hot, and cold. The safe and hot layers are loaded into the FPGA on-chip cache, and a reconfigurable parallel computing array is configured according to the resource allocation ratio. Multiple candidate action branches are decoded in parallel to generate candidate action term sequences and corresponding candidate trajectory term sequences. The candidate trajectory term sequences are input into the safety constraint verification logic embedded in the FPGA hardware pipeline for parallel hard constraint verification. Sequences that do not meet kinematic continuity or collision safety distance requirements are eliminated. The verified sequences are scored for consistency, and the optimal result is output. Simultaneously, high-hit-rate KV cache blocks are retained on-chip for reuse in the next cycle. This invention achieves efficient real-time inference and reliable output for VLA planning and decoding through hardware acceleration and dynamic scheduling strategies.

[0008] Another objective of this invention is to propose an FPGA acceleration device for VLA planning and scheduling of unmanned vehicles.

[0009] To achieve the above objectives, this invention proposes an FPGA acceleration method for VLA planning and scheduling of unmanned vehicles, comprising:

[0010] The system acquires multi-source input data for the current planning cycle of the autonomous vehicle, including environmental perception terms and vehicle status terms. Based on the vehicle speed information in the vehicle status terms and the obstacle information in the environmental perception terms, the system calculates the comprehensive risk value of the current road scenario and determines the decoding delay budget based on the comprehensive risk value. An action decoding scheduling strategy is generated based on the comprehensive risk value. The action decoding scheduling strategy includes the number of candidate action branches and the allocation ratio of computing resources. Based on the action decoding scheduling strategy, the KV cache generated during the VLA planning model decoding process is divided into a safe KV layer, a hot KV layer and a cold KV layer. The safe KV layer and the hot KV layer are loaded into the FPGA on-chip cache, while the cold KV layer is kept in off-chip storage. Configure the FPGA reconfigurable parallel computing array according to the computing resource allocation ratio. In the current decoding step, based on the shared hidden state vector in the on-chip cache and the KV data of the secure KV layer and the hot KV layer, perform forward decoding calculations of multiple candidate action branches in parallel on the FPGA reconfigurable parallel computing array to generate multiple sets of candidate action word sequences and their corresponding candidate trajectory word sequences. The candidate trajectory word sequence is input into the safety constraint verification logic embedded in the FPGA hardware pipeline. The safety constraint verification logic includes vehicle kinematic continuity constraints and collision safety distance constraints. Parallel hard constraint verification is performed on the candidate trajectory word sequence. Candidate trajectory word sequences that do not meet any constraint are eliminated, and candidate trajectory word sequences that pass the verification are retained. Consistency scores are applied to the validated candidate trajectory lexical sequences, and the optimal set is selected as the action planning result for the current planning cycle and output. At the same time, KV cache blocks with a hit rate higher than a preset threshold in the current decoding step are retained in the on-chip cache unit for use in the next planning cycle.

[0011] An FPGA acceleration method for VLA planning and scheduling of unmanned vehicles according to an embodiment of the present invention may also have the following additional technical features: In one embodiment of the present invention, calculating the comprehensive risk value of the current road scenario includes: Obtain the current vehicle speed v from the vehicle status term, and then... Output vehicle speed risk term Rv; obtain the forward obstacle distance d and relative approach speed Δv from the environmental perception terms, and the forward collision time. As input, via Output forward collision risk items ; Obtain lane risk items Road risk item Rr, and vehicle speed risk item Rv, and forward collision risk item Lane risk items The system incorporates a weighted fusion logic with the road risk factor Rr as input, performs a weighted summation according to preset weight coefficients, and outputs the comprehensive risk value of the current road scenario. .

[0012] In one embodiment of the present invention, generating an action decoding scheduling strategy based on the comprehensive risk value includes: Input the comprehensive risk value R into a risk level comparator, and compare it with the preset low risk threshold Th_low and high risk threshold Th_high: When R≤Th_low, the risk level comparator outputs a low risk mode identifier, and the action decoding scheduling strategy corresponding to the low risk mode includes: the decoding delay budget is the first delay budget T1, the number of candidate action branches is the first number N1, and the computing resource allocation ratio is the first ratio P1; When Th_low<R<Th_high, the risk level comparator outputs a medium risk mode identifier, and the action decoding scheduling strategy corresponding to the medium risk mode includes: the decoding delay budget is the second delay budget T2, the number of candidate action branches is the second number N2, and the computing resource allocation ratio is the second ratio P2, where T2<T1, N2>N1; When R≥Th_high, the risk level comparator outputs a high risk mode identifier, and the action decoding scheduling strategy corresponding to the high risk mode includes: the decoding delay budget is the third delay budget T3, the number of candidate action branches is the third number N3, and the computing resource allocation ratio is the third ratio P3; where T3<T2, N3>N2, and the proportion of computing resources allocated to safety constraint verification in the third ratio P3 is higher than the proportion of computing resources allocated to safety constraint verification in the second ratio P2 and the first ratio P1.

[0013] In one embodiment of the present invention, the KV cache generated during the decoding process of the VLA planning model is divided into a secure KV layer, a hot KV layer, and a cold KV layer, including: Acquire the semantic label of each KV cache block in the current decoding step, where the semantic label includes whether a collision constraint token is contained, whether a lane boundary token is contained, whether a braking constraint token is contained, whether the block is located within a preset forward distance range of the vehicle, and the attention hit count in the previous planning cycle; Input the semantic labels into a KV layer classifier, and the KV layer classifier outputs the layer identifier of each KV cache block according to a preset judgment rule; When the layer identifier is the secure KV layer, the corresponding KV cache block is written into the secure KV locking area in the on-chip cache unit, and the secure KV locking area is prohibited from being replaced within the current planning cycle; When the layer identifier is the hot KV layer, the corresponding KV cache block is written into the hot KV residence area in the on-chip cache unit, and the KV cache blocks in the hot KV residence area are allowed to be replaced according to a preset elimination strategy when not referenced; When the layer identifier is the cold KV layer, the corresponding KV cache block is retained in the off-chip storage unit, and no off-chip read request is initiated in the current decoding step.

[0014] In one embodiment of the present invention, the preset elimination strategy includes: Get the current used capacity and total capacity of the hot KV cache area. When the used capacity reaches the total capacity and a new hot KV cache block needs to be written, trigger the eviction logic. The eviction logic first scans the reference counters of each KV cache block in the hot KV resident area and marks the KV cache blocks with a reference count of zero as eviction candidates; If there are multiple candidate KV cache blocks that can be evicted, the priority score Q of each candidate KV cache block is read, and the KV cache block with the lowest priority score Q is determined as the evicted block; where the priority score Q is calculated based on regional relevance, security criticality and historical attention hit rate; Remove the obsolete block from the hot KV resident area, release the on-chip cache space occupied by the obsolete block, and write the metadata of the obsolete block back to the KV directory table in the off-chip storage unit.

[0015] In one embodiment of the present invention, forward decoding computation of multiple candidate action branches is performed in parallel on an FPGA reconfigurable parallel computing array, including: Read the shared hidden state vector h_t of the current decoding step from the on-chip cache unit, and broadcast the shared hidden state vector h_t to N computation partitions in the FPGA reconfigurable parallel computing array, with each computation partition corresponding to a candidate action branch; Each computation partition reads the action lexical embedding vector and key-value data corresponding to its respective candidate action branch from the on-chip cache unit in parallel. Each computation partition independently performs attention computation, feedforward network computation, and output head computation to generate the current decoding step action word probability distribution for the corresponding candidate action branch, and obtains the action word for the current decoding step based on the action word probability distribution. The action tokens generated by each candidate action branch in the current decoding step are concatenated with the trajectory state of the previous decoding step. The corresponding candidate trajectory points are generated through the projection layer. When the number of generated trajectory points reaches the preset time domain length, the decoding of the current candidate action branch is stopped, and the complete candidate trajectory token sequence is output.

[0016] In one embodiment of the present invention, the safety constraint verification logic embedded in the FPGA hardware pipeline includes a vehicle kinematics continuity verification pipeline and a collision safety distance verification pipeline, performing parallel hard constraint verification on candidate trajectory word sequences, including: The coordinates, velocity, acceleration, and heading angle data of each trajectory point in the candidate trajectory term sequence are input in parallel into the vehicle kinematics continuity verification pipeline; The vehicle kinematics continuity verification pipeline is based on a discrete kinematic bicycle model. It predicts the state of the (k+1)th trajectory point based on the state of the kth trajectory point, calculates the deviation between the predicted state and the actual state of the (k+1)th trajectory point in the candidate trajectory word sequence, and outputs the kinematic continuity flag of the candidate trajectory word sequence as invalid when the deviation exceeds the preset continuity threshold. The positions of each trajectory point in the candidate trajectory word sequence are input into the collision safety distance verification pipeline. The collision safety distance verification pipeline calculates the minimum distance between each trajectory point and the obstacle based on the obstacle position and size in the environmental perception word. When the minimum distance is less than the preset safety distance threshold, the collision safety flag of the candidate trajectory word sequence is output as invalid. Input the kinematic continuity flag and the collision safety flag into the logic AND gate. When either flag is invalid, the corresponding candidate trajectory term sequence is removed.

[0017] In one embodiment of the present invention, the consistency scoring of the verified candidate trajectory lexical sequences includes: Obtain the validated candidate trajectory word sequence and input the trajectory points in each candidate trajectory word sequence into the scoring pipeline; The scoring pipeline calculates the model confidence score, navigation consistency score, trajectory smoothness score, and safety margin score for each candidate trajectory term sequence in parallel. The model confidence score, navigation consistency score, trajectory smoothness score, and safety margin score are input into a weighted summer, which performs weighted summation according to preset scoring weight coefficients, and outputs the comprehensive score S_score of each candidate trajectory word sequence. The comprehensive score S_score of each candidate trajectory word sequence is input into the maximum value selector. The maximum value selector outputs the candidate trajectory word sequence with the highest comprehensive score S_score as the optimal action planning result for the current planning cycle.

[0018] In one embodiment of the present invention, retaining KV cache blocks with a hit rate higher than a preset threshold in the current decoding step in the on-chip cache unit for use in the next planning cycle includes: At the end of the current decoding step, read the number of times each KV cache block in the on-chip cache unit was referenced in this decoding step; KV cache blocks with reference counts greater than zero are marked as hit blocks, and the hit rate H is calculated based on the total number of hits of the hit blocks in the most recent L consecutive decoding steps; where L is the preset statistical window length. The hit rate H is input to the comparator. When H ≥ the preset hit rate threshold H_th, the corresponding KV cache block is marked as a reserved block, and the reserved block is maintained in the on-chip cache unit. When H is less than H_th, the corresponding KV cache block is marked as a to-be-released block, and when the capacity of the on-chip cache unit is insufficient, the cache space occupied by the to-be-released block is preferentially released.

[0019] To achieve the above objective, another aspect of the present invention proposes an FPGA acceleration device for VLA planning and scheduling of unmanned vehicles, comprising: A risk assessment module, configured to acquire multi-source input data of the unmanned vehicle in the current planning cycle, where the multi-source input data includes environment perception tokens and vehicle state tokens; calculate a comprehensive risk value of the current road scene according to the vehicle speed information in the vehicle state tokens and the obstacle information in the environment perception tokens, and determine a decoding delay budget according to the comprehensive risk value; A scheduling layering module, configured to generate an action decoding scheduling strategy according to the comprehensive risk value, where the action decoding scheduling strategy includes the number of candidate action branches and the calculation resource allocation ratio; divide the KV cache generated during the decoding process of the VLA planning model into a secure KV layer, a hot KV layer and a cold KV layer according to the action decoding scheduling strategy, load the secure KV layer and the hot KV layer into the FPGA on-chip cache, and retain the cold KV layer in the off-chip memory; A parallel decoding module, configured to configure the FPGA reconfigurable parallel computing array according to the calculation resource allocation ratio, and in the current decoding step, based on the shared hidden state vector in the on-chip cache and the KV data of the secure KV layer and the hot KV layer, perform forward decoding calculation of multiple candidate action branches in parallel on the FPGA reconfigurable parallel computing array, to generate multiple sets of candidate action token sequences and corresponding candidate trajectory token sequences respectively; A safety verification module, configured to input the candidate trajectory token sequences into the safety constraint verification logic solidified in the FPGA hardware pipeline, where the safety constraint verification logic includes vehicle kinematics continuity constraint and collision safety distance constraint, perform parallel hard constraint verification on the candidate trajectory token sequences, eliminate candidate trajectory token sequences that do not satisfy any of the constraints, and retain the candidate trajectory token sequences that pass the verification; A scoring feedback module, configured to perform consistency scoring on the candidate trajectory token sequences that pass the verification, select the optimal set as the action planning result of the current planning cycle and output the same; at the same time, retain the KV cache blocks with a hit rate higher than a preset threshold in the current decoding step in the on-chip cache unit for the next planning cycle.

[0020] This invention discloses an FPGA-accelerated method and apparatus for VLA planning and scheduling of unmanned vehicles. It acquires multi-source inputs and calculates a comprehensive risk value to determine the decoding latency budget, generating a dynamic scheduling strategy that includes the number of candidate action branches and resource allocation ratios. The key-value cache is divided into three layers: safe, hot, and cold, and selectively loaded onto the on-chip cache. A reconfigurable parallel computing array is configured to perform multi-branch forward decoding. Parallel hard constraint verification is performed using a hardware-encapsulated kinematic continuity and collision safety distance constraint pipeline. The optimal planning result is selected based on a consistency score, while high-frequency hit cache blocks are retained for reuse in the next cycle. This method effectively solves the problems of insufficient real-time performance and low safety verification efficiency in existing VLA planning under limited onboard resources. It achieves integrated hardware acceleration for the entire process from risk assessment, dynamic scheduling, parallel decoding, hard constraint verification to cache optimization, significantly improving the real-time response capability and trajectory safety of unmanned vehicle decision-making and planning, and enhancing the adaptability and engineering practical value of the method in complex dynamic scenarios.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an FPGA acceleration method for VLA planning and scheduling of unmanned vehicles according to an embodiment of the present invention; Figure 2 Overall flowchart of the method according to an embodiment of the present invention; Figure 3 A schematic diagram of the overall chip structure according to an embodiment of the present invention; Figure 4 A compilation mapping flowchart for an autonomous vehicle VLA planning model according to an embodiment of the present invention; Figure 5 A schematic diagram of the structure of an FPGA acceleration device for VLA planning and scheduling of unmanned vehicles according to an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] The following describes, with reference to the accompanying drawings, an FPGA acceleration method and apparatus for VLA planning and scheduling of unmanned vehicles according to an embodiment of the present invention.

[0026] The core idea of ​​this invention is to address the bottlenecks in computing power and storage resources, as well as the deterministic response requirements in safety-critical scenarios faced by VLA models in real-time inference on automotive-grade embedded platforms. This is achieved by constructing an FPGA acceleration method covering the entire process of "risk assessment—scheduling generation—caching layering—parallel decoding—hard constraint verification," thus solving the problems of decision-making latency uncertainty and low safety verification efficiency caused by the lack of dynamic risk perception and hardware adaptive scheduling in traditional solutions. First, multiple input sources are acquired and a comprehensive risk value is calculated to determine the decoding latency budget. Based on this, a scheduling strategy is generated, including the number of candidate action branches and the allocation ratio of computing resources. Accordingly, the KV cache is divided into three layers: safe, hot, and cold, and selectively loaded onto the on-chip cache. Then, the FPGA's reconfigurable parallel computing array is configured to execute multi-branch forward decoding in parallel, generating candidate action and trajectory word sequences. Parallel hard constraint verification is performed using kinematic continuity and collision safety distance constraints solidified in the hardware pipeline, eliminating sequences that do not meet safety conditions. Finally, the verified candidate sequences are scored for consistency, and the optimal result is output. High-frequency hit cache blocks are retained for reuse in the next cycle. Through the technical path of "dynamic risk-driven, software and hardware collaborative scheduling, and parallel security verification" mentioned above, this invention transforms the traditional experience-based scheduling based on fixed decoding strategies and general computing architecture into an intelligent acceleration system that can sense driving risks, adaptively allocate computing resources, and fully leverage the advantages of FPGA parallelism and pipelines. This significantly improves the real-time response capability, trajectory safety, and engineering applicability of VLA planning in resource-constrained vehicle environments.

[0027] The following describes, with reference to the accompanying drawings, an FPGA acceleration method and apparatus for VLA planning and scheduling of unmanned vehicles according to an embodiment of the present invention.

[0028] like Figure 1 As shown, the FPGA acceleration method for VLA planning and scheduling of unmanned vehicles according to the present invention includes the following steps: S1, Multi-source input data for the current planning cycle of the vehicle and pedestrian, the multi-source input data includes environmental perception terms and vehicle status terms; based on the vehicle speed information in the vehicle status terms and the obstacle information in the environmental perception terms, calculate the comprehensive risk value of the current road scene, and determine the decoding delay budget based on the comprehensive risk value; S2 describes the comprehensive risk value generation action decoding scheduling strategy, which includes the number of candidate action branches and the allocation ratio of computing resources. According to the action decoding scheduling strategy, the KV cache generated during the VLA planning model decoding process is divided into a safe KV layer, a hot KV layer and a cold KV layer. The safe KV layer and the hot KV layer are loaded into the FPGA on-chip cache, while the cold KV layer is kept in off-chip storage. S3, configure the FPGA reconfigurable parallel computing array according to the resource allocation ratio. In the current decoding step, based on the shared hidden state vector in the on-chip cache and the KV data of the secure KV layer and the hot KV layer, perform forward decoding calculations of multiple candidate action branches in parallel on the FPGA reconfigurable parallel computing array to generate multiple sets of candidate action word sequences and their corresponding candidate trajectory word sequences. S4, the candidate trajectory word sequence is input into the safety constraint verification logic embedded in the FPGA hardware pipeline. The safety constraint verification logic includes vehicle kinematic continuity constraints and collision safety distance constraints. Parallel hard constraint verification is performed on the candidate trajectory word sequence, and candidate trajectory word sequences that do not meet any constraint are eliminated, while candidate trajectory word sequences that pass the verification are retained. S5, the consistency score of the verified candidate trajectory word sequences is performed, and the best set is selected as the action planning result of the current planning cycle and output. At the same time, the KV cache blocks with a hit rate higher than the preset threshold in the current decoding step are retained in the on-chip cache unit for the next planning cycle.

[0029] The following is a detailed description of the above methods and steps: like Figure 2 As shown, the method of this embodiment of the invention specifically includes the following steps: S101. Construct the input context for VLA planning of the unmanned vehicle; S102. Generate planning task priorities and action decoding scheduling strategies based on current vehicle speed, vehicle control cycle, road scenario risks and safety constraints. S103. Dynamically schedule the action decoding data, KV cache, candidate action branches and FPGA computing resources according to the action decoding scheduling strategy. S104. Accelerate the decoding of key action terms or trajectory terms on the FPGA reconfigurable parallel computing array and generate candidate action results in parallel. S105. Perform consistency scoring and safety constraint verification on the candidate action results, and output action planning results that meet the vehicle control cycle and road safety constraints.

[0030] Further, in one embodiment of the present invention, S101 includes: The VLA planning input context building unit receives environmental perception results, local map information, vehicle status information, historical trajectory information, and navigation instructions within the current planning cycle. The environmental perception results may include vehicles ahead, pedestrians, obstacles, lane boundaries, and drivable areas; the vehicle status information may include current speed, acceleration, steering angle, and braking status; and the navigation instructions may include the target lane, target path, or upper-level behavioral decision instructions.

[0031] In this embodiment, the VLA planning input context construction unit organizes the above information into a planning input context. The planning input context includes environmental semantic terms, map terms, vehicle status terms, historical trajectory terms, and task instruction terms. The Alpamayo-type VLA planning model decodes action terms or trajectory terms based on this planning input context to generate the action planning result for the current planning cycle.

[0032] Further, in one embodiment of the present invention, S102 includes: The planning task scheduling and control logic determines the priority of the planning task based on the current vehicle speed, vehicle control cycle, distance to the vehicle in front, distance to obstacles, road speed limit, road topology, and safety distance.

[0033] When the vehicle is at a high speed, the distance to the vehicle in front is small, or the road risk is high, the planning task scheduling control logic increases the priority of safety-critical planning tasks such as deceleration, braking, lane keeping, obstacle avoidance, and maintaining a safe distance, and reduces the number of low-relevance candidate action branches to ensure that the action planning results can be output within the vehicle control cycle.

[0034] When the vehicle is in a low-speed, parking, obstacle avoidance, or low-speed oncoming traffic scenario, the planning task scheduling control logic increases the priority of planning tasks such as local trajectory adjustment, fine-grained steering, and close-range obstacle avoidance, and allows more candidate trajectory segments to be retained for safety verification.

[0035] Based on the planned task priority, the planned task scheduling control logic generates an action decoding scheduling strategy. This strategy is used to configure the number of action lexical decoding steps, the number of candidate action branches, the number of candidate trajectory segments, the KV cache access priority, on-chip cache resources, and the parallelism of the FPGA computing array.

[0036] Further, in one embodiment of the present invention, S103 includes: According to the action decoding scheduling strategy, the on-chip multiplexing pipeline unit for action decoding data retains the hidden state vector, action lexical embedding, intermediate normalization results, and candidate action outputs in the on-chip cache unit during the current decoding step, and performs pipelined multiplexing between multiple Transformer decoding layers. Since the aforementioned decoding activation data is small in size but frequently accessed, writing back to off-chip memory at each layer would result in a large number of small-granularity memory accesses. This embodiment reduces the repeated handling of hidden state vectors and intermediate results between on-chip cache and off-chip storage through on-chip multiplexing.

[0037] Meanwhile, the hierarchical KV cache scheduling unit manages the KV cache hierarchically according to the semantic importance of the planning. In this embodiment, contexts directly related to collision risk, road boundaries, route constraints, and safe distances are classified as safe KV; KV caches corresponding to the current lane, the vehicle in front, nearby pedestrians, traffic lights, and generated action words are classified as hot KV; KV caches corresponding to vehicles in adjacent lanes, local maps, and short-term historical trajectories are classified as warm KV; and KV caches corresponding to distant backgrounds, earlier historical frames, and low-relevance map information are classified as cold KV.

[0038] For secure key-value pairs and hot key-value pairs, their indices, priority tags, and key-value data blocks within the current decoding window are preferentially retained, and on-chip caching or high-priority access is employed. For warm key-value pairs, alternating on-chip and off-chip access or low-bit compression is used. For cold key-value pairs, off-chip storage, sparse access, or delayed access are employed. This ensures that contexts strongly related to the current planned task and security verification are given priority in action decoding and candidate action verification.

[0039] Further, in one embodiment of the present invention, S104 includes: The FPGA mapping and resource allocation process maps the linear projection, attention calculation, normalization, feedforward network, and output head calculation in the Alpamayo-like VLA planning model to the FPGA's reconfigurable parallel computing array. For decoding calculations corresponding to safety-critical action terms or trajectory terms, higher computational parallelism and higher cache access priority are allocated; for low-priority action branches, lower computational precision, delayed calculation, or branch pruning are used to reduce computational overhead.

[0040] After obtaining the hidden state vector in the current decoding step, the candidate action parallel prediction and security verification unit generates multiple candidate action terms or candidate trajectory fragments in parallel based on the hidden state vector. The candidate actions may include action types such as lane keeping, deceleration, braking, slight left veer, slight right veer, following another vehicle, or stopping and waiting, or may represent trajectory increments at several future time points.

[0041] By generating candidate action results in parallel, this embodiment avoids fully serially expanding all candidate action branches, which can shorten the waiting time in the process of generating action words or trajectory words.

[0042] The FPGA mapping and resource allocation process maps the linear projection, attention calculation, normalization, feedforward network, and output head calculation in the Alpamayo-like VLA planning model to an FPGA reconfigurable parallel computing array. In one embodiment, the FPGA reconfigurable parallel computing array includes a two-dimensional computing array based on DSP multiply-accumulate units, with an array size of [missing information]. Each processing unit includes a multiplier, an accumulator, an input register, and a local result register. Matrix multiplication calculations in the linear projection layer, feedforward network layer, and output header are mapped to this two-dimensional computation array using a block matrix multiplication method. Input activations are input along the row direction, weights along the column direction, and partial sums are accumulated within the array and written to the on-chip cache. For planning terms with a hidden dimension of 768, the computation array follows... The matrix calculation is performed in blocks sequentially, and the intermediate activation results are reused through on-chip cache.

[0043] For multi-head attention computation, the multiple attention heads in the VLA planning model are grouped and scheduled according to their head dimensions. Taking 12 attention heads, each with a dimension of 64, as an example, the planning task scheduling control logic divides the attention heads into multiple computation batches. Each batch performs a dot product calculation between the query vector Q and the key vector K in the hot KV cache to obtain an attention score. Then, it completes the normalization calculation through table lookup or piecewise linear approximation. Finally, it weights the attention weights and the value vector V and accumulates them to obtain the corresponding attention head output. Since multiple candidate action branches in the current action decoding step share the hot KV cache corresponding to the vehicle ahead, lane boundary, and safety constraints, the computation array does not repeatedly move the same K / V data when generating multiple candidate actions.

[0044] The computational precision is controlled by a precision mode register, eliminating the need to reconfigure the entire FPGA bitstream. Specifically, the DSP processing unit internally sets up a mixed-precision input selector, a quantization scale register, and an accumulation mode register. For safety-critical action terms, collision-constrained trajectory terms, and safety verification calculations, the precision mode register selects INT8 multiplication and FP16 accumulation; for low-priority candidate action branches, it selects low-bit-width weighted quantization and INT8 accumulation, allowing skipping of low-importance cold key-value accesses. The precision mode is updated only at decoding step boundaries or candidate branch switching boundaries, avoiding changes to the data format during a single matrix calculation. Thus, the chip achieves mixed-precision scheduling through register configuration, rather than relying on a full-chip reconfiguration.

[0045] Parallel prediction of candidate actions does not involve replicating multiple complete VLA decoder hardware copies. Instead, it employs a shared decoding backbone and parallel output of candidate branches. Specifically, the on-chip multiplexing pipeline unit for action decoding data first calculates the current hidden state vector and writes it to the on-chip buffer. The parallel prediction and safety verification unit for candidate actions allocates branch context registers and candidate action buffers for multiple candidate action branches. The shared decoding backbone is responsible for performing linear projection, attention calculation, and feedforward network calculation. Candidate action branches are expanded in parallel during the output header and trajectory generation stages. For the four candidate action branches in the medium-priority mode, the candidate action buffer opens four write ports to write the trajectory word sequences corresponding to deceleration and lane keeping, left-side avoidance, emergency braking, and maintaining current speed, respectively. The safety verification unit synchronously reads the trajectory points of each branch and performs distance, deceleration, lateral acceleration, and lane boundary constraint verification.

[0046] In terms of hardware overhead, this approach only adds candidate branch registers, candidate action buffers, and a security verification pipeline, without copying the complete attention layer and feedforward network layer. Compared to single-branch serial decoding, multiple candidate branches share hidden state vectors and hot key-value buffers, reducing off-chip memory access; compared to copying multiple complete decoders, it reduces DSP and on-chip cache usage. Taking the aforementioned medium-priority scenario as an example, four candidate action branches are generated in batches on a shared decoding backbone. The action decoding and candidate trajectory generation stages take 42ms, and candidate action output and security verification are completed within a 70ms latency budget.

[0047] Further, in one embodiment of the present invention, S105 includes: The candidate action parallel prediction and safety verification unit first performs a consistency score on the candidate action results based on the candidate action probability or candidate action score output by the VLA master decoder. The consistency score is used to determine whether the candidate action is consistent with the current planning input context, the generated action lexical sequence, and the navigation target.

[0048] Subsequently, the safety constraint verification module performs safety verification on the candidate action terms or candidate trajectory segments. The safety verification includes one or more of the following: vehicle motion continuity verification, road boundary verification, obstacle collision risk verification, safe distance verification, acceleration change verification, steering angle change verification, and route constraint verification. If the trajectory segment corresponding to a candidate action causes the vehicle to cross the road boundary, poses a collision risk with an obstacle, fails to meet the safe distance requirement, exceeds the acceleration change threshold, or exhibits discontinuous steering angle changes, then the candidate action is eliminated.

[0049] After completing the consistency scoring and safety constraint verification, the result output unit selects the action planning result that satisfies the vehicle control cycle and road safety constraints from the remaining candidate action results. The action planning result may include a sequence of trajectory points, target velocity, target acceleration, and target steering angle at several future time points, and is sent to the vehicle control module for execution. Simultaneously, the action terms, trajectory terms, and key KV cache information generated in the current planning cycle can serve as the historical context for the next planning cycle.

[0050] Subsequently, the candidate action parallel prediction and safety verification unit performs consistency scoring and safety constraint verification on candidate action terms or candidate trajectory fragments. Let the... The trajectory segments output by each candidate action branch are ,in , and Indicates the vehicle's location. Indicates speed, Indicates longitudinal acceleration. Indicates the heading angle. This indicates the front wheel steering angle. In this embodiment, the trajectory prediction time is 2 seconds, and the sampling interval is 0.2 seconds. .

[0051] Vehicle motion continuity verification employs a discrete kinematic bicycle model. For adjacent trajectory points, the safety verification unit verifies whether the trajectory satisfies vehicle kinematic continuity according to the following formula: (5) (6) (7) (8) in, This refers to the vehicle's wheelbase. The sampling interval for trajectory points is defined. The safety verification unit compares the next trajectory point actually output by the candidate trajectory with the predicted trajectory point calculated by the discrete kinematics bicycle model. If the longitudinal position deviation does not exceed 0.20m, the lateral position deviation does not exceed 0.20m, the heading angle deviation does not exceed 3°, and the speed deviation does not exceed 0.5m / s, the adjacent trajectory point is determined to meet the vehicle motion continuity requirements. If any deviation exceeds the corresponding threshold, the continuity flag for the candidate action is set to 0.

[0052] Candidate actions that pass the hard safety constraint verification enter the consistency scoring stage. Overall score. Calculate according to the following formula: (9) in, Output normalized probability values ​​for the VLA model. To ensure consistency between candidate trajectories and navigation intent, For trajectory smoothness, For safety margin. Calculated based on the distance between the candidate trajectory endpoint and the centerline of the navigation path, the heading angle deviation, and the consistency with the target lane; Calculated based on the rate of change of acceleration and the rate of change of rotation; Calculated based on the minimum forward safety margin and minimum road boundary distance margin of the candidate trajectory in the prediction time domain.

[0053] When multiple constraints conflict, the candidate action parallel prediction and safety verification unit employs arbitration logic prioritizing hard constraints. Collision constraints, road boundary constraints, and minimum safe distance constraints are considered hard constraints; if any hard constraint is not met, the candidate action is directly eliminated, and its score register is written to 0. Vehicle motion continuity, acceleration variation, steering angle variation, and navigation consistency serve as scoring constraints, participating in the comprehensive scoring as long as they do not violate the hard constraints. If multiple candidate actions pass the hard constraint verification, the result output unit selects... The highest-ranking candidate action is taken as the action planning result; if only the emergency braking branch passes the hard constraint verification, the emergency braking branch is output as the safety fallback result; if all candidate actions fail the hard constraint verification, the planning task scheduling control logic triggers a high-priority replanning.

[0054] In summary, the method of this invention prioritizes safety-critical planning tasks during the generation of action or trajectory terms by planning task priority generation and action decoding scheduling strategies; reduces small-granularity off-chip memory access and redundant KV access by using on-chip multiplexing of action decoding data and hierarchical KV cache scheduling; and reduces serial decoding waiting time by using parallel prediction of candidate actions and verification of safety constraints, while filtering out candidate action results that do not meet vehicle motion continuity, road constraints, and safety constraints before output. This improves the action decoding efficiency, planning real-time performance, and safety of the unmanned vehicle VLA planning model when deployed at the FPGA edge.

[0055] Furthermore, in one specific implementation, the VLA planning task during the operation of an autonomous vehicle on an urban expressway is used as an example. The current vehicle control cycle is 100ms, the vehicle speed is 72km / h, the distance to the vehicle in the same lane ahead is 26m, the relative approach speed is 10m / s, there are no dynamic obstacles within 60m ahead in the left lane, there is a low-speed vehicle within 30m ahead in the right lane, the road has a slight curve, and the navigation intent is to continue along the current road. The FPGA inference acceleration chip completes one action planning cycle according to steps S101 to S105 within this planning cycle.

[0056] S101. Construct the input context for VLA planning of the unmanned vehicle.

[0057] The planning input context construction unit writes the perception results, navigation instructions, vehicle status, and safety constraints into the planning context buffer in off-chip storage. The planning input context includes planning terms corresponding to the forward road area, adjacent left and right lanes, map navigation, vehicle status, safety constraints, and historical trajectories. In this embodiment, the planning input context includes 128 terms, each with a dimension of 768, stored in INT8 format. The off-chip storage access interface performs DMA transfer according to the term address table, prioritizing the transfer of vehicle status terms, safety constraint terms, and forward key area terms into the on-chip cache; left and right lane terms are temporarily stored in off-chip storage and retrieved when needed in candidate action branches. After this step, the on-chip cache saves the current hidden state vector, vehicle status register, safety constraint register, and forward key area terms, and the planning task scheduling control logic enters the current cycle scheduling state.

[0058] S102, Generate planning task priorities and action decoding scheduling strategies.

[0059] The planning task scheduling control logic calculates the current comprehensive risk value. The calculation formula is as follows: (1) Among them, the vehicle speed risk item ,in The current vehicle speed is used in this embodiment. Forward collision time ,in Distance to the vehicle in front. To achieve a relatively close speed, in this embodiment... Forward collision risk item ,therefore The left lane allows for avoidance; lane hazard item. Set the value to 0.2; the current road has a slight curve, and the road risk item is... We take 0.3. The comprehensive risk value is calculated as follows: .

[0060] Control logic will Compared with the preset threshold: Enter low-risk mode at this time. Enter medium-risk mode at this time. This period enters a high-risk mode. Therefore, it enters the medium-risk mode. The control logic writes the delay budget register to 70ms, the number of candidate action branches to 4, and allocates the FPGA reconfigurable parallel computing array to action token decoding, candidate trajectory generation, and safety constraint verification in a 7:2:1 ratio.

[0061] S103, scheduling action decoding data, KV buffer, candidate action branches and FPGA computing resources.

[0062] The task scheduling control logic calculates priority scores for KV cache blocks. The calculation formula is as follows: (2) in, For regional correlation, a value of 1 is used for the forward 40m range, 0.7 is used for the left and right adjacent lanes within a 60m range, and 0.2 is used for the far-distance area. For safety-critical terms, terms containing the vehicle ahead, lane boundary, drivable area, or safety constraints are assigned a value of 1, while all others are assigned a value of 0. For navigation relevance, terms located on the navigation path or feasible avoidance path are assigned a value of 1, and others are assigned a value of 0.3; D is the normalized value of attention hit rate in the previous planning cycle. KV cache blocks are written to the hot KV cache. KV cache blocks are written to the warm KV cache. The KV cache blocks are stored in off-chip storage.

[0063] In this embodiment, the KV cache blocks corresponding to the vehicles ahead, the forward lane boundary, the feasible avoidance area on the left, and the safety constraints are entered into the hot KV cache; the KV cache blocks corresponding to the low-speed vehicles on the right and the distant map are entered into the warm KV cache; and the KV cache blocks corresponding to the rear distant area are temporarily not read as cold KV caches.

[0064] Candidate action branches are sorted according to the following formula: (3) in, The action word probabilities output by the previous decoding step of the VLA model. As a safety-critical indicator, safety-critical branches are assigned a value of 1, while non-safety-critical branches are assigned a value of 0.4. This is the drivable space indicator; a feasible branch is assigned a value of 1, and an infeasible branch is assigned a value of 0. Because... At 2.6 seconds, the deceleration and lane-keeping branch and the emergency braking branch are marked as safety-critical branches; since the left lane is available, the left-side avoidance branch is added to the candidate queue; since there is a low-speed vehicle on the right, the right-side avoidance branch does not enter the current candidate queue. Ultimately, the candidate action queue includes the deceleration and lane-keeping branch, the left-side avoidance branch, the emergency braking branch, and the maintain current speed continuing branch. At this point, the hot KV buffer enters the reuse state, the action decoding pipeline starts, and the candidate action buffer opens 4 write ports.

[0065] S104. Perform accelerated decoding of key action terms or trajectory terms, and generate candidate action results in parallel.

[0066] The action token decoding pipeline reads the current hidden state vector, the previous action token embedding, intermediate normalization results, and hot key-value cache blocks from the on-chip cache. The four candidate action branches share the key-value vectors corresponding to the vehicle ahead, lane boundaries, and safety constraints, eliminating repeated access to off-chip storage. The FPGA's reconfigurable parallel computing array performs attention computation, feedforward network computation, and candidate trajectory generation in parallel.

[0067] Each candidate action branch generates a trajectory word sequence for the next 2 seconds, with trajectory points spaced 0.2 seconds apart. Each trajectory group includes 10 trajectory points. In the current cycle, the target deceleration for the deceleration and lane-keeping branch is -2.5 m / s², the maximum lateral displacement for the left-side avoidance branch is 1.2 m, the target deceleration for the emergency braking branch is -4.5 m / s², and the maintain current speed branch maintains the current speed. This phase takes 42 ms, and the four candidate trajectory word sequences are written to the candidate action cache.

[0068] S105. Perform consistency scoring and safety constraint verification on the candidate action results, and output the action planning results.

[0069] The safety verification unit performs parallel verification of candidate action results. The verification rules include: the minimum distance to the vehicle in front within the next 2 seconds must not be less than 8m; the longitudinal deceleration of the conventional planned action must not be less than -4.0m / s²; when the emergency braking branch is used as a safety fallback branch, the longitudinal deceleration must not be less than -6.0m / s²; the lateral acceleration must not exceed 2.5m / s²; the trajectory point must not cross the lane boundary; and the changes in speed and heading angle between adjacent trajectory points must not exceed the vehicle motion continuity threshold.

[0070] Candidate actions that pass security verification are scored using the following formula: (4) in, Output normalized probability values ​​for the model. For navigation purposes Figure 1 Devotion, For trajectory smoothness, For safety margin, in this embodiment, the "Maintain Current Speed" branch reduces the distance to the vehicle in front to 6m at the end of the next 2 seconds, which is less than the 8m safety threshold, so the safety flag is set to 0; the "Emergency Braking" branch meets the collision safety requirements, but due to the large deceleration, the overall score is 0.68; the "Left Side Avoidance" branch passes the safety verification, but the lateral acceleration is close to the threshold, so the overall score is 0.76; the "Decelerate and Maintain Lane" branch passes all safety constraints, with an overall score of 0.84. The planning task scheduling control logic selects the "Decelerate and Maintain Lane" branch and outputs its trajectory word sequence for the next 2 seconds as the current cycle action planning result.

[0071] The total planning cycle time is 61ms, which is less than the 70ms latency budget and the 100ms vehicle control cycle. After the output is completed, the safety-critical key-value blocks in the hot key-value cache are retained for the next planning cycle, the candidate action cache is cleared, and the planning task scheduling control logic returns to the cycle listening state.

[0072] The implementation of the method in this embodiment of the invention, taking an urban expressway driving scenario as an example, involves constructing a planning input context that includes a forward critical area, vehicle status, and safety constraints; calculating a comprehensive risk value based on vehicle speed and the collision time with the preceding vehicle and determining a medium-risk mode, and generating a dynamic scheduling strategy with a time delay budget, the number of candidate action branches, and the allocation ratio of computing resources accordingly; dividing the KV cache into hot, warm, and cold layers according to priority scores and selectively loading them onto the chip; and then, using an FPGA reconfigurable parallel computing array, executing the forward decoding and trajectory generation of four candidate action branches in parallel, and performing hardware-accelerated safety constraint verification and consistency scoring to select the optimal trajectory that meets collision safety, kinematic continuity, and navigation consistency as the planning result output. The total cycle time is 61ms, which is better than the 70ms time delay budget and the 100ms vehicle control cycle requirement, and the safety-critical KV block is retained for reuse in the next cycle. By using the aforementioned dynamic risk perception-driven software and hardware collaborative scheduling and full pipeline parallel solution, the problem of VLA planning on the vehicle embedded platform is effectively solved, which makes it difficult to balance real-time performance and safety in the existing technology. This significantly improves the response time, trajectory safety and computing resource utilization efficiency of decision planning in complex dynamic scenarios.

[0073] In addition, such as Figure 3 As shown, this embodiment of the invention provides an FPGA acceleration chip for VLA planning and scheduling of unmanned vehicles. The chip includes an FPGA reconfigurable parallel computing array, an on-chip cache unit, an off-chip memory access interface, planning task scheduling control logic, a VLA planning input context construction unit, an on-chip multiplexing pipeline unit for action decoding data, a hierarchical KV cache scheduling unit, a candidate action parallel prediction and security verification unit, and a result output unit.

[0074] The planning task scheduling control logic is connected to the VLA planning input context construction unit, the hierarchical KV cache scheduling unit, the action decoding data on-chip multiplexing pipeline unit, the candidate action parallel prediction and security verification unit, and the result output unit via a control configuration bus. This logic is used to issue risk levels, latency budgets, the number of candidate branches, cache hierarchical thresholds, and computational resource allocation parameters. In this embodiment, the control configuration bus is a 32-bit AXI-Lite bus. Each unit receives control commands via a cmd_valid / cmd_ready handshake and returns its running status via status_valid / status_ready.

[0075] The off-chip memory access interface connects to HBM or DDR and is connected to the on-chip cache unit via a 256-bit AXI4 data bus for transporting planning input terms, hidden state vectors, model weights, and KV cache blocks. The on-chip cache unit is connected to the hierarchical KV cache scheduling unit, the action decoding data on-chip multiplexing pipeline unit, and the FPGA reconfigurable parallel computing array. The hierarchical KV cache scheduling unit divides the KV cache blocks into hot, warm, and cold caches according to the cache hierarchy threshold written by the scheduling control logic. The action decoding data on-chip multiplexing pipeline unit reads the current hidden state vector, action term embeddings, intermediate normalization results, and hot KV cache blocks from the on-chip cache unit and sends them to the FPGA reconfigurable parallel computing array via a 256-bit internal streaming data bus. The internal streaming data bus uses a data_valid / data_ready handshake mechanism to ensure continuous transmission of action decoding data between multi-stage pipelines.

[0076] The FPGA-based reconfigurable parallel computing array is connected to a candidate action parallel prediction and safety verification unit. The former outputs candidate action terms or trajectory terms, while the latter performs consistency scoring and safety constraint verification on the candidate actions and sends the verified action planning results to the result output unit. The control logic operates in the control clock domain, the computing array, on-chip cache, and action decoding pipeline operate in the computing clock domain, and the off-chip memory access interface operates in the memory clock domain. Data and control flags are transferred between different clock domains via asynchronous FIFOs and dual-triggered synchronous registers.

[0077] Within a planning cycle, the chip's module-level timing is as follows: The VLA planning input context construction unit first reads the planning input data for the current cycle through the off-chip memory access interface and writes it to the on-chip cache; the planning task scheduling control logic then writes the scheduling parameters; after the hierarchical KV cache scheduling unit completes the hot KV cache loading, the action decoding data on-chip multiplexing pipeline unit starts decoding data multiplexing; the FPGA reconfigurable parallel computing array generates candidate action terms or trajectory terms in parallel; after the candidate action parallel prediction and security verification unit completes the scoring and security verification, it sends the final action planning result to the result output unit and returns a cycle completion flag to the planning task scheduling control logic.

[0078] This invention discloses an FPGA acceleration chip for VLA planning and scheduling of unmanned vehicles. By setting up planning task scheduling control logic, a VLA planning input context construction unit, a hierarchical KV cache scheduling unit, an on-chip multiplexing pipeline unit for action decoding data, an FPGA reconfigurable parallel computing array, and a candidate action parallel prediction and security verification unit, and employing a multi-clock domain asynchronous collaborative architecture and an AXI bus standard interface for inter-unit data interaction and control command issuance, it achieves a fully pipelined hardware-accelerated solution from planning input reading, scheduling parameter configuration, hierarchical loading of KV cache, streaming multiplexing of decoding data, parallel decoding of candidate actions, to consistency scoring and security constraint verification. This chip effectively solves the real-time bottleneck and security risks caused by the computationally and memory-intensive characteristics of VLA planning and decoding in existing vehicle solutions. It significantly improves the throughput and deterministic latency guarantee capability of unmanned vehicle decision-making and planning in resource-constrained embedded environments, and enhances the chip's engineering adaptability and large-scale deployment value in highly dynamic road scenarios.

[0079] Furthermore, in this embodiment, the Alpamayo-type autonomous vehicle VLA planning model is used as an example for illustration. This type of model uses the autonomous vehicle's environmental perception results, lane topology information, vehicle state information, historical trajectory information, and navigation commands as the planning input context. It progressively generates action terms or trajectory terms through a Transformer-type decoder and outputs trajectory points, speeds, accelerations, or steering control reference values ​​for a future period. It should be noted that this embodiment uses the Alpamayo-type VLA planning model as an example and does not limit the invention to this model; it can also be applied to other autonomous vehicle planning models based on a vision-language-action architecture or a Transformer decoding structure.

[0080] The planning task scheduling and control logic is used to generate planning task priorities and action decoding scheduling strategies based on current vehicle speed, vehicle control cycle, road scenario risks, and safety constraints. The on-chip multiplexing pipeline unit for action decoding data is used to perform on-chip cache reuse of hidden state vectors, action word embeddings, intermediate normalization results, and candidate action outputs during action word or trajectory word decoding. The hierarchical KV cache scheduling unit is used to perform hierarchical scheduling of the KV cache based on the semantic importance of the planning. The parallel prediction and safety verification unit for candidate actions is used to generate candidate action words or candidate trajectory fragments in parallel and complete consistency scoring and safety constraint verification before output. Therefore, this embodiment does not only accelerate the full inference of the VLA planning model, but also accelerates the scheduling of key planning tasks, candidate action branches, KV cache access, and safety verification tasks during the action decoding process.

[0081] The planning task scheduling and control logic includes a risk calculator, a strategy lookup table, and a scheduling register group, used to convert the vehicle speed, vehicle control cycle, road scenario risks, and safety constraints of the current planning cycle into action decoding scheduling strategies. In this embodiment, the planning task priority is determined by a comprehensive risk value. Sure, The value range is 0 to 1, and its calculation rule adopts the weighted calculation method of the aforementioned vehicle speed risk, forward collision risk, lane risk and road type risk. The specific calculation formula is shown in equation (1). The planning task scheduling control logic will Compare with the preset threshold and write to the scheduling register group according to Table 1 below: Table 1 Action Decoding Scheduling Strategy Configuration Table

[0082] Specifically, the latency budget is written to the latency budget register, the number of candidate action branches is written to the branch control register, and the computational resource allocation ratio is written to the computational array configuration register. In low-priority mode, the chip reduces the number of candidate action branches while retaining relatively complete context access; in medium-priority mode, the chip expands multiple candidate action branches in parallel and prioritizes computations related to the forward region, adjacent avoidable regions, and safety constraints; in high-priority mode, the chip increases the resource allocation to candidate trajectory generation and safety verification tasks, suspends low-importance cold key-value access, and prioritizes the use of candidate action branches related to deceleration, avoidance, and emergency braking. The emergency braking branch is only used as a safety candidate action branch in high-priority mode and does not constitute a new planning task priority level on its own.

[0083] When the vehicle speed is 30km / h, the distance to the target ahead is 80m, the relative approach speed is 3m / s, the road is a straight section, and the adjacent lanes allow for avoidance... , , , , Calculations yielded This corresponds to the low-priority mode. In this mode, the scheduling register is written with a 120ms latency budget, two candidate action branches, and a 6:3:1 resource allocation ratio. When the vehicle speed is 72km / h, the distance to the target ahead is 26m, the relative approach speed is 10m / s, the road is a curved section, and the left lane allows for avoidance... , , , , Calculations yielded This corresponds to the medium priority mode. At this time, the scheduling register is written with a 70ms delay budget, 4 candidate action branches, and a 7:2:1 resource allocation ratio.

[0084] The on-chip multiplexing pipeline unit for action decoding data is used to perform on-chip buffering and reuse of hidden state vectors, action word embeddings, intermediate normalization results, and intermediate outputs of candidate actions during the decoding process of action words or trajectory words. Specifically, within the same planning cycle, multiple candidate action branches share the current hidden state vector and safety-critical key-value data; when a candidate branch enters the next decoding step, the on-chip multiplexing pipeline unit for action decoding data prioritizes reading the intermediate results of the previous decoding step stored in the on-chip buffer, rather than rereading the same data from off-chip storage.

[0085] The hierarchical KV cache scheduling unit is used to perform hierarchical scheduling of the KV cache according to the semantic importance of the plan. Each KV cache block corresponds to a priority score calculated by equation (2). The hierarchical KV cache scheduling unit divides KV levels according to the following rules: If a KV cache block contains forward critical obstacles, lane boundary constraints, collision constraints, or braking constraints, it is classified as a safe KV; safe KVs belong to the locked subclass of hot KVs and do not participate in replacement during the current planning cycle. For other KV cache blocks, when Time is divided into thermal KV, when Time is divided into temperature KV, when Time is divided into cold KV. Taking the aforementioned urban expressway scenario as an example, the KV cache blocks corresponding to the vehicle ahead, the forward lane boundary, the feasible avoidance area on the left, and the safety constraints enter the hot KV cache, among which the KV cache blocks corresponding to the safety constraints and the vehicle ahead enter the safe KV lock area; the KV cache blocks corresponding to the low-speed vehicle on the right and the distant map enter the warm KV cache; the KV cache blocks corresponding to the distant area behind are cold KV and are not read temporarily.

[0086] In one embodiment, the total capacity of the on-chip cache unit is 2MB, of which 1MB is used as a KV hierarchical cache, 512KB as an action decoding activation cache, 256KB as a candidate action cache, and the remaining 256KB as an index and state cache. The KV hierarchical cache includes a 128KB secure KV locking area, a 640KB hot KV residency area, and a 256KB warm KV exchange area. Secure KVs are not replaced during the current planning cycle; when the hot KV residency area is insufficient, the hierarchical KV cache scheduling unit prioritizes eliminating KV blocks not referenced by the current candidate action branch; if it is still insufficient, priority scores are eliminated. The lowest-ranking KV block that has not been hit in the last four decoding steps is selected. If multiple KV blocks meet the same elimination criteria, the KV block that entered the on-chip cache earliest is prioritized for elimination. The warm KV exchange area is divided into two 128KB buffers. One buffer is used by the FPGA reconfigurable parallel computing array to read the KV blocks required for the current candidate action branch, and the other buffer is used by the off-chip memory access interface to prefetch the KV blocks required for the next candidate action branch. When the current decoding step ends, the two buffers switch read / write roles. Therefore, the warm KV is not permanently resident in the on-chip cache, but rather is prefetched and released at the branch level during the candidate action branch switching process.

[0087] The Key-Value (KV) hierarchy is dynamically updated based on the vehicle and task states within the planning cycle. When the comprehensive risk value R crosses low, medium, or high priority thresholds, the forward collision time (TTC) crosses 5s, 3s, or 2s thresholds, the candidate action branch set changes, new dynamic obstacles and safety constraints are updated in the current cycle, or the hot KV hit rate is below 80% in the last four decoding steps, the planning task scheduling control logic triggers a KV hierarchy update. During the update, the hierarchical KV cache scheduling unit recalculates the priority score Q of each KV cache block at the current decoding step boundary, first maintaining the safety KV lock unchanged, and then updating the hot, warm, and cold KV directory tables according to the Q value. For cache blocks newly promoted to hot KV, the off-chip storage access interface moves them into the hot KV residency area; for downgraded hot KV cache blocks, they are released to off-chip storage; for warm KV cache blocks that the next candidate action branch needs to access, they are written to the warm KV exchange area. If none of the above triggering conditions are met, the KV hierarchy result from the previous planning cycle is used, and only the attention hit count is updated.

[0088] The candidate action parallel prediction and safety verification unit is used to generate candidate action terms or candidate trajectory fragments in parallel according to the number of candidate action branches in the branch control register, and to complete consistency scoring and safety constraint verification before output. Therefore, this embodiment does not only accelerate the full inference of the VLA planning model, but also accelerates the scheduling of key planning tasks, candidate action branches, KV cache access and safety verification tasks in the vehicle planning constraint-oriented manner during the action decoding process.

[0089] In summary, taking the Alpamayo-type VLA planning model as an example, by setting up planning task scheduling and control logic, on-chip multiplexing pipeline units for action decoding data, hierarchical KV cache scheduling units, and candidate action parallel prediction and safety verification units, a dynamic scheduling strategy is generated by comprehensively considering vehicle speed, control cycle, road scene risks, and safety constraints. The KV cache is hierarchically layered according to semantic importance and branch-level prefetching and release are implemented. Combined with parallel decoding and consistency scoring verification of multiple candidate action branches, the problem of balancing real-time performance and safety in existing VLA planning schemes on in-vehicle embedded platforms is effectively solved. This achieves a hardware-accelerated integrated solution for the entire process from risk perception, dynamic scheduling, cache management, parallel decoding to safety verification, significantly improving the response time and trajectory safety of planning inference, and enhancing the engineering practical value of the method in dealing with dynamic risks and diverse safety constraints in complex traffic scenarios.

[0090] Furthermore, in other embodiments, the FPGA inference acceleration chip is not limited to a specific manufacturer or model of FPGA platform. When implemented with the Xilinx VERSAL AI series platform, matrix multiplication, attention weighting, and feedforward network calculations in action lexical decoding can be mapped to the AI ​​Engine array or the DSP computing array in the programmable logic, and data transfer between the hot KV cache, warm KV exchange area, and candidate action cache can be achieved through on-chip NoC and on-chip storage; security constraint verification, candidate action scoring, and scheduling register control can be deployed on the programmable logic side. When implemented with the Intel Agilex series platform, the aforementioned matrix multiplication and attention calculations can be mapped to a custom pipeline composed of DSP computing blocks and on-chip RAM, and cold KV and model weight access can be completed through off-chip DDR or HBM interfaces. The two types of platforms differ in their underlying computing units, on-chip interconnects, and storage interface forms, but both execute according to the scheduling logic of planned task priority, KV hierarchical caching, parallel prediction of candidate actions, and security verification, thus not affecting the implementation of the method of this invention.

[0091] In other embodiments, the planning task scheduling strategy can also be adjusted based on FPGA chip temperature, battery level, or power consumption status. When the chip temperature exceeds a preset temperature threshold, the planning task scheduling control logic reduces the parallelism of low-priority candidate action branches, suspends access to cold key-value pairs (KVs), and prioritizes the retention of safety KVs and hot KVs. When the battery level is below a preset power threshold, the planning task scheduling control logic reduces the number of non-safety-critical candidate action branches, reduces the computational accuracy of low-priority branches, and prioritizes the allocation of computational resources to deceleration, obstacle avoidance, braking, and safety verification tasks. Therefore, this invention can not only schedule planning tasks based on vehicle speed and road risk, but also adjust inference resource allocation based on FPGA hardware status and vehicle energy status.

[0092] Furthermore, such as Figure 4 As shown, this embodiment of the invention also provides a compilation and mapping process for VLA planning models of autonomous vehicles. This process converts the VLA planning model into hardware configuration files, instruction files, parameter files, and address tables suitable for FPGA execution before deployment to the FPGA platform, thereby enabling... Figure 3 The FPGA inference acceleration chip shown can perform action words or trajectory words decoding according to the planned task priority.

[0093] The VLA planning model to be deployed undergoes model compression, pruning, and quantization. Model compression and pruning reduce the computational load caused by low-importance weights, redundant channels, and fusionable operators. The start / stop relationships of candidate action branches are not permanently deleted at this stage; instead, branch execution templates are generated that can be selected by the runtime scheduling register. The quantization stage generates multiple sets of precision configuration parameters, including computation modes such as INT8 multiplication with FP16 accumulation and INT8 multiplication with INT16 accumulation. At runtime, the precision mode register selects the corresponding precision mode according to the planning task priority and candidate action branch type, without requiring reconfiguration of the entire FPGA bitstream.

[0094] The model structure analysis and intermediate representation generation stage analyzes the linear projection layer, multi-head attention layer, normalization layer, feedforward network layer, action terminator output head, and trajectory terminator output head in the VLA planning model, and generates a unified intermediate representation (IR). This intermediate representation records the type, input / output dimensions, tensor address, quantization scale, candidate action branch identifier, and key-value cache access relationships of each operator, providing a structured description for subsequent hardware mapping.

[0095] During the graph optimization and operator fusion stage, the compiler optimizes the computation graph based on the intermediate representation. For consecutive linear projections, normalizations, activation functions, and feedforward network computations, the compiler fuses them into operator blocks suitable for pipelined execution. For Q, K, and V projections and attention-weighted computations in the action decoding process, the compiler divides them into blocks according to the attention head dimension and the word dimension, enabling them to be mapped to the FPGA's reconfigurable parallel computing array. Through graph optimization and operator fusion, the number of times intermediate results are repeatedly written back to off-chip memory can be reduced, improving on-chip cache reuse efficiency.

[0096] In the action lexical decoding link and planning semantic strategy generation phase, the compiler generates an action decoding link based on the dependencies between action lexicals, trajectory lexicals, candidate action branches, and safety verification tasks. This link specifies the hidden state vector, action lexical embeddings, hot KV cache, warm KV cache, and intermediate results of candidate actions that need to be read at each decoding step. Simultaneously, the compiler generates a planning semantic strategy template. The risk level, KV cache hierarchy, and candidate action sorting in the planning semantic strategy follow the calculation rules of the aforementioned comprehensive risk value R, KV cache block priority score Q, and candidate branch priority P, respectively. This phase generates a runtime-callable strategy template and parameter configuration items; the actual priority selection, KV hierarchy update, and candidate branch start / stop are completed by the planning task scheduling control logic during vehicle operation.

[0097] During the hardware resource mapping and parallelism planning phase, the compiler maps matrix multiplication, attention computation, feedforward network computation, trajectory generation, and security verification tasks in the model to the reconfigurable parallel computing array, on-chip cache unit, off-chip memory access interface, and security verification pipeline in the FPGA platform. For security-critical action branches, the compiler generates execution templates with high parallelism; for low-priority action branches, the compiler generates execution templates corresponding to low-precision computation, delayed execution, or branch pruning. The runtime scheduling control logic selects the appropriate template based on the low, medium, and high priority of the planned tasks and writes it to the delay budget register, branch control register, precision mode register, and computing array configuration register.

[0098] During the on-chip cache planning and address allocation phase, the compiler allocates address ranges for the secure KV locking area, hot KV resident area, warm KV exchange area, action decoding activation cache area, and candidate action cache area according to the KV cache hierarchical strategy, and generates an on-chip address table and an off-chip address table. In one embodiment, the address allocation result corresponds to the aforementioned 2MB on-chip cache partitioning method, that is, 1MB is used as the KV hierarchical cache area, 512KB as the action decoding activation cache area, 256KB as the candidate action cache area, and the remaining 256KB as the index and state cache area. The address table is used to indicate the location and access order of model parameters, quantization parameters, KV cache blocks, and intermediate activation data between off-chip storage and on-chip cache.

[0099] After the FPGA platform completes the bitstream loading, the runtime scheduling unit does not regenerate the model structure or reconfigure the entire bitstream. Instead, it selects the appropriate instruction segment based on the current vehicle speed, road risk, safety constraints, and cache status, and updates the scheduling register, precision mode register, and address table pointer.

[0100] This invention provides a compilation and mapping process for VLA planning models of autonomous vehicles. By sequentially executing model compression and pruning with multi-precision quantization, structural analysis and intermediate representation generation, graph optimization and operator fusion, action decoding link and planning semantic strategy generation, hardware resource mapping and parallelism planning, and on-chip cache planning and address allocation, the VLA planning model is converted into hardware configuration files, instruction files, parameter files, and address tables suitable for FPGA platform execution. This effectively solves the problems of low inference efficiency and insufficient runtime flexibility caused by the mismatch between model structure and hardware resources in existing deployment schemes. It achieves integrated solution of the entire compilation and deployment process from model front-end optimization, computational graph reconstruction, hardware resource mapping to runtime multi-template selection, significantly improving the computational throughput and hardware adaptation efficiency of VLA planning models on FPGA platforms, and enhancing the operational flexibility and engineering practicality of the deployment scheme under different planning task priorities and candidate action branch configurations.

[0101] To achieve the above invention, such as Figure 5 As shown, this embodiment also provides an FPGA acceleration device 10 for VLA planning and scheduling of unmanned vehicles. The device 10 includes: The risk assessment module 100 is used to acquire multi-source input data of the current planning cycle of the unmanned vehicle. The multi-source input data includes environmental perception terms and vehicle status terms. Based on the vehicle speed information in the vehicle status terms and the obstacle information in the environmental perception terms, the module calculates the comprehensive risk value of the current road scene and determines the decoding delay budget based on the comprehensive risk value.

[0102] The scheduling layering module 200 is used to generate an action decoding scheduling strategy based on the comprehensive risk value. The action decoding scheduling strategy includes the number of candidate action branches and the allocation ratio of computing resources. According to the action decoding scheduling strategy, the KV cache generated during the VLA planning model decoding process is divided into a safe KV layer, a hot KV layer and a cold KV layer. The safe KV layer and the hot KV layer are loaded into the FPGA on-chip cache, while the cold KV layer is kept in off-chip storage.

[0103] The parallel decoding module 300 is used to configure the FPGA reconfigurable parallel computing array according to the computing resource allocation ratio. In the current decoding step, based on the shared hidden state vector in the on-chip cache and the KV data of the secure KV layer and the hot KV layer, it performs forward decoding calculations of multiple candidate action branches in parallel on the FPGA reconfigurable parallel computing array, generating multiple sets of candidate action word sequences and their corresponding candidate trajectory word sequences.

[0104] The safety verification module 400 is used to input the candidate trajectory word sequence into the safety constraint verification logic embedded in the FPGA hardware pipeline. The safety constraint verification logic includes vehicle kinematic continuity constraints and collision safety distance constraints. It performs parallel hard constraint verification on the candidate trajectory word sequence, eliminates candidate trajectory word sequences that do not meet any of the constraints, and retains candidate trajectory word sequences that pass the verification.

[0105] The scoring feedback module 500 is used to score the consistency of the verified candidate trajectory word sequences, select the best set as the action planning result of the current planning cycle and output it; at the same time, the KV cache blocks with a hit rate higher than a preset threshold in the current decoding step are retained in the on-chip cache unit for the next planning cycle.

[0106] The device of this invention effectively solves the problems of insufficient real-time performance and low safety verification efficiency of existing VLA planning schemes on vehicle-mounted embedded platforms due to resource constraints. It realizes the integrated hardware acceleration solution of the entire process from risk assessment, dynamic scheduling, parallel decoding to hard constraint verification and cache reuse, which significantly improves the real-time response capability, trajectory safety and resource utilization efficiency of autonomous vehicle decision planning, and enhances the engineering adaptability and deployment value of the device in complex dynamic scenarios.

[0107] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0108] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. An FPGA acceleration method for VLA planning and scheduling of unmanned vehicles, characterized in that, comprising: acquiring multi-source input data of the autonomous vehicle in the current planning cycle, wherein the multi-source input data includes environment perception tokens and vehicle state tokens; calculating a comprehensive risk value of the current road scene according to the vehicle speed information in the vehicle state tokens and the obstacle information in the environment perception tokens, and determining a decoding delay budget according to the comprehensive risk value; generating an action decoding scheduling strategy according to the comprehensive risk value, wherein the action decoding scheduling strategy includes the number of candidate action branches and a computing resource allocation ratio; dividing the KV cache generated during the decoding process of the VLA planning model into a secure KV layer, a hot KV layer and a cold KV layer according to the action decoding scheduling strategy, loading the secure KV layer and the hot KV layer to the FPGA on-chip cache, and retaining the cold KV layer in the off-chip memory; configuring the FPGA reconfigurable parallel computing array according to the computing resource allocation ratio, and in the current decoding step, based on the shared hidden state vector in the on-chip cache and the KV data of the secure KV layer and the hot KV layer, executing forward decoding computation of a plurality of candidate action branches in parallel on the FPGA reconfigurable parallel computing array, to generate multiple groups of candidate action token sequences and corresponding candidate trajectory token sequences respectively; inputting the candidate trajectory token sequences into the safety constraint verification logic solidified in the FPGA hardware pipeline, wherein the safety constraint verification logic includes a vehicle kinematics continuity constraint and a collision safety distance constraint, performing parallel hard constraint verification on the candidate trajectory token sequences, eliminating the candidate trajectory token sequences that do not satisfy any one of the constraints, and retaining the candidate trajectory token sequences that pass the verification; performing consistency scoring on the candidate trajectory token sequences that pass the verification, selecting the optimal group as the action planning result of the current planning cycle and outputting the same; meanwhile, retaining the KV cache blocks with a hit rate higher than a preset threshold in the current decoding step in the on-chip cache unit for use in the next planning cycle.

2. The method as described in claim 1, characterized in that, calculating the comprehensive risk value of the current road scene, comprising: Obtain the current vehicle speed v from the vehicle status lexical, and then... Output vehicle speed risk term Rv; obtain the forward obstacle distance d and relative approach speed Δv from the environmental perception terms, and the forward collision time. As input, via Output forward collision risk items ; Obtain lane risk items Road risk item Rr, and vehicle speed risk item Rv, and forward collision risk item Lane risk items The system incorporates a weighted fusion logic with the road risk factor Rr as input, performs a weighted summation according to preset weight coefficients, and outputs the comprehensive risk value of the current road scenario. .

3. The method as described in claim 1, characterized in that, generating the action decoding scheduling strategy according to the comprehensive risk value, comprising: inputting the comprehensive risk value R into a risk level comparator, and comparing the same with a preset low risk threshold Th_low and a preset high risk threshold Th_high: when R ≤ Th_low, the risk level comparator outputs a low risk mode identifier, and the action decoding scheduling strategy corresponding to the low risk mode includes: the decoding delay budget is a first delay budget T1, the number of candidate action branches is a first number N1, and the computing resource allocation ratio is a first ratio P1; when Th_low < R < Th_high, the risk level comparator outputs a medium risk mode identifier, and the action decoding scheduling strategy corresponding to the medium risk mode includes: the decoding delay budget is a second delay budget T2, the number of candidate action branches is a second number N2, and the computing resource allocation ratio is a second ratio P2, where T2 < T1, and N2 > N1; When R ≥ Th_high, the risk level comparator outputs a high-risk mode identifier. The action decoding scheduling strategy corresponding to the high-risk mode includes: decoding latency budget of the third latency budget T3, number of candidate action branches of the third quantity N3, and computational resource allocation ratio of the third ratio P3; where T3<T2,N3> N2, and the proportion of computing resources allocated to security constraint verification in the third proportion P3 is higher than the proportion of computing resources allocated to security constraint verification in the second proportion P2 and the first proportion P1.

4. The method as described in claim 1, characterized in that, The key-value cache generated during the VLA planning model decoding process is divided into a secure key-value layer, a hot key-value layer, and a cold key-value layer, including: Obtain the semantic tags of each KV cache block in the current decoding step. The semantic tags include whether it contains collision constraint terms, whether it contains lane boundary terms, whether it contains braking constraint terms, whether it is within the vehicle's forward preset distance range, and the number of attention hits in the previous planning cycle. The semantic tags are input into the KV hierarchical classifier, and the KV hierarchical classifier outputs the hierarchical identifier of each KV cache block according to the preset judgment rules. When the level is identified as a secure key-value layer, the corresponding key-value cache block is written into the secure key-value lock area in the on-chip cache unit. The secure key-value lock area is prohibited from being replaced during the current planning period. When the level is identified as a hot KV layer, the corresponding KV cache block is written into the hot KV resident area in the on-chip cache unit. The KV cache block in the hot KV resident area is allowed to be replaced according to a preset replacement policy when it is not referenced. When the layer is identified as a cold KV layer, the corresponding KV cache block is retained in the off-chip storage unit, and no off-chip read request is initiated in the current decoding step.

5. The method as described in claim 1, characterized in that, Pre-defined elimination strategies include: Get the current used capacity and total capacity of the hot KV cache area. When the used capacity reaches the total capacity and a new hot KV cache block needs to be written, trigger the eviction logic. The eviction logic first scans the reference counters of each KV cache block in the hot KV resident area and marks the KV cache blocks with a reference count of zero as eviction candidates; If there are multiple candidate KV cache blocks that can be evicted, the priority score Q of each candidate KV cache block is read, and the KV cache block with the lowest priority score Q is determined as the evicted block; where the priority score Q is calculated based on regional relevance, security criticality and historical attention hit rate; Remove the obsolete block from the hot KV resident area, release the on-chip cache space occupied by the obsolete block, and write the metadata of the obsolete block back to the KV directory table in the off-chip storage unit.

6. The method as described in claim 1, characterized in that, Perform forward decoding computations of multiple candidate action branches in parallel on an FPGA reconfigurable parallel computing array, including: Read the shared hidden state vector h_t of the current decoding step from the on-chip cache unit, and broadcast the shared hidden state vector h_t to N computation partitions in the FPGA reconfigurable parallel computing array, with each computation partition corresponding to a candidate action branch; Each computation partition reads the action lexical embedding vector and key-value data corresponding to its respective candidate action branch from the on-chip cache unit in parallel. Each computation partition independently performs attention computation, feedforward network computation, and output head computation to generate the current decoding step action word probability distribution for the corresponding candidate action branch, and obtains the action word for the current decoding step based on the action word probability distribution. The action tokens generated by each candidate action branch in the current decoding step are concatenated with the trajectory state of the previous decoding step. The corresponding candidate trajectory points are generated through the projection layer. When the number of generated trajectory points reaches the preset time domain length, the decoding of the current candidate action branch is stopped, and the complete candidate trajectory token sequence is output.

7. The method as described in claim 1, characterized in that, The safety constraint verification logic embedded in the FPGA hardware pipeline includes a vehicle kinematics continuity verification pipeline and a collision safety distance verification pipeline, which perform parallel hard constraint verification on candidate trajectory word sequences, including: The coordinates, velocity, acceleration, and heading angle data of each trajectory point in the candidate trajectory term sequence are input in parallel into the vehicle kinematics continuity verification pipeline; The vehicle kinematics continuity verification pipeline is based on a discrete kinematic bicycle model. It predicts the state of the (k+1)th trajectory point based on the state of the kth trajectory point, calculates the deviation between the predicted state and the actual state of the (k+1)th trajectory point in the candidate trajectory word sequence, and outputs the kinematic continuity flag of the candidate trajectory word sequence as invalid when the deviation exceeds the preset continuity threshold. The positions of each trajectory point in the candidate trajectory word sequence are input into the collision safety distance verification pipeline. The collision safety distance verification pipeline calculates the minimum distance between each trajectory point and the obstacle based on the obstacle position and size in the environmental perception word. When the minimum distance is less than the preset safety distance threshold, the collision safety flag of the candidate trajectory word sequence is output as invalid. Input the kinematic continuity flag and the collision safety flag into the logic AND gate. When either flag is invalid, the corresponding candidate trajectory term sequence is removed.

8. The method as described in claim 1, characterized in that, The consistency scoring of the verified candidate trajectory lexical sequences includes: Obtain the validated candidate trajectory word sequence and input the trajectory points in each candidate trajectory word sequence into the scoring pipeline; The scoring pipeline calculates the model confidence score, navigation consistency score, trajectory smoothness score, and safety margin score for each candidate trajectory term sequence in parallel. The model confidence score, navigation consistency score, trajectory smoothness score, and safety margin score are input into a weighted summer, which performs weighted summation according to preset scoring weight coefficients, and outputs the comprehensive score S_score of each candidate trajectory word sequence. The comprehensive score S_score of each candidate trajectory word sequence is input into the maximum value selector. The maximum value selector outputs the candidate trajectory word sequence with the highest comprehensive score S_score as the optimal action planning result for the current planning cycle.

9. The method as described in claim 1, characterized in that, KV cache blocks with a hit rate higher than a preset threshold in the current decoding step are retained in the on-chip cache unit for use in the next planning cycle, including: At the end of the current decoding step, read the number of times each KV cache block in the on-chip cache unit was referenced in this decoding step; Mark KV cache blocks with a citation count greater than zero as hit blocks, and calculate a hit rate H according to the total number of hits of the hit blocks in the latest consecutive L decoding steps, where L is a preset statistical window length; Input the hit rate H into a comparator, and when H is greater than or equal to a preset hit rate threshold H_th, mark the corresponding KV cache block as a reserved block, and maintain the resident state of the reserved block in the on-chip cache unit; When H is less than H_th, mark the corresponding KV cache block as a to-be-released block, and preferentially release the cache space occupied by the to-be-released block when the capacity of the on-chip cache unit is insufficient.

10. An FPGA acceleration device for VLA planning and scheduling of unmanned vehicles, characterized in that, comprising: a risk assessment module, configured to acquire multi-source input data of the current planning cycle of the unmanned vehicle, where the multi-source input data includes environment perception tokens and vehicle state tokens; calculate a comprehensive risk value of the current road scene according to the vehicle speed information in the vehicle state tokens and the obstacle information in the environment perception tokens, and determine a decoding delay budget according to the comprehensive risk value; a scheduling layering module, configured to generate an action decoding scheduling strategy according to the comprehensive risk value, where the action decoding scheduling strategy includes the number of candidate action branches and a computing resource allocation ratio; divide KV caches generated during the decoding process of the VLA planning model into a safe KV layer, a hot KV layer and a cold KV layer according to the action decoding scheduling strategy, load the safe KV layer and the hot KV layer into the FPGA on-chip cache, and retain the cold KV layer in the off-chip memory; a parallel decoding module, configured to configure the FPGA reconfigurable parallel computing array according to the computing resource allocation ratio, and in the current decoding step, based on the shared hidden state vectors in the on-chip cache and the KV data of the safe KV layer and the hot KV layer, perform forward decoding computation of multiple candidate action branches in parallel on the FPGA reconfigurable parallel computing array, to generate multiple groups of candidate action token sequences and corresponding candidate trajectory token sequences respectively; a safety verification module, configured to input the candidate trajectory token sequences into the safety constraint verification logic solidified in the FPGA hardware pipeline, where the safety constraint verification logic includes a vehicle kinematics continuity constraint and a collision safety distance constraint, perform parallel hard constraint verification on the candidate trajectory token sequences, eliminate candidate trajectory token sequences that do not satisfy any of the constraints, and retain candidate trajectory token sequences that pass the verification; a scoring feedback module, configured to perform consistency scoring on the candidate trajectory token sequences that pass the verification, select the optimal group as the action planning result of the current planning cycle and output the same; meanwhile, retain KV cache blocks with a hit rate higher than the preset threshold in the current decoding step in the on-chip cache unit for the next planning cycle.

Images