Autonomous Driving Compute Latency Budget Allocation Across Pipeline
Overview of Technical Issues:
The compute scheduling allocator insufficiently distributes the latency budget across the autonomous driving pipeline stages (perception, decision-making, control), causing pipeline imbalance where some stages become bottlenecks while others are underutilized, resulting in the system failing to meet end-to-end real-time latency requirements critical for safe vehicle operation; the goal is to achieve optimal latency allocation that ensures consistent real-time performance across all driving scenarios.
Solution directions generated for this problem
Problem Direction 1 :
ImproveLatency budget allocation precision
VSConstraintAllocator computational overhead
Inspiration 1 : Cross-domain reference
Application Principle: #10 Preliminary action
Cross-domain applicability
System, method, and apparatus for providing dynamic, prioritized spectrum management and utilization
Innovative Solution Refine solution
Offline scenario-class allocation profile library with runtime fast-match selection
Pre-compute allocation profiles offline for scenario classes to eliminate runtime overhead
How to solve :
- Offline phase: analyze historical driving data to extract six canonical scenario classes (highway-sparse, highway-dense, urban-low, urban-high, parking, transition) and compute optimal latency distribution ratios for each class using exhaustive optimization, storing results as lookup tables (18 entries: 3 stages × 6 scenarios) with allocation ratios accurate to 1ms granularity
- Runtime phase: deploy a lightweight classifier using three sensor-derived features (object count per frame, lane change frequency per 10s, average velocity) mapped through a pre-trained decision tree (depth ≤4, inference time <0.3ms) to select the matching scenario class index
- Allocation execution: retrieve the pre-computed ratio triplet (perception%, decision%, control%) from the lookup table via direct memory access (<0.2ms), apply to the 100ms total budget, and distribute to pipeline stages without iterative computation
Expected Effect : Allocation overhead reduced from 5-15ms to <0.5ms (>90% reduction); precision maintained at <10% deviation; end-to-end latency stabilized at 95-105ms across scenarios
Risk Control :
- Scenario class coverage insufficient for edge cases
- classifier misclassification under sensor noise
- lookup table memory access contention in multi-threaded execution
Problem Direction 2 :
ImproveScheduling adaptability to workload
VSConstraintScheduling algorithm complexity
Inspiration 1 : Cross-domain reference
Application Principle: #1 Segmentation
Cross-domain applicability
Methods for distributing software-determined global load information
Innovative Solution Refine solution
Scenario-specific pre-computed allocation profile library with runtime classifier selection
Offline profiling creates allocation profiles for scenarios
How to solve :
- Offline phase: Profile 8-12 representative driving scenarios (highway cruise, urban dense traffic, parking maneuver, emergency braking) under controlled testing
- extract optimal latency allocation ratios (perception:decision:control) for each scenario achieving <10% deviation from actual requirements
- store as pre-computed allocation profile library with scenario fingerprints (object count range, velocity variance, lane change frequency)
- Runtime phase: Deploy lightweight scenario classifier using decision tree (depth ≤5 levels) mapping real-time sensor metrics (detected objects 0-50, speed 0-120 km/h, steering rate 0-180°/s) to scenario class in <0.8ms
- apply corresponding pre-computed profile directly without optimization computation
- Quality control: Validate each profile achieves end-to-end latency <100ms with <15ms variance across 1000 test runs per scenario
- classifier accuracy ≥92% verified on 50,000 labeled frames
- fallback to conservative balanced allocation (40:35:25 ratio) when classifier confidence <0.7
Expected Effect : Adaptability maintained across all scenarios; algorithm overhead reduced from 5-15ms to <0.8ms (94% reduction); classifier complexity 180 nodes vs 5000+ in adaptive predictor
Risk Control :
- Scenario coverage gaps in rare edge cases
- classifier misclassification under sensor noise
- profile library storage requires 2-5 MB memory
Problem Direction 3 :
ImproveSystem response time stability
VSConstraintAllocator computational overhead
Inspiration 1 : Cross-domain reference
Application Principle: #11 Beforehand cushioning
Cross-domain applicability
Handling of parameters provided in release / suspend
Innovative Solution Refine solution
Pre-allocated time buffer zones at pipeline stage boundaries for latency stability
Reserve fixed time buffers at critical boundaries to absorb workload variations
How to solve :
- Allocate 15ms slack buffer at perception-decision boundary and 10ms buffer at decision-control boundary during system initialization
- these static reserves absorb workload spikes without triggering reallocation computation
- Implement queue depth monitoring at each buffer zone with threshold triggers: when queue occupancy exceeds 80% for 3 consecutive cycles, release additional 5ms from global reserve pool
- Deploy dual-threshold control: normal operation uses pre-allocated buffers (zero overhead), emergency mode (queue >90%) activates one-time reallocation with 8ms overhead, then returns to buffer-based operation
Expected Effect : Latency variance reduced to <30ms; overhead <0.5ms per cycle (97% reduction); end-to-end latency stable at 85-95ms across scenarios
Risk Control :
- Buffer sizing insufficient for extreme workload spikes
- queue monitoring adds 0.3-0.5ms measurement latency
- emergency reallocation frequency exceeds design assumptions
Problem Direction 4 :
ImproveLatency budget allocation precision
VSConstraintMust not deteriorate
Inspiration 1 : Cross-domain reference
Application Principle: #10 Preliminary action
Cross-domain applicability
Wireless communication method and wireless communication terminal, which use discontinuous channel
Innovative Solution Refine solution
Offline scenario-profiled latency allocation with runtime template matching
Pre-compute allocation templates offline for scenario classes then apply at runtime
How to solve :
- Offline phase: Profile 8-12 representative driving scenarios (highway cruise, urban dense traffic, intersection navigation, parking) under controlled test conditions, recording perception/decision/control stage latency distributions at 10ms resolution over 500+ cycles per scenario
- extract optimal allocation ratios (e.g., highway 30:40:30, urban 45:35:20) and encode into allocation templates with stage-specific time budgets and ±15% tolerance bands
- Runtime phase: Deploy lightweight scenario classifier using 6 input features (object count, lane complexity, velocity, GPS road type) mapped through decision tree to select matching template in <0.8ms
- apply selected template's allocation ratios directly without iterative optimization
- Quality control: Validate template coverage by testing against 200 unseen scenario samples—acceptance criterion is ≥92% scenarios achieve <10% allocation deviation and <100ms end-to-end latency
- monitor runtime queue depths at stage boundaries every 50ms, trigger template re-selection if any queue exceeds 3 frames for >200ms consecutive duration
Expected Effect : Allocation overhead reduced from 5-15ms to <0.8ms; allocation deviation <10% for 92%+ scenarios; end-to-end latency stability 85-105ms
Risk Control :
- Scenario classifier misclassification under edge cases
- template library insufficient coverage for rare scenarios
- template staleness as software versions update
