Autonomous Vehicle Runtime Monitoring With Vision-Language Risk Scoring

The patent introduces a large language model (LLM) as an intermediary component between the heuristic monitoring system and the safety assessment process. The LLM receives sensor data, obstacle detections, and vehicle behavior plans, then generates natural language explanations that capture broader environmental context. This intermediary enables the system to maintain the computational efficiency of heuristics while gaining contextual understanding through the LLM's ability to reason about environmental semantics and generate human-like safety assessments.

The patent implements a feedback mechanism where the LLM continuously monitors and evaluates the vehicle's understanding of the environment by generating natural language explanations of detected obstacles and traffic elements. The system compares these explanations against the original sensor data and heuristic assessments, providing feedback that identifies misinterpretations. This feedback loop enables the system to correct erroneous obstacle identification and improve the reliability of vehicle control decisions through iterative refinement of environmental understanding.