Multi-View Video Streaming with Driving-Based View Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video transmission approaches for teleoperated driving (TOD) focus on optimizing individual camera views or maintaining equal quality across all views, but fail to adapt effectively to the dynamic importance of camera views based on the current driving situation, particularly in mobile networks with limited resources.
Innovation Solution
A computing device that receives image data from multiple camera views, attributes priority to each view based on the current driving situation, and generates individual video streams with optimized spatial and temporal resolution, using a region of interest (ROI) mask dynamically adapted to the vehicle's state, to prioritize critical views and maximize video quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If equal quality is maintained for all camera views, then overall video transmission quality is preserved, but network resources are wasted on non-critical views and critical views do not receive sufficient bandwidth
Solution Approach 1:
The patent applies local quality by assigning different quality levels to different camera views based on their importance to the teleoperation task. Critical views (e.g., forward-facing cameras during daytime) receive higher quality encoding with more bits, while non-critical views (e.g., rear-facing cameras during nighttime) receive lower quality encoding with fewer bits. This resolves the contradiction by maintaining high reliability for critical views while improving network resource efficiency overall.
Solution Approach 2:
The patent implements dynamic quality adjustment where the quality of each camera view changes in real-time based on driving conditions, time of day, and detected events. The system continuously adapts the bitrate allocation to match current operational needs, allowing critical views to receive sufficient bandwidth when needed while reducing resources for non-critical views during normal conditions.
2Reliability
If high video quality is provided for all camera views, then situation awareness is improved, but network bandwidth consumption exceeds available transmission rates in mobile networks
Solution Approach 1:
The patent applies local quality by differentiating between critical and non-critical camera views and allocating bandwidth accordingly. During daytime, forward-facing cameras receive high quality streams while rear-facing cameras receive lower quality. During nighttime, the allocation reverses. This ensures situation awareness is maintained for critical views while keeping total bandwidth consumption within available transmission rates.
Solution Approach 2:
The patent changes encoding parameters (bitrate, resolution, frame rate) dynamically based on the importance of each camera view. The system adjusts these parameters in real-time to match network conditions and operational requirements, ensuring high quality for critical views while reducing parameters for non-critical views to fit within available bandwidth.
3Reliability
If individual camera views are optimized independently, then each view achieves its maximum quality, but the overall multi-view transmission does not efficiently utilize limited network resources
Solution Approach 1:
The patent merges the optimization of multiple camera views into a unified system that considers the driving situation as a whole. Instead of independently optimizing each view, the system jointly adapts all views based on global context (time of day, driving conditions, detected events), ensuring efficient utilization of limited network resources while maintaining appropriate quality for each view's specific importance.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A computing device (1) is programmed to: receive (21) image data corresponding to at least two individual camera views of an environment of a vehicle (3); receive (22) information about a current driving of the vehicle (3); attribute (23) a view priority to each of the individual camera views based on the information about the current driving situation; and generate (24) an individual video stream from the image data for each of the individual camera views in dependence on the view priorities.