Monocular Surveillance Camera 3D Tracking for Cooperative Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cooperative vehicle infrastructure systems (CVIS) face challenges in efficiently integrating multiple sensors for vehicle detection and tracking, leading to high costs and demanding computing resources, while autonomous vehicles lack effective cooperative sensing and maneuvering capabilities due to limited perception range and inability to coordinate with other vehicles.
Innovation Solution
A system utilizing monocular surveillance cameras and computing devices for simultaneous 2D and 3D object detection and tracking, combined with a GC-LSTM network for object association, transforms video frames into bird's-eye view for cooperative maneuvering and risk warning, leveraging 5G communication for vehicle-to-infrastructure interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple types of sensors (high resolution cameras, UHF band radio waves) are used for V2I application, then the efficiency and accuracy of vehicle detection and tracking is improved, but the cost and computing resource requirements increase significantly
Solution Approach 1:
The patent extracts and focuses on a single critical sensor type (monocular surveillance camera) from the multi-sensor suite, eliminating the need to integrate and process data from multiple sensor types while maintaining effective vehicle detection and tracking capabilities
Solution Approach 2:
The monocular surveillance camera is designed to perform multiple functions including vehicle detection, tracking, and providing spatial information for cooperative maneuvering, replacing the need for multiple specialized sensors
2Reliability
If multiple types of sensors are deployed for cooperative sensing, then the sensing range and detection capability are improved, but the computational overhead and resource integration become demanding
Solution Approach 1:
The patent extracts the essential sensing function from complex multi-sensor systems, using only monocular camera data for vehicle detection and tracking, thereby reducing computational overhead while maintaining reliable cooperative sensing
Solution Approach 2:
The system uses computationally efficient processing of single-camera data rather than expensive multi-sensor fusion, achieving reliable detection with lower computational resource consumption
3Device complexity
If onboard sensors alone are used for autonomous vehicle perception, then the system remains simple, but the perception range is limited and vehicles cannot cooperate effectively
Solution Approach 1:
The patent introduces infrastructure-based monocular surveillance cameras as intermediaries that extend the perception range of autonomous vehicles beyond what onboard sensors can achieve, enabling cooperative sensing without significantly increasing vehicle complexity
Solution Approach 2:
The system transitions from vehicle-centered onboard sensing to infrastructure-centered external sensing, fundamentally changing the dimension and scope of perception capability while maintaining system simplicity
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and a method for cooperative maneuvering and cooperative risk warning of vehicles. The system includes monocular surveillance cameras, local computing devices, and a master server. Each local computing device receives video frames from the camera, detects and tracks vehicles from the video frames, and converts the video frames to bird-view. Each detected vehicle is represented by a detection vector having first dimensions representing two dimensional (2D) parameters of the vehicle and second dimensions representing three dimensional (3D) parameters of the vehicle. Tracking of the vehicles is performed by minimizing loss calculated based on the first dimensions and the second dimensions of the detection vectors. The master server receives bird-views from different computing devices and combines the bird-views into a global bird-view, and performs cooperative maneuvering and cooperative risk warning of the vehicles using the global bird-view.