Roadside Camera Vehicle Pose Estimation Using Homography YOLO
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting and extracting vehicle kinematic data such as position, velocity, orientation, and yaw rate from real-time images are inaccurate and challenging, particularly in complex environments.
Innovation Solution
A modified neural network, specifically YOLOv5, is trained to map roadside view images to a top view perspective, incorporating yaw angle information and using a homography transformation to accurately detect and predict kinematic data of vehicles, employing image stabilization and bounding box analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a standard object detection network is used to detect vehicles and extract kinematic data, then the system is simple to implement, but the accuracy of vehicle position, orientation, and velocity extraction is insufficient
Solution Approach 1:
The neural network output is segmented into multiple independent parameters: bounding box coordinates (x, y, w, h) and yaw angle (θ). This segmentation allows the network to specialize in extracting each kinematic parameter independently, improving overall measurement precision while maintaining a manageable network structure through modular output design.
Solution Approach 2:
The network transforms 2D image coordinates into 3D kinematic space by adding the yaw angle dimension. This dimensional extension enables the system to extract orientation information that is not directly observable from standard 2D bounding boxes, significantly improving vehicle pose estimation accuracy without requiring complex 3D detection architectures.
2Measurement precision
If GPS and LIDAR sensors are used to collect training data for vehicle detection, then the training data is high-precision, but the system cost and complexity increase significantly
Solution Approach 1:
The system creates a virtual copy of the high-precision GPS-LIDAR training data in the form of synthetic bounding box annotations and ground truth kinematic parameters. These copied labels are then used to train the neural network, allowing the model to learn from high-precision data without requiring the physical sensors during deployment, thus reducing system complexity while maintaining training data accuracy.
Solution Approach 2:
Manual or automated annotation processes serve as intermediaries between the raw GPS-LIDAR data and the neural network training process. These intermediaries convert complex sensor data into simplified bounding box coordinates and kinematic parameters that the network can efficiently learn from, reducing the complexity of the overall system while preserving the precision benefits of the original sensors.
3Ease of operation
If bounding box size is used to estimate vehicle distance, then the method is simple to implement, but the accuracy of position estimation is insufficient
Solution Approach 1:
The system merges multiple features from the bounding box (position coordinates x, y; dimensions w, h; and orientation θ) into a comprehensive vehicle state estimation. By combining these features rather than relying on a single bounding box size metric, the system maintains the simplicity of bounding box-based detection while significantly improving position and distance estimation accuracy through multi-parameter analysis.
Data Source
AI summary
A methodology is developed to extract vehicle kinematic information from roadside cameras at an intersection using deep learning. The ground truth data of top view bounding boxes are collected with the help of unmanned aerial vehicles (UAVs). These top view bounding boxes containing vehicle position, size, and orientation information, are converted to the roadside view bounding boxes using homography transformation. The ground truth data and the roadside view images are used to train a modified YOLOv5 neural network, and thus, to learn the homography transformation matrix. The output of the neural network is the vehicle kinematic information, and it can be visualized in both the top view and the roadside view. In this algorithm, the top view images are only used in training, and once the neural network is trained, only the roadside cameras are needed to extract the kinematic information.