Lidar-Camera Object Tracking With ROI Prediction for Indoor Mobility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing indoor autonomous mobility systems face challenges in real-time object tracking due to limitations in sensor data fusion, low computational performance, and reliance on remote systems with potential communication disruptions.
Innovation Solution
A system and method for real-time object tracking in indoor autonomous mobility equipped with Lidar and camera, which predicts the object search range using Lidar data and mobility movement data through an artificial neural network, allowing efficient calculation and accurate tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If object search is performed through artificial neural network on entire image, then object search accuracy is improved, but processing speed deteriorates (maximum 30 FPS)
Solution Approach 1:
The patent divides the entire image into multiple regions of interest (ROIs) based on Lidar depth information and mobility movement data. Instead of performing object search on the entire image, the system segments and processes only the relevant regions where objects are likely to be located, thereby reducing computational load while maintaining search accuracy.
Solution Approach 2:
The system applies different processing strategies to different regions of the image. Regions identified as high-probability object locations through Lidar and movement data receive focused object search processing, while other regions are processed differently or skipped, optimizing the balance between accuracy and speed.
2Speed
If sensor data fusion is performed at 100 FPS (10ms control cycle), then real-time tracking performance is improved, but computational complexity increases due to neural network processing requirements
Solution Approach 1:
The system performs preliminary processing by generating regions of interest using Lidar data and mobility movement information before executing the computationally intensive object search algorithm. This preliminary segmentation reduces the amount of data that requires neural network processing, enabling real-time performance at 100 FPS without excessive computational complexity.
3Measurement precision
If object tracking relies on remote system, then tracking accuracy is improved, but system reliability deteriorates when communication is cut off or mobility deviates significantly from planned path
Solution Approach 1:
The system uses its own onboard sensors (Lidar and movement data sensors) to generate regions of interest and perform object tracking independently. This self-service capability allows the mobility to maintain accurate object tracking without relying on external remote systems, ensuring continued operation even when communication is disrupted or the mobility deviates from its planned path.
Data Source
AI summary
In an embodiment, a method of tracking an object includes acquiring an image by a camera, starting, by a controller, an object search for the entire image through an object search algorithm, determining, by the controller, whether the object search is completed, in response to determining that the object search is not completed, predicting, by the controller, an object search range based on mobility movement data and Lidar data, and performing, by the controller, the object search on the predicted object search range through the object search algorithm. In an embodiment, a system of tracking an object for a mobility is configured to perform the method of tracking an object for a mobility.