Binary Descriptor Generation Using Inertial Rotation Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion estimation algorithms for embedded video applications in smartphones and tablets face challenges in accurately estimating camera motion, especially under rotation and viewpoint changes, due to limitations in robustness to geometric variations and computational complexity.
Innovation Solution
A method is proposed to generate binary descriptors that incorporate tridimensional rotation information from inertial sensors, such as gyroscopes and accelerometers, to improve the robustness of motion estimation algorithms by processing patterns of points pairs around key points with bidimensional and tridimensional rotation information, reducing computational load by applying rotation only to specific point pairs rather than the entire image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If binary descriptors are generated using traditional methods (e.g., BRIEF) without rotation compensation, then computational complexity is low, but robustness to geometric variations and accuracy in motion estimation deteriorate under rotation and viewpoint changes
Solution Approach 1:
The patent segments the image into local patches around key points and processes each patch independently with rotation compensation. Instead of rotating the entire image, only small local regions are transformed, significantly reducing computational complexity while maintaining robustness to geometric variations.
Solution Approach 2:
The patent introduces tridimensional rotation information from inertial sensors to compensate for in-plane rotations in binary descriptors. By incorporating the third dimension (depth/rotation angle) into the descriptor generation process, the system achieves rotational invariance without significantly increasing computational burden.
2Reliability
If rotation compensation is applied to the entire image, then robustness to in-plane rotation improves, but computational load increases significantly
Solution Approach 1:
The patent divides the image into local patches around detected key points and applies rotation compensation only to these small regions. This segmentation approach reduces the computational load from rotating the entire image to rotating only small local areas, while still achieving robustness to in-plane rotation.
Solution Approach 2:
The patent applies different processing (rotation compensation) only to specific local regions (patches around key points) rather than the entire image. This local quality approach ensures that computational resources are focused where they are most needed for motion estimation accuracy.
3Measurement precision
If inertial sensor data is integrated into binary descriptor generation, then accuracy and robustness of motion estimation improve, but device complexity and computational requirements increase
Solution Approach 1:
The patent merges data from multiple sources (image sensor and inertial sensors) to generate enhanced binary descriptors. By combining visual information with motion data from gyroscopes and accelerometers, the system achieves more accurate and robust motion estimation while leveraging existing sensor capabilities.
Solution Approach 2:
The patent uses the device's own inertial sensors (already present for other functions like orientation awareness and step counting) to compensate for camera motion in video sequences. This self-service approach leverages existing hardware resources without adding external complexity.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The device (12) comprises generating means (120) configured to generate a binary descriptor associated with a given point in a current frame of a succession of video frames obtained by an apparatus such as an image sensor (10), said generating means comprising first means configured to determine a pattern of points pairs around said given point in the current frame, and comparison means configured to perform intensity comparison processing between the two points of each pair. Said apparatus (10) is likely to move in a rotation between the previous frame and the current frame, and said generating means further comprises processing means configured to process said pattern of points of said current frame with a tridimensional rotation information representative of said apparatus rotation between the previous frame and the current frame and obtained from inertial measurements provided by at least one inertial sensor (11).