Computer Vision Rotation Preprocessing via Gravity Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer vision systems face challenges in training efficiency and accuracy when dealing with images that have arbitrary rotational variances, as machine learning sub-systems are prone to errors in recognizing features across multiple images with varying rotational orientations.
Innovation Solution
The method involves rotationally preprocessing images based on a gravity direction estimation, which includes generating a rotationally preprocessed input image by applying transformations such as rectifying the image plane to be parallel or perpendicular to the gravity direction, and modifying convolutional kernels to be steered according to the gravity direction, thereby producing responses invariant to rotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine learning sub-systems process images with arbitrary rotational variances directly, then the system can handle diverse image orientations, but the recognition accuracy decreases due to errors in identifying visual features across different rotational orientations
Solution Approach 1:
The patent applies preliminary action by performing rotational preprocessing on images before they are fed into the machine learning sub-system. The system estimates gravity direction and rotates images to a standardized orientation (e.g., upright position) in advance, ensuring that all subsequent processing occurs on consistently oriented images. This pre-processing step eliminates rotational variance issues that would otherwise degrade recognition accuracy.
Solution Approach 2:
The patent changes the orientation parameter of images from arbitrary rotational states to a standardized orientation based on gravity direction estimation. By transforming the rotational state of images to a consistent reference frame, the system maintains high recognition accuracy while still being able to process images captured at various orientations.
2Measurement precision
If the system applies rotational preprocessing transformations to all input images, then recognition accuracy improves, but processing time and computational complexity increase
Solution Approach 1:
The patent applies partial action by selectively performing rotational preprocessing only when necessary. The system determines whether an image requires rotation based on gravity direction estimation and orientation analysis, applying transformations only to images that deviate from the desired orientation. This avoids unnecessary processing of already correctly oriented images, reducing overall processing time while maintaining accuracy where needed.
3Productivity
If the machine learning sub-system is trained without rotational preprocessing, then training efficiency is maintained, but the system produces erroneous results when encountering rotated images during deployment
Solution Approach 1:
The patent applies preliminary action during the deployment phase by performing rotational preprocessing on input images before they are processed by the machine learning sub-system. This allows the model to be trained on standardized oriented images for efficient training, while the preprocessing step ensures that deployed images are also in the expected orientation, guaranteeing reliable results without requiring the model to learn rotational invariance.
Data Source
AI summary
In one implementation, a method includes: obtaining an input image, wherein the input image is captured by an image sensor having a rotational orientation with respect to a direction of gravity; obtaining a gravity direction estimation associated with the rotational orientation of the sensor; generating, from the input image, a rotationally preprocessed input image by applying one or more transformations to the input image based on the gravity direction estimation; providing the rotationally preprocessed input image to the machine learning sub-system; and identifying, using the machine learning sub-system, a visual feature within the rotationally preprocessed input image.