Global Motion Compensation Using Motion Sensor Image Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies struggle to accurately compensate for global motion between image frames, leading to inefficiencies in video encoding and image signal processing due to challenges in setting effective search areas for block matching.
Innovation Solution
A processor-implemented method using a transformation determination neural network model that estimates motion data from an image sensor and a motion sensor to generate a transformation model, allowing for accurate compensation of global motion between image frames and optimizing search areas for block matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional block matching is used without motion compensation, then the process is simple, but the accuracy of motion estimation deteriorates
Solution Approach 1:
The patent applies preliminary action by using motion sensor data to estimate global motion parameters before performing block matching. The transformation model is constructed in advance based on motion sensor information, which pre-compensates for camera motion effects. This preliminary motion compensation narrows the search area and improves motion estimation accuracy without requiring complex post-processing during block matching.
Solution Approach 2:
The patent introduces motion sensor data as an intermediary element between the camera and the block matching process. The motion sensor acts as a mediator that provides independent motion information, which is then integrated into the transformation model. This intermediary motion data helps bridge the gap between simple block matching and complex optical flow methods, improving accuracy without proportional increase in complexity.
2Measurement precision
If the search area is not transformed according to global motion, then the block matching process is straightforward, but the matching accuracy deteriorates due to global motion
Solution Approach 1:
The search area transformation is performed as a preliminary action before block matching. The transformation model, constructed using motion sensor data, pre-adjusts the search area coordinates to account for global motion. This ensures that when block matching is performed, the search area is already optimally positioned, improving matching accuracy without adding complexity during the actual matching process.
Solution Approach 2:
The patent applies parameter changes by transforming the search area parameters (coordinates and dimensions) based on global motion parameters derived from motion sensor data. The transformation model adjusts the search area parameters to compensate for camera motion, ensuring that the search area remains aligned with the actual motion of objects in the scene. This dynamic parameter adjustment improves block matching accuracy while maintaining a relatively simple transformation process.
3Reliability
If motion sensor data is integrated into the transformation model, then global motion compensation is improved, but the system complexity increases
Solution Approach 1:
The motion sensor serves as an intermediary that provides reliable motion information without requiring complex system changes. The patent integrates motion sensor data into the transformation model through a straightforward process: the motion sensor outputs global motion parameters, which are directly used to construct the transformation model. This intermediary approach improves global motion compensation reliability while avoiding the need for complex multi-sensor fusion or sophisticated algorithms.
Solution Approach 2:
The transformation model serves multiple functions: it transforms search areas, compensates for global motion, and improves block matching accuracy. By making the transformation model multi-functional, the patent avoids adding separate processing stages for each function, thereby improving global motion compensation reliability without proportionally increasing system complexity. The same transformation model handles multiple tasks efficiently.
4Reliability
If a larger search area is used for block matching, then the likelihood of finding matching blocks increases, but the computational cost increases
Solution Approach 1:
The transformation model performs preliminary action by pre-adjusting the search area based on global motion parameters from motion sensor data. This preliminary adjustment concentrates the search area around the most likely location of matching blocks, eliminating the need to search large areas. The search area is optimally positioned before block matching begins, improving block matching reliability while minimizing the number of comparisons needed, thus maintaining high encoding efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processor-implemented method includes estimating a transformation model using a transformation determination neural network model, provided motion sensor detected motion data representing motion of an image sensor with respect to a first image frame and a subsequent second image frame from captured by the image sensor, to perform a transformation based on global motion between the first image frame and the second image frame; and generating output image data by combining, by using the transformation model, the first image frame and the second image frame.