Image Alignment Using Plane-Specific Projection Matrices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing techniques struggle to accurately align images with objects having depth, as they fail to account for varying misalignment amounts across different planes, leading to artifacts in combined images.
Innovation Solution
An image processing apparatus and method that calculates a projection conversion matrix for each plane with different misalignment amounts, generates alignment images, and creates a plane map to select the appropriate projection conversion matrix for each pixel based on difference values, reducing selection errors through expansion and contraction processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single projection conversion matrix is used to align images, then the alignment process is simple and fast, but artifacts occur in combined images when the subject has multiple planes with different misalignment amounts
Solution Approach 1:
The patent divides the image into multiple planar regions based on depth information, and calculates a separate projection conversion matrix for each plane. This segmentation allows each plane to be aligned with its own optimal matrix, preventing artifacts caused by using a single matrix for all planes while maintaining reasonable processing efficiency through region-based parallel computation.
Solution Approach 2:
The patent applies different projection conversion matrices to different spatial regions (planes) of the image according to their specific misalignment characteristics. By making the alignment parameters locally optimized for each plane rather than globally uniform, the patent achieves high alignment precision for multi-plane subjects while the localized processing keeps computational complexity manageable.
2Manufacturing precision
If multiple projection conversion matrices are calculated for different planes to suppress artifacts, then alignment precision is improved, but device complexity and processing time increase
Solution Approach 1:
The patent segments the image into planar regions using depth maps or stereo vision techniques, which provides a natural framework for applying multiple projection conversion matrices. This segmentation approach organizes the complexity into manageable regions, allowing parallel processing of each plane's alignment independently, thus reducing overall processing complexity despite using multiple matrices.
Solution Approach 2:
The patent performs preliminary depth estimation and plane segmentation before the actual alignment process. By pre-identifying which pixels belong to which planes and determining the appropriate projection conversion matrix for each plane in advance, the patent simplifies the subsequent alignment operation to a matter of selecting and applying pre-computed matrices, thereby reducing real-time processing complexity.
3Manufacturing precision
If projection conversion matrices are calculated for each plane to achieve accurate alignment, then alignment precision is improved, but the processing time increases
Solution Approach 1:
The patent divides the alignment task into independent plane-specific subtasks that can be processed in parallel. By segmenting the image into distinct planes and calculating projection conversion matrices for each plane simultaneously rather than sequentially, the patent reduces total processing time while maintaining high alignment precision for each plane.
Solution Approach 2:
The patent performs preliminary depth map generation and plane identification before the alignment stage. This preliminary action pre-organizes the computational workload by identifying all planes and their corresponding optimal projection conversion matrices in advance, allowing the actual alignment to be executed efficiently by simply applying the pre-determined matrices to their respective regions without iterative optimization during the time-critical alignment phase.
Data Source
AI summary
An image processing apparatus includes at least one processor configured to execute processes including: calculating a misalignment amount of each pixel of a reference image relative to a standard image; and combining the reference image converted based on the calculated amount with the standard image. The calculating includes: calculating a projection conversion matrix for each of planes with different misalignment amounts in the reference image; generating a plane map in which the plane to which each pixel of the reference image belongs and the matrix to be applied to each plane are selected based on a difference value between the standard image and each of the alignment images converted from the reference image by using each calculated matrix; suppressing a selection error of the matrix; and calculating the misalignment amount for each of the planes based on the plane map in which the selection error of the matrix is suppressed.