3D Image Alignment via Hue Texture Edge Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for constructing a three-dimensional shape from two-dimensional images, such as those obtained by an endoscope, face challenges in accurately aligning and merging partial three-dimensional shapes due to lack of rotational degree of freedom information around the image pickup direction, leading to deviations and inaccuracies in the generated three-dimensional model.
Innovation Solution
An image processing apparatus that receives two-dimensional subject images and corresponding 5-degree-of-freedom information, including xyz coordinates and rotation angles, constructs three-dimensional image data and calculates deviation angles based on hue, texture, and edge similarity to correct positional deviations and align the shapes in three-dimensional space, minimizing rotation axis deviations and enhancing accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 6-degree-of-freedom information about endoscope is used to accurately generate two-dimensional image of inner wall, then alignment precision is improved, but device complexity and information requirements increase
Solution Approach 1:
The patent extracts only the necessary rotational degree of freedom information around the image pickup direction from the full 6-DOF endoscope data, discarding redundant information. This selective extraction maintains alignment precision while reducing the information burden and system complexity.
Solution Approach 2:
The patent applies different processing strategies to different types of information: it fully utilizes 3D position information and partial rotation information (around image pickup direction) while ignoring other rotation components. This localized information usage optimizes the balance between precision and complexity by applying quality control only where necessary.
2Device complexity
If rotational degree of freedom information around image pickup direction is not available, then device complexity is reduced, but alignment accuracy deteriorates
Solution Approach 1:
The patent enables the system to self-correct alignment deviations by calculating deviation angles from image content itself (hue, texture, edge similarity) rather than relying on external sensor information. This self-service approach compensates for missing rotational data while maintaining alignment accuracy.
Solution Approach 2:
The patent implements a feedback mechanism where deviation angles are calculated by comparing image features between sequential frames, and this feedback is used to correct the arrangement of three-dimensional shapes. The system continuously refines alignment based on image content feedback, achieving accuracy without requiring additional sensor information.
3Measurement precision
If image features are used to calculate deviation angle, then alignment accuracy is improved, but computational load increases
Solution Approach 1:
The patent extracts only the most salient image features (hue, texture, edge information) that are necessary for calculating deviation angles, discarding redundant pixel data. This selective feature extraction maintains alignment accuracy while significantly reducing computational requirements compared to using full image data.
Solution Approach 2:
The patent uses a partial approach by focusing computation only on specific image regions containing edges and features of interest, rather than processing the entire image. This partial action achieves sufficient alignment accuracy with reduced computational energy consumption.
Data Source
AI summary
An image processing apparatus includes a processor including hardware, wherein the processor is configured to: construct first and second pieces of three-dimensional image data based on two different subject images received; arrange first and second three-dimensional shapes included in the first and second pieces of three-dimensional image data at positions corresponding to pieces of image pickup position information; calculate a deviation angle based on at least one of a degree of similarity between hues, a degree of similarity between textures and a degree of similarity between edges, and correct the deviation angle and arrange the first and second three-dimensional shapes in three-dimensional space to generate a three-dimensional shape image.