Smartphone Composite Image Generation via 3D Point Cloud Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Smartphones struggle to capture high-quality images of large objects or scenes due to limitations in image resolution and quality, especially when objects are close, leading to blurred text and poor detail recognition.
Innovation Solution
A method for combining multiple image frames captured by a smartphone to form a composite image, using inertial sensors and image processing techniques to align and enhance image quality, allowing for improved resolution and detail by representing image features as a three-dimensional point cloud and adjusting positional data for consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple image frames are captured and combined to form a composite image, then image resolution and quality are improved, but device complexity and processing requirements increase
Solution Approach 1:
The patent divides the large object or scene into multiple smaller regions captured by individual image frames. Each frame captures a specific portion, and the processing system segments the overall processing task into frame-level operations (alignment, feature extraction, blending) that can be performed independently and then combined, making the complex task more manageable and efficient
Solution Approach 2:
The patent transitions from two-dimensional image processing to three-dimensional processing by creating a point cloud representation with depth information. This dimensional enhancement allows for more accurate spatial relationships and depth perception in the composite image, improving overall image quality and resolution through volumetric data integration
2Measurement precision
If inertial sensors are used to align image frames, then alignment precision is improved, but device complexity increases
Solution Approach 1:
The patent introduces inertial sensors as intermediary devices that measure physical motion (acceleration, rotation) and serve as a mediator between the camera's physical movement and the digital image alignment process. These sensors provide intermediate data that bridges the gap between physical camera motion and computational image registration, enabling more accurate alignment without requiring complex direct measurement systems
Solution Approach 2:
The patent replaces complex mechanical alignment systems with sensor-based measurement and computational processing. Instead of using mechanical stages or precision positioning mechanisms to physically align images, the system uses inertial sensors to measure motion and applies computational algorithms to digitally align the frames, substituting mechanical complexity with sensor and software solutions
3Productivity
If images are captured at high speed, then productivity is improved, but image quality deteriorates due to motion blur
Solution Approach 1:
The patent implements feedback mechanisms where inertial sensors continuously monitor camera motion during the capture sequence. This motion data is fed back to the processing system, which uses it to compensate for motion effects during alignment and to determine optimal capture timing. The feedback loop allows the system to maintain high capture speeds while correcting for motion-induced quality degradation through computational methods
Solution Approach 2:
The patent performs preliminary measurements of camera motion using inertial sensors before and during image capture. By anticipating and measuring motion in advance, the system can pre-calculate alignment transformations and prepare for optimal capture timing, allowing high-speed capture without sacrificing quality since the motion parameters are already known and compensated for in the processing pipeline
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the capture of high-quality composite images of large objects or scenes with improved resolution and detail, allowing for better recognition of text and reduced motion blur, while providing real-time feedback to users for optimal image acquisition.
Implementation Method 1
Motion sensors of the portable electronic device may include an accelerometer and/or a gyroscope. The outputs of the sensors may be processed to determine a position of the portable electronic device at a time when each image frame was captured.
Implementation Method 2
A stream of image frames representing images of a scene may be captured by a portable electronic device, such as a smartphone, as the device is moved into multiple orientations
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A smartphone may be freely moved in three dimensions as it captures a stream of images of an object. Multiple image frames may be captured in different orientations and distances from the object and combined into a composite image representing an image of the object. The image frames may be formed into the composite image based on representing features of each image frame as a set of points in a three dimensional point cloud. Inconsistencies between the image frames may be adjusted when projecting respective points in the point cloud into the composite image. Quality of the image frames may be improved by processing the image frames to correct errors. Further, operating conditions may be selected, automatically or based on instructions provided to a user, to reduce motion blur. Techniques, including relocalization such that, allow for user-selected regions of the composite image to be changed.