3D Structure Construction Using Sensor-Based Extrinsic Parameters

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The existing methods for constructing 3D structures on user devices are hindered by limited data processing capabilities, requiring image transmission to remote servers and time-consuming extraction of extrinsic camera parameters, which delays the 3D structure construction process.

Innovation Solution

A system and method that utilize a gyroscope, magnetometer, and accelerometer to determine extrinsic camera parameters and compute an internal calibration matrix, allowing for on-device processing of images to generate a 3D structure using triangulation techniques, thereby reducing the need for remote processing and enhancing real-time capability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Power

If images are transmitted to a remote server for processing, then the 3D structure construction can be performed with sufficient computing power, but the overall time required for constructing the 3D structure increases due to network transmission

Engineering Contradiction:

Improvedata processing capabilityVSAvoidoverall time for constructing 3D structure

Core Design Contradiction:

PowerVSLoss of time

Solution Approach 1:

The patent segments the 3D structure construction process into two parts: initial processing on the user device (image capture, feature extraction, extrinsic parameter determination using sensors) and subsequent refinement on the remote server. This segmentation allows time-sensitive operations to be performed locally, reducing network transmission time while maintaining processing quality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary actions on the user device including capturing images, determining extrinsic camera parameters using gyroscope, magnetometer, and accelerometer, and performing initial feature extraction. These preliminary steps prepare the data in advance, reducing the computational burden and time required when data is later processed on the remote server.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If extrinsic camera parameters are extracted from images through processing, then accurate parameters can be obtained, but the time consumption for parameter extraction increases the overall construction time

Engineering Contradiction:

Improveextrinsic camera parameter accuracyVSAvoidtime for extracting extrinsic parameters

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent introduces sensor data (gyroscope, magnetometer, accelerometer) as an intermediary source for determining extrinsic camera parameters. Instead of extracting parameters solely from images through complex processing, the system uses sensor readings as a mediator to directly obtain rotation and translation information, significantly reducing extraction time while maintaining accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical/image-processing-based parameter extraction system with a sensor-based measurement system. The gyroscope, magnetometer, and accelerometer provide direct physical measurements of camera orientation and position, substituting the time-consuming image analysis process with faster sensor-based determination.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enables faster construction of 3D structures on user devices by processing images locally, reducing transmission time and optimizing extrinsic camera parameters, thus providing a near-real-time 3D structure generation experience.

Implementation Method 1

determining, by a processor, extrinsic camera parameters using a gyroscope, a magnetometer, and an accelerometer

Methodology Applied

Scientific EffectGyroscope effect: Gyroscope

Implementation Method 2

determining, by a processor, extrinsic camera parameters using a gyroscope, a magnetometer, and an accelerometer

Methodology Applied

Scientific EffectMagnetic field detection: Magnetometer

Implementation Method 3

determining, by a processor, extrinsic camera parameters using a gyroscope, a magnetometer, and an accelerometer

Methodology Applied

Scientific EffectGravitational acceleration detection: Accelerometer

Data Source

PatentEP2960859B1Constructing a 3D structure

Publication Date: 2019.05.01 TATA CONSULTANCY SERVICES LTD

Figure 1
Figure 2
Figure 3A

AI summary

Disclosed is a method and system for constructing a 3D structure. The system of the present disclosure comprises an image capturing unit for capturing images of an object. The system comprises of a gyroscope, a magnetometer, and an accelerometer for determining extrinsic camera parameters, wherein the extrinsic camera parameters comprise a rotation and a translation of the images. Further the system determines an internal calibration matrix once. The system uses the extrinsic camera parameters and the internal calibration matrix for determining a fundamental matrix. The system extracts features of the images for establishing point correspondences between the images. Further, the point correspondences are filtered using the fundamental matrix for generating filtered point correspondences. The filtered point correspondences are triangulated for determining 3D points representing the 3D structure. Further, the 3D structure may be optimized for eliminating reprojection errors associated with the 3D structure.