3D AR Sketching on Mobile Devices via Camera-Based Snapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mid-air three-dimensional (3-D) sketching systems are primarily designed for virtual reality (VR) and are not practical for augmented reality (AR), as they require expensive hardware and are inaccessible for outdoor use, making it difficult to create large-scale or distant objects in a 3-D sketch.
Innovation Solution
A mobile device with a sketch generator component that integrates multiple sources of inputs, including touch screen data, camera data, and motion tracking, to enable users to create multi-dimensional AR sketches by generating surface proxies and snapping points, allowing for the integration of drawing content with real-world visual content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional mid-air 3-D sketching systems are used, then 3-D sketching capability is provided, but expensive specialized hardware and stereoscopic displays are required
Solution Approach 1:
The patent uses a mobile device camera to capture visual content of the real-world environment, creating a 2-D image copy that serves as the basis for 3-D sketching. This replaces the need for complex motion capture hardware by using the camera feed as the working surface for 3-D drawing operations.
Solution Approach 2:
The patent replaces mechanical motion capture systems with a combination of camera-based visual content capture and software-based 3-D coordinate transformation. The mobile device's camera and processing capabilities substitute for dedicated motion capture hardware and stereoscopic display systems.
2Adaptability or versatility
If VR-based sketching systems are used, then immersive 3-D environment is achieved, but the system is not suitable for outdoor environments
Solution Approach 1:
The patent makes the 3-D sketching system universal by using a mobile device that can function both as a camera for capturing real-world environments and as a processing unit for 3-D sketch generation. The system works with any visual content source (camera feed, images, videos) making it adaptable to both indoor and outdoor environments without requiring specialized VR hardware.
3Length of stationary object
If conventional 3-D sketching systems are used, then 3-D drawing capability is provided, but creating large-scale or distant objects is difficult
Solution Approach 1:
The patent transforms 2-D drawing operations on a mobile device screen into 3-D objects in augmented reality space. By capturing visual content of the real-world environment and mapping drawing operations onto the captured image coordinates, the system enables creation of large-scale 3-D objects that correspond to the actual physical environment, overcoming the limitations of conventional 3-D sketching systems.
4Ease of operation
If mobile device is used for 3-D sketching, then accessibility is improved, but limited field of view and coupling of 2-D and 3-D inputs are challenges
Solution Approach 1:
The patent uses the captured visual content image as an intermediary between the 2-D mobile device interface and the 3-D augmented reality output. The image coordinates serve as a mapping medium that translates 2-D touch input coordinates into corresponding 3-D spatial positions, resolving the complexity of coordinating 2-D and 3-D input systems.
Data Source
AI summary
An enhanced multimodal interface and various enhanced tools are provided to enable a user to create 3-D augmented reality (AR) sketches using a mobile device. Mobile device can comprise a sketch generator component (SGC) that can integrate multiple sources of inputs with enhanced tools, comprising enhanced 3-D snapping and proxy creation tools. The inputs comprise touch screen data, motion data, and image data from a camera. SGC can provide the interface, which can support absolute and relative drawing. Via the interface, the user can select and use desired enhanced tools to create a 3-D AR sketch that augments visual content of a real-world environment captured by the camera with drawing content added by the user using the enhanced tools. SGC enables a user to create desired strokes, planar or curved surface proxies, snapping points, or other sketch features, wherein proxies can align with real-world surfaces.