Camera-Based Virtual Input Buttons from Real Drawings for AR Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality systems face challenges in controlling virtual input devices, such as virtual keyboards, which are difficult to use and require separate physical input devices like Bluetooth joysticks, incurring additional costs and not fully leveraging AR environments.
Innovation Solution
An electronic device that utilizes real drawings on paper or objects as virtual input devices, using a camera module to detect outlines, deploy virtual buttons, and process user interactions based on button movements, eliminating the need for separate input devices and online downloads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a virtual keyboard is used in AR environment, then input functionality is provided, but control difficulty increases and user convenience deteriorates
Solution Approach 1:
The patent uses image recognition to detect drawings made on paper and creates virtual buttons that replicate the functionality of physical buttons. The system captures an image of the drawing, identifies button patterns, and generates corresponding virtual buttons in the AR environment, allowing users to interact with familiar physical-like interfaces without actual physical devices.
Solution Approach 2:
The patent replaces physical mechanical input devices (like Bluetooth joysticks or physical keyboards) with a camera-based image recognition system. Instead of requiring mechanical contact with physical buttons, the system uses the camera to detect drawn patterns and processes gestures or touches on the virtual buttons displayed in the AR environment.
2Adaptability or versatility
If a physical input device like Bluetooth joystick is connected, then input control is enabled, but additional cost is incurred and AR environment potential is underutilized
Solution Approach 1:
The patent makes the camera module serve multiple functions: it is used for both capturing the drawing patterns and for AR environment rendering. The same hardware components (camera, processor, display) that are essential for AR functionality are also used for input device recognition, eliminating the need for separate input hardware and fully leveraging the AR device's capabilities.
Solution Approach 2:
The AR device uses its own built-in camera and processing capabilities to recognize drawings and create virtual buttons, without requiring external physical input devices. The system serves itself by utilizing its inherent hardware resources for both display and input functions, eliminating additional hardware costs.
3Ease of manufacture
If conventional virtual input devices are used, then basic input is possible, but user customization and immediate creation are limited
Solution Approach 1:
The patent allows users to pre-draw button patterns on paper before entering the AR environment. The drawing is captured and processed to create ready-to-use virtual buttons, so the input device configuration is prepared in advance rather than requiring complex setup procedures during AR session initiation.
Solution Approach 2:
The system copies the drawn patterns from paper directly into the virtual environment, creating functional buttons that mirror the physical drawing. This allows immediate creation of customized input devices by simply drawing on paper, without requiring online downloads or complex configuration steps.
Data Source
AI summary
An example electronic device which, by at least one processor, may obtain an image corresponding to a real drawing by the camera module, detect an outline of at least one object included in the image, obtain at least one virtual button corresponding to a pattern of the outline of the at least one object, deploy the at least one virtual button in an area of the at least one object matching at least one real object included in the real drawing, and process a user interaction corresponding to a user input according to a movement of button control in an area of the at least one virtual button.