Drawing Recognition Using Status-Based Template Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems limit user interaction with digital objects, relying on mouse and keyboard inputs for appearance and behavior, with scanned drawings offering limited opportunities for manipulation beyond resizing and color changes, lacking a natural method for richer graphical interaction.
Innovation Solution
A computer system that recognizes drawn inputs using a camera to identify and interpret drawings as virtual objects or controls, allowing users to interact with them as if they were physical peripherals, by matching the drawings to templates based on active programs or hardware status, enabling gestures and physics properties.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional mouse and keyboard inputs are used to determine digital object appearance and behavior, then system control and precision are maintained, but user interaction richness and naturalness are limited
Solution Approach 1:
The patent creates virtual copies of physical input devices (mouse, keyboard, game controller) by recognizing drawings of these devices. The system captures an image of a drawing, identifies it as a virtual input device, and creates a functional copy that can be interacted with on the display surface, allowing users to draw and interact with virtual peripherals rather than using physical ones
Solution Approach 2:
The display surface serves multiple functions: it acts as both the output display and the input surface. Users can draw on the display surface using a finger or stylus, and the system recognizes these drawings as virtual input devices. This multi-functional approach eliminates the need for separate physical input devices while maintaining interaction capabilities
2Adaptability or versatility
If scanned drawings are recognized as simple commands or alphanumeric characters, then recognition accuracy is improved, but interaction versatility is limited
Solution Approach 1:
The system dynamically adjusts the recognition process based on the active program. When a drawing is detected, the system queries the active program to determine what type of object or input device would be most useful in the current context. This dynamic adaptation allows the same drawing to be interpreted differently depending on the active application, increasing versatility without sacrificing accuracy
Solution Approach 2:
The system provides feedback by displaying the recognized virtual input device on the display surface and allowing users to interact with it. The active program's requirements guide the recognition process, and the system continuously monitors user interactions with the virtual device to ensure proper functionality
Data Source
Figure 1~2
Figure 3~4
Figure 5A
AI summary
Interactive drawing recognition is described using status determination. In one example, a computer system status is determined and then used to determine a type of drawing. A user drawing is observed and a library of drawing templates associated with the determined drawing type is accessed. The observed drawing is compared to the drawing templates to identify the observed drawing, and attributes are assigned to the identified drawing.