Optical Pen and Finger Detection on Projection Surfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing projection type display devices and manipulation detection systems fail to accurately detect the position of electronic pens and fingers on a manipulation surface without installing touch sensors, and they do not consider user operations involving gripping objects.
Innovation Solution
A projection type image display device and method using a camera and two illuminating lights to detect electronic pen and finger contact points on a manipulation surface without touch sensors, employing image processing algorithms to differentiate and accurately determine the contact positions and usability of both inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If touch sensors are installed on the manipulation surface to detect contact positions, then detection precision is improved, but device complexity and manufacturing cost increase
Solution Approach 1:
The patent replaces the mechanical/electronic touch sensor system with an optical detection system. A camera captures images of the manipulation surface, and image processing algorithms analyze brightness variations and shadow patterns to detect contact positions of fingers and electronic pens. This substitution eliminates the need for physical sensors embedded in the surface while maintaining detection capability.
Solution Approach 2:
The patent introduces light as an intermediary medium to enable non-contact detection. Illuminating lights illuminate the manipulation surface, and the camera captures the reflected light patterns. The interaction between light, the manipulation surface, and the user's inputs (fingers/pen) creates detectable optical signals that reveal contact positions without direct electrical contact.
2Ease of operation
If only finger operation is supported, then ease of operation is improved, but adaptability deteriorates
Solution Approach 1:
The patent creates a universal manipulation detection system that can recognize multiple input methods (finger tapping, electronic pen writing, and combinations) using the same camera and image processing infrastructure. The system identifies different input types through characteristic pattern recognition in the captured images, allowing a single device to support diverse user interaction modes.
Solution Approach 2:
The patent changes the detection parameters and image processing algorithms adaptively based on the detected input type. When fingers are detected, the system applies one set of processing parameters for tapping gestures, while electronic pen contacts trigger different parameter settings for writing recognition. This dynamic parameter adjustment enables the system to optimize detection for each input method.
3Adaptability or versatility
If electronic pen operation is added to finger operation, then adaptability is improved, but difficulty of detecting and measuring increases
Solution Approach 1:
The patent segments the detection task into distinct recognition modules: one for detecting electronic pen contacts and another for detecting finger contacts. Each module uses specialized image processing techniques optimized for its specific target. The electronic pen detection module analyzes bright spots and line patterns, while the finger detection module analyzes shadow patterns and contact geometry, reducing the complexity of the overall detection system.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables correct detection of electronic pen and finger contact positions on the manipulation surface without installed touch sensors, improving user operability and allowing parallel detection and processing of pen and finger inputs.
Implementation Method 1
a camera (102) to capture an image of a manipulation surface
Implementation Method 2
an image processing unit (109) to process the captured image data and detect contact points between an electronic pen or a finger and the manipulation surface
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is a projection type image display device which correctly detects the position of contact of an electronic pen and a finger on a manipulation surface without the provision of a touch sensor or the like on the manipulation surface. Light containing an image is projected, illuminating light that is different from the light containing the image is projected onto a manipulation surface which at least partially overlaps an image projection surface onto which the light has been projected, and an image of the manipulation surface is captured. A first detection process is performed to detect the position at which a first object being manipulated, a distal end of which emits light, touches the manipulation surface, a second detection process is performed to detect the position at which a second object being manipulated, which does not emit light, touches the manipulation surface, and at least part of the image being displayed is changed on the basis of the detection result from either the first detection process or the second detection process.