Image-Based Search Using Gesture Recognition for Intuitive Object Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional information search methods rely on user queries and lack intuitive and efficient ways to search for information based on displayed images, requiring more user-friendly and accurate methods to select and search relationships between objects.
Innovation Solution
An information search method and device that recognize inputs indicating selections and search relationships between objects in an image, allowing for intuitive searching by using touch-based gestures to gather, move, or display objects, and outputting relevant information based on these interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional information search methods based on user queries are used, then information can be searched, but the search process is not intuitive and requires complex user input
Solution Approach 1:
The patent replaces traditional text-based query input (mechanical typing process) with image-based visual search. Users capture or select images containing objects of interest, and the system automatically identifies objects and retrieves related information, eliminating the need for manual query formulation and making the search process more intuitive.
Solution Approach 2:
The patent uses image copying/capturing as the search input mechanism. Instead of requiring users to type queries, they capture images (either from camera or gallery) that contain the objects they want to search for. The system then processes these image copies to identify objects and retrieve information, simplifying the interaction to a simple capture-and-search workflow.
2Loss of information
If multiple objects are selected from an image for search, then more comprehensive information can be retrieved, but determining the search relationship between objects becomes complex
Solution Approach 1:
The patent introduces dynamic relationship determination based on user interaction. Instead of requiring users to pre-define search relationships, the system dynamically determines relationships (such as spatial relationships, hierarchical relationships, or contextual relationships) based on the selected objects' positions, sizes, and characteristics in the image, adapting to different search scenarios automatically.
Solution Approach 2:
The system performs self-service by automatically analyzing the selected objects and determining their search relationships without requiring explicit user input. The system uses image processing and object recognition techniques to identify objects, determine their spatial relationships, and automatically construct search queries that reflect the relationships between multiple objects.
3Ease of operation
If touch-based gestures are used to indicate search relationships, then user interaction becomes more intuitive, but the system must recognize and process multiple gesture types
Solution Approach 1:
The patent implements a universal gesture recognition system that handles multiple gesture types (such as single-tap, double-tap, long-press, drag, and swipe) through a unified framework. Each gesture type corresponds to a specific search relationship indication, but the system processes all gestures through the same object selection and relationship determination pipeline, reducing the need for separate processing logic for each gesture type.
Data Source
AI summary
An intuitive information search method and device based on a displayed image and a computer readable recording medium thereof. The information search method based on an image displayed on a device includes recognizing a first input indicating a selection related to a plurality of objects included in the image, recognizing a second input indicating a search relationship between the selected plurality of objects, and searching for information based on the first input and the second input; and outputting found information through the device.