Sketch-Based Search System Using Neural Network Image Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional search mechanisms, such as keyword-based and image-based searches, are ineffective when users cannot describe products in words or capture images due to language proficiency issues or restrictions, limiting their ability to find desired content.
Innovation Solution
A method and system utilizing a neural network to convert hand-drawn sketches into images, extract features, and identify similar images stored in a database, allowing users to search for content based on sketches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional keyword-based search or image-based search is used, then search functionality is provided, but users cannot search when they cannot describe products in words or capture images
Solution Approach 1:
The patent introduces a sketch-to-image conversion model as an intermediary between the user's sketch input and the image search system. This intermediary automatically converts hand-drawn sketches into standard images that can be processed by traditional image search algorithms, eliminating the need for users to capture photos or write keywords while maintaining search functionality
Solution Approach 2:
The system creates a copy or representation of the user's sketch in the form of a standard image through automated conversion. This copied image representation can then be used in place of actual product photos for search purposes, allowing users to search without having the physical product or being able to photograph it
2Adaptability or versatility
If users are restricted from using cameras to capture product images, then traditional image search cannot be performed, but users still need to find the product
Solution Approach 1:
The sketch conversion model serves as an intermediary that transforms the limited information in a user's memory (represented as a simple sketch) into a complete product image representation, compensating for the loss of the actual product image that the user cannot capture
Solution Approach 2:
The system performs preliminary sketch-to-image conversion before the search process, creating a usable image representation from the user's rough sketch. This preliminary action prepares the data in a format that can be immediately used by existing image search infrastructure without requiring the user to have the actual product image
3Adaptability or versatility
If users lack language proficiency to describe products, then keyword-based search becomes ineffective, but users still need to search for products
Solution Approach 1:
The patent replaces the linguistic/motor skill requirement (writing or speaking product descriptions) with a visual motor skill (drawing sketches). This substitution allows users with limited language proficiency to communicate product characteristics through visual representation, which is then converted into searchable image data
Solution Approach 2:
The system creates a visual copy or representation of the product based on the user's sketch, bypassing the need for textual description. This copied visual representation captures the essential product characteristics and can be used for search without requiring the user to articulate product details in their native language
Data Source
AI summary
The present teaching relates to a method, system, and programming for searching content. A sketch of an object is obtained from a user. The sketch is processed by a neural network to generate an image corresponding to the sketch of the object. One or more features are extracted from the image, and one or more images previously stored are identified that have features similar to the one or more features. The one or more images are provided to the user.