Sketch-Based Content Retrieval Using Neural Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in finding specific content on their devices due to the vast amount of data stored, making it cumbersome to locate images or other content without a convenient search method.
Innovation Solution
An electronic apparatus equipped with a neural network model that processes user sketch inputs to identify matching content information, using embedding vectors and similarity calculations to retrieve relevant content, such as images, audio, or documents, based on the sketch input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users manually search through large amounts of stored content data, then they can find specific content, but the search process becomes time-consuming and cumbersome
Solution Approach 1:
The system pre-processes content data by generating embedding vectors for images, audio, text, and documents, and stores them in a database with metadata. This preliminary organization enables rapid retrieval through similarity comparison without manual searching, resolving the contradiction between search accuracy and time consumption.
Solution Approach 2:
The patent replaces manual mechanical searching with an automated neural network-based similarity search system. The processor automatically compares query embedding vectors against stored content vectors using cosine similarity, eliminating the need for users to manually browse through large datasets.
2Adaptability or versatility
If the system stores diverse types of content information, then content versatility increases, but the complexity of managing and retrieving different content types increases
Solution Approach 1:
The system employs a universal embedding vector representation that can encode multiple content types (images, audio, text, documents) in a unified vector space. This allows the same neural network model and similarity comparison mechanism to handle diverse content types, reducing management complexity while maintaining versatility.
Solution Approach 2:
The patent transforms diverse content types into a common parameter representation (embedding vectors) through the neural network model. By converting different content formats into comparable vector parameters, the system simplifies the management and retrieval process while supporting multiple content types.
Data Source
AI summary
An electronic apparatus is provided. The electronic apparatus includes an input interface, a memory configured to store a plurality of first embedding vectors and content information related to each of the plurality of first embedding vectors, and at least one processor. The at least one processor is configured to obtain a sketch input of a user through the input interface, obtain a second embedding vector by inputting the sketch input to a neural network model, identify a first embedding vector, among the plurality of first embedding vectors, having a highest similarity with the second embedding vector, and obtain and output content information corresponding to the identified first embedding vector.