On-Device Image Search Index via Knowledge Graph
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current system-wide file searching on electronic devices is limited to file name, type, creation, and modification timestamps, and does not effectively incorporate metadata from various applications, such as scene analysis and user interaction data, for image files.
Innovation Solution
The technology expands search functionality by creating an on-device index that includes metadata from applications, performs optical character recognition on image files, and uses knowledge graphs to generate additional metadata, allowing for semantic searches across multiple applications and organizing results based on application donation and relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If system-wide file searching is limited to file name, type, and timestamp, then device complexity is reduced, but search functionality and relevance are insufficient
Solution Approach 1:
The search system is segmented into multiple independent components: a search module that manages queries, application modules that provide domain-specific metadata (e.g., scene analysis for images, contact info for messaging apps), and a knowledge graph that integrates all metadata sources. This segmentation allows each component to function independently while contributing to the overall search capability, resolving the contradiction between enhanced functionality and system complexity.
Solution Approach 2:
The search module serves multiple functions: it processes queries from users, collects metadata from various applications, queries the knowledge graph, ranks results, and presents findings. This multi-functionality consolidates what would otherwise be separate systems into a unified search interface, improving search capabilities without proportionally increasing user-visible complexity.
2Measurement precision
If metadata from multiple applications is incorporated into search, then search relevance is improved, but information processing complexity increases
Solution Approach 1:
The knowledge graph acts as an intermediary between applications and the search module. Instead of applications directly managing their own metadata and the search module directly querying all applications, the knowledge graph mediates by centralizing metadata storage and providing standardized query interfaces. This reduces the complexity burden on individual applications while maintaining comprehensive search relevance.
Solution Approach 2:
The system changes how metadata is structured and queried by transforming diverse application-specific data into a unified knowledge graph format. Metadata from different applications is normalized and represented through common concepts and relationships, allowing the search module to query using consistent parameters regardless of the source application, thereby reducing processing complexity while maintaining precision.
3Adaptability or versatility
If optical character recognition is performed on image files, then search capability is improved, but processing time and energy consumption increase
Solution Approach 1:
Optical character recognition is performed as a preliminary action when image metadata is first collected and added to the knowledge graph. By pre-processing images and extracting text information upfront, the system avoids the need for repeated OCR processing during search queries. This one-time processing cost is justified by the ongoing benefit of having text-searchable metadata available for all future searches, reducing cumulative energy consumption.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables comprehensive, relevance-ranked search results for image files across applications, improving user search experiences by incorporating metadata and user interaction data, and allowing for text recognition within images.
Implementation Method 1
The technology expands search functionality by creating an on-device index that includes metadata from applications, performs optical character recognition on image files, and uses knowledge graphs to generate additional metadata
Data Source
AI summary
A method is provided that includes receiving, from a first application, a first image file and a first set of metadata associated with the first image file, and updating an on-device index based on the first image file and the first set of metadata. The method may further include processing the first image file and the first set of metadata to generate a second set of metadata and updating the on-device index based on the second set of metadata. Upon receiving an image search query the process may identify a plurality of candidate image files from the on-device index based on the image search query, and providing the plurality of candidate image files for display in response to the image search query.