Interactive Sketch Feedback for Transparent Visual Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual search methods lack user control and transparency, with users unable to effectively modify search results or understand how the algorithm perceives their input, leading to unsatisfactory outcomes.
Innovation Solution
A visual search method with a feedback loop that allows users to manipulate an interactive sketch representation of the search object, enabling them to directly modify and refine their queries through graphical, textual, or vocal inputs, ensuring transparency and user control over the search process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional visual search methods are used, then search results are provided automatically, but users have no control over the search process and cannot understand how results are generated
Solution Approach 1:
The patent implements a feedback loop where the system presents extracted object attributes and sketch representation to the user, allowing users to see how the algorithm perceives their input and modify the search based on this feedback. This resolves the transparency issue by making the black-box process visible and controllable.
Solution Approach 2:
The patent introduces an interactive sketch representation as an intermediary between the user's initial input and the final search results. This sketch serves as a mediator that translates user input into search queries while allowing users to control and modify the search process step-by-step.
2Productivity
If users receive search results without modification capability, then the search process is simple, but users cannot improve results when they are unsatisfactory
Solution Approach 1:
The patent makes the search process dynamic by allowing users to interact with and modify the sketch representation and extracted attributes in real-time. Users can add, remove, or modify object properties, and the search results update dynamically accordingly, enabling continuous improvement of search quality.
Solution Approach 2:
The patent segments the search process into multiple stages: initial object recognition, attribute extraction, sketch generation, and iterative refinement. This segmentation allows users to control and modify specific aspects of the search process without overwhelming complexity.
3Ease of operation
If the search algorithm operates as a black box, then processing is fast and automated, but users cannot understand or control how results are generated
Solution Approach 1:
The feedback loop presents intermediate results (extracted attributes and sketch) to the user during the search process, allowing users to understand and control the algorithm's perception while maintaining efficient processing through automated attribute extraction and sketch generation.
Data Source
AI summary
The disclosure relates to the fields of electronic processing of images and e-commerce, specifically—to the visual search with feedback loop based on interactive sketch for transparency and user control. Presented method adds transparency and user control to the traditionally “black-box” visual search process by introducing a feedback loop, based on interactive sketch. The method comprises of following main steps: submission of an information about the object being searched for by the user (in a form of image, written text, spoken text or selection of product category from menu in GUI), identification of product attributes and verification of identified attributes against the ontology rules, selection of building blocks from database for filtered attributes, connection of selected blocks to form a sketch of object being searched for, application of projection rules to produce a form vector, comparison of vector from previous step with database, presenting user with interactive sketch of product alongside the list of matching products from database, enabling user to select a part of the object in sketch, presenting user with possible modifications for the selected part of the object, enabling user to select needed modification and re-iterate the search process as many times as needed, until the wanted results are achieved.