Messaging App AR Selection via Visual Codebook Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face inefficiencies in messaging applications when trying to find and share images, as they must manually search through numerous options, which is time-consuming and discourages the use of image-based communication, and existing systems require users to manually specify objects in images for augmented reality, leading to resource waste.
Innovation Solution
A messaging application that automatically selects and presents augmented reality experiences by scanning captured images using visual codebooks to match image features, reducing the need for user input and streamlining the process of finding matching images and associated AR items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users manually search through numerous image options in messaging applications, then they can find specific images, but the process becomes time-consuming and discourages image-based communication
Solution Approach 1:
The system performs automatic image recognition and AR experience selection without requiring user input. The messaging application scans captured images, identifies objects within them, and automatically presents relevant augmented reality experiences, allowing the system to serve itself rather than requiring manual user search and specification
Solution Approach 2:
The system pre-processes captured images by automatically detecting objects and matching them with relevant AR experiences before user interaction is needed. This preliminary automatic analysis eliminates the need for users to manually search through images or specify objects, as the matching is already prepared and presented to them
2Adaptability or versatility
If users manually specify objects in captured images for augmented reality, then AR experiences can be selected, but resources are wasted due to unnecessary processing
Solution Approach 1:
The system automatically performs object detection and AR experience selection without requiring users to manually specify objects. This self-service approach eliminates wasteful re-processing that would occur if users had to repeatedly specify objects, as the system independently completes the full workflow from image capture to AR experience presentation
Solution Approach 2:
The system extracts only the essential information needed for AR matching by automatically identifying relevant objects in the image. Rather than requiring users to specify all possible objects or process the entire image manually, the system extracts key object features and uses them to select appropriate AR experiences, reducing unnecessary computational overhead
Data Source
AI summary
Aspects of the present disclosure involve a system and a method for performing operations comprising: capturing an image that depicts currency, receiving an identification of an augmented reality experience associated with the image, the server identifying the augmented reality experience by assigning visual words to features of the image and searching a visual search database to identify a plurality of marker images associated with one or more charities, and the server retrieving the augmented reality experience associated with a given one of the plurality of marker images; automatically, in response to capturing the image that depicts currency, displaying one or more graphical elements of the identified augmented reality experience that represent a charity of the one or more charities; and receiving a donation to the charity from a user of the client device in response to an interaction with the one or more graphical elements that represent the charity.