Automated Interest Extraction from Social Media Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face inefficiencies in navigating web and app pages due to the need for manual input of interests through multi-page forms and repetitive queries, which is time-consuming and resource-intensive, and existing technologies fail to accurately capture user interests from images.
Innovation Solution
The system uses social media APIs to obtain images, processes them through computer vision and NLP models to generate detailed text descriptions, and maps these descriptions to predefined interest categories, automatically rendering web or app pages that align with user interests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users manually fill out multi-page interest forms to indicate their interests, then the system can provide relevant web pages based on user interests, but this process is time-consuming and requires significant user effort
Solution Approach 1:
The system performs preliminary actions by automatically analyzing user-uploaded images (such as product photos) and extracting interest information before the user needs to search for items. This pre-processing of user interests from images eliminates the need for time-consuming multi-page forms while maintaining accurate interest capture
Solution Approach 2:
The patent replaces the mechanical manual form-filling process with automated computer vision and natural language processing systems. Image recognition models automatically analyze uploaded photos and generate interest categories, substituting user manual input with automated image-based analysis
2Ease of operation
If users navigate through several layers of user interface to find items of interest, then they can access detailed information, but this increases the complexity of user navigation
Solution Approach 1:
The system segments the navigation process by directly presenting relevant item listings on the main page based on image analysis, rather than requiring users to traverse multiple hierarchical UI layers. The interest information is extracted and used to filter and display relevant items immediately
Solution Approach 2:
The patent adds a new dimension to the user interface by integrating image upload functionality that directly influences product recommendations. Instead of traditional text-based search or category browsing, users can upload images and receive relevant product listings, creating a new interaction dimension
3Measurement precision
If the system processes multiple images through computer vision and NLP models to derive user interests, then accurate interest contexts can be obtained, but computing resource usage increases
Solution Approach 1:
The system extracts only the essential interest-relevant features from images using computer vision models, rather than processing entire images through all analysis stages. The NLP model then processes only the extracted text descriptions and captions, reducing overall computational load while maintaining interest derivation accuracy
Solution Approach 2:
Computer vision models perform preliminary analysis to generate text descriptions and captions of images before NLP processing. This preliminary extraction of relevant visual information filters out unnecessary data, reducing the computational burden on subsequent NLP and classification stages
Data Source
AI summary
A plurality of images are received from one or more social media platforms associated with a user. For a selected image of the plurality of images, a plurality of text descriptions are generated. The plurality of text descriptions are computer-generated captions that describe features of the selected image of the plurality of images. The plurality of text descriptions are processed through a natural language processing model. Based on processing, a plurality of interest contexts are derived from the plurality of text descriptions. A mapping of each of the plurality of interest contexts to one or more predefined categories associated with an online marketplace is generated. Based the mapping of each of the plurality of interest contexts to the one or more predefined categories, a user device associated with the user is caused to display an app page or web page associated with the one or more predefined categories.