Primary Product Object Identification on Webpages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search algorithms on the Internet fail to accurately and efficiently extract product-related information from webpages, leading to irrelevant search results due to the lack of consideration for attributes like product images, titles, and prices, which are crucial for online shopping.
Innovation Solution
A method that identifies primary product objects on webpages by dividing them into sections, extracting features such as geometric and alternative text features, and computing probabilities to determine the likelihood of candidate objects being primary product objects, allowing for accurate and efficient indexing and retrieval of product information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional keyword-based search algorithms are used to index all keywords on webpages, then the search coverage is comprehensive, but the search accuracy and relevance are poor because product-related information is not differentiated from other content
Solution Approach 1:
The webpage is divided into multiple sections based on HTML tags and structural elements. The system identifies and segments product-related sections (such as product descriptions, images, prices) from other content areas, allowing focused extraction and indexing of only relevant product information while maintaining comprehensive coverage across all webpages
Solution Approach 2:
A specialized information extraction module acts as an intermediary between the crawler and the search index. This module uses wrapper induction and pattern matching to automatically identify and extract product-related information (images, titles, prices, descriptions) from diverse webpage formats, converting unstructured HTML into structured product data before indexing
2Measurement precision
If manual or semi-automatic information extraction techniques are used to extract product-related information, then the extraction accuracy improves, but the processing time and resource consumption increase significantly due to millions of webpages with different formats
Solution Approach 1:
The system performs preliminary classification of webpages during the crawling phase, identifying pages that contain product information based on URL patterns, HTML structure, and content keywords. This pre-filtering reduces the volume of pages requiring detailed extraction, allowing accurate processing of only relevant pages while maintaining high overall processing throughput
Solution Approach 2:
The extraction module dynamically adjusts extraction parameters and patterns based on the identified webpage type and format. For different e-commerce platforms and product categories, the system modifies extraction rules, priority attributes, and validation criteria, enabling accurate extraction across diverse formats without requiring manual reconfiguration for each page type
3Loss of information
If all product attributes and information are extracted and indexed from every webpage, then the search completeness is maximized, but the resource utilization and storage requirements become inefficient
Solution Approach 1:
The system extracts only essential product attributes (product images, titles, prices, descriptions, and key specifications) while discarding redundant information such as navigation menus, advertisements, and repetitive content. This selective extraction maintains search completeness for critical product information while significantly reducing storage requirements and indexing computational load
Solution Approach 2:
The system performs extraction at a partial level by focusing on the most important product attributes that determine search relevance. Rather than extracting every piece of information on a webpage, it prioritizes extraction of attributes with highest search value, achieving sufficient completeness for effective product search while minimizing resource consumption
Data Source
AI summary
A method for identifying primary product objects on webpages over the Internet. A primary product object displays the best view of the product that a webpage is detailing. Each webpage is divided into sections based on the primary product objects in the webpage. Features of candidate product objects in each section are extracted. The primary product objects are identified by computing probabilities of the candidate product objects in each section being primary product objects, based on a statistical model. The identified primary product objects are stored for subsequent retrieval and display.