Client-Side Boilerplate Detection for Web Query Recommendations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing query recommendation engines often generate unhelpful results when dealing with web resources that contain multiple content elements associated with different topics, as they fail to focus on the primary topic of interest due to the inclusion of boilerplate content, which is common across many web pages.
Innovation Solution
A client-side method that traverses the DOM tree of a web resource to identify and score nodes based on characteristics of boilerplate content, allowing for the filtering of such content before applying query recommendation techniques, thereby focusing recommendations on the primary topic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If query recommendation techniques are applied to all content elements of a resource, then the engine can process all available information, but the resulting recommendations become unfocused and less helpful due to inclusion of boilerplate content
Solution Approach 1:
The patent segments the resource's content elements into different categories (boilerplate content versus primary topic content) by analyzing DOM tree characteristics. It identifies and separates navigation elements, footer content, and other template-based elements from the main content, allowing the query recommendation engine to process only the relevant segments.
Solution Approach 2:
The patent extracts and removes boilerplate content from the resource before applying query recommendation techniques. By identifying content elements that match boilerplate patterns (such as navigation bars, footers, and template elements) and excluding them from processing, the system ensures that recommendations are generated solely from the primary topic content.
2Measurement precision
If the search engine filters boilerplate content before applying query recommendation techniques, then the recommendations become more focused on the primary topic, but the system complexity increases due to the need to identify and remove boilerplate content
Solution Approach 1:
The patent performs preliminary identification and filtering of boilerplate content before the query recommendation process begins. By pre-processing the resource to remove or mark boilerplate elements, the system simplifies the subsequent recommendation generation process rather than complicating it with real-time filtering logic.
Solution Approach 2:
The patent creates a simplified representation or copy of the resource's content structure that excludes boilerplate elements. This copied structure, which contains only the primary topic content, is then used for query recommendation processing, avoiding the need to repeatedly analyze and filter the full original structure.
3Measurement precision
If server-side processing is used to detect boilerplate content by comparing DOM trees of multiple web pages, then comprehensive boilerplate identification can be achieved, but the processing time and computational resources increase significantly
Solution Approach 1:
The patent enables the client device to autonomously identify and filter boilerplate content from the resource it receives. Instead of relying on server-side comparison of multiple web pages, the client device independently analyzes the DOM tree structure and content characteristics of the single resource, making the detection process faster and more efficient.
Solution Approach 2:
The patent applies a simplified, heuristic-based approach to boilerplate detection that processes only the necessary portions of the resource's DOM tree. Rather than performing exhaustive comparisons of entire DOM structures, the system focuses on key characteristics and patterns that indicate boilerplate content, achieving sufficient accuracy with reduced computational effort.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating query recommendations. One method provides selecting one or more nodes of an object model that models a resource as a hierarchy of nodes, and determining that the selected nodes exhibit one or more predefined traits that are characteristic of boilerplate content, wherein boilerplate content comprises content that is repeated in multiple resources of a particular web site or content which is not relevant to the main content of a resource. A score associated with the selected nodes is adjusted responsive to determining that the selected nodes exhibit the predefined traits, and information is provided to a query recommendation engine, the information including textual content associated with the selected nodes, and the information identifying the adjusted score associated with the selected nodes.