Page Element Identifier Caching for DOM Processing Bottlenecks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

DOM algorithms are computationally intensive and lead to performance bottlenecks due to the need for repeated computations on similar page elements, often resulting in inefficient execution times and potential infinite loops during page crawling.

Innovation Solution

An algorithm that computes and stores an identifier (ID) for page elements upon first encounter, allowing subsequent encounters to reference the stored ID instead of recalculating, thereby reducing computational load and avoiding redundant computations by representing elements with IDs across the page hierarchy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If DOM algorithms compute identifiers for page elements on every encounter, then measurement precision is maintained, but execution time increases and productivity decreases

Engineering Contradiction:

Improveidentifier computation accuracyVSAvoidexecution time

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent applies preliminary action by computing and storing the identifier for a page element the first time it is encountered during crawling. This pre-computed identifier is then retrieved from storage during subsequent encounters, eliminating the need for repeated computation and significantly reducing execution time while maintaining identifier accuracy.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If DOM algorithms process large amounts of text directly, then measurement precision is maintained, but use of energy increases and productivity decreases

Engineering Contradiction:

Improvetext processing accuracyVSAvoidcomputational energy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies copying by creating and storing a simplified representation (identifier) of the page element based on its text content. Instead of processing the full text repeatedly, the system uses this copied identifier for subsequent comparisons and processing, dramatically reducing computational energy consumption while preserving the essential information needed for accurate page element identification.

Inventive Principle:

Principle #26Copying

3Measurement precision

If crawlers explore all pages without skipping, then measurement precision is improved, but loss of time increases due to infinite loops

Engineering Contradiction:

Improvepage coverage completenessVSAvoidcrawling time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies feedback by using the stored identifier of a page element to determine whether the current page has already been visited. When the same identifier is encountered again, the system recognizes this as a duplicate page and skips further processing, preventing infinite loops and reducing crawling time while maintaining complete coverage of unique pages.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS9959254B2Representation of an element in a page via an identifier

Publication Date: 2018.05.01 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Embodiments are directed to computing, by an apparatus comprising a processing device, an identifier (ID) for an element in a page using an algorithm responsive to encountering the element a first time, causing, by the apparatus, the ID to be stored in a storage device, encountering, by the apparatus, the element a second time, determining, by the apparatus, that the element is encountered the second time, and responsive to determining that the element is encountered the second time, representing, by the apparatus, the element by the ID at the time of the second encounter by retrieving the ID from the storage device.