Website Auditing via Headless Browser Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing website analysis tools face difficulties in generating accurate site maps and determining website health due to issues with page loading, dynamic content, and incomplete reports on metatags and tracking pixels, leading to inaccurate navigation and optimization insights.
Innovation Solution
A system that uses web page scanning and rendering techniques to identify and execute objects on web pages, retrieve user input for fill-in forms, and combine scanned and rendered information to generate accurate site maps and assess website health and optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional page scanning mechanisms are used to generate site maps, then the generation process is simple and fast, but the accuracy and completeness of the site map deteriorates due to failed page loads, dynamic content, and forms requiring user input
Solution Approach 1:
The patent introduces a headless browser as an intermediary component that acts as a mediator between the page scanning mechanism and the dynamic web content. This headless browser renders pages completely, executes JavaScript, fills forms with pre-recorded user inputs, and extracts links from the rendered output, thereby resolving the contradiction by enabling accurate site map generation without requiring manual user interaction for each page.
Solution Approach 2:
The patent applies preliminary action by pre-recording user inputs for forms during a first pass through the website. These pre-recorded inputs are then reused during subsequent site map generation, allowing the automated system to complete forms and navigate through pages that require user input without manual intervention, thus maintaining both accuracy and automation.
2Reliability
If website health analysis only scans page source code for metatags and tracking pixels, then the analysis process is quick and simple, but the reliability of the health assessment deteriorates because it cannot verify if the code executes appropriately or captures dynamic content
Solution Approach 1:
The headless browser serves as an intermediary that executes the actual web page code and renders the dynamic content, allowing the system to verify whether metatags, tracking pixels, and JavaScript code execute correctly. This approach ensures reliable health assessment by observing actual execution behavior rather than merely scanning static source code.
Solution Approach 2:
The system creates a virtual copy of the user's browser environment through the headless browser, which replicates the actual browsing experience. This copy allows comprehensive analysis of code execution, dynamic content loading, and form behavior without requiring real user interaction, thereby maintaining reliability while reducing time loss compared to manual testing.
3Productivity
If automated page loading is used to generate site maps, then productivity increases, but the system stops progression and provides inaccurate maps when pages fail to load or require human input
Solution Approach 1:
The headless browser acts as an intermediary that handles page loading failures and forms requiring user input automatically. It renders pages that fail to load completely, executes JavaScript to reveal hidden content, and fills forms using pre-recorded user inputs, thereby maintaining automated productivity while ensuring complete and accurate site map generation.
Solution Approach 2:
The system performs preliminary action by pre-recording user inputs for forms during an initial exploration phase. These pre-recorded inputs enable the automated system to successfully navigate through forms during site map generation without manual intervention, preventing progression stops and ensuring complete coverage of all accessible pages.
Data Source
AI summary
An automated website analysis system includes mechanisms for automatically reviewing a website and identifying various features of the website. In one implementation, the analysis system can rely at least in part on the actual downloading and rendering of each page in the website to determine not only aspects of website content and layout, but also whether various content, such as third party objects or source code, execute as intended. Additional features include recording user input that may be required to progress past web pages where automated site mapping would ordinarily stall. The analysis system can identify various items of interest about each web page to the website owner, including web page parent/child relationships, as well as the extent to which the given page content is consistent with its metadata descriptions.