Dynamic Web Flows Using AI Dummy Pages for Bot Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing bot detection systems in content delivery networks (CDNs) are vulnerable to low-volume bot attacks using residential IP addresses, which evade detection and impose a burden on users with captcha challenges, impacting user experience.
Innovation Solution
Implement a bot detection service in CDNs that generates dynamic, AI/ML-driven dummy web pages to create indistinguishable navigation loops, trapping bots in non-productive navigation and enhancing detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional bot detection systems are used, then bot detection capability is provided, but low-volume bot attacks using residential IP addresses can evade detection
Solution Approach 1:
The system pre-generates synthetic web page content and navigation workflows before actual bot attacks occur. These pre-prepared dummy pages and navigation paths are stored and ready to be deployed when bot traffic is detected, allowing the system to respond rapidly without generating content in real-time during an attack
Solution Approach 2:
The system creates synthetic copies of legitimate web page content, navigation structures, and user interaction patterns. These copied elements are used to construct dummy pages that mimic real site behavior, making it difficult for bots to distinguish between authentic and synthetic content. The copying includes replicating page layouts, text content, image structures, and navigation sequences
2Reliability
If dynamic dummy content is generated during interaction, then bot detection accuracy improves, but system complexity increases
Solution Approach 1:
The system pre-generates synthetic web page content and navigation workflows before actual bot attacks occur. These pre-prepared dummy pages and navigation paths are stored and ready to be deployed when bot traffic is detected, allowing the system to respond rapidly without generating content in real-time during an attack
Solution Approach 2:
The system introduces a specialized content generation service that acts as an intermediary between the bot detection service and the web server. This mediator receives detection results, selects appropriate synthetic content from pre-generated pools, and injects it into the response stream, thereby isolating the complexity of content synthesis from the core detection and serving logic
3Productivity
If ML-generated navigation loops are deployed, then bot attack cost increases and attacks slow down, but implementation complexity increases
Solution Approach 1:
The system dynamically adjusts navigation loop characteristics based on detected bot behavior patterns. When bots are identified, the system activates adaptive navigation paths that redirect traffic through synthetic content sequences. The dynamics include real-time switching between normal and looped navigation, adjusting loop depth and complexity based on bot persistence, and modifying content delivery timing to maximize disruption while maintaining system control
Solution Approach 2:
The system introduces a specialized content generation service that acts as an intermediary between the bot detection service and the web server. This mediator receives detection results, selects appropriate synthetic content from pre-generated pools, and injects it into the response stream, thereby isolating the complexity of content synthesis from the core detection and serving logic
Data Source
AI summary
An overlay network bot detection service is augmented to include a content generation service that dynamically generates dummy web pages that are served (along with real site content) to a requesting user, This content is built using machine learning models trained on a target website's content, or that otherwise leverage generative AI to create site content that mimics the site's real content. The generated content is preferably built dynamically during an actual interaction session with the requesting user, is designed to “look” and “feel” like actual content of the website, and inclusion of the content acts to trap a requesting user's browser in one or more non-productive (fake) navigation loops within the site. This facilitates the overall bot detection because such content and such loops are not actually part of the real site, and thus the navigation of these unproductive pages is highly indicative of bot activity.


