Substitute Web Content Generation for Bot Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for detecting and preventing automated agents from accessing website content are inadequate, as CAPTCHAs can annoy users and are often bypassed by advanced technologies, necessitating a solution that distinguishes human interactions from automated activity without disrupting user experience.

Innovation Solution

A system that generates substitute web content, which appears identical to authentic content but is non-interactive to automated agents, using layers for detection and analysis of user interactions to determine whether to provide authentic or substitute content based on bot detection confidence.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If CAPTCHAs are implemented to detect automated agents, then automated agent detection capability is improved, but user experience deteriorates due to consumer annoyance and potential CAPTCHA bypassing

Engineering Contradiction:

Improveautomated agent detection capabilityVSAvoiduser experience

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The web content is segmented into multiple layers: a first layer containing non-interactive visual elements that appear identical to authentic content, and a second layer containing interactive elements. This segmentation allows the system to present different content structures to different requestors while maintaining visual consistency, thereby detecting automated agents without degrading human user experience.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different portions of the web content are assigned different interactive properties based on the requestor type. Human users perceive and interact with fully functional content, while automated agents encounter non-interactive visual elements. This local differentiation in content quality enables selective detection without affecting overall user experience.

Inventive Principle:

Principle #3Local quality

2Reliability

If substitute web content with non-interactive visual elements is provided to automated agents, then automated agent detection is improved, but content functionality is reduced for the agent

Engineering Contradiction:

Improveautomated agent detection accuracyVSAvoidcontent functionality for agent

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system creates a visual copy of the authentic web content that appears identical to human users but lacks interactive functionality for automated agents. This copy contains non-interactive visual elements that replicate the appearance and layout of the original content, enabling detection of automated agents while maintaining visual fidelity for human users.

Inventive Principle:

Principle #26Copying

3Reliability

If the system provides different content layers based on bot detection confidence, then automated agent prevention is improved, but system complexity increases

Engineering Contradiction:

Improveautomated agent prevention effectivenessVSAvoidsystem architecture complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

A bot detection service acts as an intermediary between the content service and the rendering service. This intermediary analyzes requestor characteristics, determines bot detection confidence levels, and directs traffic to appropriate content layers. This modular intermediary architecture manages system complexity by centralizing detection logic while maintaining independent content delivery components.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Ease of operation

If authentic web content with interactive elements is provided to all requestors, then user interaction capability is maintained, but automated agent access to sensitive content is enabled

Engineering Contradiction:

Improveuser interaction capabilityVSAvoidautomated agent access to sensitive content

Core Design Contradiction:

Ease of operationVSObject-affected harmful factors

Solution Approach 1:

The system dynamically adjusts the interactive properties of web content based on real-time bot detection confidence assessments. When a requestor is identified as potentially automated, the system dynamically switches from providing fully interactive authentic content to providing non-interactive substitute content. This dynamic adaptation prevents automated agents from accessing sensitive content while maintaining full interactivity for human users.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10565385B1Substitute web content generation for detection and avoidance of automated agent interaction

Publication Date: 2020.02.18 AMAZON TECH INC

AI summary

Online service providers may operate a rendering service for generating and providing substitute web content information for rendering substitute web content instead of authentic web content. The rendering service may obtain web content information for the authentic web content in response to receiving a request for web content. The rendering service may use the web content information to generate the substitute web content information. The substitute web content information is useable by the computing device to generate substitute web content that includes one or more visual elements resembling resource objects of the authentic web content. The visual elements are rendered, as a result of processing by the computing device, as image content instead of interactive objects.