Intermediary Bot Detection via Telemetry Checkpoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing security techniques for client/server systems face challenges in distinguishing between legitimate web browsers and malicious bots, making it difficult to prevent unauthorized acts such as content scraping and DDoS attacks, as bots can mimic legitimate user behavior and spoof identities.
Innovation Solution
Implementing a system that uses telemetry data and checkpoint tokens to determine whether a browser is legitimate or a bot by injecting specific instructions into web pages, which generate and validate checkpoint records, allowing the server to differentiate between human-computer interaction and automated activity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If traditional security techniques are used to block bot requests, then malicious activity is reduced, but legitimate user access is also blocked due to inability to distinguish between bots and human users
Solution Approach 1:
An intermediary system is introduced between the web server and clients that acts as a mediator to analyze incoming requests. This intermediary evaluates multiple characteristics of each request (headers, behavior patterns, device information) and makes intelligent decisions about whether to allow or block the request, thereby protecting legitimate users while blocking bots without completely blocking access.
Solution Approach 2:
The system changes multiple parameters of incoming requests for evaluation, including HTTP headers, request timing patterns, user agent strings, and behavioral characteristics. By analyzing changes and variations across these parameters, the system can distinguish between automated bot requests and legitimate human user requests, resolving the contradiction between blocking bots and allowing legitimate access.
2Measurement precision
If advanced browser detection tests are implemented, then bot identification accuracy is improved, but system complexity and processing overhead increase
Solution Approach 1:
The detection system is segmented into multiple independent evaluation modules, each analyzing specific aspects of requests (header analysis, behavior pattern recognition, device fingerprinting). This segmentation allows the system to achieve high detection accuracy through comprehensive analysis while managing complexity by organizing functions into separate, manageable components that can be independently optimized.
Solution Approach 2:
The intermediary system performs multiple functions simultaneously: it analyzes request characteristics, maintains reputation databases, evaluates behavioral patterns, and makes blocking decisions. This multi-functionality consolidates what would otherwise require multiple separate systems into a single platform, reducing overall system complexity while maintaining high detection accuracy through comprehensive analysis.
3Reliability
If multiple countermeasures are injected into web pages, then bot mitigation is enhanced, but legitimate user experience deteriorates due to additional scripts and processing
Solution Approach 1:
The system applies countermeasures selectively rather than universally. It performs basic security evaluations on all requests but only injects additional countermeasures or challenging tasks for requests that show suspicious characteristics. This partial action approach enhances security by applying targeted mitigation while minimizing the impact on legitimate users who do not trigger additional countermeasures, thus preserving their experience.
Solution Approach 2:
The system employs challenges that allow legitimate users to self-verify their humanity through simple interactions (such as solving captchas or completing behavioral verification tasks). These self-service mechanisms enable legitimate users to prove their identity and bypass restrictions without requiring intensive intervention, thereby maintaining good user experience while enhancing security against bots.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Computer systems and methods in various embodiments are configured for improving the security and efficiency of server computers interacting with client computers that may be executing malicious software. In an embodiment, a computer system comprises a memory; one or more processors coupled to the memory; a processor logic coupled to the memory and the one or more processors, and configured to: intercept, from a server computer, one or more original instructions to be sent to a browser of a client computer; send the one or more original instructions to the browser and one or more telemetry instructions, which when executed, generate telemetry data indicating one or more objects that were referenced by the browser and to send the telemetry data to the intermediary computer; receive the telemetry data and determine whether the browser is legitimate or illegitimate based on the set of telemetry data.