URL Fetch Retry Switching Across Proxy Paths and Countries

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing communication over the Internet can be hindered by network issues such as congestion, load balancing, and unpredictable behavior, leading to lost, duplicated, or out-of-order data packets, which TCP protocols struggle to handle efficiently, resulting in connection failures and reduced reliability.

Innovation Solution

Implementing a retry mechanism for content fetching using direct or indirect fetching schemes, such as different proxy servers or IP addresses, to reestablish connections and ensure reliable data transfer.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If TCP protocol is used for reliable data delivery, then data integrity is improved, but connection failures occur due to network congestion and unpredictable behavior

Engineering Contradiction:
Improvedata delivery reliabilityVSAvoidconnection failures
Core Design Contradiction:
ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary actions by establishing multiple alternative connections and preparing backup routes before actual data transfer begins. When a connection failure occurs, the pre-established alternative connections can be immediately activated without waiting for failure detection, thus preventing connection failures from affecting data delivery reliability.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces intermediary components such as connection managers and failure detection mechanisms that mediate between the data transfer process and network conditions. These intermediaries monitor connection status, detect failures, and coordinate switch-over to alternative connections, thereby maintaining data delivery reliability despite network congestion and unpredictable behavior.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If retry mechanism is implemented for content fetching, then connection reliability is improved, but data transfer time increases

Engineering Contradiction:
Improveconnection reliabilityVSAvoiddata transfer time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The system pre-establishes multiple alternative connections and prepares retry configurations in advance. When a connection failure is detected, the retry mechanism can immediately utilize the pre-prepared alternative connections, eliminating the need to wait for connection establishment during retry attempts, thus minimizing time loss while maintaining reliability.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements periodic monitoring of connection status and scheduled retry attempts with optimized intervals. Instead of continuous retry attempts that would waste time, the system uses periodic failure detection and strategic retry scheduling, balancing reliability improvement with time efficiency by retrying only when necessary and using optimal retry intervals.

Inventive Principle:
Principle #19Periodic action

Data Source

PatentUS12524490B2System and method for URL fetching retry mechanism
Publication Date: 2026.01.13 BRIGHT DATA LTD
  • US12524490B2 patent drawing
  • US12524490B2 patent drawing
  • US12524490B2 patent drawing

AI summary

A method for overcoming intermittent, temporary, or other fetching failures by using multiple attempts for retrieving a content from a web server to a client device is disclosed. The URL fetching may use direct or non-direct fetching schemes, or a combination thereof. The non-direct fetching method may use intermediate devices, such as proxy server, Data-Center proxy server, tunnel devices, or any combination thereof. Upon sensing a failure of a fetching action, the action is repeated using the same or different parameters or attributes, such as by using different intermediate devices, selected based on different parameters or attributes, such as different countries. The repetitions are limited to a pre-defined maximum number or attempts. The fetching attempts may be performed by the client device, by an intermediate device in a non-direct fetching scheme, or a combination thereof. Various fetching schemes may be used sequentially until the content is retrieved.