Distributed Token Bucket Rate Limiting for Low-Latency NF Ingress

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing token bucket algorithms are inefficient for ingress message rate limiting in distributed producer NF applications, leading to increased latency and resource utilization due to frequent access to a distributed cache for each message.

Innovation Solution

Implementing a local token bucket for each distributed producer NF application and a distributed token bucket for refilling, reducing the need for frequent access to the distributed cache by using local token buckets for message processing and refilling only when necessary.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a distributed cache is accessed for each ingress message in existing token bucket algorithms, then rate limiting can be performed across distributed producer NF applications, but resource utilization and latency increase due to frequent cache accesses

Engineering Contradiction:

Improverate limiting effectivenessVSAvoidmessage processing latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the centralized distributed cache into multiple local cache instances, each deployed with a distributed producer NF application. Each local cache maintains a portion of the token bucket state locally, eliminating the need to access the distributed cache for every message. This segmentation reduces latency while maintaining rate limiting effectiveness across the distributed system.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary action by pre-loading token bucket state into local caches before messages arrive. Each local cache is initialized with token bucket parameters and state, allowing immediate rate limiting decisions without requiring real-time access to the distributed cache. This preliminary preparation significantly reduces message processing latency.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If a distributed cache is accessed for each ingress message, then centralized rate limiting control is maintained, but resource utilization increases due to frequent cache operations

Engineering Contradiction:

Improvecentralized rate limiting controlVSAvoidresource utilization

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements local quality by allowing each distributed producer NF application to have its own local cache with locally-managed token bucket state. Each local cache operates independently with locally-available data, enabling fast rate limiting decisions without requiring centralized cache access. This local autonomy reduces resource utilization while maintaining effective rate limiting control.

Inventive Principle:

Principle #3Local quality

3Loss of time

If local token buckets are implemented for each distributed producer NF application, then cache access frequency is reduced and latency decreases, but system complexity increases

Engineering Contradiction:

Improvemessage processing latencyVSAvoidtoken bucket implementation complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent merges the local cache functionality with the existing distributed producer NF application deployment model. Each distributed producer NF application is co-located with its own local cache instance, combining what were previously separate components into a unified deployment unit. This merging simplifies the overall system architecture while enabling low-latency rate limiting.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent implements universality by making each local cache instance serve multiple functions: it acts as both a data cache and a token bucket rate limiting mechanism. This multi-functionality eliminates the need for separate rate limiting infrastructure, reducing system complexity while achieving the goal of reduced cache access frequency and latency.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12526235B2Methods, systems, and computer readable media for using optimized token bucket algorithm for ingress message rate limiting across distributed producer network function (NF) applications

Publication Date: 2026.01.13 ORACLE INT CORP

AI summary

A method for using an optimized token bucket algorithm for ingress message rate limiting across distributed producer network function (NF) applications includes implementing a producer NF instance as distributed producer NF applications and implementing distributed ingress gateways (IGWs) for performing ingress message rate limiting for the distributed producer NF applications. The method further includes maintaining, for each of the distributed IGWs, a local token bucket for rate limiting of ingress service-based interface (SBI) request messages received by each of the distributed IGWs and maintaining a distributed token bucket for refilling the local token buckets. The method further includes receiving ingress SBI request messages at the distributed IGWs and consuming, by the distributed IGWs, tokens from the local token buckets to allow processing of the ingress SBI request messages by the distributed producer NF applications and refilling the local token buckets with tokens from the distributed token bucket when numbers of tokens in the local token bucket fall below a threshold level.