Transmission Log Bucketing with Shard Keys for Low-Latency Retrieval

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional distributed database management systems face performance issues due to suboptimal partitioning, leading to high latency, load shedding, and dropped messages, as they lack technology to optimize partition size, which affects user experience.

Innovation Solution

A transmission log bucketing tool that formats partitions, generates a manifest with a transmission log table, and uses shard keys to store and retrieve logs efficiently, ensuring each partition does not exceed a threshold to maintain optimal performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Volume of stationary object

If larger partitions are used in distributed database systems, then storage capacity is improved, but latency increases and system performance deteriorates

Engineering Contradiction:
Improvestorage capacityVSAvoidlatency
Core Design Contradiction:
Volume of stationary objectVSLoss of time

Solution Approach 1:

The patent applies segmentation by dividing the distributed database into multiple smaller partitions instead of using large partitions. This allows the system to maintain storage capacity while reducing the size of individual partitions, thereby decreasing latency and improving query performance. The segmentation principle directly addresses the contradiction by breaking down the storage structure into manageable units that can be processed more efficiently.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a new dimension to partition management by implementing dynamic partitioning strategies that consider multiple factors such as data access patterns, query types, and system load. This multi-dimensional approach allows the system to optimize both storage capacity and latency simultaneously by making intelligent decisions about partition sizing and distribution based on various operational parameters.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Device complexity

If conventional partitioning techniques are used, then system simplicity is maintained, but read query performance deteriorates due to suboptimal partitions

Engineering Contradiction:
Improvepartitioning complexityVSAvoidread query performance
Core Design Contradiction:
Device complexityVSProductivity

Solution Approach 1:

The patent implements dynamic partitioning techniques that automatically adjust partition sizes and distributions based on real-time system conditions and query patterns. This dynamic approach improves read query performance by optimizing partition structures according to actual usage, while the automation reduces the operational complexity that would otherwise be required to manually manage optimal partitions.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The system incorporates feedback mechanisms that monitor query performance and system metrics, then use this information to automatically adjust partitioning strategies. This feedback loop enables the system to continuously optimize read query performance without requiring complex manual intervention, as the system self-adjusts based on observed performance patterns and operational data.

Inventive Principle:
Principle #23Feedback

3Productivity

If large partitions are used, then storage efficiency is improved, but system reliability deteriorates due to load shedding and dropped messages

Engineering Contradiction:
Improvestorage efficiencyVSAvoidmessage delivery reliability
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The patent applies segmentation by creating multiple smaller partitions instead of relying on large partitions for storage efficiency. This segmentation ensures that if one partition experiences issues, other partitions can continue operating, thereby maintaining message delivery reliability while still achieving efficient storage through distributed data placement across multiple smaller units.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements parameter changes by dynamically adjusting partition size thresholds and distribution parameters based on system conditions. This allows the system to maintain storage efficiency by optimizing how data is distributed across partitions, while simultaneously improving reliability by ensuring that no single partition becomes a bottleneck or failure point that would cause load shedding or message drops.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS12536200B2System for optimized transmission log storage and retrieval
Publication Date: 2026.01.27 JPMORGAN CHASE BANK NA
  • US12536200B2 patent drawing
  • US12536200B2 patent drawing
  • US12536200B2 patent drawing

AI summary

A system for implementing a transmission log bucketing tool that provides an optimized transmission log storage and retrieval scheme. The system may comprise a processor and memory storing instructions that cause the processor to perform operations. The operations may comprise generating a manifest that comprises a transmission log table, obtaining a first set of transmission logs, evaluating the first set of transmission logs against the manifest to determine a first set of respectively corresponding shard keys, and utilizing the first set of respectively corresponding shard keys to at least one from among store the first set of transmission logs and retrieve the first set of transmission logs.