Dynamic Graph Embeddings for Low-Latency Transaction Streams

The patent segments the gradient computation process into two distinct phases: (1) forward pass that computes intermediate states and stores them in a compressed format using hashing, and (2) backward pass that retrieves only necessary intermediate states for gradient calculation. This segmentation allows efficient gradient computation while reducing memory requirements by not storing all intermediate states in full detail.

The patent introduces hashing structures as an intermediary mechanism between the forward and backward passes. Instead of directly storing and retrieving all intermediate states, the hashing structure acts as a mediator that compresses intermediate states during the forward pass and efficiently retrieves only the necessary states during the backward pass, thereby reducing memory complexity while maintaining computational efficiency.

The patent transitions from the time dimension to a hash-based indexing dimension for storing intermediate states. Instead of sequentially storing all intermediate states in time order (which consumes linear memory), the patent uses hashing to map intermediate states to compact indices, effectively changing the storage dimension from temporal sequence to hash space. This allows O(1) retrieval of any intermediate state regardless of its position in the sequence, enabling full-sequence backpropagation with constant memory overhead.

The patent changes the parameter representation of intermediate states by applying hashing functions. Rather than storing complete intermediate state representations which consume significant memory, the patent transforms these states into compact hash indices. This parameter transformation maintains the ability to uniquely identify and retrieve intermediate states while dramatically reducing the memory footprint, thus allowing capture of long-term dependencies without proportional memory increase.

The patent inverts the traditional approach by using reverse-mode AD (backpropagation) instead of forward-mode AD (RTRL). Conventionally, RTRL was chosen to reduce memory overhead, but it incurred O(l×d^4) computational cost. The patent demonstrates that by combining reverse-mode AD with hashed intermediate state storage, one can achieve both low memory overhead and acceptable computational demand, effectively inverting the trade-off relationship between memory and computation.

The patent creates a compressed copy of intermediate states through hashing during the forward pass. Instead of storing full intermediate state representations that would consume excessive memory, the patent generates compact hash copies that serve as proxies. These hash copies enable efficient retrieval and gradient computation during the backward pass without requiring proportional memory resources, thus reducing memory overhead while maintaining computational feasibility.