Ordered Append-Only Log for Distributed Shared Memory

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current distributed computing systems face challenges in efficiently managing large-scale, computationally intensive virtual environments, such as virtual worlds and games, due to limitations in processing power, storage, and scalability, particularly in handling diverse data types and spatial dependencies, which restricts the ability to support millions of concurrent users with high performance and low latency.

Innovation Solution

The implementation of a distributed computing system that organizes computational units using object domains and processing partitions, allowing for the separation and processing of diverse data types across multiple hardware hosts, and utilizes a distributed shared memory system for low-latency data management and replication, enabling high-throughput and scalable processing of millions of data objects with spatial awareness.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If distributed computing systems use traditional storage methods to manage large-scale virtual environments, then they can support multiple customers and virtualized resources, but they face limitations in processing power, storage efficiency, and scalability when handling millions of concurrent users

Engineering Contradiction:
Improveprocessing throughputVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the distributed file system into multiple independent object stores, each handling specific data objects. This segmentation allows parallel processing across multiple storage nodes, increasing throughput while distributing system complexity across manageable units rather than a monolithic system

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a new dimensional organization of data using object domains and processing partitions, moving beyond traditional flat file system structures. This multi-dimensional organization enables efficient querying and processing by adding domain-based categorization and partitioning layers that scale independently

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If the system processes diverse data types with spatial dependencies in virtual environments, then it can support complex simulations and games, but it restricts the ability to support millions of concurrent users with high performance and low latency

Engineering Contradiction:
Improvedata type handling capabilityVSAvoidprocessing latency
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent applies local quality by organizing data objects according to their domain characteristics and spatial relationships. Each object domain can be optimized for its specific data type requirements, and spatial partitions enable localized processing that reduces latency by keeping related data together and minimizing cross-domain operations

Inventive Principle:
Principle #3Local quality

3Adaptability or versatility

If the system uses centralized computing resources to host virtual environments, then it can create electronic simulations and games, but it cannot efficiently scale to support millions of concurrent users

Engineering Contradiction:
Improvevirtual environment capabilityVSAvoidconcurrent user support
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent segments virtual environment data into distinct object domains that can be independently processed and stored. This allows the system to distribute virtual environment instances across multiple computing nodes, enabling linear scaling with concurrent user growth while maintaining full virtual environment functionality

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent creates a universal object store architecture that can handle diverse virtual environment data types through a common interface. The object domain framework provides multi-functional capabilities that work across different simulation and game types, enabling scalable support for various virtual environments without requiring separate specialized systems

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS11436217B2Ordered append-only log based data storage
Publication Date: 2022.09.06 AMAZON TECH INC
  • US11436217B2 patent drawing
  • US11436217B2 patent drawing
  • US11436217B2 patent drawing

AI summary

Technology is described for computationally intensive distributed computing systems and methods. A storage process for representing data objects in memory using a serialization format that provides low replication cost and thread-safe reads may include receiving a data object by a computing hub. A distributed shared memory of the computing hub may write the data object to a memory device associated with the computing hub using a byte array broken into a plurality of sections. The plurality of sections may describe the in-memory content of the data object and include information on how to read the plurality of sections to obtain a current state of the data object. The distributed shared memory may process requests from processing applications to modify data associated with the data object. The distributed shared memory may write mutations to the data object caused by the requests to the memory device using an ordered, log-based format to provide versioned snapshots of state.