Entity Database Sharding for Simulation Query Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database technologies fail to sustain necessary persistence, scalability, throughput, and consistency for large-scale simulations, particularly in games and simulations where updates across multiple clients require efficient tracking and execution of multidimensional query subscriptions, leading to increased latency and computational expense.
Innovation Solution
A database system comprising an entity data structure, query data structure, and query index data structure to efficiently determine affected entities and update them automatically, with additional index structures like entity position index for spatial queries, and a client library proxy for managing connections and demultiplexing responses to improve query efficiency and reduce unnecessary communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional database technologies are used to track and update entities in large-scale simulations, then persistence and query capabilities are maintained, but latency increases and throughput decreases due to quadratic growth in update operations
Solution Approach 1:
The patent segments the database into multiple shards distributed across different servers, allowing parallel processing of simulation updates. Each shard handles a subset of entities independently, reducing the quadratic complexity of global updates and enabling linear scaling with the number of shards.
Solution Approach 2:
The patent introduces a new dimensional organization of data using spatial indexes and entity-component-system architecture, transforming the traditional flat database structure into a multi-dimensional space-partitioned structure that enables efficient spatial queries and reduces update propagation overhead.
2Measurement precision
If spatial queries are executed frequently to determine affected entities, then query accuracy is maintained, but computational expense increases quadratically with simulation scale
Solution Approach 1:
The patent pre-computes and stores spatial indexes (such as R-trees or grid partitions) for all entities before queries are executed. This preliminary organization of spatial data allows queries to operate on pre-filtered subsets rather than scanning all entities, reducing computational expense from quadratic to logarithmic or linear time complexity.
Solution Approach 2:
The patent creates and maintains separate index structures that are copies or projections of the actual entity data, organized specifically for efficient spatial querying. These index copies allow fast spatial operations without directly querying the full entity dataset, significantly reducing computational expense while maintaining query accuracy.
3Productivity
If the database is divided into shards to improve scalability, then throughput increases, but device complexity increases due to distributed architecture
Solution Approach 1:
The patent introduces a client library proxy as an intermediary layer between application code and the distributed database shards. This proxy handles connection management, query routing, and result aggregation transparently, shielding users from the complexity of the distributed architecture while maintaining the throughput benefits of sharding.
Solution Approach 2:
The patent designs the sharded database architecture with universal interfaces and standardized data formats that work consistently across all shards. This multi-functional design allows the same query and update operations to work uniformly regardless of which shard handles them, reducing the complexity of managing distributed operations.
Data Source
AI summary
Database schemas, systems, and methods are described herein which enhance database operations when responding to streaming (subscribed) queries regarding records stored in the database. A database schema may include a data table storing a plurality of records each having a plurality of data fields. The schema may also include a query table, and a query index table or tree, or other data structure. The data table stores the current state of each record as defined by its associated data fields. The query table stores all queries mapped to the one or more clients that have subscribed to each query. The query index table, tree, or other data structure indexes the subscribed queries based on a data field common to or associated with the indexed queries. Subsequently, when records and data fields are updated (mutated) in the database, the query index data structure can be used to quickly assess which subscribed clients need to be sent updates with mutated data, so that only the clients who have subscribed to receive updates based on a particular data field do so and the cost of identifying those clients is greatly reduced.