Low-Latency, Distributed Applications for the Interactive World

Abstract

This technology provides computationally intensive distributed computing applications and systems in a service provider environment, which can be used to provide virtual worlds, simulations, virtual environments, games, and other multi-dimensional (e.g., 2D and 3D) virtual environments that can be viewed by users and other computing services, interact with users and other computing services, and be accessed by users and other computing services. The distributed computing applications can be large-scale, low-latency distributed applications used with simulations or persistent interactive virtual worlds capable of hosting millions of concurrent users. Through partitioning, this technology can manage computing resources such that applications or application instances can effectively utilize the CPU cores (central processing unit cores) of large hardware instances, and enable developers to effectively use fewer hardware instances.

Description

Background Art

[0001] Virtualization technologies for computing resources have provided many customers with different requirements with the benefits of managing large-scale computing resources and have allowed various different computing resources or computing services to be effectively and securely shared by multiple customers. For example, virtualization technologies can allow a single physical computing machine to be shared among multiple customers by using a hypervisor to provide each customer with one or more virtualized computing resources (e.g., computing instances and software containers) hosted by the single physical computing machine. Additionally, users or customers can access a large number of dynamic and virtualized computing resources without having to manage the computer hardware on which those resources execute. Virtualized computing resources can be configured to obtain various different additional resources or services via an API, where the API provides access to the resources and services.

[0002] Centralized computing resources can be used to create electronic 2D (two-dimensional) virtual environments, 3D (three-dimensional) virtual environments, or multi-dimensional virtual environments such as electronic simulations, virtual worlds, and electronic games, and a large number of centralized computing resources can be used to host such virtual environments. These virtual environments can be accessed by multiple users via a computer network or the Internet. For example, virtual worlds, simulations, or electronic games can be accessed by multiple users via the Internet. Examples of such virtual worlds can be physical simulators, medical simulations, driving simulations, open adventure worlds, first-person shooter games, sports games, strategy games, or massively multiplayer online (MMO) games, etc. Such virtualized worlds can be hosted using a group of virtualized resources in a service provider environment, the group of virtualized resources including distributed applications, virtualized containers, computing instances (i.e., virtual machines), virtualized data storage, virtualized networks, virtualized services, and other virtualized computing resources executing on underlying hardware devices and underlying computer networks.

[0003] Virtualization technologies for computing resources have provided many customers with different requirements with the benefits of managing large-scale computing resources and have allowed different computing resources or computing services to be effectively and securely shared by multiple customers. For example, virtualization technologies can allow a single physical computing machine to be shared among multiple customers by using a hypervisor to provide each customer with one or more virtualized computing resources (e.g., computing instances and software containers) hosted by the single physical computing machine. Additionally, users or customers can access a large number of dynamic and virtualized computing resources without having to manage the computer hardware on which those resources execute. Virtualized computing resources can be configured to obtain various different additional resources or services via an API, where the API provides access to the resources and services.

[0004] Centralized computing resources can be used to create electronic 2D (two-dimensional) virtual environments, 3D (three-dimensional) virtual environments, or multi-dimensional virtual environments such as electronic simulations, virtual worlds, and electronic games, and a large amount of centralized computing resources can be used to host such virtual environments. These virtual environments can be accessed by multiple users through a computer network or the Internet. For example, a virtual world, simulation, or electronic game can be accessed by multiple users through the Internet. Examples of such virtual worlds can be physical simulators, medical simulations, driving simulations, open adventure worlds, first-person shooter games, sports games, strategy games, or massively multiplayer online (MMO) games, etc. Such virtualized worlds can be hosted using a group of virtualized resources in a service provider environment, the group of virtualized resources including distributed applications, virtualized containers, computing instances (i.e., virtual machines), virtualized data storage, virtualized networks, virtualized services, and other virtualized computing resources executing on underlying hardware devices and underlying computer networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A system is shown that can include a service provider environment hosting a compute-intensive distributed computing system for providing a user-accessible virtual environment according to an example of the present technology.

[0006] Figure 2 Different example components are shown that are included in a service provider environment hosting a compute-intensive distributed computing system for providing a user-accessible virtual environment according to an example of the present technology.

[0007] Figure 3 A block diagram is shown that illustrates an example computing service of a compute-intensive distributed computing system using a distributed computing service manager according to an example of the present technology.

[0008] Figure 4A A system and related operations are shown for assigning computing units in a distributed computing system using processing partitions organized by an object domain according to an example of the present technology.

[0009] Figure 4B A graphical example is shown of creating processing partitions using an object domain to assign to hardware hosts in a distributed computing system according to an example of the present technology.

[0010] Figure 4C A graphical example is shown of creating processing partitions using an object domain and a spatial location to assign to hardware hosts in a distributed computing system according to an example of the present technology.

[0011] Figure 5 Various example components are shown that are included in a service provider environment for managing hardware hosts in a distributed computing system using processing partitions organized by an object domain according to an example of the present technology.

[0012] Figures 6A - 6B is a flowchart showing an example method for managing hardware hosts in a distributed computing system using processing partitions organized by an object domain.

[0013] Figure 7 is a flowchart showing an example method for processing data objects assigned to a hardware host using processing partitions organized by an object domain.

[0014] Figure 8 shows a hardware host and related operations for storing and replicating the state of data objects in a distributed computing system according to an example of the present technology.

[0015] Figure 9 shows a graphical schematic example of a serialization format for data objects in a distributed computing system for providing low replication cost and thread-safe reads according to an example of the present technology.

[0016] Figure 10 is a flowchart showing an example method for managing the storage of data objects in a distributed computing system using a serialization format that provides low replication cost and thread-safe reads according to an example of the present technology.

[0017] Figure 11 is a flowchart showing an example method for using an ordered, append-only log-based format to capture changes to data objects in a distributed computing system to provide a versioned snapshot of the state.

[0018] Figure 12 is a flowchart showing an example method for using an ordered, append-only log-based format to exchange data objects between hardware hosts in a distributed computing system to provide a versioned snapshot of the state.

[0019] Figure 13 shows a system and related operations for assigning computing units in a distributed computing system using processing partitions with data objects organized by spatial location according to an example of the present technology.

[0020] Figure 14 shows a graphical example of creating processing partitions using spatial location information associated with data objects and assigning the processing partitions to hardware hosts in a distributed computing system according to an example of the present technology.

[0021] Figures 15A - 15B is a flowchart showing an example method for managing hardware hosts in a distributed computing system using processing partitions organized by spatial location information associated with data objects according to an example of the present technology.

[0022] Figure 16 is a flowchart showing an example method for processing data objects assigned to a hardware host using processing partitions organized by spatial location information associated with the data objects according to an example of the present technology.

[0023] Figure 17 is a flowchart showing an example method for splitting processing partitions organized by spatial location information according to an example of the present technology.

[0024] Figure 18 is a flowchart showing an example method for merging processing partitions organized by spatial location information according to an example of the present technology.

[0025] Figure 19 shows an example of physical computer hardware on which the present technology can be executed. Detailed Description

[0026] The technology provides compute-intensive distributed computing applications and systems in a service provider environment, which can be used to provide virtual worlds, simulations, virtual environments, games, and other multi-dimensional (e.g., 2D and 3D) virtual environments that can be viewed by users and other computing services, interact with users and other computing services, and be accessed by users and other computing services. Examples of such distributed virtual environments can be physical simulations, first-person games, medical simulations, TV or movie sets, driving simulators, aircraft simulators, CAD (Computer-Aided Design) or CAM (Computer-Aided Manufacturing) applications, space simulators, and other virtual worlds. The distributed computing applications can be large-scale, low-latency distributed applications used with simulations or persistent interactive virtual worlds capable of hosting millions of concurrent users.

[0027] The technology can manage computing resources in the service provider environment and can scale elastically to support thousands of applications (i.e., for processing various different object types). The technology can load balance application instances across hardware in response to virtual world load, so that developers can spread the cost of large worlds and simulations across the user base. Through partitioning, the technology can manage computing resources such that applications or application instances can effectively utilize the CPU cores (Central Processing Unit cores) of large hardware instances, and enable developers to effectively use fewer hardware instances. Additionally, this technology also enables developers to use a larger number of hardware hosts to run complex worlds, even worlds with millions of users connected to a single world or simulation. The technology can automatically scale (e.g., scale down when not in use or scale up when load increases) computing resources based on usage and charge only for computing time when there is world, simulation, or game activity.

[0028] The technology may include a state structure for high-throughput computing and data services, which may support gigabytes per second of data writes of highly partitioned data from applications or application instances collocated with the data. Examples of applications executed using this technology may include destruction applications, which may simulate a large number of destructions in a game world while manipulating tens of thousands of physical entities; AI (Artificial Intelligence) applications, which enable thousands of competing AIs in a game; or "navigation mesh" generation applications, which enable developers to recalculate the navigation mesh of AI vehicles or characters on a multi-kilometer world changing at 30hz. Custom applications may also access simulated data, data objects, entities, and actors from the state structure by subscribing to filters for receiving changed data objects and world assets from a hardware host (e.g., for a region of the world or for an object type).

[0029] A distributed computing system in a service provider environment may provide high-throughput, low-latency computing and data services (or state structure) to support the placement and retrieval of millions of highly partitioned data per second using one or more hardware hosts. The hardware hosts may include computers, servers, physical hosts, or virtual machine instances that execute containerized applications or application instances and manage the memory shared across the containerized applications or application instances. A distributed shared memory (DSM) management system may facilitate client and application interaction with highly partitioned data and enable data distribution across many hardware hosts. DSM enables highly partitioned data to be read with low latency and shared via subscription-based streams. Applications or clients create data object event streams that request updates to data objects sent to the DSM, and these changes are replicated to peer hardware hosts, allowing other applications and clients to see the updates. DSM provides fault tolerance by separating code from data state. Thus, starting and stopping applications does not affect the data stored in the DSM.

[0030] DSM may also enable highly partitioned data to be backed up to long-term storage in a service provider environment. DSM may also replicate highly partitioned data to a large number (thousands, millions, or hundreds of millions) of clients. Region-of-interest management (e.g., spatial fields or object types) and bandwidth prioritization systems may be used to prioritize data updates to clients to maintain client view consistency of large-scale object worlds or simulations over consumer-grade Internet connections. Figure One consistency.

[0031] The DSM can also enable customer code (e.g., containerized applications or application instances) to modify highly partitioned data as close as possible to the data on the serverless fabric to achieve minimum latency and maximum throughput. Applications and clients read data from the DSM by executing a one-time query that subscribes them to updates to data objects matching the query filter. The filter can be a spatial construct (e.g., an axis-aligned bounding box), a label or list of labels (e.g., "robot"), or any other query that can be evaluated on a field of the data object (e.g., "enemy player with low hit point"). A manager service on the hardware host can manage the lifecycle of the applications and the DSM on the hardware host and can expose endpoints that allow DSMs on other hardware hosts to be linked together.

[0032] In an example use of the technology for games, developers building a gaming experience in a service provider environment may have been limited by the computing and storage of a single server process on a single hardware instance. For example, a game console paired with a single server process can manipulate thousands of physical objects per frame or a few dozen AIs (artificial intelligence) intelligent enough to challenge several players. Developers have been able to deploy a single server process to a computing instance on a hardware host or droplet using existing tools, but may have difficulty leveraging additional computing resources to support more users, physical objects, and AIs in the case of building complex clustering technologies that enable developers to distribute their virtual worlds, games, or simulations across servers. Building technologies that distribute computing workloads across servers and are highly stateful, low-latency, and fault-tolerant is difficult, risky, and takes years of effort for large and experienced teams.

[0033] Using this technology, developers can immediately start building virtual worlds, simulations, and games that leverage increased simulation density (e.g., 10X–10000X) compared to single-server games. This can result in manipulating tens of thousands of physical objects per frame or featuring thousands of competitive AIs. Providing high-performance runtimes, distributed game applications, and tools can help game developers build games that exceed the limitations of the single-server model in a service provider environment.

[0034] In one configuration of the present technology, a distributed computing system can use object domains to organize computing units across hardware hosts. An object domain can define processing partitions (i.e., subdivisions of data objects) using the object types of data objects in an object-based model of a virtual environment. Distributing the load for processing data objects in a distributed computing system using object types enables the handling and rendering of large virtual worlds. Data objects can be geometric objects (e.g., polygons, spheres, cylinders, etc.), player models, opponent models, buildings, vehicles, plants, animals, or any other object that can be modeled or animated in 2D or 3D. An object domain can describe the object types of data objects that can be assigned to a processing group or processing partition. An object type can be data associated with a data object that conceptually describes the data object. For example, a data object can be one of the following object types: person, animal, plant, building, vehicle, particle, weapon, or another object type. An object domain can use one object type or multiple object types to subdivide data objects. An object domain can be associated with one processing partition or multiple processing partitions. A mapping can be created between the object domain and a processing application, where the processing application processes the data objects subdivided within the object domain. Object types can be used to group data objects of the object type together, and the grouping can be regarded as a processing partition computing unit.

[0035] A processing application can be used to process data objects assigned to a processing partition by object type. Thus, an object domain can be used to define which applications and / or application instances can be used to perform the computations of the object domain and which applications and / or application instances can have exclusive change permissions (e.g., write permissions) for data objects of the associated object type. For example, data objects with a physical data type can be processed by a physics application. A physics object domain can define which object types belong to the physics object domain, define which physics processing applications have permissions for the data objects, and define the data correlations with other object types used to process object types belonging to the physics object domain.

[0036] As described, an object type can be owned by an object domain that has a processing application that represents at least one computational process to be applied to the object type owned by the object domain. Thus, an object domain can provide an indexing process to organize object data into discrete groups of computing units. An object domain can define which applications have permissions to write to a particular object type (e.g., exclusive change permissions). An object domain can define which applications have permissions to use additional object types during processing (e.g., read-only permissions). This definition of the ownership of data objects enables the implementation of a single-writer-multi-reader model for applications and / or application instances. This definition of the ownership of data objects also enables the object domain to read the replication signaling of additional object types, allowing the distributed computing system to calculate at which hardware hosts and with what priority to make what data available.

[0037] Thus, data objects in a virtual environment can be organized into processing partitions by a distributed computing system according to their object types. The distributed computing system can use the processing partitions to assign data objects to application instances of a processing application on a hardware host. For example, multiple "particle objects" can be assigned to multiple processing partitions. Then, one or more of the processing partitions can be assigned to a particle simulation application instance on a first hardware host for processing. Other processing partitions can be assigned to a particle simulation application instance on a second hardware host for processing.

[0038] In high-performance computing systems (e.g., Hadoop clusters), the distribution of data to be processed in parallel has previously been accomplished by dividing unified data into multiple parts and sending the data to separate nodes in a computing cluster. Since the data in these cases is unified, a unified type of processing is applied to the data. In contrast, partitioning data for parallel processing can be problematic in cases where different types of processing may be desired for separate parts of the data. Instead, the present technique can partition a large dataset of data objects having various data types for a virtual world (e.g., a game world) into processing partitions for distributed or parallel processing. The technique is capable of grouping different types of data objects using object types. The object types can be assigned to a hardware host that has an application instance specifically programmed to process the object types. The use of object types enables processing to be divided into processing partitions distributed among the hardware hosts in a distributed computing system based on the type of processing to be applied to the object types. Additionally, the technique can also detect and manage the correlations between data object types, and managing the correlations can overcome bottlenecks in the processing of data objects partitioned into processing partitions. The use of data correlations defined by an object domain also enables processing to be organized into processing partitions and distributed among the hardware hosts in a distributed computing system to optimize where the data used by the application instances is located in the distributed computing system.

[0039] This technology further provides high-throughput, low-latency computing and data services that support millions of placements and retrievals per second of highly partitioned data that is partitioned into processing partitions and distributed among hardware hosts in a distributed computing system. The distributed computing system can use distributed shared memory (DSM) services on the device to manage the storage of data objects mapped to processing partitions at the hardware hosts. The distributed shared memory device can write changes to the data objects using a storage format shared across the distributed shared memory services on the device to provide a versioned snapshot of the state. The distributed shared memory service can also enable the state of the data objects to be read with low latency via subscription-based streams. Developers using this technology can achieve higher levels of performance, scalability, persistence, and reliability on a single server and across multiple servers, thus enabling stable, spatially infinite, or spatially dense interactive experiences for players.

[0040] This technology can write simulation states that are several orders of magnitude more numerous than past single-server database solutions (e.g., 300X+) and replicate the states across hardware hosts in a distributed computing system and replicate the states to users across PCs, game consoles, and mobile devices. In one configuration of the present technology, the distributed computing system can use a serialization format to represent data objects to manage the storage of the data objects applied to the application. As previously described, the data objects to be stored in the shared memory can be geometric objects (e.g., polygons, spheres, cylinders, etc.), player models, opponent models, buildings, vehicles, plants, animals, images, or any other object that can be modeled in 2D or 3D. The distributed computing system can use the serialization format to manage changes to the state of the data objects using an ordered append-only log-based storage process. The ordered append-only log-based storage process provides low replication costs and thread-safe reads in the distributed computing system. Thus, the data objects can be represented using a format shared across multiple distributed shared memory services on separate devices in the distributed computing system to provide a versioned snapshot of the state and low replication costs.

[0041] The data objects can be represented in the memory of the distributed shared memory in the hardware host using a set of semantic representations built on top of a byte array. The representation of the data objects can be written to the memory device using a byte array that is divided into multiple segments. The segments can describe the in-memory content of the data objects and contain information on how to read the in-memory content to obtain the current state of the data objects. One of the segments can include a log segment containing a series of one or more log records. Changes to the data objects caused by a processing application or application instance can be written to the log segment using an ordered, log-based format to provide a versioned snapshot of the state. The log records can be used to represent a version-ordered set of changes to the data objects.

[0042] The distributed shared memory of the hardware host can access the log segment from the tail and read the log records backward to obtain the latest state of the data object. The distributed shared memory can read the log records backward to quickly identify the object fields of the data objects that may have changed during processing and collect the changes sorted by the latest versions of the object fields of the data objects. The distributed shared memory can also read the representation of the data object initially written to the memory device when a portion of the data object in the log records in the log segment has not changed. The distributed shared memory can determine the latest state of the data object by collecting the changes sorted by the latest versions of the object fields and, when needed, collecting the original in-memory content of the data object.

[0043] In high-performance computing systems (e.g., Hadoop clusters), the format of the data to be processed is typically in a human-readable format such as JSON or XML, or as a CSV file. These formats generally do not provide an efficient format for a distributed computing system to actually store data (whether in memory or on disk). Storing data in a high-level or annotated format can also be extremely inefficient for storage efficiency through wired communication and / or parallel processing.

[0044] This technology enables representing data objects in memory, on mass storage devices, or transmitting data objects over a network connection using storage procedures and serialization formats that provide low copy costs and thread-safe reads. The storage procedures according to this technology enable copying the in-memory representation of a data object to another memory location, another hardware host, or a storage device without incurring additional serialization processing due to the in-memory representation already being serialized. The storage procedures further enable the processing of data objects to be quickly and easily distributed among the hardware hosts in a distributed computing system. The storage procedures can ensure that processing applications perform operations using the correct version of the data object and that updates to the data object are properly sorted. Additionally, this technology can also manage in-memory changes to the data object to overcome bottlenecks in the processing of the data object when processing applications modify or read it. The storage procedures can further enable state exchange between the application and the hardware host and improve the speed, efficiency, and operation of data storage and replication in a distributed computing system.

[0045] According to the present technology, spatial analysis of data objects associated with a virtual environment can be used to organize computing units in a distributed computing system. The spatial analysis can determine spatial information about the data objects, such as absolute position, relative position, proximity position, spatial dependencies, etc. The data objects can be grouped together using the spatial location information, and a collection of data objects can be considered a processing partition (i.e., a sub-division of the data objects) for an application or an instance of an application that processes the data objects assigned to the processing partition. The processing partition can be defined, in part, based on how the data objects are grouped spatially. The processing partitions can be load balanced across the hardware hosts in the distributed computing system. By using the processing partitions and spatial organization to distribute the load for processing data objects in the distributed computing system, it becomes possible to process large virtual worlds and present them to a client or user.

[0046] In one configuration, the technology uses one or more object fields that provide spatial location information of the data objects to spatially subdivide a plurality of data objects into a plurality of processing partitions. The spatial location information (e.g., x, y, and z coordinates) associated with the plurality of data objects can be determined from the object fields in the data objects and can be mapped to the processing partitions. The processing partitions can identify a set of data objects in a defined location. The processing partitions can be assigned and sent to the hardware hosts in the distributed computing system for an application to execute code with the data objects. The processing partitions can be sent to the hardware hosts to configure the hardware hosts to allow the application to process the data objects in the identified location.

[0047] Thus, data objects that are spatially close together can be grouped for one or more processing applications (e.g., physics applications). In one configuration, this technology can use a tree structure to spatially subdivide the data objects. The tree structure can be a two-dimensional octree or quadtree that covers a three-dimensional or planar environment of the virtual world. Each node in the tree structure can identify a set of data objects with spatial proximity or spatial grouping that will be processed by an instance of an application configured to process one object type. The nodes in the tree can be used to assign the data objects to an instance of an application that processes the data objects. For example, a plurality of "particle objects" can be assigned to a plurality of processing partitions by spatially subdividing the particle objects by location. One or more of the processing partitions can then be assigned to one or more particle simulation application instances on a hardware host or across multiple hardware hosts for processing.

[0048] In cases where different types of processing can be applied to parts of data, partitioning the data for parallel processing can be problematic. This technique overcomes this difficulty by partitioning a large dataset of data objects with different data types for a virtual world (e.g., a game world) into processing partitions for parallel processing. The technique takes into account the spatial relationships of the data objects in order to group the data objects and process them more quickly. The use of spatial analysis enables the processing to be divided into processing partitions distributed among the hardware hosts in a distributed computing system so as to group related data as closely as possible. In addition, the technique can also detect and manage the spatial dependencies between data object types, and managing the spatial dependencies can overcome bottlenecks in the processing of data objects partitioned into processing partitions. Using data dependencies defined by an object domain also enables the processing to be organized into processing partitions among the hardware hosts in a distributed computing system to optimize where the data used by an application instance is located in the distributed computing system.

[0049] According to this technique, a distributed computing system can use an object domain to identify and control processing applications executed on hardware hosts. The hardware hosts can be pre-loaded with processing applications using an application library. The application library can be stored by the hardware hosts and used by an application manager on the hardware hosts to start instances of the processing applications according to the associated object types. As described above, the object domain can be used to define which applications and / or application instances can be used to perform the calculations of the object domain and which applications and / or application instances can have exclusive change permissions (e.g., write permissions) for data objects of the associated object type. Starting instances of processing applications in a distributed computing system using object types enables large virtual worlds to be processed and presented, since the processing can be assigned to any number of hardware hosts having a copy of the application library. According to one instance, a hardware host can receive an assigned processing partition and identify from the application library a processing application for an object type associated with (e.g., matching or capable of performing functions on) the processing partition. The hardware host can start an instance of the processing application so that the hardware host can process multiple data objects grouped by the object type, and the data objects are mapped to the processing partition.

[0050] In another configuration, a distributed computing system can enable hardware hosts to communicate via subscription-based streams. The processing applications in the application library can define subscription policies for obtaining additional data used during the processing of data objects. The subscription policies can identify what additional data object types are used during the processing and where the additional object types can be located. The hardware hosts can use the data dependencies, spatial relationships, object tracking, and queries defined in the subscription policies to send subscription requests so that the hardware hosts can transmit and replicate the data objects used by the instances of the processing applications during the processing.

[0051] Accordingly, a distributed computing system can enable hardware hosts to share data objects used by processing applications distributed across the hardware hosts. In one configuration, this technique can enable a first hardware host to receive multiple processing partition assignments and identify a first processing partition assigned to the first hardware host. The first hardware host can then determine a first object type associated with the first processing partition and use the first object type to initiate an instance of a corresponding processing application from an application library. The first hardware host can also determine a second object type on which the processing of data objects of the first object type by the processing application depends. The first hardware host can use the multiple processing partition assignments to determine a second hardware host assigned the second processing partition, and the second hardware host groups data objects into the second processing partition based on the second object type. The first hardware host can send a subscription request to the second hardware host to instruct the second hardware host to replicate (e.g., make a copy of the underlying data object or make a change to the data object) a second plurality of data objects to the first hardware host and the instance of the processing application.

[0052] In another configuration, the technique can enable a first hardware host to determine a subscription policy that identifies adjacency relationships between spatial sub-divisions associated with multiple spatial sub-divisions of a multi-dimensional virtual environment to filter data objects of the second object type that satisfy the filter. The first hardware host can use an adjacency relationship that satisfies the subscription policy between a first spatial sub-division associated with the first processing partition and a second spatial sub-division associated with the second processing partition to identify the second processing partition.

[0053] In yet another example, a first hardware host can determine a subscription policy that identifies query criteria (e.g., a vehicle with a low hit point or a specific moving object) associated with a query for filtering data objects of the second object type. The first hardware host can identify the second processing partition by matching the second plurality of data objects with the query criteria using the subscription policy. The first hardware host can also determine a subscriber list for data objects of the first object type. The first hardware host can send the data objects to the subscriber list. The first hardware host can further receive an update to the multiple processing partition assignments. The first hardware host can determine a migration of the second processing partition between the second hardware host and a third hardware host and send a second subscription request from the first hardware host to the third hardware host.

[0054] Figure 1System 100 is shown that may include a service provider environment 102 hosting a compute-intensive distributed computing system for providing a virtual environment accessible and / or viewable by a user. System 100 may include a service provider environment 102 and one or more clients 104 communicating with the service provider environment 102 using a network 106. Network 106 may include any useful computing network, including an intranet, the Internet, a local area network, a wide area network, a wireless data network, or any other such network or combination thereof. Components for such a system may depend at least in part on the type of network and / or environment selected. Communication over the network may be achieved through wired or wireless connections and combinations thereof. The service provider environment 102 is capable of managing and delivering computing, storage, and networking capabilities as software services to a community of end recipients using the clients 104. In this example, the service provider environment 102 may include an infrastructure manager 110, one or more infrastructure services 112, and one or more distributed computing systems 120.

[0055] The infrastructure manager 110 may manage and control physical machines and other physical devices for providing a compute-intensive distributed computing system. The infrastructure manager 110 may be a service including one or more computing systems such as server computers configured to control the physical and virtualized infrastructure resources of the (one or more) infrastructure services 112 and the (one or more) distributed computing systems 120. The virtualized infrastructure resources of the infrastructure services 112 and the distributed computing systems 120 may include virtualized executable resources, virtualized storage resources, virtual network interfaces, and other virtualized networking components. Some examples of virtualized executable resources may include compute instances, containers (e.g., open-source container orchestration system (Kubernetes)), compute functions, hosted applications, etc. Some instances of virtualized storage services may include database services, block storage services, content delivery services, etc. Some examples of virtualized networking components may include virtualized networking devices and physical network devices (i.e., routers, firewalls, load balancers, etc.) configured with logical roles in the infrastructure services 112 and the distributed computing systems 120.

[0056] The infrastructure manager 110 can instantiate all or part of the infrastructure services 112 and the distributed computing system 120. The infrastructure manager 110 can identify the physical hosts and physical networking devices to be managed by the infrastructure manager 110. Some examples of the physical hosts managed by the infrastructure manager 110 are server computers, embedded computers, computing hosts at the edge of the service provider environment 102, and other physical devices. Some examples of the physical networking devices used by the infrastructure manager 110 are routers, switches, firewalls, load balancers, cache services, server computers, embedded devices, and other physical networking devices.

[0057] The infrastructure services 112 can be considered on-demand computing services hosted in servers, virtual machines, grids, clusters, parallel, or distributed computing systems. Some examples of the infrastructure services 112 that can be provided by the service provider environment 102 can include one or more computing services 114, one or more storage services 116, networking services, network services, streaming services, network-accessible services, software as a service, storage as a service, on-demand applications, services for performing code functions, and so on. In this example, the (one or more) infrastructure services 112 can include a distributed computing service 118 that is used to establish and host the (one or more) distributed computing systems 120 as a large-scale, low-latency, distributed computing system.

[0058] (One or more) distributed computing systems 120 can be considered on-demand computing-intensive distributed computing systems hosted in servers, virtual machines, grids, clusters, parallel, or distributed computing systems. (One or more) distributed computing systems 120 can be used to provide virtual environments (e.g., 2D (two-dimensional) or 3D (three-dimensional) virtual environments), simulations, and "computationally intensive games" that include persistent interactive virtual worlds that can host millions of concurrent users or gamers. The distributed computing system 120 can provide high-throughput, low-latency computing and data services that support millions of puts and gets per second for highly partitioned data and data objects.

[0059] (One or more) distributed computing systems 120 may include a control plane 122, a data plane 124, and an ingress plane 126. The control plane 122 may be responsible for coordinating the creation of a distributed computing system for simulation, virtual environments, gaming, etc., and the control plane 122 may be responsible for load balancing across hardware hosts in the distributed computing system. Hardware hosts (such as hardware hosts 142 and 144) may be physical hosts, hardware computing devices, or servers capable of hosting one or more virtualized computing resources or services. The control plane 122 may manage aspects of the distributed computing system, such as running simulations and virtual environments, including deploying code for execution, collecting metrics, uploading logs, and monitoring a cluster of hardware hosts.

[0060] The control plane 122 may also provide a user-accessible dashboard for performing auditable maintenance actions across a cluster of hardware hosts, and a development team can use the dashboard to access operational characteristics of running simulations and virtual environments. The control plane 122 may include a system manager 130 that manages resources used in the distributed computing system via simulations and virtual environments, such as a hardware host warm pool 132 and one or more cluster services 134. The hardware host warm pool 132 may include one or more hardware hosts with preloaded operating systems, applications, assets, and / or data waiting for an assignment from the system manager 130. Warm hardware hosts may represent physical hosts or virtualized infrastructure resources allocated for a simulation for later use. The cluster services 134 may include services that support the operation of the distributed computing system, such as health monitoring, ticketing, logging, metrics, deployment, workflows, etc.

[0061] The data plane 124 may be responsible for the processing of simulations, virtual environments, games, etc. in a distributed computing system, and the data plane 124 may be responsible for communication across hardware hosts in the distributed computing system. The data plane 124 may include a hardware host activity pool 140, on which computing units of a simulated or virtual world are distributed using one or more hardware hosts, such as hardware hosts 142 and 144. Hardware hosts 142 and 144 include one or more files 150a-b, one or more applications 152a-b, and one or more distributed shared memories 154a-b. The files 150 may be data, resources, data objects, 3D models, 2D models, textures, images, animations, web pages, etc. This or these applications 152 may include a class of user computing or execution processes that may execute on data (e.g., data objects) associated with this or these files 150, and the applications describe multiple application instances in a collective manner. For example, an application may be a physics application that modifies a data object using a physics-based digital method. The distributed shared memory 154 may manage data in the memory used and shared across the applications 152. The distributed shared memory 154 may also replicate changes to the data in the memory across the hardware hosts in the hardware host activity pool 140. For example, the (one or more) applications 152 may subscribe to a stream of data object events by requesting that an update to the data in the memory be delivered from the distributed shared memories 154a-b to the (one or more) applications 152. In another example, the (one or more) applications 152 may create a data object event stream that includes modifications to the data in the memory, and the modifications are sent to the distributed shared memories 154a-b. The distributed shared memories 154a to 154b may replicate changes in the state of the data in the memory between the distributed shared memories (DSM) of the hardware hosts 142 and 144.

[0062] The ingress plane 126 can be responsible for facilitating communication with a distributed computing system for simulations, virtual environments, gaming, etc. For example, the ingress plane 126 can manage connections from outside the distributed computing system(s) 120. The ingress plane 126 can include a front-end service resource manager 160, one or more remote ingress points 162, and one or more client gateways 164. The client(s) 104 can utilize an application programming interface (API) to connect to the client gateway(s) 164, and the client gateway(s) 164 authenticate credentials and authorize the connection to the remote ingress point(s) 162. The remote ingress point(s) 162 can include a multi-tenant queue that copies data from the virtual world or simulation to the physical hosts of the client(s) 104. The front-end service resource manager 160 can provide a graphical user interface for accessing, managing, monitoring, updating, or developing distributed applications, data objects, and / or assets executed in the service provider environment 102.

[0063] The technology can provide serverless virtual world or simulation services for building and operating computationally large worlds, such as massively multiplayer games with millions of connected players and city-scale simulations with millions of persistent objects. Developers can create millions of data objects and launch applications (e.g., computational processes) that modify the data objects at variable rates (e.g., 10 - 60 Hz). The technology can automatically manage where and when the millions of data objects and applications are assigned to hardware hosts. The technology can load balance application instances across hardware hosts in response to various conditions, such as the load on the virtual world, or co-locate the data processed by the applications to reduce replication costs).

[0064] Figure 2 Various example components included in a service provider environment 200 of a computationally intensive distributed computing system for providing a user-visible virtual environment according to an example of the present technology are shown. In this example, the service provider environment 200 may be capable of delivering computing, storage, and networking capabilities as software services to a community of end recipients. In one example, the service provider environment 200 can be established by or on behalf of an organization for the organization. That is, the service provider environment 200 can provide a "private cloud environment." In another example, the service provider environment 200 can support a multi-tenant environment where multiple customers can operate independently (i.e., a public cloud environment). Generally, the service provider environment 200 can provide the following models: infrastructure as a service ("IaaS") and / or software as a service ("SaaS"). Other models can be provided. For the IaaS model, the service provider environment 200 can provide computers as physical or virtual machines and other physical devices to be used as virtualized infrastructure resources in a virtual infrastructure.

[0065] Application developers can develop and run their applications on the service provider environment 200 without incurring the costs of purchasing and managing the underlying hardware and software. The SaaS model allows for the installation and operation of applications in the service provider environment 200. For example, end customers can use networked client devices (such as desktop computers, laptop computers, tablet computers, smart phones, game consoles, etc.) that run web browsers or other stand-alone client applications to access the service provider environment 200. The service provider environment 200 can include a plurality of computing devices arranged, for example, in one or more server groups or computer groups or other arrangements. The computing devices can use hypervisors, virtual machine managers (VMMs), and other virtualization software to support the computing environment. In this instance, the service provider environment 200 can include one or more server computers 202. The (one or more) server computers 202 can include a system manager module 210, a cluster manager module 210a, a world manager module 210b, the (one or more) cluster service modules 212, a data repository 214, a front-end service resource manager module 216, a remote entry point module 218, a client gateway module 220, one or more warming hardware hosts 224, one or more active hardware hosts 226, one or more processors 230, and one or more memory modules 232.

[0066] The system manager module 210 can include the hardware and software configured to create, deploy, and manage high-throughput low-latency distributed computing systems. The system manager module 210 can use the cluster manager module 210a and the world manager module 210b to coordinate the use of clusters of hardware hosts to create distributed computing systems for simulations, virtual environments, etc. The cluster manager module 210a can manage various aspects of running the infrastructure associated with the distributed computing system, including deploying code for execution, collecting metrics, uploading logs, and monitoring cluster resources. The world manager module 210b can manage various aspects of running the multi-dimensional virtual environments, worlds, or simulations hosted by the distributed computing system, including allocating files and applications to the hardware hosts and monitoring changes in the processing of data on the hardware hosts by the applications. The system manager module 210 can also use the cluster manager module 210a and the world manager module 210b to perform load balancing of computing operations across the hardware hosts.

[0067] The cluster service module 212 may include hardware and software components configured to provide services that support the operation of a distributed computing system, such as a monitoring service 240, a logging service 242, a ticketing service 244, a workflow service 246, a deployment service 248, and an operations console service 250. The monitoring service 240 may be used to monitor resources used by the distributed computing system. For example, the monitoring service 240 may collect metrics associated with the active hardware host(s) 226 to determine the utilization of one or more hardware or software resources. The logging service 242 may be used to manage and analyze logs and operational data points from the distributed computing system. The ticketing service 244 may be used for ticketing issues or for checking in or checking out source code, executable code, or data. The workflow service 246 may be used to manage and execute workflows in the distributed computing system. For example, the workflow service 246 may initialize, construct, or load virtual computing resources, data objects, hardware hosts, etc. The deployment service 248 may be used to configure and deploy hardware hosts in the distributed computing system. The operations console service 250 may be used to interact with the control plane, data plane, and ingress plane associated with the distributed computing system. The operations console service 250 may provide one or more user interfaces (textual or graphical), application programming interfaces (APIs), etc., through which a user may input commands or retrieve data associated with the distributed computing system.

[0068] The data repository 214 may include hardware and software components configured to provide data services to a distributed computing system managed by the cluster manager module 210a and a multi-dimensional virtual environment managed by the world manager module 210b. The data repository 214 may include one or more world / cluster configurations 260, one or more files 262, and one or more applications 264. The world / cluster configurations 260 may define the distributed computing system for a world or simulation. For example, an object-based model of the virtual environment may be used to define the world. The virtual environment may be a 2D, 3D, or multi-dimensional virtual world. The object-based model may use data objects to represent entities within the virtual environment. The files 262 may include data, resources, data objects, 3D models, 2D models, textures, images, animations, web pages, etc. used by the distributed computing system. In one example, the data objects in the (one or more) files 262 may represent entities within the virtual environment. These entities may be characters, animations, geometric objects, actors, non-player characters, vehicles, buildings, plants, rocks, animals, etc. The data objects in the files 262 may be represented in the data repository 214 using properties with key-value or name-value pairs and object fields with field identifiers and field values. The (one or more) applications 264 may include executable code that processes the (one or more) files 262. For example, the (one or more) applications 264 may process data objects used to represent entities within the virtual environment. Some examples of the (one or more) applications 264 may include simulation, collision detection, physics engines, rendering, etc.

[0069] The front-end service resource manager module 216 may include hardware and software components configured to provide access, management, monitoring, updating, or development of the (one or more) world / cluster configurations 260, the (one or more) files 262, and the (one or more) applications 264. The front-end service resource manager module 216 may include one or more graphical user interfaces to build virtual worlds, define data objects, write applications, etc. The remote entry point module 218 may include a multi-tenant queue that copies data from the data repository 214 and / or the active hardware host 226 to the physical host of a remote client. Further, the client may utilize an API to connect to the client gateway module 220, which authenticates credentials and authorizes the connection to the remote entry point module 218.

[0070] (One or more) heating pool hardware hosts 224 may include hardware and software elements configured to execute (one or more) applications 264 using (one or more) files 262 in a standby mode. The cluster manager module 210a may assign multiple hardware hosts to (one or more) heating pool hardware hosts 224 in a standby or suspended operation mode in anticipation of future needs. The cluster manager module 210a may migrate hardware hosts between (one or more) heating pool hardware hosts 224 and (one or more) active pool hardware hosts 226 to scale computing resources when a need arises. (One or more) active pool hardware hosts 226 may include hardware and software elements configured to execute (one or more) applications 264 using (one or more) files 262 in an active mode. (One or more) active pool hardware hosts 226 may include a hardware host manager module 270, an application manager module 272, one or more application instances 284, a distributed shared memory module 274, a file system data repository 276, and an in-memory data repository 278.

[0071] The hardware host manager module 270 may include hardware and software elements configured to manage data processing performed by one or more hardware hosts. The hardware host manager module 270 may include a runtime module that receives instructions from the cluster manager module 210a, and the world manager module 210b executes the instructions on the hardware host. The instructions may configure the hardware host in an active mode to execute the application 264 using the file 262. The instructions may identify the file 262 assigned to the hardware host and which one of the applications 264 to use to process the assigned file 262. The hardware host manager module 270 may receive one or more of (one or more) files 262 for processing and store the files as (one or more) local files 280 in the file system data repository 276. According to an example of the present technology, the hardware host manager module 270 may receive one or more index structures assigned to the hardware host from the world manager module 210b. The index structures may be used by the world manager module 210b to organize the files 262 into computing units, which are referred to herein as processing partitions discussed in more detail below. The hardware host manager module 270 may use the index structures to manage the storage of the local files 280 in the file system data repository 276.

[0072] The instruction can further identify one or more application instances 264 for processing one or more of the one or more local files 280. The hardware host manager module 270 can receive one or more of the one or more applications 264 and store the applications as one or more local applications 282 in the file system data repository 276. According to an example of the present technology, the hardware host manager module 270 can receive an application library including one or more applications 264. The application library can be stored by the hardware host manager module 270 and used by the application manager module 272 to start an instance of the local application 282 in the file system data repository 276.

[0073] The application manager module 272 can include hardware and software elements configured to manage one or more application instances 284 on the hardware host. Examples of the application instance 284 can be an instance of a physical application, a rendering application, a collision application, a transformation application, an occlusion application, a sound application, or other types of applications. The application manager module 272 can identify the local files 280 assigned to the hardware host and determine which of the local applications 282 to instantiate. The application manager module 272 can determine the number of instances of the one or more local applications 282 to be instantiated as one or more application instances 284. The application manager module 272 can scale the number of application instances 284 as needed.

[0074] The distributed shared memory module 274 can include hardware and software elements configured to manage the in-memory data repository 278. The in-memory data repository 278 of the hardware host can be used by the application instance 284 for storage, and changes to the in-memory data repository 278 can be shared across the active pool of hardware hosts 226. The distributed shared memory module 274 can load one or more of the local files 280 into the in-memory data repository 278 as the shared memory 286. The distributed shared memory module 274 can receive requests to access the shared memory 286 from the application instance 284. The distributed shared memory module 274 can send data from the shared memory 286, for example, by processing requests to read data from the application instance 284. The distributed shared memory module 274 can also receive requests to modify the shared memory 286 from the application instance 284. The distributed shared memory module 274 can store data in the shared memory 286, for example, by processing requests to write data from the application instance 284. The distributed shared memory module 274 can further replicate changes to the shared memory 286 across the active hardware hosts 226.

[0075] The different processes and / or other functions included within the service provider environment 200 may be executed on one or more processors 230 that communicate with one or more memory modules 232. The service provider environment 200 may include a plurality of computing devices arranged in, for example, one or more server groups or computer groups or other arrangements. The computing devices may use a hypervisor, a virtual machine monitor (VMM), and other virtualization software to support the computing environment.

[0076] The term "data repository" may refer to any device or combination of devices capable of storing, accessing, organizing, and / or retrieving data, which may include any combination and number of data servers, relational databases, object-oriented databases, clustered storage systems, data storage devices, data warehouses, flat files, and data storage configurations in any centralized, distributed, or clustered environment. The storage system components of data repositories 214, 276, and 278 may include storage systems such as SAN (Storage Area Network), cloud storage networks, volatile or non-volatile RAM, optical media, or hard drive type media. As can be appreciated, data repositories 214, 276, and 278 may represent multiple data repositories.

[0077] Figure 2 Certain processing modules are shown that may be associated with this technology and these processing modules may be implemented as computing services. In one example configuration, a module may be considered a service having one or more processes executed on a server or other computer hardware. Such a service may be a centrally hosted function or a service application that can receive requests and provide outputs to other services or consumer devices. For example, a module providing a service may be considered on-demand computing hosted in a server, a virtualized service environment, a grid, or a clustered computing system. An API may be provided for each module such that a second module can send a request to the first module and receive an output from the first module. Such an API may also allow third parties to interface with the module, make requests, and receive outputs from the module. Although Figure 2 An example of a system in which the above technology may be implemented is shown, but many other similar or different environments are possible. The example environments discussed and shown above are merely representative and not restrictive.

[0078] Figure 3FIG. 0 is a block diagram showing an example computing service 300 that provides a computationally intensive distributed computing system using a distributed computing service manager, in accordance with an example of the present technology. The computing service 300 can be used to execute and manage multiple computing instances 304a-d on which the present technology can execute. Specifically, the depicted computing service 300 shows one environment in which the techniques described herein can be used. The computing service 300 can be a type of environment that includes different virtualized service resources that can be used, for example, to host the computing instances 304a-d.

[0079] The computing service 300 may be capable of delivering computing, storage, and networking capabilities as a software service to a community of end recipients. In one example, the computing service 300 can be established for or on behalf of an organization by the organization. That is, the computing service 300 can provide a "private cloud environment". In another example, the computing service 300 can support a multi-tenant environment in which multiple customers can operate independently (i.e., a public cloud environment). Generally, the computing service 300 can provide the following models: Infrastructure as a Service ("IaaS"), and / or Software as a Service ("SaaS"). Other models can be provided. For the IaaS model, the computing service 300 can provide computers as physical machines or virtual machines and other resources. The virtual machines can be run as guests by a hypervisor, as further described below. In another configuration, the service model delivery can include the computing of an operating system, a programming language execution environment, a database, and a web server.

[0080] Application developers can develop and run their software solutions on the computing service without incurring the costs of purchasing and managing the underlying hardware and software. The SaaS model allows for the installation and operation of application software in the computing service 300. For example, end customers can access the computing service 300 using a networked client device such as a desktop computer, laptop computer, tablet computer, smart phone, etc. running a web browser or other lightweight client application. Those skilled in the art will recognize that the computing service 300 can be described as a "cloud" environment.

[0081] The specifically illustrated computing service 300 may include multiple server computers 302a-d. The server computers 302a-d may also be referred to as physical hosts. Although four server computers are shown, any number may be used, and large data centers may include thousands of server computers. The computing service 300 may provide computing resources for executing computing instances 304a-d. The computing instances 304a-d may be, for example, virtual machines. A virtual machine may be an instance of a software implementation of a machine (i.e., a computer) that executes applications like a physical machine. In the example of a virtual machine, each of the server computers 302a-d may be configured to execute an instance manager 308a-d capable of executing instances. The instance managers 308a-d may be a hypervisor, a virtual machine manager (VMM), or another type of program configured to enable the execution of multiple computing instances 304a-d on a single server. Additionally, each of the computing instances 304a-d may be configured to execute one or more applications.

[0082] The server computer 314 may be reserved to execute software components for implementing the present technology or managing the operations of the computing service 300 and the computing instances 304a-d. For example, the server computer 314 may execute a distributed computing service manager 315 to provide high-throughput, low-latency computing and data services that support highly partitioned data placement and retrieval millions of times per second.

[0083] The server computer 316 may execute a management component 318. A user may access the management component 318 to configure various aspects of the operations of the computing instances 304a-d purchased by the customer. For example, the user may set up the computing instances 304a-d and make changes to the configurations of the computing instances 304a-d.

[0084] A deployment component 322 may be used to assist a customer in deploying the computing instances 304a-d. The deployment component 322 may access account information associated with the computing instances 304a-d, such as the name of the owner of the account, credit card information, the country of the owner, etc. The deployment component 322 may receive a configuration from the user that includes data describing how the computing instances 304a-d may be configured. For example, the configuration may include an operating system, one or more applications to be installed in the computing instances 304a-d, a script and / or other type of code to be executed for configuring the computing instances 304a-d, cache logic specifying how an application cache will be prepared, and other types of information. The deployment component 322 may use the configuration and cache logic provided by the user to configure, populate, and start the computing instances 304a-d. The configuration, cache logic, and other information may be specified by a user accessing the management component 318 or by directly providing the information to the deployment component 322.

[0085] Customer account information 324 may include any desired information associated with a customer of a multi-tenant environment. For example, customer account information 324 may include a unique identifier for the customer, a customer address, billing information, licensing information, customization parameters for launching instances, scheduling information, etc. As described above, customer account information 324 may also include security information used in encryption of asynchronous responses to API requests. By "asynchronous," it is meant that the API response may be made at any time after the initial request and using a different network connection.

[0086] The network 310 may be used to interconnect the computing service 300 and the server computers 302a-d, 316. The network 310 may be a local area network (LAN) and may be connected to a wide area network (WAN) 312 or the Internet so that end clients can access the computing service 300. In addition, the network 310 may include a virtual network overlaid on a physical network to provide communication between the server computers 302a-d. Figure 3 The network topology shown in is not limited to that shown in because more networks and networking devices can be utilized to interconnect the different computing systems disclosed herein.

[0087] Figure 4A 4 shows a system 400 and related operations for assigning computing units in a distributed computing system using processing partitions organized by object domains according to examples of the present technology. The system 400 may include a world manager 402. Some examples of the world manager 402 may include reference to Figure 1 The system manager 130 described in Figure 2 Described world manager module 210b.

[0088] The world manager 402 may identify one or more data objects 404 associated with an object-based model of a multi-dimensional virtual environment (e.g., a 3D simulation, a 3D game, etc.). The world manager 402 may retrieve the data object 404 from an object data repository. The data object 404 may include one or more object fields 420. The object fields 420 may include a set of fields, properties, or characteristics represented by a field name 422 and a field value 424, where the field name 422 may serve as an identifying tag or key. The field value 424 may include a data value, a string, an X, Y, Z coordinate, etc., which may be referenced by a processing application using an object identifier 430 in order to process the data object.

[0089] In this example, the data object 404 may include metadata that uses one or more of the object fields 420 to provide an object identifier (ID) 430, an object ID value 431, an object version 432, an object version value 433, an object type 434, and an object type value 425. An example object ID value 431 for the object identifier 430 field may be "forest_tree_object" and an example object ID value 431 for the object version 432 field may be "version 4.214". The object type 434 may map to an object domain and have object type values 435 such as "particle", "tree", "water", etc. The object domain describes which object types form part of the domain, thereby implementing permissions regarding what processing applications have exclusive change permissions, as well as data dependencies (e.g., read-only permissions) for the object types used in the processing of the object types that form part of the domain. An object type may be owned by a domain that has a processing application representing its computation. The type of processing may include any type of processing of data objects that may occur for a virtual world, simulation, or game.

[0090] Then, the world manager 402 may use the object type 434 of the data object 404 to determine one or more processing partitions 406. In addition to the data object 404 itself, the world manager 402 may use the world configuration 408 to determine the processing partitions 406, as discussed in more detail later. The processing partitions 406 may define computational units in a distributed computing system that hosts a multi-dimensional environment, and the computational units may be assigned to hardware hosts in the distributed computing system. The data object 404 may be organized or grouped into the processing partitions 406 through the object type value 435 of the object type 434 to create computational units for the hardware hosts. Thus, the processing partitions 406 may group the data object 404 into computational units according to the object domain represented by the object type. Metadata in the processing partitions may refer to the set of data objects or data object set mapped to the processing partition as a computational unit for processing.

[0091] According to an example of the present technology, the world manager 402 may use the object fields 420 of the data object 404 to determine the processing partitions 406. Since the processing partitions 406 represent groups of data objects organized as computing units to be processed by a processing application, the world manager 402 may organize the data object 404 according to one or more of the object fields 420. According to an example of the present technology, this data object management performed by the world manager 402 may include the creation of one or more index structures, known as domain indexes, which map a collection of data objects 404 to the processing partitions 406 using the object type 434 through the object domain, as will be discussed in a later section. The index structure may index the data object 404 into the processing partitions 406. According to another example of the present technology, this data object management performed by the world manager 402 may include the creation of one or more index structures, known as spatial indexes, which may be used to partition the processing of the data object 404 by spatial location information (e.g., spatial location, bounding box, etc.) in the object fields 420, as will be discussed in a later section. The world manager 402 may use the (one or more) object fields 420 to create multiple layers in grouping the data object 404 into computing units, thereby further organizing the (one or more) data objects 404 by multiple fields, attributes, and characteristics represented by the (one or more) object fields 420.

[0092] The world manager 402 may then determine one or more processing partition assignments 410 to assign the processing partitions 406 to one or more hardware hosts 412. The processing partition assignments 410 may be made based on the object type and application instances of the processable object type, based on data correlation, and based on metrics of the processing load on the hardware or software (e.g., to avoid overload, etc.). The world manager 402 may use the processing partition assignments 410 to send the processing partitions 410 to the hardware hosts 412 to allocate the processing of the data object 404 in a distributed computing system. The processing partition assignments 410 may be used to configure the hardware hosts 412 for processing the data object 404.

[0093] The hardware host 412 can process the data object 404 using the processing partition assignment 410. Examples of processing can include data object movement, data object collision, rendering, explosion, physical simulation, in-world credit transactions, or other types of processing. The data object 404 for a particular processing partition in the processing partition 406 can be loaded at one of the hardware hosts 412 and then processed by a processing application. For example, an instance of the processing application can be established on the hardware host to process the explosion of the data object 404. The instance of the processing application can iterate across one or more data objects mapped to the processing partition assigned to the hardware host to simulate what happens to the data object 404 during the explosion. The changes to the data object 404 can then be written to the memory in the hardware host (using DSM). The changes written to the memory can also be shared with other hardware hosts.

[0094] The technology can use a multi-level index to organize data objects into the processing partitions 406 representing computing units in a distributed computing system. For example, the world manager 402 can create multiple levels as a hierarchical structure. A multi-level index scheme can be used, where the object domain grouping is the most important level. Figure 4B The global view of the multi-dimensional virtual environment is separated into more than one group or processing partition according to the object domain. The multi-dimensional environment can include data objects with two object domains. In the global view, the first object domain "Domain A" is represented by a cross, while the second object domain "Domain B" is represented by solid dots. The world manager 402 can decompose the global view into "Domain A decomposition" and "Domain B decomposition", which becomes the most important level for grouping data objects according to the object types associated with "Domain A" and "Domain B". This decomposition process also allows the less important levels in the multi-level index scheme to use similar semantics while being different in terms of instances. For example, in Figure 4C if each of "Domain A" and "Domain B" includes a large number of data objects densely packed into different spaces, this decomposition process can add another level (such as a spatial index) to further index the data objects into the processing partitions using the spatial location information. As depicted, since the spatial index components of the two domains can be in different states, the world manager 402 can organize the data objects into the processing partitions 406 and determine whether to assign the data objects in different states to different hardware hosts.

[0095] Figure 4BA permission model that can be used with object domains is also shown. More specifically, the domain allows developers to declare which application owns each of the object types and which additional object types are dependent on any particular domain. While other applications may read data objects that their domain does not own, they may not mutate them (e.g., may not write to the data object). As shown, the "Domain A" permissions for the first application can include read-only permissions for data objects associated with "Domain B" and exclusive write permissions for data objects associated with "Domain A". The "Domain B" permissions for the second application can include read-only permissions for data objects associated with "Domain A" and exclusive write permissions for data objects associated with "Domain B".

[0096] Figure 4B It is further shown how the permission model can be used with object domains for replication signaling. More specifically, since the domain allows developers to declare which application owns each of the object types and which additional object types are dependent on any particular domain, the permission model can signal whether data objects are to be shared by applications or replicated across a distributed computing system. For example, a first application associated with "Domain A" can declare that the first application owns data objects associated with the object type of "Domain A". The first application can further subscribe to the data object event stream by requesting delivery of updates to data objects associated with the object type of "Domain B". This replication signaling mechanism can act as an object filtering mechanism to reduce the number of objects that need to be replicated across hardware hosts. For example, a domain that requires an object type of another domain does not imply a reverse relationship.

[0097] Figure 5 Illustrated are various example components included in a service provider environment 500 according to one example of the present technology for managing hardware hosts in a distributed computing system using processing partitions organized by object domains. The service provider environment 500 can include one or more server computers 502 that communicate with one or more hardware hosts 504 using a network 506. The network 506 can include any useful computing network, including an intranet, the Internet, a local area network, a wide area network, a wireless data network, or any other such network or combination thereof.

[0098] Server computer 502 may include a world manager module 510, a first data repository 512, a second data repository 514, a third data repository 516, one or more processors 518, and one or more memory modules 520. The world manager module 510 may be configured to create, deploy, and manage a high-throughput low-latency distributed computing system. The world manager module 510 may use the hardware host 504 to coordinate the creation of a distributed computing system for virtual worlds, games, simulations, virtual environments, etc. The world manager module 510 may manage various aspects of running the distributed computing system, including deploying code for execution (e.g., application instances), collecting metrics, uploading logs, and monitoring resources. The world manager module 510 may access the first data repository 512 to create and manage a world configuration 522 for the virtual world, one or more data objects 524 stored for the virtual world, and one or more applications 526. The world configuration 522 may identify the virtual world and define the assets and applications used by the virtual world. The world configuration 522 may specify one or more indexes for partitioning assets into processing partitions. An example of the world configuration 522 may use the following world mode:

[0099]

[0100] (One or more) data objects 524 may identify objects and entities in the virtual world and define the properties and characteristics of the assets. The data object may define a set of object fields and a set of indexable fields mapped to its existing fields. The data object 425 may use, for example, the following object definition and object schema definition:

[0101]

[0102] For the above, some examples of (one or more) data objects 524 may include:

[0103]

[0104] (One or more) applications 526 may identify and store copies of processing applications and application instances that handle the computations and processing in the virtual world. The application may specify the object domains to be processed during the computations and processing in the virtual world. The application may identify or be linked to an object type to indicate a given object domain. The application may further specify an index that can be used by the world manager module 510 to partition the data objects 524 into processing partitions or chunks. For example, the application may specify the maximum number of data objects that an application instance may write during a specific application period or processing cycle.

[0105] The application can also define a subscription policy for obtaining additional data used during the processing of data objects. The subscription policy can identify what additional data object types are used during processing and where the additional object types can be located. Additionally, the subscription policy can specify the maximum latency that an application instance can tolerate in obtaining the additional object types from a specified source. One or more applications 526 can use, for example, the following application definitions:

[0106]

[0107] For the above, some instances of application 526 can include:

[0108]

[0109]

[0110] The world manager module 510 can also access a second data repository 514 to create and manage one or more hardware host configurations 528 and store one or more hardware host metrics 530. For example, the world manager module 510 can store configurations of computing instance types to be used as hardware hosts, configurations of containers on the hardware hosts, the capacities of one or more hardware or software resources of the computing instances, and one or more types of processing partitions that the computing instances can handle. Additionally, the world manager module 510 can obtain and track one or more hardware host metrics 530. These hardware host metrics 530 can identify and track the online / offline status of the hardware hosts, the utilization of the hardware or software resources of the hardware hosts, and other performance metrics associated with the hardware hosts.

[0111] The world manager module 510 can access a third data repository 516 to create and manage one or more processing partitions 532 and one or more processing partition assignments 534. The processing partitions 532 can contain data structures that act as indexes to map data objects 524 to computing units that can be assigned to hardware hosts. The processing partitions 532 can include metadata that identifies the processing partitions and one or more of the data objects 524 mapped to the processing partitions.

[0112] As previously described, the object domain can be used to distribute the processing of data objects in a distributed computing system that hosts a virtual environment. In this instance, the world manager module 510 can access the first data repository 512 to identify the object types of the data objects 524. The world manager module 510 can create a list of object types used in the virtual world. The world manager module 510 can then use the list of object types to create processing partitions 532 to provide a mapping between the data objects 524 and the processing partitions 532. For example, a first object type (e.g., person) can be mapped to a first processing partition, and a second object type (e.g., vehicle) can be mapped to a second processing partition.

[0113] The world manager module 510 may also use the world configuration 522 and the application 526 to create processing partitions 532. For example, the world manager module 510 may use the world configuration 522 to determine the total number of processing partitions that can be supported in the virtual environment. In another example, the world manager module 510 may use the configuration of the application 526 to determine the maximum or minimum number of data objects that can be assigned to a processing partition. The world manager module 510 may also use the hardware host configuration 528 and the hardware host metrics 530 to create processing partitions 532. For example, the world manager module 510 may use the hardware host configuration 528 to determine the number of available hardware hosts, the type of hardware hosts, the hardware and software resources assigned to the hardware hosts, etc., in order to determine the total number of processing partitions. In another example, the world manager module 510 may use the hardware host configuration 528 to determine the capacity of the resources defined in the hardware host configuration 528 to determine the total number of processing partitions in the processing partitions 532 or the total number of data objects 524 mapped to one of the processing partitions 532. In another example, the world manager module 510 may use the hardware host metrics 530 to determine whether the utilization of hardware or software resources affects the number of processing partitions in the processing partitions 532 or the number of data objects 524 supported by one of the processing partitions 532.

[0114] After determining the processing partitions 532 using the data objects 524, the world manager module 510 may determine one or more processing partition assignments 534. The (one or more) processing partition assignments 534 may include a data structure that acts as an index to assign the (one or more) processing partitions 532 to the (one or more) hardware hosts 504. The processing partition assignments 534 may include metadata that identifies at least one of the one or more processing partitions and the hardware hosts 504 assigned to the processing partition. The world manager module 510 may use the world configuration 522 and the application 526 to determine the processing partition assignments 534. For example, the world manager module 510 may use the number of assigned hardware hosts defined in the world configuration 522 to determine the processing partition assignments 534. In another example, the world manager module 510 may use the number of supported objects or the subscription policy defined for the application 526 to determine the processing partition assignments 534.

[0115] The world manager module 510 may also use the hardware host configuration 528 and the hardware host metrics 530 to determine the processing partition assignments 534. For example, the world manager module 510 may use the capacity of the resources defined in the hardware host configuration 528 to determine the processing partition assignments 534. In another example, the world manager module 510 may use the hardware host metrics 530 to determine whether the utilization of hardware or software resources affects the processing partition assignments 534.

[0116] After determining the processing partition 532 using the data object 524, the world manager module 510 may send the processing partition 532 to the hardware host 504 to prepare the hardware host 504 to process the data object 524. For example, the world manager module 510 may send the processing partition assignment 534 to the hardware host 504. The first hardware host in the hardware host 504 may determine which of the processing partitions 532 are assigned to the first hardware host. The first hardware host may process one or more of the data objects 524 mapped to those processing partitions 532 assigned to the first hardware host.

[0117] The hardware host 504 may include a hardware host manager module 540, an application manager module 542, one or more application containers 544a-n, a distributed shared memory (DSM) module 546, a data repository 548, one or more processors 550, and one or more memory modules 552. The data repository 548 may contain one or more local data objects 560, an application library 562, one or more local hardware host metrics 564, one or more local processing partitions 566, and one or more local processing partition assignments 568.

[0118] The hardware host manager module 540 may be configured to manage the operation of the hardware host 504 to process the data object 524. The hardware host manager module 540 may store one or more data objects 524 as local data objects 560 in the file system data repository 548. The hardware host manager module 540 may also receive one or more applications 526 using the application library 562 for storage in the data repository 548. The hardware host manager module 540 may configure the application library 562 with all or a portion of the application(s) 526. Thus, the application library 562 may be used to provide on-demand access to one or more applications 526 on the hardware host 504 without additional configuration.

[0119] The hardware host manager module 540 may also monitor the utilization of hardware and software resources and store one or more local hardware host metrics 564 in the data repository 548. The hardware host manager module 540 may monitor the operation of the application manager module 542, the application container(s) 544a-n, and the distributed shared memory module 546 or the hardware host 504 as a whole. The hardware host manager module 540 may report the local hardware host metrics 564 back to the world manager module 510 for storage as hardware host metrics 530.

[0120] The hardware host manager module 540 may manage the storage of one or more local processing partitions 566. The hardware host manager module 540 may receive a processing partition 532 from the world manager module 510, and the hardware host manager module 540 may store a copy of the processing partition 532 in the data repository 548 using the local processing partition 566. In addition, the hardware host manager module 540 may report changes to the local processing partition 566 back to the world manager module 510, and the world manager module 510 may use the changes to update the processing partition 532 and the processing partition assignment 534.

[0121] The hardware host manager module 540 may manage the storage of one or more local processing partition assignments 568. The hardware host manager module 540 may receive one or more of the processing partition assignments 534 from the world manager module 510. The hardware host manager module 540 may store a copy of the processing partition assignment 534 in the data repository 548 using the local processing partition assignment 568, and the hardware host manager module 540 may use the copy to determine the assigned processing partitions. Further, the hardware host manager module 540 may also maintain a copy of the processing partition assignment 534 for other hardware hosts 504 to facilitate communication and data routing with the other hardware hosts 504.

[0122] According to an example of the present technology, the hardware host manager module 540 may analyze one or more local processing partition assignments 568 to determine the assigned processing partitions in one or more local processing partitions 566. The hardware host manager module 540 may request one or more of the processing partitions 532 from the world manager module 510, or the hardware host manager module 540 may receive the processing partitions 532 together with the processing partition assignments 534 from the world manager module 510.

[0123] The application manager module 542 may be configured to manage the life cycle of applications on the hardware host 504 using the application library 562. For example, the application manager module 542 may read one or more local processing partitions 566 to determine which of one or more applications 526 are used during the processing of one or more local data objects 560 mapped to the one or more local processing partitions 566. The application manager module 542 may retrieve one or more of the applications 526 from the application library 562 and instantiate the applications according to the object type to process the local data objects 560. The application manager module 542 may load the application instances into the application containers 544a-n and track the processing of the local data objects 560 by the application instances.

[0124] The distributed shared memory module 546 can manage data objects 524 using the shared memory in the memory device. The distributed shared memory module 546 can read one or more local processing partitions 566 to identify one or more local data objects 560. For example, the metadata referencing one or more local data objects 560 can be in one or more local processing partitions 566 and can be used to obtain and locally store one or more data objects 524 at the hardware host 504. The distributed shared memory module 546 can then load the local data objects 560 into the shared memory as shared memory data objects 570 for use by application instances executing in the application containers 544a-n when needed.

[0125] The distributed shared memory module 546 can also process requests from application instances executing in the application containers 544a-n to access the shared memory data objects 570. The requests to access the shared memory data objects 570 can include create, read, update, and delete operations created by application instances executing in the application containers 544a-n. Since the shared memory data objects 570 are processed by application instances executing in the application containers 544a-n, the distributed shared memory module 546 can use a thread-safe and low-copy-cost storage process and a serialization format understood by the distributed shared memory module 546 to record state changes to the shared memory data objects 570, as will be discussed in a later section.

[0126] The distributed shared memory module 546 can further replicate state changes made to the shared memory data objects 570 to other application instances and across other hardware hosts in the hardware host 504. The distributed shared memory module 546 can determine a subscriber list that identifies application instances on the hardware host or other hardware hosts registered to receive a data object event stream associated with the shared memory data objects 570. When the shared memory data objects 570 are processed by application instances executing in the application containers 544a-n, the distributed shared memory module 546 can send state changes to the shared memory data objects 570 to the subscriber list. The distributed shared memory module 546 can also subscribe to data object event streams associated with distributed shared memory modules on other hardware hosts. The distributed shared memory module 546 can receive state changes from other hardware hosts to reflect in the shared memory data objects 570.

[0127] According to an example operation of the present technology, when the shared memory data object 570 is processed by application instances executing in application containers 544a-n, the distributed shared memory module 546 may determine changes to the local processing partition 566. The distributed shared memory module 546 may change the mapping between the local data object 560 and the local processing partition 566, for example, when a data object is created (i.e., resulting in the addition of a mapping between the data object and one of the local processing partitions 566) and when a data object is deleted or removed from the virtual environment (i.e., resulting in the removal of a mapping between the data object and one of the local processing partitions 566). The distributed shared memory module 546 may change the mapping between the local data object 560 and the local processing partition 566, for example, due to the shared memory data object 570 being processed by application instances executing in application containers 544a-n, resulting in changes to the object type, spatial location, state, and other object attributes used to group the data object 524 into the processing partition 532. The distributed shared memory module 546 may use the change in the mapping between the local data object 560 and the local processing partition 566 to update the local processing partition 566. The distributed shared memory module 546 may send any updates to the local processing partition 566 to the world manager module 510.

[0128] The world manager module 510 may receive updates from the hardware host 504 that reflect changes to the mapping between one or more of the data objects 524 and the processing partition 532. For example, one of the (one or more) hardware hosts 504 may create a data object and add the data object to one of the (one or more) local processing partitions 566. The world manager module 510 may obtain the (one or more) local processing partitions 566 from the (one or more) hardware hosts for comparison with the (one or more) processing partitions 532 to detect changes. In another example, one of the (one or more) hardware hosts 504 may perform an operation to update or delete a data object, which results in a change to the mapping between the (one or more) data objects 524 and the (one or more) processing partitions 532. In yet another instance, one of the hardware hosts 504 may split or merge one or more of the local processing partitions 566. For example, a processing application may specify the ability of the processing application to process multiple data objects of a given object type. A threshold (e.g., 60%) may be established for the processing application, and once the threshold is approached, the data objects disposed of by a first instance of the processing application are split and redistributed between the first instance and a second instance of the processing application. When there is underutilization of hardware or software resources, the processing partitions may be merged. The world manager module 510 may use the updates from the hardware host to update the processing partition 532.

[0129] In response to an update to processing partition 532, world manager module 510 may also determine whether to update processing partition assignment 534. World manager module 510 may update the processing partition assignment(s) 534 to load balance the processing of data object(s) 524 among the hardware host(s) 504. World manager module 510 may also update the processing partition assignment 534 to ensure that frequently used data can be aggregated or collocated to a hardware host. World manager module 510 may then send the update to the processing partition assignment 534 to hardware host 504. Thus, world manager module 510 may control and optimize the performance of hardware host 504 during the processing of data object 524.

[0130] According to another example operation of the present technology, world manager module 510 may also monitor metrics associated with hardware host 504 to manage the performance of hardware host 504. World manager module 510 may receive data associated with metrics from the hardware host(s) 504 and use the data to determine an initial configuration for the processing partition assignment 534 or to determine whether to update the processing partition assignment(s) 534. World manager module 510 may use the data associated with the metrics to determine whether to migrate a processing partition from a first hardware host in hardware host 504 to a second hardware host in hardware host 504. For example, during the processing of data object 524, the utilization of the hardware or software resources of the first hardware host may exceed a threshold and affect the performance of the first hardware host. World manager module 510 may determine to change the assignment of some of the processing partitions 534 to balance the processing load, control hardware or software utilization, reduce network bandwidth, and latency, etc. World manager module 510 may send the updated processing partition assignment 534 to hardware host 504. Again, world manager module 510 may manage the performance of hardware host 504 during the processing of data object 524 by modifying the allocation of processing partition 532.

[0131] The various processes and / or other functions included within service provider environment 500 may be executed on processors 518 and 550 that communicate with memory modules 520 and 552, respectively. Service provider environment 500 may include a plurality of computing devices arranged in, for example, one or more server groups or computer groups or other arrangements. The computing devices may use a hypervisor, a virtual machine monitor (VMM), and other virtualization software to support the computing environment.

[0132] The term "data repository" can refer to any device or combination of devices capable of storing, accessing, organizing, and / or retrieving data, which may include any combination and number of data servers, relational databases, object-oriented databases, clustered storage systems, data storage devices, data warehouses, flat files, and data storage configurations in any centralized, distributed, or clustered environment. The storage system components of data repositories 512, 514, 516, and 548 may include storage systems such as SAN (Storage Area Network), cloud storage network, volatile or non-volatile RAM, optical media, or hard drive type media. Data repositories 512, 514, 516, and 548 may represent multiple data repositories, as can be appreciated.

[0133] Figure 5 Certain processing modules may be discussed in connection with this technology and these processing modules may be implemented as computing services. In one example configuration, a module may be considered a service having one or more processes executed on a server or other computer hardware. Such a service may be a centrally hosted function or a service application that can receive requests and provide outputs to other services or consumer devices. For example, a module providing a service may be considered on-demand computing hosted in a server, virtualized service environment, grid, or clustered computing system. An API may be provided for each module such that a second module can send a request to the first module and receive an output from the first module. Such an API may also allow third parties to interface with the module and make requests and receive outputs from the module. Although Figure 5 An example of a system that can implement the above technology is shown, but many other similar or different environments are possible. The example environments discussed and shown above are merely representative and not restrictive.

[0134] Figures 6A - 6B is a flowchart showing an example method for using object domain assignment to process partitions according to an example of the present technology. Method 600 may be executed when software (e.g., instructions or code modules) is executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or information processing device), by a hardware component of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0135] In operation 602, the world manager may receive a plurality of data objects associated with an object-based model representing a multi-dimensional virtual environment hosted by a distributed computing system. In operation 604, the world manager may determine a plurality of object types associated with the plurality of data objects. Examples of object types may be that for object_1, the type is set to "vehicle" and object_2 may be of type "building". Thus, these two groups of objects may be separated by their types. In operation 606, the world manager may generate a mapping between the plurality of data objects and a plurality of processing partitions using the plurality of object types. As in the example, objects of type "vehicle" and objects of type "building" may be mapped to processing partition 1 and processing partition 2 respectively. The world manager may generate a data structure that indexes the plurality of data objects into the plurality of processing partitions using the plurality of object types. The data structure may include metadata about the processing partitions and the data objects grouped by object type, where the data objects are mapped to the processing partitions.

[0136] In operation 608, the world manager may identify a plurality of hardware hosts in the distributed computing system to process the object-based model using the plurality of data objects. The world manager may identify the hardware hosts from a pool of hardware hosts waiting to be assigned, active hardware hosts with idle computing capacity, etc. The active hardware hosts may execute application instances in containers or in computing instances that can operate on the processing partitions. This means that the processing partitions may be assigned for processing to a hardware host that has an application instance of an application that can process the type of data objects mapped to the processing partition. In one example, the first hardware host may have an application instance that can process "vehicle" type and a second application instance that can process "building" when a physical event (e.g., a collision or explosion occurs).

[0137] In operation 610, the world manager may generate a plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts. The world manager may produce a data structure that distributes the plurality of processing partition assignments to the plurality of hardware hosts. The data structure may include metadata about the hardware hosts and the processing partitions assigned for processing to the hardware hosts. In operation 612, the world manager may send the plurality of processing partition assignments to the plurality of hardware hosts to organize the plurality of hardware hosts to process the plurality of data objects using the plurality of object domains. Thus, the data objects may be processed based on the type of the objects.

[0138] Method 600 continues to use the reference number "A" from Figure 6A to Figure 6BAbove. In operation 614, the world manager can monitor the processing of multiple data objects mapped to multiple processing partitions by multiple hardware hosts. The world manager can receive data associated with the utilization of hardware and software resources, changes to data objects, and changes to processing partitions. In operation 616, the world manager can receive a change to the processing of multiple data objects mapped to multiple processing partitions. The change can include an update to a processing partition, which can include mapping a newly created data object to the processing partition, modifying a data object mapped to the processing partition, or removing the mapping between a data object and the processing partition. For example, the number of data objects mapped to a processing partition may increase for other reasons (e.g., a player enters an area), and load balancing may need to occur by amending the assignment. In another example, the object type can change from "building" to "exploded_building", which can affect the processing partition assignment of the data object. The change to the processing of the multiple processing partitions can also include a change attributable to overutilization of hardware and software resources on a hardware host.

[0139] In operation 618, the world manager can use the change to determine whether to update the multiple processing partition assignments. The world manager can determine to update the multiple processing partitions to manage or optimize the performance of the hardware hosts. The world manager can, for example, perform load balancing on the processing of data objects across multiple hardware hosts in response to underutilization or overutilization of hardware or software resources. The world manager can also redistribute processing partitions attributable to changes to data objects or processing partitions. If the world manager uses the change in step 620 to determine not to update the multiple processing partition assignments, method 600 continues in operation 614, where the world manager returns to monitoring the multiple hardware hosts.

[0140] If the world manager uses the change in step 620 to determine to update the multiple processing partition assignments, method 600 continues in operation 622, where the world manager updates the multiple processing partition assignments using the change. The world manager can update the multiple processing partition assignments, for example, to migrate a processing partition assignment from a first hardware host to a second hardware host due to exceeding a threshold of CPU usage associated with the first hardware host. The world manager can migrate a processing partition from a first hardware host to a second hardware host to co-locate data that the second hardware host uses more frequently. The world manager can also create a new processing partition assignment for a hardware host being transferred from a warm pool to an active pool. Method 600 continues using reference "B" from Figure 6B and returns Figure 6A , where in operation 614, the world manager can send the multiple processing partition assignments (including any updates) to the multiple hardware hosts.

[0141] Figure 7 It is a flowchart showing an example method 700 for processing data objects grouped into processing partitions according to object domains. Method 700 can be executed when software (e.g., instructions or code modules) is executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or an information processing device), by a hardware component of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0142] In operation 702, the hardware host may receive a plurality of processing partition assignments, where the processing partition assignments are assignments between a plurality of processing partitions and a plurality of hardware hosts. The hardware host may receive a plurality of processing partition assignments from a world manager. In operation 704, the hardware host may use the processing partition assignments to identify the processing partitions assigned to the hardware host. In one configuration, a single processing partition may be assigned to the hardware host. In another configuration, a number of processing partitions may be assigned to the hardware host (e.g., dependent, independent, related, or unrelated processing partitions).

[0143] In operation 706, the hardware host may use a distributed shared memory to load a plurality of data objects mapped by a first object type into a processing partition in a memory device associated with the hardware host. In operation 708, the hardware host may determine data dependencies used by a processing application associated with the processing partition. The data dependencies may indicate additional object types on which the data objects mapped to the processing partition used by the processing application depend. The data dependencies may indicate data object event streams associated with the additional object types that the processing application may subscribe to receive. The data dependencies may further indicate a subscription strategy for filtering the sources from which the additional object types are obtained. An application instance on the hardware host may communicate with the distributed shared memory using a permission model for the object types of the data objects currently being processed. Read-only permissions in the permission model may be used to determine data dependencies.

[0144] In operation 710, the hardware host using the distributed shared memory may subscribe to additional data objects of a second object type using the data dependencies. The subscription may be used to identify and filter sources of data object event streams associated with the additional data objects of the second object type. The distributed shared memories of a plurality of hardware hosts may be sources for data object event streams associated with the additional data objects of the second object type. A plurality of distributed shared memories may communicate with each other to exchange data objects or exchange updates to data objects as part of the subscription strategy identified for the data dependencies.

[0145] In operation 712, a hardware host may receive a list of subscribers having data dependencies on multiple data objects grouped into a processing partition by a first object type. The subscriber list may be used to identify and filter destinations for a stream of data object events associated with the multiple data objects of the first object type. Processing applications on the same hardware host and on distributed shared memories of multiple hardware hosts may be destinations for the stream of data object events associated with the multiple data objects of the first object type. As discussed above, application instances on the hardware host may communicate with the distributed shared memory of the data objects currently being used for processing. Additionally, multiple distributed shared memories may communicate with each other to exchange data objects or exchanges updates of data objects between subscriber lists.

[0146] In operation 714, a hardware host may use a processing application to process multiple data objects grouped into a processing application using an added data object by a first object type. An application instance may read data associated with the data object from the distributed shared memory. The application instance may perform operations or calculations on the data object, some of which may result in changes to the data object. In operation 716, the hardware host may use the distributed shared memory to manage storage of the multiple data objects in a memory device using multiple requests from the processing application. The multiple requests may include requests to modify data associated with the multiple data objects mapped to the processing partition. An application instance may submit a request to modify data associated with a data object to the distributed shared memory. The distributed shared memory may process the request to record changes to the data object caused or implemented by the request.

[0147] In operation 718, the hardware host may use the distributed shared memory to send multiple changes to the multiple data objects caused by the multiple requests to the subscriber list. As discussed above, application instances on the hardware host may communicate with the distributed shared memory of the data objects currently being used for processing. Additionally, multiple distributed shared memories may communicate with each other to exchange data objects or exchanges updates of data objects between subscriber lists.

[0148] Figure 8 A system 800 and related operations for storing and replicating states of data objects in a distributed computing system according to an example of the present technology are shown. The system 800 may include a distributed shared memory 802. Some examples of the distributed shared memory 802 may include the distributed shared memory module 274 described with respect to Figure 2 and with respect to Figure 5The described distributed shared memory module 546. The distributed shared memory 802 may manage the storage of data objects 404 for the system 800. The distributed shared memory 802 may use stored procedures to manage the storage of data objects 404 to create or load data objects 404 into one or more memory devices using an in-memory representation of the data objects 404. The distributed shared memory 802 may also use stored procedures to manage the storage of data objects 404 to process requests for reads and writes to the in-memory representation of the data objects 404.

[0149] According to the present technology, the distributed shared memory 802 may use stored procedures that include a serialization format for the data objects 404. The distributed shared memory 802 may use stored procedures to represent data objects as hierarchical data in-memory and on disk using multiple segments. The distributed shared memory 802 may further utilize logging in one of the multiple segments to represent modifications to the data objects 404 to minimize the time cost of reflecting state changes in highly distributed real-time computing.

[0150] For example, the distributed shared memory 802 may use stored procedures and implement a serialization format for the data objects 404 to minimize the time cost of reflecting state changes in epoch-based simulations, where each epoch provides an execution interval. The epoch-based simulation may use a scenario with 1 writer / N reader processes for the data objects 404 and each process potentially containing multiple threads, where the epoch or execution interval of the process may be approximately 30 ms. A typical RPC (Remote Procedure Call) framework (such as grpc / protobuf) starts to lag as the epoch frequency increases, and the size of the messages for reflecting state changes increases. This may be due to the CPU time spent deserializing the data objects and the cost of synchronizing the entire data object when the data object is updated. The distributed shared memory 802 may use stored procedures according to the present technology to meet the performance requirements because the stored procedures provide platform-independent and language-independent data format specifications for use with the data objects 404.

[0151] The distributed shared memory 802 can provide small, predictable overheads because the code written to use stored procedures with a serialized format should execute CRUD (Create, Read, Update, Delete) operations as close as possible to native stored procedures. The distributed shared memory 802 can include the overhead for using stored procedures with a serialized format, which can maintain an amortized O(1) cost. By keeping the memory layout the same as the storage / wire format, the distributed shared memory 802 also does not incur the serialization cost of using stored procedures. Additionally, the distributed shared memory 802 can use stored procedures to implement updates to the data object 404 in the form of a log. The distributed shared memory 802 can group one or more modifications to the data object 404 as a log entry within a round or execution interval.

[0152] The distributed shared memory 802 can provide a mechanism for processing applications and other distributed shared memories to retrieve and consume the log entries of the data object 404. Thus, the distributed shared memory 802 can be highly concurrent because there is a separation between how the stored procedure implements the storage of the data object and the storage of the metadata used to track changes to the state of the data object 404. Finally, the distributed shared memory 802 is efficient because the stored procedure can use binary encoding to represent the data of the data object, and thus, reading data fields from the encoding simply uses the binary encoding pattern.

[0153] In this example, the distributed shared memory 802 can retrieve data objects 404 from one or more sources (not shown), such as files, object data repositories, another distributed shared memory, etc. As discussed above, the data objects 404 can be associated with an object-based model of a virtual environment (e.g., 3D simulation, 3D game, etc.). At the beginning of the life cycle of the data object 404 in the virtual environment, the distributed shared memory 802 can use the storage process according to this technology to manage the storage of the data object 404 in the memory device during object creation and loading. For example, the distributed shared memory 802 can generate one or more object representations 804 for the data object 404 in the memory device. When keeping the memory layout the same as the storage / wire format, the distributed shared memory 802 can directly copy the data object 404 to the memory device. Alternatively, the distributed shared memory 802 can parse the data object 404 to write the data object to the memory device using the storage process. The object representation 804 can include one or more sections in the memory device. Some examples of sections can include a metadata section, a virtual table section, an object data section, a log section, and so on. For example, at creation, the (one or more) object representations 804 can include an object metadata section 806, a virtual table 808, and an object data section 810. At other times during the life cycle of the data object 404, the distributed shared memory 802 can use the storage process to add one or more sections and parts of sections to the object representation 804.

[0154] The distributed shared memory 802 can use the object metadata section 806 to describe the in-memory / disk / on-wire content of the data object 404, and how to read the object representation 804. The object metadata section 806 can include one or more metadata fields. The object metadata section 806 can include a length field indicating the length of the object metadata section 806 (e.g., in bytes) and an object length field indicating the total combined length of the object metadata section 806, the virtual table 808, and the object data section 810. The object metadata section 806 can also include: an object identifier (ID) 812 field that provides a unique identifier for identifying the data object; and a base version field that indicates the version of the data object represented by the object data section 810. The object metadata section 806 can include: a virtual table offset field that provides a memory offset (e.g., from the start of the object metadata section 806) to the start of the virtual table 808, and an object data offset field that provides a memory offset (from the start of the object metadata section 806) to the start of the object data section 810.

[0155] The distributed shared memory 802 can use a virtual table 808 to describe one or more portions of an object data segment 810. The virtual table 808 can include one or more table fields and table entries. For example, the virtual table 808 can contain an offset array that points to portions of the object data segment 810 starting from the beginning of the object data segment 810. In one example, a desired memory location in the object data segment 810 can be found by using the offset found in the corresponding table entry of the virtual table 808. In another example, the Nth data field of a data object can be found by using the offset found in the Nth index of the virtual table 808. The virtual table 808 can include a length field that indicates the length of the virtual table 808 (e.g., in bytes), and one or more table entries or field offsets that provide an array of table entry or field offsets representing each of the data entries or data fields in the object data segment 810. The distributed shared memory 802 can use the object data segment 810 to contain data entries, data fields, data values, etc. of the data object 404. The object data segment 810 can contain a length field that provides the length of the object data segment 810 (e.g., in bytes) and a byte array representing the data values in the data object.

[0156] According to another example of the present technology, during the life cycle of a data object 404 in a virtual environment, the distributed shared memory 802 can manage the storage of state changes of the data object 404. For example, the distributed shared memory 802 can provide a mechanism for processing applications and other distributed shared memories to retrieve and process the data object 404. The distributed shared memory 802 can receive one or more requests 820. The requests 820 can be issued by one or more processing applications. The requests 820 can instruct the distributed shared memory 802 to create or delete an object representation 804 in a memory device, read the object representation 804 from the memory device, and write changes to the data object 404 represented by the object representation 804.

[0157] In one example, the distributed shared memory 802 can analyze the request 820 to identify read requests associated with the data object 404 that will be processed using the object representation 804 in the memory device. The distributed shared memory 802 can read the object metadata section 806 of the (one or more) object representations 804 to determine whether the (one or more) requests 820 match, for example, a specific object ID 812, an object version 814, and an object type 816. When this initial verification check is satisfied, the distributed shared memory 802 can obtain the memory location of the virtual table 808 from the object metadata section 806. The distributed shared memory 802 can read the virtual table 808 to determine the field offset in the object data section 810 to obtain the in-memory content of the data object 404. The distributed shared memory 802 can use the field offset to read the object data section 810 to obtain the in-memory content of the data object 404 from the object representation 804. The distributed shared memory 802 can include the in-memory content of the (one or more) data objects 404 obtained from the (one or more) object representations 804 and can send a response to one or more requesters associated with the (one or more) requests 820.

[0158] In another example, the distributed shared memory 802 can analyze the request 820 to identify write requests associated with the data object 404 and the data to be written to the data object 404 using the object representation 804 in the memory device. The distributed shared memory 802 can again read the object metadata section 806 of the (one or more) object representations 804 to determine whether the (one or more) requests 820 match, for example, a specific object ID and an object version. When this initial verification check is satisfied, the distributed shared memory 802 can append a version-ordered change set to the object representation 804 in the memory device.

[0159] The distributed shared memory 802 can use a series of log records in the log section to reflect the state change of the object representation 804 in the memory device. As Figure 8As shown, the (one or more) object representations 804 may also include a log segment 830. The distributed shared memory 802 may use the log segment 830 to represent state changes of the data object 404 represented in the object representation 804. The distributed shared memory 802 may access the current state of the data object 404 by reading the log segment 830 from the tail and reading backward to obtain the latest information of the data object 404 contained in the object representation 804 in the memory device. The distributed shared memory 802 may manage the current state of the (one or more) data objects 404 by appending one or more log records 832 (e.g., log records 832a - n) to the (one or more) object representations 804 to reflect version - ordered changes or write sets. The distributed shared memory 802 may use the log records 832 to describe the change set for the data object 404 and to which version of the data object the changes apply. The distributed shared memory 802 may append the log records 832 to the object representation 804 in reverse order to facilitate a tail - to - head read mechanism and single - pass writes and thread - safe reads.

[0160] Figure 9 A graphical example of an object representation 900 used by a distributed shared memory for managing storage of data objects in a distributed computing system according to an example of the present technology is shown, the distributed computing system providing a low copy cost and thread - safe reads. The distributed shared memory may implement the object representation 900 using a set of semantics built on top of a byte array. The object representation 900 may provide a low copy cost because the byte array can be copied "as is" to another memory location, to a file on disk, across a network to another hardware host, etc., because the serialization format of the object representation 900 reduces the additional processing of formatting data for different storage or transmission media.

[0161] The distributed shared memory may split the object representation 900 into one or more segments in memory, on the wire, and on disk as files. The object representation 900 may include a metadata segment 902, a virtual table segment 904, an object data segment 906, and a log segment 908. The object representation 900 may provide thread - safe reads because the organization of the segments in the object representation 900 prevents reads from segments that may be written to simultaneously. The object representation 900 may include a resizable byte array (RBArray) that provides an abstraction around the resizable byte array that allows atomic "reservation" of bytes. The object representation 900 may enable a request to resize the array to allocate additional bytes as an atomic process before another pending request to service the resize of the array. Thus, the object representation 900 enables atomic reservation to allow a queue of additional writes to the object representation 900 to be thread - safe.

[0162] A distributed shared memory can use the metadata section 902 of the object representation 900 to describe a byte array and provide information on how to read the byte array. The metadata section 902 can include one or more metadata fields, such as a metadata length 910, an object length 912, an object identifier 914, a base version 916, a virtual table offset 918, and an object data offset 920. The metadata length 910 can specify the length of the metadata section 902 in bytes. The object length 912 can specify the total length of the byte array without the log section 908. The object identifier 914 can specify a unique identifier used to identify this object. Some examples of the object identifier 914 can include UUID or GUID.

[0163] The base version 916 can specify the version of the data object represented by the object data section 906. The virtual table offset 918 can specify the offset (from the start of the byte array) to the start of the virtual table section 904. The object data offset 920 can specify the offset (from the start of the byte array) to the start of the object data section 906.

[0164] A distributed shared memory can use the virtual table section 904 of the object representation 900 to describe the offsets into specific parts of the object data section 906. The virtual table section 904 can contain a virtual table length 930, which specifies the length of the virtual table 932 in bytes. The virtual table 932 can contain an array of offsets starting from the start of the object data section 906. The Nth field of the data object represented by the byte array can be found by using the offset found in the Nth index of the array in the virtual table 932.

[0165] A distributed shared memory can use the object data section 906 of the object representation 900 to store a data object. The object data section 906 can contain an object data length 940, which specifies the length of the object data 942 in bytes. The data associated with the data object can be retrieved from a file, an object repository, etc. and copied into the object data 942. The object data 942 can contain a byte array representing the data object. The object data 942 can contain: integers, floating points, characters, strings, binary large objects (blobs), etc. and combinations thereof. As described above, the object identifier 914 identifies the data object represented in the object data section 906. The base version 916 specifies the version of the data object represented in the object data section 906.

[0166] A distributed shared memory can use a log segment 908 of an object representation 900 to represent state changes of a data object. The distributed shared memory can use the log segment 908 to write a version-ordered change set to a data object whose content is represented in an object data segment 906. The log segment 908 can include a series of one or more log records 950 (e.g., log records 950a-d). The distributed shared memory can read the log segment 908 from the tail and read backward to obtain the current state of the data object. The distributed shared memory can use the log records 950 to describe a set of changes to the data object. The distributed shared memory can write the log records 950 to the log segment 908 by appending the log records 950 in reverse order to facilitate a tail-to-head reading mechanism, single-pass writing, and thread-safe reading.

[0167] In this example, the log record 950d can include one or more log fields, such as a length 960 that specifies the length of the log record 950d in bytes. The log record 950d can include a version field 962. The distributed shared memory can use the version field 962 to specify the version of the data object to which the log record 950d is applied. The distributed shared memory can also use the version field 962 to specify the version of a part of a data record or a specific data field to which the log record 950d is applied. The log record 950d can also include an object identifier 964 and a modified field table 966, and the object identifier 964 provides the object identifier of the data object to which the log record 950d is applied.

[0168] According to an example of the present technology, the distributed shared memory can use the modified field table 966 to indicate where the state of the data object has changed. The modified field table 966 can include references indicating that a part of the log record 950d includes changes to data associated with the data object, and the data can be different from the data currently associated with the data object and stored in a part of the object data segment 906. For example, the distributed shared memory can use the modified field table 966 to include references indicating which object fields of the data object have been changed. The modified field table 966 can include a length 972 (in bytes) for describing the size of the modified field table 966, and a bit field content 974 for representing the actual modified fields using an index. In this example, the distributed shared memory can set one or more bits of the bit field content 974 to indicate that an object field has changed. Each bit in the byte can correspond to an ordered set of object fields. Setting the bit corresponding to the object field in the bit field content 974 can provide a reference indicating a change to the object field.

[0169] According to one example of the present technology, a distributed shared memory may use a translation table 968 to indicate where the state of a data object has changed. The log record 950d may also include the translation table 968 and table contents 978, where the translation table 968 provides an internal virtual table containing a length 976 (in bytes) to describe the size. The distributed shared memory may use the table contents 978 as a mapping between an offset pointing to a modified data segment 970 and a changed object field that can also be identified in the modified field table 966. The distributed shared memory may use the modified data segment 970 to store changes to the data object. The modified data segment 970 may include a byte array representing changes to the object fields in the log record 950d. According to one example of the present technology, the distributed shared memory may append the log records 950a-d one after another or continuously in memory. In another example, the distributed shared memory may include a previous record pointer or another type of linked list scheme in the log records 950a-d. The distributed shared memory may use the previous record pointer to provide gap detection to detect incomplete logs.

[0170] According to another example of the present technology, during a round or execution interval in a distributed computing system, the distributed shared memory may aggregate multiple requests to modify data associated with a data object. The distributed shared memory may use the translation table 968 and / or the modified field table 966 to include references to multiple parts of the data object affected by the multiple requests. The distributed shared memory may append a single log record for the round to the log segment 908 of the object representation 900. The single log record may include multiple changes to the data object implemented by the multiple requests, where these changes are referenced as changes in the translation table 968 and / or the modified field table 966 that affect multiple parts of the data object.

[0171] According to yet another example of the present technology, when requests are applied to the same part of a data object, during a round or execution interval in a distributed computing system, the distributed shared memory may aggregate multiple requests to modify data associated with the data object. The distributed shared memory may append a single log record for the round to the log segment 908 of the object representation 900. The single log record may include the latest series of changes to the data object implemented by the multiple requests, which are referenced as a single change in the translation table 968 and / or the modified field table 966 that affects a part of the data object.

[0172] According to another example of the present technology, the distributed shared memory may append multiple log records to the log segment 908 of the object representation 900 during a round or execution interval in the distributed computing system. The distributed shared memory may determine to merge the multiple log records into a single log record (e.g., at the end of a round, when a read is made, or at another point in time) in order to save storage space or improve the later read time to determine the current state of the data object. The distributed shared memory may write the single log record to the log segment 908 to replace the multiple log records. The single log record may include multiple references that indicate that the multiple changes to the data object aggregated from the multiple log records reflect changes to multiple parts of the data object.

[0173] According to an additional example of the present technology, the distributed shared memory may use the object representation 900 to convey state changes associated with the data object. The distributed shared memory may receive an instruction to send the current state of the data object to a recipient (such as a processing application, another distributed shared memory, an object data repository, etc.). The distributed shared memory may determine whether the recipient includes a previous state of the data object. For example, the distributed shared memory may track whether all or part of the object representation 900 has been previously sent to the recipient. When the distributed shared memory determines that the recipient includes a previous state of the data object, the distributed shared memory may send one or more log records 950 (e.g., all log records) in the log segment 950 to the recipient. The distributed shared memory may send one or more delta log records representing the difference between the current state of the data object and the previous state maintained at the recipient to the recipient. The distributed shared memory may determine that the recipient does not include a previous state of the data object and the second object representation 900 or the entire log segment 908. The distributed shared memory may merge the log segment 908 into a single log record as described above to represent the current state based on storage or network conditions. For example, the distributed shared memory may merge the log segment 908 into a single log record to reduce the storage used by the object representation 900. In another example, the distributed shared memory may merge the log segment 908 into a single log record to reduce the network bandwidth used for data transmission.

[0174] Figure 10 is a flowchart showing an example method 1000 performed by a distributed shared memory for managing the storage of data objects in a distributed computing system that provides low copy costs and thread-safe reads. The method 1000 may be performed when software (e.g., instructions or code modules) is executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or information processing device), by a hardware component of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0175] In operation 1002, the distributed shared memory of the hardware host may receive data objects associated with an object-based model of a virtual environment hosted by the distributed computing system. The distributed shared memory of the hardware host may retrieve data associated with the data objects from an object data repository. In another example, the distributed shared memory may receive data associated with the data objects from another hardware host, a world manager, a data service, or a client. In operation 1004, the distributed shared memory may determine the memory location of the data objects in a memory device associated with the hardware host. The memory device may include volatile and non-volatile storage devices. The distributed shared memory may reserve a portion of the memory in the memory device to manage a shared storage space for the representation of the data objects. The distributed shared memory may allocate memory at the memory location of the data objects and share the memory with a processing application on the hardware host and with other distributed shared memories on other hardware hosts in the distributed computing system. In this example, the distributed shared memory may share a portion of the random access memory as the shared storage space.

[0176] In operation 1006, the distributed shared memory may determine the format of the data objects. The format of the data objects may include an on-disk format and an in-memory format. The on-disk format may be the same as or different from the in-memory format. The distributed shared memory may use the same format for file storage and memory storage to reduce the need to perform serialization before storage or transmission over a network. If the distributed shared memory determines in operation 1008 that the format of the data objects is used by the distributed shared memory, the distributed shared memory may write the data objects to the memory location in the memory device in operation 1010. For example, the distributed shared memory may directly copy the data associated with the data objects to the memory device regardless of whether the data is obtained from an object data repository, a file, a network packet, etc. The distributed shared memory may perform a memory copy (memcopy) to allocate the data objects to the shared memory space.

[0177] If the distributed shared memory determines in operation 1008 that the format of the data objects is not used by the distributed shared memory, the distributed shared memory may identify metadata associated with the data objects, as in operation 1012. For example, the distributed shared memory may parse the data associated with the data objects to determine an object identifier that uniquely identifies the data objects and an object version of the data associated with the data objects. In operation 1014, the hardware host may write the metadata to an in-memory representation of the data objects at the memory location using a metadata section.

[0178] In operation 1016, the distributed shared memory can identify the object data associated with the data object. The distributed shared memory can parse the data associated with the data object to determine digital data, strings, binary blobs, etc. The distributed shared memory can apply multiple data transformation rules to the data. In one example, the distributed shared memory can determine the pattern of the data object, which identifies one or more object fields and the format of the data contained in the object fields. The distributed shared memory can apply data transformation rules to the data contained in the object fields to transform and prepare the data for storage in the memory device. In operation 1018, the distributed shared memory can use the object data section to write the object data to the representation of the data object at the memory location. Continuing with the previous example, the distributed shared memory can use the object data section to store the data contained in the object fields associated with the data object in a byte array.

[0179] In operation 1020, the distributed shared memory can identify one or more object fields associated with the data object. In one example, the distributed shared memory can identify one or more parts of the data associated with the data object to determine one or more object fields, and parse the object data into the values of the object fields. In another example, the distributed shared memory can detect the pattern of the data object and use the pattern to identify the object fields and structure of the data object. In operation 1022, the distributed shared memory can use a virtual table to write the mapping between the object fields and the parts of the object data section that contain the values of the object fields. The distributed shared memory can use the virtual table to describe an offset array starting from the beginning of the object data section. The distributed shared memory can access the memory location in the object data section of the Nth field of the data object by using the offset found at the Nth index in the virtual table.

[0180] Figure 11 is a flowchart showing an example method 1100 executed by a distributed shared memory according to an example of the present technology. The example method is for managing the storage of the state of data objects in a distributed computing system using an ordered, append-only log-based format to provide a versioned state snapshot. Method 1100 can be executed when software (e.g., instructions or code modules) is executed by the central processing unit (CPU or processor) of a logical machine (e.g., a computer system or an information processing device), by the hardware components of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0181] In operation 1102, the distributed shared memory of the hardware host may receive a request to modify data associated with a data object. The distributed shared memory may receive the request from a processing application configured to process data objects on the hardware host according to the object type of the data object. The distributed shared memory may receive the request in response to the processing application issuing the request via an API for accessing the data object, where the storage of the data object is managed by the distributed shared memory.

[0182] In operation 1104, the distributed shared memory may determine whether the request to modify data associated with a data object matches the object identifier and version associated with the representation of the data object whose storage is managed by the distributed shared memory. The distributed shared memory may analyze the request to determine whether the request specifically identifies the data object. The distributed shared memory may further analyze the request to determine whether the request is for a version of the data object whose storage is being managed by the distributed shared memory.

[0183] If the distributed shared memory determines in operation 1106 that the request does not match the object identifier and version associated with the representation of the data object, the distributed shared memory may use the request to generate an error message in operation 1108. The distributed shared memory may include in the error message details associated with the object identifier and version referenced by the request. The error message may also include data, memory content, stack, heap, and other debugging information associated with the processing application.

[0184] If the distributed shared memory determines in operation 1106 that the request matches the object identifier and version associated with the representation of the data object, the distributed shared memory may identify the changes to the data object in the request, as in operation 1110. The changes may include any change to the data object, such as a change in a field identifier, a change in a field value, or any other write to the data object. The distributed shared memory may analyze the request to identify the changes. The distributed shared memory may interpret the request according to an API for modifying the data object, where the storage of the data object is managed by the distributed shared memory. In one example, the request may indicate to modify an object field with a supplied value. The distributed shared memory may determine the object field and the supplied value from the request.

[0185] In operation 1112, the distributed shared memory may use logging to write the changes to the memory location associated with the representation of the data object. For example, the distributed shared memory may write a log record to the log section of the representation of the data object, where the storage of the data object is managed by the distributed shared memory. The distributed shared memory may append the log record to a series of log records in the log section.

[0186] In operation 1114, the distributed shared memory can identify the object fields associated with the change. The distributed shared memory can analyze the request to identify the object fields in the request. Using the previous example, the request can indicate one or more object fields to be modified and the new corresponding values of the object fields, and the distributed shared memory can track which object fields are being modified. In operation 1116, the distributed shared memory can write a conversion table to the log record, which maps the object fields to the portion of the log record that includes the change.

[0187] In operation 1118, the distributed shared memory can generate a reference indicating that the change reflects the change to the object field. The reference can include: bits, bytes, flags, resource identifiers, or other data that identifies the object field. In operation 1120, the distributed shared memory can write a modified field table including the reference to the log record. As discussed above, the distributed shared memory can use the modified field table to indicate which parts of the data object have a state change. For example, the hardware host can set one or more bits of a byte or a group of bytes to indicate that the object field corresponding to the bit has changed. Each bit in the byte can be mapped to or associated with an object field in an ordered set of object fields. Setting the bit of the corresponding object field can provide a reference indicating that the change reflects the change to the object field. The distributed shared memory can write the byte or the group of bytes with one or more bits set as the modified field table to the log record.

[0188] According to an example of the present technology, the distributed shared memory can execute method 1100 using multiple requests that modify the data associated with the data object during a round or an execution interval in the distributed computing system, to aggregate or merge the requests into a single log record. The distributed shared memory can use the modified field table to include references to multiple parts of the data object affected by the multiple requests. The distributed shared memory can append the single log record of the round to the log segment where it stores the representation of the data object managed by the distributed shared memory. According to another example of the present technology, when the requests are applied to the same part of the data object, the distributed shared memory can aggregate multiple requests that modify the data associated with the data object during a round or an execution interval in the distributed computing system. The distributed shared memory can append the single log record of the round to the log segment where it stores the representation of the data object managed by the distributed shared memory. The single log record can include the latest series of changes to the data object implemented by the multiple requests, which are referred to as a single change in the modified field table.

[0189] Figure 12FIG. is a flowchart showing an example method 1200 for exchanging the state of a data object in a distributed computing system between hardware hosts by a distributed shared memory according to an example of the present technology. The method 1200 may be performed by software (e.g., instructions or code modules) when executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or an information processing device), by a hardware component of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0190] In operation 1202, a first hardware host that stores a representation of a data object using a distributed shared memory may receive an instruction to send the current state of the data object to a second hardware host. The first hardware host may receive the instruction as a data request from the second hardware host. In another example, the first hardware host may receive the instruction in the form of a subscriber list. The first hardware host may use the subscriber list to publish changes made to the data object. The subscriber list may include the first hardware host, the second hardware host, and processing applications on other hardware hosts and services in the distributed computing system. The first hardware host may publish the changes directly to the subscribers, or the first hardware host may notify the subscribers that the changes are available. The subscribers may then retrieve or request the changes from the first hardware host.

[0191] In operation 1204, the first hardware host may determine whether the second hardware host includes a previous state of the data object. For example, the first hardware host may use the distributed shared memory to determine whether data associated with the representation of the data object has been previously sent to or obtained by the second hardware host. The first hardware host may communicate with the second hardware host to determine whether the second hardware host includes a previous state of the data object. The first hardware host may analyze the instruction for an indication of whether the second hardware host includes a previous state of the data object. In another example, the first hardware host may use the distributed shared memory to track communications sent to the second hardware host indicating whether the second hardware host contains a previous state of the data object.

[0192] If the first hardware host determines in operation 1206 that the second hardware host does not include a previous state of the data object, the first hardware host may use the distributed shared memory in operation 1208 to send the representation of the data object to the second hardware host. The distributed shared memory of the first hardware host may initiate a memory copy of the in-memory representation of the data object to the distributed shared memory of the second hardware host. The distributed shared memory of the first hardware host incurs no serialization cost when sending the in-memory representation of the data object to the distributed shared memory of the second hardware host because the in-memory layout is the same as the storage / wire format according to the present technology. The distributed shared memory of the second hardware host may then load the memory copy directly into a memory device.

[0193] If the first hardware host determines in operation 1206 that the second hardware host includes a previous state of a data object, in operation 1210, the first hardware host may use a distributed shared memory to identify a portion of a log segment associated with a representation of the data object that represents a difference between the current state and the previous state of the data object. The distributed shared memory of the first hardware host may determine that a single log record represents a difference between the current state and the previous state of the data object. In another example, the distributed shared memory of the first hardware host may determine that multiple log records represent a difference between the current state and the previous state of the data object.

[0194] In operation 1212, the first hardware host may use the distributed shared memory to send the portion of the log segment to the second hardware host. The distributed shared memory of the first hardware host may send a single log record to the second hardware host, and the single log record is identified as representing a difference between the current state and the previous state of the data object. In another instance, the distributed shared memory of the first hardware host may send multiple log records to the second hardware host, and the log records are identified as representing a difference between the current state and the previous state of the data object.

[0195] In yet another instance, the distributed shared memory of the first hardware host may merge one or more log records identified as representing a difference between the current state and the previous state of the data object into a single log record to be sent to the second hardware host. The first hardware host may use the distributed shared memory to send the single log record to the second hardware host to reduce network bandwidth used for data transfer.

[0196] According to an example of the present technology, the distributed shared memory may use method 1200 to save the current state of a data object in an object data repository or file or to archive the current state of the data object in an object data repository or file. The distributed shared memory may determine to merge multiple log records representing the current state into a single log record in order to save storage space or improve later read time to determine the current state of the data object. For example, before storing a single log record as a file on a disk, the distributed shared memory may write the single log record into a log segment to replace multiple log records. The single log record may include multiple references, and the multiple references indicate changes to the data object aggregated from the multiple log records that reflect changes to multiple portions of the representation of the data object that represents the current state of the data object.

[0197] Figure 13Illustrated are system 1300 and related operations for assigning computing units in a distributed computing system using processing partitions with data objects organized by spatial location according to an example of the present technology. System 1300 may include a world manager 1302 and one or more hardware hosts 1304. According to an example of the present technology, world manager 1302 may assign computing units to hardware hosts 1304 in a distributed computing system using processing partitions organized by spatial location.

[0198] World manager 1302 may include one or more server computers configured to manage the processing of data objects in a distributed computing system. Some examples of world manager 1302 may include the cluster manager 130 described in Figure 1 and the cluster manager module 210 described in Figure 2 To manage processing, according to one example of the present technology, world manager 1302 may determine one or more spatial sub - divisions 1306 associated with an object - based model of a virtual environment hosted by the distributed computing system. Spatial sub - divisions 1306 may include spatial regions that divide the world space into two or more subsets (e.g., non - overlapping, disjoint subsets). The spatial regions can be any and / or uniform partitioning of space. The spatial regions may define spatial units as squares or rectangles, or may define spatial units as: any point, line, area, polyhedron (e.g., polygons and polyhedra), and conic sections such as ellipses, circles, and parabolas. The space can be 2 - D space, 3 - D space, or any multi - dimensional space that can be subdivided. Spatial sub - divisions 1306 can be hierarchical, meaning that the space (or a region of the space) can be divided into a number of spatial regions, and then the spatial regions can be recursively subdivided again to create additional spatial regions. The (one or more) spatial sub - divisions 1306 may also include multiple layers or levels of sub - division, which may overlap with each other and may include different layers or levels that provide different metrics, criteria, and granularities for subdividing the virtual world.

[0199] Spatial sub - divisions 1306 may use, for example, as described below with respect to Figure 14represented by one or more of the tree structures discussed. In one example, the world space associated with a virtual environment may be subdivided using octants, and the spatial subdivision 1306 may be represented using an octree, which is a tree structure in which each internal node has eight children. An octree can be used to subdivide a three-dimensional space associated with a virtual world by recursively dividing the region into eight octants. In another example, the world space associated with a virtual environment may be subdivided using quadrants, and the spatial subdivision 1306 may be represented using a quadtree, which is a tree structure in which each internal node has four children. A quadtree can be used to divide a two-dimensional space associated with a virtual world by recursively dividing the region into four quadrants. Other methods may be used to subdivide the world space associated with a virtual environment and use a data structure to represent the spatial subdivision 1306 so as to provide nodes representing specific units of spatial information and nodes with which data (e.g., data objects) may be associated with specific units of spatial information. Some additional examples may include: grids, Voronoi diagrams, space-filling curves, KD-trees, bounding volume hierarchies, and the like.

[0200] The world manager 1302 may also determine one or more processing partitions 1308 associated with the spatial subdivision 1306. For example, the world manager 1302 may use the spatial subdivision 1306 to create the processing partitions 1308. In one example, a processing partition in the processing partitions 1308 may be associated with a single spatial subdivision in the spatial subdivision 1306. In another example, a single processing partition may be associated with multiple spatial subdivisions in the spatial subdivision 1306. In another example, multiple processing partitions may be associated with one or more spatial subdivisions in the spatial subdivision 1306. Additionally, multiple processing partitions may be associated with a single spatial subdivision.

[0201] The world manager 1302 can determine a processing partition 1308 by mapping a data object 404 to a processing partition 1308 associated with a spatial subdivision 1306. The world manager 1302 can use spatial location information to index the data object 404 into the processing partition 1308. In another example, the world manager 1302 can instruct the hardware host 1304 to use the spatial subdivision 1306 to create the processing partition 1308. The world manager 1302 can define a world space associated with a virtual environment and instruct the hardware host 1304 to create the processing partition 1308 during processing of an object-based model of the virtual environment. In another example, the hardware host 1304 can be responsible for indexing the data object 404 into the processing partition 1308 using spatial location information. The hardware host 1304 can index the data object 404 when creating, deleting, and modifying the data object 404 through an instance of a processing application on the hardware host 1304. Such operations of the instance of the processing application can cause the hardware host 1304 to add, delete, split, merge, and otherwise modify the processing partition 1308. The world manager 1302 can monitor the hardware host 1304 to receive additions, deletions, splits, merges, and other modifications to the processing partition 1308 associated with the spatial subdivision 1306. According to the present technology, one or more processing partitions 1308 can define computing units in a distributed computing system, the computing units being organized according to spatial location information associated with one or more data objects 404, the spatial location information being assignable to one or more hardware hosts 1304. As previously discussed, one or more data objects 404 can be associated with an object-based model of a virtual environment (e.g., 3D simulation, 3D game, etc.). The data object 404 can include one or more object fields 420. In this example, the data object 404 can include one or more object fields 420 to provide an object identifier 430, an object version 432, and an object type 434 as metadata of the data object 404. The object fields 420 can also include one or more object properties or attributes defined using field values or key-value pairs, such as a location field 1316 that can specify a first spatial attribute and a location field 1318 that can specify a second spatial attribute. The location fields 1316 and 1318 can contain: spatial location information, such as data values, strings, X, Y, Z coordinates, vectors, boundaries, grids, ranges, etc., which can be referenced by a processing application to process the data object 404. Other instances of spatial location information contained in the data object 404 can contain: location, place, and orientation information associated with points, lines, polyhedra (e.g., polygons and polyhedra), conic sections (e.g., ellipses, circles, parabolas), or other types of data object models.

[0202] According to an example of the present technology, the object field 420 of the data object 404 can be used to determine the processing partition 1308. Since the processing partition 1308 represents a group of data objects organized into computing units for processing by a processing application, the data object 404 can be organized according to the spatial location information in one or more of the object fields 420. According to an example of the present technology, the data object management performed by the world manager 1302 and the hardware host 1304 may include the creation of one or more index structures, called spatial indexes, which use the spatial location information from the object field 420 to map the data object 404 to the processing partition 1308. The world manager 402 can use the object field 420 to create multiple layers in the grouping, thereby further organizing the data object 404 according to multiple attributes (such as the object domain discussed previously).

[0203] The processing partition 1308 can be represented by one or more spatial index structures that group the data object 404 at or near the spatial units defined by the spatial sub-division 1306 using the spatial location information associated with the data object 404. The spatial index elements or nodes can represent the processing partition and contain data or metadata identifying the processing partition and a set or group of data objects organized into the processing partition according to the spatial units associated with the spatial sub-division (e.g., location or place). The spatial index representing the processing partition 1308 can contain a tree structure representing the spatial sub-division 1306. The nodes of the tree structure representing the spatial sub-division 1306 can map or index the data object 404 to the processing partition 1308 using the spatial location information. The spatial index representing the processing partition 1308 can also contain metadata information about the relevant regions of the spatial sub-division 1306, such as parent, neighbor (special sibling), child, level, etc.

[0204] The world manager 402 can use the processing partition 1308 to determine one or more processing partition assignments 1312. The processing partition assignment 1312 assigns the processing partition 1308 to the hardware host 1304. The processing partition assignment 1312 can be represented by a data structure (e.g., list, index, tree, etc.) that identifies the hardware host in the hardware host 1304 and the processing partition 1308 assigned to the hardware host. The processing partition assignment 1312 can be based on the number of spatial sub-divisions in the spatial sub-division 1306, the object type of the data object 404 at a given location, the application instances that can process the object type, the spatial relationships (e.g., between the spatial sub-divisions 1306 or between the data objects 404), the data correlation between the data objects 404, and the metrics associated with the hardware host 1304 (such as the processing load on the hardware or software resources (e.g., to avoid overload, etc.)).

[0205] The World Manager 1302 may send a processing partition assignment 1312 to the hardware host 1304. In one example, the World Manager 1302 may send the processing partition assignment 1312 directly to the hardware host 1304. In another example, the World Manager 1302 may notify the hardware host 1304 of the processing partition assignment 1312, and the hardware host 1304 may retrieve the processing partition assignment 1312 from another source. The hardware host 1304 may then use the processing partition assignment 1312 to process the data object 404. Examples of processing may include: data object movement, data object deletion, data object insertion, data object conflict, rendering, explosion, physical simulation, in-world credit transactions, or other types of processing. The hardware host 1304 may use the processing partition assignment 1312 to identify the allocation of processing partitions 1308. The World Manager 1302 may utilize the processing partition assignment 1312 to send the processing partitions 1308 to the hardware host 1304. In another example, the hardware host 1304 may retrieve the allocation of the processing partitions 1308 from another source.

[0206] Figure 14 A graphical example is shown of organizing computing units according to an example of the present technology to assign to hardware hosts in a distributed computing system 1400 using spatial location information associated with data objects. Figure 14 It is shown that the world space is first divided into more than one group or processing partition using object domains (e.g., "Domain A" and "Domain B") as discussed above and then using a spatial index. If the world space of a virtual environment hosted by a distributed computing system includes two domains, where each domain may have a large number of data objects densely encapsulated into different spaces, the computing units of these two domains may be in different states, as depicted. Thus, Figure 14 It is shown that the world space is further divided into more than one group or processing partition using multiple spatial sub-divisions (e.g., "Sub-division A" and "Sub-division B").

[0207] The distributed computing system 1400 may include a spatial index manager 1402, which may include one or more server computers configured to index data objects in a virtual world hosted by the distributed computing system 1400 into computing units or processing partitions using spatial location information. Some examples of the spatial index manager 1402 may include the World Manager 1302 and the hardware host 1304 described in Figure 13 reference. The spatial index manager 1402 may determine multiple processing partitions associated with multiple spatial sub-divisions of the domains of Sub-division A and Sub-division B. For example, the spatial index manager 1402 may initially create and manage one or more spatial indexes 1408 using one or more object fields 1404 and an index configuration 1406 (as described later).

[0208] The spatial index manager 1402 can analyze the spatial location information associated with the (one or more) object fields 1404 to determine a set of data objects, and the set of data objects can be organized into processing partitions according to location, proximity, size, shape, spatial relationship, spatial dependency, spatial causality, etc. For example, the spatial index manager 1402 can analyze the spatial location information to determine whether data objects can be grouped into processing partitions by location or place. In another example, data objects that are close to each other or are approximately at the same location or place can be grouped into the same processing partition. The spatial index manager 1402 can further analyze the spatial location information to determine whether there is a spatial relationship for the data objects. For example, a first data object can have a spatial relationship with a second data object by being located near the second data object. In another example, a first data object and a second data object can have a spatial relationship with a bounding box by being located within a plane or volume associated with the bounding box. The spatial index manager 1402 can determine whether some of the data objects satisfy spatial causality by being within a predefined distance of each other. Other instances can include data objects satisfying a spatial relationship with gravity or an electromagnetic field, being located within a defined geometry, being located on a defined topology, being close to or intersecting a set of vertices and edges that describe a polygon, mesh, collision box, or other shape, extent, camera view, etc.

[0209] The spatial index manager 1402 can use the index configuration 1406 to determine how to represent the processing partitions associated with the spatial index 1408. The spatial index manager 1402 can use the multiple data objects that can be assigned to a processing partition to determine the capacity of the processing partition. The index configuration 1406 can specify the number of data objects that can be indexed by the spatial index 1408. The number of data objects that can be indexed by the spatial index 1408 can be determined based on the configuration of the processing application and the capacity of the processing application to process multiple data objects. The number of data objects that can be indexed by the spatial index 1408 can also be determined based on the configuration of the virtual world, and the configuration of the virtual world can define whether there is a one-to-one, one-to-many, and many-to-one correspondence between the processing partitions and the spatial index 1408, the structure of the spatial index 1408, one or more functions for indexing the data objects 404, the configuration of the hardware and software resources, the number of hardware hosts in a distributed computing system, etc. The index configuration 1406 can also include index-specific configurations (e.g., to use an octree with a maximum level and a predefined range) that are set as part of the world definition.

[0210] In this example, the spatial index manager 1402 uses the object field(s) 1404 and the index configuration 1406 to create, update, and manage the spatial index(es) 1408. Since a processing partition represents a group of data objects that are organized into computational units to be processed by a processing application, the spatial index manager 1402 can organize the data objects according to the spatial location information in the object field 1404. The spatial index manager 1402 can use the object field(s) 1404 to create the spatial index(es) 1408 through the spatial location information to map a collection of data objects to processing partitions. As shown, the spatial index(es) 1408 can use the object field(s) 1404 to create multiple layers in the grouping, thereby further organizing the data objects by multiple attributes such as object domain and spatial location.

[0211] The spatial index 1408 can be represented by a tree structure having nodes 1410, 1412, 1414, 1416, 1418, 1420, and 1422. In this example, the node 1410 can represent the root node, and the nodes 1412, 1414, 1416, 1418, 1420, and 1422 can represent child nodes. The root node 1410 can include metadata 1430 that provides information associated with the processing partition represented by the root node 1430 and the collection of data objects 404 organized into the processing partition. The metadata 1430 can also include information about the relevant regions of the spatial index 1408, such as parent nodes, neighbor nodes (special siblings), child nodes, etc. The metadata 1430 can identify the index levels and the first level used when subdividing the space. For example, the metadata 1430 can specify that the first level for partitioning the world space is the object type, and the second level is the spatial location. The child nodes 1412, 1414, 1416, 1418, 1420, and 1422 can also include similar metadata (not shown).

[0212] As data objects are generated in world space, the spatial index manager 1402 may be responsible for indexing the data objects into processing partitions using the spatial index 1408. At initialization of the world space, the spatial index manager 1402 may allocate a single node in the spatial index 1408 of the data objects based on object type or spatial sub-division. The spatial index manager 1402 may receive an instruction to index a data object, and once located within the world space, the spatial index manager 1402 may map the data object to a processing partition by reading the object fields 1404 and by adding the metadata associated with the data object to the node associated with the processing partition in the spatial index 1408. The spatial index manager 1402 may maintain an atomic reference to a working copy of the spatial index 1408, and only the spatial index manager 1402 may access the working copy. The spatial index manager 1402 may use the working copy to index the data objects and then flip the atomic reference to the spatial index 1408 so that other threads (e.g., the world manager or the application manager) may access the current state of the processing partition.

[0213] To determine where to add a data object to the spatial index 1408, the spatial index manager 1402 may use a hash code octree implementation to calculate the location of the data object using a normalized range. The spatial index manager 1402 may calculate the hash of the data object by normalizing the spatial location information associated with the data object to the normalized range. The calculated hash may identify the node(s) in the spatial index 1408 corresponding to the spatial sub-division of the world space that includes the location of the data object. For example, the spatial index manager 1402 may normalize the location dimensions associated with the data object to floating point numbers from 0 to 1. The spatial index manager 1402 may then multiply the normalized location by a power of 2 that is associated with the level specified for the octree. The spatial index manager 1402 may effectively hash the data object into the deepest possible level of the spatial index 1408. The spatial index manager 1402 may then use the calculated hash to traverse the spatial index 1408 from the root node 1410 in order to find the interior or child node that encloses the data object. During the traversal, the spatial index manager 1402 may compare one or more hash bits that match the "next" level with the children of the current node. The spatial index manager 1402 may use the hash to determine the index of the next child in the children array.

[0214] During the life cycle of a data object in world space, a request to modify data associated with the data object may be issued from an instance of a processing application associated with the object type of the data object. The spatial index manager 1402 may re-index the data object into the spatial index 1408 using the changes to the data object caused by the request. For example, during re-indexing, the spatial index manager 1402 may determine an updated mapping between the data object and the processing partition in response to a request to modify the data. The spatial index manager 1402 may also determine the updated mapping in response to an instruction from the world manager or another process to re-index the data object. The spatial index manager 1402 may use the changes to the data object caused by the request to re-index the data object and update the metadata associated with the data object in the spatial index 1408. Re-indexing may occur when the ownership of the data object transfers from a first hardware host to a second hardware host. In such a case, the spatial index manager 1402 may acquire the ownership of the data object and update the metadata associated with the data object in the spatial index 1408.

[0215] The spatial index manager 1402 may also use the spatial location information associated with the data object to determine whether to change or maintain the mapping between the data object and the processing partition. The spatial index manager 1402 may re-index the data object by adding metadata associated with the data object at different nodes in the spatial index 1408, for example, because the movement of the data object reflected in the spatial location information causes the data object to be at a location associated with a different spatial sub-division of the world space. The spatial index manager 1402 may also re-index the data object to remove the metadata associated with the data object from a node in the spatial index 1408 because the data object is no longer in the world space (e.g., due to de-spawning or being destroyed).

[0216] After re-indexing, the spatial index manager 1402 may also determine whether to split or merge nodes in the spatial index 1408. When the number of data objects mapped to a first processing partition represented by a first node approaches, meets, or exceeds a first defined threshold, the spatial index manager 1402 may determine to split the first node into a second node and a third node. The first defined threshold may be determined as a first percentage (e.g., 60%) of the capacity of the processing application to process multiple data objects. When the number of data objects mapped to a first processing partition and a second processing partition represented by the first node and the second node approaches, meets, or is below a second defined threshold, the spatial index manager 1402 may determine to merge the first node and the second node to create a third node. The second defined threshold may be determined as a second percentage (e.g., 10%) of the capacity of the processing application to process multiple data objects.

[0217] The spatial index manager 1402 can read the spatial index 1408 and perform any splitting or merging as needed. In one example, when a first defined threshold that is determined to handle a first percentage (e.g., 60%) of the capacity of the processing application to handle multiple data objects is met or exceeded, the spatial index manager 1402 can split a node and add the node to the spatial index 1408 (or update the metadata of an existing node to reflect the split). For example, the world space region associated with the child node 1412 can be subdivided into four sub-regions that can be associated with the child node 1414 and three siblings. When the first threshold is met or exceeded, the spatial index manager 1402 can allocate some of the data objects indexed by the child node 1412 across one or more of the child node 1414 and three siblings or a combination thereof. The spatial index manager 1402 can use the spatial location information associated with the data object to index the data object into the child node 1414 and three siblings, and the spatial location information locates the data object into the corresponding sub-region.

[0218] In some cases, data objects that hash outside the normalized range into the child nodes of the (one or more) spatial index 1408 can be mapped to the root node 1410. If there are too many data objects at the root node 1410, an "upward" split can be performed. Additional nodes can be added to the spatial index 1408 as siblings of the root node 1408 representing additional quadrants or octants of the extended world space. Each additional quadrant or octant added to the side of the root node 1410 can be considered for an upward split to become a new root node. Alternatively, an entirely new root node can be created. To determine the location of the upward expansion, the data objects at the root 1410 can be classified into super-regions that cover the existing world space. Using, for example, four super-regions, the best super-region for upward expansion can be determined by which split removes the most objects from the root node 1410 and keeps the size of the node below a limit. Due to the nature of this change, the object hash can be recalculated by adding bits representing the extended space to the current object hash. For range changes that are a power of 2 (doubling, quadrupling), the existing location hash can be transformed by shifting bits for each dimension. The actual range change can also be performed by rehashing all the data objects and recreating the spatial index 1408. A "shadow" octree can be created by the spatial index manager 1402 and then published when finished.

[0219] If there are too many objects in a node near the bottom of the tree, a level addition can be performed. The level addition can include adding a bit to the end of the data object hash. The level addition can occur when nodes are created within two levels from the bottom.

[0220] In another example, the spatial index manager 1402 may merge nodes and remove nodes from the spatial index 1408 (or update the metadata of an existing node to reflect the merge) when a second defined threshold that is determined to be a second percentage (e.g., 10%) of the capacity of the processing application to handle multiple data objects is met or exceeded. For example, four sub-regions including a world space region associated with child node 1422 and three siblings may be merged into the world space region associated with child node 1420. Any child nodes associated with child node 1422 may also be merged into the world space region associated with child node 1420. The spatial index manager 1402 may combine data objects indexed by child node 1422, the three siblings, and any descendants of child node 1422 and the three siblings or a combination of one or more of them with child node 1420. To merge the nodes together, the spatial index manager 1402 may take ownership of child node 1420 at level N, child node 1422 at level N+1, and the three siblings of child node 1420 at level N+1. If the total number of data objects indexed by child node 1422 and the three siblings is close to a lower threshold or minimum number of data objects, the spatial index manager 1402 may merge child node 1422 at level N and the three siblings at level N+1 into node 1420. The spatial index manager 1402 may then delete child node 1422 and the three siblings.

[0221] Figures 15A - 15B FIG. is a flowchart illustrating an example method 1500 for managing hardware hosts in a distributed computing system using processing partitions organized by spatial location information associated with data objects according to an example of the present technology. Method 1500 may be performed when executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or an information processing device) by software (e.g., instructions or code modules), by hardware components of an electronic device or an application specific integrated circuit, or by a combination of software and hardware elements.

[0222] In operation 1502, the world manager may determine a plurality of spatial sub-divisions associated with an object-based model representing a virtual environment hosted by a distributed computing system. The world manager may use a configuration associated with the object-based model representing the virtual environment to determine the plurality of spatial sub-divisions. The world manager may parse the configuration to determine the size of the world space associated with the virtual environment, a data structure (e.g., an octree or a quadtree) for representing the plurality of spatial sub-divisions, and a configuration of the data structure (e.g., the number of levels, the number of nodes within a level, and the size of each node in terms of a minimum or maximum number of data objects that can be indexed by the node).

[0223] The world manager can also determine multiple spatial sub - divisions using a configuration associated with a processing application that processes data objects in an object - based model. The world manager can read the configuration to determine multiple object types associated with the data objects processed by the processing application, the application resources used by the processing application, the type of data structure used by the processing application to access the data objects, the minimum and maximum number of data objects that can be processed by the processing application, the subscription types that specify which additional object types to subscribe to in order to process the data objects, and the subscription strategy for obtaining the additional data. The subscription strategy can identify what additional data object types are used during processing and what source locations the additional object types can be located at.

[0224] The subscription strategy of a processing application can be used to determine which additional data objects are to be synchronized to the hardware host where the processing instance of the processing application is located. In one example, the subscription strategy can specify that the processing application subscribes to neighbor nodes in a data structure (e.g., an index) that references data objects. Using the data structure, the hardware host can discover the neighbors of the nodes to which the data objects being processed by the processing application (e.g., using spatial location information) are indexed, and the hardware host can cause the processing application to subscribe to data from those neighbor nodes. In another example, the subscription strategy can specify object tracking, where the object tracking identifies data objects, and the subscription strategy can subscribe to data from the nodes to which the data objects are indexed, the neighbors of the nodes, the parents, the children, etc.

[0225] The subscription strategy can be used to initiate subscriptions to data objects from different hardware hosts in a distributed computing system. For example, a processing application can be instantiated on a first hardware host, and neighbor nodes can be assigned to a second hardware host. The first and second hardware hosts can communicate to share state changes to the data objects indexed to the neighbor nodes. For example, the first hardware host can identify the neighbor nodes as being assigned to the second hardware host using processing partition assignments. The first hardware host can send instructions, messages, subscription requests, etc. to the second hardware host to subscribe to and obtain data associated with the data objects mapped to the neighbor nodes (e.g., for dependencies between the first hardware host and the second hardware host). The second hardware host can add the first hardware host to a subscriber list to receive data associated with the data objects mapped to the neighbor nodes. The second hardware host can determine changes to the data objects mapped to the neighbor nodes and send all or part of the data associated with the data objects mapped to the neighbor nodes to the first hardware host.

[0226] In another example, a processing application can issue a subscription for data associated with all players near an enemy for use by an AI to control enemy units. This can also be useful for processing applications or clients that perform rendering. Additionally, the subscription strategy can specify the maximum latency that an application instance can tolerate in obtaining additional object types from a specified source.

[0227] Refer again to Figure 15A , the world manager can establish multiple spatial sub - divisions as spatial regions that divide the space into two or more subsets. The multiple spatial sub - divisions can be hierarchical, meaning that the space (or a region of the space) can be divided into a number of spatial regions, and then the spatial regions can be recursively subdivided again to create additional spatial regions. The multiple spatial sub - divisions can also include multiple layers or hierarchies of sub - divisions, which can overlap with each other and can include different layers or hierarchies that provide different metrics, criteria, and granularities for subdividing the virtual world.

[0228] As discussed above, the multiple spatial sub - divisions can be represented using one or more tree structures. For example, as discussed above with respect to Figure 14 . The world space associated with the virtual environment hosted by the distributed computing system can be subdivided using octants, and an octree can be used to represent the multiple spatial sub - divisions to subdivide the three - dimensional space associated with the virtual world. In another example, the virtual world can be subdivided using quadrants, and a quadtree can be used to represent the multiple spatial sub - divisions to divide the two - dimensional space associated with the virtual world. Other methods can be used to subdivide the virtual world and use data structures to represent the multiple spatial sub - divisions in order to provide nodes to represent specific units of spatial information and which data can be associated with the specific units of spatial information.

[0229] In operation 1504, the world manager can determine the mapping between multiple data objects and multiple processing partitions associated with the multiple spatial sub - divisions. For example, the world manager can use an octree associated with the multiple spatial sub - divisions to initialize a spatial index. The world manager can utilize the nodes in the octree to represent processing partitions so as to associate a processing partition with a single spatial sub - division among the multiple spatial sub - divisions. The world manager can also utilize a single node in the octree to represent multiple processing partitions and associate the multiple processing partitions with a single spatial sub - division. In addition, the world manager can utilize multiple nodes in the octree to represent processing partitions and associate a single processing partition with the multiple spatial sub - divisions.

[0230] In operation 1506, the world manager may identify multiple hardware hosts in a distributed computing system to process an object-based model using multiple data objects. The world manager may identify hardware hosts from a pool of hardware hosts waiting for assignment, or the world manager may identify active hardware hosts with spare computing capacity, etc. The active hardware hosts may execute application instances in containers or in computing instances that may operate on processing partitions. In operation 1508, the world manager may assign multiple processing partitions to multiple hardware hosts using multiple spatial indexes. More specifically, the world manager may assign one or more spatial indexes to a hardware host. The world manager may also assign one or more hardware hosts to a spatial index. The world manager may assign processing partitions to hardware hosts based on: spare computing capacity, hardware hosts with dedicated hardware (e.g., GPUs), load balancing, co-located data, etc.

[0231] In operation 1510, the world manager may generate multiple processing partition assignments between the multiple processing partitions and the multiple hardware hosts. This means that processing partitions may be assigned to hardware hosts with application instances of a type that can process data objects mapped to the processing partitions and organized using spatial locations. For example, the world manager may assign a spatial index to a hardware host that has a physics application, a collision detection application, a rendering application, etc., where the physics application modifies data objects using physical simulation, the collision detection application determines conflicts between data objects, and the rendering application renders the data objects into one or more images or videos. In operation 1512, the world manager may send the multiple processing partition assignments to the multiple hardware hosts to organize the multiple hardware hosts to process the multiple data objects using spatial locations.

[0232] Method 1500 continues to use Figure 15A reference "A" to Figure 15B . In operation 1514, the world manager may use a map to monitor the spatial indexes of multiple hardware hosts for multiple data objects to process the multiple data objects. The world manager may monitor the multiple hardware hosts to track object changes, to determine whether the multiple processing partitions have changed, and to determine metrics associated with the hardware hosts. The data objects may be modified as a result of being processed by a processing application, and the processing application may change the spatial location information associated with the data objects. The processing partitions may also be modified due to changes in the spatial location information, which are attributed to the processing of the data objects by the processing application. Changes in the spatial location information may introduce re-indexing of the data objects by the hardware hosts. Changes in the spatial location information may also result in the splitting or merging of the processing partitions.

[0233] In operation 1516, the world manager may determine changes to a plurality of processing partitions associated with a plurality of spatial subdivisions. Updating the processing partitions with the changes may include mapping newly created data objects to the processing partitions, modifying data objects mapped to the processing partitions, or removing the mapping between data objects and the processing partitions. Updating the processing partitions with the changes may include splitting a first processing partition into a second processing partition and a third processing partition, thereby allocating the data objects indexed to the first processing partition to the second processing partition and the third processing partition. In another example, updating the processing partitions with the changes may include merging a first processing partition and a second processing partition into a third processing partition, thereby allocating the data objects indexed to the first processing partition and the second processing partition to the third processing partition.

[0234] In operation 1518, the world manager may use the changes to determine whether to update a plurality of processing partition assignments. The world manager may determine to update the plurality of processing partitions to manage or optimize the performance of a hardware host. The world manager may perform load balancing on the processing of data objects across a plurality of hardware hosts. Additionally, the world manager may redistribute processing partitions due to changes to data objects or hardware hosts. If in step 1520 the world manager determines not to update the plurality of processing partition assignments using the changes, then method 1500 continues in operation 1514, where the world manager returns to monitoring the plurality of hardware hosts.

[0235] If in step 1520 the world manager uses the changes to determine to update the plurality of processing partition assignments, then method 1500 continues in operation 1522, where the world manager updates the plurality of processing partitions using updates from the hardware host. The world manager may update the plurality of processing partitions, for example, by changing the processing partition assignment of a first hardware host and allocating the spatial index owned by the first hardware host to a second hardware host. Thereby, the world manager may migrate a processing partition from the first hardware host to the second hardware host. Method 1500 continues using reference "B" from Figure 15B and returns to Figure 15A , where in operation 1512, the world manager may send the plurality of processing partition assignments to the plurality of hardware hosts.

[0236] Figure 16 is a flowchart showing an example method 1600 for processing data objects assigned to a hardware host using processing partitions organized according to spatial location information associated with the data objects according to an example of the present technology. Method 1600 may be performed when executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or an information processing device) by software (e.g., instructions or code modules), by hardware components of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0237] In operation 1602, a distributed shared memory associated with a hardware host in a distributed computing system may receive a plurality of processing partition assignments representing an assignment between a plurality of processing partitions and a plurality of hardware hosts. The distributed shared memory may receive the plurality of processing partition assignments from a cluster or a world manager associated with the distributed computing system. The plurality of processing partition assignments may identify a plurality of hardware hosts in the distributed computing system and nodes of a tree structure assigned to the plurality of hardware hosts. The nodes may include a spatial index structure representing a set of data objects grouped or indexed by spatial location information. The set of data objects may also be organized by object type. Thus, the nodes may represent a collection of data objects of the same object type and satisfying spatial dependencies.

[0238] In operation 1604, the distributed shared memory may use the plurality of processing partition assignments to identify the processing partitions assigned to the hardware host associated with the spatial sub-division. The distributed shared memory may analyze a list providing the plurality of processing partition assignments between the plurality of hardware hosts and the plurality of processing partitions to find the assignment to the hardware host. The hardware host may filter the list to identify the processing partitions assigned to the hardware host.

[0239] In operation 1606, the distributed shared memory may load a plurality of data objects mapped to the processing partitions grouped by the location associated with the spatial sub-division into a memory device associated with the hardware host. For example, the distributed shared memory may read the nodes of the tree structure corresponding to the processing partitions to obtain metadata about the data objects mapped to the processing partitions. The distributed shared memory may retrieve the data objects from files, object repositories, across a network, from another distributed shared memory, etc.

[0240] In operation 1608, the hardware host may use an instance of a processing application on the hardware host to process the plurality of data objects. The hardware host may determine the object type of the data objects to identify the processing application configured to execute code on data objects of the object type. The hardware host may also determine the location of the data objects to identify the processing application configured to execute code on the data objects based on the location. The hardware host may create an instance of the processing application as a containerized application to process the data objects.

[0241] In operation 1610, the hardware host may determine whether to update the processing partition using changes associated with spatial sub - division. The hardware host may determine in operation 1608 to update the processing partition in response to the processing of a data object, where the processing of the data object causes a change in the spatial location information associated with the data object. For example, a processing application may create a new data object within a virtual environment, move a data object, change data associated with an existing data object, and remove a data object from the virtual environment. Create, delete, and update operations may add or remove spatial dependencies between data objects used to organize the data objects into processing partitions.

[0242] In a particular instance, a new data object may be created at a given location, and the new data object triggers a spatial dependency with one or more of the data objects. The hardware host may determine to update the processing partition to add the new data object to the processing partition because the data object satisfies the spatial dependency. In another example, a new data object introduced to a given location that triggers a spatial dependency may cause the number of data objects to be processed by the processing application to increase and approach a set upper threshold. When the number of data objects to be processed by the processing application approaches the threshold, the hardware host may determine to split the first processing partition into two processing partitions, thereby allocating some of the data objects to remain mapped to the first processing partition and allocating some of the data objects to be mapped to a new second processing partition. The hardware host may also create a second instance of the processing application to process the data objects mapped to the second processing partition.

[0243] In another example, some of the data objects may move to a different location from the remaining data objects during processing, and the spatial dependency with the remaining data objects is removed. In yet another example, some of the data objects may be destroyed virtually during processing. The hardware host may determine to update the processing partition to remove data objects that no longer satisfy the spatial dependency or no longer exist in the virtual environment from the processing partition. Removing a data object from a given location may cause the number of data objects processed by the processing application to decrease. As described above, the number of data objects processed by the processing application may approach or exceed a defined threshold. As the number of data objects disposed of by the processing application approaches the defined threshold, the hardware host may determine to merge the first processing partition with the second processing partition, thereby combining the data objects mapped to the second processing location with the data objects mapped to the first processing partition.

[0244] Return to Figure 16, if the hardware host determines not to update the processing partition in operation 1612, the hardware host can continue to process multiple data objects using an instance of the processing application in operation 1608. If the hardware host determines to update the processing partition in operation 1612, then the hardware host can generate an updated processing partition in operation 1614 to re-index the data objects associated with the spatial sub-division. As discussed above, create, delete, and update operations can add or remove spatial dependencies used to organize data objects into processing partitions associated with a spatial sub-division. The hardware host can generate the updated processing partition by re-indexing the data objects and including any additional data objects that satisfy the spatial relationship with the spatial sub-division.

[0245] As discussed above, updating the processing partition can also be used to notify a cluster or world manager associated with the distributed computing system about the utilization of hardware and software resources associated with the hardware host. Adding new data objects can increase resource utilization because the number of data objects to be processed by the processing application has increased. To load balance the processing of data objects in the distributed computing system, the cluster or world manager can evaluate whether the hardware host has sufficient resources to process the additional data objects. Removing a data object from the processing partition can decrease resource utilization because the number of data objects to be processed by the processing application has decreased. The cluster or world manager can then evaluate whether the hardware host can process the additional data objects. Thus, in operation 1616, the hardware host can send the updated processing partition to the world manager.

[0246] Figure 17 is a flowchart showing an example method 1700 for splitting a processing partition organized by spatial location information according to an example of the present technology. The method 1700 can be executed when software (e.g., instructions or code modules) is executed by a central processing unit (CPU or processor) of a logical machine (e.g., a computer system or an information processing device), by a hardware component of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0247] In operation 1702, a hardware host in a distributed computing system may process operations associated with multiple data objects. The data objects may be grouped into a first processing partition at a first location associated with a first spatial sub-division using a first spatial index node. The operations may include create, read, update, and delete (CRUD) operations processed by a distributed shared memory on the hardware host. In operation 1704, the hardware host may determine to split the first processing partition into a second processing partition and a third processing partition based on the results of the operations. The hardware host may determine to split the first processing partition into a second processing partition and a third processing partition because the number of data objects disposed of by an instance of a processing application associated with the first processing partition is approaching, equal to, or exceeding a threshold specified for the processing application. The hardware host may determine the threshold used when splitting the first processing partition as a percentage of the capacity defined by the processing application to process multiple data objects. For example, when the number of data objects disposed of by an instance of the processing application meets or exceeds 60% of the capacity defined for the processing application, the hardware host may trigger the splitting of the first processing partition into a second processing partition and a third processing partition.

[0248] For example, when a new data object is created in a virtual environment and the new data object is mapped to the first processing partition, the hardware host may determine that the number of data objects disposed of by an instance of the processing application may have increased. When a data object is moved from a first location to a second location associated with the first spatial sub-division, the hardware host may determine that the number of data objects disposed of by an instance of the processing application may have increased. The hardware host may also determine that the movement of the first data object may have triggered a spatial dependency with a second data object mapped to the first processing partition.

[0249] In operation 1706, the hardware host may split some of the multiple data objects into a second processing partition that groups the data objects by a second location associated with a second spatial sub-division. The hardware host may determine that spatial location information associated with the data objects may be used to group the data objects into the second processing partition by the second location. In another example, the hardware host may determine that some of the data objects meet a spatial relationship with the second location or have a data dependency with other objects at or near the second location. The hardware host may then allocate the data objects to the second processing partition. In operation 1708, the hardware host may generate, determine, or identify a second node representing the second processing partition in order to index the data objects grouped into the second processing partition based on the second spatial sub-division. The hardware host may use metadata that maps some of the data objects to the second processing partition to generate the second spatial index node.

[0250] In operation 1710, the hardware host may split some of the multiple data objects into a third processing partition, and the third processing partition groups the data objects according to a third position associated with a third spatial sub-division. Thus, the hardware host may allocate the data objects to the third processing partition. In operation 1712, the distributed shared memory may determine or identify a third spatial index node representing the third processing partition for indexing the data objects associated with the third spatial sub-division.

[0251] In operation 1714, the hardware host may use the second spatial index node and the third spatial index node to update the tree structure associated with the multiple spatial sub-divisions to reflect the split of the first processing partition. For the first processing partition, the hardware host may update the tree structure to remove the metadata (if any) of the data objects that are now mapped to the second processing partition and the third processing partition from the first spatial index node. The hardware host may use the second spatial index node to update the tree structure to include the metadata of the data objects that are now mapped to the second processing partition. The hardware host may further use the third spatial index node to update the tree structure to include the metadata of the data objects that are now mapped to the third processing partition. The hardware host may generate the second spatial index node and the third spatial index node in the tree structure by updating the existing spatial index nodes or adding additional spatial index nodes (e.g., children or sibling spatial index nodes of the first spatial index node associated with the first processing partition).

[0252] Splitting the first processing partition into the second processing partition and the third processing partition may also be used to notify a cluster or a world manager associated with the distributed computing system about the utilization of the hardware and software resources associated with the hardware host to which the second processing partition and the third processing partition are currently assigned. As discussed above, the second processing partition and the third processing partition may increase the resource utilization on the hardware host because the number of data objects to be processed by an instance of the processing application has increased, or another instance of the processing application may be launched. To balance the load of processing data objects in the distributed computing system, the cluster or the world manager may evaluate whether the hardware host has sufficient resources to process the data objects assigned to the second processing partition and the third processing partition.

[0253] Figure 18 FIG. is a flowchart showing an example method 1800 for merging processing partitions organized according to spatial location information according to an example of the present technology. The method 1800 may be executed when software (e.g., instructions or code modules) is executed by a central processing unit (CPU or processor) of a logic machine (e.g., a computer system or an information processing device), by hardware components of an electronic device or an application-specific integrated circuit, or by a combination of software and hardware elements.

[0254] In operation 1802, a hardware host in a distributed computing system may process operations associated with a first plurality of data objects grouped into a first processing partition according to a first location associated with a first spatial sub - division. In operation 1804, the hardware host may determine to merge the first processing partition and a second processing partition based on the result of the operation. When the number of data objects disposed of by an instance of a processing application associated with the first processing partition approaches, equals, or exceeds a merge threshold specified for the processing application, the hardware host may determine to merge the first processing partition with the second processing partition due to the operation. The merge threshold may be a percentage of the capacity of the processing application to process a plurality of data objects. For example, when the number of data objects disposed of by an instance of the processing application meets or is below 10% of the capacity of the processing application, the merge threshold may trigger the merge of the first processing partition and the second processing partition.

[0255] As discussed above, for example, when a data object moves in a virtual environment and the data object can be mapped to a different processing partition based on the new location of the data object, the number of data objects disposed of by an instance of the processing application may decrease. In another example, when a data object is removed or deleted from the virtual environment, the number of data objects disposed of by an instance of the processing application may decrease. Removing and updating data objects in the virtual environment may cause the percentage of data objects previously disposed of by an instance of the processing application to subsequently meet the merge threshold.

[0256] In operation 1806, the hardware host may merge a plurality of data objects into a second processing partition that groups the data objects according to a second location associated with a second spatial sub - division. For example, the hardware host may determine that spatial location information associated with the plurality of data objects can be used to group data objects from both the first processing partition and the second processing partition into the second processing partition according to the second location. In another example, the hardware host may determine that the spatial range defined for the second processing partition using the second location encompasses the locations associated with a plurality of data objects in the first processing partition and the second processing partition. The hardware host may then move or allocate the plurality of data objects from the first processing partition to the second processing partition.

[0257] In operation 1808, the hardware host may generate a spatial index node representing the second processing partition to index a plurality of data objects grouped into the second processing partition associated with the second spatial sub - division. The hardware host may use metadata that maps the plurality of data objects to the second processing partition to generate the spatial index node.

[0258] In operation 1810, the hardware host may use a spatial index node to update a tree structure associated with multiple spatial subdivisions to reflect the merger of a first processing partition and a second processing partition. The hardware host may use the spatial index node to update the tree structure to include metadata for data objects now mapped to the second processing partition. The hardware host may also update the tree structure to remove metadata from the spatial index node associated with the first processing partition. The hardware host may generate a spatial index node in the tree structure by updating an existing spatial index node (e.g., the parent or sibling of the spatial index node associated with the first processing partition) or adding an additional spatial index node.

[0259] Merging the first processing partition into the second processing partition may additionally be used to notify a cluster or world manager associated with the distributed computing system of underutilization of the hardware and software resources associated with the hardware host to which the second processing partition is currently assigned. As discussed above, adding a new data object to the second processing partition may increase resource utilization because the number of data objects to be processed by an instance of the processing application has increased. To load balance the processing of data objects in the distributed computing system, the cluster or world manager may evaluate whether the hardware host has sufficient resources to process the additional data objects assigned to the second processing partition.

[0260] Figure 19 System 1900 and computing device 1910 are shown, on which the modules of the present technology may be executed. Computing device 1910 is shown, and a high-level example of the technology may be executed on the computing device 1910. Computing device 1910 may include one or more processors 1912 in communication with a memory device 1920. Computing device 1910 may include a local communication interface 1918 for components in the computing device. For example, the local communication interface 1918 may be a local data bus and / or any associated address or control bus that may be required.

[0261] Memory device 1920 may contain module 1924 executable by processor 1912 and data for module 1924. Module 1924 may perform the functions described previously. Data repository 1922 may also be located in memory device 1920 for storing data related to module 1924 and other applications, as well as an operating system executable by processor 1912. Other applications may also be stored in memory device 1920 and executable by processor 1912. The components or modules discussed in this specification may be implemented in software using a high-level programming language that is compiled, interpreted, or executed using a hybrid of methods.

[0262] The computing device 1910 may also have access to I / O (input / output) devices 1914 that can be used by the computing device 1910. Networking devices 1916 and similar communication devices may be included in the computing device 1910. The networking device 1916 may be a wired or wireless networking device connected to the Internet, a LAN, a WAN, or other computing networks.

[0263] Components or modules 1924 shown as stored in the memory device 1920 may be executable by the processor 1912. The term "executable" may mean a program file in a form that can be executed by the processor 1912. For example, a program in a higher-level language may be compiled into machine code having a format that can be loaded into the random access portion of the memory device 1920 and executed by the processor 1912, or the source code may be loaded by another executable program and interpreted to generate instructions in the random access portion of the memory for execution by the processor. The executable program may be stored in any part or component of the memory device 1920. For example, the memory device 1920 may be a random access memory (RAM), a read-only memory (ROM), a flash memory, a solid state drive, a memory card, a hard disk drive, an optical disc, a floppy disk, a magnetic tape, or any other memory component.

[0264] The processor 1912 may represent multiple processors, and the memory device 1920 may represent multiple memory units operating in parallel with the processing circuitry. This may provide parallel processing channels for processes and data in the system. The local interface 1918 may be used as a network to facilitate communication between any of the multiple processors and the multiple memories. The local interface 1918 may use additional systems designed to coordinate communication, such as load balancing, bulk data transfer, and similar systems.

[0265] Although the flowcharts presented for this technology may imply a specific order of execution, the order of execution may be different from that shown. For example, the order of two or more blocks may be rearranged relative to the order shown. Further, two or more consecutive boxes shown may be executed in parallel or have partial parallelization. In some configurations, one or more boxes shown in the flowchart may be omitted or skipped. For purposes of enhancing utility, accounting, performance, measurement, troubleshooting, or for similar reasons, any number of counters, status variables, warning semaphores, or messages may be added to the logical flow.

[0266] Some of the functional units described in this specification have been labeled as services that can be implemented as modules. A module may be implemented as a hardware circuit that includes custom VLSI circuits or gate arrays, off-the-shelf semiconductors (such as logic chips), transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc.

[0267] The module can also be implemented in software for execution by different types of processors. The identified module of executable code can include, for example, one or more blocks of computer instructions, which can be organized as objects, procedures, or functions. However, the executable files of the identified module do not need to be physically located together, but can include different instructions stored in different locations, the different instructions including the module and achieving the purpose of the module when logically combined together.

[0268] In fact, a module of executable code can be a single instruction or many instructions, and can even be distributed across several different code segments, different programs, and across several memory devices. Similarly, the operating data can be identified and shown within the module herein, and can be embodied in any suitable form and organized within any suitable type of data structure. The operating data can be collected as a single data set, or can be distributed across different locations, including on different storage devices. A module can be passive or active, including an agent operable to perform a desired function.

[0269] The techniques described herein can also be stored on a computer-readable storage medium, which includes volatile and non-volatile, removable and non-removable media implemented using any technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, non-transitory machine-readable storage media such as RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other computer storage media that can be used to store the desired information and the techniques described.

[0270] The devices described herein may also include communication connections or networking means and networking connections that allow the devices to communicate with other devices. A communication connection is an example of a communication medium. A communication medium typically embodies computer-readable instructions, data structures, program modules, and other data in a modulated data signal such as a carrier wave or other transmission mechanism, and includes any information delivery medium. A "modulated data signal" is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct wired connection, and wireless media such as acoustic, radio frequency, infrared, and other wireless media. As used herein, the term computer-readable medium includes communication media.

[0271] The foregoing may be better understood in view of the following clauses

[0272] 1. A method, comprising:

[0273] Determine a plurality of spatial sub - divisions associated with a multi - dimensional virtual environment hosted by a distributed computing system;

[0274] Determine a mapping between a plurality of data objects and a plurality of processing partitions associated with the plurality of spatial sub - divisions, wherein the mapping is associated with a spatial index of the plurality of spatial sub - divisions, and the spatial index of the plurality of spatial sub - divisions represents the plurality of data objects grouped into the plurality of processing partitions using spatial location information associated with the plurality of data objects;

[0275] Identify a plurality of hardware hosts in the distributed computing system to process the plurality of data objects;

[0276] Generate a plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts in the distributed computing system using the spatial index associated with the mapping, wherein the plurality of processing partition assignments allocate the plurality of processing partitions to the plurality of hardware hosts using the spatial index; and

[0277] Send the plurality of processing partition assignments to the plurality of hardware hosts so that the plurality of hardware hosts can process the assigned plurality of data objects using spatial locality.

[0278] 2. The method according to clause 1, further comprising:

[0279] Receive a configuration associated with the multi - dimensional virtual environment hosted by the distributed computing system, wherein the configuration identifies a tree structure for representing the plurality of sub - divisions that subdivide the space associated with the multi - dimensional virtual environment; and

[0280] Use the plurality of spatial sub - divisions to determine the tree structure, wherein a plurality of nodes in the tree structure represent the spatial index and include metadata associating the plurality of data objects with the plurality of processing partitions.

[0281] 3. The method according to clause 1 or 2, further comprising:

[0282] Identify a first hardware host among the plurality of hardware hosts to process a first data object grouped at a first location in a first spatial sub - division among the plurality of spatial sub - divisions, the first data object being mapped to a first processing partition among the plurality of processing partitions; and

[0283] Identify a second hardware host among the plurality of hardware hosts to process a second data object grouped at a second location in a second spatial sub - division among the plurality of spatial sub - divisions, the second data object being mapped to a second processing partition among the plurality of processing partitions; and

[0284] Send the first processing partition and the second processing partition to the first hardware host and the second hardware host.

[0285] 4. The method according to clause 1 or 2, further comprising:

[0286] Identify a hardware host among the plurality of hardware hosts to process a first data object grouped by a first location in a first spatial sub-division of the plurality of spatial sub-divisions, the first data object being mapped to a first processing partition among the plurality of processing partitions;

[0287] Identify the hardware host to process a second data object grouped by a second location in a second spatial sub-division of the plurality of spatial sub-divisions, the second data object being mapped to a second processing partition among the plurality of processing partitions, wherein the second data object satisfies a spatial relationship with the first data object; and

[0288] Send the first processing partition and the second processing partition to the hardware host.

[0289] 5. The method according to clauses 1 to 4, further comprising:

[0290] Receive a change to the mapping between the plurality of data objects and the plurality of processing partitions, wherein the change is caused by a split or merge associated with the spatial index;

[0291] Determine that the change affects the plurality of processing partition assignments;

[0292] Use the change to update the plurality of processing partition assignments to generate the updated plurality of processing assignments; and

[0293] Send the updated plurality of processing assignments to the plurality of hardware hosts to update the processing allocation in the distributed computing system.

[0294] 6. A method, comprising:

[0295] Determine a plurality of spatial sub-divisions in a multi-dimensional virtual environment hosted by a distributed computing system;

[0296] Determine a mapping between a plurality of data objects and a plurality of processing partitions associated with the plurality of spatial sub-divisions, wherein the processing partitions are associated with processing applications that process data objects grouped using spatial location information associated with the plurality of data objects;

[0297] Identify a hardware host in the distributed computing system that processes the data objects grouped by a location associated with a spatial sub-division among the plurality of spatial sub-divisions, the data objects being mapped to the processing partitions; and

[0298] Send the processing partition to the hardware host to process the data object, where the hardware host includes an application instance of the processing application for processing the data object grouped by the location.

[0299] 7. The method according to clause 6, further comprising:

[0300] Receiving the plurality of data objects associated with the multi-dimensional virtual environment hosted by the distributed computing system, where the plurality of data objects include a plurality of object fields providing the spatial location information; and

[0301] Sending the plurality of processing partitions to the hardware host to process the data object.

[0302] 8. The method according to clause 6 or 7, further comprising:

[0303] Receiving a configuration associated with the multi-dimensional virtual environment hosted by the distributed computing system, where the configuration identifies a tree structure for using the plurality of spatial sub-divisions to represent the space associated with the multi-dimensional virtual environment; and

[0304] Using the plurality of spatial sub-divisions to determine the tree structure, where the nodes in the tree structure include metadata that associates the data object grouped by the location to the processing partition.

[0305] 9. The method according to clauses 6 to 8, further comprising:

[0306] Receiving a configuration associated with the processing application, where the configuration provides the capacity of the processing application to process a plurality of data objects at a given location; and

[0307] Using the capacity of the processing application to process a plurality of data objects at a given location to determine the size of the processing partition.

[0308] 10. The method according to clauses 6 to 9, further comprising:

[0309] Identifying a second hardware host in the distributed computing system to process a second data object grouped by a second location, where the second data object is mapped to a second processing partition; and

[0310] Sending the second processing partition to the second hardware host to process the second data object grouped by the second location, where the second hardware host includes a second application instance of a second processing application to process the second data object grouped by the second location.

[0311] 11. The method according to clauses 6 to 9, further comprising:

[0312] Identify the hardware host to process second data objects grouped by a second location, where the second data objects satisfy a spatial relationship with the data objects grouped by the location; and

[0313] Send a second processing partition to the hardware host to process the second data objects grouped by the second location to juxtapose related data, where the hardware host includes a second application instance of a second processing application to process the second data objects.

[0314] 12. The method according to clauses 6 to 11, further comprising:

[0315] Monitor a plurality of metrics associated with a plurality of hardware hosts in the distributed computing system;

[0316] Use the plurality of metrics to determine a plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts; and

[0317] Send the plurality of processing partition assignments to the hardware hosts.

[0318] 13. The method according to clauses 6 to 11, further comprising:

[0319] Receive a change to the mapping between the plurality of data objects and the plurality of processing partitions, where the change includes increasing or decreasing the number of data objects in the plurality of sub - divisions; and

[0320] Use the change to determine a plurality of processing partition assignments; and

[0321] Send the plurality of processing assignments to a plurality of hardware hosts to update the processing allocation in the distributed computing system.

[0322] 14. The method according to clauses 6 to 11, further comprising:

[0323] Receive a change to the mapping between the plurality of data objects and the plurality of processing partitions, where the change includes allocating a data object from a first processing partition to a second processing partition in response to the movement of the data object in the multi - dimensional virtual environment; and

[0324] Use the change to determine a plurality of processing partition assignments; and

[0325] Send the plurality of processing assignments to a plurality of hardware hosts so that the plurality of hardware hosts can use spatial locations to process the allocated data objects.

[0326] 15. The method according to clauses 6 to 11, further comprising:

[0327] Determine a change to the mapping between the multiple data objects and the multiple processing partitions, where changing the data includes allocating a data object from a first processing partition to a second processing partition and to a third processing partition; and

[0328] Use the change to update the multiple processing partition assignments to remove the first processing partition and add the second processing partition and the third processing partition.

[0329] 16. The method according to clauses 6 to 11, further comprising:

[0330] Determine a change to the mapping between the multiple data objects and the multiple processing partitions, where the change includes allocating the data objects mapped to the second processing partition and the data objects mapped to the third processing to the first processing partition; and

[0331] Use the change to update the multiple processing partition assignments to add the first processing partition and remove the second processing partition and the third processing partition.

[0332] 17. A system, comprising:

[0333] One or more processors; and

[0334] One or more memory devices that store instructions that, when executed by the one or more processors, cause the one or more processors to:

[0335] Receive multiple spatial sub - divisions associated with an object - based model representing a multi - dimensional virtual environment hosted by a distributed computing system;

[0336] Monitor the mapping between multiple data objects and multiple processing partitions associated with the multiple spatial sub - divisions, where the multiple processing partitions use spatial location information associated with the multiple data objects to group the multiple data objects at locations associated with the multiple spatial sub - divisions of a processing application for processing the multiple data objects;

[0337] Identify multiple hardware hosts in the distributed computing system to process the multiple data objects;

[0338] Use the mapping to determine multiple processing partition assignments between the multiple processing partitions in the distributed computing system and the multiple hardware hosts, where the multiple processing partition assignments allocate the multiple data objects grouped into the multiple processing partitions associated with the multiple spatial sub - divisions to the multiple hardware hosts; and

[0339] Send the multiple processing partition assignments to the multiple hardware hosts to manage the processing allocation of the multiple data objects.

[0340] 18. The system as described in clause 17, wherein the instructions further cause the one or more processors to:

[0341] Receive a configuration associated with the object-based model, the object-based model representing the multi-dimensional virtual environment hosted by the distributed computing system, wherein the configuration identifies a tree structure for representing a plurality of sub-divisions for subdividing a space associated with the multi-dimensional virtual environment; and

[0342] Use the plurality of spatial sub-divisions to determine the tree structure, wherein a plurality of nodes in the tree structure represent the plurality of processing partitions and include metadata associating the plurality of data objects with the plurality of processing partitions.

[0343] 19. The system as described in clause 17 or 18, wherein the instructions further cause the one or more processors to:

[0344] Determine a change to the mapping between the plurality of data objects and the plurality of processing partitions, wherein the change includes allocating data objects mapped to a first processing partition that groups data objects by a first position to a second processing partition that groups data objects by a second position and a third processing partition that groups data objects by a third position;

[0345] Update the plurality of processing partition assignments to remove the first processing partition and add the second processing partition and the third processing partition; and

[0346] Send the plurality of processing assignments to the plurality of hardware hosts to manage processing allocation in the distributed computing system.

[0347] 20. The system as described in clause 17 or 18, wherein the instructions further cause the one or more processors to:

[0348] Determine a change to the mapping between the plurality of data objects and the plurality of processing partitions, wherein the change includes allocating to a first processing partition that groups data objects by a first position, data objects mapped to a second processing partition that groups data objects by a second position and a third processing partition that groups data objects by a third position;

[0349] Update the plurality of processing partition assignments to add the first processing partition and remove the second processing partition and the third processing partition; and

[0350] Send the plurality of processing assignments to the plurality of hardware hosts to manage processing allocation in the distributed computing system.

[0351] 21. A method, comprising:

[0352] Receiving a plurality of data objects associated with an object-based model representing a multi-dimensional virtual environment hosted by a distributed computing system;

[0353] Identifying a plurality of object types associated with the plurality of data objects;

[0354] Using the plurality of object types to map the plurality of data objects to a plurality of processing partitions to distribute computing operations by object type within the distributed computing system, wherein the mapping includes a plurality of object type indexes representing sets of data objects grouped by object type into the plurality of processing partitions;

[0355] Identifying a plurality of hardware hosts in the distributed computing system to process the plurality of data objects of the object-based model;

[0356] Using the mapping to generate a plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts, wherein the plurality of processing partition assignments specify which of the plurality of object type indexes are assigned to the plurality of hardware hosts; and

[0357] Sending the plurality of processing partition assignments to the plurality of hardware hosts to enable the plurality of hardware hosts to process the plurality of data objects separated using the plurality of processing partitions.

[0358] 22. The method of clause 21, further comprising:

[0359] Receiving a configuration associated with a processing application;

[0360] Using the configuration to determine the capacity of the processing application to process a plurality of data objects of a first object type;

[0361] Using the configuration to determine the data correlation between data objects of the first object type and data objects of a second object type;

[0362] Using the capacity of the processing application to generate the mapping between the plurality of processing partitions and the plurality of data objects; and

[0363] Using the data correlation between the data objects of the first object type and the data objects of the second object type to generate the plurality of processing partition assignments.

[0364] 23. The method of clause 21 or 22, further comprising:

[0365] Determine a change to the mapping between the plurality of processing partitions and the plurality of data objects to obtain an updated mapping, wherein the change to the mapping splits or merges some of the object type indexes in the plurality of object type indexes;

[0366] Use the updated mapping to determine updated processing assignments between an updated plurality of processing partitions and the plurality of hardware hosts; and

[0367] Send the updated processing assignments to the plurality of hardware hosts to modify processing in the plurality of hardware hosts.

[0368] 24. The method according to clauses 21 to 23, further comprising:

[0369] Monitor a plurality of metrics associated with the plurality of hardware hosts;

[0370] Receive data associated with the plurality of metrics; and

[0371] Use the data associated with the plurality of metrics to determine processing assignments between the plurality of processing partitions and the plurality of hardware hosts.

[0372] 25. The method according to clauses 21 to 24, further comprising:

[0373] Use the plurality of metrics to determine to migrate at least one of the plurality of processing partitions between a first hardware host and a second hardware host;

[0374] Determine updated processing assignments between the plurality of processing partitions and the plurality of hardware hosts by assigning the at least one of the plurality of processing partitions to the second hardware host; and

[0375] Send the updated processing assignments to the plurality of hardware hosts.

[0376] 26. A method, comprising:

[0377] Determine a plurality of object types associated with a plurality of data objects in a multi-dimensional virtual environment hosted by a distributed computing system;

[0378] Generate a mapping between the plurality of data objects and a plurality of processing partitions using the plurality of object types, wherein a processing partition in the plurality of processing partitions is associated with a processing application for processing data objects of a processing object type;

[0379] Identify hardware hosts in the distributed computing system that process data objects of the object type, the data objects being mapped to the processing partitions; and

[0380] Send the processing partition to the hardware host to process the data object of the object type, where the hardware host includes an application instance of the processing application for processing the data object of the object type in the processing partition.

[0381] 27. The method according to clause 26, further comprising:

[0382] Receiving the plurality of data objects from an object data repository associated with an object-based model representing the multi-dimensional virtual environment;

[0383] Determining a plurality of object fields associated with the plurality of data objects, where the plurality of object fields specify the plurality of object types; and

[0384] Generating the mapping between the plurality of data objects and the plurality of processing partitions using the plurality of object types specified by the plurality of object fields.

[0385] 28. The method according to clauses 26 to 27, further comprising:

[0386] Using the mapping to determine a plurality of processing partition assignments between the plurality of processing partitions and a plurality of hardware hosts in the distributed computing system;

[0387] Identifying the hardware hosts using the plurality of processing assignments to process the data objects mapped to the processing partitions; and

[0388] Sending the plurality of processing partition assignments to the hardware hosts.

[0389] 29. The method according to clauses 26 to 28, further comprising:

[0390] Monitoring a plurality of metrics associated with the plurality of hardware hosts; and

[0391] Using the plurality of metrics to determine the plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts.

[0392] 30. The method according to clauses 26 to 29, further comprising:

[0393] Identifying a second hardware host in the distributed computing system to process a second data object of a second object type mapped to a second processing partition; and

[0394] Sending the second processing partition to the second hardware host to process the second data object of the second object type, where the second hardware host includes a second application instance of a second processing application to process the second data object of the second object type.

[0395] 31. The method as described in clauses 26 to 29 further includes:

[0396] identifying the hardware host to process the relevant data objects of the second object type, where the relevant data objects satisfy the data correlation with the data objects mapped to the processing partition; and

[0397] sending a second processing partition to the hardware host to process the relevant data objects mapped to the second processing partition to co-locate the relevant data, where the hardware host includes a second application instance of a second processing application to process the relevant data objects of the second object type.

[0398] 32. The method as described in clauses 26 to 31 further includes:

[0399] receiving a change to the mapping between the multiple data objects and the multiple processing partitions, where a data object is added to or removed from the multiple data objects;

[0400] determining that the change affects the multiple processing partition assignments between the multiple processing partitions and the multiple hardware hosts in the distributed computing system;

[0401] updating the multiple processing partition assignments between the multiple processing partitions and the multiple hardware hosts to generate updated multiple processing assignments; and

[0402] sending the updated multiple processing assignments to the multiple hardware hosts to update the processing allocation in the distributed computing system.

[0403] 33. The method as described in clauses 26 to 32, where generating the mapping between the multiple data objects and the multiple processing partitions using the multiple object types further includes generating the mapping using multiple data objects of the object types defined by the capacity of the processing application in the configuration of the processing application.

[0404] 34. The method as described in clauses 26 to 33 further includes generating multiple processing partition assignments between the multiple processing partitions and the multiple hardware hosts in the distributed computing system using the capacity of the hardware and software resources of the multiple hardware hosts.

[0405] 35. The method as described in clauses 26 to 29 or 31 to 34 further includes:

[0406] monitoring multiple metrics associated with the hardware host;

[0407] using the multiple metrics to determine migrating the processing partition to a second hardware host; and

[0408] Send the processing partition to the second hardware host to process the data objects of the object type, where the second hardware host includes a second application instance of the processing application to process the data objects of the object type in the processing partition.

[0409] 36. A system, comprising:

[0410] One or more processors; and

[0411] One or more memory devices storing instructions that, when executed by the one or more processors, cause the one or more processors to:

[0412] Identify a plurality of data objects associated with an object-based model representing a 3D virtual environment hosted by a distributed computing system;

[0413] Use a plurality of object types associated with the plurality of data objects to determine a mapping between the plurality of data objects and a plurality of processing partitions, where the plurality of processing partitions group the plurality of data objects by object type;

[0414] Identify a plurality of hardware hosts in the distributed computing system to process the plurality of data objects;

[0415] Use the mapping to determine a plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts, where the plurality of processing partition assignments allocate the plurality of data objects grouped by object type in the plurality of processing partitions to the plurality of hardware hosts; and

[0416] Send the plurality of processing partition assignments to the plurality of hardware hosts to manage the plurality of hardware hosts to process the plurality of data objects using the plurality of object types.

[0417] 37. The system according to clause 36, wherein the instructions further cause the one or more processors to:

[0418] Receive a configuration associated with a plurality of processing applications for processing the plurality of data objects using the plurality of object types;

[0419] Use the configuration to determine the capacity of the plurality of processing applications to process the plurality of data objects; and

[0420] Generate the mapping between the plurality of processing partitions and the plurality of data objects using the capacity of the plurality of processing applications.

[0421] 38. The system according to clause 36 or 37, wherein the instructions further cause the one or more processors to:

[0422] Monitor utilization of hardware or software resources by the plurality of hardware hosts in the distributed computing system;

[0423] Receive data associated with utilization of the hardware or software resources by the plurality of hardware hosts; and

[0424] Use the data associated with utilization of the hardware or software resources by the plurality of hardware hosts to determine the processing partition assignment to re - allocate utilization of the hardware or software resources of the plurality of hardware hosts.

[0425] 39. The system as recited in clauses 36 to 38, wherein the instructions further cause the one or more processors to:

[0426] Identify a first hardware host to process a first data object of a first object type, the first data object being mapped to a first processing partition among the plurality of processing partitions;

[0427] Identify the first hardware host to process a second data object of a second object type, the second data object being mapped to a second processing partition among the plurality of processing partitions, wherein the second data object satisfies data correlation with the first data object; and

[0428] Use the data correlation to assign the plurality of processing partitions to the plurality of hardware hosts.

[0429] 40. The system as recited in clauses 36 to 39, wherein the instructions further cause the one or more processors to:

[0430] Receive a change to the mapping between the plurality of data objects and the plurality of processing partitions, wherein the change results in an update to the plurality of data objects;

[0431] Use the update to the plurality of data objects to modify the plurality of processing partition assignments between the updated plurality of processing partitions and the plurality of hardware hosts to generate updated plurality of processing assignments; and

[0432] Send the updated plurality of processing assignments to the plurality of hardware hosts to modify the assignment of processing partitions.

[0433] 41. A method, comprising:

[0434] Receiving, by a distributed shared memory of a computing hub, a data object associated with an object - based model of a virtual environment hosted by a distributed computing system;

[0435] Writing a representation of a data object to a memory location associated with a memory device of the computing hub using the distributed shared memory, wherein the representation of the data object includes an object data section and a virtual table that maps a plurality of object fields associated with the data object to the object data section;

[0436] Receiving a request to modify the data object from an instance of a processing application on the computing hub;

[0437] Based on the request, identifying the object fields associated with the changes to the data object caused by the request;

[0438] Writing the changes to the data object caused by the request to a log record appended to a log section of the data object; and

[0439] Writing a modified field table to the log record, the modified field table including references indicating that the changes to the data object in the log record reflect changes to the object fields associated with a portion of the object data section.

[0440] 42. The method of clause 41, further comprising:

[0441] Receiving a plurality of requests to modify data associated with the data object;

[0442] Determining to aggregate the plurality of requests using the plurality of changes to the data object caused by the plurality of requests during an execution cycle in the distributed computing system; and

[0443] Appending additional log records to the log section using the plurality of changes to the data object; wherein the additional log records include the plurality of changes to the data object.

[0444] 43. The method of clause 41 or 42, further comprising:

[0445] Identifying a plurality of log records of the log section appended to the representation of the data object;

[0446] Based on storage or network conditions, determining to merge the plurality of log records into a single log record; and

[0447] Writing additional log records to the log section using the plurality of log records to replace the plurality of log records, wherein the additional log records include a section indicating which changes to the data object associated with the plurality of log records reflect a change to the representation of the data object.

[0448] 44. The method of clauses 41 to 43, further comprising:

[0449] Receive an instruction to send the current state of the data object to a second distributed shared memory of a second computing hub in the distributed computing system;

[0450] Determine whether the second distributed shared memory includes a previous state of the data object;

[0451] When the second distributed shared memory includes the previous state of the data object, send a portion of the log segment of the representation of the data object to the second distributed shared memory, where the portion of the log segment represents the difference between the current state and the previous state of the data object; and

[0452] When the second distributed shared memory does not include the previous state of the data object, send the current state of the data object to the second distributed shared memory using the representation of the data object and a single log record that combines the log segments of the representation of the data object.

[0453] 45. The method according to clauses 41 to 44, further comprising:

[0454] Retrieve the data object from an object data repository to obtain retrieved data;

[0455] Determine whether the format of the retrieved data is used by the distributed shared memory;

[0456] When the format of the retrieved data is used by the distributed shared memory, perform a memory copy of the retrieved data to a memory location; and

[0457] When the format of the retrieved data is not used by the distributed shared memory, use the retrieved data to write to the object data segment of the representation of the data object.

[0458] 46. A method, comprising:

[0459] Receive a data object at a distributed shared memory;

[0460] Use the distributed shared memory to write a representation of the data object to a memory device;

[0461] Receive a request to modify data associated with the data object;

[0462] Determine a portion of the representation of the data object associated with a change to the data object caused by the request; and

[0463] Write a log record to the memory device in a log segment associated with the representation of the data object using the change to the data object, where the log record includes a reference indicating that the change to the data object reflects a change associated with the portion of the representation of the data object.

[0464] 47. The method according to clause 46, further comprising:

[0465] Receiving a plurality of requests to modify data associated with the data object;

[0466] Determining to aggregate the plurality of requests using a plurality of changes to the data object associated with the plurality of requests;

[0467] Determining a plurality of portions of the representation of the data object associated with the plurality of changes to the data object; and

[0468] Appending additional log records to the log segment using the plurality of changes, where the additional log records include a plurality of references indicating that the plurality of changes to the data object reflect changes to the plurality of portions of the representation of the data object.

[0469] 48. The method according to clause 46 or 47, further comprising:

[0470] Determining to merge a plurality of log records into a single log record based on storage or network conditions; and

[0471] Writing an additional log record to the log segment using the plurality of log records to replace the plurality of log records.

[0472] 49. The method according to clauses 46 to 48, further comprising:

[0473] Receiving an instruction to send the current state of the data object to a second computing hub; and

[0474] Sending the current state of the data object to the second computing hub using the representation of the data object.

[0475] 50. The method according to clause 49, further comprising:

[0476] Determining that the second computing hub includes a previous state of the data object; and

[0477] Sending a portion of the log segment of the representation of the data object to the second computing hub, where the portion of the log segment represents the difference between the current state and the previous state of the data object.

[0478] 51. The method as described in clause 49 further includes:

[0479] Determining that the second computing hub does not include a previous state of the data object;

[0480] Merging the log segments to generate a single log record representing the current state of the data object; and

[0481] Sending the representation of the data object and the single log record representing the current state of the data object to the second computing hub.

[0482] 52. The method as described in clauses 46 to 51 further includes:

[0483] Identifying object fields associated with the data object, where the object fields correspond to the changed portions of the representation of the data object;

[0484] Writing a conversion table to the log record, the conversion table mapping the object fields to the portion of the log record that includes the changes to the data object; and

[0485] Writing a modified field table to the log record, the modified field table including references indicating that the mutations of the data object reflect changes associated with the portion of the representation of the data object.

[0486] 53. The method as described in clauses 46 to 52 further includes:

[0487] Receiving the data object from an object data repository to obtain a file;

[0488] Determining that the format of the file is used by the distributed shared memory of the computing hub; and

[0489] Writing the file to a memory location of the memory device associated with the distributed shared memory.

[0490] 54. The method as described in clauses 46 to 52 further includes:

[0491] Receiving the data object from an object data repository to obtain retrieved data;

[0492] Determining that the format of the retrieved data is not used by the distributed shared memory of the computing hub;

[0493] Determining multiple object fields associated with the data object;

[0494] Writing the retrieved data to an object data segment of the memory device associated with the representation of the data object; and

[0495] Writing a virtual table to the memory device includes mapping a plurality of object fields to a plurality of references in the data object portion.

[0496] 55. The method according to clauses 46 to 54, further comprising:

[0497] Determining whether a request to modify data associated with the data object matches an object identifier and a version associated with the data object;

[0498] When the request matches the object identifier and the version, writing the log record to the memory device; and

[0499] Sending an error message when the request fails to match the object identifier and the version.

[0500] 56. A system, comprising:

[0501] At least one processor; and

[0502] At least one memory device that stores instructions which, when executed by the at least one processor, cause the at least one processor to:

[0503] Receive an instruction to create a data object in the virtual environment from a distributed shared memory that manages the storage of data objects associated with an object-based model of the virtual environment;

[0504] Write a representation of the data object to a first memory location associated with the at least one memory device, wherein the representation of the data object includes an object data section that includes object data;

[0505] Receive a request to modify the object data from an instance of a processing application;

[0506] Determine a portion of the object data section associated with a change to the data object caused by the request;

[0507] Generate a log record using the change to the data object and a reference indicating that the change to the data object reflects a change associated with the portion of the object data section; and

[0508] Append the log record to a log section associated with the representation of the data object at a second memory location associated with the at least one memory device.

[0509] 57. The system according to clause 56, wherein the instructions further cause the at least one processor to:

[0510] Receive multiple requests to modify data associated with the data object;

[0511] Determine to aggregate the multiple requests using multiple changes to the data object caused by the multiple requests; and

[0512] Append additional log records to the log segment of the representation of the data object using the multiple changes, where the additional log records aggregate the multiple changes to the data object.

[0513] 58. The system according to clause 56 or 57, wherein the instructions further cause the at least one processor to:

[0514] Determine to merge multiple log records into a single log record based on storage or network conditions; and

[0515] Write an additional log record to the log segment using the multiple log records to replace the multiple log records and meet the storage or network conditions.

[0516] 59. The system according to clauses 56 to 58, wherein the instructions further cause the at least one processor to:

[0517] Receive an instruction to send the current state of the data object to a second distributed shared memory;

[0518] When the second distributed shared memory includes a previous state of the data object, send a portion of the log segment, where the portion of the log segment represents the difference between the current state and the previous state of the data object, to the second distributed shared memory;

[0519] When the second distributed shared memory does not include a previous state of the data object, merge the log segment of the representation of the data object to generate a single log record to represent the current state of the data object; and

[0520] Send the representation of the data object to the second distributed shared memory, where the representation of the data object includes the single log record.

[0521] 60. The system according to clauses 56 to 59, wherein the instructions further cause the at least one processor to:

[0522] Retrieve the data object from an object data repository to obtain retrieved data;

[0523] Determine whether the format of the retrieved data is used by the distributed shared memory;

[0524] When the format of the retrieved data is used by the distributed shared memory, directly copy the retrieved data to the memory location; and

[0525] When the format of the retrieved data is not used by the distributed shared memory, use the retrieved data to write to the object data section of the representation of the data object.

[0526] 61. A method, comprising:

[0527] Receiving, at a first hardware host in a distributed computing system hosting a multi-dimensional virtual environment, a plurality of processing partition assignments, wherein a first processing partition assignment of the plurality of processing partition assignments assigns a first processing partition to the first hardware host, and the first processing partition groups a first plurality of data objects in the multi-dimensional virtual environment by a first object type;

[0528] Determining a second object type on which processing of the first plurality of data objects by a first processing application depends;

[0529] Starting a first instance of the first processing application on the first hardware host to provide processing of the first plurality of data objects mapped to the first processing partition;

[0530] Using the plurality of processing partition assignments to determine a second hardware host assigned a second processing partition, wherein the second hardware host includes a second instance of a second processing application providing processing of a second plurality of data objects of the second object type, the second plurality of data objects being mapped to the second processing partition; and

[0531] Sending, from the first hardware host to the second hardware host, a subscription request, wherein the subscription request instructs the second hardware host to copy the second plurality of data objects to the first instance of the processing application.

[0532] 62. The method of clause 61, further comprising:

[0533] Determining a subscription policy that identifies an adjacency relationship between spatial sub-divisions associated with a plurality of spatial sub-divisions for the multi-dimensional virtual environment to filter data objects of the second object type; and

[0534] Identifying the second processing partition using an adjacency relationship between a first spatial sub-division associated with the first processing partition and a second spatial sub-division associated with the second processing partition that satisfies the subscription policy.

[0535] 63. The method of clause 61, further comprising:

[0536] Determine a subscription policy that identifies query criteria associated with a query to filter data objects of the second object type; and

[0537] Identify the second processing partition by matching the second plurality of data objects to the query criteria using the subscription policy.

[0538] 64. The method according to clauses 61 to 63, further comprising:

[0539] Determine a subscriber list for the first plurality of data objects; and

[0540] Send the first plurality of data objects to the subscriber list, wherein the subscriber list includes a third instance of a third processing application on a third hardware host.

[0541] 65. The method according to clauses 61 to 63, further comprising:

[0542] Receive an update assigned to the plurality of processing partitions;

[0543] Use the update to determine a migration of the second processing partition between the second hardware host and a third hardware host, wherein the third hardware host includes a third instance of a third processing application that processes the second plurality of data objects; and

[0544] Send a second subscription request from the first hardware host to the third hardware host to instruct the third hardware host to copy the second plurality of data objects to the first instance of the first processing application.

[0545] 66. A method, comprising:

[0546] Identify an application library at a hardware host in a distributed computing system hosting a multi-dimensional virtual environment, wherein the application library includes a plurality of processing applications that process data objects associated with a plurality of object types;

[0547] Receive a processing partition assigned to the hardware host, wherein the mapping between the data objects and the plurality of processing partitions groups the plurality of data objects into the processing partitions by object type;

[0548] Retrieve from the application library a processing application for the object type associated with the processing partition; and

[0549] Start an instance of the processing application to enable the hardware host to process the plurality of data objects.

[0550] 67. The method according to clause 66, further comprising:

[0551] Determine a second object type on which the processing of the plurality of data objects by the instance of the processing application depends;

[0552] Identify a second processing partition that groups a second plurality of data objects into the second processing partition according to the second object type; and

[0553] Send the second plurality of data objects to the instance of the processing application.

[0554] 68. The method according to clause 67, further comprising:

[0555] Determine a filter for the data objects of the second object type;

[0556] Identify the second processing partition based on the second plurality of data objects that satisfy the filter; and

[0557] Send a subscription request to a second hardware host associated with the second processing partition, where the subscription request instructs the second hardware host to copy the second plurality of data objects to the instance of the processing application.

[0558] 69. The method according to clause 66, further comprising:

[0559] Determine a subscription policy that identifies an adjacency relationship between spatial sub - divisions associated with a plurality of spatial sub - divisions of the multi - dimensional virtual environment to filter the data objects of the second object type according to spatial location;

[0560] Use the adjacency relationship that satisfies the subscription policy between a first spatial sub - division associated with the processing partition and a second spatial sub - division associated with the second processing partition to identify the second processing partition; and

[0561] Send a subscription request to a second hardware host associated with the second processing partition to receive a second plurality of data objects at the instance of the processing application.

[0562] 70. The method according to clause 66, further comprising:

[0563] Determine a subscription policy that identifies query criteria associated with a query of the data objects in the multi - dimensional virtual environment;

[0564] Identify a second processing partition that groups at least one of the data objects that match the query criteria; and

[0565] Send a subscription request to a second hardware host associated with the second processing partition to receive at least one of the data objects that match the query criteria at the instance of the processing application.

[0566] 71. The method according to clause 66, further comprising:

[0567] Receiving a change to the mapping between the data object and the plurality of processing partitions, wherein the change is caused by a movement of the data object in the multi-dimensional virtual environment;

[0568] Using the change to identify a second processing partition associated with the data object; and

[0569] Sending a subscription request to a second hardware host associated with the second processing partition to receive the data object at the instance of the processing application.

[0570] 72. The method according to clauses 66 to 71, further comprising:

[0571] Receiving a first data object event stream associated with the plurality of data objects from the processing application; and

[0572] Using a second data object event stream to send the plurality of data objects to a subscriber list.

[0573] 73. The method according to clauses 66 to 71, further comprising:

[0574] Receiving a data object event stream associated with the plurality of data objects from the processing application;

[0575] Using the data object event stream to determine a change to the mapping between the data object and the plurality of processing partitions; and

[0576] Sending the change to a world manager to enable the world manager to manage the allocation of processing partitions across multiple hardware hosts.

[0577] 74. The method according to clauses 66, 72 or 73, further comprising:

[0578] Identifying at least one of the plurality of data objects grouped into processing partitions by object type;

[0579] Migrating at least one of the plurality of data objects between the processing partition and a second processing partition, the second processing partition grouping a second plurality of data objects by the object type; and

[0580] Starting a second instance of the processing application to enable the hardware host to process the second plurality of data objects.

[0581] 75. The method as described in clause 66, 72 or 73 further includes:

[0582] Determining the number of data objects in a second processing partition, where the second processing partition groups a second plurality of data objects according to the object type;

[0583] Allocating the second plurality of data objects to processing partitions based on the number of data objects; and

[0584] Terminating a second instance of the processing application associated with the second processing partition.

[0585] 76. A system includes:

[0586] One or more processors; and

[0587] One or more memory devices that store instructions which, when executed by the one or more processors, cause the one or more processors to:

[0588] Use a plurality of processing partition assignments associated with a distributed computing system hosting a multi-dimensional virtual environment to determine a first processing application to be launched from an application library using a first processing partition assigned for processing to a first hardware host, where the first processing partition groups a first plurality of data objects according to a first object type;

[0589] Determine a second object type on which the processing of the first plurality of data objects by the first processing application depends;

[0590] Use the plurality of processing partition assignments to determine a second hardware host associated with a second plurality of data objects associated with the second object type, where the second plurality of data objects are grouped into a second processing partition assigned in the plurality of processing partition assignments for processing by the second hardware host; and

[0591] Send a subscription request for the second plurality of data objects to the second hardware host to copy changes to the second plurality of data objects to a first instance of the first processing application.

[0592] 77. The system as described in clause 76, where the instructions further cause the one or more processors to:

[0593] Determine a subscription policy for filtering data objects of the second object type using a spatial location associated with a plurality of spatial sub-divisions for the multi-dimensional virtual environment; and

[0594] Identify the second processing partition using the mapping between data objects and processing partitions in the multi-dimensional virtual environment and the subscription policy to associate the second plurality of data objects with the spatial location.

[0595] 78. The system as described in clause 76, wherein the instructions further cause the one or more processors to:

[0596] Determine a subscription policy for using a query to filter data objects of the second object type; and

[0597] Use the mapping between data objects and processing partitions in the multi-dimensional virtual environment and the subscription policy to identify the second processing partition to match the second plurality of data objects to the query.

[0598] 79. The system as described in clauses 76 to 78, wherein the instructions further cause the one or more processors to:

[0599] Receive a list of subscribers for the first plurality of data objects;

[0600] Receive a data object event stream associated with the first plurality of data objects from the first processing application; and

[0601] Send the first plurality of data objects to the list of subscribers using a second data object event stream.

[0602] 80. The system as described in clauses 76 to 79, further comprising:

[0603] Receive an update assigned to the plurality of processing partitions;

[0604] Use the update to determine a third hardware host associated with the second plurality of data objects; and

[0605] Send a second subscription request for the second plurality of data objects to the third hardware host to copy the second plurality of data objects to the first instance of the first processing application.

[0606] Reference is made to the examples shown in the drawings and specific language is used to describe these examples. However, it will be understood that this is not intended to limit the scope of the technology. Changes and further modifications of the features shown herein and additional applications of the examples shown herein should be considered within the scope of this specification.

[0607] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more examples. In the foregoing description, numerous specific details are provided, such as examples of various different configurations, to provide a thorough understanding of the examples of the described techniques. However, it will be recognized that the techniques may be practiced without one or more of the specific details or with other methods, components, devices, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the techniques.

[0608] Although the subject matter has been described in language specific to structural features and / or operations, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features and operations described above. Rather, the above specific features and acts are disclosed as example forms of implementing the claims. Many modifications and alternative arrangements can be designed without departing from the spirit and scope of the described techniques.

Claims

1. A system for distributed applications, comprising: at least one processor; at least one memory device including a data repository for storing a plurality of data and instructions, the instructions when executed causing the system to: determine a plurality of object types associated with a plurality of data objects in a multi-dimensional virtual environment hosted by a distributed computing system, each object type including data associated with the data object that conceptually describes the data object; generate a mapping between the plurality of data objects and a plurality of processing partitions, wherein a processing partition among the plurality of processing partitions is associated with a processing application for processing data objects of a processing object type, and organize each data object among the plurality of data objects into a processing partition according to the object type of the plurality of data objects; identify a hardware host in the distributed computing system that processes the data objects of the object type, the data objects being mapped to the processing partition; and send the processing partition to the hardware host to process the data objects of the object type, wherein the hardware host includes an application instance of the processing application for processing the data objects of the object type in the processing partition.

2. The system according to claim 1, further comprising instructions that when executed cause the system to: receive the plurality of data objects from an object data repository associated with an object-based model representing the multi-dimensional virtual environment; determine a plurality of object fields associated with the plurality of data objects, wherein the plurality of object fields specify the plurality of object types; and generate the mapping between the plurality of data objects and the plurality of processing partitions using the plurality of object types specified by the plurality of object fields.

3. The system according to claim 1 or 2, further comprising instructions that when executed cause the system to: use the mapping to determine a plurality of processing partition assignments between the plurality of processing partitions and a plurality of hardware hosts in the distributed computing system; identify the hardware host using the plurality of processing partition assignments to process the data objects mapped to the processing partition; and send the plurality of processing partition assignments to the hardware host.

4. The system according to claim 3, further comprising instructions that when executed cause the system to: monitor a plurality of metrics associated with the plurality of hardware hosts; and use the plurality of metrics to determine the plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts.

5. A method for distributed applications, comprising: determine a plurality of object types associated with a plurality of data objects in a multi-dimensional virtual environment hosted by a distributed computing system, each object type including data associated with the data object that conceptually describes the data object; generate a mapping between the plurality of data objects and a plurality of processing partitions, wherein a processing partition among the plurality of processing partitions is associated with a processing application for processing data objects of a processing object type, and organize each data object among the plurality of data objects into a processing partition according to the object type of the plurality of data objects; Identify a hardware host in the distributed computing system that processes the data objects of the object type, where the data objects are mapped to the processing partitions; And Send the processing partitions to the hardware host to process the data objects of the object type, where the hardware host includes an application instance of the processing application for processing the data objects of the object type in the processing partitions.

6. The method according to claim 5, further Comprising: Receive the plurality of data objects from an object data repository associated with an object-based model representing the multi-dimensional virtual environment; Determine a plurality of object fields associated with the plurality of data objects, where the plurality of object fields specify the plurality of object types; And Generate the mapping between the plurality of data objects and the plurality of processing partitions using the plurality of object types specified by the plurality of object fields.

7. The method according to claim 5 or 6, further Comprising: Use the mapping to determine a plurality of processing partition assignments between the plurality of processing partitions and a plurality of hardware hosts in the distributed computing system; Identify the hardware host using the plurality of processing partition assignments to process the data objects mapped to the processing partitions; And Send the plurality of processing partition assignments to the hardware host.

8. The method according to claim 7, further Comprising: Monitor a plurality of metrics associated with the plurality of hardware hosts; And Use the plurality of metrics to determine the plurality of processing partition assignments between the plurality of processing partitions and the plurality of hardware hosts.

9. A non-transitory machine-readable storage medium having instructions thereon that are executed by one or more processors, Comprising: Receive, by a distributed shared memory of a computing hub, data objects associated with an object-based model of a virtual environment hosted by a distributed computing system; Use the distributed shared memory to write a representation of the data object to a memory location associated with a memory device of the computing hub, where the representation of the data object includes an object data segment and a virtual table that maps a plurality of object fields associated with the data object to the object data segment; Receive, from an instance of a processing application on the computing hub, a request to modify the data object; Based on the request, identify an object field associated with a change to the data object caused by the request; Write the change to the data object caused by the request to a log record appended to a log segment of the data object; And Write a modified field table to the log record, where the modified field table includes references indicating that the change to the data object in the log record reflects a change to the object field associated with a portion of the object data segment.

10. The non-transitory machine-readable storage medium according to claim 9, further Comprising: Receive a plurality of requests to modify data associated with the data object; Determine to aggregate the plurality of requests using a plurality of changes to the data object caused by the plurality of requests during an execution cycle in the distributed computing system; And Append additional log records to the log segment using multiple changes to the data object; wherein the additional log records include the multiple changes to the data object.

11. The non-transitory machine-readable storage medium according to claim 9, further comprising: Identify multiple log records of the log segment attached to the representation of the data object; Determine to merge the multiple log records into a single log record based on storage or network conditions; and Write additional log records to the log segment using the multiple log records to replace the multiple log records, wherein the additional log records include a section indicating which changes to the data object associated with the multiple log records reflect a change in the representation of the data object.

12. The non-transitory machine-readable storage medium according to any one of claims 9 to 11, further comprising: Receive instructions to send the current state of the data object to a second distributed shared memory of a second computing hub in the distributed computing system; Determine whether the second distributed shared memory includes a previous state of the data object; When the second distributed shared memory includes the previous state of the data object, send a portion of the log segment of the representation of the data object to the second distributed shared memory, wherein the portion of the log segment represents the difference between the current state and the previous state of the data object; and When the second distributed shared memory does not include the previous state of the data object, send the current state of the data object to the second distributed shared memory using the representation of the data object and a single log record that merges the log segment of the representation of the data object.

13. A method for a distributed application, comprising: Receive multiple processing partition assignments at a first hardware host in a distributed computing system hosting a multi-dimensional virtual environment, wherein a first processing partition assignment in the multiple processing partition assignments assigns a first processing partition to the first hardware host, and the first processing partition groups a first plurality of data objects in the multi-dimensional virtual environment according to a first object type; Determine a second object type on which the processing of the first processing application for the first plurality of data objects depends; Start a first instance of the first processing application on the first hardware host to provide processing for the first plurality of data objects mapped to the first processing partition; Use the multiple processing partition assignments to determine a second hardware host assigned a second processing partition, wherein the second hardware host includes a second instance of a second processing application that provides processing for a second plurality of data objects of the second object type, and the second plurality of data objects are mapped to the second processing partition; and Send a subscription request from the first hardware host to the second hardware host, wherein the subscription request instructs the second hardware host to copy the second plurality of data objects to the first instance of the processing application.

14. The method according to claim 13, further comprising: Determine a subscription policy that identifies adjacency relationships between spatial sub - partitions associated with multiple spatial sub - divisions for the multi - dimensional virtual environment to filter data objects of the second object type; and Identify the second processing partition using an adjacency relationship that satisfies the subscription policy between a first spatial sub - partition associated with the first processing partition and a second spatial sub - partition associated with the second processing partition.

15. The method according to claim 13, further comprising: Determine a subscription policy that identifies query criteria associated with a query to filter data objects of the second object type; and Identify the second processing partition by matching the second plurality of data objects with the query criteria using the subscription policy.

Images