Distributed database system

Abstract

To provide a distributed data storage system that obtains a static portion of each client platform.SOLUTION: A distributed data storage system 200 comprises: a database server 220 configured to store key values related to a plurality of data types; and a plurality of client platforms 210. Each of the plurality of client platforms is configured to store data of at least one data type and to store indexing information, and each of the plurality of client platforms implements compilation of each data scheme by a database logic compiler (DBLC) into an object relational mapping (ORM) 216 to obtain a static portion of each client platform.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates generally to the field of databases. In particular, it relates to the location of database logic.

[0002] BST: Binary Search Tree CPU: Central Processing Unit CSDD: Client-Side Distributed Database DBLC: Database Logic Compiler ORM: Object Relational Mapping TTL: Time to Live

Background Art

[0003] In the field of databases, a common approach among those skilled in the art is that the server is a heavy monolithic device. The server is "smart" and maintains all the necessary logic such as indexing, schema enforcement, complex query (join), periodic cleanup routines, shards, replication, etc., while the client entity has thin logic (i.e., contains little logic or is a relatively "dumb" entity in terms of data processing capabilities).

[0004] Such implementations involve many difficulties. a. When each new feature is introduced, the server must be redeployed and sometimes the client entity needs to be updated. b. While client entities can use a subset of server functionality, performance can still degrade if a large number of client entities use the same code path. This difficulty worsens over time as new functionality is added to the server. c. Because client entities are thin entities, they typically do not have fine-grained control over the queries they execute. As a result, client entities cannot, for example, update entries in the database without updating the index.

[0005] Many conventional database systems implement large and scalable database architectures. Various database architectures are chosen to suit specific data requirements. However, as the number of systems supporting various architectures increases, so does the complexity of the database systems. In some cases, managing the database system becomes as complex as the architecture itself, making it impossible for administrators to make changes to large databases.

[0006] In an attempt to address at least some of this complexity, U.S. Patent Publication US10713280 discloses a cloud-distributed database comprising cloud-based resources, a database subsystem running on the cloud-based resources, and a proxy layer configured to control authentication of client systems based on connection strings communicated from client systems connected to the cloud resources. The publication discloses the database as a cloud service that eliminates many design challenges associated with distributed database implementations while allowing client input on configuration choices in database construction. Furthermore, clients can identify a fully functional, scalable, replicated, secure distributed database in the cloud within minutes, based on the number of database nodes, node capabilities, and so on.

[0007] U.S. Patent Application US20130290249 describes a system and method for managing asynchronous replication in a distributed database environment while providing scaling for a distributed database. The solution described in this publication allows clusters of nodes to be assigned roles in managing partitions of data within a database and processing database requests. Each cluster may include a node having a primary role in handling write operations and managing the asynchronous replication of operations to at least one secondary node. Each cluster or set of nodes may host one or more partitions of data in a database. Collectively, the clusters or set of nodes define a shard cluster that hosts all the data in a distributed database. Each shard cluster, individual node, or set of nodes may be configured to manage extensions of the shard cluster to include new systems and / or manage the size of any hosted partitions, isolated database partitions, or migrated partitions.

[0008] This disclosure provides a new solution for delivering a high-performance agile database system. [Overview of the Initiative] [Problems that the invention aims to solve]

[0009] This disclosure can be summarized by referring to the attached claims.

[0010] The purpose of this disclosure is to provide a novel storage system, including a thin, high-performance database server.

[0011] Another objective of this disclosure is to provide a novel storage system that includes new DBLC logic configured to compile a data scheme for low-level server operations, including pre-compiled data such as shard IDs.

[0012] Another objective of this disclosure is to provide a novel memory system in which the main logic of the system resides on the client side.

[0013] Another purpose of this disclosure is to provide a novel storage system that can be configured by the client to implement only the functions that are currently required to be performed.

[0014] Other purposes of this disclosure will become apparent from the following description. [Means for solving the problem]

[0015] A first embodiment of the present disclosure provides a distributed data storage system including a database server configured to store key values ​​related to data belonging to a plurality of data types, wherein the distributed data storage system has a plurality of client platforms, each of the plurality of client platforms configured to store data of at least one data type and to store indexing information, and each of the plurality of client platforms is configured to implement the compilation of its respective data scheme within an object relation mapping (ORM) by a database logic compiler (DBLC), thereby enabling the acquisition of a static portion of each client platform.

[0016] In other words, the proposed system includes a database server that provides a set of low-level high-performance operations while enabling the use of different data types.

[0017] Because the query and database logic are implemented close to the data source (i.e., the client side), optimizations that are not possible in current implementations known to those skilled in the art become possible. This enables higher end-to-end performance and agile feature development without performance degradation over time.

[0018] According to other embodiments, at least one data type is an element of a group consisting of hash maps, sorted sets, lists, binary search trees (BSTs), etc.

[0019] According to other embodiments, a data schema includes information regarding at least one of data types, data relationships, whether and how data should be indexed, and other applicable metadata.

[0020] These data types are used by the client platform to implement both data storage and indexing (using the appropriate data types). This is similar to a central processing unit and a compiler, where the central processing unit (similar to a database server) provides the basic operations, and the compiler (similar to the client platform) is configured to compile the complex application logic of these low-level operations.

[0021] Also, runtime flags enable direct data queries that do not check indexes (e.g., when the exact keys of the data input are known), or additional adaptability to perform non-indexed data storage.

[0022] According to other embodiments, a database logic compiler (DBLC) is configured to calculate a shard ID for each data input during compilation.

[0023] According to other embodiments, all related data inputs are stored on a single shard, and unrelated data is stored on different shards.

[0024] The implementation of this embodiment enables simple horizontal and vertical scaling (where the database server is spread across the same host to achieve horizontal growth and across other servers to achieve vertical scaling).

[0025] According to other embodiments, the periodic functions necessary to maintain the database server are implemented by an additional computer platform.

[0026] Preferably, the periodic functions necessary to maintain the database server, such as cleanup or any other periodic function, are implemented by an additional computer platform (operating as another client not included in the database server), thereby maintaining the database server as thin and fast as possible.

[0027] The accompanying drawings, which form a part of this specification and are incorporated herein, illustrate some embodiments of the present disclosure and, together with the description herein, explain the principles of the embodiments disclosed herein.

Brief Description of the Drawings

[0028] [Figure 1] It is a diagram showing an example of DBLC compilation interpreted according to an embodiment of the present invention. [Figure 2] It is a diagram showing an example of a distributed data storage system interpreted according to an embodiment of the present invention.

Modes for Carrying Out the Invention

[0029] Some of the specific details and values in the following detailed description show specific examples of the present disclosure. However, this description is exemplary and is not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that the claimed system can be implemented by other methods known in the art. Furthermore, the embodiments described herein include different steps, but not all of them are required in all embodiments of the present invention. The scope of the present invention can be summarized by referring to the appended claims.

[0030] One of the fundamental principles of this disclosure is that a distributed data storage system includes a thin, high-performance database server that enables a set of low-level, high-performance operations, and therefore, since the queries and database logic are implemented on the client side, it becomes possible to optimize the operations that need to be performed.

[0031] Figure 1 shows an example of DBLC compilation according to an embodiment of the present invention. According to this embodiment, a client platform (not shown) is configured to implement the compilation of each data scheme (100) within an object relation mapping (ORM) (120) by a database logic compiler (DBLC) (110), thereby enabling the acquisition of a static part for each client platform. As illustrated in this figure, the correct index is selected for the data model. The entity "car" is provided with shard ID:1, which is different from shard ID:0 of ​​the entity "group" or entity "person" included in the same object relation mapping (120). The present invention makes it possible to implement this option because different shards can be used for data schemes independently. This compile-time determination allows data to be sent directly from the client to the correct shard in the cluster, in contrast to a common implementation where the shard ID is calculated at runtime and then the data is redirected to the correct shard in the cluster. This embodiment serves to demonstrate how compile-time determination / optimization affects performance.

[0032] Figure 2 shows an example of a distributed data storage system (200) as interpreted according to embodiments of the present invention, in which the static and dynamic parts are shown.

[0033] The distributed data storage system (200) shown in this embodiment includes a client platform (210) (other client platforms are not shown) and a thin database server (220).

[0034] The client platform (210) comprises two main parts, one of which is a dynamic part (212) which is runtime operation performed based on information received from a user application (214). The other main part is a static part, object relation mapping (216), based on information received from a corresponding data scheme (218). The system operates by transmitting low-level commands from the client platform (210) to a database server (220) and receiving responses from that server.

[0035] Here, consider an example where a client creates a "person" entity (see Figure 1) with the name "Alice". The following processing procedure is implemented: First, "Alice" is added to the index " / person", and then the person entity "Alice" is added to the key-value storage. These low-level instructions are generated using the static part. However, the client can add a flag so that only the addition of the person entity to the key-value storage works without manipulating the index. Preferably, this is done when the client is certain that the index has already been created and only the data update is needed. Applying such a method is effective in dramatically improving performance.

[0036] The present invention is described using a detailed description of embodiments provided merely as examples and not intended to limit the scope of the invention. The embodiments described include different configurations, and not all configurations are required in all embodiments of the invention. Some embodiments of the invention use only some configurations, or possible combinations of configurations. Variations of the embodiments of the invention described, and embodiments of the invention including different combinations of configurations shown in the embodiments described, will be obvious to those skilled in the art. The scope of the invention is limited only by the following claims.

Claims

1. A distributed data storage system comprising a cluster including a plurality of client platforms belonging to the distributed data storage system, and a database server configured to store data of a plurality of data types and performing high-performance operations based on low-level instructions received from the plurality of client platforms, A distributed data storage system characterized in that each of the plurality of client platforms is configured to store data of at least one data type from the plurality of data types, each of the plurality of client platforms is configured to implement the compilation of each data scheme within an object relation mapping (ORM) by a database logic compiler (DBLC) based on information received from the corresponding data scheme, thereby enabling a static portion of each client platform, the static portion of each client platform is used to generate low-level instructions transmitted from each client platform to the database server, thereby enabling high-performance operation performed by the database server.

2. The distributed data storage system according to claim 1, characterized in that the at least one data type is an element of a group consisting of a hash map, a sorted set, a list, and a binary search tree (BST).

3. The distributed data storage system according to claim 1, characterized in that the data scheme includes information on at least one of data types, data relationships, whether the data should be indexed, and how it should be indexed.

4. The distributed data storage system according to claim 1, characterized in that the database logic compiler (DBLC) is configured to calculate a shard ID for each data input being compiled, in order to enable the client platform to send data directly to the correct shard in the cluster.

5. The distributed data storage system according to claim 3, characterized in that all relevant data inputs are sent directly from the client platform to the correct shard in the cluster and stored on a single shard, and irrelevant data is sent directly from the client platform to different shards in the cluster and stored on those different shards, thereby enabling the compilation of a data scheme into a low-level server operation including pre-compiled data.

6. The distributed data storage system according to claim 1, characterized in that the periodic cleanup function necessary for maintaining the database server is implemented by an additional computer platform.

Images