Data storage method, reading method, device and electronic device

Abstract

The embodiments of the present application provide a data storage method, a data reading method, a device, an electronic device, and a computer-readable storage medium, which relate to the field of databases, and can be used, for example, in the field of distributed databases. The method includes: receiving a first key-value pair to be written; synchronizing the first key-value pair in the form of a log entry to the respective logs of a first storage node and at least two second storage nodes through a consistency check protocol; if it is determined that the first key-value pair has been synchronized in the first storage node itself and at least two second storage nodes, the first key-value pair is stored in the cache of the first storage node; when it is determined that the newly synchronized log entry in the local log meets the preset conditions, each key-value pair corresponding to the newly synchronized log entry in the cache is flushed to the cloud as a first row storage format file and a first column storage format file. The embodiments of the present application save storage space and can realize the immediate execution of OLAP queries without waiting for delays.

Description

Technical Field

[0001] The present application relates to the field of database technology. Specifically, the present application relates to a data storage method, a reading method, a device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Hybrid Transactional / Analytical Processing (HTAP) is a database architecture paradigm that aims to merge traditional online transaction processing (OLTP) and analytical processing (OLAP) into the same database system to meet the growing demand for real-time data processing.

[0003] The HTAP database provided by the related technology, whether it is the architecture of cloud database RDS system (OLTP type) + analytical cloud database system (OLAP type), or the architecture of implementing row storage nodes and column storage nodes in one system (such as TiDB database), the synchronization of data from the row storage system (or row storage node TiKV) to the column storage system (or column storage node TiFlash) is an asynchronous replication process, which means that after the data is successfully written to the row storage system (or row storage node), it is not necessarily synchronized to the column storage system (or column storage node), but there is a certain time delay, and the real-time performance of the query cannot be guaranteed.

[0004] In addition, the deployment of distributed databases requires more hardware costs. Usually, the system will shard the data in the cluster, and each data shard will build a three-copy group. Then, at least three copies of each piece of data in the system are stored. This is only the hardware resources consumed in the scenario of supporting OLTP business. If you want to support OLAP business at the same time, you need to add column storage nodes to synchronize data and write it to the column storage engine, which is equivalent to storing at least four copies of the same data, resulting in extremely high storage costs. Summary of the invention

[0005] The embodiments of the present application provide a data storage method, a reading method, a device, an electronic device, a computer-readable storage medium, and a computer program product, which can solve the above-mentioned problems of the prior art. The technical solution is as follows:

[0006] According to a first aspect of an embodiment of the present application, a data storage method is provided, which is applied to a database cluster, wherein the database cluster includes multiple storage nodes, each storage node includes a primary copy or a secondary copy of at least one data shard, and each data shard is a set of at least one key-value pair whose key is within a corresponding value range, and the method is executed by a first storage node among the multiple storage nodes, and the method includes:

[0007] Receiving a first key-value pair to be written, wherein the first storage node includes a primary copy of a data shard corresponding to a key of the first key-value pair;

[0008] The first key-value pair is synchronized in the form of a log entry to the first storage node and the logs of at least two second storage nodes among the plurality of storage nodes through a preset consistency check protocol; the second storage node includes a slave copy of the data shard corresponding to the key of the first key-value pair;

[0009] If it is determined that the first key-value pair is synchronized in the first storage node itself and the at least two second storage nodes, storing the first key-value pair in the cache of the first storage node;

[0010] When it is determined that the newly synchronized log entry in the local log meets the preset conditions, each key-value pair corresponding to the newly synchronized log entry in the cache is flushed to the cloud as a first row storage format file and a first column storage format file.

[0011] According to a second aspect of an embodiment of the present application, a data reading method is provided, which is applied to a database cluster, wherein the database cluster includes multiple storage nodes, each storage node includes at least one data shard, and each data shard is a set of at least one key-value pair whose key is within a corresponding value range, and the method is executed by a first storage node among the multiple storage nodes, and the method includes:

[0012] Receiving first information, wherein the first information is used to indicate a target key of a value to be obtained;

[0013] If it is determined that the local cache includes the second key-value pair corresponding to the target key, obtaining the value of the second key-value pair;

[0014] Among them, the key-value pairs cached in the cache are stored by the first storage node through the data storage method provided by the first aspect.

[0015] According to a third aspect of an embodiment of the present application, a data storage device is provided, which is applied to a database cluster, wherein the database cluster includes multiple storage nodes, each storage node includes a primary copy or a secondary copy of at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, and the device is deployed on a first storage node among the multiple storage nodes, and the device includes:

[0016] A key-value pair receiving module, configured to receive a first key-value pair to be written, wherein the first storage node includes a primary copy of a data shard corresponding to a key of the first key-value pair;

[0017] A log synchronization module, configured to synchronize the first key-value pair in the form of a log entry to the first storage node and the respective logs of at least two second storage nodes among the plurality of storage nodes through a preset consistency check protocol; the second storage node includes a slave copy of the data shard corresponding to the key of the first key-value pair;

[0018] a cache module, configured to store the first key-value pair in a cache of the first storage node if it is determined that the first key-value pair is synchronized in the first storage node itself and the at least two second storage nodes;

[0019] The flushing module is used to flush each key-value pair corresponding to the newly synchronized log entry in the cache to the cloud as a first row storage format file and a first column storage format file when it is determined that the newly synchronized log entry in the local log meets the preset conditions.

[0020] According to a fourth aspect of an embodiment of the present application, a consistent data reading device is provided, which is applied to a database cluster, wherein the database cluster includes multiple storage nodes, each storage node includes at least one data shard, and each data shard is a set of at least one key-value pair whose key is within a corresponding value range, and the device is deployed on a first storage node among the multiple storage nodes, and the device includes:

[0021] An information receiving module, used for receiving first information, wherein the first information is used for indicating a target key of a value to be obtained;

[0022] A cache search module, configured to obtain a value of the second key-value pair if it is determined that the local cache includes the second key-value pair corresponding to the target key;

[0023] Among them, the key-value pairs cached in the cache are stored by the first storage node through the data storage method provided by the first aspect.

[0024] According to a fifth aspect of an embodiment of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the method provided in the first aspect or the second aspect above.

[0025] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the first aspect or the second aspect are implemented.

[0026] According to a seventh aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which implements the steps of the method provided in the first aspect or the second aspect when the computer program is executed by a processor.

[0027] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0028] First, compared with the traditional HTAP database solution, in which data needs to be asynchronously copied from the row storage node to the column storage node, there is a certain delay in the asynchronous replication method, resulting in a certain waiting delay to query the results after the writing is completed. In the present application, all storage nodes, regardless of type, can be refreshed to obtain row storage format files and column storage format files. The data accessor queries the file immediately after the file is successfully written. The storage node queries the row storage file or column storage file in the cloud, so that the OLAP query can be executed immediately without waiting for delay.

[0029] On the second aspect, the embodiment of the present application configures a master copy and multiple slave copies for each data shard, and only the storage node where the master copy of the data shard is located can obtain the column storage file and the row storage file of the data shard, while the storage node where the slave copy is located does not perform a flush operation. Compared with the prior art in which three copies are configured for the row storage node and one copy is configured for the column storage node, the embodiment of the present application only needs to store two copies (row storage format file and column storage format file) in the cloud, and the data storage capacity is reduced by 1 / 2. The row storage file and the column storage file are uploaded to the cloud, and can be directly written after elastic expansion when necessary, without the need to apply for excessive idle storage resources for the storage node in advance, which can also save a lot of storage costs;

[0030] Third, in traditional database systems built based on consistency protocols, each time a proposal is submitted successfully, the data write request is first written to the log, and then the corresponding log entry is applied and written to the stand-alone storage engine, accessing the disk at least twice. The key-value pairs to be written in this application are directly written to the log in the form of log entries, and are immediately applied to the cache after successful synchronization, which can reduce write latency as much as possible. By accumulating a certain number of log entries before flushing, the IO throughput can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in describing the embodiments of the present application.

[0032] Figure 1A A schematic diagram of a database architecture provided for related technologies;

[0033] Figure 1B A schematic diagram of another database architecture provided for related technologies;

[0034] Figure 1C A schematic diagram of another database architecture provided for related technologies;

[0035] Figure 2A A schematic diagram of a database architecture provided in an embodiment of the present application;

[0036] Figure 2B A schematic diagram of a data storage method provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of a process of uploading a file to a first storage layer provided in an embodiment of the present application;

[0038] Figure 4 A schematic diagram of a process for uploading a file to a second storage layer provided in an embodiment of the present application;

[0039] Figure 5 A schematic diagram of splitting data shards provided in an embodiment of the present application;

[0040] Figure 6 A schematic diagram of data shard migration provided in an embodiment of the present application;

[0041] Figure 7 A schematic diagram of an execution flow of multiple storage nodes in a data storage process provided by an embodiment of the present application;

[0042] Figure 8 A flowchart of a data reading method provided in an embodiment of the present application;

[0043] Fig. 9 A flowchart of another data reading method provided in an embodiment of the present application;

[0044] Fig.10 A schematic diagram of the structure of a data storage device provided in an embodiment of the present application;

[0045] Fig.11 A schematic diagram of the structure of a data reading device provided in an embodiment of the present application;

[0046] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0048] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "an" and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we call an element "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here may include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0050] First, several terms involved in this application are introduced and explained:

[0051] 1) Sharding is a database design technique used to split a large database into multiple smaller, more manageable parts, usually called shards. This technique is often used in scenarios with large amounts of data and high concurrent access to improve database performance and scalability. In data sharding, key-value pairs are a common data model that simplifies the data storage and retrieval process.

[0052] 2) Key-Value Pair, a simple data structure consisting of two elements: key and value. A key is a unique identifier used to quickly retrieve the value associated with it. In the context of data sharding, the key-value pair model can be particularly useful because it allows you to decide which data shard the data should be stored on based on the key.

[0053] In key-value-based data sharding, one or more keys (called shard keys or partition keys) are usually selected to determine how to distribute data to different shards. This process usually follows the following steps:

[0054] Step 1: Select sharding keys: First, you need to select one or more keys that can evenly distribute data as sharding keys. The selection of these keys is crucial to the balance and query performance of sharding.

[0055] Step 2, Sharding strategy: Determine which shard the data should be stored on based on the value of the shard key. Common sharding strategies include range sharding, hash sharding, and list sharding. Range sharding means dividing the value range of the shard key into multiple continuous ranges, each range corresponds to a shard, and hash sharding means performing hash calculations on the value of the shard key and then mapping the hash value to the shard. This method ensures that the data is distributed relatively evenly. List sharding means determining the storage location of the data based on the matching relationship between the shard key value and the predefined list.

[0056] Step 3: Data routing: When data needs to be stored or retrieved, the system determines which shard to interact with based on the value of the shard key and the sharding strategy.

[0057] Step 4: Query optimization: When querying data, the system may need to perform query operations across multiple shards. In order to optimize query performance, you can use parallel query, index optimization and other technologies.

[0058] 3) OLTP business: refers to the type of workload that the database system uses to process daily business transactions. These transactions usually include operations such as inserting, updating, deleting, and querying single records or small batches of records. OLTP databases usually optimize transaction processing speed, concurrency, and data consistency to support real-time, highly interactive applications.

[0059] 4) OLAP business: refers to the type of workload that the database system uses to support decision support and data analysis. OLAP systems are designed to perform complex query and analysis operations, such as data aggregation, data mining, multidimensional analysis, etc., to support business intelligence and decision making.

[0060] 5) Row-Based Storage Format refers to the storage of data in row order, that is, all field values ​​in the same row are stored together continuously. This storage method is very common in relational databases, such as MySQL's InnoDB storage engine, which uses row-based storage format.

[0061] 6) Column-Based Storage Format refers to storing data in column order, that is, all data values ​​in the same column are stored together continuously. This storage method is widely used in the field of big data processing and analysis, such as Apache Parquet and Apache ORC.

[0062] 7) In related technologies, the database architecture that supports HTAP services mainly includes the following two types:

[0063] See also Figure 1A , which exemplarily shows a schematic diagram of a database architecture of the related technology. As shown in the figure, OLTP services are provided to the outside through a transactional database RDS (such as PG or MySQL, etc.), and then the data in the RDS is synchronized to an analytical database (such as ClickHouse or Doris), and OLAP services are provided through the analytical database. This solution is a common method for implementing hybrid transaction / analytical processing (HTAP) in the field of distributed databases. Its core idea is to separate the functions of online transaction processing (OLTP) and online analytical processing (OLAP), but maintain the consistency of the two through an asynchronous replication mechanism. This type of solution can provide OLTP and OLAP services to the outside as a whole, but in essence it connects the two systems through data synchronization technology, which has the following disadvantages:

[0064] The operation and maintenance is complex, and the operation and maintenance costs of the two systems are high;

[0065] Synchronizing data from the OLTP system to the OLAP system is an asynchronous replication process. After the data is written to RDS MySQL, it cannot be analyzed and processed immediately on ClickHouse and needs to wait for the data synchronization to complete.

[0066] See also Figure 1B , which exemplarily shows a schematic diagram of another database architecture of the related technology. As shown in the figure, the architecture implements row storage nodes and column storage nodes in a system. The row storage nodes support OLTP requests, and the column storage nodes support OLAP requests. The advantage of this type of solution is that a system can support HTAP services, but it has the following disadvantages:

[0067] (1) Row-based storage is generally configured as three-copy data shards. The additional copies provided by column-based storage nodes (i.e., storage nodes used only to store column-based format files) result in excessively high storage costs overall.

[0068] (2) When storing row-format files, multi-copy data shards use the distributed consensus algorithm Paxos / Raft to synchronize data, while the column-based storage node acts as a Paxos / Raft learner to synchronize data. Because the synchronization of data from the row-based storage node (i.e., the storage node used only to store row-format files) to the column-based storage node is still an asynchronous replication process, after the data is written to the row-based storage node (OLTP), it may not be synchronized to the column-based storage node (OLAP) immediately. There is no way to perform real-time OLAP queries. It is necessary to wait for the data written to the row-based storage node to be successfully replicated to the column-based storage node before the column-based storage node returns the latest query analysis results.

[0069] The data storage method, reading method, device, electronic device, computer-readable storage medium, and computer program product provided in this application are intended to solve the following technical problems in the prior art:

[0070] 1. Asynchronous replication causes delays in OLAP queries

[0071] In the current HTAP database implementation, whether it is the architecture of cloud database RDS system (OLTP type) + analytical cloud database system (OLAP type), or the architecture of implementing row storage nodes and column storage nodes in one system (such as TiDB database), data synchronization from the row storage system (or row storage node TiKV) to the column storage system (or column storage node TiFlash) is an asynchronous replication process, which means that after the data is successfully written to the row storage node, it is not necessarily synchronized to the column storage node, but there is a certain time delay, and the real-time performance of the query cannot be guaranteed.

[0072] Second, deploying a distributed database requires more hardware costs. Usually, the system will shard the data in the cluster. Each data shard uses the Paxos or Raft consensus algorithm to build a three-copy group. At least three copies of each piece of data in the system are stored. This is only the hardware resources consumed in the scenario of supporting OLTP business. If you want to support online data analysis (OLAP) business at the same time, you need to add a column storage node as a Paxos / Raft learner to synchronize data and write it to the column storage engine, which is equivalent to storing at least 4 copies of the same data.

[0073] See also Figure 1C , which exemplarily shows another database architecture diagram provided by the related technology. As shown in the figure, the data storage system includes a computing layer, a metadata layer and a storage layer. The storage layer includes row storage nodes and column storage nodes. The same data shard Region stores 3 copies on the row storage node and one copy on the column storage node. The EBS cloud disk is configured under each storage node. This type of cloud-native distributed database architecture occupies extremely high storage costs. Log entries are stored in three copies, and data is also stored in three copies.

[0074] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0075] Figure 2A An architectural diagram of a database provided for an embodiment of the present application, as shown in the figure, the system architecture of the embodiment of the present application includes a computing layer, a metadata layer (also called a scheduling layer) and a storage layer, wherein the computing layer includes at least one computing node, and the computing node is used to receive SQL data write requests and SQL data processing requests sent by the client. For SQL data write requests, the computing node can convert them into key-value pairs to be written and send them to the storage node of the storage layer. For SQL data processing requests, the computing node can convert them into keys and send them to the storage node of the storage layer.

[0076] The metadata layer includes at least one metadata node, which is used to store various metadata of the database, such as which data shards each storage node includes as primary copies, which data shards have as secondary copies, the value range of the key corresponding to each data shard, the capacity of each storage node, etc.

[0077] The storage layer includes at least one storage node. The present application designs the storage node into a model in which row storage format files and column storage format files exist simultaneously. Each storage node has the same form, and there is no distinction between row storage nodes and column storage nodes. In this way, each storage node can support both OLTP business and OLAP business.

[0078] The present application provides a data storage method, such as Figure 2B As shown, it is applied to Figure 2A In the database cluster shown, the storage node of the embodiment of the present application includes at least one data shard, each data shard is a set of at least one key-value pair whose key is within the corresponding value range. For example, a storage node includes two data shards, one of which is a set of key-value pairs with keys ranging from 1 to 100, and the other is a set of key-value pairs with keys ranging from 101 to 200. Then, when there is a key-value pair to be written with a key of 40, the key-value pair is located in data shard 1. The data storage method of the embodiment of the present application is as follows: Figure 2A The method is executed by a first storage node in the middle storage layer, and includes:

[0079] S101. Receive a first key-value pair to be written.

[0080] The first storage node of the embodiment of the present application can receive the first key-value pair to be written from the computing layer. In the embodiment of the present application, the storage layer includes multiple storage nodes, but the computing layer does not randomly send the first key-value pair to a certain storage node, but allocates it according to the value range of the key corresponding to each data shard included in each storage node, and whether the data shard is a master copy or a slave copy. Specifically, the computing layer of the embodiment of the present application can determine the data shard corresponding to the key of the first key-value pair and the storage node where the master copy of the corresponding data shard is located by interacting with the metadata layer, and send the first key-value pair to the storage node (that is, the first storage node). In other words, the first storage node of the embodiment of the present application includes the master copy of the data shard corresponding to the key of the first key-value pair.

[0081] The embodiment of the present application configures a master copy of a data shard and multiple (for example, 2) slave copies for each data shard. A storage node either includes a master copy of a data shard or a slave copy of a data shard. There is no situation where a storage node includes both a master copy and a slave copy of a data shard. In addition, it should be noted that due to the large number of data shards involved in the database cluster, a storage node may not necessarily include master copies or slave copies of all data shards. For example, if the database cluster involves 4 data shards and there are 3 storage nodes in the database cluster, it can be:

[0082] Storage node 1 has the primary copy of data shard 1, the secondary copy of data shard 2, and the secondary copy of data shard 3, but lacks the primary copy and secondary copy of data shard 4.

[0083] Storage node 2 has a slave copy of data shard 1, a master copy of data shard 2, a slave copy of data shard 3, and a master copy of data shard 4;

[0084] Storage node 3 has a master copy of data shard 3, a slave copy of data shard 1, a slave copy of data shard 2, and a slave copy of data shard 4;

[0085] Storage node 4 has a slave copy of data shard 4, but lacks the master and slave copies of data shards 1 to 3.

[0086] S102. Synchronize the first key-value pair in the form of log entries to the respective logs of the first storage node and at least two second storage nodes among the multiple storage nodes through a preset consistency check protocol.

[0087] This application does not make any specific limitation on the consistency verification protocol, for example, it can be the Raft consistency verification protocol or the Paoxs consistency verification protocol, etc.

[0088] The following uses the Raft consistency check protocol as an example to illustrate the process of synchronizing the first key-value pair to the logs of the first storage node and at least two second storage nodes. It should be understood that the second storage node in the embodiment of the present application includes a slave copy of the data shard corresponding to the key of the first key-value pair. The synchronization of log entries is a key part to ensure the consistency of the state of the first storage node and each second storage node. The Raft algorithm manages the replication and consistency of the log through the leader node (Leader, that is, the first storage node in the embodiment of the present application). The following are the basic steps for log entry synchronization:

[0089] Step 1: Log Append

[0090] The first storage node receives the first key-value pair, and appends the first key-value pair to its own log as a new log entry;

[0091] The first storage node sends an AppendEntries RPC (remote procedure call) to all second storage nodes in parallel to request them to copy the new log entry; the AppendEntries RPC contains the following information:

[0092] Term: the term number of the first storage node as the leader;

[0093] LeaderId: the node ID of the first storage node;

[0094] PrevLogIndex: the index of the previous log entry, used for log matching and consistency checking;

[0095] PrevLogTerm: the term number of the previous log entry;

[0096] Entries[]: List of log entries to append (may be empty, for heartbeat);

[0097] LeaderCommit: The index of a committed log entry known to the leader.

[0098] Step 2 Follower Response

[0099] After receiving the AppendEntries RPC as a follower, each second storage node will check whether the Term in the RPC is greater than or equal to its current Term. If not, it will reject the RPC and update its own Term and possible leader;

[0100] If Term matches, the follower attempts to synchronize its local log with the leader's log, which typically involves deleting all entries after PrevLogIndex in the local log (if they do not match the leader's log) and appending the log entries in Entries[].

[0101] The follower replies with an AppendEntries RPC, telling the leader the log entries that it successfully replicated.

[0102] Step 3: Log consistency

[0103] If the leader's log does not match the log of a follower at PrevLogIndex, the leader decrements PrevLogIndex and resends the AppendEntries RPC until a matching log entry is found or PrevLogIndex becomes 0. This mechanism ensures that the logs of each storage node (in the same term) are completely consistent after PrevLogIndex.

[0104] Step 4: Submit log entry

[0105] Once the leader knows that a log entry has been replicated by a majority of the second storage nodes (including the first storage node itself), it marks the log entry as committed;

[0106] The leader notifies followers which log entries are committed by sending an AppendEntries RPC with a higher LeaderCommit index;

[0107] Follower nodes, upon receiving an AppendEntries RPC containing a higher LeaderCommit, also mark these log entries as committed and can safely apply them to their state machines.

[0108] Through the above steps, the Raft algorithm ensures that all nodes reach consensus on the same log entries.

[0109] In some embodiments, the first storage node may obtain the second storage node where each slave copy of the data shard is located from the metadata layer.

[0110] In some embodiments, the number of first key-value pairs to be written in the present application may be multiple, and each first key-value pair is synchronized to the log as a log entry.

[0111] S103: If it is determined that the first key-value pair has been synchronized in the first storage node itself and the at least two second storage nodes, store the first key-value pair in the cache of the first storage node.

[0112] After determining that the first key-value pair is synchronized between the first storage node itself and the at least two second storage nodes, the first storage node in the embodiment of the present application will store the first key-value pair in the cache of the first storage node, thereby responding to the query request of the data accessor to the first storage node.

[0113] S104. When it is determined that the newly synchronized log entry in the local log meets the preset condition, each key-value pair corresponding to the newly synchronized log entry in the cache is flushed to the cloud as a first row storage format file and a first column storage format file.

[0114] Flush refers to the process of writing data in the memory to a persistent storage device. In the embodiment of the present application, it refers to writing the key-value pairs in the cache to a persistent storage device in the cloud in the form of row storage format files and column storage format files.

[0115] The first storage node in the embodiment of the present application will not flush immediately after a new synchronized log entry appears in the log, but will wait until a certain number of new synchronized log entries accumulate in the log before flushing, which can effectively improve IO throughput.

[0116] The preset conditions of the embodiments of the present application include that the number of newly synchronized log entries exceeds a preset number. For example, when the number of newly synchronized log entries in the log reaches 100, it is deemed to meet the preset conditions. In some embodiments, the preset conditions may also include that the length of time synchronized to the log exceeds a preset length of time. For example, the log entries synchronized by the log center have exceeded 10 minutes, which is deemed to meet the preset conditions.

[0117] In the embodiment of the present application, for each key-value pair corresponding to a newly synchronized log entry, a first row storage format file and a first column storage format file are generated when the file is flushed. The data content contained in the first row storage format file and the first column storage format file corresponding to the same key-value pair is the same, except for the difference between the row storage format and the column storage format. In some embodiments, the row storage format file is a sorted string table (SST) file, and the column storage format file is a Parquet file.

[0118] In some embodiments, after determining that the first row storage format file and the first column storage format file have been uploaded to the cloud, the first storage node may delete the corresponding key-value pair in the local cache, thereby saving storage space.

[0119] First, compared with the traditional HTAP database solution, in which data needs to be asynchronously copied from the row storage node to the column storage node, there is a certain delay in the asynchronous replication method, resulting in a certain waiting delay to query the results after the writing is completed. In the present application, all storage nodes, regardless of type, can be refreshed to obtain row storage format files and column storage format files. The data accessor queries the file immediately after the file is successfully written. The storage node queries the row storage file or column storage file in the cloud, so that the OLAP query can be executed immediately without waiting for delay.

[0120] On the second aspect, the embodiment of the present application configures a master copy and multiple slave copies for each data shard, and only the storage node where the master copy of the data shard is located can obtain the column storage file and the row storage file of the data shard, while the storage node where the slave copy is located does not perform a flush operation. Compared with the prior art in which three copies are configured for the row storage node and one copy is configured for the column storage node, the embodiment of the present application only needs to store two copies (row storage format file and column storage format file) in the cloud, and the data storage capacity is reduced by 1 / 2. The row storage file and the column storage file are uploaded to the cloud, and can be directly written after elastic expansion when necessary, without the need to apply for excessive idle storage resources for the storage node in advance, which can also save a lot of storage costs;

[0121] Third, in traditional database systems built based on consistency protocols, each time a proposal is submitted successfully, the data write request is first written to the log, and then the corresponding log entry is applied and written to the stand-alone storage engine, accessing the disk at least twice. The key-value pairs to be written in this application are directly written to the log in the form of log entries, and are immediately applied to the cache after successful synchronization, which can reduce write latency as much as possible. By accumulating a certain number of log entries before flushing, the IO throughput can be improved.

[0122] Based on the above embodiments, as an optional embodiment, each key-value pair corresponding to the newly synchronized log entry in the cache is flushed to the cloud as a first row storage format file and a first column storage format file, including:

[0123] Creating a first task, wherein the first task is used to instruct to flush each key-value pair in the cache corresponding to the newly synchronized log entry to the cloud as a first row storage format file and a first column storage format file;

[0124] Sending the first task to a pre-started first background thread so that the first background thread executes the first task;

[0125] In response to the first background thread uploading the first row-based format file and the first column-based format file to the first storage layer of the cloud, deleting the key-value pair corresponding to the newly synchronized log entry in the cache.

[0126] In the embodiment of the present application, when the database cluster is started, the first background thread can be started in the background of the first storage node. When the first storage node determines that the newly synchronized log entry in the local log meets the preset conditions, the first storage node creates a first task, and the first task is used to indicate that each key-value pair corresponding to the newly synchronized log entry is flushed to the first row storage format file and the first column storage format file, and the first row storage format file and the first column storage format file are uploaded to the first storage layer, and the first task is sent to the first background thread, and the first background thread specifically performs the flushing operation. Since the first background thread is executed in the background, it will not block the user's data writing.

[0127] It should be noted that the judgment condition for the successful execution of the first task is that the key-value pairs corresponding to the newly synchronized log entries in the local log of the first storage node have been successfully flushed to the first row storage format file and the first column storage format file, and written to the first storage layer; if the execution fails, the first background thread will need to re-execute the first task subsequently.

[0128] In some embodiments, the first storage layer may be a distributed file system (DFS) or an object storage system, which is not specifically limited in the embodiments of the present application.

[0129] See also Figure 3 , which exemplarily shows a schematic diagram of a process of uploading a file to the first storage layer according to an embodiment of the present application, as shown in the figure, including the following steps:

[0130] S201, start the first background thread;

[0131] S202, waiting for a log entry to be written into the log;

[0132] S203, determining whether the newly synchronized log entry in the log meets the preset conditions, if not, returning to step S202, if it meets the preset conditions, executing step S204;

[0133] S204, creating a first task, and delivering the first task to a first background thread;

[0134] S205: The first background thread executes the first task.

[0135] Based on the above embodiments, as an optional embodiment, the cloud further includes a second storage layer, wherein the second storage layer includes a first log-structured merge (LSM) tree for storing row-stored format files and a second LSM tree for storing column-stored format files.

[0136] The embodiment of the present application sends the first task to a pre-started first background thread, and then further includes:

[0137] In response to the first storage layer having a first row storage format file and a first column storage format file, creating a second task;

[0138] Sending the second task to a pre-started second background thread so that the second background thread executes the second task;

[0139] In response to the second background thread merging the row-stored format file into the first LSM tree and merging the first column-stored format file into the second LSM tree, deleting the first row-stored format file and the first column-stored format file in the first storage layer.

[0140] It should be understood that, since the present application is to flush the newly synchronized log entries in the log in batches, the number of the first row storage format files and the first column storage format files flushed is often multiple. The present application periodically scans the first storage layer, and as long as the first row storage format file and the first column storage format file are newly added to the first storage layer, a second task will be created, and the second task is used to indicate that the first row storage format file is merged into the first LSM tree, and the first column storage format file is merged into the second LSM tree. In addition, the embodiment of the present application will create a second background thread when the database cluster is started. After the second task is created, the second task is sent to the second background thread, and the second background thread executes the second task. When it is determined that the second background thread merges the first row storage format file into the first LSM tree, and merges the first column storage format file into the second LSM tree, the first row storage format file and the first column storage format file in the first storage layer are deleted.

[0141] See also Figure 4, which exemplarily shows a schematic diagram of a process of uploading a file to the second storage layer according to an embodiment of the present application, as shown in the figure, including the following steps:

[0142] S301, start the second background thread;

[0143] S302, waiting for the first storage layer to generate a first row storage format file and a first column storage format file;

[0144] S303, determining whether the first storage layer has the first row storage format file and the first column storage format file, if yes, executing step S304, if no, returning to executing step S302;

[0145] S304, creating a second task, and delivering the second task to a second background thread;

[0146] S305: The second background thread executes the second task.

[0147] The data in the first data layer and the second data layer of the embodiment of the present application are stored in both rows and columns, that is, the same data has both a corresponding row storage format file and a corresponding column storage format file. The embodiment of the present application can store the row storage format file and the column storage format file in different systems respectively. For example, the DFS and S3 systems each store one type of file. When one of the cloud storage systems fails, it can be restored through the files in the other type of system.

[0148] Based on the above embodiments, as an optional embodiment, for each key-value pair corresponding to the newly synchronized log entry, the first row storage format file and the first column storage format file obtained by flushing the key-value pairs corresponding to the newly synchronized log entry meet the following conditions:

[0149] The key-value pairs corresponding to the same data shard in each key-value pair corresponding to the newly synchronized log entry are flushed to the same first-row storage format file and the same first-column storage format file.

[0150] During the flush operation, the embodiment of the present application can first determine the corresponding data shards of each key-value pair in the cache corresponding to the newly synchronized log entry, and then flush each key-value pair corresponding to the same data shard to the same first row storage format file and the same first column storage format file respectively.

[0151] That is to say, the embodiment of the present application independently refreshes the corresponding first row storage format file and first column storage format file for each data shard. This processing is conducive to the subsequent rapid recovery based on the corresponding relationship between the shards and the files when the file is found to be damaged.

[0152] Furthermore, the embodiment of the present application downloads the first row storage format file and the first column storage format file to the cloud, and then further includes at least one of the following:

[0153] If it is determined that the second row-format file stored in the cloud is damaged, determine the target shard corresponding to the second row-format file, and rewrite the row-format files corresponding to the target shard according to the column-format files corresponding to the target shard, wherein the row-format files obtained by rewriting include the undamaged second row-format file;

[0154] If it is determined that the second column-based format file stored in the cloud is damaged, the target shard corresponding to the second column-based format file is determined, and the column-based format files corresponding to the target shard are rewritten according to the row-based format files corresponding to the target shard, and the column-based format files obtained by rewriting include the undamaged second column-based format file.

[0155] In general, it can be found that the file is damaged during the data reading stage, and when reading data, the data reading request will only be sent to the storage node where the primary copy of the data shard corresponding to the key is located, and will not be sent to the storage node where the secondary copy is located. Since the storage node is based on the row storage format file and column storage format file obtained by refreshing the data shard, the storage node can determine the data shard corresponding to each file. Therefore, when it is determined that the second row storage format file stored in the cloud is damaged, first determine the target shard corresponding to the second row storage format file, and then obtain the respective column storage format files corresponding to the target shard from the cloud, and rewrite the respective row storage format files corresponding to the target shard according to the respective column storage format files corresponding to the target shard. The rewritten row storage format files include the undamaged second row storage format file. The recovery of the damaged column storage format file can be implemented based on the above logic, and the embodiments of this application will not be repeated.

[0156] It should be noted that although when a row / column storage format file of a data slice is damaged, all row / column storage format files of the data slice need to be rebuilt, since the amount of data in a data slice itself is small, the data slice can be rebuilt efficiently.

[0157] Based on the above embodiments, when a user makes rapid and continuous insertions to a table in a database, a hotspot write problem usually occurs in the database. In the storage layer, this is manifested as a rapid insertion operation on a data shard, which causes the data shard to have an obvious hotspot and the size of the data shard to grow rapidly. The embodiment of the present application can solve this problem by using shard splitting (Region split) and shard migration, that is, first splitting the hotspot data shard into two, and then migrating the split data shard to other storage nodes.

[0158] See also Figure 5, which exemplarily shows a schematic diagram of splitting data shards in an embodiment of the present application. As shown in the figure, on the one hand, a data shard in a storage node corresponds to a local log of the storage node, and the log includes various key-value pairs as log entries. The key of each key-value pair has a corresponding data shard. On the other hand, each key-value pair of a data shard will also be stored in the cloud (including the first data layer and the second data layer) in the form of row storage format files and column storage format files. For a data shard A with a key value range of [a,h), it can be split into data shard A[a,d) and data shard B[d,h).

[0159] It should be noted that, on the one hand, in the process of splitting the data shards, the embodiment of the present application does not need to migrate the logs, because the corresponding copies in the data shards before and after the split are on the same storage node, and the log storage engine Rocksdb is shared by a single storage node, so the split process does not need to migrate the logs, but it is necessary to build the cache corresponding to the logs in the new data shards; on the other hand, the split process needs to clone the row storage format files and column storage format files. Since the files in this application are stored in isolation for each shard, the split process needs to clone the files of the original data shards. In order to quickly split successfully, there is no need to scan the key-value pairs of the data layer, and the files of the data layer can be directly hard-linked. When the hard link operation is not supported, the files can be directly copied. After the split is successful, each data shard can recycle the files that do not belong to the scope of the data shard in the background; on the other hand, when the log cache of the new data shard is built and the data layer is cloned successfully, the metadata of the shard is updated, including the value range of the key corresponding to the shard and the version number of the shard.

[0160] On the basis of the above embodiments, the embodiment of the present application determines that the first key-value pair is synchronized with the first storage node itself and the at least two second storage nodes, and further includes:

[0161] The determining that the first key-value pair is synchronized on the first storage node itself and the at least two second storage nodes also includes:

[0162] Update the synchronization index to the entry number of the log entry corresponding to the first key-value pair, wherein the entry number of the synchronization index indicates that the corresponding log entry has been synchronized in the first storage node itself and the at least two second storage nodes;

[0163] The storing the first key-value pair in the cache of the first storage node further includes:

[0164] Updating the application index to the entry number of the log entry corresponding to the first key-value pair, and uploading the updated application index as metadata to the metadata layer; the entry number of the application index indicates that the corresponding log entry has been stored in the cache of the first storage node;

[0165] The step of flushing the key-value pairs corresponding to the newly synchronized log entries in the cache into a row storage format file and a column storage format file further includes:

[0166] The refresh index is updated to the entry number of the log entry corresponding to the latest refreshed key-value pair and uploaded to the metadata layer as metadata, and the entry number of the refresh index indicates that the corresponding log entry has been uploaded to the cloud.

[0167] The first storage node of the embodiment of the present application also needs to update three indexes during the data storage process, namely the synchronization index, application index and refresh index mentioned above. Since each second storage node also executes the step of synchronizing the first key-value pair to the local log in the form of a log entry, each second storage node also needs to update the synchronization index during the data storage process, and after the first storage node uploads the application index to the metadata layer, each second storage node also needs to obtain the application index uploaded by the first storage node from the metadata layer to update the local application index.

[0168] Since the flush operation is performed on the first storage node, the second storage nodes do not need to update the flush index. However, when the first storage node fails, resulting in a master-slave switch in the copies of the data shard, the new first storage node needs to obtain the flush index uploaded by the previous first storage node from the metadata layer, and reply the key-value pairs corresponding to the log entries between the flush index and the application index to the local cache. When the new first storage node receives a data read request, if there is a corresponding value in the local cache, it can be returned directly. If not in the cache, it can be obtained from the cloud.

[0169] For a data shard, shard migration includes the process of adding a replica (add peer) and removing a replica (remove peer). Figure 6, which exemplarily shows a schematic diagram of data shard migration provided by an embodiment of the present application. As shown in the figure, first add a slave copy (shard 4) to another storage node, and then delete the original copy (shard 1). Performing this process three times can migrate all three copies of this data shard to other storage nodes. It should be noted that the embodiment of the present application needs to copy log entries during the shard migration process. When adding a copy, it is necessary to copy the log entries of the node where the master copy is located to the new copy, but it should be noted that only the log entries after the index is refreshed need to be copied. In addition, it is not necessary to copy the files in the cloud during shard migration. Only the storage node corresponding to the master copy will access the cloud, and the storage node where the new copy is a slave copy will not access the cloud. When the new copy is elected as the master copy, it can also directly access the cloud, so this step does not require copying files in the cloud.

[0170] See also Figure 7 , which exemplarily shows a schematic diagram of the execution flow of multiple storage nodes in the data storage process of an embodiment of the present application. As shown in the figure, the figure shows a master copy and two slave copies of a data shard, the master copy corresponds to the first storage node, and the slave copy corresponds to the second storage node. The first storage node synchronizes the first key-value pair in the data shard in the form of a log entry to the first storage node and at least two second storage nodes in the multiple storage nodes through a preset consistency check protocol. After the synchronization is completed, the first key-value pair is stored in the cache of the first storage node. Different column families (Column Famliy, CF) is applied to the memory tables of different column families. When it is determined that the newly synchronized log entry in the log meets the preset conditions, each key-value pair corresponding to the newly synchronized log entry in the cache is flushed to the cloud as a first row storage format file and a first column storage format file. The first storage node needs to update the synchronization index, application index and flush index. For each second storage node, only the synchronization index and application index need to be updated. However, when the slave copy in the second storage node is elected as the master copy, that is, the second storage node is updated to the first storage node, the key-value pairs of the log entries between the flush index and the application index updated by the last first storage node are restored to the local cache.

[0171] It should be noted that the implementation of distributed transactions requires the maintenance of three column families, including the lock column family, the data column family, and the write column family. The flushing process also includes concatenating the data in the three column families into key-value pairs, then decoding the key-value pairs into table schema column data (such as Name: String, age: Int32, city: string), and then writing them into the column storage format file.

[0172] On the basis of the above embodiments, as an optional embodiment, before the first storage node receives the first key-value pair to be written, the data shard corresponding to the key of the first key-value pair included in the first storage node is switched from a slave copy to a master copy. That is to say, the data shard stored by the first storage node in the embodiment of the present application is not always a master copy, but there is a stage of switching from a slave copy to a master copy. In this case, the first storage node receives the first key-value pair to be written, and before that, it also includes:

[0173] The storage node where the primary copy before the switch is located is used as the third storage node, and the first application index and the first flush index uploaded to the metadata layer by the third storage node are obtained;

[0174] The key-value pairs corresponding to each log entry between the first refresh index value and the first application index are stored in a local cache.

[0175] It should be understood that after receiving the key-value pair, the storage node must first synchronize, then cache, and finally flush, so the log number of the synchronization index record is greater than the log number of the application index record, and the log number of the application index record is greater than the log number of the flush index record. When the data sharding of the storage node switches from a slave copy to a master copy, the key-value pairs corresponding to each log entry between the first flush index value and the first application index need to be stored in the local cache.

[0176] The present application also provides a data reading method, which can be applied to Figure 2A The database cluster shown in the figure includes multiple storage nodes, each storage node includes at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, and the method is executed by a first storage node among the multiple storage nodes, such as Figure 8 As shown, the method includes:

[0177] S401, receiving first information, where the first information is used to indicate a target key of a value to be obtained;

[0178] S402: If it is determined that the local cache includes the second key-value pair corresponding to the target key, obtain the value of the second key-value pair.

[0179] The first storage node of the embodiment of the present application can receive the first information sent by the computing layer. The computing layer can receive the data reading request sent by the client and convert it into a target key of the value to be obtained. The computing layer determines the first storage node where the primary copy of the data shard including the target key is located by interacting with the metadata layer.

[0180] After receiving the first information, the first storage node first searches the local cache for the target key. If found, it obtains a second key-value pair including the target key, returns the value of the second key-value pair to the computing layer, and the computing layer further returns the value to the client.

[0181] The method for storing key-value pairs in the cache of the first storage node in the embodiment of the present application can adopt the data storage method provided by the above-mentioned embodiments, and there is no waiting delay.

[0182] Based on the above embodiments, as an optional embodiment, it also includes:

[0183] If it is determined that the local cache does not include the second key-value pair corresponding to the target key, query the cloud for the second key-value pair corresponding to the target key;

[0184] If it is determined that the cloud includes a second key-value pair corresponding to the target key, the value of the second key-value pair is obtained.

[0185] Based on the above embodiments, as an optional embodiment, the first information is further used to indicate a query type, and the query type is an OLAP type or an OLTP type;

[0186] If the query type is an OLTP type, querying the second key-value pair in a row storage format file stored in the first storage layer, or querying the second key-value pair in the first LSM tree;

[0187] If the query type is an OLAP type, the second key-value pair is queried in a column-stored format file stored in the first storage layer, or the second key-value pair is queried in a second LSM tree.

[0188] See also Fig. 9 , which exemplarily shows a flow chart of another data reading method provided in an embodiment of the present application, as shown in the figure, comprising:

[0189] S501, the computing layer receives an SQL data query request initiated by a client;

[0190] S502: The computing layer parses the request to obtain a query request for querying the target key;

[0191] S503: The computing layer and the metadata layer interact to locate the first storage node, and send a query request (including the first information) to the first storage node.

[0192] S504, the first storage node searches for the target key in the local cache. If the target key is found, S508 is executed. If the target key is not found, S505 is executed.

[0193] S505, the first storage node determines whether the query type is an OLTP type, if so, executes step S506a, if not, executes step S506b;

[0194] S506a, querying the second key-value pair in the row storage format file stored in the first storage layer, or querying the second key-value pair in the first LSM tree;

[0195] S506b. Query the second key-value pair in the column storage format file stored in the first storage layer, or query the second key-value pair in the second LSM tree;

[0196] S507, determine whether the target key is found, if the target key is found, execute S508, if not, execute S509;

[0197] S508: Return the value of the second key-value pair to the client through the computing layer, and end the process;

[0198] S509: Return the query failure result to the client through the computing layer, and end the process.

[0199] The present application embodiment provides a data storage device, such as Fig.10 As shown, it is applied to a database cluster, the database cluster includes multiple storage nodes, each storage node includes a master copy or a slave copy of at least one data shard, each data shard is a set of at least one key-value pair whose key is in a corresponding value range, and the device is deployed in the first storage node among the multiple storage nodes. The device may include: a key-value pair receiving module 1001, a log synchronization module 1002, a cache module 1003 and a flush module 1004, wherein,

[0200] A key-value pair receiving module 1001 is used to receive a first key-value pair to be written, wherein the first storage node includes a primary copy of a data shard corresponding to a key of the first key-value pair;

[0201] A log synchronization module 1002, configured to synchronize the first key-value pair in the form of a log entry to the first storage node and the respective logs of at least two second storage nodes among the plurality of storage nodes through a preset consistency check protocol; the second storage node includes a slave copy of the data shard corresponding to the key of the first key-value pair;

[0202] The cache module 1003 is configured to store the first key-value pair in the cache of the first storage node if it is determined that the first key-value pair is synchronized in the first storage node itself and the at least two second storage nodes;

[0203] The flush module 1004 is used to flush each key-value pair corresponding to the newly synchronized log entry in the cache to the cloud as a first row storage format file and a first column storage format file when it is determined that the newly synchronized log entry in the local log meets the preset conditions.

[0204] The data storage device of the embodiment of the present application can execute the data storage method provided by the embodiment of the present application, and the implementation principle is similar. The actions performed by each module in the data storage device of each embodiment of the present application correspond to the steps in the data storage method of each embodiment of the present application. For the detailed functional description of each module of the data storage device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0205] The present application embodiment provides a data reading device, such as Fig.11 As shown, it is applied to a database cluster, the database cluster includes multiple storage nodes, each storage node includes at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, and the device is deployed in a first storage node among the multiple storage nodes, and the device includes:

[0206] The information receiving module 1101 is used to receive first information, where the first information is used to indicate a target key of a value to be obtained;

[0207] The cache search module 1102 is configured to obtain a value of the second key-value pair if it is determined that the local cache includes the second key-value pair corresponding to the target key;

[0208] Among them, the key-value pairs cached in the cache are stored by the first storage node through the above-mentioned various data storage methods.

[0209] The data reading device of the embodiment of the present application can execute the data reading method provided by the embodiment of the present application, and the implementation principle is similar. The actions performed by each module in the data reading device of each embodiment of the present application correspond to the steps in the data reading method of each embodiment of the present application. For the detailed functional description of each module of the data reading device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0210] In an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a data storage method or a data reading method. Compared with the related art, the following can be achieved:

[0211] First, compared with the traditional HTAP database solution, in which data needs to be asynchronously copied from the row storage node to the column storage node, there is a certain delay in the asynchronous replication method, resulting in a certain waiting delay to query the results after the writing is completed. In the present application, all storage nodes, regardless of type, can be refreshed to obtain row storage format files and column storage format files. The data accessor queries the file immediately after the file is successfully written. The storage node queries the row storage file or column storage file in the cloud, so that the OLAP query can be executed immediately without waiting for delay.

[0212] On the second aspect, the embodiment of the present application configures a master copy and multiple slave copies for each data shard, and only the storage node where the master copy of the data shard is located can obtain the column storage file and the row storage file of the data shard, while the storage node where the slave copy is located does not perform a flush operation. Compared with the prior art in which three copies are configured for the row storage node and one copy is configured for the column storage node, the embodiment of the present application only needs to store two copies (row storage format file and column storage format file) in the cloud, and the data storage capacity is reduced by 1 / 2. The row storage file and the column storage file are uploaded to the cloud, and can be directly written after elastic expansion when necessary, without the need to apply for excessive idle storage resources for the storage node in advance, which can also save a lot of storage costs;

[0213] Third, in traditional database systems built based on consistency protocols, each time a proposal is submitted successfully, the data write request is first written to the log, and then the corresponding log entry is applied and written to the stand-alone storage engine, accessing the disk at least twice. The key-value pairs to be written in this application are directly written to the log in the form of log entries, and are immediately applied to the cache after successful synchronization, which can reduce write latency as much as possible. By accumulating a certain number of log entries before flushing, the IO throughput can be improved.

[0214] In an alternative embodiment, an electronic device is provided, such as Fig.12 As shown, Fig.12 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0215] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0216] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 4002 is represented by only one thick line in the figure, but it does not mean that there is only one bus or one type of bus.

[0217] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0218] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0219] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0220] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0221] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0222] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0223] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A data storage method, characterized in that: Applied to a database cluster, the database cluster includes multiple storage nodes, each storage node includes a primary copy or a secondary copy of at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, the method is executed by a first storage node among the multiple storage nodes, and the method includes: Receiving a first key-value pair to be written, wherein the first storage node includes a primary copy of a data shard corresponding to a key of the first key-value pair; The first key-value pair is synchronized in the form of a log entry to the first storage node and the logs of at least two second storage nodes among the plurality of storage nodes through a preset consistency check protocol; the second storage node includes a slave copy of the data shard corresponding to the key of the first key-value pair; If it is determined that the first key-value pair is synchronized in the first storage node itself and the at least two second storage nodes, storing the first key-value pair in the cache of the first storage node; When it is determined that the newly synchronized log entry in the local log meets the preset condition, each key-value pair corresponding to the newly synchronized log entry in the cache is flushed to the cloud as a first row storage format file and a first column storage format file; The flushing refers to writing the key-value pairs in the cache to a persistent storage device in the cloud in the form of a row storage format file and a column storage format file; Each storage node has a row storage format file and a column storage format file, and the row storage format file and the column storage format file of the same data key-value pair contain the same data content.

2. The method according to claim 1, characterized in that The cloud comprises a first storage layer; The step of flushing each key-value pair in the cache corresponding to the newly synchronized log entry to the cloud as a first row storage format file and a first column storage format file includes: Creating a first task, wherein the first task is used to instruct to flush each key-value pair in the cache corresponding to the newly synchronized log entry to the cloud as a first row storage format file and a first column storage format file; Sending the first task to a pre-started first background thread so that the first background thread executes the first task; In response to the first background thread uploading the first row-based format file and the first column-based format file to the first storage layer of the cloud, deleting the key-value pair corresponding to the newly synchronized log entry in the cache.

3. The method according to claim 2, characterized in that The cloud further includes a second storage layer, the second storage layer including a first log structure merge LSM tree storing row storage format files and a second LSM tree storing column storage format files; The step of sending the first task to a pre-started first background thread further includes: In response to the first storage layer having the first row-stored format file and the first column-stored format file, creating a second task, wherein the second task is used to instruct merging the first row-stored format file into the first LSM tree, and merging the first column-stored format file into the second LSM tree; Sending the second task to a pre-started second background thread so that the second background thread executes the second task; In response to the second background thread merging the first row-stored format file into the first LSM tree and merging the first column-stored format file into the second LSM tree, deleting the first row-stored format file and the first column-stored format file in the first storage layer.

4. The method according to claim 2, characterized in that: The preset conditions include that the number of newly synchronized log entries exceeds a preset number, or the duration of synchronization to the log exceeds a preset duration.

5. The method according to claim 1, characterized in that The first row storage format file and the first column storage format file obtained by flushing each key-value pair corresponding to the newly synchronized log entry meet the following conditions: The key-value pairs corresponding to the same data shard in each key-value pair corresponding to the newly synchronized log entry are flushed to the same first-row storage format file and the same first-column storage format file.

6. The method according to claim 5, characterized in that The downloading to the cloud is a first row storage format file and a first column storage format file, and then further includes at least one of the following: If it is determined that the second row-format file stored in the cloud is damaged, determine the target shard corresponding to the second row-format file, and rewrite the row-format files corresponding to the target shard according to the column-format files corresponding to the target shard, wherein the row-format files obtained by rewriting include the undamaged second row-format file; If it is determined that the second column-based format file stored in the cloud is damaged, the target shard corresponding to the second column-based format file is determined, and the column-based format files corresponding to the target shard are rewritten according to the row-based format files corresponding to the target shard, and the column-based format files obtained by rewriting include the undamaged second column-based format file.

7. The method according to claim 1, characterized in that The database cluster also includes a metadata layer; The determining that the first key-value pair has been synchronized at the first storage node itself and the at least two second storage nodes further includes: updating a synchronization index to an entry number of a log entry corresponding to the first key-value pair, wherein the entry number of the synchronization index indicates that the corresponding log entry has been synchronized at the first storage node itself and the at least two second storage nodes; The storing the first key-value pair in the cache of the first storage node further includes: updating the application index to the entry number of the log entry corresponding to the first key-value pair, and uploading the updated application index as metadata to the metadata layer; the entry number of the application index indicates that the corresponding log entry has been stored in the cache of the first storage node; The method of flushing the key-value pairs corresponding to the newly synchronized log entries in the cache into row storage format files and column storage format files also includes: updating the flush index to the entry number of the log entry corresponding to the latest flushed key-value pair as metadata and uploading it to the metadata layer, wherein the entry number of the flush index indicates that the corresponding log entry has been uploaded to the cloud.

8. The method according to claim 7, characterized in that Before the first storage node receives the first key-value pair to be written, the data shard corresponding to the key of the first key-value pair included in the first storage node is switched from a slave copy to a master copy; The receiving of the first key-value pair to be written also includes: The storage node where the primary copy before the switch is located is used as the third storage node, and the first application index and the first flush index uploaded to the metadata layer by the third storage node are obtained; The key-value pairs corresponding to each log entry between the first refresh index and the first application index are stored in a local cache.

9. The method according to claim 7, characterized in that: The database cluster further includes a computing layer, configured to receive a data write request sent by a client, convert the data write request into the first key-value pair, and determine from the metadata layer a first storage node where a primary copy of a data shard corresponding to a key of the first key-value pair is located, and send the first key-value pair to the first storage node; The receiving a first key-value pair to be written includes: Receive the first key-value pair sent by the computing layer.

10. A data reading method, characterized in that: Applied to a database cluster, the database cluster includes multiple storage nodes, each storage node includes at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, the method is executed by a first storage node among the multiple storage nodes, and the method includes: Receiving first information, wherein the first information is used to indicate a target key of a value to be obtained; If it is determined that the local cache includes the second key-value pair corresponding to the target key, obtaining the value of the second key-value pair; The key-value pairs cached in the cache are stored by the first storage node through the data storage method according to any one of claims 1 to 9.

11. The method according to claim 10, characterized in that Also includes: If it is determined that the local cache does not include the second key-value pair corresponding to the target key, query the cloud for the second key-value pair corresponding to the target key; If it is determined that the cloud includes a second key-value pair corresponding to the target key, obtaining a value of the second key-value pair; The key-value pairs cached in the cloud are stored by the first storage node through the data storage method according to any one of claims 1 to 9.

12. The method according to claim 11, characterized in that The first information is also used to indicate a query type, and the query type is an OLAP type or an OLTP type; The query from the cloud includes a second key-value pair corresponding to the target key, including: If the query type is an OLTP type, querying the second key-value pair in a row storage format file stored in the first storage layer, or querying the second key-value pair in the first LSM tree; If the query type is an OLAP type, the second key-value pair is queried in a column-stored format file stored in the first storage layer, or the second key-value pair is queried in a second LSM tree.

13. A data storage device, characterized in that: Applied to a database cluster, the database cluster includes multiple storage nodes, each storage node includes a primary copy or a secondary copy of at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, the device is deployed on a first storage node among the multiple storage nodes, and the device includes: A key-value pair receiving module, configured to receive a first key-value pair to be written, wherein the first storage node includes a primary copy of a data shard corresponding to a key of the first key-value pair; A log synchronization module, configured to synchronize the first key-value pair in the form of a log entry to the first storage node and the respective logs of at least two second storage nodes among the plurality of storage nodes through a preset consistency check protocol; the second storage node includes a slave copy of the data shard corresponding to the key of the first key-value pair; a cache module, configured to store the first key-value pair in a cache of the first storage node if it is determined that the first key-value pair is synchronized in the first storage node itself and the at least two second storage nodes; A flushing module, configured to flush each key-value pair corresponding to the newly synchronized log entry in the cache to the cloud as a first row storage format file and a first column storage format file when it is determined that the newly synchronized log entry in the local log meets a preset condition; The flushing refers to writing the key-value pairs in the cache to a persistent storage device in the cloud in the form of a row storage format file and a column storage format file; Each storage node has a row storage format file and a column storage format file, and the row storage format file and the column storage format file of the same data key-value pair contain the same data content.

14. A data reading device, characterized in that: Applied to a database cluster, the database cluster includes multiple storage nodes, each storage node includes at least one data shard, each data shard is a set of at least one key-value pair whose key is within a corresponding value range, the device is deployed on a first storage node among the multiple storage nodes, and the device includes: An information receiving module, used for receiving first information, wherein the first information is used for indicating a target key of a value to be obtained; A cache search module, configured to obtain a value of the second key-value pair if it is determined that the local cache includes the second key-value pair corresponding to the target key; The key-value pairs cached in the cache are stored by the first storage node through the data storage method according to any one of claims 1 to 9.

15. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 12.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images