Data writing method and electronic device

Abstract

An embodiment of the present application discloses a data writing method and electronic device, which are applied to the first node of an etcd cluster, the first node being the master node of the etcd cluster, the method comprising: receiving a data write request including a first key-value pair; when conditions are met, writing the first key-value pair into memory by calling the data write interface of a database adaptation layer to call the data write interface of a database engine, the database adaptation layer being used to encapsulate the interface provided by the database engine into an interface identical to that of boltdb, the database engine being based on an LSM-Tree structure, and the database engine being the database engine used by the etcd storage layer. In an embodiment of the present application, a database engine based on an LSM-Tree structure can be encapsulated by the database adaptation layer, so that the first node can write data by calling the data write interface of the database adaptation layer, thereby improving the write performance of etcd.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data writing method and electronic equipment. Background Art

[0002] etcd is a highly available distributed key-value (KV) storage that can be used for microservice registration and discovery, shared configuration, distributed locking or consistency assurance, distributed data queues, distributed notification and coordination, cluster elections, and more.

[0003] Currently, etcd's storage layer uses BoltDB, a database engine that uses a B+ tree as its internal data structure. Due to the structural characteristics of B+ trees, when large amounts of data are written to etcd, BoltDB's internal B+ tree structure undergoes frequent adjustments (such as rebalancing, splitting, or merging tree nodes), which can reduce etcd's write performance. Summary of the Invention

[0004] The embodiments of the present application disclose a data writing method and an electronic device, which can improve the writing performance of etcd.

[0005] The first aspect discloses a data writing method, which can be applied to the first node of the etcd cluster, or to a module (for example, a chip) in the first node, or to a logic module or software that can realize all or part of the functions of the first node. The first node is the master node of the etcd cluster, and the following description is taken as an example of application to the first node. The data writing method may include: receiving a data write request, the data write request including a first key-value pair; when conditions are met, calling the data write interface of the database adaptation layer to call the data write interface of the database engine to write the first key-value pair into the memory, the database adaptation layer is used to encapsulate the interface provided by the database engine into the same interface as boltdb, the database engine is based on the LSM-Tree (log structured merge tree) structure, and the database engine is the database engine used by the etcd storage layer.

[0006] In the embodiment of the present application, the database engine based on the LSM-Tree structure can be encapsulated through the database adaptation layer, so that the storage interface provided to the outside remains unchanged (that is, the same as the storage interface provided by boltdb), so that other modules in etcd are not affected. However, when the data write interface encapsulated by the database adaptation layer is called to write data, the database engine based on the LSM-Tree structure is actually used at the bottom layer for data storage. Since LSM-Tree is a hierarchical, ordered, disk-oriented data structure, random write operations can be converted into sequential writes to the disk. Therefore, the first node can improve the write performance of etcd by writing data in the above manner.

[0007] As a possible implementation method, calling the data write interface of the database engine by calling the data write interface of the database adaptation layer to write the first key-value pair into the memory includes: calling the data write interface of the database adaptation layer and passing in the first key-value pair; calling the data write interface of the database engine through the database adaptation layer and passing in the first key-value pair in the data write interface of the database adaptation layer; and writing the first key-value pair in the data write interface of the database engine into the memory through the database engine.

[0008] In this embodiment of the present application, when writing data, the first node can call the data write interface of the database adaptation layer and pass in a key-value pair. After that, the database adaptation layer can call the data write interface of the underlying database engine, and then the database engine can write the first key-value pair to the memory. As can be seen, when writing data, the first node directly writes to the memory. Since the memory has a higher write efficiency, this can further improve the write efficiency of etcd.

[0009] As a possible implementation, the database engine is rocksdb.

[0010] As a possible implementation method, calling the data write interface of the database engine by calling the data write interface of the database adaptation layer to write the first key-value pair into the memory includes: calling the data write interface of the database engine by calling the data write interface of the database adaptation layer to write the first key-value pair into the variable memory table in the memory.

[0011] As a possible implementation, the method also includes: when the amount of data in the variable memory table is greater than a first threshold, converting the variable memory table into an immutable memory table; flushing the immutable memory table to the disk of the first node to generate an SSTable (sorted string table).

[0012] In an embodiment of the present application, the first node calls the data write interface of the database engine by calling the data write interface of the database adaptation layer, and can first write the first key-value pair to the variable memory table in the memory. Afterwards, when the variable memory table reaches a certain amount of data, the variable memory table can be converted into an immutable memory table, and then the immutable memory table can be flushed to the disk of the first node to generate an SSTable. In this way, in the process of data writing, sequential writing can be guaranteed without modifying the data in the previous SSTable, thereby improving writing efficiency. In addition, since the key and value in the SSTable can be any byte array, and the size of the SSTable can be configured according to the size of the key and value, variable-length storage can be achieved.

[0013] As a possible implementation, the disk includes multiple levels of SSTables, and the method also includes: when the amount of data or the number of SSTables in the first level is greater than a second threshold, merging part or all of the SSTables in the first level to the next level.

[0014] In an embodiment of the present application, the SSTables on the disk are divided into multiple levels, and each level can include multiple SSTable files. Therefore, when the amount of data or the number of SSTables in each level reaches a preset threshold, the first node can merge part or all of the SSTables in that level into the next level. This hierarchical storage can facilitate hierarchical queries (i.e., reading) after data is written.

[0015] As a possible implementation, the method further includes: receiving a data read request, the data read request including a second key; and calling the database engine through the database adaptation layer to read data corresponding to the second key.

[0016] As a possible implementation, calling the database engine through the database adaptation layer to read the data corresponding to the second key includes: calling the database engine through the database adaptation layer to read the data corresponding to the second key from the mutable memory table; if the data is not read from the mutable memory table, reading the data corresponding to the second key from the immutable memory table; if the data is not read from the immutable memory table, reading the data corresponding to the second key from the block cache; if the data is not read from the block cache, reading the data corresponding to the second key from the SSTable on the disk.

[0017] In the embodiment of the present application, since the data in the variable memory table is the newest, the probability of reading data from it is the highest; the data in the immutable memory table is newer, the probability of reading data from it is higher; the data in the block cache is older, the probability of reading data from it is lower; the data in the SSTable on the disk is the oldest, and the probability of reading data from it is the lowest. Therefore, when the first node reads data, it can read from the variable memory table, the immutable memory table, the block cache, and the SSTable on the disk in sequence, which can greatly improve the reading efficiency. In addition, the variable memory table, the immutable memory table, and the block cache can be read based on the memory, and the reading speed of the memory is faster, so the reading efficiency can be further improved.

[0018] As a possible implementation, the first node includes a quota module and a key-value server, and the method further includes: performing a quota check through the quota module; when it is determined that the quota has not been exceeded, performing speed limit, authentication, and packet size checks through the key-value server; when the speed limit, authentication, and packet size checks pass, determining that the condition is met.

[0019] In an embodiment of the present application, after the first node receives a data write request, quota, speed limit, authentication, and packet size checks can be performed. If the check fails, the first node does not need to continue processing. If the check passes, the first node can continue processing. In this way, the processing resources of the first node can be saved, and too many data write requests can be prevented from being received, thereby avoiding the collapse of the entire processing system and improving the stability of the system.

[0020] The second aspect discloses an electronic device, which can be the first node of an etcd cluster. The electronic device may include a processor, a memory and a communication interface. The communication interface is used to receive information from other electronic devices outside the electronic device, and output information to other electronic devices outside the electronic device. The processor calls the computer program stored in the memory to implement the data writing method provided in the above-mentioned first aspect and any possible implementation method of the first aspect.

[0021] The third aspect discloses a computer-readable storage medium, on which a computer program or computer instructions are stored. When the computer program or computer instructions are executed, the data writing method disclosed in the above aspects is implemented.

[0022] A fourth aspect discloses a chip, comprising a processor for executing a program stored in a memory. When the program is executed, the chip executes the data writing method disclosed in the above aspects.

[0023] As a possible implementation, the memory is located outside the chip.

[0024] A fifth aspect discloses a computer program product, which includes a computer program code. When the computer program code is run, the data writing method disclosed in the above aspects is executed.

[0025] It is understandable that the electronic device provided in the second aspect, the computer-readable storage medium provided in the third aspect, the chip provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the data writing method provided in the first aspect of this application and any possible implementation of the first aspect. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 This is a schematic diagram of the etcd architecture disclosed in the embodiments of this application;

[0028] Figure 2 This is a schematic diagram of a scenario in which etcd writes data, as disclosed in an embodiment of the present application;

[0029] Figure 3 This is a schematic diagram of a scenario of an idle list disclosed in an embodiment of the present application;

[0030] Figure 4 This is a schematic diagram of a system architecture disclosed in an embodiment of the present application;

[0031] Figure 5 This is a schematic diagram of another etcd architecture disclosed in an embodiment of this application;

[0032] Figure 6 This is a flow chart of a data writing method disclosed in an embodiment of the present application;

[0033] Figure 7 This is a schematic diagram of another scenario of etcd writing data disclosed in an embodiment of the present application;

[0034] Figure 8 This is a structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application embodiment discloses a data writing method and electronic device (such as the first node described below), which can improve the writing performance of etcd. The technical solution in the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present application.

[0036] In order to better understand the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below.

[0037] etcd is a highly available distributed key-value (KV) store that can be used for microservice registration and discovery, shared configuration, distributed locking or consistency assurance, distributed data queues, distributed notification and coordination, and cluster elections. As a result, etcd is widely used in various distributed scenarios. For example, etcd serves as the core storage for Kubernetes (k8s), storing network configuration and object status information in a k8s cluster.

[0038] Kubernetes is an open-source platform for automating the deployment, scaling, and maintenance of container clusters. It offers features such as disaster recovery, horizontal scaling (elastic scaling), load balancing, version rollback, and storage orchestration, and is widely used in the construction and management of computer clusters.

[0039] The implementation of etcd mainly relies on the Raft (replicated and fault tolerant) algorithm, the multi-version concurrency control (MVCC) mechanism, and BoltDB.

[0040] Raft is a consensus algorithm that is easy to understand and offers enhanced security. Furthermore, Raft provides a general method for deploying finite state machines across computer clusters, ensuring that all nodes within the cluster maintain consistency in certain state transitions.

[0041] MVCC is a common concurrency control method used in database management systems. MVCC aims to address the problem of multiple, long-running read operations starving write operations due to read-write locks. Under MVCC, each data item read by a transaction is a historical snapshot, dependent on the implemented isolation level. Furthermore, write operations do not overwrite existing data items but instead create a new version that becomes visible only when the operation commits. Furthermore, snapshot isolation allows transactions to see the data state at the time they start.

[0042] BoltDB is a key-value database engine implemented in Golang. BoltDB provides basic storage functionality, but does not support network connections or complex Structured Query Language (SQL) queries. Each database data is stored in a single file, and data files are read and written through an application programming interface (API) to achieve data persistence. BoltDB's internal implementation uses a B+Tree structure, where "B" stands for balance, and B+Tree is also a balanced tree.

[0043] The following is an introduction to the etcd architecture. Figure 1 , Figure 1 This is a schematic diagram of the etcd architecture disclosed in the embodiment of this application. Figure 1 As shown in the figure, the entire architecture of etcd mainly consists of five layers: client layer, API network layer (i.e. API layer), Raft layer, functional logic layer (i.e. logic layer) and storage layer.

[0044] The client layer can include client libraries (such as clientv2 and clientv3 API client libraries) and tools such as etcdctl. The client library provides a simple and easy-to-use API and supports load balancing and automatic failover between nodes.

[0045] The API layer primarily includes the communication protocols for client access to server nodes, as well as the communication protocols between server nodes. Clients can access server nodes using the gRPC (Google Remote Procedure Call) API and the V2 / V3 HTTP (Hypertext Transfer Protocol) API. Communication between server nodes can utilize Raft HTTP, the HTTP protocol used by server nodes to implement functions such as data replication and leader election using the Raft algorithm.

[0046] The Raft layer mainly includes core modules such as Leader Election, Log Replication, Read Index, Membership, and Learner, which can ensure data consistency between server nodes and improve service availability.

[0047] The logic layer mainly includes functional modules such as the key-value server (i.e., KVServer module), authentication (Auth) module, quota (Quota) module, application (Apply) module, lease (Lease) module, compression (Compactor) module, maintenance (Maintenance) module, and tree index (treeIndex) module. It can be used to perform operations such as quota, speed limit, and authentication, as well as to realize data storage and management.

[0048] The storage layer primarily includes the write-ahead log (WAL) module, the snapshot module, and the boltdb module. The WAL module ensures persistent data storage. In etcd, all data modifications are written to the WAL log on disk before being committed. The boltdb module stores cluster metadata and user-written data. The snapshot module conserves storage space. In etcd, a default snapshot is created every 10,000 records.

[0049] It should be noted that etcd can include an MVCC module, and the MVCC module can include the above-mentioned treeIndex module and boltdb module. Among them, the MVCC mechanism can be implemented through the treeIndex module and the boltdb module. The treeIndex module is implemented based on the memory version of the btree library, which will save the key and related version number information (including historical version number information), and the version number information may include the global version number corresponding to the key, the number of modifications and other attributes. The value corresponding to the key is stored in the boltdb module, so its memory requirements are relatively low. It should be understood that the modification, writing and deletion of the key-value pair can generate a new version number (revision), and accordingly, the version number can be made globally monotonically increasing.

[0050] Specifically, the process of writing data to etcd is introduced below. Figure 2 , Figure 2 This is a schematic diagram of a scenario in which etcd writes data, as disclosed in an embodiment of the present application.

[0051] like Figure 2 As shown in the figure, when etcd writes data (i.e., when a write transaction is executed), it first obtains and updates the index (i.e., revision) based on the written key. If the key does not exist, it increments the revision based on the current maximum currentRevision (i.e., the current version number). Accordingly, a key (i.e., key) can correspond to one or more version numbers. For example, Figure 2The key for writing data in is "hello", and the current version number is "revision{2,0}". It should be understood that the version number can include two values, where the first value can be a globally incremented major version number, and the second value can be a transactional sub-version number, which is incremented within a transaction.

[0052] Afterwards, etcd can store a structure consisting of information such as the version number as the key and the original key-value pair (i.e., the key-value pair carried in the write request) in blotdb. For example, a structure with the key "revision{2,0}" and the original key-value pair can be written to boltdb. Furthermore, when storing data in blotdb, etcd can write the data to a cache buffer to ensure that subsequent read requests can obtain the latest data.

[0053] It's important to note that, to improve write performance, etcd doesn't commit transactions during the above write process. Therefore, data is updated only in the in-memory data structure managed by BoltDB. To persist the data to disk, the backend asynchronous transaction commit goroutine mechanism can be used. Typically, the asynchronous mechanism commits batches of transactions at a default interval of 100ms.

[0054] It is understandable that when etcd reads data (that is, when a read transaction is executed), since the read requests submitted by the client all query the value through the key, and querying data from boltdb must be done through the version number, etcd can first obtain the key version number from the treeIndex module, and then search for data from the cache buffer based on the version number. If the data is found (that is, a hit), it will be returned directly. If not found, etcd can query the relevant data from the boltdb module based on the version number.

[0055] BoltDB is a streamlined database implementation model that uses memory mapping (mmap) to map disk pages (physical pages on disk) to memory pages, achieving zero-copy data and utilizing B+ trees for indexing. BoltDB's write transaction implementation is quite ingenious, utilizing meta-replicas and freelists for concurrency control, and copy-on-write (COW) technology for page management. This COW technology allows for lock-free read and write concurrency, but not lock-free write concurrency, resulting in poor random write performance. BoltDB uses a B+ tree as its internal data structure for data storage, with leaf nodes storing specific key-value data. The basic unit of data storage is a page, with a default size of 4 kilobytes (KB). When data is deleted, BoltDB does not directly return the deleted disk space (i.e., the pages corresponding to the deleted data) to the system. Instead, it temporarily stores these pages internally to form a pool of released pages for subsequent use. This page pool is generally called a freelist. Figure 3 As shown in the figure, it shows 10 consecutive pages 42-51. Among them, page 43, page 45, page 46, and page 50 are currently in use, that is, they are non-free pages, while page 42, page 44, page 47, page 48, page 49, and page 51 are free and can be used later. As you can see, these free pages are discrete, not continuous. Therefore, when these pages are subsequently used, they cannot be written sequentially, resulting in low write efficiency. For example, when four consecutive pages are needed to store data, it may be impossible to find four consecutive free pages. In this case, the data can only be written to four discrete pages, such as page 42, page 44, page 47, and page 48.

[0056] In addition, due to the structural characteristics of the B+ tree, when a large amount of data is written, the B+ tree structure within BoltDB will frequently undergo adjustments (such as rebalancing, splitting or merging tree nodes, etc.), which will further lead to a significant decrease in write performance. Moreover, for data updates, the B+ tree will directly modify the corresponding value at the location of the original data, which will increase the probability of random writes. Therefore, for write-intensive "business" scenarios (i.e., scenarios that require frequent data writing), BoltDB cannot meet the writing needs, the writing efficiency is low, and it may even cause write blocking, which has a significant impact on the business.

[0057] In order to solve the above problems, in the embodiment of the present application, while keeping the overall architecture of etcd unchanged, the MVCC module can be functionally expanded, and a new boltdb adaptation layer (i.e., database adaptation layer) can be added to introduce rocksdb. The boltdb adaptation layer can encapsulate the functions of rocksdb, so that the KV storage interface function provided to the outside remains unchanged, but rocksdb is actually used for data storage at the bottom layer. Rocksdb uses LSM-Tree (log structured merge tree) as the data structure for storing data internally, which can convert random write operations into sequential writes to the disk, thereby improving the write efficiency of etcd.

[0058] In order to better understand the embodiments of the present application, the system architecture used in the embodiments of the present application is described below.

[0059] See also Figure 4 , Figure 4 This is a schematic diagram of a system architecture disclosed in the embodiment of this application. Figure 4 As shown, the system architecture may include an etcd cluster 401, a network 402, a first electronic device 403, and a second electronic device 404. The etcd cluster 401 may include multiple computing nodes (i.e., etcd nodes), for example, Figure 4 Node a 4011, node b 4012, node c 4013, ..., node N 4014 are shown. It should be understood that in order to improve the availability and fault tolerance of the etcd cluster, N is generally an integer greater than or equal to 3.

[0060] It should be noted that each node in the etcd cluster 401 can be an electronic device with data processing capabilities, data receiving and sending capabilities, and data storage capabilities, such as blade servers, high-density servers, rack servers, cabinet servers, and other servers.

[0061] The etcd cluster 401 (i.e., nodes a to N), the first electronic device 403, and the second electronic device 404 can be connected to a network 402 respectively, and communicate with each other through the network 402. The N nodes in the etcd cluster 401 can also communicate with each other through the network.

[0062] It should be understood that the Raft protocol (i.e., algorithm) defines the node states in the etcd cluster, mainly including the three states of follower, candidate, and cluster leader. Among them, at any moment, each node is in one of these states. When etcd starts, the node state generally defaults to follower, at which time the log received from the leader can be synchronized. The leader node is generated through node election. It is unique and has the privilege of synchronizing logs. It needs to broadcast heartbeats to follower nodes at regular intervals to maintain its leadership status. Candidate is a state when electing a leader, which can initiate a leader election. It should be understood that the leader node can also be called the master node.

[0063] It should be noted that data read requests can be made from any node (such as node a 4011 or node b 4012) because the data stored on each node is strongly consistent. However, data write requests must be processed by the leader node. Therefore, if the node where the etcd client initiates a data write request is a follower node, the data write request must be forwarded to the leader node. Once the leader node receives the data write request, it persists the request in the WAL log and broadcasts it to each follower node. Then, once the proposal has been persisted by a majority of nodes in the cluster, the proposal content can be persisted.

[0064] In the embodiment of the present application, electronic devices such as the first electronic device 403 and the second electronic device 404 can send data operation instructions to the etcd cluster 401, such as data addition, data deletion, data modification, data query and other instructions. Figure 4 Node a (4011) shown can receive data operation instructions from electronic devices, then perform corresponding processing and return the processing results to the electronic devices. It should be understood that electronic devices such as the first electronic device 403 and the second electronic device 404 may include etcd clients, and the first electronic device 403 and the second electronic device 404 may send data operation instructions to the etcd cluster 401 through the etcd client.

[0065] In some embodiments, the first electronic device 403 and the second electronic device 404 can be mobile phones, tablet computers, laptop computers, smart cars, smart wearable devices, servers, etc., which are not limited here.

[0066] In an embodiment of the present application, after the etcd cluster 401 receives a data write request, it can execute the data writing method provided in the embodiment of the present application, which can greatly improve the data writing performance. For details, please refer to the description in the following method embodiment.

[0067] It should be noted that Figure 4 The system architecture shown is only an example and does not constitute a limitation. In other embodiments of the present application, Figure 4 The system architecture shown may include more or fewer devices or modules than shown, and is not limited to only Figure 4 , an etcd cluster 401, a network 402, a first electronic device 403, and a second electronic device 404 are shown.

[0068] The following is an introduction to the etcd architecture used in this application. Figure 5 , Figure 5 This is another schematic diagram of the etcd architecture disclosed in the embodiment of this application. Figure 5 As shown in the figure, while keeping the original etcd overall architecture unchanged, the original MVCC module's boltdb module is replaced with the rocksdb module. In order not to affect other modules, a boltdb adaptation layer (i.e., adaptation module) is added. Through the boltdb adaptation layer, rocksdb can be encapsulated into boltdb, so that the external API interface remains unchanged (i.e., the interface provided by boltdb is the same as that provided by boltdb), but the underlying storage can be achieved through rocksdb.

[0069] RocksDB is a database engine that provides key-value storage and read / write capabilities. RocksDB's internal implementation uses the LSM-Tree structure, a hierarchical, ordered, disk-oriented data structure. The core concept of LSM-Tree is to leverage the fact that sequential batch writes on disks have significantly higher performance than random writes. Design and optimization around this principle ensure optimal write performance. All writes in this structure are in append mode; there are no deletions or modifications.

[0070] It should be understood that BoltDB and RocksDB have different object concept definitions. Therefore, when performing interface encapsulation, relevant logical concept mapping is required to map BoltDB logical concepts to RocksDB logical concepts. For example, the db (database) object in BoltDB can be mapped to the db object in RocksDB, and the bucket in BoltDB can be mapped to the column family in RocksDB. The specific mapping of BoltDB logical concepts and RocksDB logical concepts is shown in Table 1 below:

[0071] Table 1

[0072]

[0073] As shown in Table 1, logical concepts such as DB, Bucket, Cursor, Transaction, and Snapshot in BoltDB can be mapped to logical concepts such as DB, ColumnFamily, Iterator, Transaction, and Checkpoint in RocksDB. Furthermore, Put, Get, and Delete operations in BoltDB can be mapped to Put, Get, and Delete operations in RocksDB. It should be understood that RocksDB supports a more comprehensive set of transactions than BoltDB, so users can choose targeted transaction processing based on their specific usage scenarios.

[0074] Based on the mapping of related concepts in BoltDB and RocksDB, an interface adaptation layer (i.e., BoltDB adaptation layer) can be developed based on the RocksDB API interface, which is fully compatible with the BoltDB API interface. For example, API interfaces such as put (writing, adding or updating data), get (reading or querying), delete (deleting), and snapshot can be used.

[0075] It should be understood that in order to keep the API interface provided to the outside unchanged, the boltdb adaptation layer needs to further encapsulate the interface provided to the outside by rocksdb, so that the API interface provided to the outside is the same as the interface provided to the outside by boltdb. In this way, there is no need to change other modules, and data can be read, written, deleted, etc. according to the original boltdb API call method. For example, suppose that the interface for reading data provided by boltdb is Get (parameter 1: key), and the interface for reading data provided by rocksdb is Get (parameter 1: key, parameter 2: column family). At this time, since the Get interface for reading data of boltdb can only include parameter 1, that is, the key for reading data, and the Get interface for reading data of rocksdb can include two parameters, parameter 1 is the key for reading data, and parameter 2 is the related column family. Therefore, in order to keep the interface for reading data provided to the outside unchanged, the boltdb adaptation layer needs to further encapsulate the Get interface for reading data provided by rocksdb, so that its external interface is Get (parameter 1: key). In one possible implementation, when the Get (parameter 1: key) interface is called externally, the boltdb adaptation layer can convert it into a call to the rocksdb Get interface, and since both include the parameter key, parameter 1 can be directly replaced and filled, and since the boltdb Get interface does not include the corresponding parameter 2, the boltdb adaptation layer can automatically fill in parameter 2, for example, filling parameter 2 with the default column family. In this way, by wrapping the rocksdb API, etcd can call rocksdb through the boltdb adaptation layer just like calling boltdb. Similarly, similar encapsulation can be performed for interfaces such as Put and Delete. It should be understood that the encapsulation of the above-mentioned Get interface is only an exemplary illustration and does not constitute a limitation.

[0076] It should be noted that in other embodiments of the present application, when encapsulating the API interface of rocksdb, some data structures can be redefined and the API interface provided to the outside world can also be changed.

[0077] For example, DB and Tx can be defined, and their data structure definitions are as follows:

[0078]

[0079] Based on the data structure defined above, relevant interfaces can be defined, as shown in Table 2 below:

[0080] Table 2

[0081]

[0082]

[0083] It can be understood that through the above-mentioned concept mapping and interface definition, when etcd calls the relevant interface, data can be written and read through RocksDB, thereby improving data writing efficiency and meeting the writing requirements in write-intensive "business" scenarios.

[0084] It should be understood that in other embodiments of the present application, other database engines (such as LevelDB) can be adapted through the boltdb adaptation layer, not limited to rocksdb. In this way, different database engines can be flexibly used for different application scenarios to improve the data writing performance and data reading performance of etcd. In one possible implementation, an abstract interface that is not bound to any storage engine can be added to the MVCC module of etcd to implement basic operations such as KV storage insertion, deletion, query, snapshot, etc. In this way, it is convenient for users to select the appropriate database engine for use according to the usage scenario.

[0085] It should be noted that Figure 5 The etcd architecture shown is only an example and does not constitute a limitation. In other embodiments of the present application, it may include Figure 5 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 5 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0086] Based on the above system architecture, please refer to Figure 6 , Figure 6 This is a flow chart of a data writing method disclosed in an embodiment of the present application. Figure 6 As shown, the data writing method may include but is not limited to the following steps:

[0087] 601. The first node receives a data write request, where the data write request includes a first key-value pair.

[0088] When the first or second electronic device needs to write data (including inserting or updating data) to the etcd cluster, it can send a data write request to the first node. Correspondingly, the first node can receive the data write request from them. The first node can be the leader node in the database cluster, and the data write request can include a first key-value pair, which can include a key and a value.

[0089] It should be understood that in some embodiments, when the first electronic device or the second electronic device needs to write data to the etcd cluster, it can also send a data write request to other follower nodes in the etcd cluster. After the follower node receives the data write request, the follower node can forward the data write request to the first node. Accordingly, the first node can receive a data write request from the follower node. It should also be understood that when the first electronic device or the second electronic device needs to write data to the etcd cluster, an etcd node can be selected through a load balancing algorithm, and then a gRPC call for a data write request (i.e., a PUT request) can be initiated to the etcd node. Accordingly, the etcd node can intercept the data write request through a gRPC interceptor.

[0090] It should be noted that in some embodiments, when the first or second electronic device needs to delete data from the etcd cluster, it can send a data write request to the first node to delete the data. Because RocksDB also adds a new record to memory when deleting data, this record is used to mark the relevant data to be deleted. Accordingly, a data write request to delete data can only include the key, not the value.

[0091] 602. When the conditions are met, the first node writes the first key-value pair into the memory through the database adaptation layer.

[0092] After receiving a data write request, the first node needs to perform a pre-check (including quotas, rate limits, authentication, etc.). If the pre-check fails, the first node does not need to proceed. If the pre-check passes, the first node can generate a corresponding write proposal log entry based on the data write request and then broadcast this log entry to all follower nodes. After a majority of nodes in the etcd cluster have completed the log entry persistence, the database adapter layer can write the first key-value pair to memory. It should be understood that meeting the above conditions can refer to passing quotas, rate limits, authentication, packet size checks, and completing the log entry persistence by a majority of nodes in the cluster. It should be noted that to avoid data loss, if the conditions are met, the first node can write the first key-value pair to the WAL log on disk. It should also be understood that before the first node writes the first key-value pair to memory through the database adapter layer, it can retrieve and update the index based on the key of the first key-value pair. If the key does not exist, the revision is automatically incremented based on the current maximum currentRevision. Afterwards, the first node may store a structure consisting of the version number as key and the value as the first key-value pair information into the memory.

[0093] Specifically, after receiving a data write request, the first node can perform a quota check through the Quota module, that is, check whether the sum of the current etcd db size plus the key-value size of the data write request (i.e., the size of the first key-value pair) exceeds the quota (quota-backend-bytes). If the quota is exceeded, it means that the current storage space is insufficient and the first key-value pair cannot be successfully written. If the quota is not exceeded, the first node can proceed to the next step, that is, it can perform speed limiting, authentication (i.e., determine whether the data write request is legal through the Auth module) and packet size check through the KVServer module. If the speed limit, authentication or packet size check fails, the first node can refuse to write the first key-value pair. If the speed limit, authentication and packet size check pass, the first node can package the content of the data write request into a proposal message through the KVServer module and submit it to the Raft module (i.e., the above-mentioned Raft layer). Afterwards, the first node can generate a log entry corresponding to the proposal through the Raft module, and can broadcast the log entry to other follower nodes in the etcd cluster. The log entry encapsulates the content of the proposal. After most nodes in the etcd cluster complete the persistence of log entries, the status corresponding to the proposal can be changed to submitted. Then, the first node can execute the proposal in the submitted state through the Apply module. Specifically, if the conditions are met, the Apply module can first determine whether the proposal has been executed. If it has been executed, no processing is required. If it has not been executed, the operation of persisting the content of the proposal can be performed through the MVCC module of the first node. At this time, the Apply module needs to call the interface of boltdb (that is, the interface provided by the adaptation layer to the outside world) to perform data writing, that is, call the API provided by rocksdb (such as PUTAPI) through the database adaptation layer to write the first key-value pair into the memory, that is, call the data write interface of rocksdb by calling the data write interface of the database adaptation layer to write the first key-value pair into the memory. It should be understood that when the Apply module calls the data write interface of the database adaptation layer, it needs to pass in the first key-value pair so that when the database adaptation layer calls the data write interface of rocksdb, it can pass the first key-value pair into the data write interface of rocksdb. Accordingly, it can be understood that the data writing interface of the database adaptation layer called by the Apply module and the parameters passed in are functions or parameters associated with the interface of boltdb.

[0094] The following combination Figure 7 The first node writes the first key-value pair into the memory through the database adaptation layer. Figure 7As shown, data writing mainly includes two steps. The first step is to write the first key-value pair sequentially to the WALLog on the disk. The second step is to write the first key-value pair to the activememtable in the memory (that is, the variable memory table), that is, to write the first key-value pair to the activememtable in the memory through the database adaptation layer. Specifically, the first node can write the first key-value pair to the activememtable in the memory by calling the data writing interface provided by the database adaptation layer. It should be noted that since the first node can immediately return the write result to the client after writing the first key-value pair to the activememtable, its writing efficiency is relatively high.

[0095] It should be understood that as data is continuously written, more and more data will be stored in the activememtable. When the amount of data in the activememtable reaches a pre-set threshold (i.e., greater than the first threshold), the first node can freeze it in the memory and turn it into an immutable memory table (i.e., immutablememtable). At the same time, a new memtable can be created as a new activememtable. After the activememtable is converted to an immutablememtable, the first node can flush the immutablememtable in the memory to the Level 1 layer of the SSTable (sorted string table) in the disk (such as a solid-state drive, mechanical hard drive, etc.), and generate the corresponding SSTable to achieve persistent storage.

[0096] It should be noted that the SSTables on the disk are divided into multiple levels, and each level can include multiple SSTable files, such as Figure 7 The three layers shown (i.e., Level 1, Level 2, and Level 3). It should be understood that the data capacity of each layer of SSTable is 10 times that of the previous layer, for example, Level 1 is 10MB, Level 2 is 100MB, and Level 3 is 1000MB. Therefore, when the amount of data or the number of SSTables in each layer reaches a preset threshold, the first node can merge part or all of the SSTables in that layer to the next layer (i.e., compaction). For example, when the amount of data or the number of SSTables in the first layer is greater than the second threshold, the first node can merge part or all of the SSTables in the first layer to the next layer. It should be understood that the first level can be any level except the last level (such as the above-mentioned Level 1 or Level 2).

[0097] As you can see, during the RocksDB write process, the first node can write the key-value pair to the activememtable. Then, when the activememtable reaches a certain amount of data, it can convert the activememtable into an immutablememtable. After that, the immutablememtable can be flushed to disk to generate an SSTable. This ensures that the data is written sequentially without modifying the data in the previous SSTable.

[0098] It should be understood that an SSTable is a persistent, ordered, and immutable key-value storage structure. Its keys and values ​​can be arbitrary byte arrays. Furthermore, the size of an SSTable can be configured based on the size of the keys and values, thus enabling variable-length storage.

[0099] The following combination Figure 7 The data reading process of the embodiment of the present application is introduced. From the above data writing process, it can be seen that the latest data is generally stored in the activememtable. Therefore, after receiving the data reading request from the client, the corresponding version number can be found according to the key (such as the second key) carried in the data reading request. After that, when reading the data, it can be read from the activememtable according to the version number first. If it is not read from the activememtable, it can be read from the immutablememtable. If it is not read from the immutablememtable, it can be read from the block cache (i.e., BlockCache). If it is not read from the BlockCache, it indicates that the data is stored on the disk. Therefore, it can be read from the SSTable on the disk. The reading can be done from top to bottom, that is, it can be read from the SSTable in Level 1 first. If it is not read in Level 1, it can be read from the SSTable in Level 2. If it is not read in Level 2, it can be read from the SSTable in Level 3. It should be understood that the data in the SSTable can be cached in the BlockCache. When the memory is large, more SSTable data can be cached, thereby improving the data reading efficiency.

[0100] In the above method flow, the first node can convert random write operations into sequential writes to the disk through the database adaptation layer. Therefore, it can greatly improve write efficiency, especially for scenarios such as solid state disks (SSDs) that are sensitive to write amplification and hard disk drives (HDDs) that are sensitive to random writes.

[0101] It should be noted that the relevant information (ie, the same information or similar information) and related descriptions in the above different embodiments can refer to each other.

[0102] It should be understood that the above Figure 6 The above processing flow is illustrated by taking the first node as the execution subject of the interactive indication as an example, but this application does not limit the execution subject of the interactive indication. Figure 6 The first node in the method may also be a chip, a chip system, or a processor that supports the first node to implement the method, or may be a logic module or software that can implement all or part of the functions of the first node.

[0103] Based on the above system architecture, please refer to Figure 8 , Figure 8 800 is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. The electronic device 800 may include: a processor 801, a communication interface 802, and a memory 803. The processor 801, the communication interface 802, and the memory 803 may be interconnected or connected to each other via a bus 804.

[0104] Exemplarily, the memory 803 is used to store computer programs and data of the electronic device 800. The memory 803 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM). The communication interface 802 is used to support the electronic device 800 to communicate, such as receiving or sending data.

[0105] Exemplarily, the processor 801 may be a central processing unit (CPU), a complex programmable logic device (CPLD), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device (PLD), a transistor logic device (TLD), a hardware component, or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0106] In one embodiment, the electronic device 800 may be the first node, and the processor 801 may be configured to read the program stored in the memory 803 and execute the program. Figure 6 The operations performed by the first node in the method embodiment shown can be referred to the above related description and will not be described in detail here. It should be understood that in one embodiment, the electronic device 800 can also be other follower nodes in other etcd clusters.

[0107] It should be noted that Figure 8 The electronic device 800 shown is only one implementation of the embodiment of the present application. In actual applications, the electronic device 800 may also include more or fewer components, which is not limited here.

[0108] An embodiment of the present application further discloses a computer-readable storage medium having instructions stored thereon, which, when executed, execute the method in the above method embodiment.

[0109] The embodiments of the present application further disclose a computer program product comprising instructions, which, when executed, perform the method in the above method embodiments.

[0110] Obviously, the embodiments described above are only some of the embodiments of this application, and not all of them. Reference to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive with other embodiments. It is understood, both explicitly and implicitly, that the embodiments described herein can be combined with other embodiments. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. In the specification, claims, and accompanying drawings of this application, the terms "first," "second," "third," and so on are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a list of steps or elements may be included, or alternatively, steps or elements not listed may be included, or alternatively, other steps or elements inherent to the process, method, product, or device may be included. It can be understood that the equal sign of the above conditional judgment can be greater than one end or less than one end. For example, the above conditional judgment of a threshold being greater than, less than or equal to can also be changed to a conditional judgment of the threshold being greater than, equal to or less than the threshold, which is not limited here.

[0111] It will be appreciated that only the parts relevant to the present application, not all, are shown in the accompanying drawings. It will be appreciated that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. When its operation is completed, the process can be terminated, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0112] As used in this specification, the terms "component," "module," "system," "unit," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or distributed between two or more computers. In addition, these units can be executed from various computer-readable media having various data structures stored thereon. Units can communicate, for example, through local and / or remote processes based on signals having one or more data packets (e.g., data from a second unit interacting with another unit in a local system, a distributed system, and / or a network. For example, the Internet interacts with other systems via signals).

[0113] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application. It should be understood that the above description is only the specific implementation methods of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of this application should be included in the scope of protection of this application.

Claims

1. A data writing method, characterized in that: Applied to a first node of an etcd cluster, where the first node is a master node of the etcd cluster, the method includes: receiving a data write request, wherein the data write request includes a first key-value pair; When the conditions are met, the data write interface of the database engine is called by calling the data write interface of the database adaptation layer to write the first key-value pair into the variable memory table in the memory. The database adaptation layer is used to encapsulate the interface provided by the database engine into an interface identical to that of boltdb. The database engine is based on the LSM-Tree structure and is the database engine used by the etcd storage layer. When the amount of data in the variable memory table is greater than a first threshold, converting the variable memory table into an immutable memory table; The immutable memory table is flushed to the disk of the first node to generate a sorted string table SSTable.

2. The method according to claim 1, characterized in that The step of calling the data writing interface of the database engine by calling the data writing interface of the database adaptation layer to write the first key-value pair into the variable memory table in the memory includes: Calling the data writing interface of the database adaptation layer and passing in the first key-value pair; Calling the data writing interface of the database engine through the database adaptation layer, and passing the first key-value pair in the data writing interface of the database adaptation layer; The database engine writes the first key-value pair in the interface into a variable memory table in the memory through the database engine.

3. The method according to claim 1, characterized in that The database engine is rocksdb.

4. The method according to claim 1, wherein The disk includes multiple levels of SSTables, and the method further includes: When the amount of data or the number of SSTables at the first level is greater than the second threshold, some or all of the SSTables at the first level are merged into the next level.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: receiving a data read request, wherein the data read request includes a second key; The database engine is called through the database adaptation layer to read data corresponding to the second key.

6. The method according to claim 5, characterized in that Calling the database engine through the database adaptation layer to read the data corresponding to the second key includes: Calling the database engine through the database adaptation layer to read data corresponding to the second key from the variable memory table; If the data corresponding to the second key is not read from the mutable memory table, reading the data corresponding to the second key from the immutable memory table; If the data corresponding to the second key is not read from the immutable memory table, reading the data corresponding to the second key from the block cache; If the data corresponding to the second key is not read from the block cache, the data corresponding to the second key is read from the SSTable on the disk.

7. The method according to any one of claims 1 to 6, characterized in that The first node includes a quota module and a key-value server, and the method further includes: Perform quota checking via the quota module; If it is determined that the quota has not been exceeded, the key value server performs speed limiting, authentication, and packet size checking; If the speed limit, authentication, and packet size checks are passed, it is determined that the conditions are met.

8. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a communication interface, wherein the communication interface is used to receive information from other electronic devices outside the electronic device and output information to other electronic devices outside the electronic device, and the processor calls the computer program stored in the memory to implement the method according to any one of claims 1 to 7.

Images