Method and system for online creation of table indexes in a distributed system

Abstract

The application discloses an online table index creation method and system in a distributed system. After the server receives the request of the client, a transaction state machine is started. The updated data is first written into a log, so that the data of the transaction can be recovered from the log when the system fails. Then, the data is uploaded to a distributed cache layer. After the uploading is successful, the transaction is committed, and the client is notified that the read-write operation is completed. In addition, a checkpointer service asynchronously flushes the data in the distributed cache to a persistent storage layer. The application creates the index of the table online based on the mechanism of the distributed global lock without blocking concurrent read-write transactions (DML), and realizes the fault recovery operation based on the log, so that the persistence and consistency of the update table index transaction in the distributed system are guaranteed. Meanwhile, the abnormal handling process of the transaction is realized based on the retry mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of online creating table index in distributed system, and particularly relates to a method and system for online creating table index in distributed system. BACKGROUND

[0002] Distributed system is a software system based on network. Because of the software characteristics, the distributed system has high cohesion and transparency. Therefore, the difference between network and distributed system is more in high-level software (especially operating system) rather than hardware.

[0003] In a distributed system, a group of independent computers presents a unified whole to the user, just like a system. The system has multiple common physical and logical resources, and can dynamically allocate tasks. The distributed physical and logical resources exchange information through a computer network. The system has a distributed operating system that manages computer resources in a global manner. Usually, the distributed system has only one model or paradigm for the user. A layer of software middleware is responsible for implementing the model above the operating system. A famous example of a distributed system is the World Wide Web, in which everything looks like a document (Web page).

[0004] In the existing distributed system, the state inconsistency of each node in the system is prone to occur due to network, clock synchronization and other problems. For the operation of creating table index, which changes the table schema, the inconsistency may cause different table schema views of each node at a certain time, and further cause data damage and other problems. For example, during the table schema change process, concurrent read and write operations use a certain table schema for encoding in one node, and use a different table schema for encoding in another node. The state inconsistency that may be caused by creating table index in the existing distributed system. SUMMARY

[0005] Therefore, in order to solve the problem of state inconsistency that may be caused by creating table index in the existing distributed system, the present application provides a method for online creating table index in distributed system.

[0006] In a first aspect, a method for online creating table index in distributed system is provided, and the method comprises the following steps.

[0007] A global write lock is acquired, and a lock upgrade operation is performed. After the global write lock is acquired, the coordinating node initiates a table-level write lock request to all participating nodes. After the global write lock is acquired, the coordinating node calculates a transaction timestamp based on the lock information returned by each node.

[0008] A global write lock is acquired, and a lock upgrade operation is performed. After the global write lock is acquired, the coordinating node initiates a table-level write lock request to all participating nodes. After the global write lock is acquired, the coordinating node calculates a transaction timestamp based on the lock information returned by each node.

[0009] New table index information and a new table structure version number are written into a first log for persistence, and the first log is defined as a PrepareLog. After the first log is written successfully, if a node fails, the index creation process is continued based on the first log information during a recovery process.

[0010] The global write lock is downgraded, a persistent storage is created, and an index value of old data in the table is updated.

[0011] A second log is written, the second log is defined as a PrepareIndexLog, and the global write lock is upgraded. After the second log is written successfully, if a node fails, the index creation process is continued based on the second log information during a recovery process.

[0012] A third log is written, the third log is defined as a CommitLog, and the global write lock is released. After the third log is written successfully, if a node fails, the index creation process is continued based on the third log information during a recovery process.

[0013] A fourth log is written, the fourth log is defined as a CleanLog, and it is identified that the table index is successfully created.

[0014] In the above scheme, optionally, the global write lock is acquired, and the coordinating node initiates a table-level lock request to all participating nodes, including:

[0015] The coordinating node initiates a table-level write lock request to all participating nodes, and waits for each node to return a write lock result. After all nodes successfully acquire the lock, the coordinating node performs the next operation.

[0016] If a lock conflict occurs, the index creation operation is terminated.

[0017] acquire a global write lock, perform a lock upgrade operation, after acquiring the global write intent lock, the coordination node initiates a table-level write lock request to all participating nodes, after acquiring the global write lock, the coordination node calculates the commit timestamp of the transaction through the locking information returned by each node, specifically: after acquiring the global write lock, the coordination node calculates the commit timestamp of the target transaction through the locking information returned by each node, and the calculation of the timestamp uses the following formula:

[0018] Ts = max (allnode's commit_ts + 1, allnode's Last_read_ts + 1)

[0019] Wherein, the commit_ts represents the commit timestamp of the last update of the table structure;

[0020] Last_read_ts represents the ts of the last time the table structure information is read, which occurs in the process of reading and writing the table.

[0021] In the above scheme, optionally, the coordination node is the node that initiates the index creation transaction;

[0022] The participating node is a node participating in the index creation transaction, receives and processes the message sent by the coordination node, including acquiring a lock on the node, creating new table structure schema information locally, and specifically, the coordination node is also a participating node.

[0023] In the above scheme, optionally, after the first log is written successfully, if a node fails during recovery, the index creation process is continued according to the first log information, including:

[0024] After the first log is written successfully, if a node fails, the transaction recovery operation is performed according to the log; the PrepareLog record exists in the log, a table-level write intent lock is added locally, and simultaneously,

[0025] If the coordination node fails, a new transaction state machine is created when the fault is restarted, and the downgrade global write lock stage is re-executed;

[0026] If the participating node fails, the state of the node is restored to the state before the PrepareIndexLog record when the fault is restarted; the table-level write intent lock on the node is restored, and the old table structure information and the new table structure DitrySchema information are created locally.

[0027] In the above scheme, optionally, the degraded global write lock, the persistent storage is created, and the index value of the old data in the table is updated, specifically: the coordination node initiates a lock degradation request to degrade the write lock in all nodes to a write intention lock; at the same time, all nodes locally create a new table structure DirtySchema;

[0028] The coordination node creates a new index persistent storage in the persistent storage layer;

[0029] In all nodes, the data in the distributed cache layer is flushed to the persistent storage layer at the same time;

[0030] The coordination node reads the old data from the persistent storage layer, generates new index data from the old data, and writes the new index data into the distributed cache layer; after the old data generates the index data, the index data in the cache layer is forced to synchronize and flush to the index table in the persistent storage layer, and the index data generated by the old data is kicked out of the distributed cache layer.

[0031] In the above scheme, optionally, the second log is written, the second log type is PrepareIndexLog, and the global write lock is upgraded; after the second log is written successfully, if a node failure occurs during the recovery process, the index creation process is continued according to the second log information, including:

[0032] The second log is written, the second log type is PrepareIndexLog, and the update operation of the index value of the old data completed by the table index transaction is identified;

[0033] If the system fails, the PrepareIndexLog record exists in the log, and the state of the fault node is restored to the state before the CommitLog;

[0034] If the coordination node fails, the local table level write lock is recovered, and a new table structure DirtySchema information is created locally; a new transaction state machine is created, and the transaction is executed from the global write lock upgrade stage again;

[0035] If the participating node fails, the local table level write lock needs to be recovered, and the old table structure information and the new table structure DirtySchema information are created locally.

[0036] In the scheme, optionally, the third log is written, the third log type is CommitLog, and the global write lock is released; after the third log is written successfully, if a node failure occurs, in a recovery process, a creation process of an index is continued according to the third log information, including: for a table index creation transaction, a third log is written, the third log type is CommitLog, and it is indicated that the transaction has completed an operation of a persistent storage layer;

[0037] If a failure occurs, there is a CommitLog record in the log, and the state of the node is restored to a state before CleanLog;

[0038] If a failure occurs, there is a CommitLog record in the log, and the state of the node is restored to a state before CleanLog;

[0039] If a failure occurs, there is a CommitLog record in the log, and the state of the node is restored to a state before CleanLog;

[0040] In the scheme, optionally, the global write lock is released in the following manner:

[0041] The coordination node initiates a lock release request, and each participating node releases a table-level write lock of a local table;

[0042] The newly created table structure information is submitted as table structure information, and each node performs table structure conversion under the condition of holding the global write lock.

[0043] In the scheme, optionally, the fourth log is written, the fourth log type is CleanLog, and it is indicated that the table index has been successfully created, including:

[0044] If a failure occurs after CleanLog is written, the table index creation transaction has been successfully completed, no failure occurs, and neither the coordination node nor the participating node needs to perform any operation;

[0045] The log layer deletes a record about the table index creation transaction in the log according to the log.

[0046] In a second aspect, an online table index creation system in a distributed system includes: a distributed cache layer, a persistent storage layer, and a log layer; wherein,

[0047] The distributed cache layer is used to cache hot data and newly written data;

[0048] The persistent storage layer is used to store baseline data;

[0049] Cold data in the distributed cache layer is written asynchronously to the persistent storage layer; the distributed cache layer and the persistent storage layer synchronize and unify data through a checkpoint service;

[0050] The log layer is responsible for persisting request information of transactions.

[0051] The present application has at least the following beneficial effects:

[0052] Based on further analysis and research on the problems of the prior art, the present application realizes that the state inconsistency problem caused by creating table indexes in the existing distributed system. The present application acquires a global write intention lock, coordinates nodes to initiate a table-level locking request to all participating nodes, waits for all nodes to return the locking result, and executes the next operation after all nodes are successfully locked. If a locking conflict occurs, the operation of creating an index this time is terminated. A global write lock is acquired, and a lock upgrade operation is performed. After acquiring the global write intention lock, the coordinating node initiates a table-level write lock request to all participating nodes. After acquiring the global write lock, the coordinating node calculates the transaction commit timestamp through the locking information returned by each node. New table index information and a new table structure version number are written into a first log for persistence, and the first log type is defined as PrepareLog. After the first log is successfully written, if a node failure occurs, the index creation process is continued according to the first log information in the recovery process. The global write lock is downgraded, a persistent storage is created, and the index value of the old data in the table is updated. A second log is written, the second log type is PrepareIndexLog, and the global write lock is upgraded. After the second log is successfully written, if a node failure occurs, the index creation process is continued according to the second log information in the recovery process. A third log is written, the third log type is CommitLog, and the global write lock is released. After the third log is successfully written, if a node failure occurs, the index creation process is continued according to the third log information in the recovery process. The global write lock is released, and a fourth log is written. The fourth log type is CleanLog, which indicates that the table index is successfully created. Based on the distributed global lock mechanism, the table index is created online without blocking concurrent read-write transactions (DML), and based on the log, the fault recovery operation is realized to ensure the persistence and consistency of the table index update transaction in the distributed system. At the same time, based on the retry mechanism, the transaction exception handling process is realized. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 The flowchart of the method for creating a table index online in a distributed system provided by an embodiment of the present application;

[0054] Figure 2This is a basic data flow diagram in a distributed system for an online table index creation method provided in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] The method for creating table indexes online in a distributed system provided in this application, such as Figure 1 As shown, it includes the following steps:

[0057] Acquire a global write intention lock, and the coordinating node initiates a table-level lock request to all participating nodes. After all nodes return the lock result and all nodes have successfully acquired the lock, the coordinating node executes the next operation. If a lock conflict occurs, the current index creation operation is terminated.

[0058] Acquire a global write lock and perform a lock upgrade operation. After acquiring the global write intention lock, the coordinating node sends a table-level write lock request to all participating nodes. After acquiring the global write lock, the coordinating node calculates the transaction commit timestamp based on the lock information returned by each node.

[0059] The new table index information and the new table structure version number are written to the first log for persistence, and the first log type is defined as PrepareLog; after the first log is successfully written, if a node failure occurs, the index creation process continues to be completed according to the first log information during the recovery process.

[0060] Degrade the global write lock, create persistent storage, and update the index values ​​of the old data in the table;

[0061] Write to the second log, which is of type PrepareIndexLog, and upgrade to a global write lock; if a node failure occurs after the second log is successfully written, the index creation process will continue to be completed based on the information in the second log during the recovery process.

[0062] Write to the third log, which is of type CommitLog, and release the global write lock; if a node failure occurs after the third log is successfully written, the index creation process will continue to be completed based on the information in the third log during the recovery process.

[0063] Write a fourth log entry, of type CleanLog, indicating that the table index was successfully created.

[0064] In one embodiment, acquiring a global write intention lock and coordinating the node to initiate a table-level lock request to all participating nodes includes:

[0065] The coordinating node initiates a table-level write intention lock request to all participating nodes, and waits for each node to return the write intention lock result; after all nodes have successfully acquired the lock, the coordinating node executes the next operation;

[0066] If a locking conflict occurs, the current index creation operation will be terminated.

[0067] Acquire a global write lock and perform a lock escalation operation. After acquiring the global write intention lock, the coordinating node initiates table-level write lock requests to all participating nodes. After acquiring the global write lock, the coordinating node calculates the transaction commit timestamp based on the locking information returned by each node. Specifically, after acquiring the global write lock, the coordinating node calculates the target transaction commit timestamp based on the locking information returned by each node, using the following formula:

[0068] Ts=max(allnode'scommit_ts+1, allnode'sLast_read_ts+1)

[0069] Wherein, commit_ts represents the commit timestamp of the last update of the table structure;

[0070] Last_read_ts represents the last time the table structure information was read, which occurred during a read / write operation on the table.

[0071] In one embodiment, the coordinating node is the node that initiates the index creation transaction;

[0072] The participating nodes are those that participate in the index creation transaction. They receive and process messages sent by the coordinating node, including acquiring locks on their own node and creating new table structure schema information locally. In particular, the coordinating node is also a participating node.

[0073] In one embodiment, after the first log is successfully written, if a node failure occurs, during the recovery process, the index creation process continues based on the first log information, including:

[0074] After the first log is successfully written, if a node failure occurs, transaction recovery operations are performed based on the log; if the PrepareLog record exists in the log, a table-level write intention lock is acquired locally, and simultaneously...

[0075] If the coordinating node fails, a new transaction state machine is created upon restart, and execution is restarted from the downgraded global write lock phase.

[0076] If a participating node fails, upon restarting, the node's state will be restored to the state prior to the PrepareIndexLog record; the write intention lock at the table level on the node will be restored, and the old table structure information and the new table structure DitrySchema information will be created locally.

[0077] In one embodiment, the process of downgrading the global write lock, creating persistent storage, and updating the index values ​​of the old data in the table specifically involves: the coordinating node initiating a lock downgrade request to downgrade all write locks in all nodes to write intention locks; and simultaneously, all nodes creating a new table structure, DirtySchema, locally.

[0078] The coordinating node creates a new persistent storage index in the persistent storage layer;

[0079] Simultaneously refresh the data in the distributed cache layer to the persistent storage layer on all nodes;

[0080] The coordinating node reads old data from the persistent storage layer, generates new index data from the old data, and writes it to the distributed cache layer. After the old data generates index data, it performs a forced synchronization operation on the index data in the distributed cache layer, refreshes it to the index table in the persistent storage layer, and removes the index data generated from the old data from the distributed cache layer.

[0081] In one embodiment, the writing of the second log, of type PrepareIndexLog, and the escalation of the global write lock; after the second log is successfully written, if a node failure occurs, during the recovery process, the index creation process continues based on the information in the second log, including:

[0082] Write to the second log, the second log type being the PrepareIndexLog, to indicate that the transaction that created the table index has completed the update operation of the index value of the old data;

[0083] If a system failure occurs and the PrepareIndexLog record exists in the log, the failed node will restore its state to the state before the CommitLog.

[0084] If the coordinating node fails, the write lock at the local table level is restored, and a new table structure, DitrySchema information, is created locally; a new transaction state machine is created, and the transaction is re-executed from the global write lock escalation phase.

[0085] If a participating node fails, it is necessary to restore the local table-level write lock and simultaneously create the old table structure information and the new table structure DitrySchema information locally.

[0086] In one embodiment, the third log is written to, the third log type is CommitLog, and the global write lock is released; after the third log is successfully written, if a node failure occurs, during the recovery process, the index creation process continues to be completed according to the third log information, including: for this table index creation transaction, writing to the third log, the third log type is CommitLog, to indicate that the transaction has completed the persistent storage layer operation;

[0087] If a failure occurs, a CommitLog record will be stored in the log, restoring the node's state to the state before the CleanLog record.

[0088] If the coordinating node fails, the old table structure information and the new table structure information are created locally. Then, a new transaction state machine is created, and the transaction is re-executed from the phase of releasing the global write lock.

[0089] If a participating node fails, a new table structure is created locally.

[0090] In one embodiment, releasing the global write lock specifically means:

[0091] The coordinating node initiates a lock release request, and each participating node releases the table-level write lock on its local table.

[0092] The newly created table structure information is submitted as the table structure information, and each node performs table structure conversion while holding a global write lock.

[0093] In one embodiment, writing to the fourth log, the fourth log type being CleanLog, indicates that the table index has been successfully created, including:

[0094] The log contains a CleanLog entry. The failure occurred after writing the CleanLog. The current table index creation transaction has been successfully completed without any failure. Neither the coordinating node nor the participating nodes need to perform any operations.

[0095] Based on this log entry, the log layer deletes the record related to the table index creation transaction. The above-described method for online table index creation in a distributed system proposes a mechanism based on distributed global locks to create table indexes online without blocking concurrent read-write transactions (DML). It also implements fault recovery operations based on logs, ensuring the persistence and consistency of table index update transactions in a distributed system. Furthermore, it implements exception handling for transactions based on a retry mechanism.

[0096] In one embodiment, such as Figure 2As shown, the system consists of a distributed caching layer, a persistent storage layer, and a log layer. The distributed caching layer caches frequently accessed data and newly written data; the persistent storage layer stores baseline data; data in the distributed caching layer is asynchronously written to the persistent storage layer; the distributed caching layer and the persistent storage layer synchronize and unify data through a checkpointing service; and the log layer is responsible for persisting transaction request information. This ensures that transactions can be correctly recovered or rolled back in the event of a system failure.

[0097] In this solution, concurrent read and write operations are not blocked during the creation of table indexes. To better illustrate the process's impact on table data handling and concurrent DML operations, a brief description of the data flow within the system is provided. After receiving a client request, the server starts a transaction state machine. Updated data is first written to the log to ensure data recovery in case of system failure. Then, the data is uploaded to the distributed cache layer. Upon successful upload, the transaction is committed, and the client is notified that the read and write operations are complete. Additionally, the checkpointer service asynchronously flushes the data from the distributed cache to the persistent storage layer.

[0098] In one embodiment, creating a table index mainly involves two aspects: creating new table structure (schema) information and generating new index data corresponding to the old data in the table. In this solution, the following terminology is used for these tasks:

[0099] Lock Types: This solution employs two types of locks: Global Write Intent Lock (WI) and Global Write Lock (WL). The Global Write Intent Lock is used to block concurrent table structure modification operations (DDL) on the same table. The Global Write Lock is used to block concurrent read and write operations (DML) on the same table.

[0100] System Node Types: In this distributed system, all nodes are equal, meaning that a request to create a table index can be initiated on any node in the system. During the table index creation process, based on the different tasks undertaken by each node, the nodes in the system can be divided into two types.

[0101] Coordinating node: The node that initiates the index creation transaction.

[0102] Participating nodes: These are nodes that participate in the index creation transaction. They receive and process messages from the coordinating node, including acquiring locks on their local machine and creating new table schema information locally. Notably, the coordinating node is also a participating node.

[0103] Data types in the table: Based on the size of the write time relative to the version number of the new table structure (schema), the data in the table can be divided into two categories:

[0104] Old data: data whose write time is less than the version number of the new table structure.

[0105] New data: The write time is greater than the version number of the new table structure.

[0106] In one embodiment, the normal process includes the following steps:

[0107] Acquiring a global write intention lock: The coordinating node initiates a table-level lock request to all participating nodes, and then waits for each node to return the locking result. Only after all nodes have successfully acquired the lock will the coordinating node execute the next operation. During this process, if a locking conflict occurs, it means that a transaction is already updating the table structure, and the current index creation operation will be terminated.

[0108] Acquiring a global write lock: Perform a lock escalation operation. After acquiring the global write intention lock, the coordinating node sends table-level write lock requests to all participating nodes. After acquiring the global write lock, the coordinating node calculates the transaction's commit timestamp based on the locking information returned by each node. This timestamp is calculated using the following formula:

[0109] Ts=max(allnode'scommit_ts+1,allnode'slast_read_ts+1)

[0110] Here, `commit_ts` represents the commit timestamp of the last update to the table structure (schema). `Last_read_ts` represents the most recent read of the table structure (schema) information, which occurred during a read / write operation on the table.

[0111] The above formula ensures that even when the clocks of different nodes in the system are out of sync, a correct and sequential timestamp can still be obtained.

[0112] If a locking conflict occurs while requesting a global write lock, the transaction will be blocked until the write lock can be acquired.

[0113] This timestamp serves two purposes: firstly, it indicates the version number of the new table schema; secondly, it serves as a time point to distinguish read and write operations (DML) occurring on the table. For transactions with timestamps later than this, the operations have been committed, and the results have been updated in the table. For transactions with timestamps later than this, during the update process, not only are existing index information updated, but also the newly created index data is updated—a so-called "double write" operation. This ensures that the index creation operation on the table can be performed without blocking concurrent read and write operations (DML).

[0114] Write to the log: Write the new table index information and the new table structure (schema) version number to the log system for persistence. Define this log type as PrepareLog.

[0115] Once the log is successfully written, if a node failure occurs in the system, the index creation process can continue based on this log information during recovery. This ensures that index creation can be completed even in the event of a system failure.

[0116] Degrading the global write lock: The coordinating node initiates a lock degradation request, downgrading all write locks on all nodes to write intention locks. Simultaneously, all nodes create a new table structure (schema) locally, referred to here as DirtySchema. With the global write lock held, each node in the system creates a new table structure, thus ensuring system state consistency.

[0117] After this point in time, concurrent read and write operations (DML) can be executed normally. Not only do they need to write data to the existing index table, but they also need to write data to the newly created index table based on the new table structure information.

[0118] Create persistent storage: The coordinating node creates new persistent storage indexes at the persistent storage layer.

[0119] Updating index values ​​for old data in a table: In this system, old data exists in two locations: a distributed cache layer (containing hot data in the table and data newly written before index creation but not yet synchronized to the persistent storage layer) and a persistent storage layer (containing the baseline data in the table).

[0120] To avoid duplicate data operations, a synchronization operation is first performed between the distributed caching layer and the persistent storage layer. This involves flushing the data in the distributed caching layer to the persistent storage layer, ensuring that the persistent storage layer includes all old data and simplifying the data processing flow. This operation is performed simultaneously on all nodes in the system.

[0121] After synchronizing the old data, the coordinating node reads the old data from the persistent storage layer, generates new index data from it, and writes it to the distributed cache layer. Here, if concurrent read and write transactions have already written index data with the same key value during the "double write" operation, the index data generated from this old data is directly discarded, ensuring the correctness of the index data.

[0122] After generating index data for the old data, a forced synchronization operation is performed on the index data in the cache layer, flushing it to the index table in the persistent storage layer. Simultaneously, the index data generated from the old data is evicted from the distributed cache layer. This operation ensures that the data in the distributed cache layer is newer than the data in the persistent storage layer (i.e., the commit timestamp of the data in the distributed cache layer is greater than the commit timestamp of the data in the persistent storage layer), thus guaranteeing that subsequent transactions will not read incorrect index data.

[0123] Write to the log: For this table index creation operation, write a log entry, defined as PrepareIndexLog, to indicate that the table index creation transaction has completed the update operation of the index values ​​of the old data. If a system failure occurs, each node can recover from this log.

[0124] Upgrade global write lock: The coordinating node initiates a lock upgrade request, upgrading all write intention locks on all nodes to write locks. Similarly, if a locking conflict occurs, the transaction will be blocked until the write lock can be acquired.

[0125] Once a global write lock is acquired, concurrent read and write (DML) operations on that table will be blocked, but the blocking time is very short. This global write lock ensures that all nodes in the system are in a consistent state.

[0126] Write to the log: For this table index creation transaction, write a log entry, defined as CommitLog, to indicate that the transaction has completed the persistent storage layer operations. In case of a failure, each node can use this log to recover to the corresponding state.

[0127] Release global write lock: The coordinating node initiates a lock release request, and each participating node releases its local table-level write lock on the table. Simultaneously, it submits the newly created table structure (DirtySchema) information as the table structure information, making the new table structure (schema) information visible to the outside world.

[0128] Similarly, while holding a global write lock, each node in the system performs table structure conversion, ensuring the consistency of the system's table structure with the outside world.

[0129] In this way, subsequent read and write transactions will read the new table structure information and then perform corresponding operations based on the new table structure information.

[0130] Write to the log: For this table index creation transaction, write a log entry, defining the log type as CleanLog, to indicate that the table index was successfully created. Subsequently, the logging layer will delete the record related to this table index creation transaction from the log based on this log entry.

[0131] In one embodiment, the recovery process includes: a log layer in the system that ensures transaction recovery operations are performed based on the logs when a node failure occurs. The node redoes the corresponding operations based on different log information, thus restoring the node's state after the failure. This state includes the lock state and local table structure (schema) information. For different log types, the following scenarios can be considered:

[0132] In one embodiment, no logging occurs: if the coordinating node fails, upon restarting, since there are no log entries, it is assumed that no table index creation operation occurred. Orphan locks may remain in participating nodes. However, the system's orphan lock detection mechanism will automatically release these orphan locks.

[0133] If a participating node fails, no additional operations are required upon restarting because there are no log entries. The system's retry mechanism will cause the coordinating node to resend the corresponding requests, ensuring that the transaction continues.

[0134] In one embodiment, the PrepareLog record exists in the log, indicating that at the time of the failure, the system held at least a global write intention lock. In other words, the system needs to recover to a state where it holds a global write intention lock. Therefore, during the recovery process, the failed node needs to directly acquire a table-level write intention lock locally. Meanwhile,

[0135] If the coordinating node fails, a new transaction state machine needs to be created upon restart, and then the process needs to be re-executed from the downgraded global write lock phase.

[0136] If a participating node fails, upon restarting, it is only necessary to restore the node's state to the state prior to the PrepareIndexLog record. That is, restore the table-level write intent locks on that node, and simultaneously create the old table structure information and the new table structure (DitrySchema) information locally.

[0137] Note: Due to the idempotency of the operation, the final result and state remain unchanged even if the same operation is executed multiple times during the fault restart and recovery process.

[0138] In one embodiment, the presence of a PrepareIndexLog record in the log indicates that the failure occurred sometime between writing to PrepareIndexLog and writing to CommitLog. Therefore, the failed node needs to be restored to its state before the CommitLog entry.

[0139] If the coordinating node fails, the write locks at the local table level are first restored, and a new table structure (DitrySchema) is created locally. Then, a new transaction state machine is created, and the transaction is re-executed from the global write lock escalation phase.

[0140] If a participating node fails, the write lock at the local table level needs to be restored, and the old table structure information and the new table structure (DitrySchema) information need to be created locally.

[0141] In one embodiment, the presence of a CommitLog record in the log indicates that the failure occurred sometime between writing to the CommitLog and writing to the CleanLog. Therefore, the failed node needs to be restored to its state before the CleanLog entry.

[0142] If the coordinating node fails, the old table structure information and the new table structure (DirtySchema) information are first created locally. Then, a new transaction state machine is created, and the transaction is re-executed from the phase of releasing the global write lock.

[0143] If a participating node fails, only the newly created table structure information needs to be created locally.

[0144] In one embodiment, the presence of a CleanLog entry in the log indicates that the failure occurred after the CleanLog was written. At this point, the table index creation transaction is considered to have completed successfully without any failure. Neither the coordinating node nor the participating nodes need to perform any operations.

[0145] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0146] In one embodiment, an online table index creation system is provided in a distributed system, comprising the following program modules: An online table index creation system in a distributed system, characterized in that the system includes: a distributed caching layer, a persistent storage layer, and a log layer; wherein,

[0147] The distributed caching layer is used to cache frequently accessed data and newly written data.

[0148] The persistent storage layer is used to store baseline data;

[0149] Data in the distributed cache layer is asynchronously written to the persistent storage layer; the distributed cache layer and the persistent storage layer synchronize and unify data through a checkpoint service.

[0150] The log layer is responsible for persisting transaction request information.

[0151] For specific limitations regarding the online table index creation system in a distributed system, please refer to the limitations on the online table index creation method in a distributed system mentioned above, which will not be repeated here. Each module in the online table index creation system described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0152] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for creating table indexes online in a distributed system, characterized in that, The method includes: Acquire a global write intention lock, and the coordinating node initiates a table-level lock request to all participating nodes. After all nodes return the lock result and all nodes have successfully acquired the lock, the coordinating node executes the next operation. If a lock conflict occurs, the current index creation operation is terminated. Acquire a global write lock and perform a lock upgrade operation. After acquiring the global write intention lock, the coordinating node sends a table-level write lock request to all participating nodes. After acquiring the global write lock, the coordinating node calculates the transaction commit timestamp based on the lock information returned by each node. The new table index information and the new table structure version number are written to the first log for persistence, and the first log type is defined as PrepareLog; after the first log is successfully written, if a node failure occurs, the index creation process continues to be completed according to the first log information during the recovery process. Degrade the global write lock, create persistent storage, and update the index values ​​of the old data in the table; Write to the second log, which is of type PrepareIndexLog, and upgrade to a global write lock; if a node failure occurs after the second log is successfully written, the index creation process will continue to be completed based on the information in the second log during the recovery process. Write to the third log, which is of type CommitLog, and release the global write lock; if a node failure occurs after the third log is successfully written, the index creation process will continue to be completed based on the information in the third log during the recovery process. Write to the fourth log, which is of type CleanLog, to indicate that the table index was successfully created.

2. The method according to claim 1, characterized in that, The process of acquiring a global write intention lock and coordinating the node to initiate a table-level lock request to all participating nodes includes: The coordinating node initiates a table-level write intention lock request to all participating nodes, and waits for each node to return the write intention lock result; after all nodes have successfully acquired the lock, the coordinating node executes the next operation; If a locking conflict occurs, the current index creation operation will be terminated. Acquire a global write lock and perform a lock escalation operation. After acquiring the global write intention lock, the coordinating node initiates table-level write lock requests to all participating nodes. After acquiring the global write lock, the coordinating node calculates the transaction commit timestamp based on the locking information returned by each node. Specifically, after acquiring the global write lock, the coordinating node calculates the target transaction commit timestamp based on the locking information returned by each node, using the following formula: Ts = max(all node's commit_ts + 1, all node's Last_read_ts + 1) Wherein, commit_ts represents the commit timestamp of the last update of the table structure; Last_read_ts represents the most recent read of table structure information, which occurred during a read / write operation on the table.

3. The method according to claim 2, characterized in that, The coordinating node is the node that initiates the index creation transaction; The participating nodes are those that participate in the index creation transaction. They receive and process messages sent by the coordinating node, including acquiring locks on their own node and creating new table structure schema information locally. The coordinating node is also a participating node.

4. The method according to claim 1, characterized in that, After the first log is successfully written, if a node failure occurs, during the recovery process, the index creation process continues based on the information in the first log, including: After the first log is successfully written, if a node failure occurs, transaction recovery operations are performed based on the log; if the PrepareLog record exists in the log, a table-level write intention lock is acquired locally, and simultaneously... If the coordinating node fails, a new transaction state machine is created upon restart, and execution is restarted from the downgraded global write lock phase. If a participating node fails, upon restarting, the node's state will be restored to the state prior to the PrepareIndexLog record; the write intention lock at the table level on the node will be restored, and the old table structure information and the new table structure DitrySchema information will be created locally.

5. The method according to claim 1, characterized in that, The process of downgrading the global write lock, creating persistent storage, and updating the index values ​​of the old data in the table involves: the coordinating node initiating a lock downgrade request to downgrade all write locks in all nodes to write intention locks; and simultaneously, all nodes creating a new table structure, DirtySchema, locally. The coordinating node creates a new persistent storage index in the persistent storage layer; Simultaneously refresh the data in the distributed cache layer to the persistent storage layer on all nodes; The coordinating node reads old data from the persistent storage layer, generates new index data from the old data, and writes it to the distributed cache layer. After the old data generates index data, it performs a forced synchronization operation on the index data in the cache layer, refreshes it to the index table in the persistent storage layer, and removes the index data generated from the old data from the distributed cache layer.

6. The method according to claim 1, characterized in that, The process involves writing to a second log, of type PrepareIndexLog, and escalating the global write lock. After the second log is successfully written, if a node failure occurs, the index creation process continues based on the second log information during recovery, including: Write to the second log, the second log type being the PrepareIndexLog, to indicate that the transaction that created the table index has completed the update operation of the index value of the old data; If a system failure occurs and the PrepareIndexLog record exists in the log, the failed node will restore its state to the state before the CommitLog. If the coordinating node fails, the write lock at the local table level is restored, and a new table structure, DitrySchema information, is created locally; a new transaction state machine is created, and the transaction is re-executed from the global write lock escalation phase. If a participating node fails, it is necessary to restore the local table-level write lock and simultaneously create the old table structure information and the new table structure DitrySchema information locally.

7. The method according to claim 1, characterized in that, The process involves writing to the third log, where the third log type is CommitLog, and releasing the global write lock. If a node failure occurs after the third log is successfully written, the index creation process continues based on the third log information during the recovery process. This includes writing to the third log, where the third log type is CommitLog, to indicate that the transaction has completed the persistent storage layer operation. If a failure occurs, a CommitLog record will be stored in the log, restoring the node's state to the state before the CleanLog record. If the coordinating node fails, the old table structure information and the new table structure information are created locally. Then, a new transaction state machine is created, and the transaction is re-executed from the phase of releasing the global write lock. If a participating node fails, a new table structure is created locally.

8. The method according to claim 1, characterized in that, The specific steps of releasing the global write lock are as follows: The coordinating node initiates a lock release request, and each participating node releases the table-level write lock on its local table. The newly created table structure information is submitted as the table structure information, and each node performs table structure conversion while holding a global write lock.

9. The method according to claim 1, characterized in that, The writing to the fourth log, of type CleanLog, indicates that the table index was successfully created, including: The log contains a CleanLog entry. The failure occurred after writing the CleanLog. The current table index creation transaction has been successfully completed without any failure. Neither the coordinating node nor the participating nodes need to perform any operations. Based on the fourth log, the log layer deletes records in the log related to the creation of table index transactions.

10. An online table index creation system in a distributed system, used to implement the method according to any one of claims 1-9, characterized in that, The system includes: a distributed caching layer, a persistent storage layer, and a log layer; wherein, The distributed caching layer is used to cache frequently accessed data and newly written data. The persistent storage layer is used to store baseline data; Data in the distributed cache layer is asynchronously written to the persistent storage layer; the distributed cache layer and the persistent storage layer synchronize and unify data through a checkpoint service. The log layer is responsible for persisting transaction request information.

Images