A method, apparatus, device and storage medium for adding an index

Abstract

The present disclosure provides a method, device and equipment for adding index, and a storage medium, relates to the technical field of data processing, and particularly relates to the fields of information flow, database, knowledge graph and the like. The specific implementation scheme is as follows: a metadata node sends a transaction check task to at least one computing node in response to the index state being changed to only visible to write operation; the at least one computing node receives the transaction check task sent by the metadata node, checks whether a target transaction is completed, and returns a task completion message to the metadata node in response to the target transaction being completed; the metadata node receives the task completion message returned by all computing nodes in the at least one computing node, and issues a write-back task to the at least one computing node, and the write-back task is used for the at least one computing node to add index. The present disclosure can reduce the influence of adding index on transaction operation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to the fields of information flow, database, knowledge graph, etc. BACKGROUND

[0002] A large amount of data is generally stored in a database, and the data can be provided with an index. The index is provided to accelerate the retrieval efficiency of the data in the database. Simply put, the index can be equivalent to the table of contents of a book, and the required content can be quickly found in the book according to the table of contents. SUMMARY

[0003] The present disclosure provides an index adding method, device, equipment and storage medium.

[0004] According to a first aspect of the present disclosure, an index adding method is provided, comprising:

[0005] In response to the index state changing to be visible only to write operations, a transaction checking task is sent to at least one computing node, the transaction checking task is used for the at least one computing node to check whether a target transaction is completed, and in response to the target transaction being completed, the at least one computing node returns a task completion message to a metadata node, wherein the target transaction includes a transaction executed by a computing node and having a transaction start time before an index state update time, and the index state update time is a time when the index state changes to be visible only to write operations;

[0006] Task completion messages returned by all computing nodes in the at least one computing node are received, and a write-back task is issued to the at least one computing node, the write-back task being used for the at least one computing node to add an index.

[0007] According to a second aspect of the present disclosure, an index adding method is provided, comprising:

[0008] A transaction checking task sent by a metadata node is received, and it is checked whether a target transaction is completed; wherein the target transaction includes a transaction executed by a computing node and having a transaction start time before an index state update time, the index state update time being a time when the index state changes to be visible only to write operations, and the transaction checking task being sent by the metadata node to at least one computing node in response to the index state changing to be visible only to write operations;

[0009] In response to the target transaction being completed, a task completion message is returned to the metadata node, the task completion message being used for the metadata node to receive task completion messages returned by all computing nodes in the at least one computing node, and a write-back task is issued to the at least one computing node, the write-back task being used for the at least one computing node to add an index.

[0010] According to a third aspect of the present disclosure, a system for index addition is provided, comprising: a metadata node and a computing node;

[0011] The metadata node is configured to send a transaction check task to at least one computing node in response to the index state being changed to be visible only to write operations;

[0012] The at least one computing node is configured to receive the transaction check task sent by the metadata node, check whether a target transaction is completed, and return a task completion message to the metadata node in response to the target transaction being completed, wherein the target transaction comprises a transaction executed by the computing node and having a transaction start time before an index state update time, and the index state update time is a time when the index state is changed to be visible only to write operations.

[0013] The metadata node is further configured to receive the task completion messages returned by all computing nodes in the at least one computing node, and send a write-back task to the at least one computing node, wherein the write-back task is used for the at least one computing node to add an index.

[0014] According to a fourth aspect of the present disclosure, a device for index addition is provided, comprising:

[0015] A sending module is configured to send a transaction check task to at least one computing node in response to the index state being changed to be visible only to write operations, wherein the transaction check task is used for the at least one computing node to check whether a target transaction is completed and return a task completion message to the metadata node in response to the target transaction being completed, and the target transaction comprises a transaction executed by the computing node and having a transaction start time before an index state update time, and the index state update time is a time when the index state is changed to be visible only to write operations.

[0016] A receiving module is configured to receive the task completion messages returned by all computing nodes in the at least one computing node.

[0017] A sending module is configured to send a write-back task to the at least one computing node, wherein the write-back task is used for the at least one computing node to add an index.

[0018] According to a fifth aspect of the present disclosure, a device for index addition is provided, comprising:

[0019] The checking module is configured to receive a transaction checking task sent by the metadata node, and check whether a target transaction is completed, wherein the target transaction comprises a transaction executed by the computing node and having a transaction start time before an index state update time, the index state update time is a time when the index state changes to be visible only to write operations, and the transaction checking task is sent by the metadata node to at least one computing node in response to the index state changing to be visible only to write operations.

[0020] The returning module is configured to return a task completion message to the metadata node in response to the target transaction being completed, the task completion message is used for the metadata node to receive the task completion message returned by all computing nodes in the at least one computing node, and a write-back task is issued to at least one computing node, the write-back task is used for at least one computing node to add an index.

[0021] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising:

[0022] at least one processor; and

[0023] a memory connected to the at least one processor in communication; wherein

[0024] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect or the second aspect.

[0025] According to a seventh aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect or the second aspect.

[0026] According to an eighth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method of the first aspect or the second aspect.

[0027] The present disclosure can reduce the impact of adding an index on transaction operations.

[0028] It should be understood that the contents described in this part are not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0030] Figure 1 is a flowchart of the method for adding an index provided by the embodiments of the present disclosure;

[0031] Figure 2 is another flow chart of the method for adding index provided by the embodiments of the present disclosure;

[0032] Figure 3 is a structural schematic diagram of the system for adding index provided by the embodiments of the present disclosure;

[0033] Figure 4 is a schematic diagram of applying the method for adding index provided by the embodiments of the present disclosure;

[0034] Figure 5 is another schematic diagram of applying the method for adding index provided by the embodiments of the present disclosure;

[0035] Figure 6 is a first structural schematic diagram of the device for adding index provided by the embodiments of the present disclosure;

[0036] Figure 7 is a second structural schematic diagram of the device for adding index provided by the embodiments of the present disclosure;

[0037] Figure 8 is a third structural schematic diagram of the device for adding index provided by the embodiments of the present disclosure;

[0038] Figure 9 is a fourth structural schematic diagram of the device for adding index provided by the embodiments of the present disclosure;

[0039] Figure 10 is a fifth structural schematic diagram of the device for adding index provided by the embodiments of the present disclosure;

[0040] Figure 11 is a sixth structural schematic diagram of the device for adding index provided by the embodiments of the present disclosure;

[0041] Figure 12 is a block diagram of an electronic device for implementing the method for adding index provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help in understanding. These should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0043] Data in the database is stored in units of tables, and the data of the tables includes metadata (schema) and data (Data). Online index adding (Online DDL) technology needs to modify the metadata and the data at the same time.

[0044] The index adding operation should be transparent to the business, and therefore concurrent execution of the index operation and normal data manipulation language (DML) operation of the business is a core requirement of the user, and is also a problem to be solved by the online index adding.

[0045] In the embodiments of the present disclosure, the metadata and the data are stored in storage nodes. A metadata node is used to control the index adding process, and the metadata node can also be understood as a control node used to control the tasks executed by the computing nodes, and the specific index adding process is executed by the computing nodes.

[0046] The metadata node can monitor the execution of transactions on the computing nodes. Specifically, in response to the index state being changed to be visible only to write operations, a transaction checking task is sent to at least one computing node. The computing node receiving the transaction checking task checks whether a target transaction is completed, and in response to the target transaction being completed, the at least one computing node returns a task completion message to the metadata node. The target transaction includes a transaction executed by the computing node and having a transaction start time before an index state update time, and the index state update time is a time when the index state is changed to be visible only to write operations.

[0047] The metadata node receives the task completion messages returned by all the computing nodes in the at least one computing node, and issues a write-back task to the at least one computing node. That is, the write-back task can be issued to all the computing nodes, or the write-back task can be issued to part of the computing nodes in all the computing nodes, for example, one computing node in all the computing nodes. The computing node receiving the write-back task adds the index. The index is for the metadata and the data stored in the storage nodes. For example, the index adding instruction can indicate information of the metadata to be added with the index, such as identification information and a location of the metadata to be added with the index. When the index is added, the index is added for the metadata to be added with the index and the data of the metadata indicated in the index adding instruction.

[0048] In the embodiments of the present disclosure, the index is added after the transaction executed by the computing node and started before the index state update time is completed, because the lock table cannot be executed after the lock table, and the transaction operation of the data targeted by the lock table cannot be executed, the lock table and the index are added after the transaction executed by the computing node and started before the index state update time is completed, which does not affect the transaction that has been executed before, and only the lock table is performed in the process of adding the index, the time of the lock table is relatively short, and the influence on the subsequent transaction operation to be executed is also relatively small. In general, the embodiments of the present disclosure can realize the compatibility of transaction operation and index addition, and can reduce the influence of index addition on transaction operation.

[0049] The compatibility of transaction operation and index addition can also be understood as online index addition. The transaction operation can also be understood as a business operation, which can include access to data stored in the storage node, such as DML operation, etc. The embodiments of the present disclosure can realize that the execution of DML is not prohibited in the process of adding the index, and can reduce the influence of index addition on transaction operation.

[0050] In the related art, the update of the lease index state is increased, and the state needs to be updated every time a lease period is waited for. The lease period can be a preset fixed time length, such as 1 minute, etc., to ensure that all nodes obtain the latest version of the schema (metadata), and the transaction whose execution time exceeds a lease period will be rolled back to ensure that the transaction uses only one schema version. However, this case will cause long transaction rollback, which can be understood as that the long transaction needs to be restarted due to expiration of the lease without being executed. The rollback not only affects the transaction operation, but also wastes resources to repeatedly execute the transaction that is rolled back, causing waste of resources. The embodiments of the present disclosure do not need to roll back the transaction, which can not only reduce the influence of index addition on transaction operation, but also reduce resource waste, and realize low-cost index addition.

[0051] The method for adding an index provided by the embodiments of the present disclosure is described in detail below.

[0052] The embodiments of the present disclosure provide a method for adding an index, comprising:

[0053] In response to the index state being changed to be visible only to write operations, a transaction check task is sent to at least one computing node, the transaction check task is used for the at least one computing node to check whether a target transaction is completed, and in response to the target transaction being completed, the at least one computing node returns a task completion message to the metadata node, wherein the target transaction includes a transaction executed by the computing node and started before the index state update time, and the index state update time is the time when the index state is changed to be visible only to write operations;

[0054] receiving a task completion message returned by all of the at least one computing node, and issuing a write-back task to the at least one computing node, the write-back task being used for the at least one computing node to add the index.

[0055] In the embodiments of the present disclosure, the metadata node monitors the execution of the index addition after the completion of the transaction whose transaction start time is before the index state update time, because the lock table cannot be executed after the lock table, and the transaction operation on the data targeted by the lock table cannot be executed after the lock table, and the lock table is performed and the index is added after the completion of the transaction whose transaction start time is before the index state update time executed by the computing node, which does not affect the transaction that has been executed before, and only the lock table is performed during the execution of the index addition process, and the time of the lock table is relatively short, so the influence on the subsequent to-be-executed transaction operation is also relatively small. In general, the embodiments of the present disclosure can realize the compatibility of the transaction operation and the index addition, and can reduce the influence of the index addition on the transaction operation.

[0056] The method for adding an index provided by the embodiments of the present disclosure can be applied to a metadata node. Referring to Figure 1 The method for adding an index provided by the embodiments of the present disclosure can include:

[0057] S101, in response to the index state being changed to be visible only to write operations, a transaction check task is sent to at least one computing node.

[0058] The transaction check task is used for the at least one computing node to check whether a target transaction is completed, and in response to the completion of the target transaction, the at least one computing node returns a task completion message to the metadata node.

[0059] The target transaction includes a transaction whose transaction start time is before the index state update time and which is executed by the computing node.

[0060] Specifically, the target transaction can include a transaction on the computing node for the metadata and the data to be added to the index. That is, the transaction targeted by the computing node when checking is a transaction on the computing node for the metadata and the data to be added to the index, and the transaction start time is before the index state update time.

[0061] Only the transaction for the metadata and the data to be added to the index is checked during the index addition process, for example, the storage node includes multiple tables, when only one table is added to the index, only the transaction whose transaction start time is before the index state update time and which is executed by the computing node for the table is checked, so it is not necessary to check all the data stored in the storage node, which can reduce the calculation amount and avoid the influence on other data that does not need to be added to the index.

[0062] The index state update time is the time when the index state is changed to be visible only to write operations.

[0063] When there are multiple computing nodes, the metadata node can broadcast a transaction checking task, each computing node receiving the transaction checking task checks whether a transaction executed by the computing node and started before the index state update time is completed, and in response to the transaction executed by the computing node and started before the index state update time being completed, returns a task completion message to the metadata node.

[0064] The transaction checking task is sent to all computing nodes, that is, for each computing node, the metadata node sends the transaction checking task to the computing node regardless of whether the computing node is in the same machine room as the storage node.

[0065] When the metadata node changes the index state to be visible only to write operations, the metadata node can record the current time, and the recorded time is the time when the index state is changed to be visible only to write operations, that is, the index state update time.

[0066] In an implementation manner, after the metadata node changes the index state to be visible only to write operations, the metadata node waits for a heartbeat period and broadcasts a transaction checking task to all computing nodes. The heartbeat period can be a preset time length, or the heartbeat signal of the storage node or the computing node can be detected. When the metadata node receives a heartbeat signal sent by the storage node or the computing node, it can be understood that the metadata node has waited for a heartbeat period.

[0067] Each computing node receiving the transaction checking task checks the transaction state of a transaction being executed on the database table of the computing node, specifically, checks the completion of a transaction executed by the computing node and started before the index state update time for the database table, wherein the transaction includes a transaction for the database table, such as a data manipulation language (DML) operation. After each computing node checks that the transaction executed by the computing node and started before the index state update time is completed, the computing node can return a task completion message to the metadata node.

[0068] S102, receiving the task completion message returned by all computing nodes in the at least one computing node, and issuing a write-back task to the at least one computing node.

[0069] The write-back task is used for the at least one computing node to add an index.

[0070] When all computing nodes return the task completion message to the metadata node, the metadata node receives the task completion message returned by all computing nodes in the at least one computing node, and can issue a write-back task to the at least one computing node. In this way, the computing node can add an index after receiving the write-back task.

[0071] The computing node receiving the write-back task performs index addition. The index is for metadata and data stored in the storage node, for example, the index addition instruction can indicate information of metadata to be added with an index, such as identification information and a location of the metadata to be added with an index, and when the index is added, the index is added for the metadata to be added with an index indicated in the index addition instruction and data of the metadata.

[0072] The storage node can store metadata and data in the form of a table, for example, a main table. The content stored in the main table can be updated, for example, added or deleted. In order to avoid inconsistency of data caused by updating of the main table during the process of adding an index, the main table is locked during the process of adding an index in the embodiment of the disclosure, and specifically, the computing node locks the main table and adds an index. For example, the access operation for the main table is temporarily stopped, and after the index is added, the access operation is received again. There can be multiple main tables in the storage node, and the main table to be added with an index can be locked during the process of adding an index.

[0073] The online index addition for the distributed database can be implemented without destroying data integrity, that is, the index is for data stored in the storage node, and the data stored in the storage node can be queried through the added index.

[0074] When there are multiple computing nodes, the process of adding an index can be performed by multiple computing nodes, for example, the write-back task of adding an index is divided into multiple index tasks, which are assigned to each computing node, and the index task assigned to each computing node is sent to each computing node, and the index addition process is completed by each computing node performing the index task respectively. Alternatively, one of the computing nodes can be selected as a target computing node, and the target computing node performs the process of adding an index.

[0075] In an implementation manner, the computing node in the same machine room as the storage node can be determined as the target computing node.

[0076] For example, the machine room position of the storage node is determined first, and then the computing node in the same machine room as the storage node is selected as the target computing node.

[0077] The write-back task is sent to at least one computing node, which can include sending the write-back task to the target computing node.

[0078] The write-back task can be sent only to the computing node in the same machine room as the storage node.

[0079] In this way, the selected target computing node completes the index addition.

[0080] The data in the database table is stored on the storage node, the index is added for the data in the database table, and the computing node in the same machine room as the storage node is selected to add the index for the database table, which can reduce network overhead and the like.

[0081] In an optional embodiment, the metadata node can control the concurrency of the write-back task through a control instruction during the write-back task process or after the write-back task is issued, so as to realize task concurrency control. The control instruction can include the commands in Table 1 below. The ddlwork can represent the write-back task.

[0082] Table 1

[0083]

[0084] In the embodiment of the disclosure, the metadata node receives the task completion messages returned by all the computing nodes in the at least one computing node, and issues a write-back task to the at least one computing node, so that the computing node adds the index. Compared with the transaction whose execution time exceeds one lease period in the related art, the transaction will be rolled back. The active detection mechanism can be used to determine the transaction execution on the computing node, so that the index is added after the transaction operation such as the write operation is completed, thereby reducing the influence of the index addition on the transaction operation. The rollback not only causes the influence on the transaction operation, but also wastes resources to repeatedly execute the rolled-back transaction, causing resource waste. The embodiment of the disclosure does not need to roll back the transaction, but also saves the resources occupied by the executed rolled-back transaction, thereby realizing low-overhead index addition.

[0085] In an optional embodiment, the method for adding an index provided by the embodiment of the disclosure further includes: S102 includes: after receiving the task completion messages returned by all the computing nodes in the at least one computing node, changing the index state from only visible to write operation to data reorganization state, and after two heartbeat periods after the index state is changed from only visible to write operation to data reorganization state, issuing a write-back task to the at least one computing node.

[0086] When the index is to be added, the user end can send an index addition instruction to the computing node. After the computing node receives the index addition instruction, the computing node forwards the index addition instruction to the metadata node. That is, the computing node interacts with the user end, and the metadata node is used to monitor the execution of the transaction on the computing node and issue a write-back task to the computing node and the like.

[0087] The metadata node controls the index state.

[0088] The index state can be changed in the following order: invisible to any operation, visible only to delete operation, visible only to write operation, data reorganization state, and available to any operation.

[0089] The index state being visible only to delete operation means invisible to insert, select, etc. The index state being visible only to write operation means visible to update, insert, delete, etc. but not readable. The data reorganization state can be understood as a process of reorganization in this state, such as adding an index in this state.

[0090] In this state where the index state is set to invisible to any operation, the metadata node can generate index information according to the metadata change information, generate a write-back task such as a ddlwork task according to the main table information and persist, the state of each task can include Idle (to be processed), Doing (being processed), Done (completed), and high availability can be achieved through raft (consensus algorithm) synchronization. The index information can include index data, index data addition position, etc.

[0091] When the metadata node issues a write-back task to at least one computing node, it can issue index information, so that the computing node can add an index according to the index information.

[0092] The metadata node waits for the heartbeat request of the computing node and the storage node, changes the index state once every two heartbeat periods, and the index state changes from invisible to any operation to visible only to delete operation to visible only to write operation.

[0093] In response to the index state change to visible only to write operation, a transaction check task is sent to at least one computing node, which can include waiting for a heartbeat period after the index state changes to visible only to write operation to issue a transaction check task to the computing node, such as broadcasting a transaction check task to all computing nodes.

[0094] The computing node executes the transaction check task, and the transaction executed by the computing node and started before the index state update time is completed, and a task completion message is returned to the metadata node. After the metadata node receives the task completion message returned by all computing nodes, the index state can be changed to a data reorganization state. The computing node selected from the same machine room as the storage node can be sent a write-back task after the index state changes to the data reorganization state for two heartbeat periods, and the metadata node changes the index state to available to any operation after the write-back task is completed. After the index state changes, the changed index state can be synchronized to the computing node and the storage node, so that the process of adding an index is completed.

[0095] In the related art, after receiving an index adding instruction, the table is directly locked and then the index is added. The table is locked to prohibit read and write operations to ensure the atomicity of metadata and data modification. However, the execution time of adding the index may be several minutes or even several hours, and long-time write prohibition in a production environment is unacceptable for business.

[0096] In the embodiments of the present disclosure, the metadata node responds to the index state change to be visible only to write operations by sending a transaction checking task to at least one computing node and responds to the index state change to be a data reorganization state by sending a write-back task to at least one computing node. The computing node adds the index, and the process of adding the index is implemented after the transaction executed by the computing node is completed, specifically, after the transaction executed by the computing node and started before the index state update time is completed. The metadata node monitors the execution of the index addition after the transaction executed by the computing node and started before the index state update time is completed. Because the table is locked, the transaction operation on the data for which the table is locked cannot be performed. The table is locked and the index is added after the transaction executed by the computing node and started before the index state update time is completed, which does not affect the transaction that has been executed before. The table is locked only during the process of adding the index, and the time of locking the table is relatively short. Therefore, the influence on the subsequent transaction operation to be executed is relatively small. In general, the embodiments of the present disclosure can realize the compatibility of transaction operation and index addition, and can reduce the influence of index addition on transaction operation.

[0097] The embodiments of the present disclosure can add a global index or / and a local index. The global index means that the index data is maintained separately, has its own region (shard), and can also be understood as the index data being stored separately from the main table. The local index means that the index data and the main table are together and belong to the same region.

[0098] In an optional embodiment, the index adding instruction includes an index type.

[0099] When the index type indicates that the index to be added is a global index, the metadata change information can also be obtained, and a global index shard can be created according to the metadata change information, wherein the global index shard is independent of the main table, and the global index shard is used to store global index data.

[0100] The computing node responds to the received write-back task, locks the main table, and adds global index data in the global index shard.

[0101] When the index type indicates that the index to be added is a local index, the computing node, in response to receiving the backwrite task, locks the master table and adds the local index data in the master table, which can also be understood as storing the local index data together with the master table, such as storing the local index data in the same shard as the master table.

[0102] In this way, the global index or / and the local index can be added according to actual needs, and the global index is maintained separately, and the local index is maintained together with the master table.

[0103] The present disclosure also provides a method for adding an index applied to a computing node, which is described in detail below.

[0104] The present disclosure provides a method for adding an index, which can include:

[0105] A transaction checking task sent by a metadata node is received, and it is checked whether a target transaction is completed; the target transaction includes a transaction executed by the computing node and having a transaction start time before an index state update time, the index state update time is a time when the index state is changed to be visible only to write operations, and the transaction checking task is sent by the metadata node to at least one computing node in response to the index state being changed to be visible only to write operations;

[0106] In response to the target transaction being completed, a task completion message is returned to the metadata node, the task completion message is used for the metadata node to receive the task completion message returned by all computing nodes in the at least one computing node, and a backwrite task is issued to the at least one computing node, the backwrite task is used for the at least one computing node to add an index.

[0107] In the present disclosure, the metadata node monitors the execution of the transaction executed by the computing node and having a transaction start time before the index state update time after the transaction is completed, and then executes the addition of the index, because the table is locked, the transaction operation of the data for which the table is locked cannot be executed, and the table is locked and the index is added after the transaction executed by the computing node and having a transaction start time before the index state update time is completed, which does not affect the transaction that has been executed before, and the table is locked only during the process of adding the index, and the time of locking the table is relatively short, so the influence on the subsequent to-be-executed transaction operation is also relatively small. Overall, the present disclosure can realize the compatibility of the transaction operation and the addition of the index, and can reduce the influence of the addition of the index on the transaction operation.

[0108] As shown in Figure 2 The method for adding an index applied to a computing node provided by the present disclosure can include:

[0109] S201, a transaction checking task sent by a metadata node is received, and it is checked whether a target transaction is completed.

[0110] The target transaction includes a transaction executed by the computing node and having a transaction start time before the index state update time.

[0111] The target transaction can include a transaction executed by the computing node and directed to metadata and data to be indexed. That is, the transaction checked by the computing node is a transaction executed by the computing node and directed to metadata and data to be indexed, and the transaction has a transaction start time before the index state update time.

[0112] In the index adding process, only transactions directed to metadata and data to be indexed are checked. For example, when indexes are added to only one table in a storage node including multiple tables, whether transactions executed by the computing node and directed to the table and having a transaction start time before the index state update time are completed can be checked, so that the computing amount can be reduced without affecting other data without indexes.

[0113] The index state update time is a time when the index state is changed to be visible only to write operations, and the transaction checking task is sent by the metadata node to at least one computing node in response to the index state being changed to be visible only to write operations.

[0114] S202, in response to the target transaction being completed, a task completion message is returned to the metadata node.

[0115] The task completion message is used for the metadata node to receive the task completion message returned by all computing nodes in the at least one computing node, and to issue a write-back task to the at least one computing node, the write-back task being used for the at least one computing node to add indexes.

[0116] When the metadata node changes the index state to be visible only to write operations, the current time can be recorded, and the recorded time is the time when the index state is changed to be visible only to write operations, that is, the index state update time.

[0117] In an implementation manner, after the metadata node changes the index state to be visible only to write operations, a heartbeat cycle is waited for, and a transaction checking task is broadcast to all computing nodes. Each computing node receives the transaction checking task, checks the transaction state of a database table being executed on the computing node, and specifically checks the completion of a transaction executed by the computing node and directed to the database table and having a transaction start time before the index state update time, wherein the transaction includes a transaction directed to the database table, such as a data manipulation language (DML) operation. After each computing node checks the transaction executed by the computing node and having a transaction start time before the index state update time, a task completion message can be returned to the metadata node.

[0118] When there are multiple computing nodes, the metadata node can issue the write-back task to multiple computing nodes, or select one of the computing nodes and issue the write-back task to the selected computing node. For the computing nodes, when the metadata node issues the write-back task to multiple computing nodes, the multiple computing nodes can receive the write-back task, and when the metadata node issues the write-back task to only one selected computing node, the selected computing node receives the write-back task.

[0119] In response to receiving the write-back task, the computing node that can receive the write-back task adds the index.

[0120] The write-back task can be implemented through all or part of the computing nodes, and the computing nodes for adding the index can be more flexibly selected.

[0121] The computing node that receives the write-back task adds the index. The index is for the metadata and data stored in the storage node, for example, the index addition instruction can indicate the information of the metadata to be added, such as the identification information and location of the metadata to be added, and when the index is added, the index is added for the metadata to be added indicated in the index addition instruction and the data of the metadata.

[0122] In an implementation manner, the metadata node can determine the computing node in the same machine room as the storage node as a target computing node, and issue the write-back task to the target computing node, that is, the computing node in the same machine room as the storage node can be selected to add the index for the database table, the data in the database table is stored in the storage node, and the index is added for the data in the database table. Selecting the computing node in the same machine room as the storage node to add the index for the database table can reduce network overhead and the like.

[0123] In an optional embodiment, the method for adding the index applied to the computing node can further include:

[0124] Receiving the index addition instruction and forwarding the index addition instruction to the metadata node.

[0125] The index addition instruction is used for the metadata node to receive the index addition instruction, set the index state to be invisible to any operation, and in response to the index state being visible only to the write operation, send a transaction check task to at least one computing node.

[0126] The index state can be changed in the following order: invisible to any operation, visible only to the delete operation, visible only to the write operation, data reorganization state, and available to any operation.

[0127] The computing node receives the index adding instruction and forwards the index adding instruction to the metadata node, and the metadata node sets the index state as invisible to any operation after receiving the index adding instruction.

[0128] While setting the index state as invisible to any operation, the metadata node can generate index information according to the metadata change information, generate a write-back task such as a ddlwork task according to the main table information and persist, and the state of each task can include Idle (to be processed), Doing (being processed), and Done (completed), which can be synchronized through raft (consensus algorithm) to achieve high availability. The index information can include index data, index data adding position, etc.

[0129] The metadata node waits for the heartbeat request of the computing node and the storage node, changes the index state every two heartbeat periods, and the index state changes from invisible to any operation to visible only to delete operation to visible only to write operation.

[0130] The metadata node can issue a transaction check task to the computing node when the index state changes to visible only to write operation, or can issue a transaction check task to the computing node after the index state changes to visible only to write operation, such as waiting for a heartbeat period to issue a transaction check task to the computing node after the index state changes to visible only to write operation, such as broadcasting a transaction check task to all computing nodes.

[0131] The computing node executes the transaction check task, and the transaction executed by the computing node and started before the index state update time can be completed, and the task completion message is returned to the metadata node, and the metadata node receives the task completion message returned by all computing nodes, and can change the index state to data reorganization state. The computing node and the storage node in the same machine room can be selected to send a write-back task after the index state changes to data reorganization state for two heartbeat periods, and the metadata node changes the index state to available to any operation after the write-back task is completed, and the changed index state can be synchronized to the computing node and the storage node after the index state is changed, and thus the process of adding the index is completed.

[0132] In the embodiments of the present disclosure, in response to the index state being changed to only visible to write operation, the metadata node sends a transaction check task to at least one computing node, and in response to the index state being changed to data reorganization state, the metadata node sends a write-back task to at least one computing node, and the computing node adds an index, so that the process of adding the index is implemented after the transaction executed by the computing node is completed, specifically, the process of adding the index is implemented after the transaction executed by the computing node and the start time of the transaction is before the index state update time, which can avoid directly locking the table to add the index after receiving the index adding instruction, and the table cannot be operated after being locked, thereby reducing the influence of the index operation on the transaction operation.

[0133] In order to ensure the consistency of data, the computing node locks the master table during the process of adding the index, and specifically, the computing node locks the master table and adds the index. For example, the access operation for the master table is temporarily stopped, and after the adding of the index is completed, the access operation is received again.

[0134] In an optional embodiment, the index adding instruction includes an index type.

[0135] When the index type indicates that the index to be added is a global index, S202 includes:

[0136] In response to receiving the write-back task, the at least one computing node locks the master table and adds index data in the global index shard. The global index shard is created according to the metadata change information after the metadata node obtains the metadata change information, and the global index shard is independent of the master table.

[0137] Specifically, the computing node can lock the data stored by the storage node of the master table, and add the index data of the global index in the storage node of the global index for the data of the master table.

[0138] When the index type indicates that the index to be added is a local index, in response to receiving the write-back task, the index is added, including:

[0139] In response to receiving the write-back task, the at least one computing node locks the master table and adds local index data in the master table. It can also be understood that the local index data is stored in the same shard as the master table.

[0140] The computing node can lock the data stored by the storage node of the master table, and directly add the index data of the local index in the master table.

[0141] According to actual needs, the global index or / and the local index can be added, so that the global index is independently maintained, and the local index is maintained together with the master table.

[0142] The embodiments of the present disclosure also provide a system for adding an index, including Figure 3As shown, it may include: metadata node 301 and compute node 302.

[0143] Metadata node 301 is used to send a transaction check task to at least one compute node 302 in response to an index state change that is visible only to write operations.

[0144] At least one compute node 302 is configured to receive a transaction check task sent by a metadata node 301, check whether the target transaction is completed, and in response to the completion of the target transaction, at least one compute node returns a task completion message to the metadata node 301, wherein the target transaction includes a transaction executed by the compute node whose transaction start time is before the index status update time, and the index status update time is the time when the index status changes to be visible only to write operations.

[0145] Metadata node 301 is also used to receive task completion messages returned by all compute nodes in at least one compute node 302, and to issue write-back tasks to at least one compute node 302. The write-back tasks are used to add indexes to at least one compute node.

[0146] At least one compute node 302 is also used to receive index addition instructions and forward the index addition instructions to the metadata node 301.

[0147] Metadata node 301 is also used to identify compute node 302, which is in the same data center as the storage node, as the target compute node; and to send write-back tasks to the target compute node.

[0148] The index addition instruction includes the index type, which indicates whether the index to be added is a global index or a local index.

[0149] At least one compute node 302 is also configured to, when the index type indicates that the index to be added is a global index, lock the main table and add global index data in the global index shard in response to receiving a write-back task; and when the index type indicates that the index to be added is a local index, lock the main table and add local index data in the main table in response to receiving a write-back task.

[0150] The various processes executed by the metadata node and compute node in the index-adding system have been described in detail in the above-described index-adding method embodiments, and will not be repeated here.

[0151] This disclosure provides a specific embodiment for adding a global index, such as... Figure 4 As shown.

[0152] For ease of description, the relevant terms in the embodiments of this disclosure are first explained as shown in Table 2.

[0153] Table 2

[0154]

[0155]

[0156] meta performs the steps performed by the metadata node in the above embodiments, and db performs the steps performed by the computing node in the above embodiments.

[0157] db receives an index addition instruction, such as a user-triggered Online DDL instruction, and forwards the index addition instruction to meta.

[0158] meta creates region shards of the global index according to the schema (metadata) change information, generates index information at the same time, sets the index state to invisible to any operation, such as state: None, generates a ddlwork task according to the region information of the primary table and persists it, and the state (Idle, Doing, Done) of each task is synchronized through raft to achieve high availability.

[0159] Wait for the heartbeat request of db and store, and meta changes the index state every two heartbeat periods. The index state changes from None to IS DELETE ONLY (visible only to delete operation) to IS WRITE ONLY (visible only to write operation).

[0160] meta maintains a list of all dbs. When the index state of meta is IS WRITE ONLY, meta waits for a heartbeat period to broadcast a txn task to all dbs, and checks the transaction state of the table being executed on each db, i.e., each db checks the transaction state of the table being executed on itself.

[0161] db records the current time as write only time, i.e., the index state update time, when updating the index state to IS WRITE ONLY. The check_txn task checks the completion of all transactions on the db whose start time is before write only time. When all transactions on the db whose start time is before write only time are completed, the db replies to meta that the task is completed. When all dbs reply to meta, meta changes the index state to IS WRITE LOCAL (data reorganization state).

[0162] The meta starts to select the db in the same machine room as the store to send a write_back task after two heartbeat periods of the index state IS_WRITE_LOCAL. Because it is a global index, the write_back task needs to return the master table data from the store lock and then write the global index data to the global index region. The data consistency is guaranteed by the distributed transaction. When sending the write_back task, the task concurrency control can be provided to control the concurrency of the ddl work. At the same time, rich control instructions are provided. The specific control commands are shown in Table 1.

[0163] When all the write_back tasks are completed, the meta changes the index state to IS_PUBLIC (available for any operation) and synchronizes it to the db and the store through the heartbeat. Thus, the global index Online DDL process is completed.

[0164] The store also has schema information in the embodiment of the present disclosure. The index state compatibility of the db and the store is well handled. At the same time, the kv encoding of the stored data can be directly performed on the store.

[0165] The embodiment of the present disclosure provides a specific embodiment of adding a local index, as shown in Figure 5 .

[0166] The db receives an index adding instruction, such as an Online DDL instruction triggered by a user, and forwards the index adding instruction to the meta.

[0167] The meta generates index information according to the schema change information, sets the index state to None, generates a ddl_task according to the master table region information and persists it. The state (Idle, Doing, Done) of each task is synchronized through raft (consensus algorithm) to achieve high availability.

[0168] The heartbeat request of the db and the store is waited for to update the schema state. The meta changes the index state once every two heartbeat periods. Finally, the index state changes from None to IS_DELETE_ONLY to IS_WRITE_ONLY.

[0169] The meta maintains a list of all the databases. When the index state of the meta is IS_WRITE_ONLY, the meta waits for a heartbeat cycle to broadcast a check_txn task to all the databases. The check_txn task checks the transaction state of all the transactions being executed on the table in all the databases. When the index state of the database is updated to the IS_WRITE_ONLY state in the previous heartbeat, the database records the current time as the write_only_time. The check_txn task checks the completion of all the transactions on the database, which started before the write_only_time. When all the transactions on the database are completed, the database replies to the meta task. When all the databases reply to the meta, the meta changes the index state to IS_WRITE_LOCAL.

[0170] The meta starts to select the databases in the same machine room as the store to send a write_back task after two heartbeat cycles in the index state IS_WRITE_LOCAL. Because the index is local, the write_back task directly locks the main table data on the store and writes the local index data. The entire execution is still completed through a distributed transaction. When the write_back task is sent, the task concurrency control can be provided to control the concurrency of the ddlwork, and rich control instructions are provided. The specific control commands are shown in Table 1.

[0171] When all the write_back task tasks are completed, the meta changes the index state to IS_PUBLIC and synchronizes the index state to the databases and the store through the heartbeat. Thus, the local index Online DDL process is completed.

[0172] The store also has schema information in the embodiments of the present disclosure. The index states of the databases and the store are compatible and well handled. The kv encoding of the stored data can be directly performed on the store.

[0173] The typical architecture of a distributed database is a structured query language distributed database (SQL on Distributed KV), that is, the system is composed of a distributed key-value (KV) storage and a plurality of stateless servers. The server is a structured query language (SQL) computing node, which can be horizontally scaled, and provides relational database query capability externally. Distributed databases such as TiDB (an open source distributed relational database), CockroachDB (a scalable open source database that can synchronize across data centers) are all similar architectures. In the related art, the SQL layer and the distributed KV storage layer are completely decoupled, the storage layer does not perceive schema (metadata) information, and KV encoding operations must be completed on the server. Some operations need to read data from the storage layer to the server first. In the embodiment of the present disclosure, the computing node and the storage node can synchronize schema information, and the KV encoding work is sunk to the storage layer, fully utilizing the computing power of the storage node. Because the index state change can be perceived, the state compatibility processing is simpler.

[0174] By actively detecting the execution of the long transaction on the db, the business long transaction does not need to be rolled back because the lease expires, ensuring that the impact on the business is minimized. Both the db and the store have schema information, the KV encoding work is sunk to the storage layer, fully utilizing the computing power of the storage node, because the index state change can be perceived, making the state compatibility processing simpler. In addition, the Online DDL task query, speed limit and suspension functions are provided, and the operation is flexible.

[0175] Corresponding to the method of adding an index provided in the above embodiment, the present embodiment also provides an apparatus for adding an index, as shown in Figure 6 may include:

[0176] The sending module 601 is configured to, in response to the index state change being only visible to write operations, send a transaction checking task to at least one computing node, the transaction checking task being used by the at least one computing node to check whether a target transaction is completed, and in response to the target transaction being completed, the at least one computing node returns a task completion message to the metadata node, wherein the target transaction includes a transaction executed by the computing node and having a transaction start time before an index state update time, and the index state update time is a time when the index state change is only visible to write operations;

[0177] The receiving module 602 is configured to receive the task completion message returned by all computing nodes in the at least one computing node;

[0178] The delivery module 603 is configured to deliver a write-back task to the at least one computing node, and the write-back task is used by the at least one computing node to add an index.

[0179] Optionally, the receiving module 602 is specifically configured to change the index state from being visible only to write operations to a data reorganization state after receiving the task completion messages returned by all the computing nodes in the at least one computing node;

[0180] The issuing module 603 is specifically configured to issue the write-back task to the at least one computing node after two heartbeat periods after the index state is changed from being visible only to write operations to the data reorganization state.

[0181] Optionally, as shown in Figure 7 the device further includes:

[0182] The determining module 701 is configured to determine a computing node in the same machine room as the storage node as a target computing node.

[0183] The issuing module 603 is specifically configured to issue the write-back task to the target computing node.

[0184] Optionally, the index adding instruction includes an index type.

[0185] As shown in Figure 8 the device includes:

[0186] The obtaining module 801 is configured to obtain metadata change information when the index type indicates that the index to be added is a global index.

[0187] The creating module 802 is configured to create a global index shard according to the metadata change information, wherein the global index shard is independent of the main table, and the global index shard is used to store global index data.

[0188] Corresponding to the method for adding an index provided in the above embodiments, the present disclosure further provides an index adding device, as shown in Figure 9 may include:

[0189] The checking module 901 is configured to receive a transaction checking task sent by the metadata node, and check whether a target transaction is completed, wherein the target transaction includes a transaction executed by the computing node and having a transaction start time before an index state update time, the index state update time is a time when the index state is changed to be visible only to write operations, and the transaction checking task is sent by the metadata node in response to the index state being changed to be visible only to write operations.

[0190] The returning module 902 is configured to return a task completion message to the metadata node in response to the target transaction being completed, the task completion message is used for the metadata node to receive the task completion messages returned by all the computing nodes in the at least one computing node, and issue a write-back task to the at least one computing node, and the write-back task is used for the at least one computing node to add an index.

[0191] Optional, such as Figure 10 As shown, the device also includes:

[0192] Add index module 1001 to add an index in response to receiving a write-back task.

[0193] Optional, such as Figure 11 As shown, the device also includes:

[0194] Receiver module 1101 is used to receive index addition instructions;

[0195] Forwarding module 1102 is used to forward index addition instructions to metadata nodes. After receiving the index addition instruction, the metadata node sets the index status to be invisible to any operation, and in response to the index status being visible only to write operations, sends a transaction check task to at least one compute node.

[0196] Optionally, the index addition instruction may include the index type;

[0197] The Add Index module 1001 is specifically used when the index type indicates that the index to be added is a global index. In response to receiving a write-back task, it locks the main table and adds global index data to the index shard. The global index shard is created by the metadata node after obtaining metadata change information. The global index shard is independent of the main table and is used to store global index data. When the index type indicates that the index to be added is a local index, in response to receiving a write-back task, it locks the main table and adds index data to the main table.

[0198] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0199] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0200] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0201] like Figure 12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0202] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of displays, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0203] The computing unit 1201 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as the method of adding an index to a metadata node, or the method of adding an index to a computing node. For example, in some embodiments, the method of adding an index to a metadata node, or the method of adding an index to a computing node, can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by computing unit 1201, it can perform one or more steps of the above-described method for adding an index to a metadata node, or the method for adding an index to a computing node. Alternatively, in other embodiments, computing unit 1201 can be configured to perform the above-described method for adding an index to a metadata node, or the method for adding an index to a computing node, by any other suitable means (e.g., by means of firmware).

[0204] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0205] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / operations specified in the flowchart and / or block diagram block or blocks. The program code can be retrieved from a machine-readable medium, a storage medium, or a memory medium associated with the machine or a server, or a combination thereof. The machine-readable medium, storage medium, or memory medium can be a machine readable signal medium or a machine readable storage medium, including but not limited to electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. The program code can be implemented in a machine language, a high-level procedural programming language, an object-oriented programming language, a scripting language, or any suitable combination thereof.

[0206] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of a program of a processor, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0207] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0208] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0209] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0210] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology disclosed in the present disclosure are achieved.

[0211] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and principles of the disclosure. Accordingly, the disclosure is not limited to the specific embodiments described above.

Claims

1. A method for adding an index, applied to a metadata node, the metadata node being a control node for monitoring execution of transactions on a computing node, the method comprising: in response to an index state being changed to be visible only to write operations, sending a transaction checking task to at least one computing node, the transaction checking task being used by the at least one computing node to check whether a target transaction is completed, and in response to the target transaction being completed, the at least one computing node returning a task completion message to the metadata node, wherein the target transaction comprises a transaction executed by the computing node on metadata and data to be added with an index and having a transaction start time before an index state update time, the index state update time being a time when the index state is changed to be visible only to write operations; receiving the task completion message returned by all of the at least one computing node, and issuing a write-back task to the at least one computing node, the write-back task being used by the at least one computing node to lock a primary table to be added with the index and add the index. 2.The method of claim 1, wherein the receiving the task completion message returned by all of the at least one computing node and issuing the write-back task to the at least one computing node comprises: after receiving the task completion message returned by all of the at least one computing node, changing the index state from being visible only to write operations to a data reorganization state, and after the index state is changed from being visible only to write operations to the data reorganization state, issuing the write-back task to the at least one computing node after two heartbeat periods. 3.The method of claim 1, further comprising: determining a computing node in a same machine room as a storage node as a target computing node; and wherein the issuing the write-back task to the at least one computing node comprises: issuing the write-back task to the target computing node. receiving an index adding instruction forwarded by the computing node, and after receiving the index adding instruction, setting the index state to be invisible to any operation, and changing the index state every two heartbeat periods; wherein the index adding instruction comprises an index type; and when the index type indicates that the index to be added is a global index, the method further comprises: obtaining metadata change information, and creating a global index shard according to the metadata change information, wherein the global index shard is independent of the primary table, and the global index shard is used to store global index data. 5.A method for adding an index, applied to a computing node, comprising: receiving a transaction checking task sent by a metadata node, and checking whether a target transaction is completed, the metadata node being a control node for monitoring execution of transactions on a computing node; wherein the target transaction comprises a transaction executed by the computing node on metadata and data to be added with an index and having a transaction start time before an index state update time, the index state update time being a time when an index state is changed to be visible only to write operations, the transaction checking task being sent by the metadata node in response to the index state being changed to be visible only to write operations. ​ ​ ​ ​ 4. The method of claim 2 or 3, further comprising: ​ ​ ​ ​ ​ ​ in response to the target transaction being completed, returning a task completion message to the metadata node, the task completion message being used for the metadata node to receive the task completion messages returned by all of the at least one computing node, and issuing a write-back task to the at least one computing node, the write-back task being used for the at least one computing node to lock the primary table to which the index is to be added and add the index.

6. The method of claim 5, further comprising: in response to receiving the write-back task, adding the index.

7. The method of claim 6, further comprising: receiving an index adding instruction, and forwarding the index adding instruction to the metadata node, the index adding instruction being used for the metadata node to, after receiving the index adding instruction, set the index state to be invisible to any operation, and in response to the index state being visible only to write operations, send a transaction checking task to the at least one computing node.

8. The method of claim 7, wherein, the index adding instruction comprising an index type; when the index type indicates that the index to be added is a global index, the in response to receiving the write-back task, adding the index comprising: in response to receiving the write-back task, locking the primary table and adding index data in a global index shard, the global index shard being created by the metadata node after obtaining the metadata change information according to the metadata change information, the global index shard being independent of the primary table, and the global index shard being used to store global index data; when the index type indicates that the index to be added is a local index, the in response to receiving the write-back task, adding the index comprising: in response to receiving the write-back task, locking the primary table and adding local index data in the primary table.

9. A system for indexing additions, comprising: a metadata node and a computing node, wherein the metadata node is a control node and is configured to monitor execution of transactions on the computing node; the metadata node is configured to, in response to the index state being changed to be visible only to write operations, send a transaction checking task to the at least one computing node; the at least one computing node is configured to receive the transaction checking task sent by the metadata node, check whether a target transaction is completed, and in response to the target transaction being completed, return a task completion message to the metadata node, wherein the target transaction comprises a transaction executed by the computing node on metadata and data of the index to be added and having a transaction start time before an index state update time, and the index state update time is a time when the index state is changed to be visible only to write operations; the metadata node is further configured to receive the task completion messages returned by all of the at least one computing node, and issue a write-back task to the at least one computing node, the write-back task being used for the at least one computing node to lock the primary table to which the index is to be added and add the index.

10. An apparatus for adding an index, applied to a metadata node, the metadata node being a control node and being configured to monitor execution of transactions on a computing node, the apparatus comprising: sending a transaction checking task to at least one computing node in response to the index state changing to be visible only to write operations, the transaction checking task being used for the at least one computing node to check whether a target transaction is completed and, in response to the target transaction being completed, the at least one computing node returning a task completion message to the metadata node, wherein the target transaction comprises a transaction executed by the computing node for metadata and data to be indexed and having a transaction start time before an index state update time, the index state update time being a time when the index state changes to be visible only to write operations; receiving a task completion message returned by all computing nodes in the at least one computing node; issuing a write-back task to the at least one computing node, the write-back task being used for the at least one computing node to lock a primary table to be indexed and add the index.

11. The apparatus of claim 10, wherein the receiving module is specifically configured to change the index state from being visible only to write operations to a data reorganization state after receiving the task completion message returned by all computing nodes in the at least one computing node. The issuing module is specifically configured to issue the write-back task to the at least one computing node after two heartbeat periods after the index state changes from being visible only to write operations to the data reorganization state.

12. The apparatus of claim 10, further comprising: a determining module configured to determine a computing node in a same machine room as the storage node as a target computing node; The issuing module is specifically configured to issue the write-back task to the target computing node.

13. The apparatus of claim 11 or 12, further comprising a receiving setting module configured to receive an index adding instruction forwarded by a computing node, and set the index state to be invisible to any operation after receiving the index adding instruction, and change the index state once every two heartbeat periods. The index adding instruction comprises an index type; The apparatus comprises: an obtaining module configured to obtain metadata change information when the index type indicates that the index to be added is a global index; a creating module configured to create a global index shard according to the metadata change information, wherein the global index shard is independent of the primary table, and the global index shard is used to store global index data.

14. An index adding apparatus applied to a computing node, comprising: a checking module configured to receive a transaction checking task sent by a metadata node, and check whether a target transaction is completed, the metadata node being a control node configured to monitor execution of a transaction on the computing node, wherein the target transaction comprises a transaction executed by the computing node for metadata and data to be indexed and having a transaction start time before an index state update time, the index state update time being a time when the index state changes to be visible only to write operations, and the transaction checking task being sent by the metadata node to at least one computing node in response to the index state changing to be visible only to write operations; a returning module configured to return a task completion message to the metadata node in response to the target transaction being completed, the task completion message being used for the metadata node to receive the task completion message returned by all computing nodes in the at least one computing node, and issue a write-back task to the at least one computing node, the write-back task being used for the at least one computing node to lock a primary table to be indexed and add the index. 15.The apparatus of claim 14, further comprising: an index adding module, configured to add an index in response to receiving the write-back task. 16.The apparatus of claim 15, further comprising: a receiving module, configured to receive an index adding instruction; a forwarding module, configured to forward the index adding instruction to a metadata node, and the metadata node is configured to, after receiving the index adding instruction, set an index state to be invisible to any operation, and send a transaction checking task to at least one computing node in response to the index state being visible to only write operation.

17. The apparatus of claim 16, wherein, the index adding instruction comprises an index type; the index adding module is specifically configured to, when the index type indicates that the index to be added is a global index, lock a master table in response to receiving the write-back task, and add index data in a global index shard, the global index shard being created by the metadata node according to metadata change information after the metadata node acquires the metadata change information, the global index shard being independent of the master table, and the global index shard being used to store global index data; and when the index type indicates that the index to be added is a local index, lock the master table in response to receiving the write-back task, and add local index data in the master table. 18.An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4 or 5-8.

19. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, the computer instructions are used to enable the computer to perform the method of any one of claims 1-4 or 5-8. 20.A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-4 or 5-8.

Images