Data processing method and device based on database table, equipment and medium

Abstract

The invention discloses a data processing method and device based on a database table, equipment and a medium. The method comprises the steps that when an index without a root page exists in a database table corresponding to insertion operation, the index is blocked; initializing root page information of the index, and sending a system table updating task to a metadata node; after response information returned by the metadata node according to the system table updating task is received, the blocking is released, and the data insertion operation is executed. According to the method, when the insertion operation occurs, the root page information is initialized for the index without the root page in the database table, and the data insertion operation is executed after the system table is correspondingly updated, so that the waste of data space when the database table is created can be reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of database technology, and in particular to a data processing method, device, equipment and medium based on a database table. Background Art

[0002] In a database system, at least one index is created for each table. The B+ tree is the most commonly used index structure for storing data in a table. In the index B+ tree, a node represents a data page. There is only one node at the top level of the B+ tree, and the corresponding data page is called the root page. When retrieving and locating data in this index structure, the search starts from the root page and goes down from top to bottom. The root page can be used to determine a unique index, and the root page address is stored in the system table as part of the index metadata. When creating an index or when the index space is insufficient, new space needs to be applied for. The unit of space application is a cluster. A specific number (usually 16, 32, or 64) of data pages constitute a cluster.

[0003] The current method of applying for space is to automatically create the corresponding index when creating a table, apply for a cluster for each index, and initialize the root page. However, when applying for table data space, there is no data stored in the data table, which leads to a waste of space and also affects the efficiency of creating tables and indexes. Summary of the invention

[0004] The present invention provides a data processing method, device, equipment and medium based on a database table, so as to solve the problems of waste of data space when creating a database table and low efficiency of creating a table and an index in the prior art.

[0005] According to one aspect of the present invention, a data processing method based on a database table is provided, which is applied to a table data node, and the method comprises:

[0006] When there is an index without a root page in the database table corresponding to the insert operation, the index is blocked;

[0007] Initialize the root page information of the index and send a system table update task to the metadata node;

[0008] After receiving the response information returned by the metadata node according to the system table update task, the blockade is released and the data insertion operation is performed.

[0009] According to another aspect of the present invention, a data processing method based on a database table is provided, which is applied to a metadata node, and the method comprises:

[0010] Receive system table update tasks sent by table data nodes;

[0011] The system table is updated according to the system table update task, and response information is returned to the table data node.

[0012] According to another aspect of the present invention, a data processing device based on a database table is provided, which is applied to a table data node, and the device comprises:

[0013] A blocking module, used for blocking the index when there is an index without a root page in the database table corresponding to the insert operation;

[0014] An initialization module, used to initialize the root page information of the index and send a system table update task to the metadata node;

[0015] The insertion module is used to release the blockade and execute the data insertion operation after receiving the response information returned by the metadata node according to the system table update task.

[0016] According to another aspect of the present invention, a data processing device based on a database table is provided, which is applied to a metadata node, and the device comprises:

[0017] A receiving module, used for receiving the system table update task sent by the table data node;

[0018] An update module is used to update the system table according to the system table update task and return response information to the table data node.

[0019] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising: at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data processing method based on the database table described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method based on a database table described in any embodiment of the present invention when executed.

[0023] The embodiment of the present invention provides a data processing method, device, equipment and medium based on a database table, the method comprising: when there is an index without a root page in the database table corresponding to the insert operation, the index is blocked; the root page information of the index is initialized, and a system table update task is sent to the metadata node; after receiving the response information returned by the metadata node according to the system table update task, the blockade is released and the data insertion operation is performed. When the insert operation occurs, the method initializes the root page information for the index without a root page in the database table, and performs the data insertion operation after the corresponding system table is updated, which can reduce the waste of data space when creating the database table; by delaying the application of space by initializing the root page information of the index when data is inserted into the database table, the efficiency of creating tables and indexes in the database system can also be improved, solving the problem of waste of data space when creating database tables and low efficiency of creating tables and indexes in the prior art.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of a flow chart of a data processing method based on a database table provided in Embodiment 1 of the present invention;

[0027] Figure 2 A schematic diagram of a data processing method based on a database table provided in Embodiment 2 of the present invention;

[0028] Figure 3 A schematic diagram of a flow chart of a data processing method based on a database table provided in an embodiment of the present invention;

[0029] Figure 4 A schematic diagram of the structure of a data processing device based on a database table provided in Embodiment 3 of the present invention;

[0030] Figure 5 A schematic diagram of the structure of a data processing device based on a database table provided in Embodiment 4 of the present invention;

[0031] Figure 6 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of the present invention. It should be understood that the various steps recorded in the method implementation of the present invention can be performed in different orders and / or in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0033] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variations of the terms "including" and "having" are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0036] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0037] Embodiment 1

[0038] Figure 1A flowchart of a data processing method based on a database table provided in Embodiment 1 of the present invention is applicable to situations in which a database table is created in a distributed database or data space is requested for a database table when an insert operation occurs. The method can be executed by a data processing device based on a database table, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes but is not limited to: computers and other devices.

[0039] like Figure 1 As shown, a data processing method based on a database table provided in Embodiment 1 of the present invention is applied to a table data node, and includes the following steps:

[0040] S110: When there is an index without a root page in the database table corresponding to the insert operation, the index is blocked.

[0041] Among them, the insert operation can be an operation of inserting data inside the database. Inside the database, both insert (INSERT) and update (UPDATE) will trigger the insert operation. For example, when data is inserted, or when cross-partition data of a partitioned table is updated, an insert operation will be triggered. The database table can be a user table in the database. The index can be a data structure that pre-sorts the values ​​of one or more columns of the database table. The root page can be the data page corresponding to the root node in the B+ tree. One or more indexes corresponding to each database table can each correspond to a B+ tree, and the root page of each B+ tree needs to be initialized and space applied. The table data node (Data Node, DN) is a node instance contained in a distributed database, which can store data in the user table. The user table stores all data inserted by the user.

[0042] In this embodiment, when DN performs an insert operation, for the database table corresponding to the insert operation, it can determine whether all indexes of the database table already have root pages by searching metadata information. If there are indexes without root pages in the database table, it means that the database table does not store data and does not apply for data space. Before inserting data, it is necessary to apply for data space first, so the indexes without root pages can be blocked. If all indexes have root pages, the insert operation can be performed directly. The blocking of indexes can be performed simultaneously or sequentially, and this embodiment does not limit this.

[0043] In one embodiment, after the index is blocked, a request may be sent to the metadata node to determine again whether there is a root page for the index by searching the metadata information. If so, the blockade is released and the data insertion operation is performed, otherwise, the subsequent steps are continued.

[0044] This embodiment can prevent the root page of the index from being initialized when the index is blocked by determining again whether the index has a root page after the index is blocked.

[0045] S120, initializing the root page information of the index, and sending a system table update task to the metadata node.

[0046] The root page information may include the page header information of the root page, the pointers of the previous page and the next page corresponding to the root page, and other information, which is not limited in this embodiment. A metadata node (MD) is a node instance in a distributed database. The MD can store data in a system table. The system table stores metadata information, including table IDs, index IDs, and root page addresses of all user tables in the database. A system table update task may be a task for instructing to update a system table.

[0047] In this embodiment, initializing the root page information of the index may specifically refer to applying for data space for the index (ie, applying for a cluster) and initializing the root page information of the index, and sending a system table update task to the metadata node.

[0048] S130. After receiving the response information returned by the metadata node according to the system table update task, release the blockade and execute the data insertion operation.

[0049] The response information may be information used to indicate to the table data node that the system table has been updated.

[0050] In this embodiment, when the table data node receives the response information returned by the metadata node according to the system table update task, it means that the initialized information of the index has been saved in the system table. At this time, the index lock can be released and the insert operation can be performed to insert the data.

[0051] A data processing method based on a database table provided in a first embodiment of the present invention includes: when an index without a root page exists in the database table corresponding to an insert operation, the index is blocked; the root page information of the index is initialized, and a system table update task is sent to a metadata node; after receiving the response information returned by the metadata node according to the system table update task, the blockade is released and the data insertion operation is performed. When an insert operation occurs, the method initializes the root page information for the index without a root page in the database table, and performs the data insertion operation after the corresponding system table is updated, which can reduce the waste of data space when creating a database table; by delaying the application of space by initializing the root page information of the index when data is inserted into the database table, the efficiency of creating tables and indexes in the database system can also be improved, solving the problems of waste of data space when creating database tables and low efficiency of creating tables and indexes in the prior art.

[0052] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0053] In one embodiment, blocking the index includes: adding a mask to an index identifier of the index, and blocking the masked index identifier by an exclusive lock.

[0054] The index identifier may be a unique identifier of the index. An exclusive lock (X lock), also known as a write lock or an exclusive lock, is a basic lock type. An exclusive lock can ensure that multiple updates to the same resource are not performed simultaneously. A transaction that obtains an exclusive lock can both read and modify data.

[0055] In this embodiment, when blocking an index, a mask may be added to the index identifier of the index first, and then the index identifier may be blocked by an exclusive lock after the mask is added, to ensure that only one session applies for a cluster for the index and initializes the root page. Since this embodiment blocks the index identifier by adding a mask, the syntax analysis of concurrent sessions will not be blocked. For example, if a concurrent session also executes an insert, it will be blocked by the temporary block containing the mask of the current session during the statement execution phase, until the current session releases the block, and then other concurrent sessions can obtain the latest metadata information from the MD and continue to execute the insert according to the root page address in the latest metadata information.

[0056] In this embodiment, the method of adding the mask can be implemented by adjusting the binary data of the index identifier to obtain a new index identifier, which is not limited in this embodiment.

[0057] In one embodiment, after initializing the root page information of the index, the method further includes: recording a rollback log of the index in a memory; the rollback log includes at least an index identifier, a table identifier of a database table to which the index belongs, and index root page address information; and performing a disk flushing operation, wherein the disk flushing operation is to write the information in the memory to the disk, and the information in the memory includes at least the rollback log and the initialized root page information.

[0058] The rollback log may include the index identifier of the index, the table identifier of the database table to which the index belongs, and the index root page address information. The table identifier may be a unique identifier of the database table, and the index root page address information may be a physical address of the root page corresponding to the index. The disk flushing operation may be a process of writing information in the memory to the disk.

[0059] In this embodiment, after the root page information of the index is initialized, the rollback log of the index can also be recorded in the memory, and a disk flushing operation can be performed to write the initialized root page information and the rollback log to the disk to prevent data loss.

[0060] In one embodiment, the index identifier, the table identifier of the database table to which the index belongs, and the index root page address information can also be added to the system table update task, so that the metadata node can update the system table based on the index identifier, the table identifier of the database table to which the index belongs, and the index root page address information.

[0061] In one embodiment, the method further includes: when an exception occurs before executing the disk flushing operation, directly releasing the blockade after the exception is recovered and re-executing the insert operation; when an exception occurs after executing the disk flushing operation, re-sending the system table update task to the metadata node after the exception is recovered.

[0062] Among them, the exception may be a situation that occurs in the database system and causes the table data node and the metadata node to fail to work normally. For example, the exception may be a node crash, a network failure, etc., which is not limited in this embodiment.

[0063] In this embodiment, when an exception occurs before DN performs a disk flushing operation, since DN has not performed a disk flushing operation at this time, the relevant information of the index is not sent to MD. That is, the metadata information does not contain the root page of the index, and DN has not released the blockade at this time, and there is no row of records in the corresponding database table. Then when the database system returns to normal, DN can directly release the blockade, and can subsequently re-execute the insert operation as needed. When an exception occurs after DN performs a disk flushing operation, since DN has already performed a disk flushing operation, a rollback operation can be performed using the rollback log. When DN rolls back, the application for the index root page has been completed, and the root page information and index information are recorded in the rollback log. DN can continue to execute the step of requesting MD to update the system table.

[0064] Embodiment 2

[0065] Figure 2 A flowchart of a data processing method based on a database table provided in Embodiment 2 of the present invention is provided. The method can be applied to the case where a database table is created in a distributed database or data space is requested for a database table when an insert operation is performed. The method can be performed by a data processing device based on a database table, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes but is not limited to: a computer and other devices. For the contents not yet fully described in this embodiment, please refer to Embodiment 1.

[0066] like Figure 2As shown, a data processing method based on a database table provided in the second embodiment of the present invention is applied to a metadata node, and includes the following steps:

[0067] S210: Receive a system table update task sent by a table data node.

[0068] The system table update task may be a task for instructing to update the system table.

[0069] In this embodiment, the metadata node may receive the system table update task sent by the table data node.

[0070] S220: Update the system table according to the system table update task, and return response information to the table data node.

[0071] The response information may be information used to indicate to the table data node that the system table has been updated.

[0072] In this embodiment, after receiving the system table update task, the metadata node may update the system table according to the system table update task, and after updating the system table, may return response information indicating that the system table has been updated to the table data node.

[0073] A second embodiment of the present invention provides a data processing method based on a database table, comprising: receiving a system table update task sent by a table data node; updating the system table according to the system table update task, and returning response information to the table data node. The method completes the application of data space by updating the system table according to the information sent by the table data node, which can reduce the waste of data space when creating a database table, and solves the problem of waste of data space when creating a database table and low efficiency of creating tables and indexes in the prior art.

[0074] In one embodiment, the system table update task includes an index identifier, a table identifier of a database table to which the index belongs, and index root page address information; accordingly, updating the system table according to the system table update task and returning response information to the table data node include: adding the system table update task to a task list and performing a disk flushing operation; after writing the index root page address information to a corresponding position in the system table according to the index identifier and the table identifier, deleting the corresponding system table update task in the task list; performing a disk flushing operation and returning response information to the table data node.

[0075] The index identifier may be a unique identifier of the index, the table identifier may be a unique identifier of the database table, and the index root page address information may be a physical address of the root page corresponding to the index. The task list may include tasks that the metadata node needs to perform. The disk flushing operation may be a process of writing information in the memory to the disk.

[0076] In this embodiment, the system table update task may include an index identifier, a table identifier of the database table to which the index belongs, and the index root page address information. After receiving the system table update task, the metadata node may first add the system table update task to the task list and perform a disk flushing operation, thereby preventing data loss. Then, the corresponding data may be found from the system table according to the index identifier and the table identifier, and the index root page address information may be written into the data to update the system table; then, the system table update task that has been executed in the task list may be deleted, and the disk flushing operation may be performed again to prevent abnormalities after the system crashes, and the response information may be returned to the table data node.

[0077] In one embodiment, the method further includes: when an exception occurs after performing a disk flushing operation, reading a task list in the disk after the exception is recovered; after executing the system table update task in the read task list, receiving and processing a task request sent by a table data node.

[0078] The task request may include a system table update task, a task requesting to query metadata information, and the like.

[0079] In this embodiment, if an exception occurs in the metadata node after executing the disk flushing operation, the task list saved in the disk can be read after the exception is recovered, and the system table update task in the task list saved in the disk can be executed first, and then the task request sent by the table data node can be received and processed. In this way, the metadata information provided to the outside when the MD provides services again can be guaranteed to be consistent with the metadata information provided to the outside just before the MD failure, thereby avoiding other abnormal situations.

[0080] For example, if there is a network failure between DN and MD, DN will automatically quit the communication. When MD returns to normal, DN will be restarted. After MD returns to normal, it will first read the task list in the disk, check the contents in the task list and process them in sequence, and provide services to the outside world after all the contents in the task list have been processed.

[0081] The embodiments of the present invention provide several specific implementation methods based on the technical solutions of the above embodiments.

[0082] As a specific implementation method of this embodiment, Figure 3 A flowchart of a data processing method based on a database table provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the operation flow when inserting data into a table in a distributed database can be as follows:

[0083] 1. When inserting data into DN, check whether all indexes of the table already have root pages. If all indexes already have root pages, no additional processing is required and data can be directly inserted. Otherwise, proceed to the subsequent steps.

[0084] 2. DN adds a special mask to the index ID to perform a temporary X-block, ensuring that only one session is applying for a cluster for the index and initializing the root page.

[0085] 3. After the DN is blocked successfully, check the latest metadata information again to determine whether the index has a root page. If so, release the blockade and insert data. Otherwise, execute the subsequent steps.

[0086] 4. DN applies for a cluster and initializes the root page information.

[0087] 5. DN records the rollback log, which contains index information, including but not limited to the index ID, the table ID to which the index belongs, the index root page address information, etc.

[0088] 6. DN flushes the disk and writes the information in the memory to the disk. The information in the memory includes the root page initialization information and rollback log, etc.

[0089] 7. DN sends a message to MD, requesting MD to update the system table. The message contains the above index information.

[0090] 8. After receiving the message, MD adds the message to the task list and performs the disk flushing operation.

[0091] 9. MD records the index root page address information into the system table, cleans up the task list (removes the current message in the task list), then performs the disk flushing operation again, and finally returns a response message to DN.

[0092] 10. DN receives the response message, releases the lock, and then executes data insertion.

[0093] Since the operation of initializing the root page also involves network communication between the table data node and the metadata node, the prior art is very inefficient in creating a table. Therefore, this embodiment delays the allocation of data space for the database table for the distributed database, and only applies for space and initializes the root page when the first row of records is inserted into the table, thereby improving the efficiency of creating a table.

[0094] In this embodiment, for a query statement, when retrieving and locating data in an index, it is necessary to start processing from the index root page, that is, it is necessary to first obtain the index root page address information from the metadata information. If there are concurrent data insertion or update statements, the metadata information obtained by the query statement may be outdated. Specifically, the data insertion statement applies for a cluster and initializes the root page for the index, while the metadata information obtained by the query statement does not yet contain the root page address information of the index. If any index of the table has not yet applied for a root page, it means that no row of records in the table exists. Therefore, if the metadata information obtained by the query statement does not contain the root page address information of the index, the latest metadata information can be obtained again before performing data retrieval. If the latest metadata information still does not contain the root page, it can be considered that there are no records in the table.

[0095] Embodiment 3

[0096] Figure 4 A schematic diagram of the structure of a data processing device based on a database table provided in Embodiment 3 of the present invention, wherein the device can be used in a distributed database to create a database table or to apply for data space for a database table when an insert operation occurs, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device.

[0097] like Figure 4 As shown, the device is applied to a table data node and includes:

[0098] The blocking module 310 is used to block the index when there is an index without a root page in the database table corresponding to the insert operation;

[0099] Initialization module 320, used to initialize the root page information of the index and send a system table update task to the metadata node;

[0100] The insertion module 330 is used to release the blockade and execute the data insertion operation after receiving the response information returned by the metadata node according to the system table update task.

[0101] This embodiment provides a data processing device based on a database table, including: a blocking module, used to block the index when there is an index without a root page in the database table corresponding to the insert operation; an initialization module, used to initialize the root page information of the index, and send a system table update task to the metadata node; an insertion module, used to release the blockade and perform the data insertion operation after receiving the response information returned by the metadata node according to the system table update task. By initializing the root page information for the index without a root page in the database table when the insert operation occurs, and performing the data insertion operation after the corresponding system table is updated, the waste of data space when creating the database table can be reduced; by the method of delaying the application of space by initializing the root page information of the index when data is inserted into the database table, the efficiency of creating tables and indexes in the database system can also be improved, solving the problem of waste of data space when creating database tables and low efficiency of creating tables and indexes in the prior art.

[0102] Furthermore, the blocking module 310 is specifically used for:

[0103] A mask is added to the index identifier of the index, and the masked index identifier is locked by an exclusive lock.

[0104] Furthermore, the initialization module 320 is further used to:

[0105] Recording a rollback log of the index in a memory; the rollback log at least includes an index identifier, a table identifier of a database table to which the index belongs, and index root page address information;

[0106] A disk flushing operation is performed, wherein the disk flushing operation is to write the information in the memory to the disk, wherein the information in the memory at least includes the rollback log and the initialized root page information.

[0107] Furthermore, the device is also used for:

[0108] When an exception occurs before the disk flushing operation is performed, the blockade is released directly after the exception is recovered and the insertion operation is performed again;

[0109] When an exception occurs after performing a disk flushing operation, the system table update task is resent to the metadata node after the exception is recovered.

[0110] The above-mentioned data processing device based on a database table can execute the data processing method based on a database table provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0111] Embodiment 4

[0112] Figure 5A schematic diagram of the structure of a data processing device based on a database table provided in Embodiment 4 of the present invention, wherein the device can be used in a distributed database to create a database table or to apply for data space for a database table when an insert operation occurs, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device.

[0113] like Figure 5 As shown, the device is applied to a metadata node, including:

[0114] Receiving module 410, used for receiving the system table update task sent by the table data node;

[0115] The updating module 420 is used to update the system table according to the system table updating task and return response information to the table data node.

[0116] This embodiment provides a data processing device based on a database table, including: a receiving module, used to receive a system table update task sent by a table data node; an updating module, used to update the system table according to the system table update task, and return response information to the table data node. By updating the system table according to the information sent by the table data node, the application of data space is completed, which can reduce the waste of data space when creating a database table, and solve the problem of waste of data space when creating a database table in the prior art, and low efficiency of creating tables and indexes.

[0117] Furthermore, the system table update task includes an index identifier, a table identifier of a database table to which the index belongs, and index root page address information; accordingly, the update module 420 is specifically used to:

[0118] Add the system table update task to the task list and perform the disk flushing operation;

[0119] After writing the index root page address information into a corresponding position in the system table according to the index identifier and the table identifier, deleting the corresponding system table update task in the task list;

[0120] Execute the disk flushing operation and return response information to the table data node.

[0121] Furthermore, the device is also used for:

[0122] When an exception occurs after performing a disk flushing operation, the task list in the disk is read after the exception is recovered;

[0123] After executing the system table update tasks in the read task list, it receives and processes the task requests sent by the table data nodes.

[0124] The above-mentioned data processing device based on a database table can execute the data processing method based on a database table provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0125] Embodiment 5

[0126] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0127] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0129] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a data processing method based on a database table.

[0130] In some embodiments, the data processing method based on the database table can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data processing method based on the database table described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the data processing method based on the database table by any other appropriate means (for example, by means of firmware).

[0131] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0133] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0136] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0137] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0138] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method based on a database table, characterized in that: Applied to a table data node, the method comprises: When there is an index without a root page in the database table corresponding to the insert operation, the index is blocked; Initialize the root page information of the index and send a system table update task to the metadata node; After receiving the response information returned by the metadata node according to the system table update task, the blockade is released and the data insertion operation is performed.

2. The method according to claim 1, characterized in that The blocking of the index comprises: A mask is added to the index identifier of the index, and the masked index identifier is locked by an exclusive lock.

3. The method according to claim 1, characterized in that After initializing the root page information of the index, the method further includes: Recording a rollback log of the index in a memory; the rollback log at least includes an index identifier, a table identifier of a database table to which the index belongs, and index root page address information; A disk flushing operation is performed, wherein the disk flushing operation is to write the information in the memory to the disk, wherein the information in the memory at least includes the rollback log and the initialized root page information.

4. The method according to claim 1, characterized in that: The method further comprises: When an exception occurs before the disk flushing operation is performed, the blockade is released directly after the exception is recovered and the insertion operation is performed again; When an exception occurs after performing a disk flushing operation, the system table update task is resent to the metadata node after the exception is recovered.

5. A data processing method based on a database table, characterized in that: Applied to a metadata node, the method comprises: Receive system table update tasks sent by table data nodes; The system table is updated according to the system table update task, and response information is returned to the table data node.

6. The method according to claim 5, characterized in that The system table update task includes an index identifier, a table identifier of a database table to which the index belongs, and index root page address information; accordingly, updating the system table according to the system table update task and returning response information to the table data node include: Add the system table update task to the task list and perform the disk flushing operation; After writing the index root page address information into a corresponding position in the system table according to the index identifier and the table identifier, deleting the corresponding system table update task in the task list; Execute the disk flushing operation and return response information to the table data node.

7. The method according to claim 5, characterized in that The method further comprises: When an exception occurs after performing a disk flushing operation, the task list in the disk is read after the exception is recovered; After executing the system table update tasks in the read task list, it receives and processes the task requests sent by the table data nodes.

8. A data processing device based on a database table, characterized in that: Applied to a table data node, the device comprises: A blocking module, used for blocking the index when there is an index without a root page in the database table corresponding to the insert operation; An initialization module, used to initialize the root page information of the index and send a system table update task to the metadata node; The insertion module is used to release the blockade and execute the data insertion operation after receiving the response information returned by the metadata node according to the system table update task.

9. An electronic device, characterized in that: The device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the database table-based data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method based on a database table according to any one of claims 1 to 7 when executed.

Images