A database table-based data processing method, device, equipment and medium

Abstract

The application discloses a database table-based data processing method and device, equipment and medium. The method comprises the following steps: when an index without a root page exists in a database table corresponding to an insertion operation, the index is locked; root page information of the index is initialized, and a system table update task is sent to a metadata node; after receiving response information returned by the metadata node according to the system table update task, the lock is released and the data insertion operation is performed. When the insertion operation occurs, the root page information of the index without the root page in the database table is initialized, and the data insertion operation is performed after the corresponding system table is updated, so that the waste of the data space existing when the database table is created can be reduced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of database, and particularly relate to a data processing method and device based on database table, equipment and medium. BACKGROUND

[0002] In a database system, at least one index is created for each table. B+ tree is the most commonly used index structure for storing data in a table. A node in the index B+ tree represents a data page, and the top layer of the B+ tree has only one node, which corresponds to a root page. When searching for data in the index structure, the search starts from the root page and proceeds from top to bottom. A unique index can be determined through the root page, and the root page address is stored as part of the index metadata in the system table. When an index is created or the index space is insufficient, a new space needs to be applied, and the unit of space application is a cluster. A cluster is composed of a specific number (usually 16, 32 or 64) of data pages.

[0003] At present, the method for applying space is to automatically create corresponding indexes when a table is created, and each index applies for a cluster and initializes a root page. However, when applying for table data space, no data is stored in the data table, resulting in waste of space and affecting the efficiency of creating tables and indexes. SUMMARY

[0004] The present application provides a data processing method and device based on a database table, equipment and medium, to solve the problem of data space waste and low efficiency of creating tables and indexes in the prior art.

[0005] According to an aspect of the present application, a data processing method based on a database table is provided, which is applied to a table data node, and the method comprises:

[0006] When an index corresponding to the insert operation exists in the database table without a root page, the index is locked;

[0007] The root page information of the index is initialized, and a system table update task is sent to a metadata node;

[0008] After receiving the response information returned by the metadata node according to the system table update task, the lock is released and the data insertion operation is performed.

[0009] According to another aspect of the present application, a data processing method based on a database table is provided, which is applied to a metadata node, and the method comprises:

[0010] A system table update task sent by a table data node is received;

[0011] The system table is updated according to the system table update task, and response information is returned to the table data node.

[0012] According to another aspect of the present application, there is provided a database table-based data processing apparatus applied to a table data node, comprising:

[0013] a locking module configured to lock an index without a root page when the index exists in a corresponding database table of an insertion operation;

[0014] an initialization module configured to initialize root page information of the index and send a system table update task to a metadata node;

[0015] an insertion module configured to release the lock and perform an operation of data insertion after receiving response information returned by the metadata node according to the system table update task.

[0016] According to another aspect of the present application, there is provided a database table-based data processing apparatus applied to a metadata node, comprising:

[0017] a receiving module configured to receive a system table update task sent by a table data node;

[0018] an updating module configured to update a system table according to the system table update task and return response information to the table data node.

[0019] According to another aspect of the present application, there is provided an electronic device, comprising: at least one processor; and

[0020] a memory connected with the at least one processor in communication; wherein,

[0021] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the database table-based data processing method according to any one of the embodiments of the present application.

[0022] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to execute the database table-based data processing method according to any one of the embodiments of the present application.

[0023] The embodiment of the application provides a database table-based data processing method, device, equipment and medium, the method comprises the following steps: when an index without a root page exists in a database table corresponding to an insertion operation, the index is locked; root page information of the index is initialized, a system table update task is sent to a metadata node; after receiving response information returned by the metadata node according to the system table update task, the locking is released and the data insertion operation is performed. The method can initialize root page information of the index without the root page in the database table when the insertion operation occurs, and perform the data insertion operation after the corresponding system table is updated, so that the waste of data space when the database table is created can be reduced; by using the method of delaying the application of space by initializing the root page information of the index only when the database table has data insertion, the efficiency of table creation and index creation in the database system can be improved, and the problems of the waste of data space when the database table is created and the low efficiency of table creation and index creation in the prior art are solved.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0026] Figure 1 A flowchart of a database table-based data processing method provided for the first embodiment of the application;

[0027] Figure 2 A flowchart of a database table-based data processing method provided for the second embodiment of the application;

[0028] Figure 3 A flowchart of a database table-based data processing method provided for the embodiment of the application;

[0029] Figure 4 A structural schematic diagram of a database table-based data processing device provided for the third embodiment of the application;

[0030] Figure 5 A structural schematic diagram of a database table-based data processing device provided for the fourth embodiment of the application;

[0031] Figure 6 A structural schematic diagram of an electronic device of the embodiment of the application. DETAILED DESCRIPTION

[0032] In order to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of the present application. It should be understood that the steps in the method embodiments of the present application can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present application is not limited in this respect.

[0033] The term "comprising" and variations thereof as used in the present document are open-ended, that is, "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions are given throughout the description.

[0034] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, any variation of the terms "comprising" and "having" and the like is intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] It should be noted that the modification of "one" and "multiple" mentioned in the present application is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0036] The names of the messages or information exchanged between the devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0037] Embodiment one

[0038] Figure 1A flowchart of a data processing method based on a database table is provided for the first embodiment of the present application. The method can be applied to the case where a database table is created or data space is applied for the database table when an insertion operation exists. The method can be executed by a data processing device based on a database table. The device can be realized by software and / or hardware and is generally integrated on an electronic device. In the present embodiment, the electronic device includes but is not limited to a computer and the like.

[0039] As shown in Figure 1 The data processing method based on a database table provided by the first embodiment of the present application is applied to a table data node and includes the following steps.

[0040] S110, when there is an index without a root page in the database table corresponding to the insertion operation, the index is locked.

[0041] The insertion operation can be an operation of inserting data in the database. Both insertion (INSERT) and update (UPDATE) in the database can trigger the insertion operation. For example, when data is inserted or cross-partition data update of a partition table occurs, the insertion operation is triggered. The database table can be a user table in the database. The index can be a data structure for pre-sorting values of one or more columns of the database table. The root page can be a data page corresponding to a root node in a B+ tree. One or more indexes corresponding to each database table can each correspond to a B+ tree. The root page of each B+ tree needs to be initialized and space is applied. The table data node (DN) is a node instance contained in a distributed database and can store data in a user table. The user table stores all data inserted by a user.

[0042] In the present embodiment, when the DN performs an insertion operation, all indexes of the database table corresponding to the insertion operation can be determined by searching metadata information whether the indexes have a root page. If there is an index without a root page in the database table, it indicates that no data is stored in the database table and no data space is applied. Therefore, the index without a root page needs to be locked before data is inserted. If all indexes have a root page, the insertion operation can be directly performed. The locking of the indexes can be performed simultaneously or sequentially, which is not limited in the present embodiment.

[0043] In one embodiment, after the index is locked, a request can be sent to a metadata node to determine whether the index has a root page by searching metadata information. If the index has a root page, the lock is released and the data insertion operation is performed. Otherwise, the subsequent steps are continued.

[0044] The embodiment can prevent the root page of the index from being initialized when the index is locked by judging whether the index exists the root page again after locking the index.

[0045] S120, initializing root page information of the index, and sending a system table update task to a metadata node.

[0046] The root page information can include page header information of the root page, pointers of a previous page and a next page corresponding to the root page, and the like, which are not limited in the embodiment. The metadata node (MD) is a node instance in a distributed database, and the MD can store data in a system table. The system table stores metadata information, including table IDs of all user tables in the database, index IDs, root page addresses of indexes, and the like. The system table update task can be a task for instructing to update the system table.

[0047] In the embodiment, initializing the root page information of the index can specifically mean applying data space (i.e., applying a cluster) for the index and initializing the root page information of the index, and sending the system table update task to the metadata node.

[0048] S130, after receiving response information returned by the metadata node according to the system table update task, releasing the lock and performing a data insertion operation.

[0049] The response information can be information for indicating that the system table has been updated to the table data node.

[0050] In the embodiment, when the table data node receives the response information returned by the metadata node according to the system table update task, it indicates that the initialized information of the index has been saved in the system table. At this time, the lock of the index can be released, and the insertion operation can be performed to insert data.

[0051] The data processing method based on the database table provided by the embodiment one includes: when an index without a root page exists in a database table corresponding to an insertion operation, locking the index; initializing root page information of the index, and sending a system table update task to a metadata node; and after receiving response information returned by the metadata node according to the system table update task, releasing the lock and performing a data insertion operation. The method can initialize root page information of an index without a root page in a database table when an insertion operation occurs, and perform a data insertion operation after updating a system table, which can reduce waste of data space when creating the database table. The method of delaying application of space by initializing root page information of an index when data is inserted in the database table can also improve the efficiency of creating a table and an index in a database system, and solves the problems of waste of data space when creating a database table and low efficiency of creating a table and an index in the prior art.

[0052] On the basis of the above-mentioned embodiments, variant embodiments of the above-mentioned embodiments are proposed, and it needs to be explained that, in order to make the description brief, only the differences from the above-mentioned embodiments are described in the variant embodiments.

[0053] In one embodiment, the locking the index comprises: adding a mask to an index identifier of the index, and locking the index identifier to which the mask is added by an exclusive lock.

[0054] The index identifier can be a unique identifier of the index. The exclusive lock (X lock) is also called a write lock or an exclusive lock, and is a basic lock type. The exclusive lock can ensure that multiple updates are not performed on the same resource at the same time. A transaction that obtains the exclusive lock can read data and modify data.

[0055] In the embodiment, when the index is locked, a mask can be added to the index identifier of the index first, and then the index identifier to which the mask is added is locked by an exclusive lock, so as to ensure that only one session applies a cluster for the index and initializes a root page. Since the index identifier is locked by adding a mask in the embodiment, syntax analysis of a concurrent session is not blocked. For example, if the concurrent session also performs insertion, the execution phase of the statement is blocked by the temporary lock containing the mask of the current session, until the current session is released from the lock, and then the other concurrent sessions can obtain the latest metadata information from the MD, and continue to perform insertion according to the root page address in the latest metadata information.

[0056] In the embodiment, the mask adding manner can be realized by adjusting binary data of the index identifier to obtain a new index identifier, which is not limited in the embodiment.

[0057] In one embodiment, after the root page information of the index is initialized, the method further comprises: recording a rollback log of the index in the memory; the rollback log at least includes an index identifier, a table identifier of a database table to which the index belongs, and index root page address information; and performing a disk flushing operation, the disk flushing operation being a process of writing information in the memory to a disk, the information in the memory at least including the rollback log and the initialized root page information.

[0058] The rollback log can include the index identifier of the index, the table identifier of the database table to which the index belongs, and the index root page address information. The table identifier can be a unique identifier of the database table, and the index root page address information can be a physical address of the root page corresponding to the index. The disk flushing operation can be a process of writing information in the memory to a disk.

[0059] In the embodiment, after the root page information of the index is initialized, the rollback log of the index can also be recorded in the memory, and the flushing operation is performed to write the initialized root page information and the rollback log into the disk, so as to prevent the loss of data.

[0060] In one embodiment, the index identifier, the table identifier of the database table to which the index belongs, and the index root page address information can also be added into the system table updating task, so that the metadata node can update the system table according to the index identifier, the table identifier of the database table to which the index belongs, and the index root page address information.

[0061] In one embodiment, the method further comprises: when an exception occurs before the flushing operation is performed, directly releasing the lock after the exception is recovered, and re-executing the insertion operation; and when an exception occurs after the flushing operation is performed, re-sending the system table updating task to the metadata node after the exception is recovered.

[0062] The exception can be a situation that occurs in the database system and causes the table data node and the metadata node to fail to work normally, for example, the exception can be node crash, network failure, etc., which are not limited in the embodiment.

[0063] In the embodiment, when an exception occurs before the DN performs the flushing operation, the related information of the index is not sent to the MD because the DN does not perform the flushing operation at this time. That is, the root page of the index is not included in the metadata information, and the DN does not release the lock at this time, and no row record in the corresponding database table. Then, when the database system recovers normally, the DN can directly release the lock, and the insertion operation can be re-executed as needed subsequently. When an exception occurs after the DN performs the flushing operation, the rollback operation can be performed by using the rollback log because the DN has performed the flushing operation at this time. When the DN rolls back, the application of the root page of the index is completed, and the root page information and the index information are recorded in the rollback log, and the DN can continue to perform the step of requesting the MD to update the system table.

[0064] Embodiment Two

[0065] Figure 2 A flowchart of a data processing method based on a database table is provided in the embodiment two of the application. The method can be applied to the case of applying data space for a database table when the database table is created or an insertion operation exists, and can be executed by a data processing device based on a database table. The device can be realized by software and / or hardware, and is generally integrated on an electronic device, which includes but is not limited to a computer and the like. The details of the embodiment are not described herein and can be referred to the embodiment one.

[0066] As Figure 2As shown, the second embodiment of the present application provides a database table-based data processing method, applied to a metadata node, including the following steps:

[0067] S210, receiving a system table update task sent by a table data node.

[0068] The system table update task can be a task for instructing to update the system table.

[0069] In this embodiment, the metadata node can receive the system table update task sent by the table data node.

[0070] S220, updating the system table according to the system table update task, and returning response information to the table data node.

[0071] The response information can be information for instructing the table data node that the system table has been updated.

[0072] In this embodiment, after the metadata node receives the system table update task, the metadata node can update the system table according to the system table update task, and after updating the system table, the metadata node can return the response information to the table data node, the response information indicating that the system table has been updated.

[0073] The second embodiment of the present application provides a database table-based data processing method, including: receiving a system table update task sent by a table data node; updating the system table according to the system table update task, and returning response information to the table data node. The method updates the system table according to the information sent by the table data node, thereby completing the application of the data space, and can reduce the waste of the data space when creating the database table, and solve the problems of the waste of the data space when creating the database table and the low efficiency of creating the table and the index in the prior art.

[0074] In one embodiment, the system table update task includes an index identifier, a table identifier of an index belonging to a database table, and index root page address information; correspondingly, the updating the system table according to the system table update task, and returning the response information to the table data node includes: adding the system table update task to a task list, and performing a disk flushing operation; after writing the index root page address information into a corresponding position in the system table according to the index identifier and the table identifier, deleting the corresponding system table update task in the task list; performing the disk flushing operation, and returning the response information to the table data node.

[0075] The index identifier can be a unique identifier of the index, the table identifier can be a unique identifier of the database table, and the index root page address information can be a physical address of a root page corresponding to the index. The task list can include tasks to be executed by the metadata node. The disk flushing operation can be a process of writing information in the memory into a disk.

[0076] In the embodiment, the system table update task can include an index identifier, a table identifier of a database table to which the index belongs, and index root page address information. After receiving the system table update task, the metadata node can first add the system table update task to a task list and perform a disk flushing operation, thereby preventing data loss. Then, the corresponding data can be found from the system table according to the index identifier and the table identifier, and the index root page address information can be written in the data to update the system table. Then, the system table update tasks that have been executed in the task list can be deleted, and the disk flushing operation is performed again to prevent abnormality after system crash and return response information to the table data node.

[0077] In one embodiment, the method further includes: when abnormality occurs after performing the disk flushing operation, reading the task list in the disk after the abnormality is recovered; and receiving and processing the task request sent by the table data node after executing the system table update task in the read task list.

[0078] The task request can include a system table update task, a task for querying metadata information, and the like.

[0079] In the embodiment, if the metadata node is abnormal after performing the disk flushing operation, the task list saved in the disk can be read after the abnormality is recovered, the system table update task in the task list saved in the disk is executed first, and then the task request sent by the table data node is received and processed, thereby ensuring that the metadata information provided by the MD when providing services again is consistent with the metadata information provided by the MD at the moment before the MD fails, and other abnormal conditions are avoided.

[0080] For example, if there is always network failure between the DN and the MD, the DN actively exits the communication. When the MD is recovered, the DN is restarted. After the MD is recovered, the task list in the disk is read first, the content in the task list is checked and processed in sequence, and the MD provides services again after the content in the task list is completely processed.

[0081] The embodiments of the present application provide several specific implementation manners on the basis of the technical solutions of the above embodiments.

[0082] As a specific implementation manner of the embodiment, Figure 3 A flowchart of a data processing method based on a database table provided by the embodiment of the present application is shown in FIG. 1. Figure 3 As shown in FIG. 1, the operation flow when data is inserted in the table of the distributed database can be as follows:

[0083] 1. When the DN inserts data, it is checked whether all indexes of the table already exist in the root page. If all indexes already exist in the root page, no additional processing is needed, and the data insertion is directly performed. Otherwise, the following steps are executed.

[0084] 2. The DN temporarily X-locks the index ID with a special mask to ensure that only one session applies for a cluster and initializes the root page for the index.

[0085] 3. After the DN succeeds in the lock, it checks the latest metadata information again to determine whether the index has a root page. If the index has a root page, the DN releases the lock and then performs data insertion. Otherwise, the DN performs the following step.

[0086] 4. The DN applies for a cluster and initializes the root page information.

[0087] 5. The DN records a rollback log, which contains index information, including but not limited to the index ID, the table ID to which the index belongs, and the root page address information of the index.

[0088] 6. The DN flushes the disk to write the information in the memory to the disk, including the root page initialization information and the rollback log.

[0089] 7. The DN sends a message to the MD to request the MD to update the system table, and the message contains the index information described above.

[0090] 8. After the MD receives the message, it adds the message to a task list and performs a disk flushing operation.

[0091] 9. The MD records the root page address information of the index in the system table, clears the task list (removes the current message from the task list), and then performs a disk flushing operation again. Finally, the MD returns a response message to the DN.

[0092] 10. The DN receives the response message, releases the lock, and then performs data insertion.

[0093] Since the operation of initializing the root page involves network communication between the table data node and the metadata node, the existing technology is inefficient in creating a table. Therefore, the embodiment delays the allocation of data space to the database table and only applies for space and initializes the root page when the first row of records is inserted into the table, thereby improving the efficiency of creating a table.

[0094] In the embodiment, for a query statement, the locating data is retrieved from the index starting from the index root page, i.e. the address information of the index root page is first obtained from the metadata information. If there is a concurrent data insertion or update statement, the metadata information obtained by the query statement can be outdated, which is embodied in that the data insertion statement applies a cluster for the index and initializes the root page, while the metadata information obtained by the query statement does not contain the root page address information of the index. If any index of the table has not applied a root page, it means that there is no record in the table. Therefore, if the metadata information obtained by the query statement does not contain the root page address information of the index, the latest metadata information can be obtained again before the data retrieval is performed, and if the latest metadata information still does not contain the root page, it can be considered that there is no record in the table.

[0095] Embodiment Three

[0096] Figure 4 A structure diagram of a data processing device based on a database table is provided for the third embodiment of the present application. The device can be applied to the case where a database table is created or data space is applied for the database table when an insertion operation exists, and the device can be realized by software and / or hardware and is generally integrated on an electronic device.

[0097] As shown in Figure 4 , the device is applied to a table data node and includes:

[0098] The locking module 310 is configured to lock the index when the index corresponding to the insertion operation does not have a root page in the database table.

[0099] The initialization module 320 is configured to initialize the root page information of the index and send a system table update task to a metadata node.

[0100] The insertion module 330 is configured to release the lock and perform the data insertion operation after receiving the response information returned by the metadata node according to the system table update task.

[0101] The embodiment provides a database table-based data processing device, which comprises: a locking module, used for locking an index without a root page in a corresponding database table when an insertion operation is performed on the index; an initialization module, used for initializing root page information of the index, and sending a system table update task to a metadata node; and an insertion module, used for releasing the lock and performing an operation of data insertion after receiving response information returned by the metadata node according to the system table update task. By initializing the root page information of the index without the root page in the database table when the insertion operation is performed, and performing the operation of data insertion after the corresponding system table is updated, the waste of data space when the database table is created can be reduced; by using the method of delaying the application of space by initializing the root page information of the index only when data is inserted in the database table, the efficiency of creating the table and creating the index in the database system can be improved, and the problems of the waste of data space when the database table is created and the low efficiency of creating the table and creating the index in the prior art are solved.

[0102] Further, the locking module 310 is specifically used for:

[0103] adding a mask to the index identifier of the index, and locking the index identifier added with the mask by using an exclusive lock.

[0104] Further, the initialization module 320 is also used for:

[0105] recording a rollback log of the index in the memory; the rollback log at least comprises an index identifier, a table identifier of a database table to which the index belongs, and root page address information of the index;

[0106] performing a disk flushing operation, wherein the disk flushing operation is used for writing information in the memory to a disk, and the information in the memory at least comprises the rollback log and the initialized root page information.

[0107] Further, the device is also used for:

[0108] when an exception occurs before the disk flushing operation is performed, directly releasing the lock and re-executing the insertion operation after the exception is recovered;

[0109] when an exception occurs after the disk flushing operation is performed, re-sending the system table update task to the metadata node after the exception is recovered.

[0110] The database table-based data processing device can execute the database table-based data processing method provided in any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0111] Embodiment four

[0112] Figure 5A structural schematic diagram of a data processing device based on a database table is provided for Embodiment Four of the present application. The device can be applied to the case where a database table is created or data space is applied for a database table when an insertion operation exists in a distributed database. The device can be realized by software and / or hardware and is generally integrated on an electronic device.

[0113] As shown in Figure 5 the device is applied to a metadata node and includes:

[0114] The receiving module 410 is configured to receive a system table update task sent by a table data node.

[0115] The updating module 420 is configured to update a system table according to the system table update task and return response information to the table data node.

[0116] The present embodiment provides a data processing device based on a database table, which includes a receiving module configured to receive a system table update task sent by a table data node and an updating module configured to update a system table according to the system table update task and return response information to the table data node. The data space application is completed by updating the system table according to the information sent by the table data node, thereby reducing the waste of data space when a database table is created and solving the problems of waste of data space when a database table is created and low efficiency of table and index creation in the prior art.

[0117] Further, the system table update task includes an index identifier, a table identifier of an index belonging to a database table and index root page address information. Correspondingly, the updating module 420 is specifically configured to:

[0118] add the system table update task to a task list and perform a disk flushing operation;

[0119] write the index root page address information into a corresponding position in the system table according to the index identifier and the table identifier and then delete the corresponding system table update task in the task list;

[0120] perform the disk flushing operation and return response information to the table data node.

[0121] Further, the device is further configured to:

[0122] read the task list in the disk after an exception is recovered when the exception occurs after the disk flushing operation is performed;

[0123] receive and process a task request sent by the table data node after the system table update task in the read task list is executed.

[0124] The database table-based data processing apparatus can execute the database table-based data processing method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0125] Embodiment Five

[0126] Figure 6 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0127] As shown in Figure 6 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, where the memory stores computer programs that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0129] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the database table-based data processing method.

[0130] In some embodiments, the database table-based data processing method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as storage unit 18. In some embodiments, portions or all of the computer program can be loaded and / or installed onto electronic device 10 via, for example, ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more of the steps of the above-described database table-based data processing method can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the database table-based data processing method by other means, such as by way of firmware.

[0131] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0132] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, can implement the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0133] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0135] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0136] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0137] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and this is not limited herein.

[0138] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method based on database tables, characterized in that, Applied to table data nodes, the method includes: If an index without a root page exists in the database table corresponding to an insert operation, the index is locked. Initialize the root page information of the index and send a system table update task to the metadata node; After receiving the response information returned by the metadata node based on the system table update task, the block is released and the data insertion operation is performed; Wherein, locking the index includes: A mask is added to the index identifier of the index, and an exclusive lock is used to block the index identifier after the mask is added.

2. The method according to claim 1, characterized in that, After initializing the root page information of the index, the method further includes: The rollback log of the index is recorded in memory; the rollback log includes at least the index identifier, the table identifier of the database table to which the index belongs, and the root page address information of the index. Perform a disk flushing operation, which involves writing information from memory to disk. The information in memory includes at least the rollback log and the initialized root page information.

3. The method according to claim 1, characterized in that, The method further includes: If an exception occurs before the disk flushing operation is performed, the block is released immediately after the exception is resolved, and the insertion operation is re-executed. If an exception occurs after performing a disk flushing operation, the system table update task will be resent to the metadata node after the exception is resolved.

4. A data processing method based on database tables, characterized in that, Applied to metadata nodes, the method includes: The system table update task is sent by the table data node. The system table update task is sent by the table data node after adding a mask to the index identifier of the index and locking the masked index identifier with an exclusive lock when there is an index without a root page in the database table corresponding to the insertion operation, and after initializing the root page information of the index. The system table update task updates the system table according to the system table and returns response information to the table data node.

5. The method according to claim 4, characterized in that, The system table update task includes an index identifier, the table identifier of the database table to which the index belongs, and the index root page address information; correspondingly, the step of updating the system table according to the system table update task and returning response information to the table data node includes: Add the system table update task to the task list and perform a disk flush operation; After writing the index root page address information into the corresponding position in the system table according to the index identifier and the table identifier, delete the corresponding system table update task in the task list; Perform a disk flush operation and return response information to the table data node.

6. The method according to claim 4, characterized in that, The method further includes: If an exception occurs after performing a disk flush operation, the task list in the disk is read after the exception is recovered. After executing the system table update tasks from the read task list, the system receives and processes task requests sent by the table data nodes.

7. A data processing device based on database tables, characterized in that, The device, applied to table data nodes, includes: The blocking module is used to block the index when there is an index without a root page in the database table corresponding to the insert operation; The initialization module is used to initialize the root page information of the index and send system table update tasks to the metadata node; The insertion module is used to release the lock and perform data insertion operation after receiving the response information returned by the metadata node according to the system table update task; Specifically, the blocking module is used for: A mask is added to the index identifier of the index, and an exclusive lock is used to block the index identifier after the mask is added.

8. An electronic device, characterized in that, The device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data processing method based on a database table as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data processing method based on a database table as described in any one of claims 1-6.

Images