Distributed database shard key update method, device, equipment and medium

Abstract

The present invention provides a distributed database shard key update method, device, equipment and medium, which can be applied to the field of distributed databases. The distributed database shard key update method includes: in response to receiving a shard key update instruction for a target data table, obtaining target data in the target data table and a first shard key value of the target data, and determining a second shard key value from the shard key update instruction; generating operation indication data for the target data according to an original storage node and a target storage node, wherein the operation indication data includes an operation to be performed for the target data, and the operation to be performed includes an update operation, a delete operation or an insert operation; generating a structured query statement according to the operation indication data; and sending the structured query statement to a storage node to be operated that matches the operation indication data, wherein the storage node to be operated includes an original storage node or a target storage node.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed databases, and more specifically, to a distributed database shard key update method, device, equipment and medium. Background Art

[0002] With the continuous development of big data technology, the amount of data has shown explosive growth, and more and more scenarios are using distributed databases, especially shared-nothing distributed databases, to store data. When storing data, distributed databases store data to designated storage nodes based on sharding key values ​​and sharding rules, thereby distributing the data on various data nodes.

[0003] When the shard key is updated, the storage node corresponding to the shard key will change. Therefore, if the shard key value of the data is updated, the shard key value will not match the storage node. Summary of the invention

[0004] In view of this, the present invention provides a distributed database shard key update method, apparatus, device, storage medium and program product.

[0005] One aspect of the present invention provides a distributed database shard key update method, comprising: in response to receiving a shard key update instruction for a target data table, obtaining the target data in the target data table and a first shard key value of the target data, and determining a second shard key value from the shard key update instruction, wherein the first shard key value is related to the original storage node of the target data before the shard key is updated, and the second shard key value is related to the target storage node storing the target data after the shard key value is updated; generating operation indication data for the target data according to the original storage node and the target storage node, wherein the operation indication data includes an operation to be performed on the target data, and the operation to be performed includes an update operation, a delete operation or an insert operation; generating a structured query statement matching the operation to be performed according to the operation indication data; and sending the structured query statement to the storage node to be operated that matches the operation indication data, so that the storage node to be operated executes the structured query statement to update, delete or insert the target data, wherein the storage node to be operated includes the original storage node or the target storage node.

[0006] According to an embodiment of the present invention, the above method also includes: obtaining the storage location of the above target data in the above original storage node; and generating target scanning data based on the above first shard key value, the above second shard key value, the above target data, the above storage location and the above original storage node.

[0007] According to an embodiment of the present invention, the above-mentioned operation indication data for the above-mentioned target data is generated based on the above-mentioned original storage node and the above-mentioned target storage node, including: in response to determining that the above-mentioned target storage node is the same as the above-mentioned original storage node, determining that the operation to be performed on the above-mentioned target data is an update operation, wherein the above-mentioned update operation is used to update the shard key value of the above-mentioned target data; in response to determining that the above-mentioned target storage node is different from the above-mentioned original storage node, determining that the operation to be performed on the above-mentioned target data includes a delete operation and an insert operation, wherein the above-mentioned delete operation is used to delete the target data before the shard key value is updated, and the above-mentioned insert operation is used to insert the target data after the shard key value is updated; and generating the above-mentioned operation indication data based on the above-mentioned operation to be performed and the above-mentioned target scan data.

[0008] According to an embodiment of the present invention, the above-mentioned operation indication data includes first operation indication data related to the above-mentioned update operation, and the above-mentioned operation indication data is generated based on the above-mentioned operation to be executed and the above-mentioned target scan data, including: in response to determining that the operation to be executed is an update operation, based on the above-mentioned update operation, and the above-mentioned second shard key value in the above-mentioned target scan data, the above-mentioned target data, the above-mentioned storage location and the above-mentioned original storage node, the first operation indication data is generated.

[0009] According to an embodiment of the present invention, the above-mentioned operation indication data includes second operation indication data related to the above-mentioned deletion operation and third operation indication data related to the above-mentioned insertion operation. The above-mentioned operation indication data is generated based on the above-mentioned operation to be executed and the above-mentioned target scan data, including: in response to determining that the operation to be executed includes the above-mentioned deletion operation, based on the above-mentioned deletion operation, and the above-mentioned first shard key value in the above-mentioned target scan data, the above-mentioned target data, the above-mentioned storage location and the above-mentioned original storage node, the above-mentioned second operation indication data is generated; in response to determining that the operation to be executed includes the above-mentioned insertion operation, based on the above-mentioned insertion operation, the above-mentioned second shard key value in the above-mentioned target scan data, the above-mentioned target data, the above-mentioned storage location and the above-mentioned original storage node, the above-mentioned third operation indication data is generated.

[0010] According to an embodiment of the present invention, the above-mentioned structured query statement matching the above-mentioned operation to be executed is generated according to the above-mentioned operation indication data, including: in response to determining that the above-mentioned operation indication data is the first operation indication data, based on the above-mentioned update operation, the above-mentioned second sharding key value, the above-mentioned target data, the above-mentioned storage location and the above-mentioned original storage node in the above-mentioned first operation indication data, an update statement for executing the above-mentioned update operation is generated; in response to determining that the above-mentioned operation indication data is the second operation indication data, based on the above-mentioned deletion operation, the above-mentioned storage location and the above-mentioned original storage node in the above-mentioned second operation indication data, a delete statement for executing the above-mentioned deletion operation is generated; in response to determining that the above-mentioned operation indication data is the third operation indication data, based on the above-mentioned insertion operation, the above-mentioned second sharding key value and the above-mentioned target data in the above-mentioned third operation indication data, an insert statement for executing the above-mentioned insertion operation is generated.

[0011] According to an embodiment of the present invention, the sending of the structured query statement to the storage node to be operated that matches the operation indication data includes: sending the update statement or the delete statement to the original storage node; and sending the insert statement to the target storage node.

[0012] Another aspect of the present invention provides a shard key update device, including: a data acquisition module, which is used to respond to receiving a shard key update instruction for a target data table, acquire the target data in the target data table, and determine the second shard key value from the shard key update instruction, wherein the first shard key value is related to the original storage node of the target data before the shard key is updated, and the second shard key value is related to the target storage node storing the target data after the shard key value is updated; an operation generation module, which is used to generate operation indication data for the target data according to the original storage node and the target storage node, wherein the operation indication data includes the operation to be performed for the target data, and the operation to be performed includes an update operation, a delete operation or an insert operation; a statement generation module, which is used to generate a structured query statement matching the operation to be performed according to the operation indication data; and a statement sending module, which is used to send the structured query statement to the storage node to be operated that matches the operation indication data, so that the storage node to be operated executes the structured query statement to update, delete or insert the target data, wherein the storage node to be operated includes the original storage node or the target storage node.

[0013] Another aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, characterized in that when the one or more programs are executed by the one or more processors, the one or more processors implement a method for updating a distributed database shard key.

[0014] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, characterized in that the above instructions are used to implement a method for updating a distributed database shard key when executed.

[0015] Another aspect of the present invention provides a computer program product, which includes computer executable instructions, and is characterized in that the instructions are used to implement a method for updating a distributed database shard key when executed.

[0016] According to an embodiment of the present invention, the original storage node and the target storage node of the target data are respectively determined based on the first shard key value and the second shard key value, and operations are performed on the corresponding storage nodes to be operated, thereby solving the problem of mismatch between the shard key value of the target data and the storage node after the shard key is updated, meeting the user's demand for updating the shard key, and saving the computing resources of the storage node by executing the shard key update method in the coordination node. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0018] Figure 1 An application scenario diagram of a distributed database shard key update method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0019] Figure 2 A flow chart of a distributed database shard key update method according to an embodiment of the present invention is shown.

[0020] Figure 3 A data flow diagram for generating a structured query statement according to a specific embodiment of the present invention is shown.

[0021] Figure 4 A schematic diagram of a shard key update method according to a specific embodiment of the present invention is shown.

[0022] Figure 5 A structural block diagram of a distributed database shard key updating device according to an embodiment of the present invention is shown.

[0023] Figure 6 A block diagram of an electronic device suitable for implementing a distributed database shard key update method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] When the shard key is updated, the storage node corresponding to the shard key may change. Therefore, if only the shard key value of the data is updated, the shard key value may not match the storage node. The embodiment of the present invention determines the storage node after the shard key is updated and before the shard key is updated, and updates the corresponding storage node according to the determination result, so that the updated shard key value matches the storage node.

[0029] An embodiment of the present invention provides a distributed database shard key update method, comprising: in response to receiving a shard key update instruction for a target data table, obtaining target data in the target data table and a first shard key value of the target data, and determining a second shard key value from the shard key update instruction, wherein the first shard key value is related to an original storage node of the target data before the shard key is updated, and the second shard key value is related to a target storage node storing the target data after the shard key value is updated; generating operation indication data for the target data based on the original storage node and the target storage node, wherein the operation indication data includes operations to be performed on the target data, and the operations to be performed include update operations, delete operations, or insert operations; generating a structured query statement matching the operations to be performed based on the operation indication data; and sending the structured query statement to the storage node to be operated that matches the operations to be performed, so that the storage node to be operated executes the structured query statement to update, delete, or insert the target data, wherein the storage node to be operated includes the original storage node or the target storage node.

[0030] Figure 1 An application scenario diagram of a distributed database shard key update method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0031] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 111, a second terminal device 112, a third terminal device 113, a network device 120, a first server 131, a second server 132, and a third server 133. The network 120 is used to provide a medium for communication links between the first terminal device 111, the second terminal device 112, the third terminal device 113 and the first server 131, the second server 132, and the third server 133. The network 120 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0032] The user may use the first terminal device 111, the second terminal device 112, and the third terminal device 113 to interact with the first server 131, the second server 132, and the third server 133 through the network 120 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 111, the second terminal device 112, and the third terminal device 113, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0033] The first terminal device 111 , the second terminal device 112 , and the third terminal device 113 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The first server 131, the second server 132, and the third server 133 may be servers that provide various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 111, the second terminal device 112, and the third terminal device 113. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0035] According to an embodiment of the present invention, the first server 131, the second server 132, and the third server 133 can be used as storage nodes of a distributed database for storing data. A user can use the first terminal device 111, the second terminal device 112, and the third terminal device 113 to send a shard key update instruction to the first server 131, the second server 132, and the third server 133 through the network 120, and the first server 131, the second server 132, and the third server 133 operate according to the shard key update instruction and return the shard key update result to the first terminal device 111, the second terminal device 112, and the third terminal device 113.

[0036] It should be noted that the distributed database shard key update method provided in the embodiment of the present invention can generally be executed by the first server 131, the second server 132 and the third server 133. Correspondingly, the distributed database shard key update device provided in the embodiment of the present invention can generally be set in the first server 131, the second server 132 and the third server 133. The distributed database shard key update method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the first server 131, the second server 132 and the third server 133 and can communicate with the first terminal device 111, the second terminal device 112, the third terminal device 113 and / or the first server 131, the second server 132 and the third server 133. Correspondingly, the distributed database shard key update device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the first server 131, the second server 132 and the third server 133 and can communicate with the first terminal device 111, the second terminal device 112, the third terminal device 113 and / or the first server 131, the second server 132 and the third server 133.

[0037] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] Figure 2 A flow chart of a distributed database shard key update method according to an embodiment of the present invention is shown.

[0039] like Figure 2 As shown, the distributed database shard key update method includes operations S210~S240.

[0040] In operation S210, in response to receiving a shard key update instruction for a target data table, target data in the target data table and a first shard key value of the target data are acquired, and a second shard key value is determined from the shard key update instruction.

[0041] According to an embodiment of the present invention, a distributed database includes a coordination node and a storage node, wherein the coordination node is used to parse the data processing instructions sent by the client, determine the storage node corresponding to the data processing instructions, and control the storage node to perform data processing, and the storage node is used to store and process data.

[0042] According to an embodiment of the present invention, after receiving a shard key update instruction for a target data table, the coordinating node processes the shard key update instruction using the distributed database shard key update method of the present invention.

[0043] According to an embodiment of the present invention, the target data may be data of a shard key to be updated in a target data table. The first shard key value may be a shard key value before the update, stored together with the target data in the target data table, and the first shard key value is related to an original storage node of the target data before the shard key is updated.

[0044] According to an embodiment of the present invention, the original storage node may be a storage node of the target data before the shard key is updated, and is related to the first shard key value, so the original storage node of the target data may be determined by the first shard key value. Specifically, the original storage node of the target data may be determined by using the sharding rule and the first shard key value.

[0045] According to an embodiment of the present invention, the second shard key value may be an updated shard key value, and the second shard key value may be related to the target storage node storing the target data after the shard key value is updated and may be obtained from the shard key update instruction. For example, the shard key update instruction may be UPDATE table SET c1=c2*2, where table is the table name of the target data table, c1 is the first shard key value, c2 is the target data, and c2*2 is the second shard key value.

[0046] According to an embodiment of the present invention, the target storage node may be a storage node that matches the second shard key value after the shard key is updated. Specifically, the target storage node may be determined according to the sharding rule and the second shard key value.

[0047] In operation S220, operation instruction data for target data is generated according to the original storage node and the target storage node.

[0048] According to an embodiment of the present invention, the operation indication data may be used to indicate the operations to be performed on the target data and the shard key, and the operations may include update operations, insert operations, and delete operations, etc.

[0049] According to an embodiment of the present invention, since changes in shard key values ​​may result in changes in storage nodes, it is necessary to determine whether the storage node of the target data changes after the shard key is updated based on the original storage node and the target storage node, and then determine the operations for the target data and the shard key.

[0050] According to an embodiment of the present invention, if the original storage node is the same as the target storage node, it is represented that only the shard key value needs to be updated. If the original storage node is different from the target storage node, it is represented that the storage node of the target data needs to be changed.

[0051] In operation S230, a structured query statement matching the operation to be performed is generated according to the operation indication data.

[0052] According to an embodiment of the present invention, a structured query language (SQL) may be used to instruct a storage node to perform operations such as inserting, deleting, updating, and querying data.

[0053] According to an embodiment of the present invention, since the operation indication data includes operation objects such as target data and sharding keys, and operations such as update, insert and delete, a structured query statement can be generated according to the operation indication data.

[0054] In operation S240, the structured query statement is sent to a storage node to be operated that matches the operation to be performed, so that the storage node to be operated executes the structured query statement and operates on the target data.

[0055] According to an embodiment of the present invention, the storage node to be operated includes an original storage node or a target storage node.

[0056] According to an embodiment of the present invention, a structured query statement is sent to the storage node to be operated, and the storage node to be operated executes the structured query statement to perform operations such as updating, inserting and deleting on the target data and sharding key, so that after the sharding key is updated, the sharding key value of the target data matches the storage node.

[0057] According to an embodiment of the present invention, the original storage node and the target storage node of the target data are respectively determined based on the first shard key value and the second shard key value, and operations are performed on the corresponding storage nodes to be operated, thereby solving the problem of mismatch between the shard key value of the target data and the storage node after the shard key is updated, meeting the user's demand for updating the shard key, and saving the computing resources of the storage node by executing the shard key update method in the coordination node.

[0058] According to an embodiment of the present invention, after obtaining the first shard key value and target data from the target data table and determining the second shard key value from the shard key update instruction, it also includes: obtaining the storage location of the target data in the original storage node; and generating target scan data based on the first shard key value, the second shard key value, the target data, the storage location and the original storage node.

[0059] According to an embodiment of the present invention, the storage location includes path information capable of acquiring target data. Specifically, the storage location can be acquired from a system table.

[0060] According to an embodiment of the present invention, the target scan data includes a first shard key value, a second shard key value, target data, a storage location, and an original storage node. Specifically, the first shard key value, the second shard key value, the target data, and the original storage node can be stored in a memory, and after the storage location is determined, the storage location is written into the memory to obtain the target scan data.

[0061] According to the embodiment of the present invention, by generating target scan data, it is convenient to implement shard key update according to the target scan data, thereby improving the efficiency of shard key update.

[0062] According to an embodiment of the present invention, operation indication data for target data is generated based on the original storage node and the target storage node, including: in response to determining that the target storage node is the same as the original storage node, determining that the operation to be performed on the target data is an update operation; in response to determining that the target storage node is different from the original storage node, determining that the operation to be performed on the target data includes a deletion operation and an insertion operation; and generating operation indication data based on the operation to be performed and the target scan data.

[0063] According to an embodiment of the present invention, since the update of the shard key may cause changes in the storage node corresponding to the shard key, it is necessary to judge the original storage node corresponding to the first shard key value and the target storage node corresponding to the second shard key value to determine whether the update of the shard key causes changes in the storage node.

[0064] According to an embodiment of the present invention, when it is determined that the target storage node is the same as the original storage node, it is characterized that the storage node corresponding to the first shard key value before the shard key is updated and the storage node corresponding to the second shard key value after the shard key is updated have not changed, and the operation to be executed on the target data is determined to be an update operation.

[0065] According to an embodiment of the present invention, when it is determined that the target storage node is different from the original storage node, it is characterized that the storage node corresponding to the first shard key value before the shard key is updated and the storage node corresponding to the second shard key value after the shard key is updated have changed, and it is determined that the operations to be performed on the target data include a deletion operation and an insertion operation.

[0066] According to an embodiment of the present invention, the update operation is used to update the shard key value of the target data, the delete operation is used to delete the target data before the shard key value is updated, and the insert operation is used to insert the target data after the shard key value is updated.

[0067] According to an embodiment of the present invention, the operation indication data includes the operation to be performed and the target scan data. Specifically, an operation identifier can be assigned to the operation to be performed, and the update operation, the delete operation and the insert operation each correspond to an operation identifier value. After determining the operation identifier value, the operation identifier value is written into the memory storing the target scan data to obtain the operation indication data.

[0068] According to an embodiment of the present invention, by determining the operations to be performed on the target data based on the original storage node and the target storage node, the efficiency of updating the shard key is improved.

[0069] According to an embodiment of the present invention, operation indication data is generated based on the operation to be executed and the target scan data, including: in response to determining that the operation to be executed is an update operation, based on the update operation, and the second shard key value, target data, storage location and original storage node in the target scan data, generating first operation indication data.

[0070] According to an embodiment of the present invention, the operation indication data includes first operation indication data related to the update operation, and the first operation indication data may include data required for performing the update operation on the target data. Specifically, the first operation indication data includes the update operation, the second shard key value, the target data, the storage location, and the original storage node.

[0071] According to an embodiment of the present invention, for an update operation, the second shard key value, target data, storage location and original storage node can be obtained from the memory to generate first operation indication data.

[0072] According to the embodiment of the present invention, targeted first operation indication data is generated for the update operation, which saves data transmission resources and facilitates further generation of a structured query statement matching the update operation.

[0073] According to an embodiment of the present invention, operation indication data is generated based on the operation to be executed and the target scan data, including: in response to the operation to be executed including a delete operation, second operation indication data is generated based on the delete operation, the first shard key value, the target data, the storage location and the original storage node in the target scan data; in response to the operation to be executed including an insert operation, third operation indication data is generated based on the insert operation, the second shard key value, the target data, the storage location and the original storage node in the target scan data.

[0074] According to an embodiment of the present invention, the operation indication data further includes second operation indication data and third operation indication data. When the target storage node is different from the original storage node, the operation to be performed includes a deletion operation and an insertion operation, so the second operation indication data and the third operation indication data need to be generated for the deletion operation and the insertion operation respectively.

[0075] According to an embodiment of the present invention, the second operation indication data may include data required for performing a delete operation on the target data, and the third operation indication data may include data required for performing an insert operation on the target data.

[0076] According to an embodiment of the present invention, for a deletion operation, a second shard key value, target data, a storage location, and an original storage node may be obtained from the memory to generate second operation indication data.

[0077] According to an embodiment of the present invention, for an insert operation, the second shard key value, target data, storage location, and original storage node may be obtained from the memory to generate third operation indication data.

[0078] According to an embodiment of the present invention, targeted second operation indication data and third operation indication data are generated for deletion operations and insertion operations respectively, saving data transmission resources and facilitating further generation of structured query statements matching the deletion operations and insertion operations.

[0079] According to an embodiment of the present invention, a structured query statement matching the operation to be executed is generated based on the operation indication data, including: in response to determining that the operation indication data is the first operation indication data, based on the update operation, the second shard key value, the target data, the storage location and the original storage node in the first operation indication data, an update statement for executing the update operation is generated; in response to determining that the operation indication data is the second operation indication data, based on the delete operation, the storage location and the original storage node in the second operation indication data, a delete statement for executing the delete operation is generated; in response to determining that the operation indication data is the third operation indication data, based on the insert operation, the second shard key value and the target data in the third operation indication data, an insert statement for executing the insert operation is generated.

[0080] According to an embodiment of the present invention, the operation identification value in the operation indication data may be used to determine whether the operation indication data is the first operation indication data, the second operation indication data, and the third operation indication data.

[0081] According to an embodiment of the present invention, the first operation indication data is operation indication data corresponding to the update operation, and therefore an update statement is generated based on the first operation indication data, and the update statement is used to instruct the original storage node to update the shard key value of the target data from the first shard key value to the second shard key value.

[0082] Specifically, the update statement may be UPDATE table SET C1=c2*2,C2=c2 WHERE t.ctid=ctidand t.xc_node_id=node_id, where C1 may be a shard key identifier, C2 may be a target data identifier, t.ctid may be a storage location identifier, ctid may be a storage location, t.xc_node_id may be a storage node identifier, and node_id may be an original storage node.

[0083] According to an embodiment of the present invention, the second operation indication data is operation indication data corresponding to a deletion operation, and thus a deletion statement is generated according to the second operation data, and the deletion statement is used to instruct the original storage node to delete the target data. Specifically, the deletion statement may be DELETE FROM table where t.ctid=ctid and t.xc_node_id=node_id.

[0084] According to an embodiment of the present invention, the third operation indication data is operation indication data corresponding to an insert operation, and therefore an insert statement is generated according to the third operation indication data, and the insert statement is used to instruct the target storage node to store target data whose shard key value is the second shard key value. Specifically, the insert statement may be INSERT INTO table(C1,C2) VALUES(c2*2,c2).

[0085] According to an embodiment of the present invention, when the original storage node is the same as the target storage node, the shard key value of the target data in the original storage node is updated through an update operation to achieve the update of the shard key. When the original storage node is different from the target storage node, the target data before the shard key update is deleted in the original storage node through a delete operation, and the target data after the shard key update is inserted into the target storage node through an insert operation, so that the shard key update is achieved while matching the updated shard key value with the storage node.

[0086] According to an embodiment of the present invention, sending a structured query statement to a storage node to be operated that matches the operation indication data includes: sending an update statement or a delete statement to an original storage node; and sending an insert statement to a target storage node.

[0087] According to an embodiment of the present invention, since update statements and delete statements are both executed in the original storage node, they need to be sent to the original storage node, and insert statements are executed in the target storage node, so insert statements need to be sent to the target storage node.

[0088] According to an embodiment of the present invention, after the structured query statement is sent to the storage node to be operated, the storage node to be operated implements a shard key update operation for the target data by executing the structured query statement.

[0089] According to an embodiment of the present invention, the coordination node parses the shard key update instruction into a structured query statement, and sends the structured query statement to the corresponding storage node to be operated, thereby implementing the shard key update operation for the target data in the storage node to be operated, thereby improving the processing efficiency of the shard key update.

[0090] Figure 3 A data flow diagram for generating a structured query statement according to a specific embodiment of the present invention is shown.

[0091] like Figure 3 As shown, generating a structured query statement includes operations S310 to S360.

[0092] In operation S310, target scan data is generated.

[0093] In operation S320, it is determined whether the original storage node and the target storage node are the same. If they are the same, operation S330 is performed, otherwise operation S350 is performed.

[0094] In operation S330, first operation data is generated.

[0095] In operation S340, an update statement is generated.

[0096] In operation S350, second operation data and third operation data are generated.

[0097] In operation S360, a delete statement and an insert statement are generated.

[0098] According to an embodiment of the present invention, the coordination node can generate an execution plan for executing the shard key update instruction according to the shard key update instruction, wherein the execution plan is used to split the shard key update task into multiple subtasks that can be executed by operators. For the shard key update task, the execution plan may include a scan operator, an operation determination operator, and a statement determination operator.

[0099] Figure 4 A schematic diagram of a shard key update method according to a specific embodiment of the present invention is shown.

[0100] As shown in FIG4 , the coordination node 410 includes a scanning operator 401, an operation determination operator 402, and a statement determination operator 403. The scanning operator 401 is a lower layer operator of the operation determination operator 402, and the operation determination operator 402 is a lower layer operator of the statement determination operator 403.

[0101] According to an embodiment of the present invention, the scanning operator 401 can be used to scan the target data table, obtain the first shard key value and the target data from the target data table, and store them in the memory. After determining the second shard key value, the original storage node, and the storage location of the target data, the second shard key value, the original storage node, and the storage location are written into the memory to obtain the target scan data, and the target scan data is returned to the operation determination operator 402. When it is determined that there is no data of the shard key to be updated in the target data table, the shard key update task is terminated.

[0102] According to an embodiment of the present invention, the operation determination operator 402 determines the operation instruction data for the target data according to the target scan data returned by the scan operator 401. When the original storage node and the target storage node are the same, the first operation instruction data is returned to the statement determination operator 403. When the original storage node and the target storage node are different, the second operation instruction statement and the third operation instruction statement are returned to the statement determination operator 403 respectively, and when it is determined that there is no operation instruction statement to be returned to the operation determination operator 402, the scan operator 401 is called to perform the scan operation.

[0103] According to an embodiment of the present invention, the statement determination operator 403 is used to generate a structured query statement corresponding to the operation indication data according to the operation indication data returned by the operation determination operator 402, and send the structured query statement to a remote storage node. Specifically, the update statement or the delete statement is sent to the original storage node 420, and the insert statement is sent to the target storage node 430.

[0104] Based on the above distributed database shard key update method, the present invention also provides a distributed database shard key update device. Figure 5 The device is described in detail.

[0105] Figure 5 A structural block diagram of a distributed database shard key updating device according to an embodiment of the present invention is shown.

[0106] like Figure 5As shown, the distributed database shard key updating device 500 of this embodiment includes a data acquisition module 510, an operation generation module 520, a statement generation module 530 and a statement sending module 540.

[0107] The data acquisition module 510 is used to, in response to receiving a shard key update instruction for a target data table, acquire the target data in the target data table and the first shard key value of the target data, and acquire the first shard key value and the target data from the target data table, and determine the second shard key value from the shard key update instruction, wherein the first shard key value is related to the original storage node of the target data before the shard key update, and the second shard key value is related to the target storage node storing the target data after the shard key value is updated. In one embodiment, the data acquisition module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0108] The operation generation module 520 is used to generate operation indication data for the target data according to the original storage node and the target storage node, wherein the operation indication data includes the operation to be performed for the target data, and the operation to be performed includes an update operation, a delete operation or an insert operation. In one embodiment, the operation generation module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0109] The statement generation module 530 is used to generate a structured query statement matching the operation to be performed according to the operation indication data. In one embodiment, the statement generation module 530 can be used to perform the operation S230 described above, which will not be described in detail here.

[0110] The statement sending module 540 is used to send the structured query statement to the storage node to be operated that matches the operation to be performed, so that the storage node to be operated executes the structured query statement and performs an update, delete or insert operation on the target data, wherein the storage node to be operated includes the original storage node or the target storage node. In one embodiment, the statement sending module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0111] According to an embodiment of the present invention, the distributed database shard key updating device 500 further includes a location acquisition module and a data generation module.

[0112] The location acquisition module is used to obtain the storage location of the target data in the original storage node.

[0113] The data generation module is used to generate target scan data based on the first shard key value, the second shard key value, the target data, the storage location and the original storage node.

[0114] According to an embodiment of the present invention, the operation generation module 530 includes a first determination submodule, a second determination submodule and an indication generation submodule.

[0115] The first determination submodule is used to determine, in response to determining that the target storage node is the same as the original storage node, that the operation to be performed on the target data is an update operation, wherein the update operation is used to update the shard key value of the target data.

[0116] The second determination submodule is used to determine, in response to determining that the target storage node is different from the original storage node, that the operations to be performed on the target data include a delete operation and an insert operation, wherein the delete operation is used to delete the target data before the shard key value is updated, and the insert operation is used to insert the target data after the shard key value is updated.

[0117] The instruction generation submodule is used to generate operation instruction data based on the operation to be performed and the target scanning data.

[0118] According to an embodiment of the present invention, the operation indication data includes first operation indication data related to the update operation, and the first determination submodule includes a first generation unit.

[0119] The first generating unit is used to generate first operation indication data in response to determining that the operation to be performed is an update operation, based on the update operation, and the second shard key value, target data, storage location and original storage node in the target scan data.

[0120] According to an embodiment of the present invention, the operation indication data includes second operation indication data related to a delete operation and third operation indication data related to an insert operation. The second determining submodule includes a second generating unit and a third generating unit.

[0121] The second generating unit is used to generate second operation indication data in response to determining that the operation to be performed includes a deletion operation, based on the deletion operation, and the first shard key value, target data, storage location and original storage node in the target scan data.

[0122] The third generating unit is used to generate third operation indication data in response to determining that the operation to be performed includes an insert operation, based on the insert operation, and the second shard key value, target data, storage location and original storage node in the target scan data.

[0123] According to an embodiment of the present invention, the statement generation module 530 includes an update generation submodule, a delete generation submodule and an insert generation submodule.

[0124] The update generation submodule is used to generate an update statement for performing the update operation in response to determining that the operation indication data is the first operation indication data, based on the update operation in the first operation indication data, the second shard key value, the target data, the storage location and the original storage node.

[0125] The deletion generation submodule is used to generate a deletion statement for executing the deletion operation based on the deletion operation, storage location and original storage node in the second operation indication data in response to determining that the operation indication data is the second operation indication data.

[0126] The insert generation submodule is used to generate an insert statement for executing the insert operation based on the insert operation, the second shard key value and the target data in the third operation indication data in response to determining that the operation indication data is the third operation indication data.

[0127] According to an embodiment of the present invention, the statement sending module 540 includes a first sending submodule and a second sending submodule.

[0128] The first sending submodule is used to send the update statement or the delete statement to the original storage node.

[0129] The second sending submodule is used to send the insert statement to the target storage node.

[0130] According to an embodiment of the present invention, any multiple modules among the data acquisition module 510, the operation generation module 520, the statement generation module 530 and the statement sending module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the data acquisition module 510, the operation generation module 520, the statement generation module 530 and the statement sending module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the data acquisition module 510, the operation generation module 520, the statement generation module 530 and the statement sending module 540 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.

[0131] Figure 6A block diagram of an electronic device suitable for implementing a distributed database shard key update method according to an embodiment of the present invention is shown.

[0132] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0133] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0134] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0135] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0136] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0137] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the distributed database shard key update method provided by the embodiment of the present invention.

[0138] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0139] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0140] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0141] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0142] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0144] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A distributed database shard key update method, characterized in that: The method comprises: In response to receiving a shard key update instruction for a target data table, obtaining target data in the target data table and a first shard key value of the target data, and determining a second shard key value from the shard key update instruction, wherein the first shard key value is associated with an original storage node storing the target data before the shard key value is updated, and the second shard key value is associated with a target storage node storing the target data after the shard key value is updated; Generate operation indication data for the target data according to the original storage node and the target storage node, wherein the operation indication data includes an operation to be performed for the target data, and the operation to be performed includes an update operation, a delete operation or an insert operation; generating, according to the operation indication data, a structured query statement matching the operation to be performed; and Sending the structured query statement to a storage node to be operated that matches the operation to be performed, so that the storage node to be operated executes the structured query statement and performs an update, delete or insert operation on the target data, wherein the storage node to be operated includes the original storage node or the target storage node; Also includes: Obtaining the storage location of the target data in the original storage node; and Generate target scan data based on the first shard key value, the second shard key value, the target data, the storage location, and the original storage node; The step of generating operation instruction data for the target data according to the original storage node and the target storage node includes: In response to determining that the target storage node is the same as the original storage node, determining that the operation to be performed on the target data is an update operation, wherein the update operation is used to update a shard key value of the target data; In response to determining that the target storage node is different from the original storage node, determining that the operation to be performed on the target data includes a delete operation and an insert operation, wherein the delete operation is used to delete the target data before the shard key value is updated, and the insert operation is used to insert the target data after the shard key value is updated; and The operation indication data is generated based on the operation to be performed and the target scan data.

2. The method according to claim 1, characterized in that The operation instruction data includes first operation instruction data related to the update operation, and the generating the operation instruction data based on the to-be-performed operation and the target scan data includes: In response to determining that the operation to be performed is an update operation, first operation indication data is generated based on the update operation, and the second shard key value in the target scan data, the target data, the storage location, and the original storage node.

3. The method according to claim 1, characterized in that The operation indication data includes second operation indication data related to the deletion operation and third operation indication data related to the insertion operation, and the generating of the operation indication data based on the operation to be performed and the target scan data includes: In response to determining that the operation to be performed includes the deletion operation, generating the second operation indication data based on the deletion operation, and the first shard key value in the target scan data, the target data, the storage location, and the original storage node; In response to determining that the operation to be performed includes the insert operation, the third operation indication data is generated based on the insert operation, and the second shard key value in the target scan data, the target data, the storage location, and the original storage node.

4. The method according to claim 1, characterized in that: The step of generating a structured query statement matching the operation to be performed according to the operation indication data includes: In response to determining that the operation indication data is the first operation indication data, based on the update operation in the first operation indication data, the second shard key value, the target data, the storage location, and the original storage node, an update statement for executing the update operation is generated; In response to determining that the operation indication data is second operation indication data, generating a deletion statement for executing the deletion operation based on the deletion operation, the storage location and the original storage node in the second operation indication data; In response to determining that the operation indication data is third operation indication data, an insert statement for executing the insert operation is generated based on the insert operation, the second shard key value, and the target data in the third operation indication data.

5. The method according to claim 4, characterized in that The step of sending the structured query statement to a storage node to be operated that matches the operation indication data includes: Sending the update statement or the delete statement to the original storage node; and The insert statement is sent to the target storage node.

6. A distributed database shard key update device, characterized in that: The device comprises: a data acquisition module, configured to, in response to receiving a shard key update instruction for a target data table, acquire target data in the target data table and a first shard key value of the target data, and determine a second shard key value from the shard key update instruction, wherein the first shard key value is associated with an original storage node of the target data before the shard key is updated, and the second shard key value is associated with a target storage node storing the target data after the shard key value is updated; An operation generating module, configured to generate operation indication data for the target data according to the original storage node and the target storage node, wherein the operation indication data includes an operation to be performed for the target data, and the operation to be performed includes an update operation, a delete operation or an insert operation; A statement generation module, used to generate a structured query statement matching the operation to be performed according to the operation indication data; and A statement sending module, used for sending the structured query statement to a storage node to be operated that matches the operation to be performed, so that the storage node to be operated executes the structured query statement and performs an update, delete or insert operation on the target data, wherein the storage node to be operated includes the original storage node or the target storage node; Also includes: A location acquisition module, used to acquire the storage location of the target data in the original storage node; and A data generation module, configured to generate target scan data based on the first shard key value, the second shard key value, the target data, the storage location, and the original storage node; The operation generation module includes: A first determination submodule is configured to, in response to determining that the target storage node is the same as the original storage node, determine that the operation to be performed on the target data is an update operation, wherein the update operation is used to update a shard key value of the target data; A second determination submodule is configured to, in response to determining that the target storage node is different from the original storage node, determine that the operation to be performed on the target data includes a delete operation and an insert operation, wherein the delete operation is used to delete the target data before the shard key value is updated, and the insert operation is used to insert the target data after the shard key value is updated; and The indication generating submodule is used to generate the operation indication data based on the operation to be performed and the target scanning data.

7. An electronic device comprising: one or more processors; a memory for storing one or more programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor implements the method according to any one of claims 1 to 5.

Images