Method, device and electronic device for checking table data balance of distributed database

Abstract

The present invention provides a method, device and electronic device for checking table data balance of a distributed database, which can be applied to the field of distributed database technology. The method includes: obtaining a user management table and a system table corresponding to each of a plurality of database nodes; based on the management table version identifier of the i-th business table in the user management table and the sub-table version identifier of the i-th business table sub-table of each of the plurality of system tables, updating and identifying the i-th business table sub-table, and determining the table update identification result; in the case where it is determined that the table update identification result indicates that the i-th business table sub-table in at least one database node has been updated, determining the storage space of the i-th business table sub-table of each of the plurality of database nodes; in the case where it is determined that the storage spaces of the plurality of i-th business table sub-tables are unbalanced with each other, balancing the plurality of i-th business table sub-tables, and storing the balanced plurality of i-th business table sub-tables in the plurality of database nodes.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed databases, and in particular to a method, device and electronic device for checking table data balance of a distributed database. Background Art

[0002] Distributed database clusters can store, query, and analyze data. The total data of a business table needs to be scattered and distributed to all database nodes in the database, so that the computing resources of all nodes in the distributed database can be fully utilized. If a business table is unevenly distributed on each node, it will seriously affect the efficiency of querying and analyzing the data of this business table.

[0003] However, in actual production applications, a distributed database cluster has tens of thousands of business tables, and it is impossible to determine which business table has a balanced distribution, so it is necessary to traverse all business tables for investigation. Therefore, it takes a long time and affects the execution performance of the current cluster business. Summary of the invention

[0004] In view of the above problems, the present invention provides a method, device and electronic device for checking table data balance of a distributed database.

[0005] According to a first aspect of the present invention, a method for checking table data balance of a distributed database is provided, comprising:

[0006] A user management table and a system table corresponding to each of the plurality of database nodes are obtained, wherein the distributed database includes a plurality of database nodes and a central database node, the user management table is stored in the central database node, the system table is stored in the database node, the user management table stores table information of an i-th business table, and a plurality of system tables store table information of an i-th business table sub-table, where i ≥ 1 and is an integer; based on a management table version identifier of an i-th business table in the user management table and a sub-table version identifier of an i-th business table sub-table of each of the plurality of system tables, an update identification is performed on the i-th business table sub-table, and a table update identification result is determined, wherein an i-th business table sub-table of the i-th business table is stored on each of the database nodes; when it is determined that the table update identification result indicates that the i-th business table sub-table in at least one database node has been updated, storage spaces of the i-th business table sub-tables of each of the plurality of database nodes are determined; when it is determined that the storage spaces of the plurality of i-th business table sub-tables are unbalanced, the plurality of i-th business table sub-tables are balanced, and the balanced plurality of i-th business table sub-tables are stored in the plurality of database nodes.

[0007] Optionally, obtaining the user management table and the system tables corresponding to the multiple database nodes respectively includes:

[0008] A timed scheduling event is set in multiple database nodes; the timed scheduling event is run according to a preset time interval to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes respectively.

[0009] Optionally, the system table includes a storage space query time attribute, and the method for troubleshooting table data balance of a distributed database also includes:

[0010] For the system table stored in the jth database node, multiple table information in the system table is arranged in ascending order according to the storage space query time attribute value in the system table to obtain a first sequence; based on the first sequence, an update identification order between multiple business table sub-tables is determined.

[0011] Optionally, determining storage spaces of the ith business table sub-tables of the plurality of database nodes includes:

[0012] Call a query instruction to obtain the storage space query time and the i-th business table sub-table storage space corresponding to the storage space query time; based on the storage space query time and the i-th business table sub-table storage space corresponding to the storage space query time, update the storage space query time attribute value and storage space attribute value corresponding to the i-th business table sub-table in the system table stored in the database node.

[0013] Optionally, the user management table includes a business table number attribute. When it is determined that multiple business tables need to be balanced, the method for checking table data balance of a distributed database further includes:

[0014] According to the business table number attribute value in the user management table, multiple table information in the user management table is arranged in ascending order to obtain a second sequence; according to the second sequence, multiple business tables are respectively subjected to balancing processing.

[0015] Optionally, determining that storage spaces of multiple i-th business table sub-tables are unbalanced includes:

[0016] A balance standard value is determined based on the balance judgment coefficient, the number of database nodes and the total storage space of multiple i-th business table sub-tables; when the storage space of the i-th business table sub-table of each of the multiple database nodes is greater than the balance standard value, it is determined that the storage spaces of the multiple i-th business table sub-tables are unbalanced with each other.

[0017] Optionally, performing balancing processing on multiple i-th business table sub-tables, and storing the balanced multiple i-th business table sub-tables in multiple database nodes, includes:

[0018] According to the total storage space of the multiple i-th business table sub-tables and the number of database nodes, a balanced allocation strategy for the i-th business table is determined; based on the balanced allocation strategy, the multiple i-th business table sub-tables are balanced, and the balanced multiple i-th business table sub-tables are stored in multiple database nodes.

[0019] Optionally, running a scheduled event according to a preset time interval to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes respectively include:

[0020] The disk monitoring program is called to query the utilization rate of the disk related to the distributed database; when the utilization rate is less than a preset threshold, the user management table in the distributed database and the system tables corresponding to the multiple database nodes are traversed and checked.

[0021] A second aspect of the present invention provides a device for balancing table data of a distributed database, comprising:

[0022] An acquisition module is used to acquire a user management table and a system table corresponding to each of the plurality of database nodes, wherein the distributed database includes a plurality of database nodes and a central database node, the user management table is stored in the central database node, the system table is stored in the database node, the user management table stores table information of the i-th business table, and the plurality of system tables all store table information of the i-th business table sub-table, where i ≥ 1 and is an integer;

[0023] An update module, configured to perform update identification on the i-th business table sub-table based on the management table version identifier of the i-th business table in the user management table and the sub-table version identifiers of the i-th business table sub-tables of each of the plurality of system tables, and determine a table update identification result, wherein the i-th business table sub-table of the i-th business table is stored on each database node;

[0024] A determination module, configured to determine storage spaces of the i-th business table sub-tables of the respective multiple database nodes when the determination table update identification result indicates that the i-th business table sub-table in at least one database node has been updated;

[0025] The balancing module is used to balance the multiple i-th business table sub-tables when it is determined that the storage spaces of the multiple i-th business table sub-tables are unbalanced, and store the balanced multiple i-th business table sub-tables in multiple database nodes.

[0026] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned method for checking table data balancing of a distributed database.

[0027] According to the method, device and electronic device for checking the table data balance of the distributed database provided by the present invention, the table update identification result is determined by the management table version identifier of the i-th business table in the user management table and the sub-table version identifier of the i-th business table sub-table of each of the multiple system tables, thereby determining the storage space of multiple i-th business table sub-tables; when it is determined that the storage spaces of multiple i-th business table sub-tables are unbalanced, the multiple i-th business table sub-tables are balanced. Since system tables related to the storage spaces of the business table sub-tables are established in multiple database nodes, the storage spaces of the business table sub-tables can be determined according to the management table version identifier and the sub-table version identifier. In the case of basically no impact on the cluster business, the business table data distribution balance is checked in advance, and when it is determined that the storage spaces of multiple business table sub-tables are unbalanced, the balancing process is performed in advance, thereby ensuring the query performance and stability of the distributed database and improving the execution speed of the cluster business. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0029] Figure 1 The application scenarios of the method, device and apparatus for checking table data balance of a distributed database according to an embodiment of the present invention are shown;

[0030] Figure 2 A flowchart of a method for checking table data balance of a distributed database according to an embodiment of the present invention is shown;

[0031] Figure 3 An example schematic diagram showing that storage spaces of the i-th business table sub-tables are not balanced with each other according to an embodiment of the present invention;

[0032] Figure 4 It shows a structural block diagram of a device for balancing table data of a distributed database according to an embodiment of the present invention;

[0033] Figure 5 A block diagram of an electronic device suitable for implementing a method for balancing table data in a distributed database according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] In the process of implementing the present invention, it is found that when performing operations such as business table data query and analysis, it is necessary to return the execution results of the business table sub-tables in all database nodes. The total execution time depends on the business table sub-table that takes the longest time. If a business table is unevenly distributed on each node, the efficiency of data query and analysis on this business table is seriously affected. In a large-scale production environment, the total data volume of a single business table can often reach tens of GB or even hundreds of GB. When querying the balance of a single business table in real time, it takes a long time and occupies disk utilization, affecting the execution speed of normal business of the cluster. Therefore, it is often not feasible to perform real-time inspection and check the balance operation of the business table data.

[0039] In view of this, an embodiment of the present invention provides a method, device and electronic device for checking table data balance of a distributed database. The method includes: identifying results; determining the storage space of the i-th business table sub-table of each of the multiple database nodes when it is determined that the table update identification result indicates that the i-th business table sub-table in at least one database node has been updated; and performing balancing processing on the multiple i-th business table sub-tables when it is determined that the storage spaces of the multiple i-th business table sub-tables are not balanced with each other, and storing the balanced multiple i-th business table sub-tables in the multiple database nodes.

[0040] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0041] Figure 1 The application scenarios of the method, device, and apparatus for checking table data balance in a distributed database according to an embodiment of the present invention are shown.

[0042] like Figure 1 As shown, the business system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0043] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0044] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0045] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0046] It should be noted that the method for balancing table data of a distributed database provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the device for balancing table data of a distributed database provided in the embodiment of the present invention can generally be set in the server 105. The method for balancing table data of a distributed database provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the device for balancing table data of a distributed database provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0047] Alternatively, the method for balancing table data of a distributed database provided in the embodiment of the present invention may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the apparatus for balancing table data of a distributed database provided in the embodiment of the present invention may also be arranged in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may be arranged in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0048] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0049] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0050] Figure 2 A flow chart of a method for checking table data balance of a distributed database according to an embodiment of the present invention is shown.

[0051] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0052] In operation S210 , a user management table and system tables corresponding to respective database nodes are acquired.

[0053] Optionally, the distributed database includes multiple database nodes and a central database node, the user management table is stored in the central database node, the system table is stored in the database node, the user management table stores table information of the i-th business table, and multiple system tables store table information of the i-th business table sub-table, i≥1 and is an integer.

[0054] Optionally, the user management table stores table information of multiple business tables. For example, the table information may include table name information of the business table, database name information of the database where the business table is located, operation type information, operation time information, management table version identification information, etc.

[0055] Optionally, each database node stores a system table, which stores table information of multiple business sub-tables. For example, the table information may be table name information of the business sub-table, database name information of the business table sub-table where the database name information is located, sub-table version identification information, business table sub-table storage space information, etc.

[0056] For example, i represents the order of the business tables in the user management table. Multiple table information in the user management table is sorted according to a certain rule to obtain table information of the user management table including the first business table, the second business table, the third business table, and the fourth business table.

[0057] For example, taking the above-mentioned first business table, second business table, third business table, and fourth business table as examples, the table name information of the first business table in the user management table is t1, and the distributed database includes 4 database nodes. The system table stored in each database node contains the table information of the first business table sub-table, wherein the table name information of the first business table sub-table in each system table is t1_1, t1_2, t1_3, and t1_4 ​​respectively.

[0058] In operation S220, based on the management table version identifier of the i-th business table in the user management table and the sub-table version identifiers of the i-th business table sub-tables of the plurality of system tables, update identification is performed on the i-th business table sub-table to determine a table update identification result.

[0059] Optionally, each database node stores an i-th business table sub-table of the i-th business table.

[0060] Optionally, the business table sub-table data of the distributed database is distributed and stored on multiple database nodes, and each database node stores a corresponding business table sub-table, so that each database node can independently process its own business table data.

[0061] Optionally, the management table version identifier represents a unique identifier for performing a data modification operation on the i-th business table. After a data modification operation on the i-th business table subtable on at least one database node is successfully performed, the latest version identifier is generated, and the management table version identifier of the i-th business table in the user management table is updated accordingly.

[0062] For example, data "ab" is inserted into the first business table sub-tables t1_2 and t1_3 respectively. After the insertion is successful, the latest version identifier 123 is generated, and the management table version identifier of the first business table in the user management table is updated to 123 according to the version identifier 123.

[0063] Optionally, the i-th business table sub-table of each system table represents a unique identifier for a data modification operation on the i-th business table sub-table, and the sub-table version identifier will not be automatically updated according to the latest generated version identifier.

[0064] Optionally, whether the management table version identifier of the i-th business table in the user management table and the sub-table version identifiers of the i-th business table sub-tables of the multiple system tables are the same can be used to identify whether the storage space and sub-table version identifier of the i-th business table sub-table need to be updated to determine the table update identification result.

[0065] In operation S230, when it is determined that the table update identification result indicates that the i-th business table sub-table in at least one database node has been updated, storage spaces of the i-th business table sub-tables of the plurality of database nodes are determined.

[0066] For example, taking the first business table sub-table t1_1 corresponding to the first business table stored in the first database node as an example, if the management table version identifier of the first business table in the user management table and the sub-table version identifier of the first business table sub-table t1_1 are the same, then the table update identification result is determined to indicate that the data of the first business table sub-table on all database nodes has not been updated; if the management table version identifier of the first business table in the user management table and the sub-table version identifier of the first business table sub-table t1_1 are not the same, then the table update identification result is determined to indicate that the data of the first business table sub-table on at least one database node has been updated.

[0067] Optionally, when the i-th business table sub-table in at least one database node has been updated, the data update of the i-th business table sub-table will affect the storage space occupied by the i-th business table sub-table on disk. Therefore, it is necessary to re-read the storage space of the i-th business table sub-table; when the i-th business table sub-table in all database nodes has not been updated, it is not necessary to re-read the storage space of the i-th business table sub-table.

[0068] For example, taking the example that the management table version identifier of the above-mentioned first business table and the sub-table version identifier of the first business table sub-table t1_1 are different, it is determined that the table update identification result indicates that the data of the first business table sub-table on at least one database node has been updated, and then the storage space of the first business table sub-table t1_1 on the first database node is re-read.

[0069] Optionally, each business table sub-table is traversed in sequence according to a preset order, so as to determine the business table sub-table storage space of each of the multiple database nodes.

[0070] In operation S240 , when it is determined that storage spaces of the plurality of i-th business table sub-tables are unbalanced, the plurality of i-th business table sub-tables are balanced, and the balanced plurality of i-th business table sub-tables are stored in a plurality of database nodes.

[0071] Optionally, performing balancing processing on multiple i-th business table sub-tables may be performing data storage processing on an average basis, and each database node stores the data of the i-th business table sub-table after balancing processing.

[0072] For example, for the 1st business table, after determining the storage space of the 1st business table sub-table t1_1, the storage space of the 1st business table sub-table t1_2, the storage space of the 1st business table sub-table t1_3, and the storage space of the 1st business table sub-table t1_4, we then determine whether the four 1st business sub-table storage spaces are balanced with each other. If they are not balanced with each other, we balance the total data of the four 1st business sub-tables and store the balanced four 1st business table sub-tables back in the four database nodes.

[0073] Optionally, since system tables related to the storage space of business table sub-tables are established in multiple database nodes, the storage space of business table sub-tables can be determined based on the management table version identifier and the sub-table version identifier. The business table data distribution balance can be checked in advance without basically affecting the cluster business. If it is determined that the storage space of multiple business table sub-tables is unbalanced, balancing processing can be performed in advance, thereby ensuring the query performance and stability of the distributed database and improving the execution speed of the cluster business.

[0074] Optionally, before obtaining the system tables corresponding to the multiple database nodes, it is necessary to establish the system tables. The row data in the system tables may include the table name attribute value, database attribute value, storage space query time attribute value, storage space attribute value, sub-table version identification attribute value, etc. of the business table sub-table. Triggers are set on each database node. When the business table is created or deleted in the central database node, the business table sub-table is created or deleted in each database node accordingly, and row data is added or deleted in the system table on each database node.

[0075] For example, when the central database node deletes the first business table, the table information corresponding to the first business table subtable in the system table of each database node is deleted through a trigger; when the central database node modifies the table name of the first business table, the table name information corresponding to the first business table subtable in the system table of each database node is updated through a trigger.

[0076] For example, if business table t3 is created under database d1 of the distributed database, a data entry {d1,t3_1,1, ,} is inserted into the system table in database node 1, and a data entry {d1,t3_2,1, ,} is inserted into the system table in database node 2. The system table operations on other database nodes are similar.

[0077] Optionally, obtaining the user management table and the system tables corresponding to the multiple database nodes respectively includes:

[0078] A timed scheduling event is set in multiple database nodes; the timed scheduling event is run according to a preset time interval to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes respectively.

[0079] Optionally, the scheduled event can be an event scheduler, which is a set of structured query statements used to execute scheduled tasks. The scheduled event is set in each database node.

[0080] Optionally, the scheduled event is run once every preset time interval, and the preset time interval is set according to database parameters, for example, the default is 30 seconds.

[0081] Optionally, the server runs a timed scheduling event to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes.

[0082] Optionally, by setting up scheduled events in multiple database nodes, you can traverse and check the user management tables in the distributed database and the system tables corresponding to the multiple database nodes in advance, and then determine whether the storage space of the business table sub-tables and the business table sub-tables are balanced, and balance the unbalanced business table sub-tables in advance, effectively solving the problem of instant query table storage space affecting cluster business and taking a long time.

[0083] Optionally, the system table includes a storage space query time attribute, and the method for troubleshooting table data balance of a distributed database also includes:

[0084] For the system table stored in the jth database node, multiple table information in the system table is arranged in ascending order according to the storage space query time attribute value in the system table to obtain a first sequence; based on the first sequence, an update identification order between multiple business table sub-tables is determined.

[0085] Optionally, the storage space query time attribute represents the time of querying the storage space of data in the sub-table of the business table.

[0086] For example, for the system table stored in the first database node, the system table stores table information of two business table sub-tables, and the table name attribute value, database attribute value, storage space query time attribute value, storage space attribute value, and sub-table version identification attribute value correspond to "t1_1, db_1, 4.00, 20, 145" and "t2_1, db_1, 5.00, 25, 115", respectively. After arranging the multiple table information in the system table in ascending order according to the storage space query time attribute value in the system table, the first sequence containing each table information is obtained: business table sub-table t1_1, business table sub-table t2_1.

[0087] For example, according to the first sequence: business table sub-table t1_1, business table sub-table t2_1, it is determined to update the identification business table sub-table t1_1 first, and then update the identification business table sub-table t2_1.

[0088] Optionally, multiple business table sub-tables in the system table are updated and identified in sequence according to the update identification sequence between the multiple business table sub-tables, and then the storage spaces of the business table sub-tables are determined in sequence.

[0089] Optionally, identification is performed based on the table name attribute value, library name attribute value, sub-table version identification attribute value of a business table sub-table in the system table and the table name attribute value, library name attribute value, management table version identification attribute value of the business table in the user management table. When the table name attribute value and the library name attribute value are the same, and the management table version identification attribute value and the sub-table version identification attribute value are different, it is determined that the table update identification result indicates that the business table sub-table in at least one database node has been updated, and the storage space of this business table sub-table is re-read.

[0090] Optionally, for the system table stored in the jth database node, the update identification order among multiple business table sub-tables is determined according to the attribute value at the storage space query time, and then the business table sub-table with a small attribute value at the storage space query time is updated and identified first to determine the storage space of the business table sub-table. When a large number of cluster businesses occur, the update identification will be stopped, ensuring that the storage space of multiple business table sub-tables whose storage space has not been updated on the jth database node is updated as much as possible.

[0091] Optionally, determining storage spaces of the ith business table sub-tables of the plurality of database nodes respectively includes:

[0092] Call a query instruction to obtain the storage space query time and the i-th business table sub-table storage space corresponding to the storage space query time; based on the storage space query time and the i-th business table sub-table storage space corresponding to the storage space query time, update the storage space query time attribute value and storage space attribute value corresponding to the i-th business table sub-table in the system table stored in the database node.

[0093] Optionally, when it is determined that the table update identification result of the i-th business table sub-table indicates that the i-th business table sub-table in at least one database node has been updated, the server calls a query instruction at the storage space query time to obtain the i-th business table sub-table storage space.

[0094] Optionally, when it is determined that the table update identification result of the i-th business table sub-table indicates that the i-th business table sub-table in at least one database node has been updated, the sub-table version identification attribute value corresponding to the i-th business table sub-table in the system table stored in the database node may be updated according to the management table version identification attribute value of the i-th business table in the user management table.

[0095] For example, taking the system table stored in the first database node as an example, the management table version identification attribute value of the first business table in the user management table is 134, and the table update identification result of the first business table subtable t1_1 is that the first business table subtable in at least one database node has been updated. The server calls the query instruction at 6.00, reads the storage space 50 occupied by the first business table subtable t1_1 on the disk, and obtains the storage space query time: 6.00 and the storage space of the first business table subtable t1_1: 50. The storage space attribute value corresponding to the first business table subtable t1_1 in the system table stored in the first database node is updated to 50, the storage space query time attribute value is 6.00, and the subtable version identification attribute value is 134.

[0096] Optionally, the user management table includes a business table number attribute. When it is determined that multiple business tables need to be balanced, the method for checking table data balance of a distributed database further includes:

[0097] According to the business table number attribute value in the user management table, multiple table information in the user management table is arranged in ascending order to obtain a second sequence; according to the second sequence, multiple business tables are respectively subjected to balancing processing.

[0098] Optionally, the business table number attribute represents a unique numeric identifier of the business table, for example, business table 001.

[0099] For example, the business tables in the user management table need to be balanced, and the business table number attribute values ​​are 001, 011, and 004. According to the business table number attribute values, the multiple table information in the user management table are arranged in ascending order to obtain the second sequence: 001, 004, 011, and then the first business table is determined to be business table 001, the second business table is determined to be business table 004, and the third business table is determined to be business table 011. The storage space of the business table sub-tables of the first business table, the second business table, and the third business table is checked in turn to see if they are balanced.

[0100] For example, when checking the first business table, obtain the table name t1 and the database name db_1 of the business table 001, and then query the storage space of the first business table subtable t1_1, the storage space of the first business table subtable t1_2, the storage space of the first business table subtable t1_3, and the storage space of the first business table subtable t1_4 ​​on the four database nodes according to the structured query statement. When the first business table subtable t1_1, the first business table subtable t1_2, the first business table subtable t1_3, and the first business table subtable t1_4 ​​are unbalanced, balance the first business table.

[0101] Optionally, determining that storage spaces of multiple i-th business table sub-tables are unbalanced includes:

[0102] A balance standard value is determined based on the balance judgment coefficient, the number of database nodes and the total storage space of multiple i-th business table sub-tables; when the storage space of the i-th business table sub-table of each of the multiple database nodes is greater than the balance standard value, it is determined that the storage spaces of the multiple i-th business table sub-tables are unbalanced with each other.

[0103] Optionally, the total storage space of multiple i-th business table sub-tables is the sum of the storage spaces of multiple i-th business table sub-tables.

[0104] In one embodiment, the equilibrium standard value As shown in formula (1):

[0105] (1);

[0106] in, Characterizes the equilibrium judgment coefficient, Represents the total storage space of multiple i-th business table sub-tables, Indicates the number of database nodes.

[0107] Figure 3 An exemplary schematic diagram is shown in which storage spaces of sub-tables of the i-th business table are not balanced with each other according to an embodiment of the present invention.

[0108] like Figure 3 As shown, the distributed database includes three database nodes. The central database node stores the first business table. The first database node stores the first business table sub-table t1_1. The second database node stores the first business table sub-table t1_2. The third database node stores the first business table sub-table t1_3. is 1.2, and the balancing standard value is 18.14 GB. The storage space of the first business table subtable t1_1 is 11.29GB, the storage space of the first business table subtable t1_2 is 24.22GB, and the storage space of the first business table subtable t1_3 is 9.85GB. It is obtained that the storage space of the first business table subtable t1_1 is 11.29GB, which is less than 18.14 GB, the storage space of the first business table subtable t1_2 is 24.22GB, which is greater than 18.14 GB, and the storage space of the first business table subtable t1_3 is 11.29GB, which is less than 18.14 GB. Therefore, it is determined that there is a first business table subtable t1_2 that is greater than the balancing standard value, and the storage spaces of multiple first business table subtables are unbalanced, and the first business table needs to be balanced.

[0109] Optionally, the table name information, library name information, troubleshooting time information, etc. of the business table that needs to be balanced can be added to the event troubleshooting result table on the central database node. When the cluster business is not busy, the business table in the event troubleshooting result table is balanced.

[0110] Optionally, performing balancing processing on multiple i-th business table sub-tables, and storing the balanced multiple i-th business table sub-tables in multiple database nodes, includes:

[0111] According to the total storage space of the multiple i-th business table sub-tables and the number of database nodes, a balanced allocation strategy for the i-th business table is determined; based on the balanced allocation strategy, the multiple i-th business table sub-tables are balanced, and the balanced multiple i-th business table sub-tables are stored in multiple database nodes.

[0112] For example, based on the total storage space of multiple i-th business table sub-tables of 100 GB and the number of database nodes of 4, the balanced allocation strategy of the i-th business table is determined as follows: data is reallocated for each i-th business table sub-table and the data storage space is 25 GB. The balanced multiple i-th business table sub-tables are stored in multiple database nodes respectively.

[0113] Optionally, you can run a scheduled schedule to check the distribution balance of business tables in advance without affecting cluster services, to ensure the query performance and stability of the distributed database.

[0114] Optionally, running a scheduled event according to a preset time interval to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes respectively include:

[0115] The disk monitoring program is called to query the utilization rate of the disk related to the distributed database; when the utilization rate is less than a preset threshold, the user management table in the distributed database and the system tables corresponding to the multiple database nodes are traversed and checked.

[0116] Optionally, the disk utilization represents the busy state of the current cluster service.

[0117] Optionally, after a preset time interval is reached, the server calls a disk monitoring program to query the utilization of disks related to the distributed database.

[0118] For example, the preset threshold is 50%. If the utilization rate is greater than or equal to 50%, the scheduled scheduling event is terminated; if the utilization rate is less than 50%, the user management table in the distributed database and the system tables corresponding to multiple database nodes are traversed and checked.

[0119] Optionally, the busyness of the cluster business using the distributed database at this moment is determined based on the queried disk utilization, so as to flexibly select to traverse the user management table and the system table when the distributed database is idle to avoid affecting the cluster business.

[0120] Based on the above method for checking the table data balance of a distributed database, the present invention also provides a device for checking the table data balance of a distributed database. Figure 4 The device is described in detail.

[0121] Figure 4 A structural block diagram of a device for balancing table data in a distributed database according to an embodiment of the present invention is shown.

[0122] like Figure 4 As shown, the apparatus 400 for balancing table data of a distributed database in this embodiment includes an acquisition module 410 , an update module 420 , a determination module 430 and a balancing module 440 .

[0123] The acquisition module 410 is used to acquire the user management table and the system tables corresponding to the multiple database nodes, wherein the distributed database includes multiple database nodes and a central database node, the user management table is stored in the central database node, the system table is stored in the database node, the user management table stores the table information of the i-th business table, and the multiple system tables store the table information of the i-th business table sub-table, i≥1 and is an integer. In one embodiment, the acquisition module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0124] The update module 420 is used to update and identify the i-th business table sub-table based on the management table version identifier of the i-th business table in the user management table and the sub-table version identifiers of the i-th business table sub-tables of the multiple system tables, and determine the table update identification result, wherein the i-th business table sub-table of the i-th business table is stored on each database node. In one embodiment, the update module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0125] The determination module 430 is used to determine the storage space of the i-th business table sub-table of each of the plurality of database nodes when the determination table update identification result indicates that the i-th business table sub-table in at least one database node has been updated. In one embodiment, the determination module 430 can be used to perform the operation S230 described above, which will not be described in detail here.

[0126] The balancing module 440 is used to balance the multiple i-th business table sub-tables when it is determined that the storage spaces of the multiple i-th business table sub-tables are unbalanced, and store the balanced multiple i-th business table sub-tables in multiple database nodes. In one embodiment, the balancing module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0127] Optionally, the acquisition module 410 includes a setting submodule and an operation submodule.

[0128] The settings submodule is used to set timed scheduling events in multiple database nodes.

[0129] The running submodule is used to run the scheduled event according to the preset time interval to traverse and check the user management table in the distributed database and the system table corresponding to each of the multiple database nodes.

[0130] Optionally, the apparatus 400 for balancing table data of a distributed database further includes a system table sorting module and an identification module.

[0131] The system table sorting module is used to sort multiple table information in the system table stored in the j-th database node in ascending order according to the storage space query time attribute value in the system table to obtain a first sequence.

[0132] The identification module is used to determine the update identification order between the multiple business table sub-tables according to the first sequence.

[0133] Optionally, the determination module 430 includes a first determination submodule and a second determination submodule.

[0134] The first determination submodule is used to call the query instruction to obtain the storage space query time and the i-th business table sub-table storage space corresponding to the storage space query time.

[0135] The second determination submodule is used to update the storage space query time attribute value and storage space attribute value corresponding to the i-th business table subtable in the system table stored in the database node according to the storage space query time and the i-th business table subtable storage space corresponding to the storage space query time.

[0136] Optionally, the apparatus 400 for balancing table data of a distributed database further includes a business table sorting module and a processing module.

[0137] The business table sorting module is used to sort the multiple table information in the user management table in ascending order according to the business table number attribute value in the user management table to obtain a second sequence.

[0138] The processing module is used to perform balancing processing on multiple service tables respectively according to the second sequence.

[0139] Optionally, the balancing module 440 includes a first balancing submodule and a second balancing submodule.

[0140] The first balancing submodule is used to determine a balancing standard value according to a balancing judgment coefficient, the number of database nodes and a total storage space of a plurality of i-th business table subtables.

[0141] The second balancing submodule is used to determine that the storage spaces of the multiple i-th business table sub-tables are unbalanced when the storage spaces of the i-th business table sub-tables of the multiple database nodes are greater than the balancing standard value.

[0142] Optionally, the balancing module 440 further includes a third balancing submodule and a fourth balancing submodule.

[0143] The third balancing submodule is used to determine a balanced allocation strategy for the i-th business table according to the total storage space of the plurality of i-th business table sub-tables and the number of database nodes.

[0144] The fourth balancing submodule is used to perform balancing processing on the multiple i-th business table sub-tables based on the balanced allocation strategy, and store the balanced multiple i-th business table sub-tables in multiple database nodes.

[0145] Optionally, the running submodule includes a calling unit and a troubleshooting unit.

[0146] The calling unit is used to call the disk monitoring program to query the utilization rate of the disk related to the distributed database.

[0147] The checking unit is used to traverse and check the user management table and the system tables corresponding to the multiple database nodes in the distributed database when the utilization rate is less than a preset threshold.

[0148] Optionally, any multiple modules among the acquisition module 410, the update module 420, the determination module 430 and the equalization module 440 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. Optionally, at least one of the acquisition module 410, the update module 420, the determination module 430 and the equalization module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 410, the update module 420, the determination module 430 and the equalization module 440 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0149] Figure 5 A block diagram of an electronic device suitable for implementing a method for balancing table data in a distributed database according to an embodiment of the present invention is shown.

[0150] Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0151] like Figure 5 As shown, the computer electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 509 to the random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an on-board memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0152] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0153] Optionally, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0154] Optionally, the method flow according to the embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. Optionally, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0155] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method for checking table data balancing of a distributed database according to an embodiment of the present invention is implemented.

[0156] Optionally, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0157] For example, optionally, the computer-readable storage medium may include the ROM 502 and / or the RAM 503 described above and / or one or more memories other than the ROM 502 and the RAM 503 .

[0158] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for checking table data balancing of a distributed database provided by the embodiment of the present invention.

[0159] When the computer program is executed by the processor 501, the above functions defined in the system / device of the embodiment of the present invention are performed. Optionally, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0160] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0161] Optionally, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0162] Optionally, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present invention can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teaching of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0163] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for checking table data balance of a distributed database, characterized in that: The method comprises: Obtain a user management table and a system table corresponding to each of the plurality of database nodes, wherein the distributed database includes the plurality of database nodes and a central database node, the user management table is stored in the central database node, the system table is stored in the database node, the user management table stores table information of an i-th business table, and the plurality of system tables all store table information of an i-th business table sub-table, where i≥1 and is an integer; Based on the management table version identifier of the i-th business table in the user management table and the sub-table version identifiers of the i-th business table sub-tables of the plurality of system tables, update identification is performed on the i-th business table sub-table to determine a table update identification result, wherein the i-th business table sub-table of the i-th business table is stored on each of the database nodes; In a case where it is determined that the table update identification result indicates that an i-th business table sub-table in at least one of the database nodes has been updated, determining storage spaces of the i-th business table sub-tables of the respective database nodes; When it is determined that the storage spaces of the plurality of i-th business table sub-tables are unbalanced, the plurality of i-th business table sub-tables are balanced, and the balanced plurality of i-th business table sub-tables are stored in the plurality of database nodes.

2. The method according to claim 1, characterized in that The method of obtaining the user management table and the system tables corresponding to the plurality of database nodes respectively comprises: Setting a timed scheduling event in a plurality of the database nodes; The timed scheduling event is run according to a preset time interval to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes respectively.

3. The method according to claim 2, characterized in that The system table includes a storage space query time attribute, and the method further includes: For the system table stored in the j-th database node, multiple table information in the system table are arranged in ascending order according to the storage space query time attribute value in the system table to obtain a first sequence; According to the first sequence, an update identification order between the plurality of the business table sub-tables is determined.

4. The method according to claim 1, characterized in that Determining the storage space of the ith business table sub-table of each of the plurality of database nodes includes: Calling a query instruction to obtain a storage space query time and the i-th business table sub-table storage space corresponding to the storage space query time; According to the storage space query time and the i-th business table subtable storage space corresponding to the storage space query time, the storage space query time attribute value and the storage space attribute value corresponding to the i-th business table subtable in the system table stored in the database node are updated.

5. The method according to claim 1, characterized in that The user management table includes a service table number attribute. When it is determined that multiple service tables need to be balanced, the method further includes: Arrange the plurality of table information in the user management table in ascending order according to the business table number attribute value in the user management table to obtain a second sequence; According to the second sequence, the multiple service tables are respectively subjected to balancing processing.

6. The method according to claim 1, characterized in that Determining that storage spaces of the plurality of i-th business table sub-tables are unbalanced with each other includes: Determine a balance standard value according to a balance judgment coefficient, the number of database nodes and a total storage space of a plurality of the i-th business table sub-tables; When the storage space of the i-th business table sub-table of each of the plurality of database nodes is greater than the balance standard value, it is determined that the storage spaces of the plurality of i-th business table sub-tables are unbalanced with each other.

7. The method according to claim 1, characterized in that Performing balancing processing on the plurality of the i-th business table sub-tables, and storing the balanced plurality of the i-th business table sub-tables in the plurality of database nodes, comprises: Determining a balanced allocation strategy for the i-th business table according to a total storage space of a plurality of the i-th business table sub-tables and the number of the database nodes; Based on the balanced allocation strategy, the plurality of the i-th business table sub-tables are balanced, and the balanced plurality of the i-th business table sub-tables are stored in the plurality of database nodes.

8. The method according to claim 2, characterized in that: Running the timed scheduling event according to a preset time interval to traverse and check the user management table in the distributed database and the system tables corresponding to the multiple database nodes respectively include: Call the disk monitoring program to query the utilization of the disks related to the distributed database; When the utilization rate is less than a preset threshold, the user management table and the system tables corresponding to the plurality of database nodes in the distributed database are traversed and checked.

9. A device for balancing table data in a distributed database, characterized in that: The device comprises: an acquisition module, configured to acquire a user management table and system tables corresponding to respective database nodes, wherein the distributed database includes the plurality of database nodes and a central database node, the user management table is stored in the central database node, the system table is stored in the database node, the user management table stores table information of an i-th business table, and the plurality of system tables all store table information of an i-th business table sub-table, where i≥1 and is an integer; an update module, configured to perform update identification on the i-th business table sub-table based on the management table version identifier of the i-th business table in the user management table and the sub-table version identifiers of the i-th business table sub-tables of each of the plurality of system tables, and determine a table update identification result, wherein the i-th business table sub-table of the i-th business table is stored on each of the database nodes; A determination module, configured to determine storage spaces of the i-th business table sub-tables of the respective database nodes when it is determined that the table update identification result indicates that the i-th business table sub-table in at least one of the database nodes has been updated; The balancing module is used to balance the multiple i-th business table sub-tables when it is determined that the storage spaces of the multiple i-th business table sub-tables are unbalanced, and store the balanced multiple i-th business table sub-tables in the multiple database nodes.

10. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 8.

Images