Database table redistribution method and system based on distributed database

Abstract

The invention discloses a database table redistribution method and system based on a distributed database. The method comprises the steps of performing threshold judgment on an increment time delay result between a source basic library table and a target library table through an acquisition playback component, and if a preset time delay threshold is met, generating a table locking request and sending the table locking request to a distributed database; performing state modification on the source basic library table based on the table locking request through the distributed database to generate a source lock library table; sending a service request containing the table name of the library table to be cached to the distributed database through the target service program; performing state evaluation on the to-be-cached library table based on the source lock library table through the distributed database, generating a lock table identifier corresponding to the source lock library table based on a state evaluation result and a preset window period, and sending the lock table identifier to an acquisition playback component; and executing margin data synchronization through the acquisition playback component. By means of the technical scheme, data redistribution can be conducted on the specified database table in the distributed database, and the efficiency and accuracy of data redistribution are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a library table redistribution method and system based on a distributed database. Background Art

[0002] With the rapid development of computer technology, distributed databases have become an important tool for storing massive data. However, since distributed databases contain the database tables of various local databases, it is particularly important to redistribute data in designated database tables in distributed databases in actual application scenarios.

[0003] In the prior art, the designated library table that needs to be redistributed is usually locked by event monitoring. That is, during the library table locking stage, it is necessary to continuously monitor the unfinished transactions that have involved the redistribution of the library table until the transaction ends and the library table is locked successfully. After that, the data of the designated library table is synchronized and the library table is unlocked.

[0004] However, if the method of the prior art is used, if the transaction involving the redistribution table is always in a suspended state or the transaction is a very long transaction, the table lock phase will time out, causing the entire redistribution task to fail. This reduces the efficiency and accuracy of data redistribution. Therefore, how to realize data redistribution of specified tables in a distributed database and improve the efficiency and accuracy of data redistribution is a problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a library table redistribution method and system based on a distributed database, which can solve the problem of low efficiency and accuracy of data redistribution.

[0006] According to one aspect of the present invention, a library table redistribution method based on a distributed database is provided, comprising:

[0007] The incremental delay result between the source basic library table and the target library table is judged by the acquisition and playback component. If the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database.

[0008] A lock table request is received through a distributed database, and a state of the source basic library table is modified based on the lock table request to generate a source lock library table;

[0009] Sending a business request to the distributed database through a target business program; wherein the business request includes the table name of the to-be-cached library table corresponding to the business request;

[0010] Receiving a business request through a distributed database, performing a status evaluation on the to-be-cached library table based on the source lock library table, generating a lock table identifier corresponding to the source lock library table based on the status evaluation result and a preset window period, and sending the lock table identifier to a collection and playback component;

[0011] The lock table identifier is received by the acquisition and playback component, and the surplus data synchronization is performed to complete the library table redistribution of the source basic library table.

[0012] According to another aspect of the present invention, a library table redistribution system based on a distributed database is provided, comprising:

[0013] The acquisition and playback component is used to perform a threshold judgment on the incremental delay result between the source basic library table and the target library table. If the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database;

[0014] A distributed database, used for receiving a table lock request, and modifying the state of the source basic library table based on the table lock request to generate a source lock library table;

[0015] A target business program, used to send a business request to the distributed database; wherein the business request includes the table name of the to-be-cached library table corresponding to the business request;

[0016] A distributed database, used for receiving a business request, performing a status evaluation on the to-be-cached library table based on the source lock library table, generating a lock table identifier corresponding to the source lock library table based on the status evaluation result and a preset window period, and sending the lock table identifier to a collection and playback component;

[0017] The acquisition and playback component is used to receive the lock table identifier, perform surplus data synchronization, and complete the library table redistribution of the source basic library table.

[0018] The technical solution of the embodiment of the present invention is to perform a threshold judgment on the incremental delay result between the source basic library table and the target library table through the acquisition and playback component. If the incremental delay result meets the preset delay threshold, a lock table request is generated, and the lock table request is sent to the distributed database; the lock table request is received through the distributed database, and the state of the source basic library table is modified based on the lock table request to generate a source lock library table; a business request containing the table name of the library table to be cached is sent to the distributed database through the target business program; the business request is received through the distributed database, and the state of the library table to be cached is evaluated based on the source lock library table, and a lock table identifier corresponding to the source lock library table is generated based on the state evaluation result and the preset window period, and the lock table identifier is sent to the acquisition and playback component; the lock table identifier is received through the acquisition and playback component, and the surplus data synchronization is performed to complete the library table redistribution of the source basic library table. Since the preset window period is used to limit the unfinished transactions involving the redistribution library table, the problem of low efficiency and accuracy of data redistribution is solved, and data redistribution of the specified library table in the distributed database can be realized, which improves the efficiency and accuracy of data redistribution.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a flow chart of a library table redistribution method based on a distributed database provided according to the prior art;

[0022] Figure 2 is a flow chart of a library table redistribution method based on a distributed database provided according to Embodiment 1 of the present invention;

[0023] Figure 3 is a flow chart of a library table redistribution method based on a distributed database provided according to Embodiment 2 of the present invention;

[0024] Figure 4 is a flowchart of an optional library table redistribution method based on a distributed database provided according to Embodiment 2 of the present invention;

[0025] Figure 5It is a structural diagram of a library table redistribution system based on a distributed database provided according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "objective", "basis", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Figure 1 The flowchart of a library table redistribution method based on a distributed database provided by the prior art is shown. Specifically, first, a redistribution task is created based on the current actual demand, and a shadow table of the table to be redistributed is generated. After that, full data collection and playback and incremental data collection and playback are performed. When the delay between the incremental data collection and playback and the source end, that is, the table to be redistributed, is lower than the set value, the source table lock stage is performed. In the source table lock stage, the state of the source table to be redistributed is set to an unavailable state. After the state is set, all structured query languages ​​(SQL) involving the table terminate execution, all return failures, and roll back the transaction where the SQL statement is located. Among them, it includes SQL statements that are being executed and those to be executed later. At the same time, it is necessary to continuously monitor transactions that have already involved the redistribution table and have not ended until the transaction ends and the source table lock is successful. Further, incremental synchronization completes data comparison, and renames the source table and shadow table. Finally, the source table is unlocked and data cleaning is performed to complete the redistribution task.

[0029] However, if the data redistribution method in the prior art is used, in the source table lock stage, if the transaction involving the redistribution table is always in a suspended state or the transaction is an overlong transaction, the source table lock will time out, the entire redistribution task will fail, and the source table state will be restored to an available state. In addition, after the source table lock is successfully executed until the source table is successfully unlocked, during this period of time, all SQL statements involving the redistribution table cannot be executed and all return failure. As a result, the efficiency and accuracy of data redistribution are reduced. Therefore, in order to solve the problem of low efficiency and accuracy of data redistribution, an embodiment of the present invention proposes a library table redistribution method based on a distributed database.

[0030] Embodiment 1

[0031] Figure 2 This is a flowchart of a library table redistribution method based on a distributed database provided in the first embodiment of the present invention. This embodiment is applicable to the case where data redistribution is completed without the business being aware of it. The method can be executed by a library table redistribution system based on a distributed database. The library table redistribution system based on a distributed database can be implemented in the form of hardware and / or software. Figure 2 As shown, the method includes:

[0032] S110. Perform a threshold judgment on the incremental delay result between the source basic library table and the target library table through the acquisition and playback component. If the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database.

[0033] The acquisition and playback component may refer to a component that acquires and redistributes data. The acquisition and playback component may acquire data from the library table to be redistributed and then insert the data into the new library table after redistribution according to the new distribution rule. The source basic library table may refer to the source library table that needs to redistribute data. The target library table may refer to the new library table after redistribution. That is, the library table to which the data needs to be stored again.

[0034] In an optional embodiment, before the threshold judgment is performed on the incremental delay result between the source basic library table and the target library table by the acquisition and playback component, it also includes: receiving a library table redistribution task through a distributed database; determining the corresponding source basic library table and target library table based on the library table redistribution task through a distributed database. Among them, the library table redistribution task may refer to a task for redistributing data from a source library table. Exemplarily, the library table redistribution task may include the table name of the source basic library table and the table name of the target library table. Usually, the user can establish a corresponding library table redistribution task according to actual application requirements. Therefore, after receiving the library table redistribution task through a distributed database, the corresponding source basic library table and the target library table can be directly determined according to the library table redistribution task, providing an effective basis for subsequent operations.

[0035] Among them, the incremental delay result may refer to the delay value in the incremental update process. The preset delay threshold may refer to a preset value for evaluating the incremental delay result. Generally, the preset delay threshold can be customized according to user needs. Exemplarily, the preset delay threshold may be 1 second, or 2 seconds, etc., which is not specifically limited in the embodiment of the present invention. The lock table request may refer to a request to lock the source basic library table. In an optional embodiment, the lock table request may include: the table name of the source basic library table.

[0036] S120: receiving a table lock request through a distributed database, and modifying the state of the source basic library table based on the table lock request to generate a source lock library table.

[0037] A distributed database refers to a physically distributed and logically centralized database system that is formed by connecting multiple physically dispersed data storage units using a high-speed computer network. Physical distribution means that the data in a distributed database system is distributed on different sites that are connected by a network and geographically dispersed. Logically centralized means that the database sites are logically a whole and are managed by a unified database management system, and each site has the ability to manage local data.

[0038] Among them, the state modification may refer to an operation of modifying the usage state of a library table. Exemplarily, in an embodiment of the present invention, the state modification may be to modify an available state to an unavailable state, or to modify an unavailable state to an available state. The source lock library table may refer to a library table obtained by modifying the usage state of a source base library table from an available state to an unavailable state.

[0039] S130. Send a business request to the distributed database through a target business program; wherein the business request includes a table name of a to-be-cached library table corresponding to the business request.

[0040] The target business program may refer to a program corresponding to a user using a distributed database. The target business program may implement a corresponding business function. Generally, SQL statements related to the business function may be generated in the target business program and sent to the distributed database for execution.

[0041] Among them, the business request may refer to an implementation request corresponding to a business function. Exemplarily, the business request may include an SQL statement corresponding to the business function, and the SQL statement may also include a library table name corresponding to the business function. The library table to be cached may refer to the library table involved in the business request. That is, after the business function related to the business request is implemented, the library table for storing the generated business data.

[0042] S140, receiving a business request through a distributed database, performing a status evaluation on the to-be-cached library table based on the source lock library table, generating a lock table identifier corresponding to the source lock library table based on the status evaluation result and a preset window period, and sending the lock table identifier to a collection and playback component.

[0043] Among them, the status evaluation may refer to an operation to determine the status of the library table. The preset window period may refer to a preset value for evaluating the execution time of the business function corresponding to the business request. Generally, the preset window period can be customized according to user needs. Exemplarily, the preset window period may be 2. The lock table identifier may refer to an identifier indicating that the source lock library table has successfully locked the table. Exemplarily, the lock table identifier may be a lock table success field.

[0044] S150: Receive the lock table identifier through the acquisition and playback component, and perform surplus data synchronization to complete the library table redistribution of the source basic library table.

[0045] Among them, the remaining data synchronization can refer to the operation of synchronizing the newly added remaining data after the source lock library table is successfully locked. Usually, after the source lock library table is successfully locked, the source lock library table stops adding new data, but new data will be generated within 1-2 seconds after the source lock library table is successfully locked. Therefore, after receiving the lock table identifier, the acquisition and playback component can update the remaining 1-2 seconds of the newly added data in the source lock library table to the target library table for the last time, thereby realizing data synchronization between the source basic library table and the target library table.

[0046] The technical solution of the embodiment of the present invention is to perform a threshold judgment on the incremental delay result between the source basic library table and the target library table through the acquisition and playback component. If the incremental delay result meets the preset delay threshold, a lock table request is generated, and the lock table request is sent to the distributed database; the lock table request is received through the distributed database, and the state of the source basic library table is modified based on the lock table request to generate a source lock library table; a business request containing the table name of the library table to be cached is sent to the distributed database through the target business program; the business request is received through the distributed database, and the state of the library table to be cached is evaluated based on the source lock library table, and a lock table identifier corresponding to the source lock library table is generated based on the state evaluation result and the preset window period, and the lock table identifier is sent to the acquisition and playback component; the lock table identifier is received through the acquisition and playback component, and the surplus data synchronization is performed to complete the library table redistribution of the source basic library table. Since the preset window period is used to limit the unfinished transactions involving the redistribution library table, the problem of low efficiency and accuracy of data redistribution is solved, and data redistribution of the specified library table in the distributed database can be realized, which improves the efficiency and accuracy of data redistribution.

[0047] Embodiment 2

[0048] Figure 3A flowchart of a library table redistribution method based on a distributed database is provided for the second embodiment of the present invention. This embodiment is refined based on the above embodiment. In this embodiment, the operation of generating a lock table identifier corresponding to the source lock library table based on a status evaluation result and a preset window period through a distributed database is specifically refined. Specifically, it may include: performing a status judgment on the status evaluation result through a distributed database, and if the status evaluation result is an unavailable state, determining the transaction type corresponding to the business request, and generating a lock table identifier corresponding to the source lock library table based on the transaction type and the preset window period. Figure 3 As shown, the method includes:

[0049] S210: Acquire incremental data in the source basic library table through the acquisition and playback component, and record the acquisition time point.

[0050] Among them, incremental data may refer to the data newly added to the source basic library table after the start of data redistribution. Usually, the process of data redistribution involves two types of data, full data and incremental data. Full data may refer to the data that already exists in the source basic library table before the start of redistribution. After all the data are inserted into the target library table, the incremental data is updated to the target library table. The collection time point may refer to the time point when the incremental data is collected from the source basic library table. In an embodiment of the present invention, the collection time point may be the time point when the incremental data is collected from the source basic library table for the last time.

[0051] S220. Update the incremental data to the target library table through the acquisition and playback component, and record the storage time point.

[0052] The storage time point may refer to the time point when the incremental data is updated to the target library table. In the embodiment of the present invention, the storage time point may be the time point when the incremental data is last updated to the target library table.

[0053] S230. Determine the incremental delay result between the source basic library table and the target library table based on the difference between the storage time point and the collection time point through the collection and playback component, and perform a threshold judgment on the incremental delay result based on a preset delay threshold. If the incremental delay result meets the preset delay threshold, generate a table lock request, and send the table lock request to the distributed database.

[0054] Specifically, the acquisition and playback component records the acquisition time point of the incremental data in the source basic library table and the storage time point when the incremental data is updated to the target library table, and the difference between the storage time point and the acquisition time point is calculated to obtain the incremental delay result. Then, the incremental delay result is judged by the preset delay threshold. When the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database. In this way, the timing of locking the source basic library table can be determined to avoid table lock delays, providing an effective basis for subsequent operations.

[0055] S240: Receive a table lock request through a distributed database.

[0056] S250: Determine the corresponding target library table location based on the table name of the source basic library table through the distributed database.

[0057] The target library table location may refer to a location in a distributed database where a source basic library table is stored. Typically, a distributed database stores multiple library tables, and the storage location of each library table can be determined by the table name of each library table.

[0058] S260. Modify the library table state of the source basic library table corresponding to the target library table position to an unavailable state through a distributed database, and generate a source lock library table.

[0059] Specifically, when the distributed database receives a table lock request and determines the target table location corresponding to the table lock request, the table state of the source basic table corresponding to the target table location can be modified to an unavailable state. Exemplarily, the memory value corresponding to the source basic table can be modified to achieve the table state modification. Thus, the table lock of the source basic table is achieved.

[0060] S270. Send a business request to the distributed database through a target business program; wherein the business request includes a table name of a to-be-cached library table corresponding to the business request.

[0061] S280. Receive a business request through a distributed database.

[0062] S290. Perform a consistency comparison on the table name of the to-be-cached library table based on the table name of the source lock library table through a distributed database, and generate a status evaluation result corresponding to the to-be-cached library table based on the consistency comparison result.

[0063] The consistency comparison result may refer to the repetitive comparison result of the table name of the source lock library table and the table name of the to-be-cached library table. Exemplarily, the consistency comparison result may be that the table names are consistent or inconsistent.

[0064] Among them, the status evaluation result may refer to the table status result of the library table to be cached. Exemplarily, the status evaluation result corresponding to the library table to be cached may be determined according to the consistency comparison result. Specifically, if the consistency comparison result is that the table name is consistent, the status evaluation result corresponding to the library table to be cached may be an unavailable state; conversely, if the consistency comparison result is that the table name is inconsistent, the status evaluation result corresponding to the library table to be cached may be an available state.

[0065] S2100. Perform status judgment on the status evaluation result through a distributed database. If the status evaluation result is an unavailable state, determine the transaction type corresponding to the business request, and generate a lock table identifier corresponding to the source lock library table based on the transaction type and a preset window period, and send the lock table identifier to the acquisition and playback component.

[0066] A transaction may refer to the smallest logical unit of work in a distributed database. Generally, a transaction may be a SQL statement, a group of SQL statements, or an entire program. A transaction type may refer to the operation status corresponding to a business request. For example, a transaction type may be an operated type or an unoperated type.

[0067] In an optional embodiment, a lock table identifier corresponding to the source lock library table is generated through a distributed database based on the transaction type and a preset window period, including: performing type judgment on the transaction type through a distributed database, and if the transaction type is an operated type, determining the business execution duration corresponding to the business request; performing numerical judgment on the business execution duration based on a preset window period through a distributed database, and if the business execution duration exceeds the preset window period, rolling back the business request and generating a lock table identifier corresponding to the source lock library table.

[0068] Among them, the business execution duration may refer to the length of time consumed to implement the business function corresponding to the business request. Specifically, after determining that the transaction type is an operated type, a numerical judgment can be made on the business execution duration corresponding to the business request based on a preset window period. If the business execution duration does not exceed the preset window period, the transaction that has operated the source lock library table can be executed normally until the end; if the business execution duration exceeds the preset window period, the business request is rolled back and a lock table identifier corresponding to the source lock library table is generated. As a result, suspended transactions or overlong transactions involving the redistribution table will not cause the redistribution impact to expand or fail. At the same time, online businesses will not have SQL statement errors caused by redistribution, which improves the efficiency and accuracy of data redistribution.

[0069] It is worth noting that in an embodiment of the present invention, after determining that the status evaluation result is an unavailable state, the business request can be cached. If the subsequent business execution time does not exceed the preset window period, the business request is executed normally and a response is returned; conversely, if the subsequent business execution time exceeds the preset window period, the business request is backed up and executed after the redistribution table is unlocked.

[0070] S2110: Receive the lock table identifier through the acquisition and playback component, and perform surplus data synchronization to complete the library table redistribution of the source basic library table.

[0071] In an optional embodiment, after receiving the lock table identifier through the acquisition and playback component, performing the balance data synchronization, and completing the library table redistribution of the source basic library table, it also includes: generating an unlock request through the acquisition and playback component, and sending the unlock request to the distributed database; wherein the unlock request includes the table name of the source lock library table; receiving the unlock request through the distributed database, and modifying the library table status of the source lock library table to an available state.

[0072] Among them, the unlock request may refer to a request to unlock the source lock library table. Specifically, after the library table redistribution of the source basic library table is completed, the unlock request can be generated by the acquisition and playback component and sent to the distributed database. Then, the library table state of the source lock library table is modified to an available state through the distributed database. In this way, the suspended transaction can be replayed and cached, ensuring the efficiency of transaction execution.

[0073] It is worth noting that for suspended transactions or overlong transactions involving redistribution tables, after the preset window period has passed, the SQL that has been executed but not submitted is backed up first, and such transactions are rolled back to release the metadata lock of the table to be redistributed, ensuring that the source lock library table lock is executed. After the source lock library table is unlocked, the backed-up suspended transaction is replayed, and the rollback and replay processes do not respond to the business. Therefore, the embodiment of the present invention can achieve business non-perception.

[0074] The technical solution of the embodiment of the present invention obtains the incremental data in the source basic library table through the acquisition and playback component, and records the acquisition time point. The incremental data is updated to the target library table through the acquisition and playback component, and the storage time point is recorded. The incremental delay result between the source basic library table and the target library table is determined by the acquisition and playback component based on the difference between the storage time point and the acquisition time point, and the incremental delay result is threshold judged based on the preset delay threshold. If the incremental delay result meets the preset delay threshold, a lock table request is generated, and the lock table request is sent to the distributed database. The lock table request is received through the distributed database. The corresponding target library table position is determined based on the table name of the source basic library table through the distributed database. The library table state of the source basic library table corresponding to the target library table position is modified to an unavailable state through the distributed database, and a source lock library table is generated. The business request is sent to the distributed database through the target business program. The business request is received through the distributed database. The table name of the cached library table is compared for consistency based on the table name of the source lock library table through the distributed database, and the state evaluation result corresponding to the cached library table is generated based on the consistency comparison result. The status evaluation result is judged by the distributed database. If the status evaluation result is unavailable, the transaction type corresponding to the business request is determined, and the lock table identifier corresponding to the source lock table is generated based on the transaction type and the preset window period, and the lock table identifier is sent to the acquisition and playback component. The lock table identifier is received by the acquisition and playback component, and the remaining data synchronization is performed to complete the library table redistribution of the source basic library table. Since the preset window period is used to limit the unfinished transactions involving the redistributed library table, the problem of low efficiency and accuracy of data redistribution is solved, and data redistribution of the specified library table in the distributed database can be realized, which improves the efficiency and accuracy of data redistribution.

[0075] Figure 4The flowchart of an optional library table redistribution method based on a distributed database provided by an embodiment of the present invention is shown. Specifically, first, the incremental delay result between the source basic library table and the target library table is judged by a threshold through the acquisition and playback component. If the incremental delay result meets the preset delay threshold, a lock table request is generated and the lock table request is sent to the distributed database. Then, the state of the source basic library table is modified based on the lock table request by the distributed database, the source basic library table is set to an unavailable state, and a source lock library table is generated. After that, a business request is sent to the distributed database through the target business program. And, the SQL statement involving the source lock library table to be executed in the business request is cached by the distributed database, and the transaction whose source lock library table is backed up and rolled back and the business execution time exceeds the preset window period is backed up, and the lock table identifier corresponding to the source lock library table is generated and sent to the acquisition and playback component. Further, the acquisition and playback component performs the synchronization of the residual data, completes the library table redistribution of the source basic library table, generates an unlock request after completing the library table redistribution of the source basic library table, and sends the unlock request to the distributed database. Finally, the table status of the source lock table is modified to an available state through the distributed database, and the cached SQL and backed-up transactions are replayed. The cached SQL needs to return a response to the target business program.

[0076] Embodiment 3

[0077] Figure 5 The structure diagram of a library table redistribution system based on a distributed database provided by the third embodiment of the present invention is shown in FIG. Figure 5 As shown, the system includes: a collection and playback component 310, a distributed database 320 and a target business program 330;

[0078] The acquisition and playback component 310 is used to perform a threshold judgment on the incremental delay result between the source basic library table and the target library table. If the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database.

[0079] The distributed database 320 is used to receive a lock table request, and based on the lock table request, modify the state of the source basic library table to generate a source lock library table;

[0080] The target business program 330 is used to send a business request to the distributed database; wherein the business request includes the table name of the to-be-cached library table corresponding to the business request;

[0081] The distributed database 320 is used to receive a business request, perform a status evaluation on the to-be-cached library table based on the source lock library table, generate a lock table identifier corresponding to the source lock library table based on the status evaluation result and a preset window period, and send the lock table identifier to the acquisition and playback component;

[0082] The acquisition and playback component 310 is used to receive the lock table identifier, perform surplus data synchronization, and complete the library table redistribution of the source basic library table.

[0083] The technical solution of the embodiment of the present invention is as follows: the acquisition and playback component 310 is used to perform a threshold judgment on the incremental delay result between the source basic library table and the target library table. If the incremental delay result meets the preset delay threshold, a lock table request is generated and the lock table request is sent to the distributed database; the distributed database 320 is used to receive the lock table request, and based on the lock table request, the state of the source basic library table is modified to generate a source lock library table; the target business program 330 is used to send a business request to the distributed database; wherein the business request includes the table name of the to-be-cached library table corresponding to the business request; the distributed database 320 is used to receive the business request, and based on the source lock library table, the state of the to-be-cached library table is evaluated, and based on the state evaluation result and the preset window period, a lock table identifier corresponding to the source lock library table is generated, and the lock table identifier is sent to the acquisition and playback component; the acquisition and playback component 310 is used to receive the lock table identifier, and perform surplus data synchronization to complete the library table redistribution of the source basic library table. Since the preset window period is used to limit the unfinished transactions involving the redistribution of library tables, the problem of low efficiency and accuracy of data redistribution is solved, and data redistribution of designated library tables in the distributed database can be achieved, thereby improving the efficiency and accuracy of data redistribution.

[0084] Optionally, the acquisition and playback component 310 may be specifically used for:

[0085] Obtain incremental data from the source base database table and record the collection time point;

[0086] Update the incremental data to the target database table and record the storage time point;

[0087] Based on the difference between the storage time point and the collection time point, an incremental delay result between the source basic library table and the target library table is determined, and a threshold judgment is performed on the incremental delay result based on a preset delay threshold.

[0088] Optionally, the table lock request includes: a table name of a source basic library table.

[0089] Optionally, the distributed database 320 may be used for:

[0090] Determine the corresponding target library table location based on the table name of the source basic library table;

[0091] The library table state of the source basic library table corresponding to the target library table position is modified to an unavailable state, and a source lock library table is generated.

[0092] Optionally, the distributed database 320 may be specifically used to: perform consistency comparison on the table name of the to-be-cached library table based on the table name of the source lock library table, and generate a status evaluation result corresponding to the to-be-cached library table based on the consistency comparison result.

[0093] Optionally, the distributed database 320 can be specifically used to: perform status judgment on the status evaluation result, if the status evaluation result is an unavailable state, determine the transaction type corresponding to the business request, and generate a lock table identifier corresponding to the source lock library table based on the transaction type and a preset window period.

[0094] Optionally, the distributed database 320 may be used for:

[0095] Performing a type judgment on the transaction type, and if the transaction type is an operated type, determining the service execution duration corresponding to the service request;

[0096] A numerical judgment is made on the service execution duration based on a preset window period. If the service execution duration exceeds the preset window period, the service request is rolled back and a lock table identifier corresponding to the source lock library table is generated.

[0097] Optionally, the acquisition and playback component 310 may also be used to: after receiving the lock table identifier through the acquisition and playback component 310, performing the surplus data synchronization, and completing the library table redistribution of the source basic library table, generate an unlock request, and send the unlock request to the distributed database 320; wherein the unlock request includes the table name of the source lock library table;

[0098] The distributed database 320 is used to receive an unlock request and modify the library table state of the source lock library table to an available state.

[0099] Optionally, the distributed database 320 can also be used to: receive a library table redistribution task before performing a threshold judgment on the incremental delay result between the source basic library table and the target library table through the acquisition and playback component 310; and determine the corresponding source basic library table and target library table based on the library table redistribution task.

[0100] The library table redistribution system based on a distributed database provided in an embodiment of the present invention can execute the library table redistribution method based on a distributed database provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0101] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0102] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A library table redistribution method based on a distributed database, characterized in that: include: The incremental delay result between the source basic library table and the target library table is judged by the acquisition and playback component. If the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database. A lock table request is received through a distributed database, and a state of the source basic library table is modified based on the lock table request to generate a source lock library table; Sending a business request to the distributed database through a target business program; wherein the business request includes the table name of the to-be-cached library table corresponding to the business request; Receiving a business request through a distributed database, performing a status evaluation on the to-be-cached library table based on the source lock library table, generating a lock table identifier corresponding to the source lock library table based on the status evaluation result and a preset window period, and sending the lock table identifier to a collection and playback component; The lock table identifier is received by the acquisition and playback component, and the surplus data synchronization is performed to complete the library table redistribution of the source basic library table.

2. The method according to claim 1, characterized in that The step of performing threshold judgment on the incremental delay result between the source basic library table and the target library table by the acquisition and playback component includes: The incremental data in the source basic library table is obtained through the collection and playback component, and the collection time point is recorded; The incremental data is updated to the target library table through the acquisition and playback component, and the storage time point is recorded; The incremental delay result between the source basic library table and the target library table is determined by the acquisition and playback component based on the difference between the storage time point and the acquisition time point, and a threshold judgment is performed on the incremental delay result based on a preset delay threshold.

3. The method according to claim 1, characterized in that The table lock request includes: the table name of the source basic library table.

4. The method according to claim 3, characterized in that Modifying the state of the source basic library table based on the lock table request through a distributed database to generate a source lock library table includes: Determine the corresponding target database table location based on the table name of the source basic database table through the distributed database; The library table state of the source basic library table corresponding to the target library table position is modified to an unavailable state through a distributed database to generate a source lock library table.

5. The method according to claim 1, characterized in that The state of the to-be-cached library table is evaluated based on the source lock library table through a distributed database, including: A consistency comparison is performed on the table name of the to-be-cached library table based on the table name of the source lock library table through a distributed database, and a status evaluation result corresponding to the to-be-cached library table is generated based on the consistency comparison result.

6. The method according to claim 1, characterized in that Generate a lock table identifier corresponding to the source lock library table through a distributed database based on the status evaluation result and a preset window period, including: The status evaluation result is judged through a distributed database. If the status evaluation result is an unavailable state, the transaction type corresponding to the business request is determined, and a lock table identifier corresponding to the source lock library table is generated based on the transaction type and a preset window period.

7. The method according to claim 6, characterized in that Generating a lock table identifier corresponding to the source lock library table based on the transaction type and the preset window period through a distributed database includes: The transaction type is judged by a distributed database, and if the transaction type is an operated type, the service execution duration corresponding to the service request is determined; The distributed database performs a numerical judgment on the service execution duration based on a preset window period. If the service execution duration exceeds the preset window period, the service request is rolled back and a lock table identifier corresponding to the source lock library table is generated.

8. The method according to claim 1, characterized in that After receiving the lock table identifier through the acquisition and playback component, performing the surplus data synchronization, and completing the library table redistribution of the source basic library table, the method further includes: Generate an unlock request through the acquisition and playback component, and send the unlock request to the distributed database; wherein the unlock request includes the table name of the source lock library table; An unlock request is received through a distributed database, and a library table state of the source lock library table is modified to an available state.

9. The method according to claim 1, characterized in that: Before the threshold value judgment is performed on the incremental delay result between the source basic library table and the target library table by the acquisition and playback component, the method further includes: Receive library table redistribution tasks through distributed databases; The corresponding source basic library table and target library table are determined based on the library table redistribution task through the distributed database.

10. A library table redistribution system based on a distributed database, characterized in that: include: The acquisition and playback component is used to perform a threshold judgment on the incremental delay result between the source basic library table and the target library table. If the incremental delay result meets the preset delay threshold, a table lock request is generated and sent to the distributed database; A distributed database, used for receiving a table lock request, and modifying the state of the source basic library table based on the table lock request to generate a source lock library table; A target business program, used to send a business request to the distributed database; wherein the business request includes the table name of the to-be-cached library table corresponding to the business request; A distributed database, used for receiving a business request, performing a status evaluation on the to-be-cached library table based on the source lock library table, generating a lock table identifier corresponding to the source lock library table based on the status evaluation result and a preset window period, and sending the lock table identifier to a collection and playback component; The acquisition and playback component is used to receive the lock table identifier, perform surplus data synchronization, and complete the library table redistribution of the source basic library table.

Images