A data synchronization method and device of a shared storage cluster database

Abstract

The application relates to a data synchronization method and equipment of a shared storage cluster database. The method part mainly comprises the following steps: a source end database synchronization system initializes a log reading thread for each instance of the shared storage cluster; the log reading thread sends the read log to a to-be-sequenced queue of a log sequencing thread; the log sequencing thread sequences the log through a pre-set log dependency relationship, and adds the log to a to-be-analyzed queue of a log analyzing thread in sequence; the log analyzing thread extracts the log from the to-be-analyzed queue in sequence for analysis and processing, and adds the processed log to a to-be-delivered message queue of a log delivering thread; and the log delivering thread sends the to-be-delivered message to a target end for warehousing. The application can guarantee the correct order of transaction synchronization, efficiently sequence the log, and improve the overall data synchronization performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database data processing, in particular to a data synchronization method and device of a shared storage cluster database. BACKGROUND

[0002] The shared storage cluster database has multiple instances providing services, each instance sharing a storage data, and each instance maintaining a log system independently, such as the typical databases of Oracle RAC, DM8DSC, etc. Since the logs of each instance of the database are dispersed and the operated data is the same data, when synchronizing the data in real time based on log analysis, the logs of each instance need to be merged, and when merging, a certain algorithm needs to be used to sort the logs to ensure the order of transactions, and then the ordered logs are parsed and sent to the target database for storage, which is called the data synchronization process of the shared cluster database. In this process, the merging and sorting algorithm of each instance log is a difficulty. If the sorting is incorrect, the order of transaction synchronization will be disordered, resulting in data synchronization failure. At present, the common method is to sort according to the time stamp of each instance log, and sort according to the log SCN again for the same time stamp, or directly sort according to the SCN. The disadvantage of this method is that there is a situation of waiting for a time stamp to sort, and sometimes the log amount of a time stamp is large, so the overall sorting performance is also low, resulting in low data synchronization performance.

[0003] In view of this, how to overcome the defects of the prior art, how to solve the technical problem that the log amount of a time stamp is large, resulting in low overall sorting performance, and thus low data synchronization performance, has become an important technical problem to be solved in the industry. SUMMARY

[0004] In view of the above defects or improvement needs of the prior art, the present application provides a data synchronization method and device of a shared storage cluster database to solve the problem that the log amount of a time stamp is large, resulting in low overall sorting performance, and thus low data synchronization performance. The core of the data synchronization of the shared storage cluster database is how to merge and sort the logs, and the present application sorts the logs using a log dependency method and according to the dependency relationship. This dependency relationship is recorded in the logs of each instance of the shared cluster, and the content includes the dependent node number and the maximum LSN value. By comparing the current node LSN of the log with the maximum LSN of the dependent node, when the maximum LSN of all dependent nodes of the log is greater than the current node LSN of the log, the log is sent to the analysis queue of the log analysis thread. Sending the log to the analysis queue of the log analysis thread according to this dependency rule can ensure the correct order of transaction synchronization and efficiently sort the logs, thereby improving the overall data synchronization performance.

[0005] The embodiment of the present application adopts the technical scheme as follows:

[0006] In a first aspect, the present application provides a data synchronization method for a shared storage cluster database, comprising:

[0007] The source database synchronization system initializes a log reading thread for each instance of the shared storage cluster;

[0008] The log reading thread sends the read logs to a sorting queue of the log sorting thread, the log sorting thread sorts the logs according to the pre-set log dependency relationship, and adds the logs to a to-be-analyzed queue of the log analysis thread in sequence;

[0009] The log analysis thread extracts the logs from the to-be-analyzed queue in sequence for analysis and processing, and adds the processed logs to a to-be-delivered message queue of the log delivery thread, and the log delivery thread sends the to-be-delivered message to the target end for warehousing.

[0010] Further, the log reading thread sends the read logs to a sorting queue of the log sorting thread, the log sorting thread sorts the logs according to the pre-set log dependency relationship, and adds the logs to a to-be-analyzed queue of the log analysis thread in sequence, specifically comprising:

[0011] The log reading thread reads the logs from the corresponding instance node, adds the read logs to a log sorting queue of the log sorting thread, and sets the current node LSN;

[0012] The log sorting thread traverses the logs in the sorting queue in sequence, obtains the log dependency relationship of the traversed log, and the log dependency relationship includes an array formed by the node number and the maximum LSN of the dependent node on which the log depends;

[0013] Compare the current node LSN of the log with the maximum LSN of the dependent node, if the maximum LSN of all dependent nodes is greater than the current node LSN, then add the log to the to-be-analyzed queue of the log analysis thread.

[0014] Further, when obtaining the log dependency relationship of the traversed log, if the log does not exist the log dependency relationship, then directly add the log to the to-be-analyzed queue of the log analysis thread.

[0015] Further, when comparing the current node LSN of the log with the maximum LSN of the dependent node, if there is a dependent node whose maximum LSN is less than the current node LSN, then add the current log of the dependent node to the to-be-analyzed queue of the log analysis thread.

[0016] Further, when comparing the current node LSN of the log with the maximum LSN size of the dependent node, if there are multiple dependent nodes whose maximum LSN is less than the current node LSN, the current log of the dependent node with the minimum LSN is selected to be added to the analysis queue of the log analysis thread.

[0017] Further, when the log reading thread reads the log from the corresponding instance node, if no log is read, the loop continues to read until the log is read.

[0018] Further, when the log sorting thread traverses the logs in the sorting queue in sequence, the corresponding log basic information is also obtained, and the log basic information includes one or more of the node number of the current log, the minimum LSN and the maximum LSN of the current log, and the maximum LSN of the previous log.

[0019] Further, the method further comprises:

[0020] The source database and the destination database are both deployed with the synchronization system; wherein the source database synchronization system reads the log from the source shared storage cluster database, and the destination database synchronization system is responsible for applying the synchronization operation from the source to the destination database.

[0021] Further, the method further comprises:

[0022] After the destination database synchronization system receives the message delivered by the log delivery thread of the source database synchronization system, the message is unpacked, and then the message is stored through the interface of the destination database.

[0023] On the other hand, the application provides a data synchronization device of a shared storage cluster database, specifically comprising: at least one processor and a memory, the at least one processor and the memory are connected through a data bus, the memory stores instructions executable by the at least one processor, and the instructions are used to complete the data synchronization method of the shared storage cluster database in the first aspect after being executed by the processor.

[0024] Compared with the prior art, the application has the beneficial effects that: by setting the log dependency relationship, the array formed by the dependent node number and the maximum LSN of the dependent node of each log is obtained, so that the maximum LSN of the dependent node is compared with the size of the current node LSN, and only when the maximum LSN of all dependent nodes is greater than the current node LSN, the corresponding log is sent to the analysis queue of the log analysis thread. According to this dependency rule, the log is sent to the analysis queue of the log analysis thread, which can ensure the correct order of transaction synchronization and efficiently sort the log, thereby improving the overall data synchronization performance. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced. Obviously, the drawings described below are only some of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained from these drawings without creative labor.

[0026] Figure 1 A data synchronization method flow chart of a shared storage cluster database provided for the embodiment 1 of the present application;

[0027] Figure 2 A specific flow chart of the step 200 provided for the embodiment 1 of the present application;

[0028] Figure 3 A data synchronization schematic diagram of a two-node shared storage cluster provided for the embodiment 2 of the present application;

[0029] Figure 4 A flow chart of a log reading thread provided for the embodiment 2 of the present application;

[0030] Figure 5 A two-node sorting algorithm flow chart provided for the embodiment 2 of the present application;

[0031] Figure 6 A multi-node sorting algorithm flow chart provided for the embodiment 3 of the present application;

[0032] Figure 7 A data synchronization device structure schematic diagram of a shared storage cluster database provided for the embodiment 4 of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application, and the embodiments described here are part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application. In addition, the technical features of each embodiment or single embodiment provided by the present application can be combined with each other to form a feasible technical solution, but it should be considered that the combination of the technical solutions does not exist and is not within the scope of protection of the present application when the combination of the technical solutions appears contradictory or unachievable on the basis of the technical solution that can be realized by those skilled in the art.

[0034] The present application is a kind of architecture of specific function system, therefore in specific embodiment, the function logical relation of each structure module is mainly explained, and the specific software and hardware implementation is not limited.

[0035] The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0036] Embodiment 1:

[0037] As shown in the figure, the embodiment of the present application provides a data synchronization method of shared storage cluster database, and the specific steps are as follows. Figure 1

[0038] Step 100: The source database synchronization system initializes a log reading thread for each instance of the shared storage cluster.

[0039] Step 200: The log reading thread sends the read log to the sorting queue of the log sorting thread, and the log sorting thread sorts the log through the pre-set log dependency relationship, and adds the log to the analysis queue of the log analysis thread in order.

[0040] Step 300: The log analysis thread extracts the log from the analysis queue in order for analysis and processing, and adds it to the message queue of the log delivery thread after processing, and the log delivery thread sends the message to the destination for warehousing. The analysis and processing of the log analysis thread in this step include: extracting the log from the analysis queue in order, extracting the transaction information, operation type and data involved in the operation in the log, and formatting it into the message format to be synchronized, and adding it to the message queue to be delivered.

[0041] It should be noted that the embodiment also includes a preparation step: deploying the synchronization system in the source database and the destination database; wherein the source database synchronization system reads the log from the source shared storage cluster database, and the destination database synchronization system is responsible for applying the synchronization operation from the source to the destination database. The destination database synchronization system of the embodiment also unpacks the message after receiving the message delivered by the log delivery thread of the source database synchronization system, and then performs warehousing through the interface of the destination database.

[0042] ​Specifically, in step 100, the source database synchronization system initializes a log reading thread for each instance of the shared storage cluster, for example, two log reading threads are initialized for a two-node shared storage cluster, three log reading threads are initialized for a three-node shared storage cluster, and so on. In addition, the source database synchronization system also initializes other threads, including: a log sorting thread, a log analysis thread, and a log delivery thread. The log sorting thread sorts the logs read by each reading thread, and the sorted logs are added to the log analysis queue of the log analysis thread; the log analysis thread has a log analysis queue, and the log analysis thread is responsible for organizing and dispatching log operation restoration tasks in addition to screening and classifying operations and dictionary maintenance; the log delivery thread has a message queue, and is responsible for delivering the parsed operations to the destination.

[0043] As shown in the preferred embodiment, for step 200 (the log reading thread sends the read logs to the log sorting thread, the log sorting thread sorts the logs according to the pre-set log dependency relationship, and adds the logs in order to the log analysis thread), the following steps can be included. Figure 2

[0044] Step 201: The log reading thread reads the logs from the corresponding instance node, adds the read logs to the log sorting thread log queue, and sets the current node LSN; the current node LSN indicates the LSN of the logs read by the node, and each log records the LSN, which is the current LSN, which changes according to different logs read.

[0045] Step 202: The log sorting thread traverses the logs in the log sorting queue in order, and obtains the log dependency relationship according to the traversed logs, the log dependency relationship including an array formed by the node number and the maximum LSN of the dependent node; here, "dependent" means the logical order caused by concurrent conflicts for the same data page in the database implementation, which can represent the logical order of transactions. For example: node 0 updates the value of table T(c1) to 1, and node 1 updates the same row of T(c1) to 2, so there is a sequence, and the dependency relationship is represented as: node 1 depends on the update of node 0, which is represented in the log as: node 1 log (LSN1) depends on node 0 log (LSN0), and LSN0 is the maximum LSN that LSN1 depends on. If the read log LSNx on node 0 is smaller than LSN0, then it needs to wait until the maximum LSN0.

[0046] ​Step 203: Compare the current node LSN of the log with the maximum LSN size of the dependent nodes. If the maximum LSN of all dependent nodes is greater than the current node LSN, the log is added to the analysis queue of the log analysis thread. The dependent information obtained from the log can be represented as P(x, y), where P(x, y) is the log information of the dependent other nodes, x represents the node number of the dependency, and y represents the maximum LSN of the dependency. This dependency information P(x, y) represents a pair of dependency information, that is, there is more than one. The dependent nodes are traversed in turn until all dependent nodes satisfy the condition that the dependent LSN is greater than the node LSN, and then the log is added to the analysis queue of the log analysis thread.

[0047] In step 201 of the preferred embodiment, when the log reading thread reads the log from the corresponding instance node, if no log is read, the loop continues to read until a log is read.

[0048] In step 202 of the preferred embodiment, when obtaining the log dependency relationship according to the traversed log, if the log does not exist, the log is directly added to the analysis queue of the log analysis thread. No dependency relationship means that the current log does not depend on the log of other nodes, so the log can be directly added to the analysis queue.

[0049] In step 202 of the preferred embodiment, when the log sorting thread traverses the logs in the sorting queue in order, it also obtains the corresponding log basic information, which includes one or more of the node number of the current log, the minimum LSN and the maximum LSN of the current log, and the maximum LSN of the previous log.

[0050] In step 203 of the preferred embodiment, when comparing the current node LSN of the log with the maximum LSN size of the dependent nodes, if only one dependent node has a maximum LSN less than the current node LSN, the current log of the dependent node is added to the analysis queue of the log analysis thread. This is because when only one dependent node has a maximum LSN less than the current node LSN, it means that the maximum LSN of the dependent node is the earliest in time, so its current log is added to the analysis queue of the log analysis thread.

[0051] In step 203 of the preferred embodiment, when comparing the current node LSN of the log with the maximum LSN size of the dependent nodes, if there are multiple dependent nodes with maximum LSN less than the current node LSN, the current log of the dependent node with the smallest LSN is selected and added to the analysis queue of the log analysis thread.

[0052] The format of the log dependency relationship in the above process is: the number of dependent nodes, the array of node numbers and LSNs dependent on. Specifically: n[(EPNOi, PLSNi),...], n represents the number of dependent nodes, EPNOi represents the node number dependent on, and PLSNi represents the LSN dependent on.

[0053] For example, in a 2-node shared cluster environment, the node number of node 01 is 1, the log is log1, the node number of node 02 is 2, the log is log2, and the LSN of log1 dependent on log2 is 200. The dependency format of log1 is: 1[(2, 200)], 1 represents 1 dependent node, 2 represents the node with a dependent node number of 2, and 200 represents the LSN of the dependent node 2. If in a 3-node shared cluster environment, there are node 01, node 02, and node 03, the corresponding node numbers are 1, 2, and 3, the corresponding logs are log1, log2, and log3, and the corresponding log LSNs are 100, 200, and 300, then the dependency of log1 is 2[(2, 200), (3, 300)], which means that log1 depends on 2 other nodes, and depends on 200 of log2 of node 02 and 300 of log3 of node 03.

[0054] The three basic rules of the log dependency relationship are as follows:

[0055] (1) There is no self-dependence, i.e., there is no dependence on itself, for example, the LSN of log1 of node 01 is 100, the LSN of log2 is 200, and log1 depends on 200 of log2, which does not exist.

[0056] (2) There is no mutual dependence, for example, the current LSN of log1 of node 01 is 100, the current LSN of log2 of node 02 is 200, log1 depends on 200 of log2, and log2 depends on 100 of log1, which does not exist.

[0057] (3) There is no ring dependence, for example, the current LSN of log1 of node 01 is 100, the current LSN of log2 of node 02 is 200, and the current LSN of log3 of node 03 is 300, log1 depends on 200 of log2, log2 depends on 300 of log3, and log3 depends on 100 of log1, which does not exist.

[0058] In summary, the preferred embodiment sets a log dependency relationship, obtains an array formed by the dependent node number and the maximum LSN of the dependent node of each log, compares the maximum LSN of the dependent node with the size of the current node LSN, and only when the maximum LSN of all dependent nodes is greater than the current node LSN, the current node log is sent to the analysis queue of the log analysis thread. According to the log sent to the analysis queue of the log analysis thread according to the dependency rule, the order of transaction synchronization can be guaranteed to be correct, and the logs can be efficiently sorted, thereby improving the overall data synchronization performance.

[0059] Embodiment 2

[0060] Based on the data synchronization method of the shared storage cluster database provided in Embodiment 1, Embodiment 2 will be described in more detail through a specific application example.

[0061] As shown in Figure 3 , it is a data synchronization diagram of a two-node shared storage cluster of the embodiment, and the embodiment takes a two-node shared storage cluster as an example, but is not limited to the two-node shared storage cluster. As shown in Figure 3 , the log reading thread of the embodiment has two, respectively corresponding to two instance nodes, and the log sorting thread is located in the middle layer of the data synchronization software, which merges and sorts the instance logs obtained by the log reading thread, and then sends the sorted logs to the log analysis thread for analysis, and finally sends the analyzed logs to the log sending thread for sending.

[0062] As shown in Figure 4 , it is a flowchart of the log reading thread of the embodiment, and the log reading thread reads logs from the corresponding instance node. If no log is read, the loop continues to read. If a log is read, the log is added to the sorting queue of the log sorting thread, and the corresponding current node LSN is set.

[0063] As shown in Figure 5 , it is a flowchart of the sorting algorithm of the embodiment. Assuming that there are 2 nodes in the shared storage cluster, respectively ep01 and ep02, the node numbers are 1 and 2, the current logs are log1 and log2, the current log LSNs are CLSN1 and CLSN2, the dependencies of the current logs are P1 and P2, P1 is empty or P1 = 1 [(2, PSLN2)], P2 is empty or P2 = 1 [(1, PSLN1)], and the next log of node 02 is logi, corresponding to log LSN CLSNi and dependency Pi.

[0064] Referring to Figure 5 , the one-pass sorting algorithm starting from log1 is as follows:

[0065] Step 1. If P1 is empty, i.e. log1 has no dependency, proceed to step 2; if P1 is 1[(2, PLSN2)], proceed to step 3.

[0066] Step 2. Add log1 to the queue to be parsed, and proceed to step 8.

[0067] Step 3. Determine if PLSN2 is less than CLSN2. If yes, proceed to step 2; otherwise, proceed to step 4.

[0068] Step 4. If P2 is empty, that is, log2 has no dependency, proceed to step 6; if P2 is 1[(1, PLSN1)], proceed to step 5.

[0069] Step 5. Determine if PLSN1 is less than CSLN1. If yes, proceed to step 6; otherwise, there is a mutual dependency, which does not meet the dependency rule, and proceed to step 8.

[0070] Step 6. Add log2 to the queue to be parsed, and proceed to Step 7.

[0071] Step 7. Take the next log i of node 02, whose log LSN is CLSNi and whose dependency is Pi. Set log2 = logi, CLSN2 = CLSNi, P2 = Pi, and proceed to step 3.

[0072] Step 8. This sorting process is now complete.

[0073] After sorting log1, continue sorting log2 using the same sorting algorithm until the logs of both nodes are sorted.

[0074] Based on the above process, this embodiment sets log dependencies to obtain an array consisting of the node numbers that each log depends on and the maximum LSNs of the dependent nodes. The maximum LSNs of the dependent nodes are then compared with the current node's LSN. Only when the maximum LSNs of all dependent nodes are greater than the current node's LSN is the log corresponding to the current node sent to the log analysis thread's queue for analysis. Sending logs to the log analysis thread's queue according to this dependency rule ensures the correct order of transaction synchronization and efficiently sorts logs, thereby improving overall data synchronization performance.

[0075] Example 3:

[0076] like Figure 6 As shown, based on the data synchronization method of the shared storage cluster database provided in Embodiments 1 and 2, this Embodiment 3 provides a more detailed explanation of the sorting algorithm process with multiple dependent nodes.

[0077] If the shared cluster has 3 instance nodes, EP01, EP02, EP03, corresponding node numbers are 1, 2, 3, node logs are log1, log2, log3, corresponding log LSNs are CLSN1, CLSN2, CLSN3, corresponding dependencies are P1, P2, P3. The next logs corresponding to log2 and log3 are logi and logj, the next dependencies are Pi and Pj, the next log LSNs are CLSNi and CLSNj. P1 is empty or P1 = 2[(2, PLSN12), (3, PLSN13)], P2 is empty or P2 = 2[(1, PLSN21), (3, PLSN23)], P3 is empty or P3 = 2[(1, PLSN31), (2, PLSN32)].

[0078] Reference Figure 6 The one-pass sorting algorithm starting from log1 is as follows:

[0079] Step 1: Determine whether P1 is empty. If yes, go to step 2; otherwise, go to step 3.

[0080] Step 2: Add log1 to the to-be-resolved queue, and go to step 25.

[0081] Step 3: Determine whether PLSN12 is less than CLSN2. If yes, go to step 4; otherwise, go to step 5.

[0082] Step 4: Determine whether PLSN13 is less than CLSN3. If yes, go to step 2; otherwise, go to step 6.

[0083] Step 5: Determine whether P2 is empty. If yes, go to step 7; otherwise, go to step 11.

[0084] Step 6: Determine whether P3 is empty. If yes, go to step 8; otherwise, go to step 12.

[0085] Step 7: Add log2 to the to-be-resolved queue, and go to step 9.

[0086] Step 8: Add log3 to the to-be-resolved queue, and go to step 10.

[0087] Step 9: Take the next log logi of node 02, whose log LSN is CLSNi and whose dependency is Pi, set log2 = logi, CLSN2 = CLSNi, P2 = Pi, and go to step 3.

[0088] Step 10: Take the next log of node 03, logj, whose log LSN is CLSNj and whose dependency is Pj, set log3 = logj, CLSN3 = CLSNj, P3 = Pj, go to step 4.

[0089] Step 11: Judge whether PLSN21 is less than CLSN1, if yes, go to step 13; otherwise, there is mutual dependency, which does not conform to the rule, go to step 25.

[0090] Step 12: Judge whether PLSN31 is less than CLSN1, if yes, go to step 14; otherwise, there is ring dependency, which does not conform to the rule, go to step 25.

[0091] Step 13: Judge whether PLSN23 is less than CLSN3, if yes, go to step 7; otherwise, go to step 15.

[0092] Step 14: Judge whether PLSN32 is less than CLSN2, if yes, go to step 8; otherwise, go to step 16.

[0093] Step 15: Judge whether P3 is empty, if yes, go to step 21; otherwise, go to step 17.

[0094] Step 16: Judge whether P2 is empty, if yes, go to step 22; otherwise, go to step 18.

[0095] Step 17: Judge whether PLSN31 is less than CLSN1, if yes, go to step 19; otherwise, there is ring dependency, which does not conform to the rule, go to step 25.

[0096] Step 18: Judge whether PLSN21 is less than CLSN1, if yes, go to step 20; otherwise, there is ring dependency, which does not conform to the rule, go to step 25.

[0097] Step 19: Judge whether PLSN32 is less than CLSN2, if yes, go to step 21; otherwise, there is ring dependency, which does not conform to the rule, go to step 25.

[0098] Step 20: Judge whether PLSN23 is less than CLSN3, if yes, go to step 22; otherwise, there is ring dependency, which does not conform to the rule, go to step 25.

[0099] Step 21: Add log3 to the to-be-resolved queue, go to step 23.

[0100] Step 22: Add log2 to the to-be-resolved queue, go to step 24.

[0101] Step 23: taking the next log logj of node 03, whose log LSN is CLSNj and whose dependency is Pj, setting log3 = logj, CLSN3 = CLSNj and P3 = Pj, and entering step 13.

[0102] Step 24: taking the next log logi of node 02, whose log LSN is CLSNi and whose dependency is Pi, setting log2 = logi, CLSN2 = CLSNi and P2 = Pi, and entering step 14.

[0103] Step 25: the current sorting is completed.

[0104] The log1 completes sorting, and the log2 and log3 are sorted according to the same algorithm, until the logs of the three nodes are all sorted.

[0105] Embodiment 4

[0106] Based on the data synchronization method of the shared storage cluster database provided in Embodiments 1 to 3, the application further provides a data synchronization device of a shared storage cluster database, which can be used to implement the above method, as shown in Figure 7 The device architecture of the embodiment of the application is shown in the figure. The data synchronization device of the shared storage cluster database of the embodiment comprises one or more processors 21 and a memory 22. In the embodiment, the processor 21 is taken as an example. Figure 7

[0107] The processor 21 and the memory 22 can be connected through a bus or other means. Figure 7 In the embodiment, the connection through the bus is taken as an example.

[0108] The memory 22 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the data synchronization method of the shared storage cluster database in Embodiments 1 to 3. The processor 21 executes the non-volatile software programs, instructions and modules stored in the memory 22, thereby performing various functions and data processing of the data synchronization device of the shared storage cluster database, i.e. implementing the data synchronization method of the shared storage cluster database in Embodiments 1 to 3.

[0109] The memory 22 can comprise a high-speed random access memory, and can also comprise a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device or other non-volatile solid-state storage device. In some embodiments, the memory 22 can optionally comprise a memory remotely arranged relative to the processor 21, and these remote memories can be connected to the processor 21 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network and a combination thereof.​

[0110] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, perform the data synchronization method of the shared storage cluster database in the above embodiments 1 to 3, for example, perform the above-described various steps. Figure 1 、 Figure 5 、 Figure 6 as shown.

[0111] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing the relevant hardware by a program, and the program can be stored in a computer readable storage medium, which can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0112] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data synchronization method of a shared storage cluster database, characterized by, The method comprises the following steps: The source database synchronization system initializes a log reading thread for each instance of the shared storage cluster; The log reading thread sends the read log to the log sorting thread, and the log sorting thread sorts the log according to the pre-set log dependency relationship and adds the log to the log analysis thread in sequence; including: the log reading thread reads the log from the corresponding instance node, adds the read log to the log sorting thread, and sets the current node LSN; the log sorting thread traverses the log in the sorting queue in sequence, obtains the log dependency relationship according to the traversed log, and the log dependency relationship includes an array formed by the node number and the maximum LSN of the dependent node; compare the current node LSN of the log with the maximum LSN of the dependent node, if the maximum LSN of all dependent nodes is greater than the current node LSN, add the log to the log analysis thread; The log analysis thread extracts the log from the analysis queue in sequence for analysis and processing, and adds the processed log to the log delivery thread after processing.

2. The data synchronization method of the shared storage cluster database according to claim 1, characterized in that, When the log dependency relationship of the traversed log is obtained, if the log does not exist, the log is directly added to the log analysis thread.

3. The data synchronization method of the shared storage cluster database according to claim 1, characterized in that, When comparing the current node LSN of the log with the maximum LSN of the dependent node, if the maximum LSN of the dependent node is less than the current node LSN, the current log of the dependent node is added to the log analysis thread.

4. The data synchronization method of the shared storage cluster database according to claim 1, characterized in that, When comparing the current node LSN of the log with the maximum LSN of the dependent node, if the maximum LSN of the dependent node is less than the current node LSN, the current log of the dependent node is added to the log analysis thread.

5. The data synchronization method of the shared storage cluster database according to claim 1, wherein, When comparing the current node LSN of the log with the maximum LSN of the dependent node, if the maximum LSN of the dependent node is less than the current node LSN, the current log of the dependent node is added to the log analysis thread.

6. The data synchronization method of the shared storage cluster database according to claim 1, wherein, When the log reading thread reads the log from the corresponding instance node, if no log is read, the reading continues until a log is read.

7. The data synchronization method of the shared storage cluster database according to any one of claims 1-6, characterized in that, When the log sorting thread traverses the log in the sorting queue in sequence, the corresponding log basic information is also obtained, and the log basic information includes one or more of the node number of the current log, the minimum LSN and the maximum LSN of the current log, and the maximum LSN of the previous log. The method further comprises:

8. The data synchronization method of the shared storage cluster database according to any one of claims 1-6, characterized in that, Deploying a synchronization system on the source database and the target database; wherein the source database synchronization system reads the log from the source shared storage cluster database, and the target database synchronization system is responsible for applying the synchronization operation from the source to the target database. The method further comprises: After the target database synchronization system receives the message delivered by the log delivery thread of the source database synchronization system, the message is unpacked, and then the interface of the target database is used for database storage.

9. A data synchronization device for a shared storage cluster database, characterized in that: The shared storage cluster database data synchronization method comprises at least one processor and a memory, the at least one processor and the memory are connected through a data bus, the memory stores instructions executable by the at least one processor, and the instructions are used for completing the shared storage cluster database data synchronization method in any one of claims 1-8 after being executed by the processor.

Images