Data synchronization method and accounting service cluster

Abstract

The invention provides a data synchronization method and an accounting service cluster, and relates to the technical field of computers. The accounting service cluster comprises a master node, a plurality of slave nodes and a synchronization component in communication connection with each node; the main node is in communication connection with a file system, and accounting data of a plurality of accounts are stored in a memory of each node. When receiving the accounting request, the main node generates a modification proposal based on the accounting request and allocates a unique identification number to the modification proposal; the main node adds a write-in operation log to the file system based on the modification proposal and the identification number; the master node controls each slave node to execute a modification proposal through a synchronization component; and when receiving a confirmation message returned by the synchronization component, the main node updates the accounting data of the specified account in the memory of the main node based on the modification proposal. According to the method, a data storage mode of a pure memory is adopted, request processing is faster, the throughput is higher, and the performance under a high-concurrency scene can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a data synchronization method and an accounting service cluster. Background Art

[0002] In fintech, e-commerce, and distributed service systems, the consistency of account accounting data is a core requirement for ensuring transaction reliability and system security. Account operations (such as balance deduction, fund entry, cross-account transfer, etc.) usually need to ensure the atomicity and consistency of data through a transaction mechanism to avoid problems such as abnormal account balances, fund losses, or duplicate deductions caused by partial operation failures or concurrent conflicts. In the prior art, the consistency of account accounting data is mostly ensured by the following two methods: (1) Transaction atomicity solution based on a structured database: Traditional relational databases (such as MySQL, PostgreSQL) achieve strong consistency of account operations through the ACID (a general term for the four characteristics of atomicity, consistency, isolation, and durability) transaction model. In this solution, the deduction and entry operations of an account are encapsulated in the same database transaction, and rely on row-level locks or optimistic lock mechanisms to ensure the correctness of concurrent operations. However, in a high-concurrency scenario, this solution will cause lock competition for operating accounts (that is, the increase / decrease of the account balance needs to be queued for operation); (2) Cross-system consistency solution based on distributed transactions: To meet the requirements of sharding and sub-tabling of the account system or cross-service calls under the microservices architecture, the prior art uses a distributed transaction framework (such as two-phase commit 2PC, TCC flexible transactions, Saga mode) to coordinate multi-node operations. Although such solutions can achieve data consistency across databases or services, distributed transactions need to go through multiple stages of coordination (such as Prepare / Commit), network communication, and log persistence, and the overall latency is much higher than that of local transactions, making it difficult to meet the business requirements of millisecond-level responses.

[0003] Therefore, the above two solutions of the prior art both have poor performance in the face of high-concurrency scenarios. Summary of the Invention

[0004] The purpose of the present invention is to provide a data synchronization method and an accounting service cluster to improve the problems existing in the prior art.

[0005] The embodiments of the present invention can be implemented as follows: In a first aspect, the present invention provides a data synchronization method, which is applied to an accounting service cluster. The accounting service cluster includes a master node, multiple slave nodes, and a synchronization component communicatively connected to each node; the master node is communicatively connected to a file system, and the accounting data of multiple accounts is stored in the memory of each node; the method includes: When the master node receives an accounting request, it generates a modification proposal based on the accounting request and assigns a unique identification number to the modification proposal; The master node appends an operation log to the file system based on the modification proposal and the identification number; The master node controls each slave node to execute the modification proposal through the synchronization component; When the master node receives the confirmation message returned by the synchronization component, it updates the accounting data of the specified account in its own memory based on the modification proposal, where the confirmation message received by the master node is generated and sent by the synchronization component after confirming that each slave node has completed the execution of the modification proposal.

[0006] In an alternative embodiment, the step of the master node controlling each slave node to execute the modification proposal through the synchronization component includes: The master node notifies each slave node of the modification proposal and its corresponding identification number through the synchronization component; Each slave node updates the accounting data of the specified account in its own memory based on the modification proposal, and returns a confirmation message to the synchronization component when the update is completed; The synchronization component returns a confirmation message to the master node when it receives confirmation messages from more than half of the slave nodes.

[0007] In an alternative embodiment, the accounting service cluster further includes a transaction component, and the method further includes: The master node sends a transaction message to the transaction component based on the modification proposal; The transaction component records the transaction message into the corresponding consumer queue.

[0008] In an alternative embodiment, the file system includes a log file and a snapshot file, and the method further includes: The master node periodically generates a snapshot based on the identification number corresponding to the most recently submitted modification proposal to the synchronization component and the accounting data of all accounts in its own memory, and saves the generated snapshot to the snapshot file.

[0009] In an alternative embodiment, the method further includes: After each time the master node saves the snapshot, it deletes all operation logs before a preset duration from the log file; and / or, the master node deletes other snapshots except the newly generated snapshot from the snapshot file.

[0010] In an alternative embodiment, the method further includes: After the new node synchronizes the data in its own memory based on the snapshot file and the log file, it joins the accounting service cluster and starts listening to the synchronization component.

[0011] In an alternative embodiment, the step of the new node synchronizing the data in its own memory based on the snapshot file and the log file includes: The new node obtains the latest snapshot from the snapshot file and loads the accounting data of all accounts in the latest snapshot into its own memory; The new node uses the identification number in the latest snapshot as the first reference identification number; The new node obtains each first operation log after the first reference identification number from the log file; The new node sequentially updates the accounting data of each account in its own memory based on each first operation log.

[0012] In an alternative embodiment, the modification proposal is used to indicate that the accounting data of a specified account is modified from the original value to the target value; After the step that the new node synchronizes the data in its own memory based on the snapshot file and the log file, joins the accounting service cluster and starts listening to the synchronization component, the method further includes: The new node receives the new modification proposal announced by the synchronization component and its corresponding identification number; When the new node confirms that the original value in the new modification proposal is inconsistent with the accounting data of the specified account in the memory, it uses the identification number corresponding to the new modification proposal as the second reference identification number; The new node obtains each second operation log after the second reference identification number from the log file; The new node sequentially updates the accounting data of each account in its own memory based on each second operation log; The new node updates the accounting data of the specified account in its own memory based on the new modification proposal.

[0013] In an alternative embodiment, the operation log includes the account ID of the specified account, the original value before the accounting data is modified, the target value after the accounting data is modified, and the identification number; the method further includes: The new primary node obtains the latest operation log from the log file and determines whether the original value in the latest operation log is consistent with the accounting data of the specified account in its own memory; the new primary node is elected from each slave node after the primary node fails. If they are consistent, the new primary node directly starts providing services externally; If they are inconsistent, the new master node updates the accounting data of the specified account in its own memory based on the latest operation log and then starts to provide services externally.

[0014] In a second aspect, the present invention provides an accounting service cluster, which includes a master node, a plurality of slave nodes, and a synchronization component communicatively connected to each node; the master node is communicatively connected to a file system, and the accounting data of multiple accounts are stored in the memory of each node; The master node is configured to, when receiving an accounting request, generate a modification proposal based on the accounting request and assign a unique identification number to the modification proposal; The master node is configured to append and write an operation log to the file system based on the modification proposal and the identification number; The master node is configured to control each slave node to execute the modification proposal through the synchronization component; The master node is configured to, when receiving a confirmation message returned by the synchronization component, update the accounting data of the specified account in its own memory, where the confirmation message received by the master node is generated and sent by the synchronization component after confirming that each slave node has completed the execution of the modification proposal.

[0015] Compared with the prior art, the embodiments of the present invention provide a data synchronization method and an accounting service cluster. The accounting service cluster includes a master node and a plurality of slave nodes; the master node is communicatively connected to a file system, each node is communicatively connected to a synchronization component, and the accounting data of multiple accounts are stored in the memory of each node. When the master node receives an accounting request, it generates a modification proposal based on the accounting request and assigns a unique identification number to the modification proposal; the master node appends and writes an operation log to the file system based on the modification proposal and the identification number; the master node controls each slave node to execute the modification proposal through the synchronization component; the master node updates the accounting data of the specified account in its own memory based on the modification proposal when receiving a confirmation message returned by the synchronization component. The present invention adopts a pure in-memory data storage method, with faster request processing and higher throughput, and can ensure performance in high-concurrency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a schematic structural diagram of an accounting service cluster provided by an embodiment of the present invention.

[0018] Figure 2 The flowchart of a data synchronization method provided by an embodiment of the present invention.

[0019] Figure 3 An example diagram of the processing flow of a financial service cluster in response to a financial request provided by an embodiment of the present invention.

[0020] Figure 4 An example diagram of a time reference provided by an embodiment of the present invention.

[0021] Figure 5 The flowchart of synchronizing financial information before a new node joins the cluster provided by an embodiment of the present invention. Detailed implementation manners

[0022] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0024] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0025] It should be noted that, without conflict, the features in the embodiments of the present invention can be combined with each other.

[0026] Herein, the application scenario of this solution will be introduced first.

[0027] Please refer to Figure 1 , Figure 1 a schematic diagram of a financial service cluster provided by the present invention. The financial service cluster includes a master node and multiple slave nodes ( Figure 1 only nodes A, B, and C are shown in

[0028] In the accounting service cluster, the memory of each node can be used to store accounting information, which includes accounting data of multiple accounts, and the accounting information in the memory of each node needs to be kept consistent. Among them, only the master node is responsible for providing accounting services to the external business system, such as registration, cancellation, query, modification, etc. This application mainly describes the process of modifying accounting data related to the modification function.

[0029] In different application scenarios, accounting data is different concepts. For example, in the financial transaction scenario, accounting data can be the account balance; in the e-commerce sales scenario, accounting data can be the product inventory or store sales of merchants, etc. This example is only for illustration, and the specific meaning of the accounting data in the present invention is not limited.

[0030] In the present invention, the file system includes a log file and a snapshot file maintained by the master node.

[0031] The synchronization component can be ZooKeeper (a distributed application coordination service). In the present invention, the synchronization component mainly maintains the consistency of the accounting information in the memory of each node. In addition, the synchronization component is also responsible for Leader election (i.e., master node election). When the master node fails, the synchronization component will elect a new master node from each slave node. After the master node or the slave node fails, only a new slave node needs to be pulled up. This part of the function is a mature high-availability solution and is not the focus of this solution.

[0032] Please refer to Figure 2 , Figure 2 which is a schematic flow diagram of a data synchronization method provided by an embodiment of the present invention. The execution subject of this method can be the above-mentioned accounting service cluster, and this method includes the following steps: S101. When the master node receives an accounting request, it generates a modification proposal based on the accounting request and assigns a unique identification number to the modification proposal.

[0033] In this embodiment, the accounting request can be to modify (such as increase or decrease) the accounting data of any specified account. The modification proposal is the modification proposal, and the identification number is the ZXID, which is used to uniquely identify the modification proposal. The modification proposal can be used to indicate modifying the accounting data of the specified account from the original value to the target value.

[0034] S102. The master node appends an operation log to the file system based on the modification proposal and the identification number.

[0035] In this embodiment, the master node generates an operation log based on the modification proposal and the identification number, and writes this operation log to the log file in the file system in an append-write manner.

[0036] Optionally, the operation log may include the account ID of the specified account, the original value of the accounting data before modification, the target value of the accounting data after modification, and the identification number.

[0037] By maintaining the log file, the master node ensures that even if the master node fails or shuts down, the system can still recover the unfinished operations through the log file. The log file uses the append-write method to ensure the continuity of the log, which is convenient for subsequent access and search.

[0038] S103. The master node controls each slave node to execute the modification proposal through the synchronization component.

[0039] In this embodiment, the master node enables each slave node to update the accounting data in its respective memory based on the modification proposal through the synchronization component.

[0040] S104. When the master node receives the confirmation message returned by the synchronization component, it updates the accounting data of the specified account in its own memory based on the modification proposal.

[0041] In this embodiment, when the master node receives the confirmation message returned by the synchronization component, it indicates that the update of each slave node is completed. At this time, the master node can update the accounting data of the specified account in its own memory based on the modification proposal, thus completing the synchronous update of the accounting information in the memory of each node in the cluster.

[0042] For the data synchronization method provided by the present invention, when the master node receives an accounting request, it generates a modification proposal based on the accounting request and assigns a unique identification number to the modification proposal; the master node appends and writes an operation log to the file system based on the modification proposal and the identification number; the master node controls each slave node to execute the modification proposal through the synchronization component; when the master node receives the confirmation message returned by the synchronization component, it updates the accounting data of the specified account in its own memory based on the modification proposal. The present invention adopts a pure in-memory data storage method, with faster request processing and higher throughput, and can ensure the performance in high-concurrency scenarios.

[0043] In an optional implementation manner, the sub-steps of the above step S103 may include S1031 to S1033: S1031. The master node notifies each slave node of the modification proposal and its corresponding identification number through the synchronization component; S1032. Each slave node updates the accounting data of the specified account in its own memory based on the modification proposal, and returns a confirmation message to the synchronization component when the update is completed; S1033. When the synchronization component receives the confirmation messages from more than half of the slave nodes, it returns a confirmation message to the master node.

[0044] In this embodiment, the synchronization component adopts a majority write mechanism, that is: when more than half of the secondary slave nodes return an acknowledgment message (ACK message) to the synchronization component, the synchronization component considers that all slave nodes have completed the execution of the modification proposal and can send an acknowledgment message to the master node. Alternatively, the synchronization component can also consider that all slave nodes have completed the execution of the modification proposal and return an acknowledgment message to the master node only after receiving the acknowledgment messages from all slave nodes.

[0045] If the master node does not receive an acknowledgment message returned by the synchronization component within a certain period of time (for example, 0.5s), it may be a network failure or the synchronization component is unstable. The master node needs to keep retrying until it finally receives the acknowledgment message returned by the synchronization component.

[0046] Taking Figure 1 the three nodes in Figure 3 as an example, assuming the accounting request is: "The original balance of the specified account acct1 is 1000, and the request is to increase the balance by 300". Combining it can be known that after the master node responds to this accounting request, the processing process of the entire cluster is as follows: Figure 3 1. The master node A generates a modification proposal ( Figure 3 abbreviated as proposal in ) based on the accounting request and assigns a unique identification number ( zxid1 in Figure 3 ) to the modification proposal; 2. The master node A appends a write operation log to the file system based on the modification proposal and the identification number, and this operation log can be expressed as (acct1, 1000, 1300, zxid1);

[0047] This example is only for illustration and is not limited here.

[0048] Through the synchronization component of the present invention, the modification synchronization of the accounting information in the memory of each node is realized, ensuring the consistency of the accounting information in the memory of each node in the cluster within millisecond delay.

[0049] In an optional implementation manner, in order to facilitate tracing the modification records of the account, the master node can also record the modification log through the transaction component in the accounting service cluster. That is, this method can also include: S105. The master node sends a transaction message to the transaction component based on the modification proposal; S106. The transaction component records the transaction message to the corresponding consumer queue.

[0050] Optionally, the streaming component can be a Kafka component, and the consumer queue can be used to store the modification streams of each account.

[0051] Since the accounting information is stored in the memory of each node, once each node crashes, all the memory data of that node will be lost. To ensure fault recovery after a node crash, the master node can also periodically take snapshots of the accounting data in the memory to ensure that the new node / new master node can load the accounting data into the memory based on the snapshot file and the log file. That is, the method can further include the following step S107: S107. The master node periodically generates a snapshot based on the identification number corresponding to the most recently submitted modification proposal to the synchronization component and the accounting data of all accounts in its own memory, and saves the generated snapshot to a snapshot file.

[0052] In this embodiment, the generated snapshot is in binary form, including the identification number (i.e., zxid) corresponding to the most recently submitted modification proposal by the master node to the synchronization component and the accounting data of each account in the master node's memory.

[0053] For example, assume that the identification number of the most recent modification proposal is 10011, and there are a total of 100 accounts' accounting data (assuming the accounting data is the balance) in the master node's memory. Then the snapshot generated by the master node includes the following content: {zxid:10011, acct1:1300, acct2:4300.66, ... Acct100:1284} This example is only for illustration, and the present invention does not limit the number of accounts in the account information and the format of the zxid.

[0054] In the snapshot file, actually only the latest snapshot is useful, while the historical snapshots are redundant data, and the operation logs with too long existence time in the log file are also redundant data. Therefore, the master node can clean the log file and / or the snapshot file to free up storage space. That is, after the master node saves a snapshot each time, the method can further include S108: S108. Delete all operation logs in the log file before a preset duration; and / or, the master node deletes other snapshots in the snapshot file except the newly generated snapshot.

[0055] Optionally, the preset duration can be 1h, 6h, one day, one week, etc., and the size of the preset duration can be flexibly set based on the concurrency of accounting requests. After the master node saves a snapshot each time, it deletes the previous snapshot in the snapshot file, so as to ensure that there is only one latest snapshot in the snapshot file, which can save storage space.

[0056] If a node in the cluster fails, the high-availability module will subsequently restart a new node. After the accounting information of the new node is synchronized, it will join the cluster. Therefore, this method further includes: S110. After the new node synchronizes the data in its own memory based on the snapshot file and the log file, it joins the accounting service cluster and starts listening to the synchronization component.

[0057] In this embodiment, the new node obtains the account information based on the snapshot file and the log file, and loads the account information into its own memory.

[0058] Optionally, in step S110, the process of the new node synchronizing the data in its own memory based on the snapshot file and the log file may include the following sub-steps S1101 to S1104: S1101. The new node obtains the latest snapshot from the snapshot file, and loads the accounting data of all accounts in the latest snapshot into its own memory; S1102. The new node uses the identification number in the latest snapshot as the first reference identification number; S1103. The new node obtains each first operation log after the first reference identification number from the log file; S1104. The new node sequentially updates the accounting data of each account in its own memory based on each first operation log.

[0059] In this embodiment, combined with Figure 4 , it is assumed that the master node saves a snapshot at regular intervals of T (T can be 10 minutes or 20 minutes, etc.). The latest snapshot in the snapshot file is saved at time t2 (t6 is the future time for the next snapshot save), and the new node obtains the latest snapshot at time t3. The new node loads all the accounting information in the latest snapshot into the memory during the period from t3 to t4.

[0060] However, during the period from t2 to t4, the master node may very likely respond to multiple accounting requests, resulting in a discrepancy between the accounting information consistently maintained by each node in the cluster at time t4 and that in the latest snapshot. Therefore, the new node needs to obtain each operation log during the period from t2 to t4 from the log file to update the accounting information loaded into the memory. Assuming that the update is completed at time t5 (in fact, the interval between t4 and t5 is extremely short, only for illustration in the figure), then the new node can join the cluster at time t5, thus ensuring the consistency of the accounting information in the memory between the new node and other nodes in the cluster.

[0061] In an optional example, on the basis of Figure 4 , please combine Figure 5, assuming that at time t4, the identification number in the latest operation log in the log file is zxid20, the process of the new node implementing the accounting information based on the snapshot file and the log file is as follows Figure 4 shown.

[0062] Figure 5 In the figure, the new node reads 5 operation logs after zxid15 (5 operation logs associated with zxid16~zxid20). The new node will update the accounting information in the memory in sequence according to these operation logs. For example, if the operation log corresponding to zxid16 is (acct9, 500, 190, zxid16), then the new node needs to first update the accounting data of account acct9 from 500 to 190. Then, if the operation log corresponding to zxid17 is (acct3, 1425, 1600, zxid17), then the new node needs to first update the accounting data of account acct3 from 1425 to 1600, and so on. This example is only for illustration and is not limited here.

[0063] It should be noted that the above combination Figure 4 , Figure 5 The examples are only for illustration and are not limited here.

[0064] In a possible situation, combined with Figure 4 , during the time period from t4 to t5, the master node may also respond to the accounting request and record the operation log. During this time period, since the new node has not yet joined the cluster, it cannot receive the notification from the synchronization component. The new node needs to synchronize the changes in the accounting information caused by the master node's corresponding accounting requests during the time period from t4 to t5 to its own memory after joining the cluster.

[0065] Therefore, after the new node joins the cluster, the method may further include S111~S115.

[0066] S111. The new node receives the newly modified proposal notified by the synchronization component and its corresponding identification number; S112. When the new node confirms that the original value in the newly modified proposal is inconsistent with the accounting data of the specified account in the memory, it uses the identification number corresponding to the newly modified proposal as the second reference identification number; S113. The new node obtains each second operation log in the log file that is after the second reference identification number; S114. The new node sequentially updates the accounting data of each account in its own memory based on each second operation log; S115. The new node updates the accounting data of the specified account in its own memory based on the newly modified proposal.

[0067] After a new node joins the cluster, the first few modification proposals received by the new node may all execute the above steps S112 - S114 until the accounting information of the new node is completely synchronized with that of the master node.

[0068] In a possible scenario, after the master node crashes, the entire cluster stops responding. A new master node needs to be elected from each slave node. Before crashing, the original master node may respond to an accounting request A, but the response process is not completed (i.e., just wrote the operation log A or sent a proposal to the synchronization component and then the original master node crashed, and the new master node did not receive the proposal when it was a slave node). The new master node needs to first synchronize the biang of the operation log A to its own memory and then start responding externally.

[0069] The process of the new master node synchronization includes the following steps: S201: The new master node obtains the latest operation log from the log file and determines whether the original value in the latest operation log is consistent with the accounting data of the specified account in its own memory; S202: If they are consistent, the new master node directly starts providing services externally; S203: If they are not consistent, the new master node updates the accounting data of the specified account in its own memory based on the latest operation log and then starts providing services externally.

[0070] It should be noted that the execution order of each step in the above method embodiments is not limited by the step numbers, and the execution order of each step is subject to the actual application situation.

[0071] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention adopts a pure - memory data storage method, with faster request processing and higher throughput, and can ensure performance in high - concurrency scenarios; In the present invention, through the snapshot file and the log file, it is ensured that the accounting information can be synchronized before a new node joins, and it also ensures the synchronization of the accounting information of the elected new master node; enabling the accounting information consistency without locking, avoiding lock contention caused by the unique index of the relational database.

[0072] The embodiments of the present invention also provide an accounting service cluster, which includes a master node, multiple slave nodes, and a synchronization component communicatively connected to each node; the master node is communicatively connected to the file system, and the accounting data of multiple accounts are stored in the memory of each node; wherein: The master node is used to generate a modification proposal based on the accounting request and assign a unique identification number to the modification proposal when receiving an accounting request; The master node is used to append and write the operation log to the file system based on the modification proposal and the identification number; The master node is used to control each slave node to execute the modification proposal through the synchronization component; The master node is used to update the accounting data of the specified account in its own memory based on the modification proposal when receiving the confirmation message returned by the synchronization component.

[0073] In this accounting service cluster, the process principle of each part cooperating to maintain the consistency of accounting information storage is as described above and will not be elaborated here.

[0074] In summary, the embodiment of the present invention provides a data synchronization method and an accounting service cluster. The accounting service cluster includes a master node, multiple slave nodes, and a synchronization component communicatively connected to each node; the master node is communicatively connected to the file system, and the accounting data of multiple accounts is stored in the memory of each node. When receiving an accounting request, the master node generates a modification proposal based on the accounting request and assigns a unique identification number to the modification proposal; the master node appends an operation log to the file system based on the modification proposal and the identification number; the master node controls each slave node to execute the modification proposal through the synchronization component; when receiving the confirmation message returned by the synchronization component, the master node updates the accounting data of the specified account in its own memory. The present invention adopts a pure memory data storage method, with faster request processing and higher throughput, and can ensure the performance in high-concurrency scenarios.

[0075] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any change or replacement that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A data synchronization method, characterized in that, Applied to an accounting service cluster, the accounting service cluster includes a master node, multiple slave nodes, and a synchronization component communicatively connected to each node; The master node is communicatively connected to a file system, and accounting data of multiple accounts is stored in the memory of each node; The method includes: When receiving an accounting request, the master node generates a modification proposal based on the accounting request and assigns a unique identification number to the modification proposal; The master node appends an operation log to the file system based on the modification proposal and the identification number; The master node controls each slave node to execute the modification proposal through the synchronization component; When receiving a confirmation message returned by the synchronization component, the master node updates the accounting data of a specified account in its own memory based on the modification proposal, where the confirmation message received by the master node is generated and sent by the synchronization component after confirming that each slave node has completed executing the modification proposal.

2. The data synchronization method according to claim 1, wherein The step that the master node controls each slave node to execute the modification proposal through the synchronization component includes: The master node notifies each slave node of the modification proposal and its corresponding identification number through the synchronization component; Each slave node updates the accounting data of a specified account in its own memory based on the modification proposal, and returns a confirmation message to the synchronization component when the update is completed; When receiving confirmation messages from more than half of the slave nodes, the synchronization component returns a confirmation message to the master node.

3. The data synchronization method according to claim 1, wherein The accounting service cluster further includes a transaction component, and the method further includes: The master node sends a transaction message to the transaction component based on the modification proposal; The transaction component records the transaction message to a corresponding consumer queue.

4. The data synchronization method according to any one of claims 1 to 3, characterized in that The file system includes a log file and a snapshot file, and the method further includes: The master node periodically generates a snapshot based on the identification number corresponding to the most recently submitted modification proposal to the synchronization component and the accounting data of all accounts in its own memory, and saves the generated snapshot to the snapshot file.

5. The data synchronization method according to claim 4, wherein The method further includes: After each saving of the snapshot, the master node deletes all operation logs before a preset duration from the log file; and / or, the master node deletes other snapshots except the newly generated snapshot from the snapshot file.

6. The data synchronization method according to claim 4, wherein The method further includes: After performing data synchronization processing on its own memory based on the snapshot file and the log file, a new node joins the accounting service cluster and starts listening to the synchronization component.

7. The data synchronization method according to claim 6, wherein The step that the new node performs data synchronization processing on its own memory based on the snapshot file and the log file includes: The new node obtains the latest snapshot from the snapshot file and loads the accounting data of all accounts in the latest snapshot into its own memory; The new node uses the identification number in the latest snapshot as a first reference identification number; The new node obtains each first operation log after the first reference identification number from the log file; The new node sequentially updates the accounting data of each account in its own memory based on each first operation log.

8. The data synchronization method according to claim 7, wherein, The modification proposal is used to indicate modifying the accounting data of a specified account from the original value to the target value; After the new node performs data synchronization processing on its own memory based on the snapshot file and the log file and then joins the accounting service cluster and starts listening to the synchronization component, the following steps are further included: The new node receives the new modification proposal announced by the synchronization component and its corresponding identification number; When the new node confirms that the original value in the new modification proposal is inconsistent with the accounting data of the specified account in the memory, the new node uses the identification number corresponding to the new modification proposal as the second reference identification number; The new node obtains each second operation log located after the second reference identification number from the log file; The new node sequentially updates the accounting data of each account in its own memory based on each second operation log; The new node updates the accounting data of the specified account in its own memory based on the new modification proposal.

9. The data synchronization method according to claim 4, wherein The operation log includes the account ID of the specified account, the original value before the accounting data modification, the target value after the accounting data modification, and the identification number; the method further includes: The new master node obtains the latest operation log from the log file and determines whether the original value in the latest operation log is consistent with the accounting data of the specified account in its own memory; the new master node is elected from each of the slave nodes after the master node fails. If they are consistent, the new master node directly starts providing services externally; If they are inconsistent, the new master node updates the accounting data of the specified account in its own memory based on the latest operation log and then starts providing services externally.

10. A financial service cluster, characterized in that, The accounting service cluster includes a master node, multiple slave nodes, and a synchronization component communicatively connected to each node; the master node is communicatively connected to the file system, and the accounting data of multiple accounts is stored in the memory of each node; The master node is used to generate a modification proposal based on the accounting request and assign a unique identification number to the modification proposal when receiving the accounting request; The master node is used to append and write the operation log to the file system based on the modification proposal and the identification number; The master node is used to control each slave node to execute the modification proposal through the synchronization component; The master node is used to update the accounting data of the specified account in its own memory based on the modification proposal when receiving the confirmation message returned by the synchronization component, where the confirmation message received by the master node is generated and sent by the synchronization component after confirming that each slave node has completed executing the modification proposal.

Images