A distributed system and data processing method

Abstract

Embodiments of the present application relate to the technical field of computer, and particularly relate to a distributed system and a data processing method, which are used to solve the problem of limited capacity of database nodes in a deposit core system and the upper limit of capacity expansion. The distributed system comprises N nodes, wherein each node comprises a database, the database structures and parameters of any two nodes are the same, and N≥2; any node in the N nodes is taken as a first type node, and the first type node is used for: receiving a first instruction; analyzing the first instruction to determine identification information involved in the first instruction; when the identification information involved in the first instruction further comprises second identification information, processing the first instruction, and generating a second instruction according to the first instruction, wherein the second identification information is located in a database of a second type node; and sending the second instruction to the second type node, so that the second type node processes the second instruction.

Description

Technical Field

[0001] This invention relates to the field of computer technology in financial technology (Fintech), and more particularly to a distributed system and data processing method. Background Technology

[0002] With the continuous development of fintech, especially internet fintech, more and more technologies (such as distributed systems, blockchain, artificial intelligence, etc.) are being applied in the financial field. However, the financial industry is also placing higher demands on technology, such as data processing for core deposit systems.

[0003] Existing core bank deposit systems are based on a single-node model using expensive hardware and software from IOE (IBM, Oracle, and EMC, where IBM is the server provider, Oracle is the database software provider, and EMC is the storage device provider). Account data, transaction records, and other information are all stored centrally on a single database node. While the application layer can scale horizontally, the limited performance and capacity of the single database node necessitate upgrades to the database hardware and software as data volumes grow. In the long run, this approach cannot meet the demands of increasing user and transaction volumes. Summary of the Invention

[0004] This application provides a distributed system and data processing method to solve the problem of limited database node capacity and upper limit for expansion in the core deposit system.

[0005] This invention provides a distributed system comprising N nodes, each node containing a database, wherein the database structure and parameters are identical between any two nodes, and N≥2;

[0006] Any one of the N nodes is designated as a first-type node, and the first-type node is used for:

[0007] Receive the first instruction;

[0008] Parse the first instruction to determine the identification information involved in the first instruction;

[0009] When the identification information involved in the first instruction also includes the second identification information, the first instruction is processed and a second instruction is generated based on the first instruction, wherein the second identification information is located in the database of the second type of node;

[0010] The second instruction is sent to the second type of node so that the second type of node processes the second instruction.

[0011] In one optional embodiment, the first type of node is a first object node; the first object node is used for:

[0012] The system receives the first instruction sent by the message middleware, the first instruction containing first identification information located in the database of the first object node.

[0013] In one optional embodiment, the first instruction is a message-type instruction;

[0014] The first object node is specifically used for:

[0015] Parse the first message to determine the M sub-processes of the first message;

[0016] The identification information corresponding to the M sub-processes is determined sequentially;

[0017] For the sub-processes whose corresponding identification information is located in the database of the first object node, the sub-processes are processed;

[0018] For each sub-process whose corresponding identification information is located in the database of the second type of node, a second message is generated according to the sub-process and sent to the second type of node.

[0019] In one optional embodiment, the second type of node is a second object node; the first type of node is specifically used for:

[0020] Within a predetermined time period, the second instruction is sent to the second object node.

[0021] In one optional embodiment, the first type of node is a second object node, the second type of node is a first object node, and the first instruction is a first file download instruction;

[0022] The second object node is specifically used for:

[0023] Receive the first file download instruction sent by the external system;

[0024] According to the first file download instruction, download the total file from the file transfer module;

[0025] The total file is divided into L file fragments based on the L first object nodes involved, with each file fragment corresponding to one first object node.

[0026] Upload the L files to the file transfer module;

[0027] Generate L second file download instructions, where each second file download instruction corresponds to a first object node;

[0028] For any one of the L second file download instructions, the second file download instruction is sent to the corresponding first object node;

[0029] The first object node is used for:

[0030] According to the second file download instruction, file fragments are downloaded from the file transfer module and processed.

[0031] In an optional embodiment, the first object node is further configured to: upload the processed file fragments to the file transfer module, and send a file fragment download message to the second object node;

[0032] The second object node is also used for: downloading processed file fragments from the file transfer module according to the file fragment download message; merging L processed file fragments and uploading them to the file transfer module; and sending a file processing completion message to the peripheral system.

[0033] In an optional embodiment, the first type of node is further configured to:

[0034] Receive account opening instructions;

[0035] Based on the account opening instruction, identification information is generated, and the relationship between the identification information and the first type of node is recorded in the database.

[0036] In an optional embodiment, the first type of node is further configured to:

[0037] Collect unuploaded operation data from the first type of nodes at set time intervals;

[0038] The unuploaded operation data is uploaded to the file transfer module so that the unuploaded operation data is downloaded and saved to the main database.

[0039] A data processing method for a distributed system, comprising:

[0040] The first type of node receives the first instruction. The first type of node is any one of the N nodes and contains a database. The database structure and parameters are the same between any two of the N nodes.

[0041] The first type of node parses the first instruction to determine the identification information involved in the first instruction;

[0042] When the identification information involved in the first instruction also includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, wherein the second identification information is located in the database of the second type of node;

[0043] The first type of node sends the second instruction to the second type of node so that the second type of node processes the second instruction.

[0044] In one optional embodiment, the first type of node is a first object node; the first type of node receives a first instruction, including:

[0045] The first object node receives the first instruction sent by the message middleware. The first instruction contains first identification information, which is located in the first object node.

[0046] In one optional embodiment, the first instruction is a message-type instruction;

[0047] The first type of node parses the first instruction to determine the identification information involved in the first instruction, including:

[0048] The first object node parses the first message and determines the M sub-processes of the first message;

[0049] The first object node sequentially determines the identification information corresponding to the M sub-processes;

[0050] When the identification information involved in the first instruction also includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, including:

[0051] For the sub-process whose corresponding identification information is located in the database of the first object node, the first object node processes the sub-process;

[0052] For the sub-process whose corresponding identification information is located in the database of the second type of node, the first object node generates a second message according to the sub-process and sends it to the second type of node.

[0053] In one optional embodiment, the second type of node is a second object node; the first type of node sends the second instruction to the second type of node, including:

[0054] The first type of node sends the second instruction to the second object node within a predetermined time period.

[0055] In one optional embodiment, the first type of node is a second object node, the second type of node is a first object node, and the first instruction is a first file download instruction;

[0056] The first type of node receives a first instruction, including:

[0057] The second object node receives the first file download instruction sent by the peripheral system;

[0058] The first type of node parses the first instruction to determine the identification information involved in the first instruction, including:

[0059] The second object node downloads the total file from the file transfer module according to the first file download instruction;

[0060] The second object node divides the total file into L file fragments according to the L first object nodes involved in the total file, and one file fragment corresponds to one first object node;

[0061] The second object node uploads the L files to the file transfer module;

[0062] When the identification information involved in the first instruction further includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, including:

[0063] The second object node generates L second file download instructions, where each second file download instruction corresponds to a first object node;

[0064] The first type of node sends the second instruction to the second type of node, including:

[0065] For any one of the L second file download instructions, the second object node sends the second file download instruction to the corresponding first object node, so that the first object node downloads the file fragment from the file transfer module according to the second file download instruction and processes the file fragment.

[0066] In an optional embodiment, after the second object node sends the second file download instruction to the corresponding first object node, the method further includes:

[0067] The second object node receives the file fragment download message sent by the first object node;

[0068] The second object node downloads the processed file fragments from the file transfer module based on the file fragment download message;

[0069] The second object node merges the L processed file fragments and uploads them to the file transfer module;

[0070] The second object node sends a file processing completion message to the peripheral system.

[0071] In an optional embodiment, before the first type of node receives the first instruction, it further includes:

[0072] The first type of node receives the account opening instruction;

[0073] The first type of node generates identification information according to the account opening instruction and records the relationship between the identification information and the first type of node in the database.

[0074] In an optional embodiment, it further includes:

[0075] The first type of nodes collects operation data that has not been uploaded from the first type of nodes at set time intervals;

[0076] The first type of node uploads the unuploaded operation data to the file transfer module so that the unuploaded operation data is downloaded and saved to the main database.

[0077] This invention also provides an electronic device, comprising:

[0078] At least one processor; and,

[0079] A memory communicatively connected to the at least one processor; wherein,

[0080] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0081] This invention also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the method described above.

[0082] The distributed system of this invention includes N nodes, each containing a database, and the database structure and parameters are identical between any two nodes. Each of the N nodes has the same function; for ease of description, any node is designated as a first-type node. The first-type node is used to: receive a first instruction; parse the first instruction to determine the identification information involved in the first instruction; when the identification information involved in the first instruction includes second identification information, process the first instruction and generate a second instruction based on the first instruction, where the second identification information is located in the database of the second-type node; and send the second instruction to the second-type node so that the second-type node processes the second instruction. In this invention, since the database structure and parameters of the nodes are identical, when the database capacity of the core deposit system is insufficient, a new database can be cloned from the existing databases in the nodes, and then added to the new nodes, increasing the number of nodes and achieving the purpose of expansion. Furthermore, for first instructions that cross nodes, the first-type node can process the first instruction and send instructions to other nodes based on the identification information involved in the first instruction, thereby realizing cross-node instruction processing. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] Figure 1 This is a schematic diagram of a possible system architecture provided in an embodiment of the present invention;

[0085] Figure 2 A schematic diagram of a distributed system architecture provided in an embodiment of the present invention;

[0086] Figure 3 A flowchart illustrating a data processing method for a distributed system provided in an embodiment of the present invention;

[0087] Figure 4 A schematic diagram of the core deposit system provided in a specific embodiment of the present invention;

[0088] Figure 5 This is a schematic diagram of the structure of a data processing device for a distributed system provided in an embodiment of the present invention;

[0089] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0091] like Figure 1 As shown, the system applicable to this embodiment of the invention includes a peripheral system 101 and a deposit core system 102. The peripheral system 101 can be a client installed on a terminal or an operating system installed on a server. The terminal can be an electronic device with wireless communication capabilities, such as a mobile phone, tablet computer, or dedicated handheld device, or a device connected to the Internet via a wired connection, such as a personal computer (PC), laptop computer, or server. A terminal can be a single device or a cluster of multiple terminals. Preferably, the terminal can use cloud computing technology for information processing.

[0092] The core deposit system 102 can be an operating system installed on a computer or other network device. It can be a standalone device or a server cluster consisting of multiple servers. The core deposit system 102 includes N account nodes 103. These N account nodes 103 can be installed on N servers, or on fewer than N servers, meaning that two or more account nodes 103 may be installed on the same server. Preferably, the N account nodes can use cloud computing technology for information processing.

[0093] The peripheral system 101 can remotely call the deposit core system 102, communicate with the deposit core system 102 through the Internet, or communicate with the deposit core system 102 through mobile communication systems such as Global System for Mobile Communications (GSM) and Long Term Evolution (LTE).

[0094] For ease of understanding, the terms that may be involved in the embodiments of the present invention are defined and explained below.

[0095] Message middleware: Infrastructure systems responsible for message transmission, such as ESB, Dubbo, RocketMQ, Kafka, and Solace, are suitable for distributed environments requiring reliable data delivery. In systems employing message middleware mechanisms, different objects activate each other's events and complete corresponding operations by passing messages. The sender sends a message to the message server, which stores the message in several queues and forwards it to the receiver at the appropriate time. Message middleware enables communication between different platforms. It is often used to shield the characteristics of various platforms and protocols, enabling collaboration between applications. Its advantages include providing synchronous and asynchronous connections between clients and servers, and the ability to send or store and forward messages at any time.

[0096] Routing service system: An infrastructure system used to maintain the relationship between card numbers, account numbers, customer numbers, etc., and node numbers. Distributed service architecture is a cluster network, meaning that multiple instances of a service are deployed within the cluster. The routing service system can select the appropriate service provider from the service list for invocation. The distributed service framework must be able to meet users' flexible routing needs.

[0097] Spring: An open-source application framework based on the Java programming language.

[0098] AOP: Aspect-Oriented Programming, used to define interceptors.

[0099] Reversal: When a transaction fails due to an anomaly, the previous operations are forced to be rolled back.

[0100] Reversal: A rollback transaction initiated by the external system after a successful transaction.

[0101] To address the aforementioned problems, based on the above architecture, embodiments of the present invention provide a distributed system, such as... Figure 2 As shown, the data processing architecture of the deposit core system provided in this embodiment of the invention includes N nodes, each node containing a database. The database structure and parameters between any two nodes are the same, and N≥2. Therefore, each node can achieve the same function.

[0102] The following section introduces data processing methods for distributed systems, detailing the role and function of each node in a distributed system. For ease of description, any node among the N nodes will be considered as a first-class node.

[0103] The data processing method for a distributed system provided in this embodiment of the invention is as follows: Figure 3 As shown, it includes the following steps:

[0104] Step 301: The first type of node receives the first instruction.

[0105] Step 302: The first type of node parses the first instruction and determines the identification information involved in the first instruction.

[0106] Step 303: When the identification information involved in the first instruction also includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, wherein the second identification information is located in the database of the second type of node.

[0107] Step 304: The first type of node sends the second instruction to the second type of node so that the second type of node processes the second instruction.

[0108] The distributed system of this invention includes N nodes, each containing a database, and the database structure and parameters are identical between any two nodes. Each of the N nodes has the same function; for ease of description, any node is designated as a first-type node. The first-type node is used to: receive a first instruction; parse the first instruction to determine the identification information involved in the first instruction; when the identification information involved in the first instruction includes second identification information, process the first instruction and generate a second instruction based on the first instruction, where the second identification information is located in the database of the second-type node; and send the second instruction to the second-type node so that the second-type node processes the second instruction. In this invention, since the database structure and parameters of the nodes are identical, when the database capacity of the core deposit system is insufficient, a new database can be cloned from the existing databases in the nodes, and then added to the new nodes, increasing the number of nodes and achieving the purpose of expansion. Furthermore, for first instructions that cross nodes, the first-type node can process the first instruction and send instructions to other nodes based on the identification information involved in the first instruction, thereby realizing cross-node instruction processing.

[0109] In the following description, for ease of understanding, the core deposit system is set up as a distributed system as an example. Therefore, corresponding to the above description, the account node is the node mentioned above, the first account node is the first type of node, the second account node is the second type of node, the account information is the identification information, the customer account node is the first object node, and the internal account node is the second object node.

[0110] In this embodiment of the invention, before processing data using the deposit core system, an account needs to be opened in the deposit core system first. Before the first account node receives the first instruction, the process further includes:

[0111] The first account node receives the account opening instruction;

[0112] The first account node generates account information according to the account opening instruction and records the relationship between the account information and the first account node in the database.

[0113] Specifically, accounts in the core deposit system include customer accounts and internal accounts. The methods for opening different types of accounts vary.

[0114] For opening customer accounts: Since there is more than one customer account node, a message middleware is needed to handle instruction forwarding. The first account node is the first customer account node; the first account node receives the first instruction, including:

[0115] The first customer account node receives the first instruction sent by the message middleware. The first instruction contains first account information, which is located in the first customer account node.

[0116] In practice, the peripheral system first obtains the identifier of a customer account node in the core deposit system from the routing service system, and then sends the account opening instruction to that customer account node via message middleware. Upon receiving the instruction, the customer account node verifies it. If successful, it generates customer account information for the customer and stores the correspondence between the customer account information and the customer account node in the current node's database. Simultaneously, it sends the specific account information items to the peripheral system. After successful account opening, either the customer account node can write the relationship between the card number, account number, and other customer account information and the customer account node to the routing service system, or the peripheral system can write it.

[0117] For opening internal accounts: Due to the limited number of internal account information in the core deposit system, only one internal account node is set up. Therefore, external systems can directly send account opening messages to the internal account node. After successful account opening, the internal account node synchronizes the internal account information to each customer account node to support subsequent transactions between customer accounts and internal accounts.

[0118] In this embodiment of the invention, the instructions processed by the core deposit system can be categorized into message-based instructions and file-based instructions based on the data volume. Message-based instructions include transfer messages, query messages, maintenance messages, etc., and generally involve only one or two account nodes. File-based instructions, in addition to messages, generally include files with large data volumes, and may involve customer accounts distributed across multiple account nodes, such as in a company payroll scenario. The two types of instructions are described below.

[0119] When the first instruction is a message-type instruction, the first account node parses the first instruction to determine the account information involved in the first instruction, including:

[0120] The first customer account node parses the first message and determines the M sub-processes of the first message;

[0121] The first customer account node sequentially determines the account information corresponding to the M sub-processes;

[0122] When the account information involved in the first instruction also includes second account information, the first account node processes the first instruction and generates a second instruction based on the first instruction, including:

[0123] For the sub-process whose corresponding account information is located in the database of the first customer account node, the first customer account node processes the sub-process.

[0124] For the sub-process where the corresponding account information is located in the database of the second account node, the first customer account node generates a second message according to the sub-process and sends it to the second account node.

[0125] In practice, message-type instructions may involve two accounts; for example, a transfer message involves a debit account and a credit account. If the two accounts are in the same node, the message-type instruction is sent directly to that node and processed directly. If the two accounts are in different nodes, either account node can be chosen to receive the message-type instruction. The first account node that receives the first message parses the first message and determines its M sub-processes, each corresponding to an account. The corresponding account can be located in either the first or second account node. The first account node processes the sub-processes of its own first account node and generates a second message based on the sub-processes of the second account node, sending it to the second account node. Here, the processing order is based on the order of the sub-processes. After all sub-processes are processed, the first account node reports the processing results back to the external system. In this way, the external system does not need to pay attention to the cross-node nature of the message; it only needs to send the message to the core deposit system, saving the external system workload and manpower costs.

[0126] Specifically, if the second account node is an internal account node, in order to reduce the pressure on internal account nodes, the second account node is an internal account node; the first account node sends the second instruction to the second account node, including:

[0127] The first account node sends the second instruction to the internal account node within a predetermined time period.

[0128] In specific implementation, the first account node can determine the type of the first instruction. If real-time processing is required, the first account node will immediately send the second instruction to the internal account node after processing, so that the internal account node can immediately process the second instruction and provide feedback. If the type of the first instruction is maskable, the first account node will not immediately send the second instruction after processing the corresponding sub-process and generating it. Instead, it can send the second instruction to the internal account node after a period of time. Alternatively, the first account node can cache multiple second instructions that require processing by the internal account node and send them to the internal account node in batches within a predetermined time period. Generally, to reduce the pressure on the internal account node, this embodiment of the invention uses a delayed internal processing mode.

[0129] Furthermore, for instructions involving only one account node, such as query messages, maintenance messages, or transfers between internal accounts, in this embodiment of the invention, the first instruction is directly sent to the corresponding account node, and the instruction is processed within the account node.

[0130] If the first instruction is a file-related instruction, and the file contains a large amount of data, the file involved in the first instruction needs to be uploaded to the file transfer module, and then the total file needs to be split. This splitting can be performed by the internal account node. In this case, the first account node is an internal account node, and the second account node is a client account node. The first account node receives the first instruction, including:

[0131] The internal account node receives the first file download instruction sent by the external system;

[0132] The first account node parses the first instruction to determine the account information involved in the first instruction, including:

[0133] The internal account node downloads the total file from the file transfer module according to the first file download instruction;

[0134] The internal account node divides the total file into L file fragments according to the L customer account nodes involved in the total file, and each file fragment corresponds to one customer account node.

[0135] The internal account node uploads the L files to the file transfer module;

[0136] When the account information involved in the first instruction also includes second account information, the first account node processes the first instruction and generates a second instruction based on the first instruction, including:

[0137] The internal account node generates L second file download instructions, where each second file download instruction corresponds to a customer account node;

[0138] The first account node sends the second instruction to the second account node, including:

[0139] For any one of the L second file download instructions, the internal account node sends the second file download instruction to the corresponding client account node, so that the client account node downloads the file fragment from the file transfer module according to the second file download instruction and processes the file fragment.

[0140] In practice, the external system uploads the master file to the file transfer module and sends a first file download command to the internal account nodes, enabling them to download the master file from the file transfer module. The internal account nodes, based on the account identifiers in the file, query the corresponding node identifiers and segment the master file accordingly, treating the data corresponding to each account within the same account node as a file fragment. The internal account nodes can either send the segmented file fragments to the client account nodes individually, or they can re-upload the segmented file fragments to the file transfer module and send a second file download command to the corresponding client account nodes, allowing them to download the corresponding file fragments from the file transfer module. By uploading the file to the file transfer module, large-scale data transfers between account nodes are eliminated, thus alleviating the transmission pressure between nodes.

[0141] In addition, the master file can also be directly segmented by the peripheral system. Specifically, the peripheral system retrieves the identifier of the corresponding account node from the routing service system based on the card number or account number. The master file is then segmented into several different file fragments according to the different node numbers. These file fragments are uploaded to the file transfer module, and then a message is sent to each client account node via a message middleware. Upon receiving the notification, each client account node downloads the file fragments and processes them.

[0142] After file processing is complete, the file is still transferred via the file transfer module. After the internal account node sends the second file download instruction to the corresponding client account node, the process further includes:

[0143] The internal account node receives file fragment download messages sent by the client account node;

[0144] The internal account node downloads the processed file fragments from the file transfer module based on the file fragment download message;

[0145] The internal account node merges the L processed file fragments and uploads them to the file transfer module;

[0146] The internal account node sends a file processing completion message to the external system.

[0147] In practice, after the client account node processes the file fragments, it uploads the processed file fragments to the file transfer module and sends a file fragment download message to the internal account node. Based on the file fragment download message, the internal account node downloads the processed file fragments from the file transfer module, merges all file fragments, uploads them again, and finally sends a message to notify the external system.

[0148] Furthermore, embodiments of the present invention also include:

[0149] The first account node collects operation data that has not been uploaded from the first account node at set time intervals;

[0150] The first account node uploads the unuploaded operation data to the file transfer module so that the unuploaded operation data is downloaded and saved to the main database.

[0151] In the specific implementation process, the data operation logs in the customer account node need to be synchronized to the internal account node, and the data operation logs in the customer account node and the internal account node need to be synchronized to the general database of the small ledger system.

[0152] In each customer account node, a first scheduled task is created (e.g., every 2 minutes). This first scheduled task collects out-of-sync operational data, such as delayed internal account transaction logs, generates a file, uploads it to the file transfer system, and sends a notification to the internal account node. Upon receiving the notification, the internal account node downloads and parses the operational data, updates its internal account balance, and stores the operational data in its internal account database.

[0153] Furthermore, a second scheduled task is created for each customer account node (e.g., every 3 minutes). This second scheduled task collects accounting records (including transaction records, end-of-day accrual records, and total / minimum transaction records) from unsynchronized customer accounts and / or delayed internal accounts, generates a file, uploads it to the file transfer system, and notifies the general ledger subsystem. Upon receiving the notification, the general ledger subsystem downloads and parses the file, then appends the accounting records to its database.

[0154] In addition, a third scheduled task (e.g., every 4 minutes) is created within the internal account node. This third scheduled task collects the accounting records (accounting records, total and branch records, etc.) of the unsynchronized real-time internal accounts, generates a file, uploads it to the file transfer system, and notifies the general ledger subsystem. Upon receiving the notification, the general ledger subsystem downloads and parses the file, then appends the accounting records to its database.

[0155] To better understand the present invention, the above process will be described in detail below with specific embodiments. In a specific real-time example, the structure of the deposit core system can be as follows: Figure 4 As shown, it includes one internal account node and N customer account nodes. The customer account nodes support data and transactions related to customer accounts. The internal account nodes support data and transactions related to internal accounts. For ease of management and resource conservation, the internal account nodes also include a small ledger subsystem for handling administrative and statistical tasks.

[0156] Since the internal account node includes both the internal account database and the small ledger subsystem database, the internal account database and the small ledger subsystem database can be configured separately, i.e., each includes an internal account node and a small ledger node. Furthermore, compared to the massive amount of data in customer accounts, the amount of data in internal accounts is much smaller. Therefore, in this embodiment of the invention, only one internal account node is set up, but the number of internal account nodes can be increased accordingly based on actual needs.

[0157] In the above structure, each account node has the same database structure and parameters, but the stored account and transaction data differ. The sub-ledger database, because it only involves data statistics and summarization, can use a different structure and parameters.

[0158] Functionally, the core deposit system can be divided into the following subsystems:

[0159] 1. Online Transaction Subsystem: Responsible for handling online transactions related to customer accounts, such as account opening, transfers, inquiries, and maintenance.

[0160] 2. Batch Processing Subsystem: Responsible for processing batch tasks related to customer accounts, such as file-based transactions, synchronizing transaction logs to the ADM batch processing subsystem, synchronizing accounting logs to the general ledger subsystem, and end-of-day tasks.

[0161] 3. ADM Online Transaction Subsystem: Responsible for handling online transactions related to internal accounts, such as account opening, transfers, inquiries, and maintenance.

[0162] 4. ADM Batch Processing Subsystem: Responsible for processing synchronized data from customer account nodes, generating internal account transaction logs, accounting logs, as well as file-based transactions and end-of-day tasks.

[0163] 5. Small General Ledger Subsystem: Responsible for summarizing accounting transactions, registering and verifying accounting records, and reporting the account balance data to the Large General Ledger System.

[0164] 6. Console Subsystem: Responsible for task monitoring, task re-running, and account adjustment for each batch processing subsystem, as well as online transaction subsystem.

[0165] In addition, the core deposit system also includes auxiliary units such as message middleware, routing service system, and file transfer system.

[0166] It should be understood that the above system is only one embodiment of the present invention. The specific structure and auxiliary system in each account node can be set according to specific needs. The embodiments of the present invention are only examples and are not intended to limit the scope.

[0167] This invention also provides a data processing device for a distributed system, such as... Figure 5 As shown, it includes:

[0168] Transceiver unit 501 is used to receive the first instruction;

[0169] The parsing unit 502 is used to parse the first instruction and determine the account information involved in the first instruction;

[0170] The processing unit 503 is configured to, when the account information involved in the first instruction also includes second account information, process the first instruction and generate a second instruction based on the first instruction, wherein the second account information is located in the database of the second account node;

[0171] The transceiver unit 501 is further configured to send the second instruction to the second account node so that the second account node can process the second instruction.

[0172] Optionally, the transceiver unit 501 is further configured to receive the first instruction sent by the message middleware, wherein the first instruction contains first account information and the first account information is located in the first customer account node.

[0173] Optionally, the first instruction is a message-type instruction;

[0174] The parsing unit 502 is specifically used to parse the first message, determine the M sub-processes of the first message, and sequentially determine the account information corresponding to the M sub-processes.

[0175] The processing unit 503 is specifically used for processing sub-processes whose corresponding account information is located in the database of the first customer account node; for sub-processes whose corresponding account information is located in the database of the second account node, the first customer account node generates a second message according to the sub-process and sends it to the second account node.

[0176] Optionally, the transceiver unit 501 is specifically used to send the second instruction to the internal account node within a predetermined time period.

[0177] Optionally, the first instruction is a first file download instruction;

[0178] The transceiver unit 501 is specifically used to receive the first file download instruction sent by the peripheral system;

[0179] The parsing unit 502 is specifically used to download the total file from the file transfer module according to the first file download instruction; divide the total file into L file fragments according to the L customer account nodes involved in the total file, wherein one file fragment corresponds to one customer account node; and upload the L files to the file transfer module.

[0180] The processing unit 503 is specifically used to generate L second file download instructions, wherein each second file download instruction corresponds to a customer account node;

[0181] The transceiver unit 501 is specifically used to send the second file download instruction to the corresponding client account node for any one of the L second file download instructions, so that the client account node downloads file fragments from the file transfer module according to the second file download instruction and processes the file fragments.

[0182] Optionally, the transceiver unit 501 is specifically used to receive file fragment download messages sent by client account nodes; download processed file fragments from the file transfer module according to the file fragment download messages; merge L processed file fragments and upload them to the file transfer module; and send a file processing completion message to the peripheral system.

[0183] Optionally, the transceiver unit 501 is used to receive an account opening instruction;

[0184] The processing unit 503 is used to generate account information according to the account opening instruction and record the relationship between the account information and the first account node in the database.

[0185] Optionally, a synchronization unit 504 is also included, for:

[0186] Collect unuploaded operation data from the first account node at set time intervals; upload the unuploaded operation data to the file transfer module so that the unuploaded operation data is downloaded and saved to the main database.

[0187] Based on the same principle, the present invention also provides an electronic device, such as... Figure 6 As shown, it includes:

[0188] It includes a processor 601, a memory 602, a transceiver 603, and a bus interface 604, wherein the processor 601, the memory 602, and the transceiver 603 are connected through the bus interface 604;

[0189] The processor 601 is used to read the program in the memory 602 and execute the following methods:

[0190] Receive the first instruction;

[0191] Parse the first instruction to determine the account information involved in the first instruction;

[0192] When the account information involved in the first instruction also includes the second account information, the first instruction is processed and a second instruction is generated based on the first instruction, wherein the second account information is located in the database of the second account node;

[0193] The second instruction is sent to the second account node so that the second account node processes the second instruction.

[0194] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0195] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0196] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0197] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0198] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A distributed system, characterized in that, The distributed system comprises N nodes, each containing a database. The database structure and parameters are identical between any two nodes, and N≥2. When the database capacity of the distributed system is insufficient, a new database is cloned from the existing database in a node, and then the new database is added to a new node, thereby increasing the number of nodes. Any one of the N nodes is designated as a first-type node, and the first-type node is used for: Receive the first instruction; Parse the first instruction to determine the identification information involved in the first instruction; When the identification information involved in the first instruction also includes the second identification information, the first instruction is processed and a second instruction is generated based on the first instruction, wherein the second identification information is located in the database of the second type of node; The second instruction is sent to the second type of node so that the second type of node processes the second instruction; Wherein, the first type of node is a second object node, the second type of node is a first object node, and the first instruction is a first file download instruction; The second object node is specifically used for: Receive the first file download instruction sent by the external system; According to the first file download instruction, download the total file from the file transfer module; The total file is divided into L file fragments based on the L first object nodes involved, with each file fragment corresponding to one first object node. Upload the L files to the file transfer module; Generate L second file download instructions, where each second file download instruction corresponds to a first object node; For any one of the L second file download instructions, the second file download instruction is sent to the corresponding first object node; The first object node is used for: According to the second file download instruction, file fragments are downloaded from the file transfer module and processed.

2. The system as described in claim 1, characterized in that, The first type of node is a first object node; the first object node is used for: The system receives the first instruction sent by the message middleware, the first instruction containing first identification information located in the database of the first object node.

3. The system as described in claim 2, characterized in that, The first instruction is a message instruction; The first object node is specifically used for: Parse the first message to determine the M sub-processes of the first message; The identification information corresponding to the M sub-processes is determined sequentially; For the sub-processes whose corresponding identification information is located in the database of the first object node, the sub-processes are processed; For each sub-process whose corresponding identification information is located in the database of the second type of node, a second message is generated according to the sub-process and sent to the second type of node.

4. The system as described in claim 3, characterized in that, The second type of node is the second object node; the first type of node is specifically used for: Within a predetermined time period, the second instruction is sent to the second object node.

5. The system as described in claim 1, characterized in that, The first object node is further configured to: upload the processed file fragments to the file transfer module, and send a file fragment download message to the second object node; The second object node is also used for: downloading processed file fragments from the file transfer module according to the file fragment download message; merging L processed file fragments and uploading them to the file transfer module; and sending a file processing completion message to the peripheral system.

6. The system according to any one of claims 1 to 5, characterized in that, The first type of node is also used for: Receive account opening instructions; Based on the account opening instruction, identification information is generated, and the relationship between the identification information and the first type of node is recorded in the database.

7. The system according to any one of claims 1 to 5, characterized in that, The first type of node is also used for: Collect unuploaded operation data from the first type of nodes at set time intervals; The unuploaded operation data is uploaded to the file transfer module so that the unuploaded operation data is downloaded and saved to the main database.

8. A data processing method for a distributed system, applied to a distributed system, characterized in that, The distributed system comprises N nodes, each containing a database. The database structure and parameters are identical between any two nodes, and N ≥ 2. When the database capacity of the distributed system is insufficient, a new database is cloned from an existing node, and then added to a new node to increase the number of nodes. The method includes: The first type of node receives the first instruction. The first type of node is any one of the N nodes and contains a database. The database structure and parameters are the same between any two of the N nodes. The first type of node parses the first instruction to determine the identification information involved in the first instruction; When the identification information involved in the first instruction also includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, wherein the second identification information is located in the database of the second type of node; The first type of node sends the second instruction to the second type of node so that the second type of node processes the second instruction; Wherein, the first type of node is a second object node, the second type of node is a first object node, and the first instruction is a first file download instruction; The first type of node receives a first instruction, including: The second object node receives the first file download instruction sent by the peripheral system; The first type of node parses the first instruction to determine the identification information involved in the first instruction, including: The second object node downloads the total file from the file transfer module according to the first file download instruction; The second object node divides the total file into L file fragments according to the L first object nodes involved in the total file, and one file fragment corresponds to one first object node; The second object node uploads the L files to the file transfer module; When the identification information involved in the first instruction further includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, including: The second object node generates L second file download instructions, where each second file download instruction corresponds to a first object node; The first type of node sends the second instruction to the second type of node, including: For any one of the L second file download instructions, the second object node sends the second file download instruction to the corresponding first object node, so that the first object node downloads the file fragment from the file transfer module according to the second file download instruction and processes the file fragment.

9. The method as described in claim 8, characterized in that, The first type of node is the first object node; The first type of node receives a first instruction, including: The first object node receives the first instruction sent by the message middleware. The first instruction contains first identification information, which is located in the first object node.

10. The method as described in claim 9, characterized in that, The first instruction is a message instruction; The first type of node parses the first instruction to determine the identification information involved in the first instruction, including: The first object node parses the first message and determines the M sub-processes of the first message; The first object node sequentially determines the identification information corresponding to the M sub-processes; When the identification information involved in the first instruction also includes second identification information, the first type of node processes the first instruction and generates a second instruction based on the first instruction, including: For the sub-process whose corresponding identification information is located in the database of the first object node, the first object node processes the sub-process; For the sub-process whose corresponding identification information is located in the database of the second type of node, the first object node generates a second message according to the sub-process and sends it to the second type of node.

11. The method as described in claim 10, characterized in that, The second type of node is a second object node; the first type of node sends the second instruction to the second type of node, including: The first type of node sends the second instruction to the second object node within a predetermined time period.

12. The method as described in claim 8, characterized in that, After the second object node sends the second file download instruction to the corresponding first object node, it also includes: The second object node receives the file fragment download message sent by the first object node; The second object node downloads the processed file fragments from the file transfer module based on the file fragment download message; The second object node merges the L processed file fragments and uploads them to the file transfer module; The second object node sends a file processing completion message to the peripheral system.

13. The method according to any one of claims 8 to 12, characterized in that, Before the first type of node receives the first instruction, it also includes: The first type of node receives the account opening instruction; The first type of node generates identification information according to the account opening instruction and records the relationship between the identification information and the first type of node in the database.

14. The method according to any one of claims 8 to 12, characterized in that, Also includes: The first type of nodes collects operation data that has not been uploaded from the first type of nodes at set time intervals; The first type of node uploads the unuploaded operation data to the file transfer module so that the unuploaded operation data is downloaded and saved to the main database.

15. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 8-14.

16. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method described in any one of claims 8-14.

Images