A transaction flow distributed batch processing method and system

Abstract

The present invention discloses a distributed batch processing method for transaction flows and a system thereof, comprising a scheduling module, a transaction flow module, a clearing module and a profit-sharing calculation module, wherein the scheduling module, the transaction flow, the clearing module and the profit-sharing calculation module are connected in sequence, and the scheduling module is used to trigger the clearing module to execute clearing transactions, and the transaction flow will be synchronized to the transaction storage database in the clearing module through the database of the transaction flow module in real time, and various types of order consumption data will be checked at regular intervals, and filtered through cleaning, conversion and filtering, and stored in the database of the clearing module after successful clearing, for the profit-sharing calculation module to extract and calculate the profit-sharing handling fee. The present invention can support the clearing of rising data volume in business dimensions without affecting the batch execution time of other dimensions, can support rapid data migration, and facilitate various business processing.

Description

Technical Field

[0001] The present invention relates to the technical field of network business transaction, and in particular to a transaction flow distributed batch processing method and system thereof. Background Art

[0002] In the existing technology, the system is based on the business (such as customer) dimension for clearing and calculation. This method will cause the transaction volume of a certain business dimension to be unevenly distributed, the transaction volume of individual businesses to grow rapidly, and the data to be too large. This business will affect the clearing and calculation time of all customers, causing the system to execute slowly, thereby affecting the running time of the entire system. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides a method and system for distributed batch processing of transaction flows. When the transaction volume of a single business increases, the present invention solves the problem that the increase in data volume in the business dimension affects the batch execution time of other dimensions.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a distributed batch processing method for transaction flow, the improvement of which is that the method comprises the following steps:

[0005] S1: The scheduling module imports the transaction flow in the transaction flow module into the clearing module at a regular interval;

[0006] S2: The database of the transaction flow module synchronizes the transaction flow with the transaction storage database of the clearing module;

[0007] S3: The clearing module finds out the transaction data, and converts it into a clearing flow with a clearing batch number after filtering through a cleaning conversion filter, thereby forming clearing batch number data. The multiple clearing batch number data are stored in the database of the clearing module in the format of a clearing batch table.

[0008] S4: The scheduling module extracts the clearing batch table from the database of the clearing module to the profit calculation module to calculate the profit sharing fee.

[0009] As an improvement of the above technical solution, in the above step S2, the step of cleaning, converting, filtering and storing in the database of the clearing module includes:

[0010] S21: The clearing module queries for successful transaction records;

[0011] S22: Import successful transaction flow into the clearing module;

[0012] S23: The clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, thereby forming a clearing flow with a clearing batch number or a clearing flow with a blank clearing batch number;

[0013] S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module through the scheduling module.

[0014] As an improvement of the above technical solution, the transaction flow includes data on consumption, orders, payments on behalf of others, refunds or other types of transactions.

[0015] As an improvement of the above technical solution, the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions.

[0016] A distributed batch processing system for transaction flows, the improvement of which lies in that it comprises a scheduling module, a transaction flow module, a clearing module and a profit-sharing calculation module, the scheduling module, the transaction flow, the clearing module and the profit-sharing calculation module are connected in sequence, the scheduling module is used to trigger the clearing module to execute clearing transactions, the transaction flow will be synchronized to the transaction storage database in the clearing module through the database of the transaction flow module in real time, various types of order consumption data will be checked at regular intervals, and filtered through cleaning conversion, and stored in the database of the clearing module after successful clearing, for the profit-sharing calculation module to extract and calculate the profit-sharing handling fee.

[0017] As an improvement of the above technical solution, the steps of cleaning, converting, filtering and storing in the database of the clearing module include:

[0018] S21: The clearing module queries for successful transaction records;

[0019] S22: Import successful transaction flow into the clearing module;

[0020] S23: The clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, thereby forming a clearing flow with a clearing batch number or a clearing flow with a blank clearing batch number;

[0021] S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module.

[0022] As an improvement of the above technical solution, the transaction flow cleared by the clearing module includes consumption, order or other transaction type data.

[0023] As an improvement of the above technical solution, the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions.

[0024] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the improvement lies in that the processor implements any one of the above methods when executing the computer program.

[0025] A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing any of the methods described above.

[0026] The beneficial effects of the present invention are as follows: the present invention can support the clearing of increasing data volume in business dimensions without affecting the batch execution time of other dimensions, can support rapid data migration, and facilitate various business processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] Figure 1 It is a system flow chart of the present invention.

[0029] Figure 2 It is a system flow chart of the present invention.

[0030] Figure 3 It is a flow chart of the clearing transaction of the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.

[0032] Reference Figure 1 and Figure 2 The present invention discloses a distributed batch processing method for transaction flow, the method comprising the following steps:

[0033] S1: Scheduling module 1 imports the transaction flow in transaction flow module 2 into clearing module 3 at regular intervals;

[0034] S2: The database of the transaction flow module 2 synchronizes the transaction flow with the transaction storage database of the clearing module 3;

[0035] S3: The clearing module 3 finds out the transaction data, and converts it into a clearing flow with a clearing batch number after filtering through a cleaning conversion filter, thereby forming clearing batch number data. The multiple clearing batch number data are stored in the database of the clearing module 3 in the format of a clearing batch table.

[0036] S4: The scheduling module 1 extracts the clearing batch table from the database of the clearing module 3 to the profit calculation module 4 to calculate the profit handling fee.

[0037] In the above step S2, the steps of cleaning, converting, filtering and storing in the database of the clearing module 3 include:

[0038] S21: the clearing module 3 searches for successful transaction records;

[0039] S22: Import the successful transaction flow into the clearing module 3;

[0040] S23: the clearing module 3 converts the successful clearing flow into the required fields by associating with other transaction flow tables, thereby forming a clearing flow with a clearing batch number or a clearing flow with a blank clearing batch number;

[0041] S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module 3 through the scheduling module 1.

[0042] In the above embodiments, reference Figure 3 The transaction flow will be synchronized in real time through the database of the transaction flow module 2 to the transaction storage database of the clearing module 3 to form a clearing transaction table. The clearing transaction table will be triggered and executed by the scheduling module 1. A batch of clearing transaction flow data of different types of order consumption will be found out at regular intervals. After cleaning, conversion and filtering, successful transaction flows will be queried, and after being converted into the required fields by associating other transaction flow tables, they will be converted into clearing flows and stored in the database.

[0043] Furthermore, the transaction flow includes data on consumption, orders, payments on behalf of others, refunds or other types of transactions, and supports multiple types of clearing transactions.

[0044] Furthermore, the liquidation batch table includes the liquidation date, batch number, group number, liquidation status and number of transactions. The liquidation date, batch number, group number, liquidation status and number of transactions are fully integrated, making the transaction flow data fully transparent and facilitating the liquidation of flow transaction data.

[0045] When the scheduling module 1 of the present invention extracts data from the transaction flow module 2 to the clearing module 3 at a fixed time, a clearing batch number field will be added to each clearing flow. After the transaction flow is imported into the clearing module 3, a clearing batch number will be set for each clearing flow for every N transactions. The group number is determined by the number of machine nodes. The number of machine nodes X is obtained from the configuration file and can be dynamically modified. The X group number is allocated and executed according to the number of machine nodes X. Assuming that the number of nodes is X, that is, 1 to X, each group number is allocated with N transaction flows, and according to the clearing date, batch number, group number, clearing status, and number of transactions, it is saved as a clearing batch table. When the scheduling module 1 executes the clearing batch task, as shown in FIG. Figure 1 As shown, the scheduling module 1 will send execution commands to multiple servers in the profit calculation module 4 according to the number of machine nodes X allocated by the clearing batch table. Each server will select a machine according to the command sent by the scheduling module 1. The server will obtain the transaction flow related to this group number to clear the calculation flow, thereby achieving the purpose of multi-service distributed batch processing.

[0046] A distributed batch processing system for transaction flows comprises a scheduling module 1, a transaction flow module 2, a clearing module 3 and a profit calculation module 4, wherein the scheduling module 1, the transaction flow module 2, the clearing module 3 and the profit calculation module 4 are connected in sequence, and the scheduling module 1 is used to trigger the clearing module 3 to execute a clearing transaction, and the transaction flow will be synchronized to the transaction storage database in the clearing module 3 through the database of the transaction flow module 2 in real time, and various types of order consumption data will be checked at regular intervals, and filtered through cleaning, conversion and filtering, and stored in the database of the clearing module 3 after successful clearing, for the profit calculation module 4 to extract and calculate the profit fee. When the data volume of the business dimension increases, the batch execution time of other dimensions is reduced, and fast data migration is supported to facilitate various business processing.

[0047] The steps of cleaning, converting, filtering and storing in the database of the clearing module include:

[0048] S21: the clearing module 3 searches for successful transaction records;

[0049] S22: Import the successful transaction flow into the clearing module 3;

[0050] S23: the clearing module 3 converts the successful clearing flow into the required fields by associating with other transaction flow tables, thereby forming a clearing flow with a clearing batch number or a clearing flow with a blank clearing batch number;

[0051] S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module 3.

[0052] In the above embodiment, if Figure 3As shown, the transaction flow will be synchronized to the transaction storage database of the clearing module 3 in real time through the database to form a clearing transaction table. The clearing transaction table will be triggered for execution by the scheduling module 1. A batch of clearing transaction flow data of different types of order consumption will be checked out at regular intervals. Through cleaning, conversion and filtering, the success status that needs to be imported into the clearing flow is queried, and after being converted into the required fields by associating other transaction flow tables, it is converted into a clearing flow and stored in the database of the clearing module 3.

[0053] Furthermore, the transaction flow includes data on consumption, orders, payments on behalf of others, refunds or other types of transactions. Multiple types of clearing transactions are supported.

[0054] Furthermore, the liquidation batch table includes the liquidation date, batch number, group number, liquidation status and number of transactions. The liquidation date, batch number, group number, liquidation status and number of transactions are fully integrated, making the transaction flow data fully transparent and facilitating the liquidation of flow transaction data.

[0055] When the scheduling module 1 of the present invention extracts data from the transaction flow module 2 to the clearing module 3 at a regular interval, a clearing batch number field will be attached to each clearing flow. After the transaction flow is imported into the clearing module 3, a clearing batch number will be set for each N clearing flow. The group number is determined by the number of machine nodes. The number of machine nodes X is obtained from the configuration file and can be dynamically modified. The execution group number X is allocated according to the number of machine nodes X. Assuming the number of nodes is X, that is, 1 to X, each group number is allocated N transaction flows, and is saved as a clearing batch table according to the clearing date, batch number, group number, clearing status, and number of transactions. When the scheduling module 1 executes the clearing batch task, the scheduling module 1 will send an execution command to multiple servers in the profit sharing calculation module 4 according to the number of machine nodes X allocated by the clearing batch table. Each server will select a machine according to the command sent by the scheduling module. The server will obtain the transaction flow related to this group number to clear the calculation flow, thereby achieving the purpose of multi-service distributed batch processing.

[0056] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein any one of the above methods is implemented when the processor executes the computer program.

[0057] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing any of the above methods.

[0058] The beneficial effects of the present invention are as follows: the present invention can support the clearing of increasing data volume in business dimensions without affecting the batch execution time of other dimensions, can support rapid data migration, and facilitate various business processing.

[0059] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A distributed batch processing method for transaction flow, characterized in that: The method comprises the following steps: S1: The scheduling module imports the transaction flow in the transaction flow module into the clearing module at regular intervals; S2: The database of the transaction flow module synchronizes the transaction flow with the transaction storage database of the clearing module; In the above step S2, the steps of cleaning, converting, filtering and storing in the database of the clearing module include: S21: The clearing module queries for successful transaction records; S22: Import successful transaction flow into the clearing module; S23: The clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, thereby forming a clearing flow with a clearing batch number or a clearing flow with a blank clearing batch number; S24: updating the clearing batch number information of each transaction to form a clearing batch number table, and storing it in the database of the clearing module through the scheduling module; The transaction flow includes data on consumption, orders, payments, refunds or other types of transactions; S3: The clearing module finds out the transaction data, filters it through a cleaning conversion filter, and converts it into a clearing flow with a clearing batch number to form clearing batch number data. The multiple clearing batch number data are stored in the database of the clearing module in the format of a clearing batch table; S4: The scheduling module extracts the clearing batch table from the database of the clearing module to the profit calculation module to calculate the profit sharing fee; The liquidation batch table includes the liquidation date, batch number, group number, liquidation status and number of transactions.

2. A distributed batch processing system for transaction flow, characterized in that: The system is used to implement a transaction flow distributed batch processing method as claimed in claim 1; The system includes a scheduling module, a transaction flow module, a clearing module and a profit calculation module. The scheduling module, transaction flow, clearing module and profit calculation module are connected in sequence. The scheduling module is used to trigger the clearing module to execute a clearing transaction. The transaction flow will be synchronized to the transaction storage database in the clearing module through the database of the transaction flow module in real time. Various types of order consumption data are checked at regular intervals, and filtered through cleaning, conversion and filtering. After successful clearing, it is stored in the database of the clearing module for the profit calculation module to extract and calculate the commission fee.

3. A transaction flow distributed batch processing system according to claim 2, characterized in that: The steps of cleaning, converting, filtering and storing in the database of the clearing module include: S21: The clearing module queries for successful transaction records; S22: Import successful transaction flow into the clearing module; S23: The clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, thereby forming a clearing flow with a clearing batch number or a clearing flow with a blank clearing batch number; S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module.

4. A transaction flow distributed batch processing system according to claim 2, characterized in that: The transaction flow includes data on consumption, orders, payments on behalf of others, refunds or other types of transactions.

5. A transaction flow distributed batch processing system according to claim 2, characterized in that: The liquidation batch table includes the liquidation date, batch number, group number, liquidation status and number of transactions.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method of claim 1 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method of claim 1.

Images