A method and device for reconciling and generating transaction data, and a payment system
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CICC PAYMENT CO LTD
- Filing Date
- 2023-08-22
- Publication Date
- 2026-06-16
Smart Images
Figure CN117078418B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of financial reconciliation technology, and in particular to a method, apparatus and payment system for reconciling and generating transaction data. Background Technology
[0002] Currently, the service architecture adopted by large-scale payment systems differs from that of monolithic applications, involving dozens or even hundreds of business subsystems. Large-scale payment systems have numerous business subsystems, many transaction processing steps, and complex processing flows. To effectively reduce the risk of problematic transactions, a reconciliation system is particularly necessary in large-scale payment systems. Reconciliation systems face several challenges when verifying transaction data in large-scale payment systems. (1) The reconciliation system needs to be capable of processing massive amounts of transaction data, efficiently extracting and processing large volumes. To achieve this, the reconciliation system inevitably introduces additional complexity. (2) The reconciliation system also needs to adapt to the differences between different business subsystems. For example, the data table designs differ between different business subsystems, and the reconciliation system needs to implement adaptation strategies to ensure the execution of unified reconciliation logic. (3) The reconciliation system must be able to cope with the differences in transaction reconciliation time caused by different business characteristics. For example, the reconciliation time for some businesses is only tens of milliseconds, while for others it may be several hours, days, or even months. For transactions with long reconciliation times, the reconciliation system needs to select differentiated processing and verification strategies.
[0003] Therefore, reconciliation in large-scale payment systems is quite challenging. The numerous business subsystems, complex interactions, massive data volumes, architectural evolution leading to differences, and diverse business characteristics all contribute to the increased challenges of end-to-end reconciliation within large-scale payment systems. Currently, the industry offers two solutions to the reconciliation problem in large-scale payment systems. Solution 1: Each business subsystem periodically generates standardized reconciliation files and regularly transmits these files to a unified location. The reconciliation system then reads these files and performs unified reconciliation. The disadvantage of this solution is that it requires embedding reconciliation-related logic into the business subsystems, and developers of each subsystem need to support the implementation of reconciliation data conversion. For large-scale payment systems with a large number of business subsystems, especially in scenarios with complex interactions and intricate branching and convergence of links, the development cost is high and the flexibility is relatively poor. Solution 2: Real-time acquisition of transaction data from each business subsystem, inter-subsystem reconciliation of the acquired transaction data to generate edge reconciliation results, and then aggregation of these edge reconciliation results. The limitation of this solution is that the reconciliation system still relies on globally unique transaction serial numbers. Transaction data whose status cannot be determined is cached and re-verified when the global serial number reappears. If the global serial number no longer appears, the transaction data is difficult to re-verify. In addition, the initial step of this reconciliation process requires transaction deduplication, which cannot detect duplicate data caused by payment system failures or operational errors.
[0004] In summary, existing reconciliation systems in the industry cannot meet the more general reconciliation needs of large payment systems, such as flexibly adapting to extremely complex links between internal business subsystems, not relying heavily on globally unique identifiers of transaction events, reconciling transactions whose status cannot be determined for a long time, and accurately identifying duplicate transactions caused by system failures. Summary of the Invention
[0005] Therefore, it is necessary to provide a method, apparatus, and payment system for reconciling and generating transaction data to address the aforementioned technical problems.
[0006] Firstly, a method for reconciling transaction data is provided, the method being applied to a reconciliation system, the method comprising:
[0007] Obtain the DAG graph of the transaction chain of the payment system, and split the DAG graph into multiple sub-transaction chains;
[0008] For each sub-transaction link, the transaction data of each business node in the sub-transaction link is obtained, and the transaction data is divided into multiple transaction reconciliation groups according to the association key in each transaction data; the association key includes upstream association key and downstream association key;
[0009] For each transaction reconciliation group, reconcile the transaction data contained in that group.
[0010] As an optional implementation, the in-degree and out-degree of each business node in each sub-transaction link are not greater than 1.
[0011] As an optional implementation, the step of dividing each transaction data into multiple transaction reconciliation groups based on the association key in each transaction data includes:
[0012] In the transaction data of each business node, if the upstream association key in the first transaction data is the same as the downstream association key in the second transaction data, and the business node to which the first transaction data belongs is the downstream business node adjacent to the business node to which the second transaction data belongs in the sub-transaction link, then the first transaction data and the second transaction data are divided into a transaction reconciliation group.
[0013] As an optional implementation, the method further includes:
[0014] If the downstream association key of the third transaction data is different from the upstream association key of other transaction data in the transaction data of each business node, it is determined that the business node to which the third transaction data belongs has a shortfall in the downstream business node adjacent to it in the sub-transaction link.
[0015] As an optional implementation, the method further includes:
[0016] If, in the transaction data of each business node, the downstream association key of the fourth transaction data is the same as the upstream association key of multiple other transaction data, and the business node to which the multiple other transaction data belongs is the downstream business node adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link, then it is determined that there is a duplicate transaction in the downstream business node adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link.
[0017] As an optional implementation, before performing reconciliation processing on the transaction data contained in the transaction reconciliation group, the method further includes:
[0018] Based on pre-stored standard parameter conversion rules, the parameter fields and parameter values in the transaction data contained in the transaction reconciliation group are converted into standard parameter fields and standard parameter values, respectively.
[0019] Secondly, a method for generating transaction data is provided, the method being applied to a business subsystem, the method comprising:
[0020] Receive the first business request, and perform business operations based on the first business request to generate the first transaction data;
[0021] If the first business request carries a first association key, then the first association key is used as the upstream association key in the first transaction data;
[0022] Generate a second association key, and use the second association key as the downstream association key in the first transaction data;
[0023] A second business request is sent to the downstream business subsystem. The second business request carries the second association key. The second association key carried in the second business request is used by the downstream business subsystem as the upstream association key in the generated second transaction data. The downstream association key in the first transaction data and the upstream association key in the second transaction data are used to enable the reconciliation system to divide the first transaction data and the second transaction data into a transaction reconciliation group for reconciliation processing.
[0024] Thirdly, a transaction data reconciliation device is provided, the device being used in a reconciliation system, the device comprising:
[0025] The splitting module is used to obtain the directed acyclic DAG graph of the transaction links of the payment system and split the transaction link DAG graph into multiple sub-transaction links;
[0026] The grouping module is used to obtain the transaction data of each business node in each sub-transaction link, and divide each transaction data into multiple transaction reconciliation groups according to the association key in each transaction data; the association key includes upstream association key and downstream association key;
[0027] The reconciliation module is used to reconcile the transaction data contained in each transaction reconciliation group.
[0028] As an optional implementation, the grouping module specifically includes:
[0029] In the transaction data of each business node, if the upstream association key in the first transaction data is the same as the downstream association key in the second transaction data, and the business node to which the first transaction data belongs is the downstream business node adjacent to the business node to which the second transaction data belongs in the sub-transaction link, then the first transaction data and the second transaction data are divided into a transaction reconciliation group.
[0030] As an optional implementation, the device further includes:
[0031] The first determining module is used to determine that if the downstream association key of the third transaction data is different from the upstream association key in other transaction data, the business node to which the third transaction data belongs has a shortfall in the adjacent downstream business node in the sub-transaction link.
[0032] As an optional implementation, the device further includes:
[0033] The second determining module is used to determine that, in the transaction data of each business node, if the downstream association key of the fourth transaction data is the same as the upstream association key of multiple other transaction data, and the business node to which the multiple other transaction data belongs is the downstream business node adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link, then the downstream business node adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link has a duplicate account situation.
[0034] As an optional implementation, the device further includes:
[0035] The conversion module is used to convert the parameter fields and parameter values in the transaction data contained in the transaction reconciliation group into standard parameter fields and standard parameter values, respectively, based on pre-stored standard parameter conversion rules.
[0036] Fourthly, a device for generating transaction data is provided, the device being applied to a business subsystem, the device comprising:
[0037] The generation module is used to receive a first business request, and perform business operations based on the first business request to generate first transaction data;
[0038] The first determining module is used to use the first association key as the upstream association key in the first transaction data if the first business request carries a first association key.
[0039] The second determining module is used to generate a second association key and use the second association key as the downstream association key in the first transaction data;
[0040] The sending module is used to send a second business request to the downstream business subsystem. The second business request carries the second association key, which is used by the downstream business subsystem as the upstream association key in the generated second transaction data. The downstream association key in the first transaction data and the upstream association key in the second transaction data are used to enable the reconciliation system to divide the first transaction data and the second transaction data into a transaction reconciliation group for reconciliation processing.
[0041] Fifthly, a payment system is provided, the payment system comprising: a reconciliation system and multiple business subsystems; wherein the reconciliation system performs the method as described in any one of the first aspects, and the multiple business subsystems perform the method as described in the second aspect.
[0042] In a sixth aspect, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the method described in the first aspect.
[0043] In a seventh aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.
[0044] Eighthly, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to perform the steps of the method described in the second aspect.
[0045] In a ninth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the second aspect.
[0046] This application provides a method, apparatus, and payment system for reconciling and generating transaction data. The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:
[0047] This invention provides a universal reconciliation solution for large-scale payment systems. Through flexible configuration, it can adapt to various complex transaction chains and different return time characteristics, thereby solving the complex reconciliation problems within large-scale payment systems.
[0048] This invention allows for flexible configuration of reconciliation rules and parameters based on different business scenarios and needs, ensuring that the reconciliation system can accurately detect all problematic transactions, including those with missing balances, inconsistent statuses, and discrepancies in amounts. Furthermore, this solution successfully addresses the internal reconciliation challenges of many large-scale payment systems.
[0049] This application allows transaction data to not support globally unique transaction identifiers; by splitting transaction links, this application can adapt to the internal transaction links of large payment systems with complex topologies, supporting transaction links with forks and intersections; it supports the verification of transactions with various reconciliation timeframes, such as millisecond reconciliation, cross-hour and cross-day reconciliation, and cross-month reconciliation; it does not require development support from business subsystems, nor does it require business lines to write and generate unified reconciliation files, and can perform non-intrusive reconciliation through configuration; it can detect problems such as duplicate accounts caused by system failures and operational errors of maintenance personnel; the reconciliation system does not require the configuration of a specified database, and only needs to obtain reconciliation data through "table ID" information.
[0050] This application enables the payment system to conduct internal reconciliation more flexibly and efficiently, and to promptly identify and resolve issues of missing or duplicate transactions. It ensures that all transaction data generated at each stage of the payment system can be fully checked and verified, thereby reducing errors and discrepancies, improving the stability, reliability, and security of the payment system, reducing the cost of manual intervention and error handling, and enhancing overall operational efficiency.
[0051] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 This application provides a schematic diagram of a business subsystem of a transaction link in a large-scale payment system.
[0054] Figure 2 A schematic diagram of a transaction link DAG provided in an embodiment of this application;
[0055] Figure 3 A flowchart illustrating a transaction data reconciliation method provided in this application embodiment;
[0056] Figure 4 A schematic diagram of a sub-transaction link provided in an embodiment of this application;
[0057] Figure 5 A schematic diagram illustrating transaction data association provided in an embodiment of this application;
[0058] Figure 6 A schematic diagram illustrating a transaction data conversion method provided in an embodiment of this application;
[0059] Figure 7 A schematic diagram of transaction data provided in an embodiment of this application;
[0060] Figure 8 A flowchart illustrating a method for generating transaction data provided in this application embodiment;
[0061] Figure 9 A schematic diagram of the structure of a transaction data reconciliation device provided in this application embodiment;
[0062] Figure 10This is a schematic diagram of the structure of a transaction data generation device provided in an embodiment of this application;
[0063] Figure 11 This application provides a schematic diagram of the structure of a payment system according to an embodiment of the present application.
[0064] Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0066] The transaction data reconciliation method provided in this application can be applied to large-scale payment systems, and further, to the reconciliation system within such systems. Currently, large-scale payment systems have numerous internal business subsystems. Processing a single business request often requires navigating through many subsystems, such as order processing services, payment centers, financial gateways, clearing, and accounting. Each subsystem generates transaction data during transaction processing, and this data is typically stored in a relational database. Figure 1 This is a schematic diagram of a business subsystem of a transaction link in a large-scale payment system provided as an embodiment of this application. For example... Figure 1 As shown, after the payment gateway subsystem receives the business request from the merchant system, it performs a series of security checks and forwards the request to the corresponding payment acquiring subsystem. The payment acquiring subsystem accepts the request, records the relevant transaction data in its database, processes it, and then forwards the request to the corresponding transaction center subsystem. The transaction center subsystem records the request information and requests the financial gateway subsystem and clearing subsystem to process the payment and prepare for subsequent deposit and clearing processes. The clearing subsystem records the relevant transaction data in its database, processes it, and then forwards the request to the accounting subsystem and withdrawal gateway subsystem for further processing.
[0067] The above transaction scenario depicts a situation where the transaction chain branches. There may be a lot of mutual conversion between different businesses, and there may also be situations where the chains intersect. For example, a business request handled by a business subsystem in the middle layer may come from several order handling subsystems.
[0068] To address the complexity of the transaction link topology within a large payment system, this application abstracts the transaction link structure of the business request path into a DAG (Directed Acyclic Graph). For example... Figure 2 As shown in the diagram, nodes A through I represent tables in the database. Transaction data in these tables is written by the corresponding business subsystems. Therefore, the reconciliation problem is transformed into how to correlate and verify the transaction data of the business nodes in the transaction chain DAG diagram.
[0069] The following will describe in detail a transaction data reconciliation method provided in this application embodiment, with reference to specific implementation methods. This method is applied to a reconciliation system. Figure 3 A flowchart of a transaction data reconciliation method provided in this application embodiment is shown below. Figure 3 As shown, the specific steps are as follows:
[0070] Step 301: Obtain the DAG graph of the transaction chain of the payment system, and split the DAG graph of the transaction chain into multiple sub-transaction chains.
[0071] In implementation, to simplify the complex transaction chain, this application can decompose the complex transaction chain DAG graph into multiple simple sub-transaction chains. For example, Figure 2 As shown, a complex transaction chain DAG graph refers to a transaction chain DAG graph with business nodes having an out-degree or in-degree greater than 1, i.e., there are chain forks or chain intersections. The out-degree is the number of edges originating from a single business node, and the in-degree is the number of edges pointing to a particular business node. For example, Figure 2 In the transaction chain DAG graph, business node D has an in-degree of 2, indicating a link intersection; business node E has an out-degree of 2, indicating a link branch; business node F has an out-degree of 2, indicating a link branch; and business node G has an in-degree of 2, indicating a business intersection. When splitting the transaction chain DAG graph, the reconciliation system can first determine the splitting nodes (i.e., business nodes with an out-degree or in-degree greater than 1) in the transaction chain DAG graph. Then, based on the first and last business nodes and the splitting business nodes, the transaction chain DAG graph is split into multiple sub-transaction chains. In each sub-transaction chain, the out-degree or in-degree of the business nodes can still be greater than 1. For example, a sub-transaction chain includes business nodes A, B, C, and D. Of course, the in-degree and out-degree of the business nodes in each sub-transaction chain can both be less than or equal to 1. Each sub-transaction chain is as follows: Figure 4 As shown, Figure 4 In each transaction link, the in-degree and out-degree of each business node are no greater than 1. The reconciliation system splits the transaction link DAG graph as follows: traverse all business nodes in the transaction link DAG graph. If the out-degree of a business node is greater than 1, then split the business node into multiple sub-transaction links. If the in-degree of a business node is greater than 1, then the business node is shared by multiple sub-transaction links, that is, each sub-transaction link must contain the business node.
[0072] by Figure 4 Taking the sub-transaction chain "A→B→D" as an example, business requests sequentially generate transaction data at business nodes A, B, and D. The business node that generates transaction data first is called the upstream business node, and the business node that generates transaction data later is called the downstream business node. In this example, business node A is an upstream business node relative to business node B, and business node D is a downstream business node relative to business node B. For the split sub-transaction chain, transaction data can be associated and reconciled between adjacent business nodes. In this example, transaction data is associated and reconciled between business nodes A and B, and between business nodes B and D.
[0073] Step 302: For each sub-transaction link, obtain the transaction data of each business node in that sub-transaction link, and divide each transaction data into multiple transaction reconciliation groups according to the association keys in each transaction data. The association keys include upstream association keys and downstream association keys.
[0074] In implementation, transaction reconciliation involves verifying two or more transaction data to determine if the transaction status and transaction amount are consistent. Existing technologies typically add a "globally unique transaction identifier" to all transaction data across all business nodes in the business request path, using this identifier to link all transaction data for reconciliation. This application decomposes the transaction link DAG into multiple sub-transaction links. Each sub-transaction link only needs to associate the transaction data of adjacent upstream and downstream business nodes, eliminating the need for a "globally unique transaction identifier." Figure 4 Taking the sub-transaction link "A→B→D" as an example, it is necessary to associate the transaction data of business node A with the transaction data of business node B, and associate the transaction data of business node B with the transaction data of business node D, without having to associate the transaction data of business node A, business node B and business node D using a "globally unique transaction identifier".
[0075] To link transaction data between adjacent upstream and downstream business nodes in a sub-transaction chain, this application adds link key information, including upstream and downstream link keys, to the transaction data. Business requests propagate along the transaction chain in a large payment system, with upstream business nodes typically receiving the request before downstream business nodes. Therefore, transaction data for the same business request is usually generated upstream before downstream business node transaction data. Thus, downstream business nodes can use a value generated by the upstream business node as a link key, linking the transaction data of the same business request between upstream and downstream business nodes. The link key connecting the downstream business node to the upstream business node is called the upstream link key (Link-From Key), and the link key connecting the upstream business node to the downstream business node is called the downstream link key (Link-To Key). The process of adding upstream and downstream link keys to the transaction data by the business subsystem will be described in detail later and will not be repeated here.
[0076] like Figure 5 As shown, still based on Figure 4 Taking the sub-transaction link "A→B→D" as an example, from the perspective of business request acceptance, the business request on the sub-transaction link starts from the head business node and propagates along the link to the downstream business node. However, from the perspective of transaction data association, the association direction is from the transaction data of the downstream business node to the downstream association key (Link-To Key) in the transaction data of the upstream business node through the upstream association key (Link-From Key).
[0077] Large-scale payment systems involve a wide range of business operations and a large volume of transaction data. Each business node corresponds to at least one table in a database, and each table contains a significant amount of data. This is especially true when a large-scale payment system has numerous transaction links, where the overall data volume can be enormous. To address the challenges of big data processing, this application employs the mature Hadoop technology (a distributed system infrastructure technology). This technology is based on the MapReduce programming model (a programming model for large-scale datasets). To adapt sub-transaction link reconciliation to Hadoop's MapReduce programming model, the reconciliation in this embodiment is performed on a per-configured sub-transaction link basis during the Map phase. For each sub-transaction link, the reconciliation system can obtain the transaction data of each business node within that sub-transaction link within the same time window and divide the transaction data into multiple transaction reconciliation groups based on the association keys in each transaction data. The transaction data contained in a transaction reconciliation group constitutes the transaction data requiring reconciliation. The process of dividing the transaction data into multiple transaction reconciliation groups based on the association keys in each transaction data will be described in detail later and will not be repeated here.
[0078] Step 303: For each transaction reconciliation group, perform reconciliation processing on the transaction data contained in that transaction reconciliation group.
[0079] In practice, after the reconciliation system divides each transaction data into multiple transaction reconciliation groups, it can further reconcile the transaction data contained in each transaction reconciliation group to determine whether the transaction status and transaction amount are consistent.
[0080] Optionally, before performing reconciliation processing on the transaction data contained in the transaction reconciliation group, the reconciliation system can also convert the parameter fields and parameter values in the transaction data contained in the transaction reconciliation group into standard parameter fields and standard parameter values respectively based on pre-stored standard parameter conversion rules.
[0081] In implementation, the naming of parameter fields in tables across different business subsystems within a large payment system may be inconsistent. For example, the amount field might be named "AMOUNT" in some tables and "AMT" in others. Similarly, the definitions of transaction status may differ. Some tables define transaction data status as only "processing," "success," and "failure," while others may have dozens of different statuses. To address these inconsistencies, this application performs data conversion on parameter fields and values in transaction data. All transaction data in all tables must be uniformly converted to a standard data format according to pre-configured standard parameter conversion rules. Table 1 provides examples of some key fields in a standard parameter conversion rule provided in an embodiment of this application.
[0082] Standard parameter fields illustrate SYSTEMNO System serial number, i.e., the primary key of a transaction record. SYSTEMTIME System time, i.e., the time when the transaction record was generated. TXTYPE Transaction type coding STATUS Transaction status is limited to "Processing," "Successful," and "Failed." AMOUNT Transaction amount (unit: cents)
[0083] like Figure 6 As shown, the parameter fields and values in the table on the left do not conform to the standard data format, while the table on the right shows the standard parameter fields and values after data conversion. The reconciliation system can be configured with standard parameter conversion rules as shown in Table 2.
[0084] Table 2
[0085]
[0086]
[0087] Among them, “SYSTEMNO”, “SYSTEMTIME”, “TXTYPE” and “AMOUNT” are simple parameter field mappings, while “STATUS” is a rule-based mapping, that is, the value 20 is mapped to the “successful” transaction status, the values 30, 31 and 32 are mapped to the “failed” transaction status, and the remaining values are automatically mapped to the “processing” transaction status.
[0088] Optionally, the reconciliation system divides each transaction data into multiple transaction reconciliation groups based on the association key in each transaction data as follows: In the transaction data of each business node, if the upstream association key in the first transaction data is the same as the downstream association key in the second transaction data, and the business node to which the first transaction data belongs is the downstream business node adjacent to the business node to which the second transaction data belongs in the sub-transaction link, then the first transaction data and the second transaction data are divided into a transaction reconciliation group.
[0089] In implementation, this application uses SEQ to identify the relative position of each service node on the sub-transaction link. SEQ is a sequential positive integer. The SEQ of the first service node is specified as 1, and the SEQs of subsequent service nodes increment sequentially. Let's take... Figure 4 Taking the sub-transaction link "A→B→D" as an example, business node A has SEQ=1, business node B has SEQ=2, and business node D has SEQ=3.
[0090] like Figure 7 As shown, still based on Figure 4 Taking the sub-transaction link "A→B→D" as an example, the reconciliation system obtains three transaction data from business node A (SEQ=1), business node B (SEQ=2), and business node D (SEQ=3) respectively in the same time window. Figure 7 Each row below each business node represents a transaction data point retrieved from that business node. The Link-From Key in the transaction data of a downstream business node is the same as the Link-To Key in the transaction data of an upstream business node. A business request can be traced back along its path in the sub-transaction chain using the same association key between upstream and downstream business nodes. This fact can be utilized in the Map phase of MapReduce.
[0091] In the Map phase, the reconciliation system reads every transaction from each business node and uses the association key as the Map Key. The output Map Value consists of the transaction data itself, the business node's SEQ (Search Engine Number), and the association key types (i.e., upstream and downstream association keys). For intermediate business nodes, the Map phase outputs two Map Values for each transaction, one using the Link-From Key and the other using the Link-To Key. One Map Value matches the transaction data from the upstream business node with the Link-From Key, while the other Map Value matches the transaction data from the downstream business node with the Link-To Key. The data format output in the Map phase is shown in Table 3.
[0092] Table 3
[0093]
[0094] The MapReduce process will group identical key values together for processing. However, since the Link-From Key and Link-To Key in the transaction data of intermediate business nodes can be the same or different, the grouping of specific transaction reconciliation groups can be divided into the following three cases.
[0095] Scenario 1: The Link-From Key and Link-To Key values in the transaction data of each intermediate business node are different. Figure 7 Taking the first transaction data in the sub-transaction chain "A→B→D" as an example, as shown in Table 4, the grouping results of the MapReduce stage can be divided into 2 transaction reconciliation groups based on the Map Key.
[0096] Table 4
[0097]
[0098] In each transaction reconciliation group, transaction data appears in pairs, meaning that transaction data from two business nodes with a SEQ difference of 1 can be matched into a pair.
[0099] Scenario 2: If the Link-From Key and Link-To Key values of a certain intermediate business node are the same, then... Figure 7Taking the first transaction data in the sub-transaction link "A→B→D" as an example, assuming that the Link-From Key and Link-To Key values in the transaction data of business node B are f361cedb, as shown in Table 5, the grouping results of the Map Reduce stage can be divided into 1 transaction reconciliation group based on the Map Key.
[0100] Table 5
[0101]
[0102] Whether it's one transaction reconciliation group or two transaction reconciliation groups, the transaction data of two adjacent business nodes with a SEQ difference of 1 always appear in pairs.
[0103] Scenario 3: If the Link-From Key and Link-To Key values are the same for all business nodes, then... Figure 7 Taking the first transaction data in the sub-transaction link "A→B→D" as an example, assuming that the Link-From Key and Link-To Key values in the transaction data of business nodes A, B, and D are all f361cedb, as shown in Table 6, the grouping results of the MapReduce stage can be divided into 1 transaction reconciliation group based on the Map Key.
[0104] Table 6
[0105]
[0106] Similarly, whether it is a single transaction reconciliation group or two transaction reconciliation groups, the transaction data of two adjacent business nodes with a SEQ difference of 1 also appear in pairs.
[0107] As an optional implementation, if the downstream association key of the third transaction data is different from the upstream association key of other transaction data in the transaction data of each business node, it is determined that the business node to which the third transaction data belongs has a shortfall in the downstream business node adjacent to it in the sub-transaction link.
[0108] In implementation, if the downstream association key of the third transaction data is different from the upstream association key in all other transaction data, it indicates that there is no transaction data that can match the third transaction data. In this case, the reconciliation system can determine that the business node to which the third transaction data belongs has a shortfall in its adjacent downstream business node in that sub-transaction link.
[0109] As an optional implementation, in the transaction data of each business node, if the downstream association key of the fourth transaction data is the same as the upstream association key of multiple other transaction data, and the business nodes to which the multiple other transaction data belong are the downstream business nodes adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link, then it is determined that there is a duplicate transaction in the downstream business nodes adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link.
[0110] In implementation, if the downstream association key of the fourth transaction data is the same as the upstream association key of multiple other transaction data in the transaction data of each business node, and the business nodes to which the multiple other transaction data belong are the downstream business nodes adjacent to the business node to which the fourth transaction data belongs in that sub-transaction link, it indicates that there are multiple transaction data reconciling with the fourth transaction data. In this case, the reconciliation system can determine that there is duplicate accounting in the downstream business nodes adjacent to the business node to which the fourth transaction data belongs in that sub-transaction link.
[0111] In summary, by screening the Map grouping results during the Map Reduce phase, abnormal transactions such as missing or duplicate transactions can be detected. At the same time, the Map Reduce phase also checks the transaction amount and transaction status of the paired transaction data of adjacent business nodes to detect discrepancies in transaction amount and transaction status.
[0112] As an optional implementation method, such as Figure 8 As shown, this application also provides a method for generating transaction data, which is applied to a business subsystem, and the specific processing steps are as follows.
[0113] Step 801: Receive the first business request, and perform business operations according to the first business request to generate the first transaction data.
[0114] In implementation, when a business subsystem receives a business request (i.e., the first business request), it executes the business operation according to the first business request and generates the first transaction data. Afterwards, the business subsystem stores the transaction data in the database.
[0115] Step 802: If the first business request carries a first association key, then the first association key is used as the upstream association key in the first transaction data.
[0116] In implementation, after receiving the first business request, the business subsystem can determine whether the first business request carries a first association key. If the first business request carries a first association key, it indicates that the first business request was sent by an upstream business subsystem, which is either an intermediate or tail business subsystem. In this case, to associate transaction data, the business subsystem can use the first association key as the upstream association key in the first transaction data. If the first business request does not carry a first association key, it indicates that the first business request was sent by the customer system, which is a head business subsystem. In this case, the business subsystem does not need to operate on the upstream association key in the transaction data.
[0117] Step 803: Generate a second association key and use the second association key as the downstream association key in the first transaction data.
[0118] In implementation, regardless of whether the business subsystem is a head business subsystem or an intermediate business subsystem, after generating the first transaction data, in order to establish a connection with the transaction data of downstream business subsystems, the business subsystem can generate a second association key and use the second association key as the downstream association key in the first transaction data. It should be noted that if the business subsystem is a tail business subsystem, it is not necessary to generate a second association key.
[0119] Step 804: Send a second business request to the downstream business subsystem. The second business request carries a second association key. The second association key carried in the second business request is used by the downstream business subsystem as an upstream association key in the generated second transaction data. The downstream association key in the first transaction data and the upstream association key in the second transaction data are used to enable the reconciliation system to divide the first transaction data and the second transaction data into a transaction reconciliation group for reconciliation processing.
[0120] In implementation, after processing the first business request, the business subsystem can send a second business request to the downstream business subsystem. This second business request carries a second association key. This second association key is used by the downstream business subsystem as the upstream association key in the generated second transaction data. In this way, the reconciliation system can group the first and second transaction data into a single reconciliation group based on the downstream association key in the first transaction data and the upstream association key in the second transaction data, and then perform reconciliation processing.
[0121] In this application, the transaction data reconciliation process is mainly divided into three stages: data synchronization, data conversion, and data reconciliation.
[0122] Firstly, in the data synchronization phase, a data synchronization platform is used to perform data synchronization operations. This platform has a unified operation entry point and can use corresponding interface components for different databases. The data synchronization platform in this application queries the near real-time backup database of the source database periodically, and stores the retrieved data in a pre-defined format on the HDFS distributed file system (a type of Hadoop Distributed File System). Because large-scale payment systems involve massive amounts of data and employ a database sharding and table partitioning scheme, a single logical data table may exist across hundreds of partitioned tables, which may be distributed across multiple physical databases. Therefore, the data platform uses partitioned table query infrastructure technology to read the data. By configuring the partitioned table routing strategy of the large-scale payment system through system parameters, the data synchronization platform only needs to know the name (or table ID) of the logical table that the business depends on to query and synchronize the data of all partitioned tables corresponding to that logical table to HDFS within a specified time window. Once synchronization is complete, the platform automatically sends a notification message to the end-to-end reconciliation system.
[0123] Upon receiving the notification, the reconciliation system converts the raw data into the standard data format to be reconciled, based on pre-configured conversion rules. Simultaneously, the system automatically checks for the existence of data on other nodes related to the data's link within the same time window. If data exists, reconciliation operations are triggered on those links.
[0124] For certain payment transactions, the reconciliation time may be lengthy. For transactions with an average reconciliation time of several minutes or hours, this application supports a retry reconciliation strategy. For transactions where a definitive result cannot be obtained immediately, the reconciliation system will selectively store them temporarily in a distributed in-memory database or HDFS based on the characteristics of the suspicious transaction. When the actual reconciliation execution occurs in a subsequent time window, these transactions will be included in the reconciliation data for retrying. When the number of retries reaches the configured limit, the reconciliation system will determine the suspicious transaction as a discrepancy and record it in the discrepancy table of the relational database. For transactions with reconciliation spanning multiple days, the reconciliation system uses a daily scheduled batch reconciliation method to verify suspicious transactions.
[0125] It should be understood that, although Figure 3 and Figure 8 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 3 and Figure 8At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.
[0126] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.
[0127] This application also provides a transaction data reconciliation device, such as... Figure 9 As shown, the device is used in an accounting system, and the device includes:
[0128] The splitting module 910 is used to obtain the directed acyclic DAG graph of the transaction links of the payment system and split the transaction link DAG graph into multiple sub-transaction links;
[0129] Grouping module 920 is used to obtain the transaction data of each business node in each sub-transaction link, and divide each transaction data into multiple transaction reconciliation groups according to the association key in each transaction data; the association key includes upstream association key and downstream association key;
[0130] The reconciliation module 930 is used to reconcile the transaction data contained in each transaction reconciliation group.
[0131] As an optional implementation, the in-degree and out-degree of each business node in each sub-transaction link are both no greater than 1.
[0132] As an optional implementation, the grouping module 920 specifically includes:
[0133] If the upstream association key in the first transaction data is the same as the downstream association key in the second transaction data, and the business node to which the first transaction data belongs is the downstream business node adjacent to the business node to which the second transaction data belongs in the sub-transaction link, then the first transaction data and the second transaction data will be divided into a transaction reconciliation group.
[0134] As an optional implementation, the device further includes:
[0135] The first determination module is used to determine, in the transaction data of each business node, if the downstream association key of the third transaction data is different from the upstream association key in other transaction data, that the business node to which the third transaction data belongs has a shortfall in the adjacent downstream business node in the sub-transaction link.
[0136] As an optional implementation, the device further includes:
[0137] The second determining module is used to determine that, in the transaction data of each business node, if the downstream association key of the fourth transaction data is the same as the upstream association key of multiple other transaction data, and the business nodes to which the multiple other transaction data belong are the downstream business nodes adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link, then the business node to which the fourth transaction data belongs has a duplicate transaction situation in the downstream business node adjacent to the business node in the sub-transaction link.
[0138] As an optional implementation, the device further includes:
[0139] The conversion module is used to convert the parameter fields and parameter values in the transaction data contained in the transaction reconciliation group into standard parameter fields and standard parameter values, respectively, based on pre-stored standard parameter conversion rules.
[0140] This application also provides a device for generating transaction data, such as... Figure 10 As shown, the device is applied to the business subsystem, and the device includes:
[0141] The generation module 1010 is used to receive the first business request, and perform business operations according to the first business request to generate the first transaction data;
[0142] The first determining module 1020 is used to use the first association key as the upstream association key in the first transaction data if the first business request carries a first association key.
[0143] The second determining module 1030 is used to generate a second association key and use the second association key as a downstream association key in the first transaction data;
[0144] The sending module 1040 is used to send a second business request to the downstream business subsystem. The second business request carries a second association key, which is used by the downstream business subsystem as the upstream association key in the generated second transaction data. The downstream association key in the first transaction data and the upstream association key in the second transaction data are used to enable the reconciliation system to divide the first transaction data and the second transaction data into a transaction reconciliation group for reconciliation processing.
[0145] This application also provides a payment system, such as... Figure 11 As shown, the payment system includes: a reconciliation system 1110 and multiple business subsystems 1120; wherein, the reconciliation system 1110 executes the reconciliation method steps for transaction data as described above, and the multiple business subsystems 1120 execute the transaction data generation method steps as described above.
[0146] In one embodiment, a computer device is provided, such as Figure 12 As shown, it includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the above-mentioned transaction data reconciliation method steps.
[0147] In one embodiment, a computer device is provided, such as Figure 12 As shown, it includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the above-mentioned method steps for generating transaction data.
[0148] In one embodiment, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described transaction data reconciliation method.
[0149] In one embodiment, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method for generating transaction data.
[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0151] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0152] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0153] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A method for reconciling transaction data, characterized in that, The method is applied to an accounting system, and the method includes: Obtain the directed acyclic graph (DAG) of the transaction links in the payment system, and then split the DAG into multiple sub-transaction links. For each sub-transaction link, the transaction data of each business node in the sub-transaction link is obtained, and the transaction data is divided into multiple transaction reconciliation groups according to the association key in each transaction data; the association key includes upstream association key and downstream association key; For each transaction reconciliation group, reconcile the transaction data contained in that transaction reconciliation group; The step of dividing each transaction data into multiple transaction reconciliation groups based on the association keys in each transaction data includes: in the transaction data of each business node, if the upstream association key in the first transaction data is the same as the downstream association key in the second transaction data, and the business node to which the first transaction data belongs is the downstream business node adjacent to the business node to which the second transaction data belongs in the sub-transaction link, then the first transaction data and the second transaction data are divided into a transaction reconciliation group.
2. The method according to claim 1, characterized in that, The in-degree and out-degree of each business node in each sub-transaction link are not greater than 1.
3. The method according to claim 1, characterized in that, The method further includes: If the downstream association key of the third transaction data is different from the upstream association key of other transaction data in the transaction data of each business node, it is determined that the business node to which the third transaction data belongs has a shortfall in the downstream business node adjacent to it in the sub-transaction link.
4. The method according to claim 1, characterized in that, The method further includes: If, in the transaction data of each business node, the downstream association key of the fourth transaction data is the same as the upstream association key of multiple other transaction data, and the business node to which the multiple other transaction data belongs is the downstream business node adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link, then it is determined that there is a duplicate transaction in the downstream business node adjacent to the business node to which the fourth transaction data belongs in the sub-transaction link.
5. The method according to claim 1, characterized in that, Before performing reconciliation processing on the transaction data contained in the transaction reconciliation group, the method further includes: Based on pre-stored standard parameter conversion rules, the parameter fields and parameter values in the transaction data contained in the transaction reconciliation group are converted into standard parameter fields and standard parameter values, respectively.
6. A method for generating transaction data, characterized in that, The method is applied to a business subsystem, and the method includes: Receive the first business request, and perform business operations based on the first business request to generate the first transaction data; If the first business request carries a first association key, then the first association key is used as the upstream association key in the first transaction data; Generate a second association key, and use the second association key as the downstream association key in the first transaction data; A second business request is sent to the downstream business subsystem. The second business request carries the second association key. The second association key carried in the second business request is used as the upstream association key in the second transaction data generated by the downstream business subsystem. The downstream association key in the first transaction data and the upstream association key in the second transaction data are used to enable the reconciliation system to divide the first transaction data and the second transaction data into a transaction reconciliation group for reconciliation processing.
7. A transaction data reconciliation device, characterized in that, The device is used in an accounting reconciliation system, and the device includes: The splitting module is used to obtain the directed acyclic graph (DAG) of the transaction links in the payment system and split the DAG into multiple sub-transaction links. The grouping module is used to acquire the transaction data of each business node in each sub-transaction link, and divide the transaction data into multiple transaction reconciliation groups according to the association key in each transaction data; the association key includes an upstream association key and a downstream association key; in the transaction data of each business node, if the upstream association key in the first transaction data is the same as the downstream association key in the second transaction data, and the business node to which the first transaction data belongs is the downstream business node adjacent to the business node to which the second transaction data belongs in the sub-transaction link, then the first transaction data and the second transaction data are divided into a transaction reconciliation group; The reconciliation module is used to reconcile the transaction data contained in each transaction reconciliation group.
8. A device for generating transaction data, characterized in that, The device is applied to a business subsystem, and the device includes: The generation module is used to receive a first business request, and perform business operations based on the first business request to generate first transaction data; The first determining module is used to use the first association key as the upstream association key in the first transaction data if the first business request carries a first association key. The second determining module is used to generate a second association key and use the second association key as the downstream association key in the first transaction data; The sending module is used to send a second business request to the downstream business subsystem. The second business request carries a second association key, which is used as an upstream association key in the second transaction data generated by the downstream business subsystem. The downstream association key in the first transaction data and the upstream association key in the second transaction data are used to enable the reconciliation system to divide the first transaction data and the second transaction data into a transaction reconciliation group for reconciliation processing.
9. A payment system, characterized in that, The payment system includes: a reconciliation system and multiple business subsystems; wherein the reconciliation system performs the method as described in any one of claims 1 to 5, and the multiple business subsystems perform the method as described in claim 6.