Distributed Reconciliation Method, Server, Device and System

Abstract

The present application discloses a distributed reconciliation method, a server, a device and a system. Among them, the method executed by the reconciliation scheduling center includes: obtaining transaction data to be reconciled, where the transaction data includes first transaction data from a transaction institution and second transaction data from a transaction platform; sharding each piece of transaction data according to the first field of each piece of transaction data to obtain a plurality of reconciliation shards; and sending the plurality of reconciliation shards to the reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain a reconciliation result according to the second field of each piece of transaction data in the received reconciliation shards. The beneficial effect is that the transaction data is sent to each reconciliation execution node in the form of reconciliation shards, and the reconciliation execution nodes use the local computing resources of the reconciliation execution nodes to perform reconciliation, rather than occupying the cache of the distributed reconciliation system to perform reconciliation, effectively avoiding excessive occupation of the cache.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular to a distributed reconciliation method, server, device, and system. Background Art

[0002] In a scenario where a trading platform is connected to multiple trading institutions, both the trading institutions and the trading platform will record transactions that occur to meet their own needs. However, for the same transaction, the recording results of the trading institution and the trading platform may be different, and reconciliation is a very important link in this scenario.

[0003] Due to the extremely large amount of transaction data generated by recording, how to effectively and reasonably utilize computing resources, perform reconciliation based on transaction data, and ensure the accuracy of reconciliation is a problem that needs to be solved. Summary of the Invention

[0004] Embodiments of this application provide a distributed reconciliation method, device, and system to at least partially solve the problem of resource allocation in the reconciliation process.

[0005] Embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a first distributed reconciliation method, which is executed by a reconciliation scheduling center and includes: obtaining transaction data to be reconciled, where the transaction data includes first transaction data from a trading institution and second transaction data from a trading platform; slicing each transaction data according to a first field of each transaction data to obtain a plurality of reconciliation slices; and sending the plurality of reconciliation slices to reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain a reconciliation result according to a second field of each transaction data in the received reconciliation slices.

[0007] In some embodiments, in the first distributed reconciliation method, slicing each transaction data according to a first field of each transaction data includes: performing a hash calculation on the first field of each transaction data to obtain a hash value of each transaction data; and slicing each transaction data according to the hash value of each transaction data.

[0008] In some embodiments, in the first distributed reconciliation method, slicing each transaction data according to the hash value of each transaction data includes: using the number of reconciliation execution nodes as the divisor and the hash values of each transaction data as the dividends respectively to perform a modulo operation to obtain a modulo value of each transaction data; and dividing each transaction data with the same modulo value into the same reconciliation slice.

[0009] In some embodiments, in the first distributed reconciliation method, dividing each transaction data with the same modulus value into the same reconciliation shard includes: after obtaining the modulus value of a piece of transaction data, writing the piece of transaction data into the shard directory corresponding to the modulus value; obtaining the corresponding reconciliation shard according to the transaction data under each shard directory.

[0010] In some embodiments, in the first distributed reconciliation method, sending multiple reconciliation shards to the reconciliation execution nodes includes: determining the reconciliation execution nodes corresponding to each reconciliation shard according to the shard identification information of the reconciliation shards; sending each reconciliation shard to the corresponding reconciliation execution node respectively.

[0011] In some embodiments, in the first distributed reconciliation method, the first field includes at least one of the following information: the working day of the trading institution, the identifier of the trading institution, the transaction serial number identifier, and the deposit and withdrawal identifier of the transaction.

[0012] In a second aspect, an embodiment of the present application further provides a second distributed reconciliation method, which is executed by a reconciliation execution node. The method includes: receiving a reconciliation shard sent by a reconciliation scheduling center, where the reconciliation shard is obtained by the reconciliation scheduling center by sharding each transaction data according to the first field of each transaction data; the transaction data includes first transaction data from a trading institution and second transaction data from a trading platform; for each received reconciliation shard, using the computing resources local to this reconciliation execution node, performing a reconciliation task according to the second field of each transaction data in the reconciliation shard to obtain a reconciliation result.

[0013] In some embodiments, in the second distributed reconciliation method, performing a reconciliation task according to the second field of each transaction data in the reconciliation shard includes: removing duplicates from the first transaction data and the second transaction data in the reconciliation shard respectively; performing a reconciliation task according to the second field of each transaction data after duplicate removal.

[0014] In some embodiments, in the second distributed reconciliation method, removing duplicates from the first transaction data and the second transaction data in the reconciliation shard respectively includes: using all the first transaction data or all the second transaction data in the reconciliation shard as the data to be de-duplicated; calculating the similarity between each transaction data in the data to be de-duplicated; if the similarity between two transaction data in the data to be de-duplicated is greater than a first threshold and the serial number identifiers of the two transaction data are the same, then marking one of the two transaction data as duplicate data.

[0015] In some embodiments, in the second distributed reconciliation method, a reconciliation task is performed according to the second fields of the transaction data in the reconciliation slice, and the obtained reconciliation result includes: determining whether there is a difference between the first transaction data and the second transaction data in the reconciliation slice according to the second fields of the transaction data; if there is a difference, generating a reconciliation result corresponding to the difference, otherwise, generating a reconciliation result indicating no error.

[0016] In some embodiments, in the second distributed reconciliation method, determining whether there is a difference between the first transaction data and the second transaction data in the reconciliation slice according to the second fields of the transaction data includes: calculating the similarity between each piece of first transaction data and each piece of second transaction data according to the second fields; if the similarity between a piece of first transaction data and a piece of second transaction data is greater than a second threshold, marking these two pieces of transaction data as a group of successfully reconciled data; if there are several pieces of transaction data in the reconciliation slice that are not marked as successfully reconciled data, determining that there is a difference between the first transaction data and the second transaction data in the reconciliation slice, and the difference is these several pieces of transaction data that are not marked as successfully reconciled data.

[0017] In some embodiments, in the second distributed reconciliation method, the second fields include at least one of the following information: the working day of the trading institution, the identifier of the trading institution, the transaction serial identifier, the transaction sequence identifier, the transaction details list, the transaction amount, the deposit / withdrawal identifier of the transaction, the type of the transaction, the currency used in the transaction, the status of the transaction.

[0018] In some embodiments, the second distributed reconciliation method further includes: writing the reconciliation result into the cache of the reconciliation scheduling center.

[0019] In a third aspect, an embodiment of the present application further provides a distributed reconciliation server, which is applied to a reconciliation scheduling center, and the distributed reconciliation server is used to implement any one of the first distributed reconciliation methods described above.

[0020] In a fourth aspect, an embodiment of the present application further provides a distributed reconciliation device, which is applied to a reconciliation execution node, and the distributed reconciliation device is used to implement any one of the second distributed reconciliation methods described above.

[0021] In a fifth aspect, an embodiment of the present application further provides a distributed reconciliation system, including a reconciliation scheduling center and several reconciliation execution nodes. Among them, the distributed reconciliation server described above is deployed in the reconciliation scheduling center, and the distributed reconciliation device described above is deployed in the reconciliation execution node.

[0022] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores one or more programs. When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute any of the above-mentioned first distributed reconciliation methods; or, the electronic device is caused to execute any of the above-mentioned second distributed reconciliation methods.

[0023] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: After receiving the transaction data, the reconciliation scheduling center distributes the transaction data to each reconciliation execution node in the form of reconciliation shards. The reconciliation execution node uses the local computing resources of the reconciliation execution node to perform reconciliation, rather than occupying the cache of the distributed reconciliation system to perform reconciliation, effectively avoiding excessive occupation of the cache. Moreover, the reconciliation scheduling center processes the transaction data into multiple reconciliation shards, which is conducive to distributing the transaction data to the reconciliation execution nodes. Sharding through the first field has high sharding efficiency and also ensures sharding accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0025] Figure 1 A flowchart showing a distributed reconciliation method according to an embodiment of the present application;

[0026] Figure 2 A flowchart showing another distributed reconciliation method according to an embodiment of the present application;

[0027] Figure 3 A schematic structural diagram of a distributed reconciliation server according to an embodiment of the present application;

[0028] Figure 4 A schematic structural diagram of a distributed reconciliation device according to an embodiment of the present application;

[0029] Figure 5 A schematic structural diagram of a distributed reconciliation system according to an embodiment of the present application;

[0030] Figure 6 A schematic structural diagram of the first electronic device in the embodiments of the present application;

[0031] Figure 7 A schematic structural diagram of the second electronic device in the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To make the objectives, technical solutions, and advantages of this application more clear, the following will clearly and completely describe the technical solutions of this application in combination with specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0033] Distributed reconciliation is a way of performing reconciliation using multiple reconciliation execution nodes, enabling a large amount of transaction data to be distributed to each reconciliation execution node. Each reconciliation execution node only reconciles a part of the transaction data, improving the reconciliation efficiency.

[0034] However, this method requires placing all transaction data in a distributed cache that can be accessed by each reconciliation execution node, and the situation where the distributed cache is often full frequently occurs. Through analysis, it is found that this method not only requires performing reconciliation in the distributed cache but also storing the reconciliation results in the distributed cache.

[0035] Thus, on the one hand, the amount of transaction data is huge, and on the other hand, each reconciliation execution node can perform parallel reconciliation, which also causes the distributed cache to be easily filled up.

[0036] The technical concept of this application is to construct a distributed reconciliation system including a reconciliation scheduling center and multiple reconciliation execution nodes. Instead of directly storing transaction data in the distributed cache, the transaction data is first segmented, and the obtained reconciliation shards are sent to each reconciliation execution node for reconciliation. In this way, the local computing resources of each reconciliation execution node can be utilized without occupying the distributed cache, and the distributed cache can be used only to store the reconciliation results. This realizes the reasonable allocation and utilization of resources and also ensures the reconciliation efficiency.

[0037] The following will, in combination with the drawings, detail the technical solutions provided by each embodiment of this application.

[0038] Figure 1 The flowchart shows a distributed reconciliation method according to an embodiment of this application, and this distributed reconciliation method can be executed by the reconciliation scheduling center in the distributed reconciliation system. As Figure 1 shown, this method includes:

[0039] Step S110, obtaining the transaction data to be reconciled, where the transaction data includes the first transaction data from the transaction institution and the second transaction data from the transaction platform.

[0040] The reconciliation process is the process of checking the first transaction data from the trading institution and the second transaction data from the trading platform. A specific scenario is that the trading platform can connect with multiple trading institutions as an intermediary to form a structure centered on the trading platform.

[0041] In the embodiments of the present application, the trading platform can be a transfer platform that only performs transaction transfer processing, or a clearing platform that clears the transaction data generated by the transaction, or can have the functions of both types of platforms, etc., and the present application does not limit this. In addition, the trading institution can be either a transaction initiating institution or a transaction receiving institution.

[0042] For a transaction, the transaction institution involved in the transaction (transaction initiator and / or transaction recipient) will generate the first transaction data of the transaction, and the transaction platform will generate the second transaction data of the transaction. In theory, in this scenario, the transaction information such as the transaction amount contained in the first transaction data and the second transaction data should be completely consistent. However, in reality, due to various reasons, there may be inconsistencies between the first transaction data and the second transaction data. Reconciliation is to screen out these inconsistencies.

[0043] Step S120: shard each transaction data according to the first field of each transaction data to obtain a plurality of reconciliation shards.

[0044] Transaction data can show various types of transaction information, such as the working day of the transaction institution, the identification of the transaction institution, the transaction flow identification, the transaction amount, the deposit and withdrawal identification of the transaction, the type of transaction, the currency used in the transaction, the status of the transaction, etc. Specifically, the transaction data may include multiple fields, each of which may show one of the aforementioned transaction information.

[0045] In the embodiments of the present application, the first field is a general term for the fields used to distinguish each transaction data. In other words, without considering the existence of duplication in transaction data, the first fields of each transaction data should not be completely the same. The not completely the same here means that if the first field only includes one field, then the first fields of any two transaction data are completely different; if the first field includes multiple fields, then the first fields of any two transaction data are spliced ​​based on these multiple fields, and the splicing results are also different, but several of the fields used therein may be the same, for example, the transaction identifier may be the same, but the amount is different.

[0046] In some embodiments, the first field includes at least one of the following information: the working day of the trading institution, the identification of the trading institution, the transaction flow identification, and the transaction deposit and withdrawal identification.

[0047] For example, in a case where the transaction serial identifier can represent the uniqueness of transaction data (for example, the transaction volume is small), the first field may be a field showing the transaction serial identifier.

[0048] Sharding the transaction data according to the first field can ensure that one piece of transaction data will not be divided into multiple reconciliation shards, avoiding duplicate reconciliation.

[0049] Step S130: Send multiple reconciliation shards to the reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain a reconciliation result according to the second fields of the transaction data in the received reconciliation shards.

[0050] Among them, the reconciliation execution node can be a physical machine, or a virtual machine such as a JVM (Java Virtual Machine), and can also be implemented using technologies such as containers.

[0051] It can be seen that Figure 1 In the method shown, after receiving the transaction data, the transaction data is sent to each reconciliation execution node in the form of reconciliation shards, and the reconciliation execution node uses the local computing resources of the reconciliation execution node to perform reconciliation, rather than occupying the cache of the distributed reconciliation system to perform reconciliation, effectively avoiding excessive occupation of the cache. Moreover, the reconciliation scheduling center processes the transaction data into multiple reconciliation shards, which is conducive to distributing the transaction data to the reconciliation execution nodes. Sharding through the first field has high sharding efficiency and also ensures sharding accuracy.

[0052] In some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation scheduling center, sharding the transaction data according to the first field of each transaction data includes: performing a hash calculation on the first field of each transaction data to obtain a hash value of each transaction data; sharding each transaction data according to the hash value of each transaction data.

[0053] Although the first field can uniquely identify transaction data, it may not be in a neat format. For example, the transaction amount of one transaction is 300 yuan, and the transaction amount of another transaction is 0.5 yuan. Then the string lengths formed by the first fields of these two pieces of data may not be equal. Therefore, directly sharding according to the first field is not appropriate in some scenarios, and this problem can be overcome by performing a hash operation.

[0054] In some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation scheduling center, sharding each transaction data according to the hash value of each transaction data includes: using the number of reconciliation execution nodes as the divisor, and using the hash values of each transaction data as the dividends respectively to perform a modulo operation to obtain a modulo value of each transaction data; dividing the transaction data with the same modulo value into the same reconciliation shard.

[0055] For example, if the number of reconciliation execution nodes is 8, then the modulus values for modulo operation include 0, 1, 2, 3, 4, 5, 6, 7, a total of eight kinds, and eight reconciliation shards can be obtained. Each reconciliation shard can be assigned to a reconciliation execution node.

[0056] In some other embodiments, the divisor can also be selected as an integer multiple of the number of reconciliation execution nodes. For example, if it is 16, then 16 reconciliation shards can be obtained, and each reconciliation execution node can be assigned two reconciliation shards. In still some other embodiments, the divisor can be set arbitrarily according to requirements. However, correspondingly, the process of assigning reconciliation shards to each reconciliation execution node will become cumbersome.

[0057] In some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation scheduling center, dividing each transaction data with the same modulus value into the same reconciliation shard includes: after obtaining the modulus value of a piece of transaction data, writing this piece of transaction data into the shard directory corresponding to the modulus value; obtaining the corresponding reconciliation shard according to the transaction data under each shard directory.

[0058] For example, set 8 shard directories such as index0, index1... index7. index0 is used to store transaction data with a modulus value of 0, and in this way, 8 reconciliation shards can be obtained very conveniently.

[0059] In some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation scheduling center, sending multiple reconciliation shards to the reconciliation execution nodes includes: determining the reconciliation execution nodes corresponding to each reconciliation shard according to the shard identification information of the reconciliation shard; sending each reconciliation shard to the corresponding reconciliation execution node respectively.

[0060] For example, in the foregoing embodiment, the "0" in the shard directory index0 can be used as the shard identification information of the reconciliation shard. Further, the corresponding reconciliation shard can be sent to the reconciliation execution node node0.

[0061] Figure 2 The flowchart of another distributed reconciliation method according to an embodiment of the present application is shown. This distributed reconciliation method is executed by the reconciliation execution node. As Figure 2 shown, this method includes:

[0062] Step S210, receiving the reconciliation shards sent by the reconciliation scheduling center. The reconciliation shards are obtained by the reconciliation scheduling center through sharding each transaction data according to the first field of each transaction data. The transaction data includes the first transaction data from the transaction institution and the second transaction data from the transaction platform.

[0063] Among them, the introduction of the reconciliation shards, the transaction data required for reconciliation, and the reconciliation principle can refer to the foregoing embodiments and will not be elaborated herein.

[0064] Step S220: For each received reconciliation shard, use the computing resources local to this reconciliation execution node to perform a reconciliation task based on the second field of each transaction data in the reconciliation shard to obtain a reconciliation result.

[0065] Among them, the second field in the transaction data is used for reconciliation, which can be determined according to actual reconciliation requirements. For example, in some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation execution node, the second field includes at least one of the following information: the working day of the trading institution, the identifier of the trading institution, the transaction serial number, the transaction sequence identifier, the transaction details list, the transaction amount, the in / out fund identifier of the transaction, the transaction type, the currency used in the transaction, and the transaction status.

[0066] For example, in an actual business scenario, the following three items have different reconciliation requirements:

[0067] Item 1: Reconcile the working day of the trading institution, the identifier of the trading institution, the transaction serial number, and the transaction amount.

[0068] Item 2: Reconcile the transaction serial number, the identifier of the trading institution, the in / out fund identifier of the transaction, the transaction type, the currency of the transaction, and the transaction amount.

[0069] Item 3: Reconcile the transaction status, the transaction serial number, the transaction details list, and the transaction status.

[0070] Then, the second field can be determined accordingly according to business requirements, so as to perform the reconciliation task.

[0071] It can be seen that Figure 2 The method shown, in cooperation with the method executed by the foregoing reconciliation scheduling center, uses the computing resources local to the reconciliation execution node to perform the reconciliation task, and can select the second field accordingly according to business requirements, with low coupling with the business scenario and more flexible and accurate reconciliation.

[0072] In some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation execution node, performing the reconciliation task based on the second field of each transaction data in the reconciliation shard includes: removing duplicates from the first transaction data and the second transaction data in the reconciliation shard respectively; performing the reconciliation task based on the second field of each de-duplicated transaction data.

[0073] As mentioned in the foregoing embodiments, without considering the duplication of transaction data, the first fields of each piece of transaction data should not be exactly the same. In actual business scenarios, due to reasons such as network jitter, there may be transaction data with the same first field, or even all fields being exactly the same. Two pieces of transaction data with the same first field actually correspond to the same transaction, that is, duplicate accounting is formed. If deduplication is not performed during reconciliation, unnecessary anomalies may be shown in the reconciliation result.

[0074] In some embodiments, in the distributed reconciliation method executed by the reconciliation execution node, the deduplication of the first transaction data and the second transaction data in the reconciliation slice respectively includes: taking all the first transaction data or all the second transaction data in the reconciliation slice as the data to be deduplicated; calculating the similarity between each piece of transaction data in the data to be deduplicated; if the similarity between two pieces of transaction data in the data to be deduplicated is greater than the first threshold and the flow identifiers of the two pieces of transaction data are the same, then mark one of the two pieces of transaction data as duplicate data.

[0075] In the embodiments of the present application, a method for deduplicating the first transaction data and the second transaction data respectively is proposed. This is because for the same transaction, under normal circumstances, there must be corresponding records in the first transaction data and the second transaction data. Otherwise, it means that the transaction platform or transaction institution has missed this transaction, and a reconciliation result with differences should be obtained. Therefore, only the first transaction data and the second transaction data can be deduplicated respectively. Otherwise, the transaction data that should be reconciled will be removed as duplicate data, affecting the accuracy of reconciliation.

[0076] When deduplicating, the similarity between transaction data can be calculated at a relatively coarse granularity first. Only after obtaining pairs of transaction data with a relatively high similarity, a fine-grained judgment is made. Thus, the deduplication efficiency can be improved.

[0077] Generally, the flow identifier can uniquely identify each transaction. Therefore, when the similarity between two pieces of transaction data is greater than the first threshold and the flow identifiers of the two pieces of transaction data are the same, it can be considered that duplicate data is found, and one of them can be deleted, or one of them can be marked as not available for subsequent reconciliation.

[0078] In some embodiments, in the distributed reconciliation method executed by the reconciliation execution node, performing the reconciliation task according to the second field of each piece of transaction data in the reconciliation slice and obtaining the reconciliation result includes: determining whether there are differences between the first transaction data and the second transaction data in the reconciliation slice according to the second field of each piece of transaction data; if there are differences, generating a reconciliation result corresponding to the differences, otherwise, generating a reconciliation result indicating no errors.

[0079] If there is no difference, it means that each item in the first transaction data can be matched one by one with each item in the second transaction data, that is, each transaction recorded in the trading platform also has a corresponding record in the trading institution, and vice versa. In this way, a reconciliation result that is verified without error is obtained. However, if there are differences, for example, the trading platform records a transaction that occurred at a certain point in time, that is, there is a piece of first transaction data a; while no matching second transaction data is found in the trading institution, then this piece of first transaction data a needs to be marked as a difference to obtain a reconciliation result corresponding to the difference.

[0080] Specifically, in some embodiments, in the above-mentioned distributed reconciliation method executed by the reconciliation execution node, to determine whether there are differences between the first transaction data and the second transaction data in this reconciliation slice according to the second field of each transaction data includes: calculating the similarity between each piece of first transaction data and each piece of second transaction data according to the second field; if the similarity between a piece of first transaction data and a piece of second transaction data is greater than the second threshold, then mark these two pieces of transaction data as a group of successfully reconciled data; if there are several pieces of transaction data remaining in this reconciliation slice that are not marked as successfully reconciled data, then it is determined that there are differences between the first transaction data and the second transaction data in this reconciliation slice, and the differences are these several pieces of transaction data that are not marked as successfully reconciled data. Finally, if there are several pieces of transaction data remaining that are not marked as successfully reconciled data, manual reconciliation can also be performed.

[0081] In some embodiments, the above-mentioned distributed reconciliation method executed by the reconciliation execution node further includes: writing the reconciliation result into the cache of the reconciliation scheduling center. This cache can be the distributed cache mentioned above. In addition, the reconciliation result can be further persisted to generate a reconciliation statement, etc.

[0082] Based on the same concept, the embodiments of the present application also provide a distributed reconciliation server, which is applied to the reconciliation scheduling center in the distributed reconciliation system and can be used to implement the above-mentioned distributed reconciliation method executed by the reconciliation scheduling center; and, the embodiments of the present application also provide a distributed reconciliation device, which is applied to the reconciliation execution node in the distributed reconciliation system and can be used to implement the above-mentioned distributed reconciliation method executed by the reconciliation execution node.

[0083] Specifically, Figure 3 shows a schematic structural diagram of a distributed reconciliation server according to an embodiment of the present application, which is applied to the reconciliation scheduling center in the distributed reconciliation system. As Figure 3 shown, the distributed reconciliation server 300 includes:

[0084] An obtaining unit 310, configured to obtain the transaction data to be reconciled, where the transaction data includes the first transaction data from the trading institution and the second transaction data from the trading platform.

[0085] The sharding unit 320 is configured to shard each transaction data according to the first field of each transaction data to obtain multiple reconciliation shards.

[0086] The sending unit 330 is configured to send the multiple reconciliation shards to the reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain a reconciliation result according to the second field of each transaction data in the received reconciliation shards.

[0087] It can be seen that Figure 3 For the server shown, after receiving the transaction data, the transaction data is sent to each reconciliation execution node in the form of reconciliation shards, and the reconciliation execution nodes use the local computing resources of the reconciliation execution nodes to perform reconciliation instead of occupying the cache of the distributed reconciliation system to perform reconciliation, effectively avoiding excessive occupation of the cache. Moreover, the reconciliation scheduling center processes the transaction data into multiple reconciliation shards, which is conducive to distributing the transaction data to the reconciliation execution nodes. Sharding is performed through the first field, which has high sharding efficiency and also ensures sharding accuracy.

[0088] In some embodiments, in the above-mentioned distributed reconciliation server, the sharding unit 320 is configured to perform a hash calculation on the first field of each transaction data to obtain the hash value of each transaction data; and shard each transaction data according to the hash value of each transaction data.

[0089] In some embodiments, in the above-mentioned distributed reconciliation server, the sharding unit 320 is configured to use the number of reconciliation execution nodes as the divisor and the hash values of each transaction data as the dividends respectively to perform a modulo operation to obtain the modulo values of each transaction data; and divide the transaction data with the same modulo value into the same reconciliation shard.

[0090] In some embodiments, in the above-mentioned distributed reconciliation server, the sharding unit 320 is configured to, after obtaining the modulo value of a piece of transaction data, write the piece of transaction data into the sharding directory corresponding to the modulo value; and obtain the corresponding reconciliation shard according to the transaction data in each sharding directory.

[0091] In some embodiments, in the above-mentioned distributed reconciliation server, the sending unit 330 is configured to determine the reconciliation execution nodes corresponding to each reconciliation shard according to the sharding identification information of the reconciliation shards; and send each reconciliation shard to the corresponding reconciliation execution node respectively.

[0092] In some embodiments, the first field includes at least one of the following information: the working day of the trading institution, the identification of the trading institution, the transaction serial number, and the identification of the incoming and outgoing funds of the transaction.

[0093] It can be understood that the above-mentioned distributed reconciliation server can implement each step of the distributed reconciliation method (the first distributed reconciliation method) executed by the reconciliation scheduling center in the foregoing embodiments. The relevant explanations regarding the first distributed reconciliation method are applicable to the distributed reconciliation server and will not be elaborated here.

[0094] Figure 4 FIG. shows a schematic structural diagram of a distributed reconciliation device according to an embodiment of the present application, which is applied to a reconciliation execution node in a distributed reconciliation system. As Figure 4 shown, the distributed reconciliation device 400 includes:

[0095] A receiving unit 410, configured to receive reconciliation shards sent by a reconciliation scheduling center. The reconciliation shards are obtained by the reconciliation scheduling center by sharding each transaction data according to a first field of each transaction data. The transaction data includes first transaction data from a transaction institution and second transaction data from a transaction platform.

[0096] A reconciliation unit 420, configured to, for each received reconciliation shard, use the computing resources local to this reconciliation execution node to execute a reconciliation task according to a second field of each transaction data in the reconciliation shard, and obtain a reconciliation result.

[0097] It can be seen that Figure 4 the device shown cooperates with the method executed by the foregoing reconciliation scheduling center, uses the computing resources local to the reconciliation execution node to execute the reconciliation task, and can, according to service requirements, correspondingly select the second field, with low coupling with the service scenario and more flexible and accurate reconciliation.

[0098] In some embodiments, in the above-mentioned distributed reconciliation device, the reconciliation unit 420 is configured to deduplicate the first transaction data and the second transaction data in the reconciliation shard respectively; and execute a reconciliation task according to the second field of each transaction data after deduplication.

[0099] In some embodiments, in the above-mentioned distributed reconciliation device, the reconciliation unit 420 is configured to use all the first transaction data or all the second transaction data in the reconciliation shard as data to be deduplicated; calculate the similarity between each transaction data in the data to be deduplicated; if the similarity between two transaction data in the data to be deduplicated is greater than a first threshold and the flow identifiers of the two transaction data are the same, then mark one of the two transaction data as duplicate data.

[0100] In some embodiments, in the above-mentioned distributed reconciliation device, the reconciliation unit 420 is configured to determine whether there is a difference between the first transaction data and the second transaction data in the reconciliation shard according to the second field of each transaction data; if there is a difference, generate a reconciliation result corresponding to the difference, otherwise, generate a reconciliation result indicating no error.

[0101] In some embodiments, in the above-mentioned distributed reconciliation device, the reconciliation unit 420 is configured to calculate the similarity between each piece of first transaction data and each piece of second transaction data according to the second field; if the similarity between a piece of first transaction data and a piece of second transaction data is greater than the second threshold, then mark these two pieces of transaction data as a group of successfully reconciled data; if there are several pieces of transaction data in the reconciliation shard that are not marked as successfully reconciled data, then it is determined that there is a difference between the first transaction data and the second transaction data in the reconciliation shard, and the difference is the several pieces of transaction data that are not marked as successfully reconciled data.

[0102] In some embodiments, the second field includes at least one of the following information: the working day of the trading institution, the identifier of the trading institution, the transaction serial number identifier, the transaction sequence identifier, the transaction details list, the transaction amount, the deposit / withdrawal identifier of the transaction, the type of the transaction, the currency used in the transaction, the status of the transaction.

[0103] In some embodiments, in the above-mentioned distributed reconciliation device, the reconciliation unit 420 is configured to write the reconciliation result into the cache of the reconciliation scheduling center.

[0104] It can be understood that the above-mentioned distributed reconciliation device can implement each step of the distributed reconciliation method (the second distributed reconciliation method) executed by the reconciliation execution node provided in the foregoing embodiments. The relevant explanations regarding the second distributed reconciliation method are applicable to the distributed reconciliation device and will not be elaborated here.

[0105] Figure 5 shows a schematic structural diagram of a distributed reconciliation system according to an embodiment of the present application, as Figure 5 shown, the distributed reconciliation system 500 includes a reconciliation scheduling center 510 and multiple reconciliation execution nodes 520. In the reconciliation scheduling center 510, any one of the above-mentioned distributed reconciliation servers 300 is deployed, and in each reconciliation execution node 520, any one of the above-mentioned distributed reconciliation devices 400 is deployed.

[0106] The reconciliation scheduling center 510 can be communicatively connected to the trading platform and the trading institution, so that the distributed reconciliation server 300 can respectively obtain the first transaction data and the second transaction data, and then perform sharding on the transaction data by calculating the hash value of the first field of the transaction data and then taking the modulus of the hash value by the number of reconciliation execution nodes to obtain the transaction data modulus value, etc., to obtain reconciliation shards, and then send the reconciliation shards to the distributed reconciliation devices 400 in the multiple reconciliation execution nodes 520. The distributed reconciliation devices 400 select the second field in the transaction data according to business requirements, execute the reconciliation task, and can perform deduplication, etc., which can be implemented with reference to the foregoing embodiments of the distributed reconciliation server and the distributed reconciliation device and will not be elaborated here.

[0107] Embodiments of this application rely on the first transaction data and the second transaction data for reconciliation. Since these two types of transaction data come from two parties, namely the transaction platform and the transaction institution, the reconciliation scheduling center and the reconciliation execution nodes in the embodiments of this application can be deployed on the transaction platform, and the corresponding technical solutions can be implemented by the transaction platform, or can be deployed on the transaction institution, and the corresponding technical solutions can be implemented by the transaction institution. It is also possible to deploy the reconciliation scheduling center on the transaction platform and the reconciliation execution nodes on the transaction institution, and so on.

[0108] As mentioned above, the transaction platform can specifically be a transfer platform and / or a clearing platform. In this scenario, deploying the reconciliation scheduling center and the reconciliation execution nodes on the transaction platform and implementing the technical solutions of this application by the transaction platform has better technical effects.

[0109] The reason is that a transaction institution can only reconcile the transaction data related to this transaction institution and cannot reconcile all the transaction data by one transaction institution. Therefore, each transaction institution needs to perform reconciliation separately. The more transaction institutions there are, the more times of reconciliation there will be. However, when the transaction platform performs reconciliation, only one centralized reconciliation is required, and the utilization rate of computing resources is also higher.

[0110] In summary, after receiving the transaction data, the reconciliation scheduling center distributes the transaction data to each reconciliation execution node in the form of reconciliation shards. The reconciliation execution node uses the computing resources local to the reconciliation execution node to perform reconciliation, rather than occupying the cache of the distributed reconciliation system to perform reconciliation, effectively avoiding excessive occupation of the cache. Moreover, the reconciliation scheduling center processes the transaction data into multiple reconciliation shards, which is conducive to distributing the transaction data to the reconciliation execution nodes. Sharding through the first field has a high sharding efficiency and also ensures the sharding accuracy.

[0111] Figure 6 Shows the structural schematic diagram of the first electronic device of an embodiment of this application. Please refer to Figure 6 At the hardware level, this electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory, etc. Of course, this electronic device may also include other hardware required for other services.

[0112] The processor, network interface, and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 only a bidirectional arrow is used in

[0113] Memory, used to store programs. Specifically, the program can include program code, and the program code includes computer operation instructions. The memory can include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0114] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a distributed reconciliation server at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0115] Obtain transaction data to be reconciled, where the transaction data includes first transaction data from a transaction institution and second transaction data from a transaction platform; slice each transaction data according to the first field of each transaction data to obtain multiple reconciliation slices; send the multiple reconciliation slices to reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain a reconciliation result according to the second field of each transaction data in the received reconciliation slices.

[0116] This electronic device can also execute Figure 1 the method executed by the distributed reconciliation server in Figure 3 The functions of the distributed reconciliation server in the illustrated embodiment are not described herein again.

[0117] Figure 7 shows a schematic structural diagram of a second electronic device according to an embodiment of the present application. The same parts as Figure 6 are not described again. Figure 7 In the electronic device shown, the processor executes the program stored in the memory and is specifically used to perform the following operations:

[0118] Receive the reconciliation shards sent by the reconciliation scheduling center. The reconciliation shards are obtained by the reconciliation scheduling center by sharding each transaction data according to the first field of each transaction data. The transaction data includes the first transaction data from the transaction institution and the second transaction data from the transaction platform. For each received reconciliation shard, use the computing resources local to this reconciliation execution node to perform the reconciliation task according to the second field of each transaction data in the reconciliation shard to obtain the reconciliation result.

[0119] Figure 7 The electronic device shown can also execute Figure 2 the method executed by the distributed reconciliation device in Figure 4 the embodiment shown, and implement the functions of the distributed reconciliation device in

[0120] As described above in this application Figure 1 the method executed by the distributed reconciliation server disclosed in the embodiment shown in this application, and as described above in this application Figure 2 the method executed by the distributed reconciliation device disclosed in the embodiment shown can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or by the instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of this application can be directly embodied as being executed and completed by the hardware decoding processor, or by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0121] The embodiments of the present application also propose a computer-readable storage medium, which stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including a plurality of application programs, can cause the electronic device to execute Figure 1 the method executed by the distributed reconciliation server in the illustrated embodiment, and specifically used to execute:

[0122] Obtain the transaction data to be reconciled. The transaction data includes the first transaction data from the transaction institution and the second transaction data from the transaction platform; slice each transaction data according to the first field of each transaction data to obtain a plurality of reconciliation slices; send the plurality of reconciliation slices to the reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain the reconciliation result according to the second field of each transaction data in the received reconciliation slices.

[0123] The embodiments of the present application also propose another computer-readable storage medium, which stores one or more programs. The one or more programs include instructions that, when executed by an electronic device including a plurality of application programs, can cause the electronic device to execute Figure 2 the method executed by the distributed reconciliation device in the illustrated embodiment, and specifically used to execute: receive the reconciliation slices sent by the reconciliation scheduling center. The reconciliation slices are obtained by the reconciliation scheduling center slicing each transaction data according to the first field of each transaction data; the transaction data includes the first transaction data from the transaction institution and the second transaction data from the transaction platform; for each received reconciliation slice, use the local computing resources of this reconciliation execution node to execute the reconciliation task according to the second field of each transaction data in the reconciliation slice to obtain the reconciliation result.

[0124] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0126] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0128] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0129] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0130] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0131] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0132] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0133] The above description is only for the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A distributed reconciliation method, which is executed by a reconciliation scheduling center. The method includes: Obtain transaction data to be reconciled. The transaction data includes first transaction data from a transaction institution and second transaction data from a transaction platform; Slice each piece of transaction data according to the first field of each piece of transaction data to obtain multiple reconciliation slices; Send the multiple reconciliation slices to reconciliation execution nodes, so that the reconciliation execution nodes use local computing resources to obtain a reconciliation result according to the second field of each piece of transaction data received in the received reconciliation slices; Slicing each piece of transaction data according to the first field of each piece of transaction data includes: Perform a hash calculation on the first field of each piece of transaction data to obtain the hash value of each piece of transaction data; slice each piece of transaction data according to the hash value of each piece of transaction data; Slicing each piece of transaction data according to the hash value of each piece of transaction data includes: Use the number of the reconciliation execution nodes as the divisor and the hash value of each piece of transaction data as the dividend respectively to perform a modulo operation to obtain the modulo value of each piece of transaction data; divide each piece of transaction data with the same modulo value into the same reconciliation slice.

2. The method according to claim 1, wherein, Saying that dividing each piece of transaction data with the same modulo value into the same reconciliation slice includes: After obtaining the modulo value of a piece of transaction data, write the piece of transaction data into the slice directory corresponding to the modulo value; Obtain the corresponding reconciliation slice according to the transaction data in each slice directory.

3. The method according to claim 1, wherein, There are multiple reconciliation execution nodes. Sending the multiple reconciliation slices to the reconciliation execution nodes includes: Determine the reconciliation execution node corresponding to each reconciliation slice according to the slice identification information of the reconciliation slice; Send each reconciliation slice to the corresponding reconciliation execution node respectively.

4. The method according to claim 1, wherein, The first field includes at least one of the following information: the working day of the transaction institution, the identification of the transaction institution, the transaction serial number identification, and the in-out fund identification of the transaction.

5. A distributed reconciliation method, which is executed by a reconciliation execution node. The method includes: Receive the reconciliation slice sent by the reconciliation scheduling center. The reconciliation slice is obtained by the reconciliation scheduling center slicing each piece of transaction data according to the first field of each piece of transaction data; the transaction data includes first transaction data from a transaction institution and second transaction data from a transaction platform; For each received reconciliation slice, use the local computing resources of this reconciliation execution node to perform a reconciliation task according to the second field of each piece of transaction data in the reconciliation slice to obtain a reconciliation result; wherein the reconciliation scheduling center performs a hash calculation on the first field of each piece of transaction data to obtain the hash value of each piece of transaction data; Use the number of the reconciliation execution nodes as the divisor and the hash value of each piece of transaction data as the dividend respectively to perform a modulo operation to obtain the modulo value of each piece of transaction data; divide each piece of transaction data with the same modulo value into the same reconciliation slice.

6. The method according to claim 5, wherein, Saying that performing a reconciliation task according to the second field of each piece of transaction data in the reconciliation slice includes: Deduplicate the first transaction data and the second transaction data in the reconciliation slice respectively; Execute the reconciliation task according to the second field of each transaction data after deduplication.

7. The method according to claim 6, wherein, the step of respectively deduplicating the first transaction data and the second transaction data in the reconciliation slice includes: Regarding all the first transaction data or all the second transaction data in the reconciliation slice as the data to be deduplicated; Calculate the similarity between each pair of transaction data in the data to be deduplicated; If, among the data to be deduplicated, the similarity between two transaction data is greater than the first threshold and the transaction identifiers of the two transaction data are the same, then mark one of the two transaction data as duplicate data.

8. The method according to claim 5, wherein, the step of executing the reconciliation task according to the second field of each transaction data in the reconciliation slice and obtaining the reconciliation result includes: According to the second field of each transaction data, determine whether there is a difference between the first transaction data and the second transaction data in the reconciliation slice; If there is a difference, generate a reconciliation result corresponding to the difference, otherwise, generate a reconciliation result indicating no error.

9. The method according to claim 8, wherein, the step of determining whether there is a difference between the first transaction data and the second transaction data in the reconciliation slice according to the second field of each transaction data includes: Calculate the similarity between each first transaction data and each second transaction data according to the second field; If the similarity between a first transaction data and a second transaction data is greater than the second threshold, then mark the two transaction data as a group of successfully reconciled data; If there are several transaction data in the reconciliation slice that have not been marked as successfully reconciled data, then determine that there is a difference between the first transaction data and the second transaction data in the reconciliation slice, and the difference is the several transaction data that have not been marked as successfully reconciled data.

10. The method according to claim 5, wherein, the second field includes at least one of the following information: the working day of the trading institution, the identifier of the trading institution, the transaction identifier, the sequence identifier of the transaction, the details list of the transaction, the amount of the transaction, the in / out fund identifier of the transaction, the type of the transaction, the currency used in the transaction, the status of the transaction.

11. The method according to claim 5, wherein, the method further includes: Writing the reconciliation result into the cache of the reconciliation scheduling center.

12. A distributed reconciliation server, applied to a reconciliation scheduling center, wherein, the server is used to implement the method according to any one of claims 1 to 4.

13. A distributed reconciliation device, applied to a reconciliation execution node, wherein, the device is used to implement the method according to any one of claims 5 to 11.

14. A distributed reconciliation system, including a reconciliation scheduling center and several reconciliation execution nodes, wherein, the reconciliation scheduling center deploys the distributed reconciliation server according to claim 12, and the reconciliation execution nodes deploy the distributed reconciliation device according to claim 13.

15. A computer-readable storage medium storing one or more programs which, when executed by an electronic device including a plurality of application programs, cause the electronic device to execute the method according to any one of claims 1 to 4; or cause the electronic device to execute the method according to any one of claims 5 to 11.

Images