Distributed heterogeneous financial data processing method and device based on process orchestration

Abstract

The present invention can be used in the field of big data technology. The present invention provides a distributed heterogeneous financial data processing method and device based on process orchestration. The distributed heterogeneous financial data processing method based on process orchestration includes: receiving first financial data and second financial data; determining the key value and value of the first financial data and the key value and value of the second financial data respectively based on the first financial data and the second financial data in a mapping table manner; and comparing the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data. On the one hand, the present invention implements parallelization of serial processing by introducing process orchestration, which can effectively improve processing speed and efficiency and reduce processing time. On the other hand, it introduces distributed technology, fully utilizes the advantages of distributed technology, breaks down the whole into parts, greatly reduces the hardware requirements of a single device, and further improves processing time.

Description

Technical Field

[0001] The present invention belongs to the field of big data technology, and specifically relates to a distributed heterogeneous financial data processing method and device based on process orchestration. Background Art

[0002] In the prior art, for heterogeneous data fund reconciliation, in order to avoid the impact of the reconciliation function on other business functions, batch machines are usually used to process the reconciliation separately. The processing flow is roughly as follows: Figure 1 Specifically, the reconciliation server periodically scans for file arrivals from both parties. Reconciliation begins only when both files are complete. During reconciliation, both files are processed together, typically by reading the entire file into memory or writing it to a database. Reconciliation is then completed through a memory or table data comparison. Reconciliation results are generated only after the entire file has been processed.

[0003] Existing reconciliation methods work well when business data volumes are relatively small and data files are relatively small. However, as business volume and data file size increase, significant hardware investments (disk, memory, and CPU) are required to barely meet demand. For example, a daily reconciliation flow for provincial medical insurance can be several gigabytes or even dozens of gigabytes. In this case, the reconciliation server's disk, memory, and CPU requirements are high, otherwise there is a risk of insufficient disk space or memory overflow.

[0004] In addition, existing reconciliation methods are usually serialized, and processing timeliness and performance are a major bottleneck. When the data file size reaches 10G or even larger, it may take several hours to complete the reconciliation. Summary of the Invention

[0005] The present invention belongs to the field of big data technology. The distributed heterogeneous financial data processing method and device provided by it based on process orchestration, on the one hand, implements the parallelization of serial processing by introducing process orchestration, which can effectively improve processing speed and efficiency and reduce processing time (business innovation point). On the other hand, it introduces distributed technology, fully utilizes the advantages of distributed technology, breaks the whole into parts, greatly reduces the hardware requirements of a single device, and further improves processing time.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a distributed heterogeneous financial data processing method based on process orchestration, comprising:

[0008] receiving first financial data and second financial data;

[0009] Determine, according to the mapping table, the key value and value of the first financial data and the key value and value of the second financial data based on the first financial data and the second financial data respectively;

[0010] The first financial data is compared with the second financial data based on the key value and the value of the first financial data and the key value and the value of the second financial data.

[0011] In one embodiment, determining the key value and value of the first financial data and the key value and value of the second financial data respectively based on the first financial data and the second financial data in a mapping table manner includes:

[0012] Determine the unique identification codes of the first financial data and the second financial data using a map method to determine the key value of the first financial data and the key value of the second financial data;

[0013] Determine the value of the first financial data and the value of the second financial data using a map method;

[0014] The key value and its corresponding value have a one-to-one correspondence.

[0015] In one embodiment, the distributed heterogeneous financial data processing method based on process orchestration further includes:

[0016] splitting the first financial data and the second financial data according to the attribute value of the first financial data and the attribute value of the second financial data to generate a splitting result;

[0017] In the splitting result, the number of splits of the first financial data is consistent with the number of splits of the second financial data.

[0018] In one embodiment, comparing the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data includes:

[0019] Generate, based on the splitting results, a first character string corresponding to the splitting result of the first financial data and a second character string corresponding to the splitting result of the second financial data using a delimiter and in a splitting order;

[0020] The first financial data and the second financial data are compared based on the first character string, the second character string, the key value, and the value.

[0021] In a second aspect, the present invention provides a distributed heterogeneous financial data processing device based on process orchestration, comprising:

[0022] A data receiving module for receiving the first financial data and the second financial data

[0023] a key-value pair determination module, configured to determine the key value and value of the first financial data and the key value and value of the second financial data respectively according to the first financial data and the second financial data in a mapping table;

[0024] The data comparison module is configured to compare the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data.

[0025] In one embodiment, the key-value pair determination module includes:

[0026] a key value determining unit, configured to determine the unique identification codes of the first financial data and the second financial data by using a map method, so as to determine the key value of the first financial data and the key value of the second financial data;

[0027] a value determining unit, configured to determine the value of the first financial data and the value of the second financial data by using a map method;

[0028] The key value and its corresponding value have a one-to-one correspondence.

[0029] In one embodiment, the distributed heterogeneous financial data processing device based on process orchestration further includes:

[0030] a data splitting module, configured to split the first financial data and the second financial data according to the attribute value of the first financial data and the attribute value of the second financial data, so as to generate a splitting result;

[0031] In the splitting result, the number of splits of the first financial data is consistent with the number of splits of the second financial data.

[0032] In one embodiment, the data comparison module includes:

[0033] a character string generating unit, configured to generate, according to the splitting results, a first character string corresponding to the splitting result of the first financial data and a second character string corresponding to the splitting result of the second financial data, respectively, using a delimiter and in a splitting order;

[0034] A data comparison unit is configured to compare the first financial data with the second financial data based on the first character string, the second character string, the key value, and the value.

[0035] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a distributed heterogeneous financial data processing method based on process orchestration are implemented.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a distributed heterogeneous financial data processing method based on process orchestration.

[0037] As can be seen from the above description, the distributed heterogeneous financial data processing method and apparatus based on process orchestration provided by the embodiments of the present invention first receives first and second financial data; then, based on the first and second financial data, determines the key value and value of the first financial data and the key value and value of the second financial data, respectively, in accordance with a mapping table; and finally, compares the first and second financial data based on the key value and value of the first and second financial data. By introducing process orchestration, the present invention fully utilizes the advantages of distributed technology in payment fund reconciliation, parallelizing and distributing serial processing, reducing the hardware requirements of a single device while effectively improving processing efficiency and speed. Furthermore, the present invention, on the one hand, implements parallelization of serial processing by introducing process orchestration, effectively improving processing speed and efficiency and reducing processing time (a business innovation point). On the other hand, by introducing distributed technology, it fully utilizes the advantages of distributed technology and breaks down the whole into parts, significantly reducing the hardware requirements of a single device while further improving processing time. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of a distributed heterogeneous financial data reconciliation method in the prior art;

[0040] Figure 2 Schematic diagram of the distributed heterogeneous financial data processing method based on process orchestration in an embodiment of the present invention Figure 1 ;

[0041] Figure 3 200 is a flow chart of step 200 in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the distributed heterogeneous financial data processing method based on process orchestration in an embodiment of the present invention Figure 2 ;

[0043] Figure 5 300 is a flowchart of an embodiment of the present invention;

[0044] Figure 6 This is a conceptual diagram of a distributed heterogeneous financial data processing method based on process orchestration in a specific application example of the present invention;

[0045] Figure 7 This is a flow chart of a distributed heterogeneous financial data processing method based on process orchestration in a specific application example of the present invention;

[0046] Figure 8 Schematic diagram of the structure of a distributed heterogeneous financial data processing device based on process orchestration in an embodiment of the present invention Figure 1 ;

[0047] Figure 9 2 is a schematic diagram of the structure of the key-value pair determination module 20 in an embodiment of the present invention;

[0048] Figure 10 Schematic diagram of the structure of a distributed heterogeneous financial data processing device based on process orchestration in an embodiment of the present invention Figure 2 ;

[0049] Figure 11 Schematic diagram of the structure of the data comparison module 30 in an embodiment of the present invention;

[0050] Figure 12 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or apparatuses.

[0054] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] The embodiment of the present invention provides a specific implementation of a distributed heterogeneous financial data processing method based on process orchestration, see Figure 2 , the method specifically includes the following contents:

[0056] Step 100: Receive first financial data and second financial data.

[0057] Specifically, the first and second financial data are verified to ensure that the data to be processed is provided in accordance with the agreed rules and is complete. This mainly requires completing file name verification, file size verification, file MD5 value verification, etc.

[0058] It's understandable that the data structures of the first and second financial data recording the same fund flow are completely different. For example, the first financial data may be generated by a business system, which is business-driven and whose core data is business elements. The data structures recorded by different business systems are related to their respective businesses and can vary. The second financial data may be generated by a banking system, which is a single system that supports a wide range of businesses, with its core data being the flow of funds between accounts.

[0059] Step 200: Determine the key value and value of the first financial data and the key value and value of the second financial data according to the first financial data and the second financial data respectively in accordance with the mapping table;

[0060] It is understandable that a map is a type of table with y / value (key / value) structure, which is essentially a more powerful array. yA map is an array index, and value is the value corresponding to the index. Modern languages have corresponding implementations for this. A map is an object that stores associations between keys and values (i.e., key-value pairs). Given a key, its value can be found. Both keys and values are objects. Keys must be unique, but values can be repeated. You can think of a map container as having two parts: one for storing keys and one for storing values, with the two parts being associated (one key corresponds to one value).

[0061] Step 300: Compare the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data.

[0062] Specifically, firstly, based on the key value of the first financial data, a map is traversed key value by key value, and the key value, ie, the unique identification code, is used to check whether the corresponding key value exists in the second financial data.

[0063] As can be seen from the above description, the distributed heterogeneous financial data processing method based on process orchestration provided by the embodiment of the present invention first receives first financial data and second financial data; then, according to the mapping table, determines the key value and value of the first financial data and the key value and value of the second financial data respectively based on the first financial data and the second financial data; finally, compares the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data. By introducing process orchestration, the present invention can fully utilize the advantages of distributed technology in payment fund reconciliation, parallelizing and distributing serial processing, reducing the hardware requirements of a single device while effectively improving processing efficiency and speed. Furthermore, the present invention, on the one hand, implements parallelization of serial processing by introducing process orchestration, which can effectively improve processing speed and efficiency and reduce processing time (a business innovation point). On the other hand, by introducing distributed technology, it fully utilizes the advantages of distributed technology and breaks down the whole into parts, significantly reducing the hardware requirements of a single device while further improving processing time.

[0064] In one embodiment, see Figure 3 , step 200 further includes:

[0065] Step 201: Determine the unique identification codes of the first financial data and the second financial data using a map method to determine the key value of the first financial data and the key value of the second financial data.

[0066] Step 202: Determine the value of the first financial data and the value of the second financial data using the map method.

[0067] In step 201 and step 202 , the key value and its corresponding value are in a one-to-one correspondence.

[0068] In one embodiment, see Figure 4 ,The distributed heterogeneous financial data processing method based on process orchestration also includes:

[0069] Step 400: Splitting the first financial data and the second financial data according to the attribute value of the first financial data and the attribute value of the second financial data to generate a splitting result;

[0070] In the splitting result, the number of splits of the first financial data is consistent with the number of splits of the second financial data.

[0071] In step 400, after the data integrity check passes, data splitting can begin without waiting for the other party's reconciliation file to arrive, splitting a data file into multiple small data files, such as splitting data file A into n small data files such as A1, A2, ..., An. Specifically:

[0072] Data splitting is done by configuring rules, splitting a source data file into multiple smaller data files according to the configured rules. The file naming rules obtained after splitting are processed by adding modifications to the source file name. For example, if data file A is split modulo n, the file names after splitting are A1, A2, ..., An (where 1 to n are the remainders after the modulo, and n is the case when the remainder is 0); if data file A is split by attribute value, the file names after splitting are Aa, Ab, ..., An (where a to n are the specific values of the attributes).

[0073] Specific splitting rules can be flexibly configured based on actual business data. Splitting can be performed based on attribute value modulo, such as splitting data into 10 parts based on the order number modulo 10, splitting data file A into n small data files, such as A1, A2, ..., An. Splitting can also be performed based on specific attribute values. For example, in medical insurance reconciliation, the provincial medical insurance provides summary files, which are split by each municipal medical insurance unit. Data file A can be split into n small data files, such as Aa, Ab, ..., An, each containing data from only one institution. Splitting can also be performed based on multiple attributes, first by the specific value of attribute a, then by the modulo value of attribute b, splitting data file A into m×n small data files, such as A-a1...A-an, A-b1...A-bn, ..., A-m1...A-mn, etc.

[0074] In one embodiment, see Figure 5 , step 300 further includes:

[0075] Step 301: using a delimiter and in a splitting order, generating a first character string corresponding to the split result of the first financial data and a second character string corresponding to the split result of the second financial data according to the splitting result;

[0076] Step 302: Compare the first financial data with the second financial data based on the first character string, the second character string, the key value, and the value.

[0077] In steps 301 and 302, when comparing data, A1 and B1 (the first sub-file after splitting the second financial data) are first read into memory and stored in a map format to obtain map4A and map4B. The map key is a unique identifier, i.e., Aa or Ba, and the map value is other attributes other than the unique identifier, which are connected in sequence by the delimiter | to obtain the string Ac|Af|Ag or Bg|Bm|Bp. Then, based on map4A, the map is traversed key by key, and the key value, i.e., the unique identifier, is checked from map4B to see if the corresponding key value exists in B. If not, the key value is greater in A than in B. The result is written to the result file and the key value is removed from map4A. If found, the values of the two are compared. If the values are the same, the reconciliation is consistent. Otherwise, both A and B exist but with different values. The result is written to the result file and the key value is removed from map4A and Map4B. After map4A is traversed, map4A is cleared, and the remaining elements in map4B are all B more than A. Then the remaining elements in map4B are traversed again and the results are written to the result file. After two loop traversals, the comparison results of A1 and B1 are obtained.

[0078] To further illustrate this solution, the present invention takes the reconciliation of data provided by the business system and the bank system as an example to provide a specific application example of the distributed heterogeneous financial data processing method based on process orchestration, see Figure 6 as well as Figure 7 .

[0079] Explanation of terms:

[0080] Funds reconciliation: This process reconciles the account transactions stored in the business system with the bank's clearing transactions, verifying the consistency of the business system's account data with the bank's clearing data to ensure that the estimated daily balances of each payment institution's reserve bank account are consistent with the actual balances. This process also involves verifying the consistency of the bank's actual clearing results for transactions such as deposits, withdrawals, transfers, and refunds.

[0081] Heterogeneous data: The data structures used to record the same fund flow in business systems and banking systems are completely different. Business systems are business-driven, with the core data being business elements. The data structures recorded in different business systems are related to their respective businesses and can vary. Banking systems, on the other hand, are a single system that supports a wide range of businesses, with the core data being the flow of funds between accounts.

[0082] Process orchestration: The entire reconciliation process is broken down into steps such as data reception, data splitting, data cleaning, data comparison, and result generation. Each step is both interdependent and independent. Through process orchestration, serial operations can be parallelized to improve efficiency.

[0083] Data reception: Receive reconciliation source data from both parties through the network. Source data files vary in format, but each has its own set of format specifications, such as fixed-length files and delimited files.

[0084] Data splitting: Split a large data file into several small data files according to certain rules; splitting heterogeneous data according to relevant rules can ensure that data of the same dimension is included in the corresponding results.

[0085] Data cleaning: According to agreed rules, filter and eliminate interfering and irrelevant junk data in data files, leaving only the valid data that really needs to be compared. At the same time, normalize heterogeneous data with the same attributes but different representations. For example, if the dictionary values of the result status in the accounting system and the liquidation system are different, normalize them. If the amount units are different, both the amount and the unit need to be normalized.

[0086] Data comparison: The cleaned and normalized data files are compared in two directions to obtain the reconciliation results. The results include A and B are the same, A has some but B does not, B has some but A does not, and A and B both have some but are inconsistent.

[0087] Result generation: The data comparison results are summarized to obtain the reconciliation results. If necessary, the specific attributes of the results can be customized as required, and the results can be compiled and output.

[0088] S1: Data reception.

[0089] Data reception verifies and processes data submitted by business and banking systems to ensure that the data being processed complies with agreed-upon rules and is complete. This primarily involves verifying file name, file size, and file MD5 values.

[0090] S2: Data splitting.

[0091] Based on step S1, after the data integrity check is passed, there is no need to wait for the other party's reconciliation file to arrive, and data splitting can be started to split a data file into multiple small data files, such as splitting data file A into n small data files such as A1, A2, ..., An.

[0092] Data splitting is done by configuring rules, splitting a source data file into multiple smaller data files according to the configured rules. The file naming rules obtained after splitting are processed by adding modifications to the source file name. For example, if data file A is split modulo n, the file names after splitting are A1, A2, ..., An (where 1 to n are the remainders after the modulo, and n is the case when the remainder is 0); if data file A is split by attribute value, the file names after splitting are Aa, Ab, ..., An (where a to n are the specific values of the attributes).

[0093] Of course, specific splitting rules can also be flexibly configured based on actual business data needs. Splitting can be done by modulo attribute values, such as splitting data into 10 parts by taking the order number modulo 10, splitting data file A into n small data files, such as A1, A2, ..., An. Splitting can also be done based on specific attribute values, such as by payer institution. In medical insurance reconciliation, the provincial medical insurance provides summary files, which are split by each municipal medical insurance unit. Data file A can be split into n small data files, such as Aa, Ab, ..., An, each containing data from only one institution. Splitting can also be done based on multiple attributes, first by taking the specific value of attribute a and then taking the modulo attribute b, splitting data file A into m×n small data files, such as A-a1...A-an, A-b1...A-bn, ..., A-m1...A-mn, etc.

[0094] S3: Data cleaning.

[0095] Data cleaning involves three steps: removing attributes not involved in reconciliation, normalizing attribute values, and formatting files. The primary purpose of data cleaning is to facilitate subsequent data comparison, simplifying the comparison process as much as possible and improving performance and efficiency.

[0096] Eliminating irrelevant attributes not involved in reconciliation cleans up irrelevant data, eliminating interfering data while reducing data file size and easing the burden of data comparison. For example, if data file A is a delimited file with each line formatted as a|b|c|d|e|f|g|h|i|j|k|l|m|n, containing 14 attributes, but only four attributes (a, c, f, and g) are actually involved in reconciliation, the first step in data cleaning is to extract the values of these four attributes from each line. If data file B is a fixed-length file, similar rules can be used to extract the corresponding attribute values from each line. Data files often contain many elements, sometimes dozens, but the attributes that are actually involved in reconciliation are generally under 10. Therefore, by eliminating irrelevant attributes, the data file size can be reduced by over 50%.

[0097] Attribute value normalization primarily considers that different systems may use different enumeration values for the same attribute. To simplify data comparison, normalization is required so that both parties using the same value use the same value for attributes with the same meaning. For example, if the currency attribute in data file A is represented by 1 in RMB, and in data file B by CNY, normalization requires converting 1 to CNY, or vice versa, or converting both CNY and 1 to an intermediate value, RMB.

[0098] File formatting converts heterogeneous data into data with a uniform structure. After file formatting, data comparison can be performed through simple text comparison. For example, data file A is a delimited file, with each line formatted as Aa|Ab|Ac|Ad|Ae|Af|Ag|Ah|Ai|Aj|Ak|Al|Am|An. It contains 14 attributes, but only four—Aa, Ac, Af, and Ag—are actually involved in reconciliation. Data file B is a fixed-length file containing 20 attributes. Bits 1 through 5 are attribute Ba, bits 6 through 12 are attribute Bb, and bits 1234 through 1300 are attribute Bt. However, only four—Ba, Bg, Bm, and Bp—are actually involved in reconciliation. Aa corresponds to Ba, Ac corresponds to Bg, Af corresponds to Bm, and Ag corresponds to Bp. Aa and Ba are both unique identifiers. After formatting, Aa, Ac, Af, Ag and Ba, Bg, Bm, and Bp are connected by the delimiter | according to their attributes to obtain a row of formatted data.

[0099] S4: Data comparison.

[0100] After the data is formatted, as long as the data on both sides are in place, such as A1 and B1 are in place, data comparison can begin.

[0101] When comparing data, A1 and B1 are first read into memory and stored as maps, resulting in map4A and map4B. The map key is a unique identifier, either Aa or Ba, and the map value is the string Ac|Af|Ag or Bg|Bm|Bp, concatenated in sequence using the delimiter |. Using map4A as a base, the map is then traversed key by key, checking map4B for the corresponding key in B using the key (i.e., unique identifier). If not, the key value is greater than A and less than B. The result is written to the result file and the key value is removed from map4A. If found, the values are compared. If they match, the reconciliation is consistent. Otherwise, both A and B exist but with different values. The result is written to the result file and the key value is removed from both map4A and map4B. After traversing map4A, map4A is cleared, and the remaining elements in map4B are all in the case of B greater than A. The remaining elements in map4B are then traversed and the result is written to the result file.

[0102] After two iterations of this loop, the comparison result A1B1 between A1 and B1 is generated. Once A1B1, A2B2, ..., AnBn are all output, a simple file merge can be used to generate the complete reconciliation result.

[0103] As can be seen from the above description, the distributed heterogeneous financial data processing method based on process orchestration provided by the embodiment of the present invention first receives first financial data and second financial data; then, according to the mapping table, determines the key value and value of the first financial data and the key value and value of the second financial data respectively based on the first financial data and the second financial data; finally, compares the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data. By introducing process orchestration, the present invention can fully utilize the advantages of distributed technology in payment fund reconciliation, parallelizing and distributing serial processing, reducing the hardware requirements of a single device while effectively improving processing efficiency and speed. Furthermore, the present invention, on the one hand, implements parallelization of serial processing by introducing process orchestration, which can effectively improve processing speed and efficiency and reduce processing time (a business innovation point). On the other hand, by introducing distributed technology, it fully utilizes the advantages of distributed technology and breaks down the whole into parts, significantly reducing the hardware requirements of a single device while further improving processing time.

[0104] Based on the same inventive concept, the embodiments of the present application also provide a distributed heterogeneous financial data processing device based on process orchestration, which can be used to implement the method described in the above embodiments, such as the following embodiments. Since the principle of solving the problem by the distributed heterogeneous financial data processing device based on process orchestration is similar to that of the distributed heterogeneous financial data processing method based on process orchestration, the implementation of the distributed heterogeneous financial data processing device based on process orchestration can refer to the implementation of the distributed heterogeneous financial data processing method based on process orchestration, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceived.

[0105] The embodiment of the present invention provides a specific implementation of a distributed heterogeneous financial data processing device based on process orchestration that can implement a distributed heterogeneous financial data processing method based on process orchestration, see Figure 8 The distributed heterogeneous financial data processing device based on process orchestration specifically includes the following contents:

[0106] The data receiving module 10 is used to receive the first financial data and the second financial data.

[0107] a key-value pair determination module 20 for determining the key value and value of the first financial data and the key value and value of the second financial data respectively according to the first financial data and the second financial data in a mapping table manner;

[0108] The data comparison module 30 is configured to compare the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data.

[0109] In one embodiment, see Figure 9 , the key-value pair determination module 20 includes:

[0110] A key value determining unit 201 is configured to determine the unique identification codes of the first financial data and the second financial data by using a map method, so as to determine the key value of the first financial data and the key value of the second financial data;

[0111] a value determination unit 202, configured to determine the value of the first financial data and the value of the second financial data by using a map method;

[0112] The key value and its corresponding value have a one-to-one correspondence.

[0113] In one embodiment, see Figure 10 The distributed heterogeneous financial data processing device based on process orchestration also includes:

[0114] a data splitting module 40, configured to split the first financial data and the second financial data according to the attribute value of the first financial data and the attribute value of the second financial data to generate a splitting result;

[0115] In the splitting result, the number of splits of the first financial data is consistent with the number of splits of the second financial data.

[0116] In one embodiment, see Figure 11 , the data comparison module 30 includes:

[0117] A character string generating unit 301 is configured to generate, according to the splitting results, a first character string corresponding to the splitting result of the first financial data and a second character string corresponding to the splitting result of the second financial data, respectively, using a delimiter and in a splitting order;

[0118] The data comparison unit 302 is configured to compare the first financial data with the second financial data based on the first character string, the second character string, the key value, and the value.

[0119] As can be seen from the above description, the distributed heterogeneous financial data processing device based on process orchestration provided by an embodiment of the present invention first receives first and second financial data; then, based on the first and second financial data, determines the key value and value of the first financial data and the key value and value of the second financial data, respectively, using a mapping table; and finally, compares the first and second financial data based on the key value and value of the first and second financial data. By introducing process orchestration, the present invention fully utilizes the advantages of distributed technology in payment fund reconciliation, parallelizing and distributing serial processing, reducing the hardware requirements of a single device while effectively improving processing efficiency and speed. Furthermore, the present invention, on the one hand, implements parallelization of serial processing through the introduction of process orchestration, effectively improving processing speed and efficiency and reducing processing time (a business innovation point). On the other hand, by introducing distributed technology, it fully utilizes the advantages of distributed technology and breaks down the whole into parts, significantly reducing the hardware requirements of a single device while further improving processing time.

[0120] Reference below Figure 12 , which shows a structural diagram of an electronic device 600 suitable for implementing an embodiment of the present application.

[0121] like Figure 12As shown, electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate operations and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. Various programs and data required for the operation of system 600 are also stored in RAM 603. CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0122] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed in the storage section 608 as needed.

[0123] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method for determining the distance between people in a data center scenario are implemented, including:

[0124] Step 100: Receive first financial data and second financial data;

[0125] Step 200: Determine the key value and value of the first financial data and the key value and value of the second financial data according to the first financial data and the second financial data respectively in accordance with the mapping table;

[0126] Step 300: Compare the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data.

[0127] In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 609 , and / or installed from the removable media 611 .

[0128] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0129] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0132] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0133] The above are merely examples of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A distributed heterogeneous financial data processing method based on process orchestration, characterized in that: include: Receive first financial data and second financial data; wherein the first financial data and the second financial data record the same fund flow in different data structures, the first financial data is generated by a business system, which is business-driven, and the data core is business elements. The data structures recorded in different business systems are related to their respective businesses and are diverse; the second financial data is generated by a banking system, which is a system compatible with various businesses and the data core is fund transactions between accounts; Determine, according to the mapping table, the key value and value of the first financial data and the key value and value of the second financial data respectively based on the first financial data and the second financial data; Comparing the first financial data with the second financial data based on the key value and the value of the first financial data and the key value and the value of the second financial data includes: Based on the key value of the first financial data, the map is traversed key by key value, and the corresponding key value, i.e., the unique identification code, is checked to see if it exists in the second financial data. The distributed heterogeneous financial data processing method further includes: splitting the first financial data and the second financial data according to the attribute value of the first financial data and the attribute value of the second financial data to generate a splitting result; In the splitting result, the number of splits of the first financial data is consistent with the number of splits of the second financial data; The comparing the first financial data with the second financial data according to the key value and value of the first financial data and the key value and value of the second financial data includes: Generate, based on the splitting results, a first character string corresponding to the splitting result of the first financial data and a second character string corresponding to the splitting result of the second financial data using a delimiter and in a splitting order; The first financial data and the second financial data are compared based on the first character string, the second character string, the key value, and the value, specifically: First, read the first sub-file A1 after splitting the first financial data and the first sub-file B1 after splitting the second financial data into memory, and store them in a map manner to obtain map4A and map4B, where the key value of the map is the unique identification code, that is, Aa or Ba, and the value of the map is other attributes other than the unique identification code, which are connected in sequence by the delimiter | to obtain the string Ac|Af|Ag or Bg|Bm|Bp. Then, based on map4A, traverse the map key value one by one, and check whether the corresponding key value exists in the second financial data B from map4B through the key value, that is, the unique identification code. If not, the key value is the case where there is more first financial data A and less second financial data B. The result is written to the result file, and the key value is removed from map4A at the same time; if found, the values of the two are compared. If the values are the same, it means that the reconciliation is consistent. Otherwise, the first financial data A and the second financial data B both exist but the values are different. The result is written to the result file, and the key value is removed from map4A and Map4B at the same time. In this way, after map4A is traversed, map4A has been cleared, and the remaining elements in map4B are all the second financial data B More and the first financial data A Less, then the remaining elements of map4B are traversed again, and the results are written to the result file; after two loop traversals, the comparison results of the first sub-file A1 after splitting the first financial data and the first sub-file B1 after splitting the second financial data are completed.

2. The distributed heterogeneous financial data processing method according to claim 1, characterized in that: Determining the key value and value of the first financial data and the key value and value of the second financial data respectively according to the first financial data and the second financial data in a mapping table manner includes: Determine the unique identification codes of the first financial data and the second financial data using a map method to determine the key value of the first financial data and the key value of the second financial data; Determine the value of the first financial data and the value of the second financial data using a map method; The key value and its corresponding value have a one-to-one correspondence.

3. A distributed heterogeneous financial data processing device based on process orchestration, characterized in that: include: A data receiving module is configured to receive first and second financial data; wherein the first and second financial data record the same fund flow in different data structures. The first financial data is generated by a business system, which is business-driven and has business elements as its core data. The data structures recorded by different business systems are related to their respective businesses and are diverse. The second financial data is generated by a banking system, which is a system compatible with various businesses and has fund transactions between accounts as its core data. a key-value pair determination module, configured to determine the key value and value of the first financial data and the key value and value of the second financial data respectively according to the first financial data and the second financial data in a mapping table; A data comparison module, configured to compare the first financial data with the second financial data based on the key value and value of the first financial data and the key value and value of the second financial data, including: Based on the key value of the first financial data, the map is traversed key by key value, and the corresponding key value, i.e., the unique identification code, is checked to see if it exists in the second financial data. The distributed heterogeneous financial data processing device is characterized by further comprising: a data splitting module, configured to split the first financial data and the second financial data according to the attribute value of the first financial data and the attribute value of the second financial data, so as to generate a splitting result; In the splitting result, the number of splits of the first financial data is consistent with the number of splits of the second financial data; The data comparison module includes: a character string generating unit, configured to generate, according to the splitting results, a first character string corresponding to the splitting result of the first financial data and a second character string corresponding to the splitting result of the second financial data, respectively, using a delimiter and in a splitting order; A data comparison unit is configured to compare the first financial data with the second financial data based on the first character string, the second character string, the key value, and the value, specifically: First, read the first sub-file A1 after splitting the first financial data and the first sub-file B1 after splitting the second financial data into memory, and store them in a map manner to obtain map4A and map4B, where the key value of the map is the unique identification code, that is, Aa or Ba, and the value of the map is other attributes other than the unique identification code, which are connected in sequence by the delimiter | to obtain the string Ac|Af|Ag or Bg|Bm|Bp. Then, based on map4A, traverse the map key value one by one, and check whether the corresponding key value exists in the second financial data B from map4B through the key value, that is, the unique identification code. If not, the key value is the case where there is more first financial data A and less second financial data B. The result is written to the result file, and the key value is removed from map4A at the same time; if found, the values of the two are compared. If the values are the same, it means that the reconciliation is consistent. Otherwise, the first financial data A and the second financial data B both exist but the values are different. The result is written to the result file, and the key value is removed from map4A and Map4B at the same time. In this way, after map4A is traversed, map4A has been cleared, and the remaining elements in map4B are all the second financial data B More and the first financial data A Less, then the remaining elements of map4B are traversed again, and the results are written to the result file; after two loop traversals, the comparison results of the first sub-file A1 after splitting the first financial data and the first sub-file B1 after splitting the second financial data are completed.

4. The distributed heterogeneous financial data processing device according to claim 3, characterized in that: The key-value pair determination module includes: a key value determining unit, configured to determine the unique identification codes of the first financial data and the second financial data by using a map method, so as to determine the key value of the first financial data and the key value of the second financial data; a value determining unit, configured to determine the value of the first financial data and the value of the second financial data by using a map method; The key value and its corresponding value have a one-to-one correspondence.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the distributed heterogeneous financial data processing method based on process orchestration according to any one of claims 1 to 2 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the distributed heterogeneous financial data processing method based on process orchestration as described in any one of claims 1 to 2 are implemented.

Images