A data matching method, a Redis server, and a readable storage medium.

Abstract

This application provides a data matching method, a Redis server, and a readable storage medium. The method includes: writing a first set of data to be matched into a first Redis set according to a preset rule; writing a second set of data to be matched into a second Redis set according to a preset rule; and performing set operations on the first and second Redis sets to obtain a matching result. This application utilizes the characteristics of Redis sets for data matching, which can improve the efficiency of data matching, is applicable to various types of data matching, has strong versatility, and solves the problems of time consumption and low performance in existing data matching methods.

Description

Technical Field

[0001] This application relates to the field of transaction data processing technology, and in particular to a data matching method, a Redis server, and a readable storage medium. Background Technology

[0002] In the third-party payment sector, the most common data matching and comparison scenario is reconciliation. Every financial payment system inevitably involves numerous reconciliation scenarios within the system, between systems, and with external systems. For reconciliation, the comparison of large volumes of data cannot be adequately met by traditional database join queries alone, and it also places high demands on the database performance itself.

[0003] Because it's a pairwise reconciliation method, it typically involves two data tables, A and B. The current reconciliation process involves: querying table A to retrieve all data matching the reconciliation criteria and adding it to a List; querying table B to retrieve all data matching the reconciliation criteria and adding it to a List; and then performing the data comparison in memory using Java's List class (using the `contains` interface) or other collection classes like Set and Map. When dealing with large amounts of data, this current method requires processing entirely within the Java Virtual Machine (JVM), significantly impacting the application server and potentially leading to memory overflow. Therefore, this method is inefficient for reconciling large datasets quickly. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a data matching method, a Redis server, and a readable storage medium to improve the efficiency of data matching. This method is applicable to various types of data matching, has strong versatility, and solves the problems of time consumption and low performance in existing data matching methods. The specific technical solution is as follows:

[0005] Firstly, this application provides a data matching method applied to a Redis server, the method comprising:

[0006] Write the first set of data to be matched into the first Redis set according to the preset rules;

[0007] Write the second set of data to be matched into the second Redis set according to the preset rules;

[0008] Perform set operations on the first Redis set and the second Redis set to obtain the matching result.

[0009] In one possible implementation, performing set operations on the first Redis set and the second Redis set to obtain a matching result includes:

[0010] Perform an intersection operation on the first Redis set and the second Redis set to obtain the first intersection;

[0011] Perform a difference operation on the first Redis set and the first intersection to obtain the first difference set;

[0012] The first difference set is pruned to obtain the second difference set;

[0013] Perform a difference operation on the intersection of the second Redis set and the first set to obtain a third difference set;

[0014] The third difference set is pruned to obtain the fourth difference set;

[0015] Perform an intersection operation on the second difference set and the fourth difference set to obtain the second intersection set;

[0016] The matching result is generated based on the first intersection and the second intersection.

[0017] In one possible implementation, writing the first set of data to be matched into the first Redis set according to a preset rule includes:

[0018] According to the concatenation rules of the matching items and verification items in the preset rules, the data to be matched in the first set of data to be matched is concatenated to generate one or more strings;

[0019] Write the string into the first Redis collection.

[0020] In one possible implementation, before writing the first set of data to be matched into the first Redis set according to a preset rule, the method further includes:

[0021] Retrieve the first and second data tables;

[0022] Based on the rows of data to be matched that meet the matching conditions in the first data table, the first set of data to be matched is generated;

[0023] Based on the rows of data in the second data table that meet the matching conditions, a second set of data to be matched is generated.

[0024] In one possible implementation, before obtaining the first data table and the second data table, the method further includes:

[0025] Receive a data matching instruction, which is sent after a timed task is triggered, and the data matching instruction includes a data matching type;

[0026] The process of obtaining the first data table and the second data table includes:

[0027] Obtain the first data table and the second data table that match the data type.

[0028] Secondly, this application also provides a Redis server, the Redis server comprising:

[0029] The first write module is used to write the first set of data to be matched into the first Redis set according to preset rules;

[0030] The second write module is used to write the second set of data to be matched into the second Redis set according to the preset rules;

[0031] The operation module is used to perform set operations on the first Redis set and the second Redis set to obtain the matching result.

[0032] In one possible implementation, the arithmetic module is specifically used for:

[0033] Perform an intersection operation on the first Redis set and the second Redis set to obtain the first intersection;

[0034] Perform a difference operation on the first Redis set and the first intersection to obtain the first difference set;

[0035] The first difference set is pruned to obtain the second difference set;

[0036] Perform a difference operation on the intersection of the second Redis set and the first set to obtain a third difference set;

[0037] The third difference set is pruned to obtain the fourth difference set;

[0038] Perform an intersection operation on the second difference set and the fourth difference set to obtain the second intersection set;

[0039] The matching result is generated based on the first intersection and the second intersection.

[0040] In one possible implementation, the first writing module is specifically used for:

[0041] According to the concatenation rules of the matching items and verification items in the preset rules, the data to be matched in the first set of data to be matched is concatenated to generate one or more strings;

[0042] Write the string into the first Redis collection.

[0043] In one possible implementation, the Redis server further includes:

[0044] The acquisition module is used to acquire the first data table and the second data table.

[0045] The first generation module is used to generate the first set of data to be matched based on the row data to be matched in the first data table that meet the matching conditions;

[0046] The second generation module is used to generate the second set of data to be matched based on the row data to be matched in the second data table that meet the matching conditions.

[0047] Thirdly, this application also provides a readable storage medium storing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect or any one of the first aspects.

[0048] The method provided in this application includes: writing a first set of data to be matched into a first Redis set according to a preset rule; writing a second set of data to be matched into a second Redis set according to a preset rule; and performing set operations on the first and second Redis sets to obtain a matching result. This application utilizes the characteristics of Redis sets for data matching, which can improve the efficiency of data matching, is applicable to various types of data matching, has strong versatility, and solves the problems of time consumption and low performance in existing data matching methods. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart of an embodiment of a data matching method provided in this application is shown;

[0051] Figure 2 A flowchart of another data matching method embodiment provided by this application is shown;

[0052] Figure 3 A flowchart illustrating an application scenario provided by an embodiment of this application is shown;

[0053] Figure 4 A schematic diagram of a set operation provided in an embodiment of this application is shown;

[0054] Figure 5 A schematic diagram of another set operation provided in an embodiment of this application is shown;

[0055] Figure 6 A schematic diagram of the structure of a Redis server provided in an embodiment of this application is shown. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] First, some terms that may appear in the embodiments of this application will be explained.

[0058] E-commerce settlement system: A settlement platform for importing files from various data sources such as payment merchants, bank deposit slips, business orders, payment orders, TCN flight data, and supplier data; matching and reconciling data; handling discrepancies; generating journal entries; aggregating accounts; mapping accounts; managing accounting periods; and outputting accounting interfaces.

[0059] Reconciliation: This refers to the process of verifying transaction information from the previous clearing cycle to confirm the consistency and accuracy of the transaction information. By reconciling accounts receivable with each other from multiple data sources, the clearing center establishes a link between accounts receivable and actual funds received, thereby achieving accurate reconciliation of accounts receivable.

[0060] Redis, short for Remote Dictionary Server, is a high-performance key-value database. It primarily provides five data structures: strings, hashes, lists, sets, and sorted sets.

[0061] Intersection: A basic set operation that finds elements that exist in both sets (common elements).

[0062] Difference: A basic set operation that removes elements from one set that belong to another set.

[0063] Matching items: During the reconciliation process, one party searches for corresponding attributes (fields) in the other party based on certain attributes (fields).

[0064] Verification items: During the reconciliation process, data with the same matching items need to be compared and verified, which are usually amounts or products.

[0065] Matching status: For data where both the matching items and the check items are the same, the matching status is Y, which means a complete match; for data where the matching items are the same but the check items are different, the matching status is E, which means a partial match; for data where the matching items are different, the matching status is N, which means a failed match.

[0066] With the development of civil aviation e-commerce, the complexity of airline sales and intermodal clearing and settlement operations is constantly increasing. Due to the high complexity, long links, numerous stages, and multiple external systems, there is a risk of data inconsistency between multiple systems and the inability to accurately and timely collect accounts receivable. To ensure the safety of airline funds, the e-commerce settlement system should provide reconciliation processing between multiple systems and data sources. As a subsystem, the settlement center needs to receive data from multiple parties and perform pairwise reconciliation to assist airlines in e-commerce settlement. The data involved includes bank deposit slips, payment merchant settlement data, business order data, payment order data, ticket data, and third-party supplier data. Reconciliation operations are required after all data sources are connected to the e-commerce settlement system. The reconciliation steps involved in the settlement center are as follows:

[0067] (1) Reconcile bank deposit slip data with payment provider settlement data;

[0068] (2) Reconciling payment merchant settlement data with business order data;

[0069] (3) Reconcile business order data with payment order data;

[0070] (4) Reconcile business order data with flight ticket data;

[0071] (5) Reconcile business order data with third-party supplier data.

[0072] Payment merchant settlement data includes merchant ID, transaction type, transaction date, settlement currency, settlement gross amount, settlement fee, settlement fee rate, product code, submitted bank order number, payment merchant code, etc.; bank deposit slip data includes bank posting date, debit / credit direction, posting amount, bank account number, bank account abbreviation, counterparty account number, counterparty account name, voucher number, payment channel merchant ID, etc.; business order data includes transaction sequence number, transaction type, payment order number, refund transaction number, submitted bank order number, payment time, payment currency, payment amount (mileage), mileage value, payment merchant, payment product, payment fee, payment fee rate, etc.; payment order data includes flow... The data includes: transaction number, record serial number, business platform type, payment platform order number, order date, transaction currency, transaction amount, transaction date, transaction status, etc.; airline ticket data includes: airline 3-letter code, station code, transaction code, ticket number, ticketing date, international / domestic identifier, agent number, currency type, ticket type, ticket price, actual fare, tax rate, taxes, ticket serial number, old ticket number, etc.; third-party supplier data includes: supplier name, supplier code, product type, product code, product name, billing document type, business order number, transaction type, transaction date, order billing amount, commission calculation basis, commission, passenger name, flight number, flight date, sales channel, payment order number, etc.

[0073] Currently, e-commerce settlement systems typically involve a series of processes including data file import, reconciliation, discrepancy handling, account closure, and accounting. Reconciliation, as a crucial step, identifies discrepancies between the data source parties, providing a basis for subsequent discrepancy processing. Traditional reconciliation methods primarily include the following two:

[0074] The first option is as follows:

[0075] Step 1: Import the data into the settlement center system by manual upload or automatic import and store it in the local database. Since it is a pairwise reconciliation, the reconciliation generally involves two data tables, A1 and B1.

[0076] Step 2: Using table A1 as the base, query all data in table A1 that meets the reconciliation criteria in one go;

[0077] Step 3: Iterate through all the data retrieved in Step 2, and search in table B1 according to the matching items to see if there is any data with the same matching items. If the matching items are the same, check if the verification items are the same, find them, and mark the matching status in tables A1 and B1.

[0078] The second option is as follows:

[0079] Step 1: Import the data into the settlement center system by manual upload or automatic import and store it in the local database. Since it is a pairwise reconciliation, the reconciliation generally involves two data tables, table A2 and table B2.

[0080] Step 2: Query all data in table A2 that meets the reconciliation criteria and add them to a List collection;

[0081] Step 3: Query all data in table B2 that meets the reconciliation criteria and add them to a List collection;

[0082] Step 4: Compare data in memory using the contains interface provided by Java's List collection class, or collection classes such as set and map.

[0083] While both traditional reconciliation methods can complete the reconciliation process, they each have drawbacks. The first method requires iterating through the data in tables A1 and B1. After retrieving a record from table A1, a full table scan of table B1 is needed to check for a matching entry. For a settlement center, this can involve processing millions of records at a time, leading to frequent database read / write operations and potential database I / O bottlenecks. Searching each record individually is also very time-consuming and inefficient. The second method avoids the database I / O bottleneck issue of the first method, but when dealing with large volumes of data, the entire process must be handled within the JVM, significantly impacting the application server and potentially causing memory overflow. This method is unsuitable for sufficiently large datasets.

[0084] Therefore, to address the aforementioned problems, this application proposes a data matching method that can process large amounts of data to be matched, improving matching efficiency. Because the Redis server in the e-commerce settlement system is memory-based, the data matching method is highly efficient. The characteristics of the Set data structure in the Redis server can be utilized to perform the entire matching process within the Redis server. Only one query is needed to retrieve the data to be matched from the database. Using a data structure called Set in the Redis server, the retrieved data is placed into two Sets. Specifically, the data to be matched retrieved from one data table is placed into one Set. Basic set operations are then used to process the two Sets. For millions of data points to be matched, this application embodiment can complete the process in a very short time, significantly improving matching efficiency and performance.

[0085] Please see Figure 1 The diagram illustrates a flowchart of an embodiment of a data matching method provided by this application. This embodiment is applied to a Redis server and includes at least the following steps:

[0086] S1, write the first set of data to be matched into the first Redis set according to the preset rules.

[0087] S2, write the second set of data to be matched into the second Redis set according to the preset rules.

[0088] Before performing data matching, this application embodiment needs to obtain or receive a set of data to be matched from the local database. Since it is a pairwise matching, data matching generally involves two sets of data to be matched. One of the two sets of data to be matched is determined as the first set of data to be matched, and the other set of data to be matched is determined as the second set of data to be matched.

[0089] After obtaining the first and second sets of data to be matched, the first set of data to be matched is written to a first Redis set according to a preset rule, and the second set of data to be matched is written to a second Redis set according to a preset rule. A Redis set is a collection within a Redis server. This embodiment of the application writes the sets of data to be matched obtained from the database into Redis sets, enabling the Redis server to perform data matching, thereby improving data matching efficiency.

[0090] It should be noted that the order of S1 and S2 is not limited in the embodiments of this application. The second data set to be matched can be written into the second Redis set according to the preset rules before the first data set to be matched is written into the first Redis set according to the preset rules; the second data set to be matched can be written into the second Redis set according to the preset rules after the first data set to be matched is written into the first Redis set according to the preset rules; or the first data set to be matched can be written into the first Redis set according to the preset rules and the second data set to be matched can be written into the second Redis set according to the preset rules at the same time.

[0091] S3 performs set operations on the first and second Redis sets to obtain the matching results.

[0092] The Redis server writes the first set of data to be matched into a first Redis set, and the second set of data to be matched into a second Redis set. Then, it performs set operations on the first and second Redis sets to obtain the matching result. Set operations include intersection, difference, union, complement, and subset operations.

[0093] In this embodiment, a first set of data to be matched is written into a first Redis set according to a preset rule; a second set of data to be matched is written into a second Redis set according to a preset rule; set operations are performed on the first and second Redis sets to obtain the matching result. This embodiment utilizes the characteristics of Redis sets for data matching, which can improve the efficiency of data matching, is applicable to various types of data matching, has strong versatility, and solves the problems of time consumption and low performance of existing data matching methods.

[0094] Please see Figure 2 The flowchart illustrates another embodiment of the data matching method provided in this application, which includes at least the following steps:

[0095] S4, retrieve the first data table and the second data table.

[0096] S5. Based on the rows of data to be matched that meet the matching conditions in the first data table, generate the first set of data to be matched.

[0097] S6. Based on the rows of data in the second data table that meet the matching conditions, generate a second set of data to be matched.

[0098] In this embodiment, the Redis server can directly retrieve a first data table and a second data table from the database. These two tables are the data tables that need to be matched. After obtaining the first and second data tables, it is necessary to evaluate the tables and query the tables to find rows that meet the matching conditions, thereby generating a set of data to be matched.

[0099] The code involved in retrieving rows from a data table that meet matching criteria is as follows:

[0100] Select ORDER_ID | detailed_transaction_type='SAL_NOR'and Account_flag='Y'and MATCH_PRO_CONFIG='Y'and(match_pro_flag is null or match_pro_flag='N')

[0101] The matching conditions in this application embodiment can be time conditions, amount conditions, or fund type conditions. For example, when the matching condition is transaction dates from July 1, 2022 to September 1, 2022, rows of data whose transaction dates fall within this date range are retrieved from the data table, and these rows are identified as the rows to be matched. It should be noted that the matching conditions can be set by technical personnel according to the actual needs of the scenario, and this application embodiment does not impose any limitations.

[0102] S7, write the first set of data to be matched into the first Redis set according to the preset rules.

[0103] S8: Write the second set of data to be matched into the second Redis set according to the preset rules.

[0104] In this embodiment, the preset rules include concatenation rules for matching items and verification items. It can be understood that the set of data to be matched includes the data to be matched corresponding to the matching items and the data to be matched corresponding to the verification items, and the row data to be matched is composed of the data to be matched corresponding to the matching items and the data to be matched corresponding to the verification items. One implementation of the preset rules is to concatenate the data to be matched corresponding to the matching items and the verification items into a single string, with the matching items and verification items connected by the special character @. For example, if the data to be matched corresponding to the matching items is 1218098701,8849172284714570,SAL,CNY,2022 / 7 / 12, and the data to be matched corresponding to the verification items is 5000, then the string generated according to the preset rules is 1218098701,8849172284714570,SAL,CNY,2022 / 7 / 12@5000.

[0105] In view of this, one way S7 can write the first set of data to be matched into the first Redis set according to preset rules is as follows:

[0106] S701, according to the concatenation rules of matching items and verification items in the preset rules, concatenate the data to be matched in the first set of data to be matched to generate one or more strings;

[0107] S702, write the string to the first Redis collection.

[0108] One possible way to write a string to the first Redis set is to write the string to the first set, and then write the first set to the first Redis set. The first set resides in memory, allowing for fast data retrieval. For example, the first set might look like this:

[0109] 2218098701,5849172284714570,SAL,CNY,2022 / 5 / 12@500;

[0110] 2218098702,6849172284714570,SAL,CNY,2022 / 6 / 14@240;

[0111] 2218098703,7849172284714570,SAL,CNY,2022 / 7 / 10@400;

[0112] 2218098704,8849172284714570,SAL,CNY,2022 / 8 / 11@300.

[0113] It should be noted that the preset rule can also be that the matching item and the verification item are connected by other special characters, or other concatenation methods, as long as the data to be matched can be concatenated into a string. The embodiments of this application do not limit the specific content of the preset rule.

[0114] The possible implementation of writing the second set of data to be matched into the second Redis set according to the preset rules in S8 of this application is similar to that in S7, and can be referred to in S7, and will not be repeated here.

[0115] S9 performs set operations on the first and second Redis sets to obtain the matching results.

[0116] In this embodiment of the application, set operations are performed using intersection and difference operations.

[0117] One implementation of embodiment S9 of this application, which performs set operations on the first Redis set and the second Redis set to obtain a matching result, is as follows:

[0118] S901, Perform an intersection operation on the first Redis set and the second Redis set to obtain the first intersection;

[0119] S902, perform a difference operation on the first Redis set and the first intersection to obtain the first difference set;

[0120] S903, perform data reduction on the first difference set to obtain the second difference set;

[0121] S904, perform a difference operation on the intersection of the second Redis set and the first set to obtain the third difference set;

[0122] S905, the third difference set is pruned to obtain the fourth difference set;

[0123] S906, perform an intersection operation on the second difference set and the fourth difference set to obtain the second intersection set;

[0124] S907, generate matching results based on the first intersection and the second intersection.

[0125] Another way to implement the matching result by performing set operations on the first Redis set and the second Redis set in embodiment S9 of this application is as follows:

[0126] S908, perform an intersection operation on the first Redis set and the second Redis set to obtain the first intersection;

[0127] S909, perform a difference operation on the first Redis set and the first intersection to obtain the first difference set;

[0128] S910, perform data reduction on the first difference set to obtain the second difference set;

[0129] S911, perform a difference operation on the second Redis set and the first intersection set to obtain the third difference set;

[0130] S912, the third difference set is pruned to obtain the fourth difference set;

[0131] S913, perform an intersection operation on the second difference set and the fourth difference set to obtain the second intersection set;

[0132] S914, perform a difference operation on the second difference set and the second intersection set to obtain the fifth difference set;

[0133] S915, perform a difference operation on the fourth difference set and the second intersection set to obtain the sixth difference set;

[0134] S916, generate matching results based on the first intersection, the second intersection, the fifth difference, and the sixth difference.

[0135] In this embodiment, the intersection of the first Redis set and the second Redis set is performed. The data in the first intersection set is the fully matched data, that is, the matching items and the verification items are completely identical. The data in the first intersection set is marked as Y, and the matching status Y indicates that the data is fully matched.

[0136] Perform a difference operation on the first Redis set and the first intersection to obtain the first difference set. The data in the first difference set will have two cases: one is data with the same matching items but different verification items, i.e., data with a matching status of E; the other is data with both the matching items and the verification items being different, i.e., data with a matching status of N.

[0137] Perform a difference operation on the intersection of the second Redis set and the first set to obtain the third difference set. The data in the third difference set will have two cases: one is data with the same matching items but different verification items, i.e., data with a matching status of E; the other is data with both the matching items and the verification items being different, i.e., data with a matching status of N.

[0138] The first difference set is pruned by removing data from the strings, specifically the strings containing the check items. This yields the second difference set. The third difference set is pruned by removing data from the strings containing the check items. This yields the fourth difference set. For example, if the string in the first difference set is 2218098701,5849172284714570,SAL,CNY,2022 / 5 / 12@500, then the string in the second difference set will be 2218098701,5849172284714570,SAL,CNY,2022 / 5 / 12.

[0139] Perform an intersection operation on the second difference set and the fourth difference set to obtain the second intersection set. The data in the second intersection set consists of data with consistent matching items but inconsistent verification items, i.e., data with a matching status of E.

[0140] Perform a difference operation on the second difference set and the second intersection set to obtain the fifth difference set. The data in the fifth difference set consists of data where neither the matching item nor the checking item matches, i.e., data with a matching status of N.

[0141] Perform a difference operation on the fourth difference set and the second intersection set to obtain the sixth difference set. The data in the sixth difference set consists of data where neither the matching item nor the checking item is consistent, i.e., data with a matching status of N.

[0142] In this embodiment, the matching result includes the data corresponding to the matching item, the data corresponding to the verification item, the matching status, and the matching association number. The matching association number is the association number corresponding to the data when the matching status is Y or E, to establish the association between the matching data.

[0143] In another embodiment of the data matching method, before obtaining the first data table and the second data table in S4, this embodiment of the application further includes the following steps:

[0144] Receive data matching instructions. These instructions are sent after a scheduled task is triggered and include the data matching type.

[0145] The e-commerce settlement system in this embodiment configures data matching job names in the job manager interface, and sets the trigger time for each data matching type and the dependencies between data matching jobs. One data matching type corresponds to one data matching job name.

[0146] In e-commerce checkout systems, scheduled jobs send data matching instructions, including the data matching type, to the Redis server when the trigger time for the data matching type is reached. The scheduled task is then activated upon reaching the trigger time for the data matching type.

[0147] After receiving a data matching instruction, one way to retrieve the first and second data tables is to retrieve the first and second data tables corresponding to the data matching type. That is, retrieve the two data tables corresponding to the data matching type from the database, and then perform data matching.

[0148] In this embodiment, a first set of data to be matched is written into a first Redis set according to a preset rule; a second set of data to be matched is written into a second Redis set according to a preset rule; set operations are performed on the first and second Redis sets to obtain the matching result. This embodiment utilizes the characteristics of Redis sets for data matching, which can improve the efficiency of data matching, is applicable to various types of data matching, has strong versatility, and solves the problems of time consumption and low performance of existing data matching methods.

[0149] To facilitate a further understanding of the technical solutions provided in the embodiments of this application, the following example illustrates the data matching method provided in this application when applied to the reconciliation of payment merchant settlement data and business order data. Please refer to [link to relevant documentation]. Figure 3 The diagram shows a flowchart of an application scenario provided by an embodiment of this application.

[0150] Job configuration: Configure the specific reconciliation job name in the job manager interface, and configure the trigger time for each reconciliation step and the dependencies between jobs;

[0151] Step 1: Before reconciliation, various data source files need to be imported, including bank deposit slip data, payment merchant settlement data, business order data, payment order data, airline ticket / EMD sales report data, and third-party supplier data;

[0152] Imported into the e-commerce settlement system via manual upload or automatic import through the system interface, the data is first stored in the local database. The reconciliation involves two data tables: the payment merchant details table (i.e., the first data table) and the business order payment information table (i.e., the second data table).

[0153] Step Two: The task scheduling service will trigger a scheduled task at the designated time to begin reconciliation processing, which involves reconciling payment merchant details and business order payment information. This process is called fund reconciliation.

[0154] Step 3: Read the system configuration items, which include matching items and verification items. Query the data in the first data table that meets the reconciliation conditions (i.e., matching conditions). Based on the system configuration items, concatenate the matching items (merchant number, sending bank order number, fund type, transaction currency, transaction date) and the verification item (transaction amount) into a string and put it into the first set collection. Then, put the first set collection into the first Redis set AA.

[0155] The system configuration items differ for each reconciliation stage, meaning the matching and verification items vary for each stage. For example: Matching items for reconciling payment merchant settlement data and bank deposit slip data: payment merchant code, settlement currency, bank posting date; verification items: settlement gross amount, settlement net amount. Matching items for reconciling business order data and payment merchant settlement data: submitted bank order number, transaction type, merchant number, transaction date, currency; verification item: original currency gross amount. Matching items for reconciling business order data and payment order data: payment order number, transaction type; verification items: payment merchant, merchant number, transaction currency, transaction amount, payment product. Matching items for reconciling business order data and airline ticket data: ticket number, ticketing date, transaction type, currency type; verification item: ticket price. Matching items for reconciling business order data and supplier data: product ID, transaction type; verification item: order opening amount.

[0156] Step 4: Query the data in the second data table that meets the reconciliation criteria. According to the system configuration items, concatenate the matching items (merchant number, sending bank order number, fund type, transaction currency, transaction date) and the verification item (transaction amount) into a string and put it into the second set collection. Then put the second set collection into the second Redis collection BB.

[0157] Step 5: Find the intersection of sets AA and BB, and put the intersection into a new set CC. The data in set CC (i.e., the first intersection) is the completely matched data (the matching item and the verification item are exactly the same), that is, the records with a matching status of Y, such as... Figure 4 ;

[0158] Step Six: Divide the sets AA and CC to obtain a new set DD (containing data from E and N).

[0159] The DD set (i.e., the first difference set) can have two cases: one is that the matching items are consistent but the check items are inconsistent (data with a matching status of E), and the other is that the matching items are inconsistent (records with a matching status of N).

[0160] Step 7: Compare BB and CC and perform a difference operation to obtain a new set EE (containing data with matching states E and N).

[0161] The EE set (i.e., the third difference set) can have two cases: one is that the matching items are consistent but the check items are inconsistent (data with a matching status of E), and the other is that the matching items are inconsistent (records with a matching status of N).

[0162] Step 8: Extract each record from the DD set, removing the amount from the end of the string to obtain the DD' set (the second difference set). Extract each record from the EE set, removing the amount from the end of the string to obtain the EE' set (the fourth difference set). Find the intersection of DD' and EE' to obtain the F set (records with a matching status of E). Figure 5 ;

[0163] Step 9: Perform a second difference operation on the DD' set and the F set (i.e., the second intersection). At this point, the data in the set is the record that needs to be identified as N in the first data table.

[0164] Step 10: Perform a difference operation between the EE' set and the F set again. The data in the set at this point will be the records in the second data table that need to be identified as N.

[0165] Step 11: Generate reconciliation results. Identify data with a matching status of Y / E / N in the reconciliation results and tag them in their respective data tables. For data with matching statuses of Y and E, a matching association number needs to be marked in their respective data tables. The tagged data tables are shown in Table 1 and Table 2.

[0166] Table 1: First Data Table After Marking

[0167]

[0168] Table 2. Second data table after labeling.

[0169]

[0170]

[0171] The data table after labeling shows that there are three matching states: the matching item and the verification item are completely consistent, and the matching state is assigned the value Y; the matching item is consistent, but the verification item is inconsistent, and the matching state is assigned the value E; the matching item is inconsistent, and the matching state is assigned the value N.

[0172] Previous e-commerce settlement systems used traditional reconciliation methods, requiring a task scheduling operation each time. This was extremely time-consuming for millions of records, potentially taking 1-2 hours. Comparing each record in the database individually also resulted in poor performance, consuming significant database I / O resources and negatively impacting the e-commerce settlement system.

[0173] This application utilizes Redis set features for reconciliation, offering advantages such as fast reconciliation speed, high efficiency, and good performance. It is suitable for reconciliation scenarios with large data volumes and high business volumes, perfectly matching the characteristics of e-commerce settlement systems. Because the entire reconciliation process is processed and completed within the Redis server, the reconciliation efficiency is very high. For millions of data points, the entire process, from putting the data into the Redis server to completing various set operations and obtaining the corresponding results, can be completed in seconds. Furthermore, based on configuration, the entire reconciliation process can be applied to various types of reconciliation, exhibiting high versatility. It is suitable not only for one-to-one scenarios but also for one-to-many, many-to-many, and many-to-one scenarios.

[0174] The following section introduces a Redis server provided in this application. The Redis server described below can be referred to in correspondence with the data matching method described above.

[0175] Please see Figure 6 The diagram illustrates the structure of a Redis server provided in this application, the Redis server comprising:

[0176] The first write module 601 is used to write the first set of data to be matched into the first Redis set according to preset rules;

[0177] The second write module 602 is used to write the second set of data to be matched into the second Redis set according to the preset rules;

[0178] The operation module 603 is used to perform set operations on the first Redis set and the second Redis set to obtain the matching result.

[0179] In this embodiment of the application, the computing module 603 is specifically used for:

[0180] Perform an intersection operation on the first Redis set and the second Redis set to obtain the first intersection;

[0181] Perform a difference operation on the first Redis set and the first intersection to obtain the first difference set;

[0182] The first difference set is pruned to obtain the second difference set;

[0183] Perform a difference operation on the intersection of the second Redis set and the first set to obtain a third difference set;

[0184] The third difference set is pruned to obtain the fourth difference set;

[0185] Perform an intersection operation on the second difference set and the fourth difference set to obtain the second intersection set;

[0186] The matching result is generated based on the first intersection and the second intersection.

[0187] In this embodiment of the application, the first writing module 601 is specifically used for:

[0188] According to the concatenation rules of the matching items and verification items in the preset rules, the data to be matched in the first set of data to be matched is concatenated to generate one or more strings;

[0189] Write the string into the first Redis collection.

[0190] In this embodiment of the application, the Redis server further includes:

[0191] The acquisition module is used to acquire the first data table and the second data table.

[0192] The first generation module is used to generate the first set of data to be matched based on the row data to be matched in the first data table that meet the matching conditions;

[0193] The second generation module is used to generate the second set of data to be matched based on the row data to be matched in the second data table that meet the matching conditions.

[0194] In this embodiment of the application, the Redis server further includes:

[0195] A receiving module is used for data matching instructions, which are sent after a timed task is triggered, and the data matching instructions include a data matching type.

[0196] The acquisition module is specifically used for:

[0197] Obtain the first data table and the second data table that match the data type.

[0198] This application provides a readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods described in the above-described method embodiments.

[0199] In this embodiment, a first set of data to be matched is written into a first Redis set according to a preset rule; a second set of data to be matched is written into a second Redis set according to a preset rule; set operations are performed on the first and second Redis sets to obtain the matching result. This embodiment utilizes the characteristics of Redis sets for data matching, which can improve the efficiency of data matching, is applicable to various types of data matching, has strong versatility, and solves the problems of time consumption and low performance of existing data matching methods.

[0200] It should be noted that similar or identical parts between the various embodiments can be referred to interchangeably. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to in the description of the method embodiments.

[0201] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0202] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0203] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0204] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data matching method, characterized in that, Applied to a Redis server, the method includes: Write the first set of data to be matched into the first Redis set according to a preset rule, wherein the preset rule includes the concatenation rule for matching items and verification items; the step of writing the first set of data to be matched into the first Redis set according to the preset rule includes: concatenating the data to be matched in the first set of data to be matched according to the concatenation rule for matching items and verification items in the preset rule to generate one or more strings; and writing the strings into the first Redis set. Write the second set of data to be matched into the second Redis set according to the preset rules; Perform set operations on the first Redis set and the second Redis set to obtain a matching result. The matching result includes data corresponding to the matched items, data corresponding to the verified items, matching status, and matching association number. The set operations on the first Redis set and the second Redis set to obtain the matching result include: performing an intersection operation on the first Redis set and the second Redis set to obtain a first intersection; performing a difference operation on the first Redis set and the first intersection to obtain a first difference; performing data deletion on the first difference set to obtain a second difference; and performing data deletion on the second Redis set... The difference operation is performed between the Redis set and the first intersection to obtain a third difference set; data is removed from the third difference set to obtain a fourth difference set; the intersection operation is performed between the second difference set and the fourth difference set to obtain a second intersection set; the matching result is generated based on the first intersection set and the second intersection set; the data removal operation of the first difference set to obtain the second difference set includes: removing the data corresponding to the check items from the strings in the first difference set to obtain the second difference set; the data removal operation of the third difference set to obtain the fourth difference set includes: removing the data corresponding to the check items from the strings in the third difference set to obtain the fourth difference set.

2. The method according to claim 1, characterized in that, Before writing the first set of data to be matched into the first Redis set according to preset rules, the method further includes: Retrieve the first and second data tables; Based on the rows of data to be matched that meet the matching conditions in the first data table, the first set of data to be matched is generated; Based on the rows of data in the second data table that meet the matching conditions, a second set of data to be matched is generated.

3. The method according to claim 2, characterized in that, Before obtaining the first data table and the second data table, the method further includes: Receive a data matching instruction, which is sent after a timed task is triggered, and the data matching instruction includes a data matching type; The process of obtaining the first data table and the second data table includes: Obtain the first data table and the second data table that match the data type.

4. A Redis server, characterized in that, The Redis server includes: The first write module is used to write the first set of data to be matched into the first Redis set according to preset rules, the preset rules including the concatenation rules of matching items and verification items; the first write module is specifically used to: concatenate the data to be matched in the first set of data to be matched according to the concatenation rules of matching items and verification items in the preset rules to generate one or more strings; and write the strings into the first Redis set; The second write module is used to write the second set of data to be matched into the second Redis set according to the preset rules; The operation module is used to perform set operations on the first Redis set and the second Redis set to obtain a matching result. The matching result includes data corresponding to the matching item, data corresponding to the verification item, matching status, and matching association number. Specifically, the operation module is used to: perform an intersection operation on the first Redis set and the second Redis set to obtain a first intersection; perform a difference operation on the first Redis set and the first intersection to obtain a first difference set; perform data deletion on the first difference set to obtain a second difference set; perform a difference operation on the second Redis set and the first intersection to obtain a third difference set; perform data deletion on the third difference set to obtain a fourth difference set; perform an intersection operation on the second difference set and the fourth difference set to obtain a second intersection; and generate the matching result based on the first intersection and the second intersection. The operation module is also used to: remove data corresponding to the verification item from the strings in the first difference set to obtain the second difference set; and remove data corresponding to the verification item from the strings in the third difference set to obtain the fourth difference set.

5. The Redis server according to claim 4, characterized in that, The Redis server also includes: The acquisition module is used to acquire the first data table and the second data table. The first generation module is used to generate the first set of data to be matched based on the row data to be matched in the first data table that meet the matching conditions; The second generation module is used to generate the second set of data to be matched based on the rows of data to be matched in the second data table that meet the matching conditions.

6. A readable storage medium, characterized in that, The device stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 3.

Images