Data pairing method, distribution real-time calculation method and stream processing device

Abstract

The embodiment of the present application discloses a data pairing method, a distribution real-time calculation method and a stream processing device, wherein the data pairing method comprises the following steps: obtaining a data stream identifier and an identifier code of current arrived data of the stream processing framework; reading the data stream identifier and the identifier code of the stack bottom data of a first storage unit from a second storage unit, wherein the first storage unit is a first-in-first-out storage unit for storing arrived data to be paired, and the second storage unit is used for storing the data stream identifier and the identifier code of the stack bottom data of the first storage unit; judging whether the data stream identifier of the current arrived data is consistent with the data stream identifier of the stack bottom data of the first storage unit; judging whether the identifier code of the current arrived data is consistent with the identifier code of the stack bottom data of the first storage unit when the data stream identifier is inconsistent; and pairing the stack bottom data of the first storage unit with the current arrived data when the data stream identifier is consistent.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a data pairing method, a real-time profit sharing calculation method, and a stream processing device. Background Technology

[0002] In acquiring business, acquiring institutions need to calculate revenue sharing based on transaction documents received from merchants before settling accounts. This involves first calculating transaction fees based on a predetermined rate, and then transferring the remaining amount (after deducting transaction fees) to the merchant. Typically, for transactions on day T (the transaction day), the acquiring institution will calculate revenue sharing on day T+1 and transfer the merchant's total receivables from the previous trading day (the sum of all transactions from the previous trading day minus transaction fees). This process is called "settlement."

[0003] To improve user experience, some acquiring institutions offer real-time settlement services, settling transactions as quickly as possible after they occur. This places high demands on the real-time performance of revenue sharing calculations. Real-time revenue sharing calculations can be achieved using stream processing frameworks such as Flink. When a consumer makes a card payment or electronic payment, a pair of transaction data streams and billing data streams are generated at the merchant's end. Transaction data may include order numbers and transaction amounts, while billing data may include order numbers and merchant rates, agent rates, etc., that are identical to the corresponding transaction data. These transaction and billing data streams from numerous merchants reach the acquiring institution's stream processing framework. The framework first pairs transaction and billing data with the same order numbers before calculating the revenue sharing.

[0004] However, due to various factors such as network and system, the arrival times of transaction data and billing data with the same order number to the stream processing framework are different. The time difference between the arrival of transaction data and billing data to the stream processing framework is also not fixed and is difficult to predict. Furthermore, the time required to pair transaction data and billing data with the same order number is relatively long, resulting in poor real-time performance of profit sharing calculation. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a data pairing method, a real-time profit sharing calculation method, and a stream processing device to shorten the time consumed by data pairing and ensure the real-time performance of profit sharing calculation.

[0006] According to a first aspect, embodiments of the present invention provide a data pairing method, which pairs data with the same identifier code in two data streams through a stream processing framework. The method includes: obtaining the data stream identifier and identifier code of the data currently arriving at the stream processing framework; reading the data stream identifier and identifier code of the data at the bottom of the stack of a first storage unit from a second storage unit, wherein the first storage unit is a first-in-first-out (FIFO) storage unit for storing the arrived data to be paired, and the second storage unit is used to store the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit; determining whether the data stream identifier of the currently arrived data is consistent with that of the data at the bottom of the stack of the first storage unit; if the data stream identifier of the currently arrived data is inconsistent with that of the data at the bottom of the stack of the first storage unit, determining whether the identifier code of the currently arrived data is consistent with that of the data at the bottom of the stack of the first storage unit; and if the identifier code of the currently arrived data is consistent with that of the data at the bottom of the stack of the first storage unit, retrieving the data at the bottom of the stack from the first storage unit and pairing it with the currently arrived data.

[0007] Optionally, the method further includes: when the data stream identifier of the currently arriving data is consistent with that of the data at the bottom of the first storage unit, storing the currently arriving data into the first storage unit.

[0008] Optionally, the identifier of the data is related to the generation time of the data, and the method further includes: when the identifier of the currently arriving data is inconsistent with the identifier of the data at the bottom of the first storage unit, determining whether the generation time of the currently arriving data is later than that of the data at the bottom of the first storage unit based on the identifier; when the generation time of the currently arriving data is later than that of the data at the bottom of the stack, sequentially retrieving the data at the bottom of the stack from the first storage unit, and comparing the identifier of the currently arriving data with that of the new data at the bottom of the stack, until the identifier of the currently arriving data is consistent with that of the new data at the bottom of the stack, retrieving the new data at the bottom of the stack from the first storage unit and pairing it with the currently arriving data.

[0009] Optionally, the method further includes: when the generation time of the currently arriving data is earlier than that of the bottom stack data, sequentially obtaining the data stream identifier and identifier code of the next arriving data; when the data stream identifier of the next arriving data is inconsistent with that of the bottom stack data, comparing the identifier code of the next arriving data with that of the bottom stack data until they are consistent, and retrieving the bottom stack data from the first storage unit to pair with the next arriving data.

[0010] Optionally, the method further includes: storing the unpaired stack bottom data retrieved from the first storage unit into a third storage unit; and / or storing the unpaired next arriving data into a fourth storage unit.

[0011] Optionally, after retrieving the stack bottom data from the first storage unit, the method further includes: updating the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the new stack bottom data in the first storage unit.

[0012] Optionally, the second storage unit is further used to store the number of data stored in the first storage unit, and the method further includes: decrementing the number of data by 1 when retrieving stack bottom data from the first storage unit; and / or incrementing the number of data by 1 when storing newly arrived data into the first storage unit.

[0013] Optionally, before the step of reading the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit from the second storage unit, the method further includes: reading the data quantity of the data stored in the first storage unit from the second storage unit; determining whether the data quantity is 0; if the data quantity is not 0, performing the step of reading the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit from the second storage unit; or if the data quantity is 0, storing the currently arriving data into the first storage unit, updating the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the currently arriving data, and updating the data quantity to 1; or determining whether the data quantity exceeds a predetermined threshold; if the data quantity exceeds the predetermined threshold, requesting the system to allocate more storage space to the first storage unit or allocate a new storage unit, updating the predetermined threshold according to the newly added storage space, and performing the step of reading the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit from the second storage unit; or if the data quantity does not exceed the predetermined threshold, performing the step of reading the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit from the second storage unit.

[0014] According to a second aspect, embodiments of the present invention provide a real-time profit sharing calculation method, comprising: pairing data with the same identifier code in a transaction data stream and a billing data stream according to the data pairing method described in any one of the first aspects; and calculating profit sharing based on the paired transaction data and billing data.

[0015] According to a third aspect, embodiments of the present invention provide a stream processing device for pairing data with the same identifier code in two data streams, comprising: a first storage unit, a second storage unit, and a processor, wherein the first storage unit is a first-in-first-out (FIFO) storage unit for storing arrived data to be paired; the second storage unit is used to store the data stream identifier and identifier code of the stack bottom data of the first storage unit; and the processor executes computer instructions to perform the method described in any one of the first aspects or the method described in the second aspect.

[0016] According to the data pairing method, profit sharing real-time calculation method, and stream processing device of the present invention, the stream processing framework can achieve data pairing of two data streams simply by reading and logical judgment instructions, without executing any query instructions, thereby shortening the time spent on data pairing. Since the first storage unit only stores data from one data stream and retrieves the paired data promptly after pairing, storage space utilization is improved, and the risk of data loss due to data overflow is reduced. This ensures the real-time performance of profit sharing calculation and enhances the user experience. Attached Figure Description

[0017] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0018] Figure 1 A flowchart of a data pairing method according to an embodiment of the present invention is shown;

[0019] Figure 2A An example of the frame structure of transaction data according to an embodiment of the present invention is shown;

[0020] Figure 2B An example of the frame structure of billing data according to an embodiment of the present invention is shown;

[0021] Figure 3 A flowchart is shown in the case where the identifier code of the currently arriving data is inconsistent with the identifier code of the data at the bottom of the first storage unit stack, according to an embodiment of the present invention.

[0022] Figure 4 An example is shown in the data pairing method according to an embodiment of the present invention, where the generation time of the currently arriving billing data is later than that of the transaction data at the bottom of the stack;

[0023] Figure 5 A flowchart is shown in the data pairing method according to an embodiment of the present invention when the generation time of the currently arriving data is earlier than the data at the bottom of the first storage unit stack;

[0024] Figure 6 An example is shown in the data pairing method according to an embodiment of the present invention, where the generation time of the currently arriving billing data is earlier than that of the transaction data at the bottom of the stack;

[0025] Figure 7 A flowchart of a data pairing method according to another embodiment of the present invention is shown;

[0026] Figure 8 A flowchart of a real-time profit sharing calculation method according to an embodiment of the present invention is shown;

[0027] Figure 9 A schematic diagram of a data pairing device according to an embodiment of the present invention is shown;

[0028] Figure 10 A schematic diagram of a stream processing apparatus according to an embodiment of the present invention is shown. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Figure 1 A data pairing method according to an embodiment of the present invention is illustrated. This data pairing method can be applied, for example, to the server side of an acquiring institution. It pairs data with the same identifier code in two data streams through a stream processing framework. The method may include the following steps:

[0031] S11. Obtain the data stream identifier and identifier code of the data currently arriving at the stream processing framework.

[0032] When consumers make card or electronic payments at merchants, a transaction data stream and a billing data stream are generated in pairs. These two data streams are sent to the acquiring institution's server via the network. The acquiring institution's server uses a stream processing framework such as Flink to match the data in the arriving data streams. This data may include a stream identifier and an identifier code. The stream identifier identifies whether the data belongs to a transaction data stream or a billing data stream, and the identifier code uniquely identifies the transaction that generated the data. In one specific implementation, the identifier code may be, for example, an order number. When matching data, the stream processing framework needs to pair transaction data and billing data with the same identifier code.

[0033] Specifically, the transaction data in the embodiments of the present invention may have, for example, the following characteristics: Figure 2A The frame structure shown indicates that the transaction data may include at least a data stream identifier field, an identifier code field, and a transaction amount field; the billing data in this embodiment may, for example, have the following characteristics: Figure 2B The frame structure shown indicates that the billing data may include at least a data stream identifier field, an identifier code field, and a charge rate field. The stream processing framework can obtain the data stream identifier and identifier code of the arriving data by reading the data stream identifier field and identifier code field, without having to read the entire data.

[0034] S12. Read the data stream identifier and identifier code of the stack bottom data of the first storage unit from the second storage unit.

[0035] The stream processing framework of this embodiment may include a first storage unit and a second storage unit. The first storage unit is a first-in-first-out (FIFO) storage unit used to store the arrived data to be paired, while the second storage unit is used to store the data stream identifier and identifier code of the data at the bottom of the first storage unit's stack. The data stored in the second storage unit changes as the data at the bottom of the first storage unit's stack changes, thus requiring a high read / write speed. As an optional implementation, the second storage unit can be a high-speed storage unit, such as a random access memory (RAM) unit. Since the second storage unit only stores the data stream identifier and identifier code of the data at the bottom of the first storage unit's stack, rather than the entire stack data, and the second storage unit has a high read / write speed, the processor can quickly read the data stream identifier and identifier code of the data at the bottom of the first storage unit's stack from the second storage unit.

[0036] In the initial stage, both the first and second storage units are empty. When the first data arrives at the stream processing framework, the framework stores the data in the first storage unit, which becomes the stack bottom data of the first storage unit. At the same time, the data stream identifier and identifier code of the data are stored in the second storage unit. After that, the stream processing framework waits for the second data to arrive in order to continue to perform data pairing processing.

[0037] S13. Determine whether the data stream identifier of the currently arriving data is consistent with that of the data at the bottom of the first storage unit stack. If they are inconsistent, proceed to step S14. If they are consistent, proceed to step S16.

[0038] The stream processing framework only needs to execute logical judgment instructions on the data stream identifier of the acquired data to complete step S13, and the stream processing framework has extremely high processing speed for executing logical judgment instructions.

[0039] S14. Determine whether the identifier code of the currently arriving data is consistent with the identifier code of the data at the bottom of the first storage unit stack. If they are consistent, proceed to step S15. If they are inconsistent, proceed to the subsequent process A.

[0040] As described in step S13, since the stream processing framework has a very high processing speed for executing logical judgment instructions, the stream processing framework can quickly complete steps S13 and S14.

[0041] S15. Retrieve the stack bottom data from the first storage unit and pair it with the currently arriving data.

[0042] Since the data stream identifiers of the currently arriving data and the data at the bottom of the first storage unit stack are inconsistent, it can be determined that the currently arriving data and the data at the bottom of the first storage unit stack belong to different data streams; that is, one belongs to the transaction data stream, and the other belongs to the billing data stream. Furthermore, since the identifier codes of the currently arriving data and the data at the bottom of the first storage unit stack are consistent, it can be determined that the currently arriving data and the data at the bottom of the first storage unit stack are generated as a pair from the same transaction. Therefore, the data at the bottom of the stack can be retrieved from the first storage unit and paired with the currently arriving data.

[0043] S16. Update the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the new stack bottom data in the first storage unit.

[0044] For a first-in, first-out (FIFO) storage unit, since the previous bottom data has been retrieved, the data stored after it becomes the new bottom data. To facilitate pairing the next arriving data, while retrieving the bottom data from the first storage unit, the data stream identifier and identifier code stored in the second storage unit are updated to the data stream identifier and identifier code of the new bottom data. After completing step S16, the stream processing framework returns to step S11, waiting for the next data to arrive to continue data pairing processing.

[0045] S17. Store the currently arriving data into the first storage unit.

[0046] In step S17, since the data stream identifier of the currently arriving data is consistent with that of the data at the bottom of the first storage unit stack, it can be determined that the currently arriving data and the data at the bottom of the first storage unit stack belong to the same data stream and are both arriving data to be paired. Therefore, the stream processing framework stores the currently arriving data into the first storage unit to wait for pairing. After completing step S17, the stream processing framework returns to step S11 to wait for the next data to arrive to continue the data pairing process.

[0047] Existing data matching methods typically involve querying data with the same order number from the data arriving at the acquiring institution's server using a query command. However, the computer processor takes a significant amount of time to process the query command, resulting in a lengthy data matching process. Furthermore, the server's storage unit simultaneously holds data from two data streams, requiring substantial storage space. This is especially problematic when there is a significant time difference between the arrival of transaction data and billing data with the same order number at the streaming processing framework. The large volume of stored data further increases the time consumed by the query command, and in severe cases, may even lead to data overflow and loss.

[0048] Through the above steps, the stream processing framework can achieve data pairing between two data streams simply by reading and logical judgment instructions, without executing any query instructions. Since the stream processing framework has extremely high processing speed for reading and logical judgment instructions, it can shorten the time spent on data pairing. Furthermore, the first storage unit in this embodiment only stores the data of one data stream, and the paired data is retrieved in a timely manner after pairing, which improves the utilization of storage space and reduces the risk of data loss due to data overflow.

[0049] In the data pairing method of this invention, in the event of data disorder or data loss due to various unpredictable factors such as network and system issues, a situation may arise where the identifier code of the currently arriving data is inconsistent with the identifier code of the data at the bottom of the first storage unit stack. To address this situation, such as... Figure 3 As shown, the data pairing method of this embodiment of the invention may further include the following steps:

[0050] S21. Determine whether the generation time of the currently arriving data is later than the data at the bottom of the first storage unit stack based on the identifier code. If the generation time of the currently arriving data is later than the data at the bottom of the first storage unit stack, execute step S22. If the generation time of the currently arriving data is earlier than the data at the bottom of the first storage unit stack, execute subsequent process B.

[0051] In this embodiment, when a consumer makes a card payment or electronic payment at a merchant, the merchant automatically generates transaction data and billing data corresponding to that transaction. The identifier codes of the generated transaction data and billing data are related to the time the transaction was completed; that is, the identifier codes of the generated transaction data and billing data are related to the time the data was generated. For example, if a consumer completes a transaction at 9:00 AM on January 1, 2020, the identifier code corresponding to that transaction could be, for example, 20200101090000. If a consumer completes a transaction at 9:01:30 AM on January 1, 2020, the identifier code corresponding to that transaction could be, for example, 20200101090130. Since the identifier codes of the generated transaction data and billing data are related to the time the data was generated, the stream processing framework can determine whether the time of the currently arriving data is later or earlier than the data at the bottom of the first storage unit stack based on the correspondence between the identifier codes and the data generation time. Step S21 still only needs to be completed through logical judgment instructions, so the stream processing framework can quickly complete step S21.

[0052] S22. Retrieve the data from the bottom of the stack from the first storage unit.

[0053] For ease of description, please refer to Figure 4Assuming the data stored in the first storage unit is transaction data and the currently arriving data is billing data, those skilled in the art should understand that it is also feasible for the data stored in the first storage unit to be billing data and the currently arriving data to be transaction data. In step S21, the stream processing framework determines that the generation time of the currently arriving data is later than the data at the bottom of the first storage unit stack, that is, the generation time of the currently arriving billing data is later than the transaction data at the bottom of the stack. Figure 4 In the example, the currently arriving billing data is at time t3, while the transaction data at time t1 is at the bottom of the first storage unit stack. Since time t3 is later than time t1, it's clear that the billing data at time t1, which is paired with the transaction data at the bottom of the first storage unit stack, has not yet arrived at the stream processing framework or has been lost due to network, system, or other uncertainties. Therefore, pairing processing of the data at the bottom of the first storage unit stack cannot be performed temporarily. To find the transaction data paired with the billing data at time t3, it's necessary to search in the data after the generation time of the data at the bottom of the first storage unit stack. Therefore, it's necessary to retrieve the transaction data at time t1 from the first storage unit. After retrieving the transaction data at time t1, the data at the bottom of the first storage unit stack becomes the transaction data at time t2. The data stream identifier and identifier code stored in the second storage unit are updated to the data stream identifier and identifier code of the new data at the bottom of the stack, i.e., updated to the data stream identifier and identifier code of the transaction data at time t2.

[0054] S23. Determine whether the identifier of the currently arriving data is consistent with that of the new stack bottom data. If they are inconsistent, return to step S22. If they are consistent, execute step S24.

[0055] The stream processing framework reads the identifier of the new bottom-of-stack data from the second storage unit and continues to compare whether the identifier of the currently arriving data matches that of the new bottom-of-stack data. Let's go back to... Figure 4 For example, after retrieving the transaction data at time t1, the data at the bottom of the first storage unit becomes the transaction data at time t2. Time t2 is later than time t1 but earlier than time t3. After logical judgment, the stream processing framework finds that the identifier code of the currently arriving billing data at time t3 is still inconsistent with the identifier code of the transaction data at time t2 at the bottom of the stack. Therefore, it returns to step S22 and retrieves the transaction data at time t2 from the first storage unit again. At this time, the data at the bottom of the first storage unit becomes the transaction data at time t3. After logical judgment, the stream processing framework finds that the identifier code of the currently arriving billing data at time t3 is consistent with the identifier code of the transaction data at time t3 at the bottom of the stack. Then, it can execute step S24. In this embodiment, the stream processing framework only needs to repeatedly execute the retrieve, read, and logical judgment instructions, sequentially retrieve the data at the bottom of the stack from the first storage unit, and compare the identifier code of the currently arriving data with the identifier code of the new bottom data until the identifier code of the currently arriving data is consistent with the identifier code of the new bottom data, without executing any query instructions.

[0056] As an alternative implementation, the unpaired stack bottom data retrieved from the first storage unit is stored in the third storage unit for later pairing processing.

[0057] S24. Retrieve the new stack bottom data from the first storage unit and pair it with the currently arriving data.

[0058] exist Figure 4 In the example, the stream processing framework retrieves the transaction data at time t3 from the first storage unit to pair it with the billing data that has arrived at time t3, thereby completing the data pairing process between the transaction data at time t3 and the billing data at time t3.

[0059] S25. Update the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the new stack bottom data in the first storage unit.

[0060] After completing step S25, the stream processing framework returns to step S11, waiting for the next data to arrive in order to continue performing data pairing processing.

[0061] Through steps S21 to S25, when the generation time of the currently arriving data is later than the data at the bottom of the first storage unit, the data at the bottom of the stack is sequentially retrieved from the first storage unit, and the identifier codes of the currently arriving data and the new data at the bottom of the stack are compared until the identifier codes of the currently arriving data and the new data at the bottom of the stack match. Then, the new data at the bottom of the stack is retrieved from the first storage unit and paired with the currently arriving data. This ensures that data pairing can still be achieved even if data is out of order or lost due to various unpredictable factors such as network or system issues, without requiring any query operations, thus guaranteeing the real-time nature of data pairing. In this embodiment, the data at the bottom of the stack that has not been paired is retrieved from the first storage unit, which does not occupy the storage space of the first storage unit, reducing the risk of data loss due to data overflow.

[0062] Figure 5 It shows Figure 4 The subsequent process B, such as Figure 5 As shown, when the generation time of the currently arriving data is earlier than the data at the bottom of the first storage unit stack, the data pairing method of this embodiment may further include the following steps:

[0063] S31. Obtain the data stream identifier and identifier code of the next arriving data.

[0064] For ease of description, please refer to Figure 6 Here, it is also assumed that the data stored in the first storage unit is transaction data, and the currently arriving data is billing data. In step S21, the stream processing framework determines that the generation time of the currently arriving data is earlier than that of the data at the bottom of the first storage unit stack, that is, the generation time of the currently arriving billing data is earlier than that of the transaction data at the bottom of the stack. Figure 6In the example, the currently arriving billing data is at time t1, while the transaction data at time t3 is at the bottom of the first storage unit stack. Since time t1 is earlier than time t3, it's clear that the transaction data at time t1 that is paired with the currently arriving billing data has not yet been stored in the first storage unit or has been lost due to uncertainties such as network and system issues. Therefore, data pairing processing cannot be performed on the currently arriving data. To find the transaction data that pairs with the transaction data at time t3 at the bottom of the first storage unit stack, it's necessary to search among the billing data generated after the time the currently arriving billing data at time t1 was generated. Therefore, it's necessary to receive the next arriving billing data.

[0065] S32. Determine whether the data stream identifier of the next arriving data is consistent with that of the data at the bottom of the stack. If they are inconsistent, proceed to step S33. If they are consistent, proceed to step S36.

[0066] Similar to step S13, the stream processing framework reads the data stream identifier and identifier code of the stored data at the bottom of the first storage unit from the second storage unit. Step S32 can be completed by executing a logical judgment instruction on the data stream identifier of the next arriving data and the data at the bottom of the stack. When the data stream identifier of the next arriving data is consistent with that of the data at the bottom of the first storage unit, the next arriving data and the data at the bottom of the first storage unit belong to the same data stream. Therefore, the stream processing framework executes step S36 to store the next arriving data into the first storage unit to wait for pairing.

[0067] S33. Determine whether the identifier of the next arriving data is consistent with the identifier of the data at the bottom of the stack. If they are inconsistent, return to step S31. If they are consistent, execute step S34.

[0068] Let's go back to Figure 6 For example, when the data stream identifier of the next arriving data is inconsistent with that of the bottom data in the stack, the next arriving data is billing data at time t2, which is later than time t1 but earlier than time t3. After logical judgment, the stream processing framework finds that the identifier of the currently arriving billing data at time t2 is still inconsistent with the identifier of the transaction data at time t3 at the bottom of the stack. Therefore, it returns to step S31 to receive the next arriving data. When the next arriving data is billing data at time t3 that is inconsistent with the data stream identifier of the bottom data in the stack, the stream processing framework finds that the identifier of the arriving billing data at time t3 is consistent with the identifier of the transaction data at time t3 at the bottom of the stack. Then, it can execute step S34. In this embodiment, the stream processing framework only needs to repeatedly execute the read and logical judgment instructions to sequentially obtain the data stream identifier and identifier of the next arriving data. When the data stream identifier of the next arriving data is inconsistent with that of the bottom data in the stack, it compares the identifier of the next arriving data with that of the bottom data in the stack until the identifier of the next arriving data is consistent with that of the bottom data in the stack, without executing any query instructions.

[0069] As an optional implementation, the next arriving data that is not paired is stored in the fourth storage unit for later pairing processing. In this embodiment, the third storage unit can be the same storage unit as the fourth storage unit, or it can be a different storage unit.

[0070] S34. Retrieve the stack bottom data from the first storage unit and pair it with the next arriving data.

[0071] exist Figure 6 In the example, the stream processing framework retrieves the transaction data at time t3 from the first storage unit to pair it with the billing data that arrives at time t3, thereby completing the data pairing process between the transaction data at time t3 and the billing data at time t3.

[0072] S35. Update the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the new stack bottom data in the first storage unit.

[0073] After completing step S35, the stream processing framework returns to step S11, waiting for the next data to arrive in order to continue performing data pairing processing.

[0074] Through steps S31 to S35 above, when the generation time of the current arriving data is earlier than that of the bottom stack data, the data stream identifier and identifier code of the next arriving data are obtained sequentially. When the data stream identifier of the next arriving data is inconsistent with that of the bottom stack data, the identifier codes of the next arriving data and the bottom stack data are compared until they match. Then, the bottom stack data is retrieved from the first storage unit and paired with the next arriving data. Thus, even if data is out of order or lost due to various unpredictable factors such as network and system issues, data pairing can still be achieved without any query operation, ensuring the real-time performance of data pairing.

[0075] In some optional embodiments of the present invention, the second storage unit is further used to store the amount of data stored in the first storage unit. The amount of data stored in the second storage unit is updated simultaneously when data is stored in or retrieved from the first storage unit. Since the stream processing framework can know the amount of data stored in the first storage unit in real time by reading the second storage unit, it can request the system to allocate more storage space to the first storage unit or allocate a new storage unit before the storage space of the first storage unit is exhausted, thereby avoiding data loss due to data overflow. In this optional embodiment, such as... Figure 7 As shown, the data pairing method may include the following steps:

[0076] S401. Obtain the data stream identifier and identifier code of the data currently arriving at the stream processing framework. For details, please refer to the description of step S11.

[0077] S402. Read the amount of data stored in the first storage unit from the second storage unit.

[0078] Since the second storage unit also stores the amount of data stored in the first storage unit, the stream processing framework can directly read that amount of data from the second storage unit without having to count the storage space occupied by the first storage unit, thus saving the processing time of the stream processing framework.

[0079] S403. Determine if the number of data is 0. If the number of data is 0, proceed to step S404. If the number of data is not 0, proceed to step S405.

[0080] S404. Store the currently arriving data into the first storage unit and increment the data count by 1.

[0081] When the data quantity is 0, it indicates that the first storage unit is empty. In this case, the stream processing framework directly stores the currently arriving data into the first storage unit, and simultaneously updates the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the currently arriving data. Since the first storage unit now contains the currently arriving data, it is no longer empty. Therefore, the stream processing framework updates the data quantity stored in the second storage unit to 1. After completing step S404, the stream processing framework waits for the next arriving data to continue performing data pairing processing.

[0082] S405. Determine whether the number of data exceeds a predetermined threshold. If the number of data exceeds the predetermined threshold, proceed to step S406. If the number of data does not exceed the predetermined threshold, proceed to step S407.

[0083] In this embodiment, those skilled in the art can reasonably set the predetermined threshold according to the storage space of the first storage unit, thereby avoiding data loss due to data overflow.

[0084] S406. Request the system to allocate more storage space to the first storage unit or allocate a new storage unit, and update the predetermined threshold according to the newly added storage space.

[0085] When the amount of data exceeds a predetermined threshold, it indicates that the storage space of the first storage unit will be exhausted. To avoid data loss due to data overflow, the stream processing framework can request the system to allocate more storage space to the first storage unit or allocate a new storage unit. After the storage space is increased, the stream processing framework updates the predetermined threshold based on the increased storage space. After completing step S406, the stream processing framework executes step S407.

[0086] Of course, in this embodiment, if the system enters an idle period, such as in the early morning when consumer card swiping or electronic payment is relatively sparse, and the amount of data is much less than the predetermined threshold, the stream processing framework can release some of the storage space of the first storage unit and update the predetermined threshold at the same time to save system resources.

[0087] Since the frequency of consumer card swiping or electronic payment varies greatly between busy and off-peak hours, in this embodiment, the system can flexibly adjust the storage space of the first storage unit according to the frequency of consumer card swiping or electronic payment, instead of allocating a fixed large storage space to the first storage unit to avoid data overflow during busy hours, thus saving system resources.

[0088] S407. Read the data stream identifier and identifier code of the stack bottom data of the first storage unit from the second storage unit. For details, please refer to the description of step S12.

[0089] S408. Determine whether the data stream identifier of the currently arriving data is consistent with that of the data at the bottom of the first storage unit stack. If they are consistent, proceed to step S409. If they are inconsistent, proceed to step S410. For details, please refer to the description of step S13.

[0090] S409. Store the currently arriving data into the first storage unit and increment the data count by 1.

[0091] Since the first storage unit contains the currently arriving data, the stream processing framework needs to update the amount of data stored in the second storage unit by incrementing the data count by 1. After completing step S409, the stream processing framework waits for the next data to arrive in order to continue performing data pairing processing.

[0092] S410. Determine whether the identifier code of the currently arriving data is consistent with the identifier code of the data at the bottom of the first storage unit stack. If they are consistent, execute step S411. If they are inconsistent, execute the subsequent process A. For details, please refer to the description of step S14.

[0093] S411. Retrieve the stack bottom data from the first storage unit and pair it with the currently arrived data. For details, please refer to the description of step S15.

[0094] S412. Update the data stream identifier and identifier code stored in the second storage unit to the data stream identifier and identifier code of the new stack bottom data in the first storage unit, and decrement the data quantity by 1.

[0095] Since the previous stack bottom data has been retrieved, the stream processing framework needs to update the data stream identifier and identifier code stored in the second storage unit with the data stream identifier and identifier code of the new stack bottom data in the first storage unit, and decrement the number of data stored in the second storage unit by 1. After completing step S412, the stream processing framework waits for the next data to arrive in order to continue performing data pairing processing.

[0096] Through steps S401 to S412, the stream processing framework can achieve data pairing of two data streams by simply reading and logical judgment instructions, which shortens the time spent on data pairing. It can also adjust the storage space of the first storage unit according to the amount of data, avoid data loss due to data overflow, and save system resources.

[0097] like Figure 8 As shown, this embodiment of the invention also provides a real-time profit-sharing calculation method, which can be applied, for example, to the server side of an acquiring institution, and may include the following steps:

[0098] S51. Pair data with the same identifier code in the transaction data stream and the billing data stream.

[0099] When a consumer makes a card payment or electronic payment at a merchant, a transaction data stream and a billing data stream are generated in pairs. In step S51, the generated transaction data stream and billing data stream are paired using a stream processing framework to obtain transaction data and billing data with the same identifier. This can be achieved using the method described above. Figures 1 to 6 The data pairing method shown in the embodiment is used to pair transaction data and billing data with the same identifier code, and will not be described in detail here.

[0100] S52. Calculate profit sharing based on the paired transaction data and billing data.

[0101] After obtaining transaction data and billing data with the same identification code, the profit sharing can be calculated based on the transaction amount in the transaction data and the deduction rate in the billing data.

[0102] In the real-time profit sharing calculation method of this embodiment, since the stream processing framework can achieve data pairing of transaction data stream and billing data stream by simply reading and logical judgment instructions, without executing any query instructions, the time spent on data pairing can be shortened and the risk of data loss due to data overflow can be reduced. Thus, real-time profit sharing calculation is realized, and the funds after deducting transaction fees can be transferred to merchants in real time, improving the user experience.

[0103] Accordingly, such as Figure 9As shown, this embodiment of the invention also provides a data pairing device, which can be applied to the server side of an acquiring institution. This device pairs data with the same identifier code in two data streams using a stream processing framework. The data pairing device may include:

[0104] The acquisition module 61 is used to acquire the data stream identifier and identifier code of the data currently arriving at the stream processing framework. For details, please refer to the description of step S11.

[0105] The reading module 62 is used to read the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit from the second storage unit. The first storage unit is a first-in-first-out type storage unit used to store the arrived data to be paired. The second storage unit is used to store the data stream identifier and identifier code of the data at the bottom of the stack of the first storage unit. For details, please refer to the description of step S12.

[0106] The first judgment module 63 is used to determine whether the data stream identifier of the currently arriving data is consistent with the data at the bottom of the first storage unit stack. For details, please refer to the description of step S13.

[0107] The second judgment module 64 is used to determine whether the identifier code of the currently arrived data is consistent with that of the data stream identifier of the data at the bottom of the first storage unit stack when the data stream identifier of the currently arrived data is inconsistent with that of the data at the bottom of the first storage unit stack. For details, please refer to the description of step S14.

[0108] The pairing module 65 is used to retrieve the stack bottom data from the first storage unit and pair it with the currently arriving data when the identifier code of the currently arriving data matches that of the data at the bottom of the first storage unit. For details, please refer to the description of step S14.

[0109] Through the modules described above, the stream processing framework can achieve data pairing between two data streams simply by reading and logical judgment instructions, without executing any query instructions, thereby shortening the time spent on data pairing. Furthermore, the first storage unit in this embodiment stores only the data of one data stream and retrieves the paired data promptly after pairing, improving storage space utilization and reducing the risk of data loss due to data overflow.

[0110] For details of the various components of the data pairing device according to embodiments of the present invention, please refer to the relevant references. Figures 1 to 8 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0111] like Figure 10As shown, this embodiment of the invention also provides a stream processing device, which can be applied to the server side of an acquiring institution for pairing data with the same identification code in two data streams. The stream processing device may include a first storage unit 71, a second storage unit 72, and a processor 73.

[0112] The first storage unit 71 is a first-in-first-out (FIFO) storage unit used to store the arrived data to be paired, and may include at least one solid-state storage device such as a disk storage device or a flash memory device.

[0113] The second storage unit 72 is used to store the data stream identifier and identifier code of the stack bottom data of the first storage unit, and may include high-speed storage devices such as random access memory.

[0114] Processor 73 can be a Central Processing Unit (CPU). Processor 52 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. Processor 73 executes various functional applications and data processing by running software instructions, that is, it implements the data pairing method and the real-time calculation method in the above method embodiments.

[0115] For specific details regarding the aforementioned stream processing equipment, please refer to the relevant documentation. Figures 1 to 8 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0117] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for distributing real-time computations, characterized in that, The method for pairing data with the same identification code in a transaction data stream and a billing data stream through a stream processing framework comprises: obtaining a data stream identification and an identification code of current arrived data in the stream processing framework, the data stream identification being used to identify whether the data belongs to the transaction data stream or the billing data stream, and the identification code being used to uniquely identify a transaction generating the data; reading the data stream identification and the identification code of the bottom data of the first storage unit from a second storage unit, wherein the first storage unit is a first-in-first-out storage unit used to store arrived data to be paired, and the second storage unit is used to store the data stream identification and the identification code of the bottom data of the first storage unit; judging whether the data stream identification of the current arrived data is consistent with that of the bottom data of the first storage unit; when the data stream identification of the current arrived data is not consistent with that of the bottom data of the first storage unit, judging whether the identification code of the current arrived data is consistent with that of the bottom data of the first storage unit; when the identification code of the current arrived data is consistent with that of the bottom data of the first storage unit, pairing the bottom data of the first storage unit with the current arrived data; calculating a commission according to the paired transaction data and billing data.

2. The method of claim 1, wherein, Further comprising: when the data stream identification of the current arrived data is consistent with that of the bottom data of the first storage unit, storing the current arrived data in the first storage unit.

3. The method of claim 1, wherein, The identification code of the data is related to the time of generating the data, and the method further comprises: when the identification code of the current arrived data is not consistent with that of the bottom data of the first storage unit, judging whether the time of generating the current arrived data is later than that of the bottom data of the first storage unit according to the identification code; when the time of generating the current arrived data is later than that of the bottom data, sequentially taking out the bottom data from the first storage unit, and comparing the identification code of the current arrived data with that of the new bottom data until the identification code of the current arrived data is consistent with that of the new bottom data, and then pairing the new bottom data of the first storage unit with the current arrived data.

4. The method of claim 3, wherein, Further comprising: when the time of generating the current arrived data is earlier than that of the bottom data, sequentially obtaining the data stream identification and the identification code of the next arrived data, and comparing the identification code of the next arrived data with that of the bottom data until the identification code of the next arrived data is consistent with that of the bottom data, and then pairing the bottom data of the first storage unit with the next arrived data.

5. The method of claim 3, wherein, Further comprising: storing the unpaired bottom data taken out from the first storage unit in a third storage unit; and / or storing the unpaired next arrived data in a fourth storage unit.

6. The method according to any one of claims 1-5, characterized in that, After taking out the bottom data from the first storage unit, the method further comprises: updating the data stream identification and the identification code stored in the second storage unit to the data stream identification and the identification code of the new bottom data of the first storage unit.

7. The method according to any one of claims 1-5, characterized in that, The second storage unit is also used to store the data quantity of the data stored in the first storage unit, and the method further comprises: decrementing the data quantity by 1 when the stack bottom data of the first storage unit is fetched; and / or incrementing the data quantity by 1 when newly arrived data is stored into the first storage unit.

8. The method of claim 7, wherein, Before the step of reading the data stream identifier and the identification code of the stack bottom data of the first storage unit from the second storage unit, the method further comprises: reading the data quantity of the stored data of the first storage unit from the second storage unit; determining whether the data quantity is 0, and when the data quantity is not 0, performing the step of reading the data stream identifier and the identification code of the stack bottom data of the first storage unit from the second storage unit; or when the data quantity is 0, storing the current arrived data into the first storage unit, updating the data stream identifier and the identification code stored in the second storage unit to be the data stream identifier and the identification code of the current arrived data, and updating the data quantity to be 1; or determining whether the data quantity exceeds a predetermined threshold, and when the data quantity exceeds the predetermined threshold, requesting the system to allocate more storage space to the first storage unit or to allocate a new storage unit, updating the predetermined threshold according to the newly added storage space, and performing the step of reading the data stream identifier and the identification code of the stack bottom data of the first storage unit from the second storage unit; or when the data quantity does not exceed the predetermined threshold, performing the step of reading the data stream identifier and the identification code of the stack bottom data of the first storage unit from the second storage unit.

9. A stream processing device for pairing data having the same identification code in a transaction data stream and a billing data stream, characterized by, comprises: a first storage unit, a second storage unit, and a processor, wherein the first storage unit is a first-in-first-out storage unit for storing arrived data to be paired; the second storage unit is used for storing the data stream identifier and the identification code of the stack bottom data of the first storage unit; the processor executes the method according to any one of claims 1-8 by executing computer instructions.

Images