Merchant reconciliation file processing method and device based on burying point log

Abstract

The invention provides a merchant reconciliation file processing method and device based on buried point logs, and solves the problems that in the prior art, merchant file data processing speed is low, hardware resource dependency is high, and data cannot be generated in real time. After the merchant reconciliation system is transformed by adopting the scheme, the data is uniformly processed, the stability of a transaction system is remarkably improved, the pressure on a database is reduced, on the other hand, the data can be accurately processed in real time, and a merchant reconciliation file can be issued at any time; and the hysteretic quality of big data processing such as Hive and the increase of transaction volume are avoided, so that the increase of processing time is avoided.

Description

Technical Field

[0001] The present invention relates to the field of third-party payment technology, and in particular to a merchant reconciliation file processing method and device based on buried point logs. Background Art

[0002] The merchant reconciliation system is mainly used to issue various merchant transaction data files to merchants. When the transaction volume is large, the transaction cycle is long, and the number of related tables increases, the time it takes to generate merchant reconciliation data is also getting longer and longer.

[0003] Currently, merchant reconciliation systems in third-party payment systems mostly use SQL statements to associate and process data. Generally, there are three processing methods: one is to directly query transaction data through the software system, and generate data in a loop by associating multiple detailed tables based on key fields such as order numbers. The second is to use big data processing systems such as Hive to associate all detailed tables row by row based on key fields such as order numbers to generate all the data. The third is to clean the data into a wide table and then associate the tables to generate data.

[0004] The existing merchant file data processing technologies mainly include the three methods mentioned above. First, although the speed of issuing documents can reach real time, the processing speed is slow, the data is unevenly distributed, and the pressure on the database is high. Second, as the amount of data increases, the number of related tables also increases, the speed of generating data becomes slower and slower, which depends on hardware resources and has a lag. Third, based on the second solution, the number of related tables is controlled, but it also has a lag and cannot generate data in real time. Summary of the Invention

[0005] The present invention provides a merchant reconciliation file processing method and device based on buried point logs to solve the problems existing in the prior art of slow merchant file data processing speed, strong dependence on hardware resources, and inability to generate data in real time.

[0006] In a first aspect, the present invention provides a merchant reconciliation file processing method based on tracking logs, which specifically includes the following steps:

[0007] Step S1: A first system receives an order transaction request initiated by a merchant and generates a first log file based on the order transaction request; wherein the first system includes a head node subsystem, a tail node subsystem, and multiple (in this application, the "multiple" means at least two) intermediate node subsystems (in this application, the "head node subsystem", the "tail node subsystem", and the "intermediate node" subsystems are divided according to the transmission order of the information flow within the first system);

[0008] Step S2, the ELK system (ELK system composed of Elasticsearch, Logstash, Kibana, referred to as ELK system, used for log analysis and management) reads and parses the first log file, forms the parsed log file, encapsulates the parsed first log file, forms a to-be-sent Kafka message object (ProducerRecord object), and sends the to-be-sent Kafka message;

[0009] Step S3, the second system (also referred to as a merchant reconciliation system) receives the Kafka message and saves it, and verifies the integrity of the transaction order. When the transaction order information is missing data, the transaction order information is completed to form complete transaction order information.

[0010] Preferably, the step S1 specifically comprises the following steps:

[0011] Step S101, receiving the order transaction request initiated by the merchant through the first node subsystem, obtaining the basic information in the order transaction request according to the order transaction request, converting the name of the current node subsystem, the transaction type, the transaction process node and the basic information into json format data, and printing into the first log file of the second system;

[0012] Step S102, the order transaction request is sequentially transferred to a plurality of intermediate node subsystems, the name of the current node subsystem, the transaction type, the transaction process node and the contained transaction information in each intermediate node subsystem are converted into a plurality of json format data, and printed into the first log file of the second system;

[0013] Step S103, the order transaction request is transferred to the tail node subsystem, the name of the tail node subsystem, the transaction type, the transaction process node and the contained transaction completion information are converted into json format data, and printed into the first log file of the second system.

[0014] In step S101, the basic information includes merchant number, terminal, order number, amount and transaction type.

[0015] In step S101, the first log file is used only for storing the json format data required by the second system.

[0016] Preferably, the step S2 specifically comprises the following steps:

[0017] Step S201, the ELK system configures the first log file of the second system, and obtains the first log file of the second system;

[0018] In step S202, the ELK system reads the first log file, parses the data in json format piece by piece, and acquires field information according to the parsed data in json format.

[0019] In step S203, the ELK system encapsulates the parsed first log file to form a to-be-sent Kafka message object (ProducerRecord object).

[0020] In step S204, the ELK system defines a topic subject specified by the second system, and transmits the to-be-sent Kafka message object to the topic subject.

[0021] In step S202, the field information includes subsystem name, transaction type, process node, order number, merchant number, and amount.

[0022] Preferably, the step S3 specifically includes the following steps.

[0023] In step S301, a string for representing the transaction order processing state of all N node subsystems is set, wherein the length of the string is N, N is a positive integer, and each digit of the string is 0 or 1.

[0024] In step S302, the second system subscribes to the topic subject defined by the ELK system, receives the Kafka message object, and forms transaction order information.

[0025] Step S303, judging the current node subsystem according to the transaction order information; if the current node subsystem is the first node subsystem (also referred to as the first node subsystem), then according to the current node subsystem name, transaction type and order number, it is queried whether the order exists in the database, if not, then the node subsystem name, transaction type, flow node and order number contained in the transaction order message are added in the database, and the flow node state is changed to 100…0 (N-1 zeros), otherwise, if it exists, then no operation is needed; if the current node subsystem is the i-th node subsystem (i.e. the intermediate node subsystem), then according to the current node subsystem name, transaction type and order number, it is queried whether the order exists in the database, if it exists, then the transaction data contained in the current node subsystem is updated and the state of the current node subsystem is changed to 1…100…0 (i ones and N-i zeros), otherwise, if it does not exist, then the order information is queried, after the order information of all node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 10…1…0 (the first and i-th bits are 1, and the rest are 0); wherein, 2≤i≤N-1, and i is a positive integer; if the current node subsystem is the last node subsystem, then according to the current node subsystem name, transaction type and order number, it is queried whether the order exists in the database, if it exists, then the transaction data contained in the current node subsystem is updated and the state of the current node subsystem is changed to 11…1 (N ones), otherwise, if it does not exist, then the order information is queried, after the order information of all node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 100…01 (N-2 zeros).

[0026] Wherein, the first bit represents the first node subsystem, the last bit represents the last node subsystem, and the middle bits represent the intermediate node subsystems. For example, a string of characters 0000000, the first bit represents the state of the first node subsystem, and the seventh bit represents the state of the last node subsystem. If the first node subsystem receives a message, then the first bit of the string changes from 0 to 1, indicating that the message is received.

[0027] Preferably, in step S301, the string for representing the state of the N node subsystems further includes data for representing the state of the string data, denoted as state; wherein, when state is 0, it indicates that the node subsystem is in the process of flow transfer, when state is 1, it indicates that there is a missing data and it needs to be supplemented, and when state is 2, it indicates that the data is complete.

[0028] Preferably, every five minutes, the first system is queried for a transaction order set A of all transactions successful within thirty minutes, and according to the node subsystem information and transaction type contained in the transaction order set A, a transaction order set B of all transaction orders with a string of 11…1 (N 1s) within sixty minutes in the second system is queried, the transaction order set A and the transaction order set B are data compared, if the data of the transaction order set A and the transaction order set B are the same, no processing is needed; if the data of the same transaction order in the transaction order set A and the transaction order set B are not completely the same, the state of the string of the transaction order with data missing in the transaction order set A and the transaction order set B is changed to 1; if there is a transaction order in the transaction order set A but not in the transaction order set B, the basic information of the transaction order is deleted and the string of the transaction order is 100…1 (N-2 0s).

[0029] Preferably, in order to cope with data missing caused by system instability, every five minutes, the second system is queried for transaction order string data with a state of 1, according to the transaction type and order number of the transaction order, information related to the transaction order in all node subsystems is queried, and after the transaction order is completed, the state of the string data of the transaction order is changed to 2.

[0030] Preferably, the method further comprises the following step S4: when the merchant needs transaction order information data, according to the needs of the merchant and the merchant reconciliation file issued by the first system, the merchant reconciliation file is verified, if the merchant reconciliation file is inconsistent with the transaction order data in the second system, the merchant reconciliation file is regenerated, otherwise, the issued merchant reconciliation file is sent to the merchant or an API interface for downloading is provided.

[0031] In a second aspect, the application further provides a merchant reconciliation file processing device based on a buried point log, which comprises the following modules:

[0032] A log file generation module is configured to receive an order transaction request initiated by a merchant through a first system, and form a first log file according to the order transaction request; wherein the first system comprises a head node subsystem, a tail node subsystem and a plurality of intermediate node subsystems.

[0033] A log file processing and sending module is configured to read and analyze the first log file through an ELK system, form an analyzed log file, encapsulate and process the analyzed first log file, form a to-be-sent Kafka message object, and send the to-be-sent Kafka message.

[0034] The transaction order processing module is configured to receive the Kafka message through the second system and save the message, verify the integrity of the transaction order, perform data completion on the transaction order information when the transaction order information is incomplete, and form complete transaction order information.

[0035] Preferably, the log file generation module specifically comprises the following sub-modules:

[0036] The log file generation first sub-module is configured to receive a merchant-initiated order transaction request through the first node subsystem, acquire basic information in the order transaction request according to the order transaction request, perform format conversion on the name of the current node subsystem, the transaction type, the transaction process node, and the basic information, form data in the json format, and print the data into a first log file of the second system.

[0037] The log file generation second sub-module is configured to sequentially transfer the order transaction request to a plurality of intermediate node subsystems, perform format conversion on the name of the current node subsystem, the transaction type, the transaction process node, and the contained transaction information in each intermediate node subsystem, form a plurality of data in the json format, and print the data into the first log file of the second system.

[0038] The log file generation third sub-module is configured to transfer the order transaction request to a tail node subsystem, perform format conversion on the name of the tail node subsystem, the transaction type, the transaction process node, and the contained transaction completion information, form data in the json format, and print the data into the first log file of the second system.

[0039] In the log file generation first sub-module, the basic information includes a merchant number, a terminal, an order number, an amount, and a transaction type.

[0040] In the log file generation first sub-module, the first log file is used only for storing data in the json format required by the second system.

[0041] Preferably, the log file processing and sending module specifically comprises the following sub-modules:

[0042] The log file processing and sending first sub-module is configured to configure the first log file of the second system through the ELK system, and acquire the first log file of the second system.

[0043] The log file processing and sending second sub-module is configured to read the first log file through the ELK system, analyze the data in the json format piece by piece, and acquire field information according to the analyzed data in the json format.

[0044] The log file processing sending third submodule is configured to encapsulate the parsed first log file to form a to-be-sent Kafka message object through the ELK system;

[0045] The log file processing sending fourth submodule is configured to define a topic subject specified by the second system through the ELK system, and transmit the to-be-sent Kafka message object to the topic subject.

[0046] In the log file processing sending second submodule, the field information includes sub-system name, transaction type, process node, order number, merchant number, and amount of money.

[0047] Preferably, the transaction order processing module specifically includes the following submodules.

[0048] The transaction order processing first submodule is configured to set a string for representing the transaction order processing state of all N node subsystems; wherein the length of the string is N, N is a positive integer, and each digit of the string is 0 or 1.

[0049] The transaction order processing second submodule is configured to subscribe to a topic subject defined by the ELK system through the second system, and receive the Kafka message object to form transaction order information.

[0050] The transaction order processing third submodule is configured to judge the current node subsystem according to the transaction order information; if the current node subsystem is the first node subsystem, the database is queried to determine whether the order exists according to the current node subsystem name, the transaction type and the order number; if the order does not exist, the node subsystem name, the transaction type, the flow node and the order number contained in the transaction order information are added to the database, and the flow node state is changed to 100…0 (N-1 zeros); otherwise, if the order exists, no operation is needed; if the current node subsystem is the i th node subsystem, the database is queried to determine whether the order exists according to the current node subsystem name, the transaction type and the order number; if the order exists, the transaction data contained in the current node subsystem is updated, and the state of the current node subsystem is changed to 1…100…0 (i ones or N-i zeros); otherwise, if the order does not exist, the order information is queried, and after the order information of all the node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 10…1…0 (the first and i th bits are 1, and the remaining bits are 0); wherein, 2≤i≤N-1, and i is a positive integer; if the current node subsystem is the last node subsystem, the database is queried to determine whether the order exists according to the current node subsystem name, the transaction type and the order number; if the order exists, the transaction data contained in the current node subsystem is updated, and the state of the current node subsystem is changed to 11…1 (N ones); otherwise, if the order does not exist, the order information is queried, and after the order information of all the node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 100…01 (N-2 zeros).

[0051] The first bit represents the state of the first node subsystem, the last bit represents the state of the last node subsystem, and the middle bits represent the states of the intermediate node subsystems. For example, a string of characters 0000000, the first bit represents the state of the first node subsystem, and the seventh bit represents the state of the last node subsystem. If the first node subsystem receives a message, the first bit of the string changes from 0 to 1, indicating that the message is received.

[0052] Preferably, in the transaction order processing first submodule, the string representing the states of the N node subsystems further includes data for representing the state of the string data, which is set as state; when the state is 0, it indicates that the node subsystem is in the process of flow transfer; when the state is 1, it indicates that the data is missing and needs to be supplemented; and when the state is 2, it indicates that the data is complete.

[0053] Preferably, every five minutes, the first system is queried for a transaction order set A of all transactions that have been successful within the last thirty minutes for each node subsystem, and based on the node subsystem information and transaction type contained in the transaction order set A, a transaction order set B of all transaction orders with the string 11…1 (N ones) within the last sixty minutes is queried in the second system, the transaction order set A and the transaction order set B are compared, if the transaction order set A and the transaction order set B are the same, no processing is required; if the data of the same transaction order in the transaction order set A and the transaction order set B are not completely the same, the state of the string of the transaction order with missing data in the transaction order set A and the transaction order set B is changed to 1; if there is a transaction order in the transaction order set A that does not exist in the transaction order set B, the basic information of the transaction order is deleted and the string of the transaction order is 100…1 (N-2 zeros).

[0054] Preferably, in order to cope with data loss caused by system instability, every five minutes, the second system is queried for transaction order string data state 1, and based on the transaction type and order number of the transaction order, information related to the transaction order in all node subsystems is queried, and after the transaction order is completed, the string data state of the transaction order is changed to 2.

[0055] Preferably, the merchant reconciliation file processing device based on the embedded point log further comprises a transaction information issuing module for issuing a merchant reconciliation file based on the needs of the merchant and the first system when the merchant needs transaction order information data, and verifying the merchant reconciliation file, if the merchant reconciliation file is inconsistent with the transaction order data in the second system, a new merchant reconciliation file is generated, otherwise, the issued merchant reconciliation file is sent to the merchant or an API interface is provided for download.

[0056] In a third aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the merchant reconciliation file processing method based on the embedded point log according to any one of the first aspect.

[0057] In a fourth aspect, the present application also provides an electronic device, comprising a memory storing a computer program, and a processor communicatively connected to the memory, wherein the computer program is executed by the processor to implement the merchant reconciliation file processing method based on the embedded point log according to any one of the first aspect.

[0058] Compared with the prior art, the present application has the following obvious and substantial features and advantages:

[0059] The application provides a merchant reconciliation file processing method and device based on a buried point log, to solve the problems of slow merchant file data processing speed, strong hardware resource dependency and inability to generate data in real time in the prior art. After the merchant reconciliation system is modified by using the above scheme, data is uniformly processed, the stability of the transaction system is significantly improved, the pressure on the database is reduced, on the other hand, data can be processed in quasi-real time, merchant reconciliation files can be provided at any time, and the lag of big data processing such as Hive and the increase of processing time caused by the increase of transaction volume are avoided. BRIEF DESCRIPTION OF DRAWINGS

[0060] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their descriptions serve the purpose of explaining the present application, and do not constitute improper limitations on the present application. In the drawings:

[0061] Figure 1 is a flowchart of a merchant reconciliation file processing method based on a buried point log according to a preferred embodiment of the present application.

[0062] Figure 2 is a system overall architecture diagram in the preferred embodiment of the present application.

[0063] Figure 3 is a first node processing flowchart in the preferred embodiment of the present application.

[0064] Figure 4 is an intermediate node processing flowchart in the preferred embodiment of the present application.

[0065] Figure 5 is a tail node processing flowchart in the preferred embodiment of the present application.

[0066] Figure 6 is a merchant reconciliation transaction order verification processing flowchart in the preferred embodiment of the present application.

[0067] Figure 7 is a merchant reconciliation data completion processing flowchart in the preferred embodiment of the present application.

[0068] Figure 8 is a structure schematic diagram of a merchant reconciliation file processing device based on a buried point log according to a preferred embodiment of the present application. DETAILED DESCRIPTION

[0069] The application provides a merchant reconciliation file processing method and device based on a buried point log, to solve the problems of slow merchant file data processing speed, strong hardware resource dependency and inability to generate data in real time in the prior art. After the merchant reconciliation system is modified by using the above scheme, data is uniformly processed, the stability of the transaction system is significantly improved, the pressure on the database is reduced, on the other hand, data can be processed in quasi-real time, merchant reconciliation files can be provided at any time, and the lag of big data processing such as Hive and the increase of processing time caused by the increase of transaction volume are avoided.

[0070] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence, and it should be understood that the data thus used can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0071] Example 1

[0072] As Figures 1-7 shown, the method for processing a merchant statement file based on a buried point log comprises the following steps:

[0073] Step S1, the first system receives an order transaction request initiated by a merchant, and forms a first log file according to the order transaction request; wherein the first system comprises a head node subsystem, a tail node subsystem and a plurality of intermediate node subsystems.

[0074] Preferably, the step S1 specifically comprises steps S101-S103.

[0075] Step S101, receiving an order transaction request initiated by a merchant through the head node subsystem, and obtaining basic information in the order transaction request according to the order transaction request, and performing format conversion on the name of the current node subsystem, the transaction type, the transaction process node and the basic information, forming json format data, and printing into the first log file of the second system; wherein the basic information includes merchant number, terminal, order number, amount and transaction type, etc. Basic information; the first log file is only used to store the json format data required by the second system.

[0076] Step S102, the order transaction request is sequentially transferred to a plurality of intermediate node subsystems, and the name of the current node subsystem, the transaction type, the transaction process node and the contained transaction information in each intermediate node subsystem are format converted to form a plurality of json format data, and printed into the first log file of the second system.

[0077] Step S103, the order transaction request is transferred to the tail node subsystem, the name of the tail node subsystem, the transaction type, the transaction process node and the contained transaction completion information are format converted to form json format data, and printed into the first log file of the second system.

[0078] Step S2, the ELK system reads and parses the first log file, forms a parsed log file, encapsulates the parsed first log file, forms a to-be-sent Kafka message object, and sends the to-be-sent Kafka message.

[0079] In the specific implementation of the embodiment, the step S2 specifically includes steps S201-S204.

[0080] Step S201, the ELK system configures the first log file of the second system, and acquires the first log file of the second system.

[0081] Step S202, the ELK system reads the first log file, parses the data in json format piece by piece, and acquires field information according to the parsed data in json format; in step S202, the field information includes information such as a subsystem name, a transaction type, a process node, an order number, a merchant number, and an amount.

[0082] Step S203, the ELK system encapsulates the parsed first log file, and forms a to-be-sent Kafka message object.

[0083] Step S204, the ELK system defines a topic subject specified by the second system, and transmits the to-be-sent Kafka message object to the topic subject.

[0084] Step S3, the second system receives the Kafka message and saves it, verifies the integrity of the transaction order, performs data completion on the transaction order information when the transaction order information has data missing, and forms complete transaction order information.

[0085] In the specific implementation of the embodiment, the step S3 specifically includes steps S301-S303.

[0086] Step S301, a string for indicating the processing state of a transaction order of all N node subsystems is set; the length of the string is N, N is a positive integer, and each digit of the string is 0 or 1.

[0087] The string for indicating the state of the N node subsystems further includes data for indicating the state of the string data, which is set as state; when the state is 0, it indicates that the node subsystem is in the process of flow transfer; when the state is 1, it indicates that there is missing data and the missing data needs to be supplemented; and when the state is 2, it indicates that the data is complete.

[0088] Step S302, the second system subscribes to the topic topic defined by the ELK system, and receives the Kafka message object to form the transaction order information.

[0089] Step S303, according to the transaction order information, the current node subsystem is judged; if the current node subsystem is the first node subsystem, according to the current node subsystem name, transaction type and order number, whether the order exists in the database is inquired, if not, the node subsystem name, transaction type, process node and order number contained in the transaction order message are added in the database, and the process node state is changed to 100…0 (N-1 0), otherwise, if it exists, no operation is needed; if the current node subsystem is the i-th node subsystem, according to the current node subsystem name, transaction type and order number, whether the order exists in the database is inquired, if it exists, the transaction data contained in the current node subsystem is updated and the state of the current node subsystem is changed to 1…100…0 (i 1+N-i 0), otherwise, if it does not exist, the order information is inquired, after the order information of all the node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 10…1…0 (the first and i-th bits are 1, and the rest are 0); wherein, 2≤i≤N-1, and i is a positive integer; if the current node subsystem is the tail node subsystem, according to the current node subsystem name, transaction type and order number, whether the order exists in the database is inquired, if it exists, the transaction data contained in the current node subsystem is updated and the state of the current node subsystem is changed to 11…1 (N 1), otherwise, if it does not exist, the order information is inquired, after the order information of all the node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 100…01 (N-2 0).

[0090] Wherein, the first bit represents the first node subsystem, the last bit represents the tail node subsystem, and the middle bit represents the intermediate node subsystem, for example, a string of characters 0000000, the first bit represents the state of the first node subsystem, and the seventh bit represents the state of the tail node subsystem, if the first node subsystem receives the message, the first bit of the string changes from 0 to 1, indicating that the message is received.

[0091] Optionally, every five minutes, all transaction order sets A of all transactions successful within thirty minutes in each node subsystem in the first system are inquired, and according to the node subsystem information and the transaction type contained in the transaction order set A, all transaction order sets B with the string being 11…1 (N 1s) within sixty minutes in the second system are inquired, the transaction order set A and the transaction order set B are data compared, if the data of the transaction order set A and the transaction order set B are the same, no processing is needed; if the data of the same transaction order in the transaction order set A and the transaction order set B are not completely the same, the state of the string of the transaction order with data missing in the transaction order set A and the transaction order set B is changed to 1; if there is a transaction order in the transaction order set A but not in the transaction order set B, the basic information of the transaction order is deleted and the string of the transaction order is 100…1 (N-2 0s).

[0092] Optionally, in order to cope with data missing caused by unstable system, every five minutes, the data of the transaction order with the string data state being 1 in the second system is inquired, according to the transaction type and the order number of the transaction order, the information related to the transaction order in all node subsystems is inquired, and after the transaction order is completed, the string data state of the transaction order is changed to 2.

[0093] Optionally, the method further comprises the following step: when the merchant needs transaction order information data, a merchant account file is generated according to the needs of the merchant and the first system, and the merchant account file is verified, if the merchant account file is inconsistent with the transaction order data in the second system, the merchant account file is regenerated, otherwise, the generated merchant account file is sent to the merchant or an API interface for downloading is provided.

[0094] Example 2

[0095] As shown in Figure 8 the merchant account file processing device based on the log file, specifically includes a log file generation module, a log file processing and sending module and a transaction order processing module.

[0096] The log file generation module is used for receiving an order transaction request initiated by a merchant through a first system, and forming a first log file according to the order transaction request; wherein the first system includes a head node subsystem, a tail node subsystem and a plurality of intermediate node subsystems.

[0097] The log file generation module specifically includes a log file generation first submodule, a log file generation second submodule and a log file generation third submodule.

[0098] The log file generating first submodule is configured to receive a merchant-initiated order transaction request through the first node subsystem, acquire basic information in the order transaction request according to the order transaction request, perform format conversion on the name of the current node subsystem, the transaction type, the transaction process node and the basic information, form data in the json format, and print the data into a first log file of the second system.

[0099] The basic information includes a merchant number, a terminal, an order number, an amount and a transaction type. In the log file generating first submodule, the first log file is used only for storing data in the json format required by the second system.

[0100] The log file generating second submodule is configured to sequentially transfer the order transaction request to a plurality of intermediate node subsystems, perform format conversion on the name of the current node subsystem, the transaction type, the transaction process node and the contained transaction information in each intermediate node subsystem, form a plurality of data in the json format, and print the data into the first log file of the second system.

[0101] The log file generating third submodule is configured to transfer the order transaction request to a tail node subsystem, perform format conversion on the name of the tail node subsystem, the transaction type, the transaction process node and the contained transaction completion information, form data in the json format, and print the data into the first log file of the second system.

[0102] The log file processing and sending module is configured to read and analyze the first log file through an ELK system, form an analyzed log file, perform encapsulation processing on the analyzed first log file, form a to-be-sent Kafka message object, and send the to-be-sent Kafka message.

[0103] The log file processing and sending module specifically includes a log file processing and sending first submodule, a log file processing and sending second submodule, a log file processing and sending third submodule and a log file processing and sending fourth submodule.

[0104] The log file processing and sending first submodule is configured to configure the first log file of the second system through the ELK system, and acquire the first log file of the second system.

[0105] The log file processing and sending second submodule is configured to read the first log file through the ELK system, analyze the data in the json format piece by piece, and acquire field information according to the analyzed data in the json format. The field information includes a subsystem name, a transaction type, a process node, an order number, a merchant number and an amount.

[0106] The log file processing sending third submodule is configured to encapsulate the parsed first log file to form a to-be-sent Kafka message object through the ELK system.

[0107] The log file processing sending fourth submodule is configured to define a topic subject specified by the second system through the ELK system, and transmit the to-be-sent Kafka message object to the topic subject.

[0108] The transaction order processing module is configured to receive the Kafka message through the second system and save it, and verify the integrity of the transaction order, and perform data completion on the transaction order information when the transaction order information has data loss, to form complete transaction order information.

[0109] The transaction order processing module specifically includes a transaction order processing first submodule, a transaction order processing second submodule, and a transaction order processing third submodule.

[0110] The transaction order processing first submodule is configured to set a string for representing the transaction order processing state of all N node subsystems; wherein the length of the string is N, N is a positive integer, and each digit of the string is 0 or 1.

[0111] The transaction order processing second submodule is configured to subscribe to the topic subject defined by the ELK system through the second system, and receive the Kafka message object to form transaction order information.

[0112] The transaction order processing third submodule is configured to judge the current node subsystem according to the transaction order information; if the current node subsystem is the first node subsystem, the database is queried to determine whether the order exists according to the current node subsystem name, the transaction type and the order number; if the order does not exist, the node subsystem name, the transaction type, the flow node and the order number contained in the transaction order information are added to the database, and the flow node state is changed to 100…0 (N-1 zeros); otherwise, if the order exists, no operation is needed; if the current node subsystem is the i th node subsystem, the database is queried to determine whether the order exists according to the current node subsystem name, the transaction type and the order number; if the order exists, the transaction data contained in the current node subsystem is updated, and the state of the current node subsystem is changed to 1…100…0 (i ones and N-i zeros); otherwise, if the order does not exist, the order information is queried, and after the order information of all the node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 10…1…0 (the first and i th bits are 1, and the remaining bits are 0); wherein, 2≤i≤N-1, and i is a positive integer; if the current node subsystem is the last node subsystem, the database is queried to determine whether the order exists according to the current node subsystem name, the transaction type and the order number; if the order exists, the transaction data contained in the current node subsystem is updated, and the state of the current node subsystem is changed to 11…1 (N ones); otherwise, if the order does not exist, the order information is queried, and after the order information of all the node subsystems before the current node subsystem is completed, the state of the current node subsystem is changed to 100…01 (N-2 zeros).

[0113] The first bit represents the state of the first node subsystem, the last bit represents the state of the last node subsystem, and the middle bits represent the states of the intermediate node subsystems. For example, a string of characters 0000000, the first bit represents the state of the first node subsystem, and the seventh bit represents the state of the last node subsystem. If the first node subsystem receives a message, the first bit of the string changes from 0 to 1, indicating that the message is received.

[0114] Optionally, in the transaction order processing first submodule, the string for representing the states of the N node subsystems further includes data for representing the state of the string data, which is set as state; when the state is 0, it indicates that the node subsystem is in the process of flow transfer; when the state is 1, it indicates that the data is missing and needs to be supplemented; and when the state is 2, it indicates that the data is complete.

[0115] Optionally, every five minutes, all transaction order sets A of all transaction successes in each node subsystem in the first system within thirty minutes are inquired, and according to the node subsystem information and the transaction type contained in the transaction order set A, all transaction order sets B with the string of 11…1 (N 1s) within sixty minutes in the second system are inquired, the transaction order set A and the transaction order set B are data compared, if the data of the transaction order set A and the transaction order set B are the same, no processing is needed; if the data of the same transaction order in the transaction order set A and the transaction order set B are not completely the same, the state of the string of the transaction order with data loss in the transaction order set A and the transaction order set B is changed to 1; if there is a transaction order in the transaction order set A but not in the transaction order set B, the basic information of the transaction order is deleted and the string of the transaction order is 100…1 (N-2 0s).

[0116] Optionally, in order to cope with data loss caused by system instability, every five minutes, the data of the transaction order with the string data state of 1 in the second system is inquired, according to the transaction type and the order number of the transaction order, the information related to the transaction order in all node subsystems is inquired, and after the transaction order is completed, the string data state of the transaction order is changed to 2.

[0117] Optionally, the device for processing the merchant reconciliation file based on the buried point log further comprises a transaction information issuing module, which is used for issuing the merchant reconciliation file according to the needs of the merchant and the first system when the merchant needs transaction order information data, and verifying the merchant reconciliation file, if the merchant reconciliation file is inconsistent with the transaction order data in the second system, the merchant reconciliation file is regenerated, otherwise, the issued merchant reconciliation file is sent to the merchant or an API interface for downloading is provided.

[0118] The specific embodiments of the present application are described in detail above, but it is only as an example, the present application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modification and alternative of the present application are also within the scope of the present application. Therefore, any equivalent transformation and modification without departing from the spirit and scope of the present application should be covered in the scope of the present application.

Claims

1. A merchant reconciliation file processing method based on embedded point logs, characterized in that: The specific steps include: Step S1: A first system receives an order transaction request initiated by a merchant and generates a first log file according to the order transaction request; wherein the first system includes a head node subsystem, a tail node subsystem, and multiple intermediate node subsystems; Step S2: The ELK system reads and parses the first log file to form a parsed log file, encapsulates the parsed first log file to form a Kafka message object to be sent, and sends the Kafka message to be sent; Step S3: The second system receives and saves the Kafka message, and verifies the integrity of the transaction order. If there is data missing in the transaction order information, the second system completes the data to form complete transaction order information.

2. A merchant reconciliation file processing method based on embedded point logs according to claim 1, characterized in that: The step S1 specifically includes the following steps: Step S101: Receive an order transaction request initiated by a merchant through the first node subsystem, obtain basic information in the order transaction request based on the order transaction request, convert the name of the current node subsystem, transaction type, transaction process node, and the basic information into JSON format, and print it to the first log file of the second system; Step S102: The order transaction request is sequentially transferred to multiple intermediate node subsystems. The name of the current node subsystem, transaction type, transaction process node, and transaction information contained in each intermediate node subsystem are formatted and converted into multiple JSON-formatted data. The data is then printed into the first log file of the second system. Step S103: The order transaction request is transferred to the tail node subsystem. The tail node subsystem name, transaction type, transaction process node, and transaction completion information included are converted into JSON format data and printed into the first log file of the second system. The basic information includes merchant number, terminal, order number, amount and transaction type; the first log file is only used to store data in json format required by the second system.

3. A merchant reconciliation file processing method based on embedded point logs according to claim 1, characterized in that: The step S2 specifically includes the following steps: Step S201: The ELK system configures the first log file of the second system and obtains the first log file of the second system; Step S202: The ELK system reads the first log file, parses the data in JSON format one by one, and obtains field information based on the parsed data in JSON format; Step S203: The ELK system encapsulates the parsed first log file to form a Kafka message object to be sent; Step S204: The ELK system defines a topic specified by the second system, and transmits the Kafka message object to be sent to the topic; In step S202, the field information includes subsystem name, transaction type, process node, order number, merchant number, and amount.

4. A merchant reconciliation file processing method based on embedded point logs according to claim 1, characterized in that: The step S3 specifically includes the following steps: Step S301: Set a string for representing the status of all N node subsystems processing a transaction order; wherein the length of the string is N, N is a positive integer, and each digit of the string is 0 or 1; Step S302: The second system subscribes to the topic defined by the ELK system and receives the Kafka message object to form transaction order information; Step S303: The current node subsystem is judged based on the transaction order information; if the current node subsystem is the first node subsystem, the database is queried based on the current node subsystem name, transaction type and order number to see if the order already exists; if not, the node subsystem name, transaction type, process node and order number contained in the transaction order message are added to the database, and the process node status is changed to a string in the form of 1 1 + N-1 0s from left to right; otherwise, if it exists, no operation is required; if the current node subsystem is the i-th node subsystem, the database is queried based on the current node subsystem name, transaction type and order number to see if the order already exists; if so, the transaction data contained in the current node subsystem is updated and the status of the current node subsystem is changed to a string in the form of i 1s + Ni 0s from left to right; otherwise, If it does not exist, query the order information, and after completing the order information of all node subsystems before the current node subsystem, change the state of the current node subsystem to the form of a string with the first and i-th bits from left to right being 1 and the remaining bits being 0; where 2≤i≤N-1, and i is a positive integer; if the current node subsystem is the last node subsystem, query the database based on the current node subsystem name, transaction type, and order number to see if the order already exists; if so, update the transaction data contained in the current node subsystem and change the state of the current node subsystem to the form of a string with N 1s from left to right; otherwise, if it does not exist, query the order information, and after completing the order information of all node subsystems before the current node subsystem, change the state of the current node subsystem to the form of a string with 1 1+N-2 0s+1 1 from left to right; The first digit represents the first node subsystem, the last digit represents the last node subsystem, and the middle digit represents the middle node subsystem.

5. A merchant reconciliation file processing method based on embedded point logs according to claim 4, characterized in that: In step S301, the character string used to represent the status of N node subsystems also includes data used to represent the data status of the character string, which is set to state; when state is 0, it indicates that the node subsystem is in circulation; when state is 1, it indicates that data is missing and needs to be supplemented; when state is 2, it indicates that the data is complete.

6. A merchant reconciliation file processing method based on embedded point logs according to claim 5, characterized in that: Every five minutes, a query is made for a transaction order set A of all successful transactions within thirty minutes in each node subsystem of the first system. Based on the node subsystem information and transaction type contained in the transaction order set A, a query is made in the second system for a transaction order set B within sixty minutes whose character strings are in the form of N 1s from left to right. The transaction order set A and the transaction order set B are compared. If the data of the transaction order set A and the transaction order set B are the same, no processing is required. If the data of the same transaction order in the transaction order set A and the transaction order set B are not exactly the same, the state of the character strings of the transaction orders with missing data in the transaction order set A and the transaction order set B is changed to 1. If there is a transaction order that exists in the transaction order set A but not in the transaction order set B, the basic information of the transaction order is deleted and the character string of the transaction order is changed to the form of 1 1 + N-2 0s + 1 1 from left to right.

7. A merchant reconciliation file processing method based on embedded point logs according to claim 5, characterized in that: To address data loss caused by system instability, the second system queries the transaction order string data state 1 every five minutes. Based on the transaction type and order number of the transaction order, all node subsystems are queried for information related to the transaction order. After completing the transaction order, the string data state of the transaction order is changed to 2.

8. The merchant reconciliation file processing method based on the embedded point log according to claim 1 is characterized in that: A merchant reconciliation file processing method based on tracking logs also includes step S4: when the merchant needs transaction order information data, a merchant reconciliation file is issued according to the merchant's needs and the first system, and the merchant reconciliation file is checked. If the merchant reconciliation file is inconsistent with the transaction order data in the second system, the merchant reconciliation file is regenerated; otherwise, the issued merchant reconciliation file is sent to the merchant or an API interface is provided for download.

9. A merchant reconciliation file processing device based on embedded point logs, characterized in that: Specifically, it includes the following modules: a log file generation module, configured to receive an order transaction request initiated by a merchant through a first system and generate a first log file based on the order transaction request; wherein the first system includes a head node subsystem, a tail node subsystem, and a plurality of intermediate node subsystems; A log file processing and sending module is used to read and parse the first log file through the ELK system to form a parsed log file, encapsulate the parsed first log file to form a Kafka message object to be sent, and send the Kafka message to be sent; The transaction order processing module is used to receive and save the Kafka message through the second system, and verify the integrity of the transaction order. When there is data missing in the transaction order information, the transaction order information is supplemented to form complete transaction order information.

Images