Transaction flow processing method and device, equipment and storage medium
By asynchronously distributing transaction flow files to multiple Topics in the accounting engine system and performing subpoena split processing, the problems of low efficiency of transaction flow processing and waste of resources in the existing technology are solved, and efficient transaction flow data processing and system are realized Performance improvement.
Patent Information
- Application Number
- CN202411072448.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
AI Technical Summary
When handling credit card transaction flows, the existing accounting engine system is inefficient and has serious waste of resources, which affects the timeliness of accounting.
The transaction flow processing method is adopted, and the transaction flow files are distributed asynchronously to multiple Topics in the kafka message queue through the data processing layer, and the summons are split based on the configuration information of the parameter configuration layer to realize efficient processing of transaction flow data.
It improves the processing efficiency of transaction flow data, reduces resource waste, solves the problem of uneven busy and idle application containers, and improves the scalability and performance of the system.
Smart Images

Figure CN118939507A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of financial technology, and in particular to a transaction flow processing method, device, equipment and storage medium. Background Art
[0002] The existing accounting engine system has a low efficiency in splitting transactions due to the large amount of transaction flow data, which takes a long time, thus affecting the performance of the accounting engine system. For example, the credit card accounting engine system is a system that splits credit card transaction flows into accounting vouchers through accounting entry rule parameters. With the continuous development of business and the continuous changes in accounting standards, the average daily credit card accounting transaction flow (i.e. GL flow) is about 80 million, with a peak of about 140 million. The accounting entry rules are set according to the flow elements, and the number of parameters is as high as 60,000 sets. 140 million transaction flows are gradually matched with more than 60,000 sets of entry rules, which not only consumes resources but also takes time, and affects the timeliness of accounting.
[0003] Therefore, there is an urgent need for a transaction flow processing method that can not only improve the processing efficiency of transaction flow data but also reduce resource waste. Summary of the invention
[0004] The main purpose of the present invention is to provide a transaction flow processing method, device, equipment and storage medium, aiming to solve the technical problems in the prior art that the processing efficiency of transaction flow data is low and resources cannot be effectively utilized.
[0005] To achieve the above object, the present invention provides a transaction flow processing method, which is applied to an accounting engine system, the system comprising an access layer, a data processing layer and a parameter configuration layer, and the method comprises:
[0006] Acquiring a transaction flow file from the access layer based on the data processing layer;
[0007] The transaction flow data in each of the transaction flow files are asynchronously distributed to multiple Topics of the Kafka message queue one by one in turn through the data processing layer;
[0008] The data processing layer is used to read configuration information from the parameter configuration layer, and based on the configuration information, the transaction flow data in each Topic is subjected to ticket splitting processing to obtain a transaction flow splitting result.
[0009] Optionally, the system further comprises a basic platform layer, and the data processing layer comprises a pipeline file splitting application;
[0010] Correspondingly, the step of asynchronously distributing the transaction flow data in each of the transaction flow files to multiple Topics of the Kafka message queue one by one through the data processing layer includes:
[0011] Using the transaction flow file splitting application to write the transaction flow data in each of the transaction flow files into multiple accounting engine libraries of the basic platform layer;
[0012] After the transaction flow data is written into each of the accounting engine libraries, reading the transaction flow data in each of the accounting engine libraries;
[0013] The transaction flow data in each accounting engine library is asynchronously distributed one by one in turn to multiple Topics of the Kafka message queue in the basic platform layer.
[0014] Optionally, the data processing layer further includes an engine pipeline processing application, and the parameter configuration layer includes an engine parameter application;
[0015] Accordingly, the step of using the data processing layer to read the configuration information from the parameter configuration layer, and performing ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result includes:
[0016] Utilize the engine flow processing application to consume the transaction flow data in each of the topics to obtain the consumed transaction flow data;
[0017] Reading configuration information from the engine parameter application via the engine pipeline processing application;
[0018] Based on the configuration information, the post-consumption transaction flow data is subjected to invoice splitting processing to obtain a transaction flow splitting result, and the transaction flow splitting result is written into the accounting engine library.
[0019] Optionally, after the step of performing invoice splitting processing on the post-consumption transaction flow data based on the configuration information to obtain the transaction flow splitting result, the step further includes:
[0020] The transaction flow splitting results are sent back to the corresponding engine flow processing applications through multiple asynchronous update topics of the Kafka message queue for consumption;
[0021] After the engine pipeline processing application consumes the corresponding asynchronously updated Topics, the transaction pipeline splitting result is updated.
[0022] Optionally, the configuration information includes entry rules, and the step of using the data processing layer to read the configuration information from the parameter configuration layer includes:
[0023] When the data processing layer receives the transaction flow data, the MD5 value corresponding to the transaction flow data is calculated according to the accounting elements;
[0024] Based on the MD5 value corresponding to the transaction flow data, a preset entry rule-MD5 value mapping relationship table is queried to obtain a query result;
[0025] If the query result indicates that the MD5 value corresponding to the transaction flow data has a corresponding target entry rule, the target entry rule is read from the entry rules in the parameter configuration layer.
[0026] Optionally, after the step of performing ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result, the step further includes:
[0027] The transaction flow data of the failed splitting is written into the transaction flow splitting failure table, and the transaction flow splitting failure table is scanned through the abnormal re-entry scheduled task to obtain the splitting failure information;
[0028] Writing the split failure information into the abnormal reentry Topic of the kafka message queue;
[0029] The abnormal reentry Topic is consumed by the engine pipeline processing application of the data processing layer to obtain abnormal transaction flow data after consumption, and the abnormal transaction flow data after consumption is subjected to ticket splitting processing to obtain a re-splitting result;
[0030] If the re-splitting result indicates that the transaction is successfully split, the re-splitting result is written into the database, and general ledger accounting and account-by-account accounting are performed;
[0031] If the re-splitting result indicates that the pipeline splitting fails, an abnormal notification message is generated and sent to the system user.
[0032] Optionally, the system further comprises a system monitoring layer;
[0033] The system monitoring layer is used to monitor the operation of the access layer, the data processing layer, the parameter configuration layer and the basic platform layer to obtain monitoring results;
[0034] The system monitoring layer is further used to generate daily monitoring reports and / or alarm information according to the monitoring results;
[0035] The system monitoring layer is also used to push the daily monitoring report and / or alarm information to system users.
[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes a transaction flow processing device, which is applied to an accounting engine system, the system comprising an access layer, a data processing layer and a parameter configuration layer, and the device comprises:
[0037] A file acquisition module, used for acquiring a transaction flow file from the access layer based on the data processing layer;
[0038] A data distribution module, used to asynchronously distribute the transaction flow data in each of the transaction flow files to multiple Topics of the Kafka message queue one by one in turn through the data processing layer;
[0039] The data processing module is used to use the data processing layer to read the configuration information from the parameter configuration layer, and perform ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result.
[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes a transaction flow processing device, which includes: a memory, a processor, and a transaction flow processing program stored in the memory and executable on the processor, and the transaction flow processing program is configured to implement the steps of the transaction flow processing method described above.
[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a transaction flow processing program is stored, and when the transaction flow processing program is executed by a processor, the steps of the transaction flow processing method described above are implemented.
[0042] The present invention discloses a transaction flow processing method, which is applied to an accounting engine system, and the system includes an access layer, a data processing layer, and a parameter configuration layer; the present invention obtains a transaction flow file from the access layer based on the data processing layer; the transaction flow data in each transaction flow file is asynchronously distributed to multiple Topics of a kafka message queue one by one in turn through the data processing layer; the data processing layer is used to read configuration information from the parameter configuration layer, and the transaction flow data in each Topic is subjected to ticket splitting processing based on the configuration information to obtain a transaction flow splitting result. Since the present invention asynchronously distributes the transaction flow data in each transaction flow file to multiple Topics of a kafka message queue one by one in turn through the data processing layer, and then performs ticket splitting processing on the transaction flow data in each Topic based on the configuration information, compared with the prior art, the present invention solves the problem of uneven busyness of the processing application containers of each transaction flow data when processing the transaction flow file, thereby improving the processing efficiency of the transaction flow data and reducing resource waste. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 This is a flow chart of the first embodiment of the transaction flow processing method of the present invention;
[0044] Figure 2It is a schematic diagram of the architecture of the accounting engine system in the transaction flow processing method of the present invention;
[0045] Figure 3 This is a flow chart of a second embodiment of the transaction flow processing method of the present invention;
[0046] Figure 4 This is a flow chart of entry rule matching in the transaction flow processing method of the present invention;
[0047] Figure 5 This is a flow chart of a third embodiment of the transaction flow processing method of the present invention;
[0048] Figure 6 It is a structural block diagram of the first embodiment of the transaction flow processing device of the present invention;
[0049] Figure 7 It is a structural diagram of a transaction flow processing device in a hardware operating environment involved in an embodiment of the present invention.
[0050] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0051] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0052] The embodiment of the present invention provides a transaction flow processing method, referring to Figure 1 , Figure 1 It is a flowchart of the first embodiment of the transaction flow processing method of the present invention.
[0053] In this embodiment, the transaction flow processing method is applied to an accounting engine system, the system includes an access layer, a data processing layer and a parameter configuration layer, and the method includes steps S10 to S30:
[0054] Step S10: Acquire transaction flow files from the access layer based on the data processing layer.
[0055] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication and program running functions, such as a server, a tablet computer, a personal computer, etc., or an electronic device capable of realizing the above functions, a transaction flow processing device, etc. The following takes an accounting engine system including a transaction flow processing device as an example to illustrate this embodiment and the following embodiments.
[0056] It should be noted that the access layer can include transaction flow files generated by the distribution center and various credit card business parties (such as basic accounting, expense adjustments, small loan joint loan systems, etc.), providing the main data input source for the entire accounting engine system.
[0057] The data processing layer may include a transaction flow file splitting application (i.e., EngineSplitFileBatch) and an engine flow processing application (CGMA), wherein the main function of the transaction flow file splitting application is to send the transaction flow data after the transaction flow file is split to the engine flow processing application, and the main function of the engine flow processing application is to consume the transaction flow data, split the transaction flow data into corresponding subpoenas and write the subpoena details into the database.
[0058] Parameter configuration layer: mainly includes engine parameter application (CGMP), whose function is to provide configuration information for the data processing layer, such as general ledger entry rule configuration information, general ledger account configuration information, general ledger account configuration information, account type grouping configuration information and funding channel grouping configuration information.
[0059] In a specific implementation, the transaction flow files can be pulled from the distribution center of the access layer through the flow file splitting application's various sub-containers.
[0060] Step S20: The transaction flow data in each of the transaction flow files are asynchronously distributed to multiple Topics of the Kafka message queue one by one in turn through the data processing layer.
[0061] It should be explained that the above-mentioned accounting engine system also includes a basic platform layer, which may include an accounting engine library, an accounting general ledger library, a redis cache component, a kafka message queue component, etc. Its main function is to provide data storage and data transmission components for the parameter configuration layer and the data processing layer.
[0062] It should be understood that after each sub-container of the old accounting engine system grabs the transaction flow file from the distribution center, it processes the transaction flow data in the transaction flow file in the current container, and then grabs a new transaction flow file from the distribution center for processing. Since the data volume of each transaction flow file varies greatly, the processing time of the sub-container that grabs a large file is much longer than that of a small file. This time difference (uneven busy and idle) affects the time of the entire data processing process and makes the system scalability very poor.
[0063] Therefore, in order to improve the processing efficiency of transaction flow data and make full use of system resources, this embodiment reads the transaction flow data in each transaction flow file through the flow file splitting application, and then asynchronously distributes them to multiple Topics of the kafka message queue one by one (for example, 8 Topics, Topic01-Topic08). In this way, the transaction flow data in each transaction flow file is evenly distributed to the various containers of the engine flow processing application for processing, solving the problem of uneven busyness of the containers of the engine flow processing application, thereby improving the processing efficiency of the transaction flow data and reducing resource waste.
[0064] In a specific implementation, the transaction flow file splitting application can be used to write the transaction flow data in each of the transaction flow files into multiple accounting engine libraries in the basic platform layer; after the transaction flow data is written into each of the accounting engine libraries, the transaction flow data in each of the accounting engine libraries is read; and the transaction flow data in each of the accounting engine libraries is asynchronously distributed one by one in turn to multiple Topics of the kafka message queue in the basic platform layer.
[0065] Step S30: using the data processing layer to read configuration information from the parameter configuration layer, and performing ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain a transaction flow splitting result.
[0066] It should be noted that this embodiment consumes the transaction flow data in each Topic by utilizing the engine flow processing application of the data processing layer to obtain the consumed transaction flow data; then reads the configuration information from the engine parameter application through the engine flow processing application; finally, performs voucher splitting processing on the consumed transaction flow data based on the configuration information to obtain the transaction flow splitting result, and writes the transaction flow splitting result into the accounting engine library.
[0067] In order to improve data processing efficiency, in a specific implementation, after the above-mentioned step of performing ticket splitting processing on the consumed transaction flow data based on the configuration information and obtaining the transaction flow splitting result, it also includes: sending the transaction flow splitting result back to each corresponding engine flow processing application through multiple asynchronous update Topics of the kafka message queue for consumption; after the engine flow processing application consumes the corresponding asynchronous update Topic, updating the transaction flow splitting result.
[0068] It should be understood that the above-mentioned update of the transaction flow splitting result realizes asynchronous processing. For example, after the engine flow processing application splits the transaction flow data, it sends the result of the transaction flow splitting processing back to the corresponding engine flow processing application (CGMA) through Topic5X (Topic51-Topic58) for processing. The engine flow processing application consumes Topic5X and updates the transaction flow splitting result.
[0069] Furthermore, since the accounting engine system of this embodiment is executed concurrently on multiple nodes and fully utilizes the characteristics of Kafka's asynchronous sending mechanism and high throughput, the data processing capability of the accounting engine system of this embodiment is greatly improved compared with the old accounting engine system.
[0070] In order to understand the operation of the accounting engine system, the system also includes a system monitoring layer; the system monitoring layer is used to monitor the operation of the access layer, the data processing layer, the parameter configuration layer and the basic platform layer to obtain monitoring results; the system monitoring layer is also used to generate daily monitoring reports and / or alarm information based on the monitoring results; the system monitoring layer is also used to push the daily monitoring reports and / or alarm information to system users.
[0071] It should be understood that the system monitoring layer is mainly used to monitor the operation of the above access layer, data processing layer, parameter configuration layer and basic platform layer, including unified log and Beidou log monitoring, greeting alarms, email alarms and daily monitoring reports.
[0072] It should be noted that since the transaction flow data in the transaction flow file is evenly distributed to each engine data processing application, when the subsequent business volume increases, it is only necessary to appropriately increase the number of sub-databases and Kafka topics to easily achieve capacity expansion. The scalability of the accounting engine system of this embodiment is also reflected in the ability to flexibly configure resources according to actual needs, and modify configuration information according to the actual needs of the test data volume, so as to achieve an effective balance between resource conservation and high performance requirements.
[0073] For example, refer to Figure 2 , Figure 2The schematic diagram of the architecture of the accounting engine system in the transaction flow processing method of the present invention is shown in FIG. Since the accounting elements are irrelevant to the customer information and the transaction flow data volume is large, the architecture of the accounting engine system is designed as a stateless concurrent vertically distributed application architecture of sub-libraries and sub-tables, which can be divided into an access layer, a data processing layer, a parameter configuration layer, a basic platform layer, and a data monitoring layer. The access layer mainly includes the transaction flow files (i.e. GL flow files) produced by the distribution center and various credit card business parties; the data processing layer includes the flow file splitting application (i.e. EngineSplitFileBatch) and the engine flow processing application (i.e. CGMA); the parameter configuration layer mainly includes the engine parameter application (i.e. CGMP); the basic platform layer mainly includes the accounting engine database (i.e. ORACLE Oracle database, for example, 8 sub-databases, CGMADB01-CGMADB08), accounting general ledger database (CGMCDB00), redis cache component and kafka message queue component, etc.; system monitoring layer: mainly used to monitor the operation of the above levels, including unified log and Beidou log collection, greeting alarm, email alarm and daily monitoring report.
[0074] The present embodiment discloses a transaction flow processing method, which is applied to an accounting engine system, and the system includes an access layer, a data processing layer, and a parameter configuration layer; the present embodiment obtains a transaction flow file from the access layer based on the data processing layer; the transaction flow data in each transaction flow file is asynchronously distributed to multiple Topics of a kafka message queue one by one through the data processing layer; the data processing layer reads configuration information from the parameter configuration layer, and performs a subpoena splitting process on the transaction flow data in each Topic based on the configuration information to obtain a transaction flow splitting result. Since the present embodiment asynchronously distributes the transaction flow data in each transaction flow file to multiple Topics of a kafka message queue one by one through the data processing layer, and then performs a subpoena splitting process on the transaction flow data in each Topic based on the configuration information, compared with the prior art, the present embodiment solves the problem of uneven busyness of the processing application containers of each transaction flow data when processing the transaction flow file, thereby improving the processing efficiency of the transaction flow data and reducing resource waste.
[0075] refer to Figure 3 , Figure 3 It is a flowchart of the second embodiment of the transaction flow processing method of the present invention.
[0076] Based on the above first embodiment, in this embodiment, step S30 includes steps S301 to S303:
[0077] Step S301: When the data processing layer receives the transaction flow data, the MD5 value corresponding to the transaction flow data is calculated according to the accounting elements.
[0078] Step S302: querying a preset entry rule-MD5 value mapping relationship table based on the MD5 value corresponding to the transaction flow data to obtain a query result.
[0079] Capture S303: If the query result indicates that the MD5 value corresponding to the transaction flow data has a corresponding target entry rule, the target entry rule is read from the entry rules in the parameter configuration layer.
[0080] It should be noted that since the dimensions in the entry rules are composed of unchanging customer information, these dimensions will not change with the transaction changes. Therefore, a unique MD5 value can be calculated for each set of entry rules based on the 14 accounting elements in the accounting entry rules configured for the business, and a preset entry rule-MD5 value mapping relationship table can be established.
[0081] It should be understood that when the engine pipeline processing application of the data processing layer receives the transaction flow data, it calculates the MD5 value corresponding to the transaction flow data based on the accounting elements.
[0082] For example, refer to Figure 4 , Figure 4 The following is a flowchart of entry rule matching in the transaction flow processing method of the present invention. First, when receiving the transaction flow data (GL flow), the corresponding MD5 value is calculated, and then it is determined whether the MD5 value corresponding to the transaction flow data exists in the preset entry rule-MD5 value mapping relationship table. If it exists (Y), the target entry rule is directly read from the entry rule in the parameter configuration layer; if it does not exist (N), the entry rule list is obtained according to TC (total cost), product type, transaction type, and price and tax identification, and then the account type grouping, business type grouping, balance type grouping, clearing channel grouping, accounting status grouping, foreign exchange purchase identification, hold account identification, and installment identification are sequentially filtered, and finally the target entry rule is obtained. The entry rules are generated, and the MD5 value corresponding to the transaction flow data and the target entry rules are written into the preset entry rule-MD5 value mapping relationship table. Compared with the prior art, when the transaction flow data (GL flow) is received, the entry rule list is obtained according to TC (total cost), product type, transaction type, and price and tax identification. Then, the entry rules are filtered in turn according to the account type group, the business type, the balance type, the installment identification, the hold account identification, the foreign exchange purchase identification, the accounting status group and the clearing channel group. Finally, the target entry rules are obtained. This embodiment optimizes the entry rule parameter matching mechanism, thereby reducing time consumption and improving data processing efficiency.
[0083] This embodiment discloses that when the data processing layer receives the transaction flow data, the MD5 value corresponding to the transaction flow data is calculated according to the accounting elements; based on the MD5 value corresponding to the transaction flow data, the preset entry rule-MD5 value mapping relationship table is queried to obtain the query result; if the query result indicates that the MD5 value corresponding to the transaction flow data has a corresponding target entry rule, the target entry rule is read from the entry rules in the parameter configuration layer. Since this embodiment queries the preset entry rule-MD5 value mapping relationship table according to the MD5 value corresponding to the transaction flow data, if the query result indicates that the MD5 value corresponding to the transaction flow data has a corresponding target entry rule, the target entry rule is read from the entry rules in the parameter configuration layer. Compared with the prior art, this embodiment optimizes the entry rule parameter matching mechanism, thereby reducing time consumption and improving data processing efficiency.
[0084] refer to Figure 5 , Figure 5 It is a flowchart diagram of the third embodiment of the transaction flow processing method of the present invention.
[0085] Based on the above embodiments, in this embodiment, after step S30, steps S40 to S702 are further included:
[0086] Step S40: Write the transaction flow data of the failed splitting into the transaction flow splitting failure table, and scan the transaction flow splitting failure table through the abnormal re-entry scheduled task to obtain the splitting failure information.
[0087] Step S50: Write the split failure information into the abnormal reentry Topic of the Kafka message queue.
[0088] Step S60: The engine pipeline processing application of the data processing layer consumes the abnormal re-entry Topic to obtain abnormal transaction flow data after consumption, and performs ticket splitting processing on the abnormal transaction flow data after consumption to obtain a re-splitting result.
[0089] Step S701: If the re-splitting result indicates that the flow splitting is successful, the re-splitting result is written into the database, and general ledger accounting and account-by-account accounting are performed.
[0090] Step S702: If the re-splitting result indicates that the flow splitting fails, an abnormal notification message is generated and sent to a system user.
[0091] It should be noted that the present embodiment designs an automatic re-entry mechanism for GL flow. First, the idempotence of each interface of the engine flow processing application (CGMA) is ensured through technical means such as unique keys and cleaning up old data. Then, for the GL flow that fails to be split (the transaction flow data that fails to be split), it is written into the flow splitting failure table CGM_GL_TRANS_ERROR_T. This table is scanned at intervals through the abnormal re-entry timed task to obtain the splitting failure information (i.e., the GL flow information that fails to be split) and write it into the Kafka abnormal re-entry Topic. The engine flow processing application (CGMA) consumes this abnormal re-entry Topic to obtain the abnormal transaction flow data after consumption, and performs subpoena splitting processing on the abnormal transaction flow data after consumption to obtain a re-splitting result. If the re-splitting result indicates that the flow splitting is successful, the re-splitting result is written into the database, and general ledger accounting and account accounting are performed. If the re-splitting result indicates that the flow splitting fails, an abnormal notification message is generated and sent to the system user.
[0092] This embodiment discloses writing the transaction flow data of failed splitting into a flow splitting failure table, and scanning the flow splitting failure table through an abnormal re-entry timed task to obtain splitting failure information; writing the splitting failure information into the abnormal re-entry Topic of the kafka message queue; consuming the abnormal re-entry Topic through the engine flow processing application of the data processing layer to obtain the abnormal transaction flow data after consumption, and performing subpoena splitting processing on the abnormal transaction flow data after consumption to obtain a re-splitting result; if the re-splitting result indicates that the flow splitting is successful, then the re-splitting result is written into the database, and general ledger accounting and account accounting are performed; if the re-splitting result indicates that the flow splitting fails, then an abnormal notification message is generated, and the abnormal notification message is sent to the system user. Compared with the prior art, this embodiment realizes the automatic re-entry of transaction flow data splitting exceptions, enhances the robustness and fault tolerance of the accounting engine system, realizes the self-repair of the accounting engine system, and reduces the work pressure of operation and maintenance personnel.
[0093] In addition, an embodiment of the present invention further proposes a storage medium, on which a transaction flow processing program is stored. When the transaction flow processing program is executed by a processor, the steps of the transaction flow processing method described above are implemented.
[0094] Reference Figure 6 , Figure 6 It is a structural block diagram of the first embodiment of the transaction flow processing device of the present invention.
[0095] like Figure 6As shown, the transaction flow processing device proposed in the embodiment of the present invention is applied to an accounting engine system. The system includes an access layer, a data processing layer and a parameter configuration layer. The device includes: a file acquisition module 601, a data distribution module 602 and a data processing module 603.
[0096] The file acquisition module 601 is used to acquire the transaction flow file from the access layer based on the data processing layer.
[0097] The data distribution module 602 is used to asynchronously distribute the transaction flow data in each transaction flow file to multiple Topics of the Kafka message queue one by one in turn through the data processing layer.
[0098] The data processing module 603 is used to read the configuration information from the parameter configuration layer using the data processing layer, and perform ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result.
[0099] The data distribution module 602 is also used to use the transaction flow file splitting application to write the transaction flow data in each of the transaction flow files into multiple accounting engine libraries in the basic platform layer; after the transaction flow data is written into each of the accounting engine libraries, read the transaction flow data in each of the accounting engine libraries; and asynchronously distribute the transaction flow data in each of the accounting engine libraries to multiple Topics of the Kafka message queue in the basic platform layer one by one.
[0100] The data processing module 603 is also used to consume the transaction flow data in each of the Topics using the engine flow processing application to obtain the consumed transaction flow data; read the configuration information from the engine parameter application through the engine flow processing application; perform invoice splitting processing on the consumed transaction flow data based on the configuration information to obtain the transaction flow splitting result, and write the transaction flow splitting result into the accounting engine library.
[0101] The data processing module 603 is also used to send the transaction flow splitting results back to the corresponding engine flow processing applications through multiple asynchronous update Topics of the Kafka message queue for consumption; after the engine flow processing application consumes the corresponding asynchronous update Topics, the transaction flow splitting results are updated.
[0102] The embodiment of the present device is applied to an accounting engine system, and the system includes an access layer, a data processing layer, and a parameter configuration layer; the embodiment of the present device obtains a transaction flow file from the access layer based on the data processing layer; the transaction flow data in each of the transaction flow files are asynchronously distributed to multiple Topics of the Kafka message queue one by one in turn through the data processing layer; the data processing layer is used to read the configuration information from the parameter configuration layer, and the transaction flow data in each Topic is subjected to ticket splitting processing based on the configuration information to obtain the transaction flow splitting result. Since the embodiment of the present device asynchronously distributes the transaction flow data in each of the transaction flow files to multiple Topics of the Kafka message queue one by one in turn through the data processing layer, and then performs ticket splitting processing on the transaction flow data in each Topic based on the configuration information, compared with the prior art, the embodiment of the present device solves the problem of uneven busyness of the processing application containers of each transaction flow data when processing the transaction flow file, thereby improving the processing efficiency of the transaction flow data and reducing resource waste.
[0103] Based on the above-mentioned first embodiment of the transaction flow processing device of the present invention, a second embodiment of the transaction flow processing device of the present invention is proposed.
[0104] In this embodiment, the data processing module 603 is also used to calculate the MD5 value corresponding to the transaction flow data according to the accounting elements when the data processing layer receives the transaction flow data; query the preset entry rule-MD5 value mapping relationship table based on the MD5 value corresponding to the transaction flow data to obtain the query result; if the query result indicates that the MD5 value corresponding to the transaction flow data has a corresponding target entry rule, then read the target entry rule from the entry rules in the parameter configuration layer.
[0105] Other embodiments or specific implementations of the transaction flow processing device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.
[0106] The present application provides a transaction flow processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the transaction flow processing method in the above-mentioned embodiment one.
[0107] Reference below Figure 7, which shows a schematic diagram of the structure of a transaction flow processing device suitable for implementing the embodiment of the present application. The transaction flow processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The transaction flow processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0108] like Figure 7 As shown, the transaction flow processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the transaction flow processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the transaction flow processing device to communicate with other devices wirelessly or wired to exchange data. Although the transaction flow processing device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.
[0109] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0110] The transaction flow processing device provided by the present application adopts the transaction flow processing method in the above embodiment, which can solve the technical problems of low efficiency in processing transaction flow data and ineffective use of resources in the prior art. Compared with the prior art, the beneficial effects of the transaction flow processing device provided by the present application are the same as those of the transaction flow processing method provided by the above embodiment, and other technical features in the transaction flow processing device are the same as those disclosed in the method of the previous embodiment, which will not be described in detail here.
[0111] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0113] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0114] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0116] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A transaction flow processing method, characterized in that: The method is applied to an accounting engine system, the system comprising an access layer, a data processing layer and a parameter configuration layer, and the method comprises: Acquiring a transaction flow file from the access layer based on the data processing layer; The transaction flow data in each of the transaction flow files are asynchronously distributed to multiple Topics of the Kafka message queue one by one in turn through the data processing layer; The data processing layer is used to read configuration information from the parameter configuration layer, and based on the configuration information, the transaction flow data in each Topic is subjected to ticket splitting processing to obtain a transaction flow splitting result.
2. The transaction flow processing method according to claim 1, characterized in that: The system further comprises a basic platform layer, and the data processing layer comprises a pipeline file splitting application; Correspondingly, the step of asynchronously distributing the transaction flow data in each of the transaction flow files to multiple Topics of the Kafka message queue one by one through the data processing layer includes: Using the transaction flow file splitting application to write the transaction flow data in each of the transaction flow files into multiple accounting engine libraries of the basic platform layer; After the transaction flow data is written into each of the accounting engine libraries, reading the transaction flow data in each of the accounting engine libraries; The transaction flow data in each accounting engine library is asynchronously distributed one by one in turn to multiple Topics of the Kafka message queue in the basic platform layer.
3. The transaction flow processing method according to claim 2, characterized in that: The data processing layer also includes an engine pipeline processing application, and the parameter configuration layer includes an engine parameter application; Accordingly, the step of using the data processing layer to read the configuration information from the parameter configuration layer, and performing ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result includes: Utilize the engine flow processing application to consume the transaction flow data in each of the topics to obtain the consumed transaction flow data; Reading configuration information from the engine parameter application via the engine pipeline processing application; Based on the configuration information, the post-consumption transaction flow data is subjected to invoice splitting processing to obtain a transaction flow splitting result, and the transaction flow splitting result is written into the accounting engine library.
4. The transaction flow processing method according to claim 3, characterized in that: After the step of performing invoice splitting processing on the post-consumption transaction flow data based on the configuration information to obtain the transaction flow splitting result, the method further includes: The transaction flow splitting results are sent back to the corresponding engine flow processing applications through multiple asynchronous update topics of the Kafka message queue for consumption; After the engine pipeline processing application consumes the corresponding asynchronously updated Topics, the transaction pipeline splitting result is updated.
5. The transaction flow processing method according to claim 1, characterized in that: The configuration information includes entry rules, and the step of using the data processing layer to read the configuration information from the parameter configuration layer includes: When the data processing layer receives the transaction flow data, the MD5 value corresponding to the transaction flow data is calculated according to the accounting elements; Based on the MD5 value corresponding to the transaction flow data, a preset entry rule-MD5 value mapping relationship table is queried to obtain a query result; If the query result indicates that the MD5 value corresponding to the transaction flow data has a corresponding target entry rule, the target entry rule is read from the entry rules in the parameter configuration layer.
6. The transaction flow processing method according to claim 1, characterized in that: After the step of performing ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result, the method further includes: The transaction flow data of the failed splitting is written into the transaction flow splitting failure table, and the transaction flow splitting failure table is scanned through the abnormal re-entry scheduled task to obtain the splitting failure information; Writing the split failure information into the abnormal reentry Topic of the kafka message queue; The abnormal reentry Topic is consumed by the engine pipeline processing application of the data processing layer to obtain abnormal transaction flow data after consumption, and the abnormal transaction flow data after consumption is subjected to ticket splitting processing to obtain a re-splitting result; If the re-splitting result indicates that the transaction is successfully split, the re-splitting result is written into the database, and general ledger accounting and account-by-account accounting are performed; If the re-splitting result indicates that the pipeline splitting fails, an abnormal notification message is generated and sent to the system user.
7. The transaction flow processing method according to claim 2, characterized in that: The system also includes a system monitoring layer; The system monitoring layer is used to monitor the operation of the access layer, the data processing layer, the parameter configuration layer and the basic platform layer to obtain monitoring results; The system monitoring layer is further used to generate daily monitoring reports and / or alarm information according to the monitoring results; The system monitoring layer is also used to push the daily monitoring report and / or alarm information to system users.
8. A transaction flow processing device, characterized in that: The device is applied to an accounting engine system, the system includes an access layer, a data processing layer and a parameter configuration layer, and the device includes: A file acquisition module, used for acquiring a transaction flow file from the access layer based on the data processing layer; A data distribution module, used to asynchronously distribute the transaction flow data in each of the transaction flow files to multiple Topics of the Kafka message queue one by one in turn through the data processing layer; The data processing module is used to use the data processing layer to read the configuration information from the parameter configuration layer, and perform ticket splitting processing on the transaction flow data in each Topic based on the configuration information to obtain the transaction flow splitting result.
9. A transaction flow processing device, characterized in that: The device comprises: a memory, a processor and a transaction flow processing program stored in the memory and executable on the processor, wherein the transaction flow processing program is configured to implement the steps of the transaction flow processing method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores a transaction flow processing program, and when the transaction flow processing program is executed by the processor, the steps of the transaction flow processing method according to any one of claims 1 to 7 are implemented.
