Multi-source heterogeneous enterprise-level fund intelligent auditing system

Abstract

The invention discloses a system for realizing multi-source heterogeneous enterprise-level fund intelligent auditing. The system comprises a management platform and a batch processing system, the management platform is used for configuring and generating a merchant reconciliation file strategy, task scheduling related configuration and task execution log viewing; and the batch processing system is used for downloading and importing a data source side account checking detail file at regular time, reading production merchant account checking file strategy information configured by an enterprise from a management end and an account checking file required by production configuration, and uploading the information to the enterprise. The method has the advantages that repeated work and repeated labor cost of account checking projects are reduced, project cost is saved, and project use efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fund auditing, and in particular to a multi-source heterogeneous enterprise-level fund intelligent auditing system. BACKGROUND

[0002] With the increase of MIS transaction services in various parts of the country, the demand for post-transaction merchant reconciliation also follows, and various reconciliation requirements and reconciliation projects come one after another, and the project development workload and labor cost are greatly improved. However, these reconciliation projects have the same point, that is, they all read the reverse transmission file (i.e. reconciliation file) from the data source side, import it into the local library, and then process it into the reconciliation file required by each enterprise. In short, with the diversification of enterprise fund management needs, various reconciliation projects have emerged in large numbers, resulting in a significant increase in project development workload and labor cost. The existing reconciliation system has the following problems. Repeated development: different projects need to repeatedly implement bank reconciliation file acquisition, import and processing functions; low efficiency: lack of unified platform management, inconvenient task scheduling and log viewing; insufficient flexibility: difficult to meet the individual reconciliation needs of different merchants. SUMMARY

[0003] The present application relates to the technical field of fund auditing, and in particular to a multi-source heterogeneous enterprise-level fund intelligent auditing system.

[0004] In order to achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0005] A multi-source heterogeneous enterprise-level fund intelligent auditing system, comprising a management platform and a batch processing system;

[0006] The management platform is used for configuring the generation of merchant reconciliation file strategy and task scheduling related configuration and task execution log viewing; the management platform comprises a merchant hub, a standard reconciliation mode, a custom reconciliation mode, a reconciliation configuration center, an execution engine, an execution task management, a task scheduling tracking, a task operation insight, and an abnormal transaction automatic early warning;

[0007] The batch processing system is used for downloading and importing the data source side reconciliation detail file at regular intervals, and reading the production merchant reconciliation file strategy information configured by the enterprise from the management end, producing the reconciliation file required by the configuration and uploading it to the enterprise; the batch processing system comprises a regular download data source side reverse transmission file task, a regular import data source side reverse transmission file task, a read strategy configuration production merchant reconciliation file task, an upload merchant reconciliation file to the merchant task, a regular data file cleaning task, and a regular monthly creation of detail summary table task.

[0008] Preferably, the merchant hub is used for entering the merchant number and merchant name information of this reconciliation, and is also used for screening out only the merchant reconciliation data participating in the reconciliation;

[0009] The account configuration center is used to check and confirm whether the configuration strategy for generating the merchant account file is correct according to the merchant account file requirements;

[0010] The execution engine is used to execute related tasks;

[0011] The execution task management is used to manage the execution of related tasks, modify the timing execution time of each task as needed, and start the task, so that the task is executed in batches according to the configured time at the arrival time point;

[0012] The task scheduling tracking can view all task batch running logs;

[0013] The task operation insight is used to count the number of task executions and results according to dates;

[0014] The abnormal transaction automatic early warning is used to integrate the email sending capability by using the automatic configuration function of Spring Boot, initiate email early warning for the abnormal task of generating the account file, notify the operation and maintenance personnel, and achieve early warning to discover problems as soon as possible.

[0015] Preferably, in the execution task management, each task is re-run after the execution of the function once, the execution task management provides a re-run mechanism, supports specified date / period re-run, and when it is found that a task fails to execute within a specified date / period and needs to be re-run, the batch running date is input according to the prompt and the re-run is completed by clicking the execution.

[0016] Preferably, the fields commonly used for the merchant account file are screened out, the standard account mode is opened by default, the account fields are checked according to the requirements of the merchant account file, and if the requirements are not met, the custom account mode is selected;

[0017] The standard account mode specifically includes selecting the data source for generating the merchant account file, selecting the fields used for generating the merchant account file, sorting and setting the account fields according to the up and down button, selecting the name, suffix, format, and generation path of the merchant account file to be generated, selecting whether to generate one account file for each merchant, and selecting the upload processing method of the merchant account file.

[0018] The custom account mode can customize the SQL statement for generating the merchant account file; the custom account mode specifically includes selecting the data source for generating the merchant account file, manually writing the SQL for generating the merchant account file according to the requirements of the merchant custom account file, selecting the name, suffix, format, and generation path of the merchant account file to be generated, selecting whether to generate one account file for each merchant, and selecting the upload processing method of the merchant account file.

[0019] Preferably, the data source of the reconciliation file includes the China Construction Bank channel, the Bank of Communications channel and the Agricultural Bank channel.

[0020] Preferably, the batch processing system realizes a unified data access layer design and supports multi-source heterogeneous data import; by constructing a standardized data access adapter framework, the batch processing system supports plug-in access of various data sources of banks, third-party payment and ERP systems; the batch processing system realizes automatic identification and conversion of data formats, including CSV, XML, JSON, EXCEL and TXT; by performing MD5 check on the downloaded bill file, the batch processing system ensures the integrity and authenticity of the file; and by Sharding-JDBC, the batch processing system realizes horizontal table splitting of related forms according to months.

[0021] Preferably, the task of downloading and importing the data source side feedback file at a fixed time specifically includes the following steps: checking whether the T-day task is executed, if not, checking whether the feedback file exists, if so, downloading the feedback file, screening the data of the merchant to be reconciled according to the configured merchant number, and importing the data into the database, and recording the success of the T-day batch running.

[0022] The task of generating a merchant reconciliation file and uploading specifically includes the following steps: checking whether the T-day production merchant feedback file task is executed, if not, reading the production merchant reconciliation file strategy configuration information configured by the user in the management end, producing the merchant reconciliation file corresponding to the configuration information and uploading the file to the merchant, recording the success of the T-day batch running, supporting FTPS secure transmission, and realizing the function of breakpoint resume;

[0023] The task of cleaning data files at a fixed time specifically includes the following steps: checking whether the T-day cleaning data task is executed, if not, cleaning the data and files generated by the batch running N days ago, and recording the success of the T-day batch running.

[0024] The task of creating a detailed summary table at a fixed time specifically includes the following steps: generating a detailed summary table and a bill summary table of the next month at a fixed time at the end of each month.

[0025] Preferably, the audit system realizes merchant reconciliation, including the following steps:

[0026] S1, adding a reconciliation merchant to the merchant hub of the management platform;

[0027] S2, setting a reconciliation batch running strategy in the reconciliation configuration center of the management platform;

[0028] S3, selecting a standard reconciliation mode or a custom reconciliation mode according to actual conditions;

[0029] S4, configuring a fixed-time task in the execution task management of the management platform;

[0030] S5, record the relevant policy database, when the timing task reaches the time node, check whether the log has been run batch, if yes, the timing run batch process ends, otherwise, generate the reconciliation file according to the reconciliation policy, upload the reconciliation file to the merchant through SFTP, record the run batch log, and the timing run batch process ends.

[0031] The beneficial effects of the present application are: 1. Provide double reconciliation mode, standard reconciliation mode: pre-set commonly used reconciliation field, fast configuration; Custom reconciliation mode: support SQL level flexible customization, meet complex needs. 2. Intelligent task scheduling: based on XXL-JOB to realize distributed task scheduling; Support task dependency management (such as import first and then generate); Provide a mechanism for running, support specified date / time period. 3. Efficient file processing: support FTPS secure transmission, realize breakpoint resume function, automatically clean up expired data. 4. Multi-source heterogeneous data integration, provide unified data access layer design, through building a standardized data access adapter framework, support plug-in access of various banks, third-party payment, ERP system and other data sources; Realize data format automatic identification and conversion, including CSV, XML, JSON, EXCEL, TXT and other common financial formats. 5. Use the automatic configuration function of Spring Boot to integrate the mail sending ability. 6. Integrated Sharding-JDBC horizontal table technology. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is the working principle diagram of the intelligent auditing system in the embodiment of the present application;

[0033] Figure 2 is the flow chart of the merchant reconciliation operation in the embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present application more clear and explicit, the present application will be further described in detail below with reference to the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0035] As shown in Figure 1 In the embodiment, in order to reduce the repeated work of reconciliation items and the repeated labor cost, in the principle of saving project cost and improving the use efficiency of the project, the common points of the put-into-production reconciliation items are summarized, the common reconciliation function is found out, and a multi-source heterogeneous enterprise-level intelligent auditing system of funds is provided, including a management platform and a batch processing system.

[0036] 1. The management platform is used for configuring the generation of merchant account file strategy and task scheduling related configuration and task execution log viewing; the functions of the management platform include: merchant hub, standard reconciliation mode, custom reconciliation mode, reconciliation configuration center, execution engine, execution task management, task scheduling tracking, task operation insight, and abnormal transaction automatic early warning.

[0037] (1) Merchant hub: enter the merchant number participating in this reconciliation, merchant name information, and use it to filter out only the merchant reconciliation data participating in reconciliation.

[0038] (2) Standard reconciliation mode: (standard reconciliation mode and custom reconciliation mode can only be one of the two) The standard reconciliation mode is to filter out the commonly used fields of the merchant reconciliation file. By default, the standard reconciliation mode is turned on. According to the merchant billing file requirements, check if the reconciliation field meets the requirements of the merchant reconciliation file. If it does not meet the requirements, select the custom reconciliation mode. Select the data source for generating the merchant reconciliation file (ICBC channel, CCB channel, ABC channel), select the fields used to generate the merchant reconciliation file, and set the order of the reconciliation fields according to the up and down button (used to set the order of each record in the generated billing file). Then select the merchant reconciliation file name, suffix, format, generation path, whether to generate one reconciliation file for each merchant, and select the merchant reconciliation file upload processing method and other information selection.

[0039] The standard reconciliation mode is to display only the

common reconciliation fields

common reconciliation fields

[0040] (3) Custom reconciliation mode: (standard reconciliation mode and custom reconciliation mode can only be one of the two). When the reconciliation field in the standard reconciliation mode configuration cannot meet the requirements of the merchant reconciliation file (such as custom billing or additional fields or more fields are required for the reconciliation file, or more complex billing files, etc.), select the custom reconciliation mode. The difference between custom mode and standard mode is that there are no too many constraints to generate the SQL statement of the merchant reconciliation file, and the other options are the same as the custom reconciliation mode. Select the data source for generating the merchant reconciliation file (ICBC channel, CCB channel, ABC channel), and the developer manually writes the SQL statement for generating the merchant reconciliation file according to the requirements of the custom billing file. Select the merchant reconciliation file name, suffix, format, generation path, whether to generate one reconciliation file for each merchant, and select the merchant reconciliation file upload processing method and other information selection.

[0041] When the account field in the standard reconciliation mode configuration cannot meet the requirements of the merchant reconciliation file (such as customer customized bills, or additional fields or more fields required for the reconciliation file), the standard reconciliation mode field does not meet the requirements. For special and complex customized reports, developers need to manually write and generate customized report SQL in the configuration window (how to write the configuration window is illustrated by examples). This system is used to reduce the repetitive workload of developers, and simple and commonly used bill files can be directly configured manually. For complex customized bill SQL statements that need to be written and configured by developers, the repetitive development work can be avoided.

[0042] Different customers have different reconciliation requirements. Some customers require the generated bill amount to be yuan, some require the amount to be fen, some have common reconciliation fields, and some do not have common reconciliation fields or have more other non-common fields, and so on. Because the requirements of each merchant for reconciliation are not unique, standard mode and custom mode are provided according to the needs of the customers to facilitate the customers to select the mode for configuration according to their needs.

[0043] (4) Reconciliation configuration center: According to the requirements of the merchant bill file, check and confirm whether the configuration strategy information for generating the merchant reconciliation file is correct. For example, if merchant A uses the standard reconciliation mode, the source of the merchant bill data is the ICBC channel, the reconciliation fields include merchant number, terminal number, transaction date, transaction time, transaction amount (unit: fen), bank card number, transaction type, two-dimensional code order number, each field is separated by a comma, the bill file name is date+merchant number.csv, and the SFTP form is uploaded to the upload path / upload path and other requirements. Check whether the configuration is consistent with the requirements according to the requirements.

[0044] (5) Execution engine: By default, the runner in the batch project has been added to the configuration, and no configuration is required here.

[0045] (6) Execute task management: the default execution task has been added, including: import data source side daily transaction detail account bill file task, import data source side daily transaction summary account bill file task, generate enterprise reconciliation file according to configuration, clean up data file task, create monthly detail summary table task, etc. Here, you need to modify the timing of each task according to your needs, and start the task. In this way, the task will be executed according to the configured time, and the time point will be reached. Need to pay attention to the three import tasks, which need to be consistent with the selected standard reconciliation data source configuration. If two data sources are selected, only two corresponding tasks need to be started, and the unnecessary ones do not need to be started. Each task also has an execute once function, that is, it can be re-run. Provide a re-run mechanism to support specified date / time period re-run. For example, when a T-6 day execution task fails, it needs to be re-run, at which point the user is prompted to enter the batch date and click execute to complete the re-run effect. This re-run can be for a certain day or a certain time period.

[0046] (7) Task scheduling tracking: you can view all task batch logs, that is, the management end online log. Solve the problem of only being able to view log files to view logs.

[0047] (8) Task operation insight: according to the date, the number of task execution times and other conditions are counted.

[0048] (9) Abnormal transaction automatic early warning: the system starts the abnormal check function, integrates the mail sending ability by using the automatic configuration function of Spring Boot, sends an email warning for the abnormal task of generating account files, notifies the system operation and maintenance personnel, and achieves early warning to find problems as soon as possible, rather than being passive and waiting for customers to find problems.

[0049] 2. The batch processing system is used for downloading and importing data source side reconciliation detail files, and reading enterprise production merchant reconciliation file strategy information configured from the management end, producing configured reconciliation files, and uploading to enterprises; The functions of the batch processing system include: downloading data source side reverse transmission file task, importing data source side reverse transmission file task, reading strategy configuration production merchant reconciliation file task, uploading merchant reconciliation file to merchant task, cleaning up data file task, and creating monthly detail summary table task.

[0050] (1) Reverse transmission file download and import timing task: check whether the T-day task is executed, if not, check whether the reverse transmission file exists (including daily detail account bill file and daily summary account bill file), if it exists, download the reverse transmission file, and according to the configured merchant number, filter out the reconciliation merchant data and import it into the database, and record the T-day batch execution success.

[0051] (2) Generate merchant reconciliation file and upload task: check if the T-day production merchant reverse transmission file task is executed, if not, read the production merchant reconciliation file strategy configuration information configured by the user in the management end, the production configuration information corresponding to the merchant reconciliation file is uploaded to the merchant, and the task T-day run batch success is recorded. This function supports FTPS secure transmission and realizes the breakpoint resume function.

[0052] (3) Timely clean up data file task: check if the T-day clean up data task is executed, if not, clean up the data and files generated by the run batch N days ago (N can be configured), record the task T-day run batch success.

[0053] (4) Timely create monthly detailed summary table task: generate the next month's bill detail table and bill summary table according to the configured time at the end of each month.

[0054] The batch processing system realizes unified data access layer design, supports multi-source heterogeneous data import, builds a standardized data access adapter framework, supports plug-in access of various data sources such as banks, third-party payment, ERP system, etc., realizes automatic identification and conversion of data formats, including CSV, XML, JSON, EXCEL, TXT, etc. Financial common format technology. The downloaded bill file is checked by MD5 to ensure the integrity and authenticity of the file. Considering the large transaction volume, the read-write table efficiency is optimized, and the bill detail table and the bill summary table are horizontally divided into tables by Sharding-JDBC according to the month.

[0055] As shown in Figure 2 , in the embodiment, the audit system realizes the merchant reconciliation, including the following steps,

[0056] (1) Add reconciliation merchant to the merchant hub of the management platform.

[0057] (2) Configure the generation of merchant reconciliation file configuration strategy in the reconciliation configuration center of the management platform and submit, assuming that the data source is selected as the China Construction Bank channel.

[0058] (3) Select standard reconciliation mode or custom reconciliation mode according to actual situation.

[0059] (4) Configure in the task management on the management platform: import daily transaction detail bill file into database, import daily transaction summary bill file into database, generate reconciliation file, and configure the run batch time of the data cleaning task in the order of import first and then generate, and start the four tasks.

[0060] (5) On the management platform, the time point tasks are respectively run in batches, that is, the daily transaction detail account bill file is imported into the database and run in batches, the daily transaction summary account bill file is imported into the database and run in batches, and the reconciliation file is generated and run in batches. The task scheduling tracking checks whether the three tasks are successfully run in batches. If all the tasks are successfully run in batches, it is checked whether the reconciliation file is generated according to the configuration strategy and uploaded to the merchant through SFTP.

[0061] By adopting the above technical solutions disclosed in the application, the following beneficial effects are obtained:

[0062] The application provides a multi-source heterogeneous enterprise-level intelligent fund auditing system, a double-reconciliation mode, a standard reconciliation mode, preset common reconciliation fields, rapid configuration, a self-defined reconciliation mode, support for SQL-level flexible customization, satisfaction of complex requirements, intelligent task scheduling, distributed task scheduling based on XXL-JOB, support for task dependency management (such as first import and then generation), a re-run mechanism, support for specified date / time period re-run, efficient file processing, support for FTPS secure transmission, realization of a breakpoint resume function, automatic cleaning of expired data, multi-source heterogeneous data integration, unified data access layer design, construction of a standardized data access adapter framework, support for plug-in type access of various banks, third-party payment, ERP systems and other data sources, realization of automatic data format identification and conversion, including CSV, XML, JSON, EXCEL, TXT and other common financial formats, integration of a mail sending capability by using the automatic configuration function of Spring Boot, and integration of Sharding-JDBC horizontal table splitting technology.

[0063] The above description is only the preferred embodiments of the application, and it should be pointed out that, for ordinary skilled persons in the art, without departing from the principles of the application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the application.

Claims

1. A multi-source heterogeneous enterprise-level funds intelligent auditing system is implemented, characterized by: The management platform and the batch processing system are comprised; The management platform is used for configuring generation of merchant account statement file strategy and task scheduling related configuration and task execution log viewing; the management platform comprises a merchant hub, a standard account reconciliation mode, a custom account reconciliation mode, an account reconciliation configuration center, an execution engine, an execution task management, a task scheduling tracking, a task operation insight, and an abnormal transaction automatic early warning; The batch processing system is used for downloading and importing data source side account detail file in a timing manner, reading production merchant account statement file strategy information configured by an enterprise from a management end, producing account statement file meeting configuration requirements and uploading to the enterprise; the batch processing system comprises a timing download data source side reverse transmission file task, a timing import data source side reverse transmission file task, a read strategy configuration production merchant account statement file task, an upload merchant account statement file to a merchant task, a timing data file cleaning task, and a timing monthly creation of detail summary table task.

2. The system for implementing multi-source heterogeneous enterprise-level funds intelligent auditing according to claim 1, characterized in that: The merchant hub is used for inputting merchant number and merchant name information of this time account reconciliation, and screening out only account reconciliation data of the merchant; The account reconciliation configuration center is used for checking and confirming whether the generation of merchant account statement file configuration strategy is correct according to merchant bill file requirements; The execution engine is used for executing related tasks; The execution task management is used for managing the execution of related tasks, modifying the timing execution time of each task as required, and starting the task, so that the task is executed in a timing manner after reaching the time point according to the configuration time; The task scheduling tracking can view all task batch running logs; The task operation insight is used for counting the number of task executions and results according to dates; The abnormal transaction automatic early warning is used for integrating the email sending capability by using the automatic configuration function of Spring Boot, initiating email early warning for the generation of bill file abnormal tasks, notifying operation and maintenance personnel, achieving early warning, and discovering problems as early as possible.

3. The system for multi-source heterogeneous enterprise-level funds intelligent auditing according to claim 1, characterized in that: In the execution task management, each task can be re-run after the execution of the function once, the execution task management provides a re-run mechanism, supports the specified date / period re-run, and when it is found that a certain task fails to execute in a specified date / period, the re-run needs to be re-run, the re-run date is input according to the prompt, and the re-run can be completed by clicking execution.

4. The system for implementing multi-source heterogeneous enterprise-level funds intelligent auditing according to claim 1, characterized in that: The fields commonly used for merchant account statement file are screened out, the standard account reconciliation mode is opened by default, the account reconciliation fields are checked according to the requirements of the merchant bill file, and if the requirements are not met, the custom account reconciliation mode is selected; The standard account reconciliation mode specifically comprises selecting a data source for generating the merchant account statement file, selecting fields used for generating the merchant account statement file, sorting and setting the account reconciliation fields according to the up and down button, selecting the name, suffix, format, and generation path of the merchant account statement file, selecting whether to generate one account statement file for each merchant, and selecting the upload processing mode of the merchant account statement file. The custom reconciliation mode can customize the SQL statement for generating the merchant reconciliation file; the custom reconciliation mode specifically includes selecting the data source for generating the merchant reconciliation file, manually writing the SQL for generating the merchant reconciliation file according to the merchant's customized billing file requirements, selecting the name, suffix, format and generation path of the merchant reconciliation file, selecting whether to generate one reconciliation file for each merchant, and selecting the upload processing mode of the merchant reconciliation file.

5. The system for implementing multi-source heterogeneous enterprise-level funds intelligent auditing of claim 1, wherein: The data source of the reconciliation file includes the Bank of Communications channel, the Bank of China channel and the Agricultural Bank channel.

6. The system for implementing multi-source heterogeneous enterprise-level funds intelligent auditing of claim 1, wherein: The batch processing system realizes a unified data access layer design and supports multi-source heterogeneous data import; by constructing a standardized data access adapter framework, the system supports plug-in access of various data sources such as banks, third-party payment and ERP systems; the system realizes automatic identification and conversion of data formats, including CSV, XML, JSON, EXCEL and TXT; the system performs MD5 check on the downloaded billing file to ensure the integrity and authenticity of the file; and the system realizes horizontal table splitting of related forms by month through Sharding-JDBC.

7. The system for implementing multi-source heterogeneous enterprise-level funds intelligent auditing according to claim 6, characterized in that: The timed download and import of the data source side feedback file task specifically includes checking whether the T-day task is executed, if not, checking whether the feedback file exists, if so, downloading the feedback file, screening the merchant data required for reconciliation according to the configured merchant number and importing the data into the database, and recording the success of the T-day batch running; The generation of the merchant reconciliation file and the uploading task specifically includes checking whether the T-day production of the merchant feedback file task is executed, if not, reading the production of the merchant reconciliation file corresponding to the configuration information configured by the user in the management end, uploading the configuration information to the merchant, recording the success of the T-day batch running; supporting FTPS secure transmission and realizing the breakpoint resume function; The timed data file cleaning task specifically includes checking whether the T-day cleaning data task is executed, if not, cleaning the data and files generated by the batch running N days ago, and recording the success of the T-day batch running; The timed monthly creation of the detail summary table task specifically includes generating the next month's billing detail table and billing summary table according to the configured time at the end of each month.

8. The system according to any one of claims 1 to 7, wherein: The auditing system realizes the reconciliation of the merchant, including the following steps, S1, adding a reconciliation merchant in the merchant hub of the management platform; S2, setting a reconciliation running strategy in the reconciliation configuration center of the management platform; S3, selecting a standard reconciliation mode or a custom reconciliation mode according to the actual situation; S4, configuring a timed task in the execution task management of the management platform; S5, recording the related strategy in the database, checking whether the log has been run when the timed task reaches the time node, if so, ending the timed running process, otherwise, generating a reconciliation file according to the reconciliation strategy, uploading the reconciliation file to the merchant through SFTP, recording the running log, and ending the timed running process.

Images