Method and device for performing consistency verification on financial data and readable storage medium

Abstract

The invention discloses a method and a device for performing consistency verification on financial data and a readable storage medium. The method comprises the following steps: acquiring a first data verification task for indicating to perform consistency verification on first financial data in an original data source and second financial data in a backup data source; determining business types of the first financial data and the second financial data; according to the business types, a plurality of first business tasks corresponding to the first financial data and a plurality of second business tasks corresponding to the second financial data are inquired; and determining a plurality of comparison groups according to task attributes of the business tasks, and determining a consistency verification result of the first financial data and the second financial data based on execution results of the tasks in the comparison groups. According to the embodiment of the invention, different business tasks are inquired according to the business categories, then the comparison group is determined according to the task attributes of the business tasks, and then the comparison result is determined based on the execution result of the tasks in the comparison group, so that the verification speed can be increased, and the verification efficiency can be improved.

Description

Technical Field

[0001] This article relates to data processing technology, and in particular to a method, apparatus and readable storage medium for performing consistency verification of financial data. Background Technology

[0002] Currently, most software systems, especially financial systems, store the same data on different hosts, in different data centers, and even in different cities to ensure data security and reliability—that is, to prevent data loss and ensure continuous and stable service in the event of power outages, hardware failures, or other unforeseen circumstances. This creates source and backup databases. Furthermore, to ensure data consistency between the source and backup databases, data consistency checks are performed on both.

[0003] In related technologies, the MySQL checksum function is typically used to verify data consistency.

[0004] However, the Checksum function uses the Cyclic Redundancy Check (CRC) algorithm to retrieve data from the source and backup databases line by line and then calculate the checksum. When the amount of data in the database is large, the data verification efficiency is low. Summary of the Invention

[0005] This application provides a method, apparatus, and readable storage medium for consistency verification of financial data, which can avoid the problem of huge verification data volume faced when using the Checksum function to verify the database, greatly speeding up the verification of financial data and improving the efficiency of financial data verification.

[0006] On the one hand, this application provides a method for consistency verification of financial data, the method comprising: Obtain a first data verification task, which instructs to perform a consistency verification between the first financial data in the original data source and the second financial data in the backup data source. Determine the business categories of the first financial data and the second financial data; Based on the business category, query several first business tasks corresponding to the first financial data, and query several second business tasks corresponding to the second financial data; Based on the task attributes of the business tasks, several comparison groups are determined. Based on the execution results of the tasks in the comparison groups, the consistency verification results of the first financial data and the second financial data are determined.

[0007] On the other hand, this application provides an apparatus for performing consistency verification on financial data, comprising: a memory and a processor, wherein the memory is used to store an executable program; the memory is used to store the executable program; The processor is used to read and execute the executable program to implement the above-described method for verifying the consistency of financial data.

[0008] In another aspect, this application provides a readable storage medium storing computer-executable commands for performing the above-described method for verifying the consistency of financial data.

[0009] Compared with related technologies, this application queries different business tasks based on the business categories of financial data to compare the data consistency between the source database and the backup database. Then, it determines multiple comparison groups based on the task attributes of the business tasks. Finally, it determines the final comparison result based on the execution results of the tasks in the comparison groups. This avoids the problem of huge amount of verification data faced when using the Checksum function to verify the database, greatly speeds up the verification of financial data, and improves the efficiency of financial data verification.

[0010] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0011] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0012] Figure 1 This is a flowchart illustrating a method for performing consistency verification on financial data according to an embodiment of this application. Figure 2 This is a flowchart illustrating another method for performing consistency verification on financial data according to an embodiment of this application. Figure 3 This is a flowchart illustrating another method for performing consistency verification on financial data according to an embodiment of this application. Figure 4 This is a flowchart illustrating another method for verifying the consistency of financial data according to an embodiment of this application. Detailed Implementation

[0013] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0014] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0015] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0016] This application provides a method for consistency verification of financial data, such as... Figure 1 As shown, the method includes: Step 101: Obtain the first data verification task, which instructs to perform consistency verification on the first financial data in the original data source and the second financial data in the backup data source. Step 102: Determine the business categories of the first financial data and the second financial data; Step 103: Based on the business category, query out several first business tasks corresponding to the first financial data, and query out several second business tasks corresponding to the second financial data; Step 104: Determine several comparison groups based on the task attributes of the business tasks, and determine the consistency verification result of the first financial data and the second financial data based on the execution results of the tasks in the comparison groups.

[0017] In related technologies, Checksums can only perform validation at the database or table level, resulting in a large amount of data to be validated. Furthermore, Checksum operations require the source and backup databases to have the same operating system version, database version, and table structure. If the source database is sharded and the backup database is not, data consistency comparison cannot be performed. Additionally, Checksums require the source and backup databases to use the same storage medium, i.e., both must be MySQL databases. If the backup database uses other relational or non-relational databases such as TiDB, MongoDB, or ClickHouse, data consistency comparison cannot be performed.

[0018] The method for consistency verification of financial data provided in this application involves querying different business tasks based on the business category of the financial data to compare the data consistency between the source database and the backup database. Then, multiple comparison groups are determined based on the task attributes of the business tasks. Finally, the final comparison result is determined based on the execution results of the tasks in the comparison groups. This avoids the problem of huge amount of verification data when using the Checksum function to verify the database, greatly speeds up the verification of financial data, and improves the efficiency of financial data verification.

[0019] The method for consistency verification of financial data provided in this application breaks down the financial data consistency comparison at the business level, avoiding the problem of excessive data verification pressure caused by performing a full data consistency comparison at once. This makes the granularity of each comparison task smaller, significantly reducing the data verification pressure. Furthermore, the method for consistency verification of financial data provided in this application focuses on data consistency from the business side, is independent of storage media, and can perform consistency comparisons in scenarios where the primary and backup database table structures and database types are different.

[0020] In one exemplary instance, the task attributes include business logic attributes and business batch attributes.

[0021] For example, the business logic attributes include: accounting reconciliation and / or account balance verification. Business batch attributes can record information generated by the business process to ensure traceability.

[0022] In one exemplary instance, the business category includes: pending reconciliation category.

[0023] For example, the categories of financial data to be reconciled may include: accounts receivable reconciliation, accounts payable reconciliation, bank reconciliation, inventory reconciliation, billing reconciliation, financial statement reconciliation, and tax reconciliation.

[0024] For example, suppose the categories of financial data to be reconciled include accounts receivable reconciliation, and the business logic attributes include accounting reconciliation and account balance verification. The accounting reconciliation business includes batches 001 and 002, and the account balance verification business includes batches 012 and 015. Then, based on the accounts receivable reconciliation category, several first business tasks corresponding to the first financial data are retrieved from the original data source. Based on the accounts receivable reconciliation category, several second business tasks corresponding to the second financial data are retrieved from the backup data source. Then, from the first and second business tasks of the accounts receivable reconciliation categories, several comparison groups of batches 001 of accounting reconciliation are determined. From the first and second business tasks of the accounts receivable reconciliation categories, several comparison groups of batches 001 of accounting reconciliation are determined. A comparison group for batch 002 of accounting reconciliation transactions is established. From the first and second business tasks of several accounts receivable reconciliation categories, a comparison group for batch 012 of account balance verification transactions is established. From the first and second business tasks of several accounts receivable reconciliation categories, a comparison group for batch 015 of account balance verification transactions is established. Based on the execution results of the tasks in the comparison groups, the consistency verification results of the first financial data and the second financial data are determined.

[0025] In one exemplary instance, determining the consistency verification result between the first financial data and the second financial data based on the execution results of the tasks in the comparison group includes: If the execution results of the tasks in each comparison group are the same, then the first financial data and the second financial data are determined to be consistent. If there are target comparison groups with different task execution results, then a consistency check is performed on each data item contained in the first financial data and the second financial data to obtain the consistency check result.

[0026] In one exemplary instance, the method further includes: monitoring whether new financial data is written to the original data source and monitoring whether new financial data is written to the backup data source; wherein, the first data verification task is issued whenever new financial data is detected being written to the original data source and / or new financial data is detected being written to the backup data source. The first financial data refers to the newly added first financial data, and the second financial data refers to the newly added second financial data.

[0027] For example, monitoring whether new financial data is written to the original data source and monitoring whether new financial data is written to the backup data source; whenever new financial data is detected being written to the original data source and / or the backup data source, such as... Figure 2 As shown, perform the following operations: Step 201: Obtain instructions to perform a consistency check on the first financial data added to the original data source and the second financial data added to the backup data source. (First data check task) Step 202: Determine the business categories of the newly added first financial data and the newly added second financial data; Step 203: Based on the business category, query out several first business tasks corresponding to the newly added first financial data, and query out several second business tasks corresponding to the newly added second financial data; Step 204: Determine several comparison groups based on the task attributes of the business tasks, and determine the consistency verification results of the newly added first financial data and the newly added second financial data based on the execution results of the tasks in the comparison groups.

[0028] Whenever new financial data is detected being written to the original data source and / or the backup data source, a comparison with the newly added financial data can ensure the finest granularity of the comparison. However, from a resource consumption perspective, financial data verification consumes a significant amount of resources.

[0029] The method for verifying the consistency of financial data provided in this application breaks down the task of financial data consistency comparison according to the time dimension, resulting in a smaller granularity of each comparison task. This avoids the situation where a full data consistency comparison times out, and only compares incremental data, thus avoiding repeated comparisons of historical data. Furthermore, the method for verifying the consistency of financial data provided in this application focuses on data consistency from the business side, is independent of the storage medium, and can perform consistency comparisons in scenarios where the primary and backup database table structures and database types are different.

[0030] In one exemplary instance, the method further includes: monitoring the writing status of financial data in the original data source and monitoring the writing status of financial data in the backup data source; wherein the first data verification task is periodically issued. The first financial data refers to the first financial data added in each period of the original data source, and the second financial data refers to the second financial data added in each period of the backup data source.

[0031] For example, when a cycle is reached, such as Figure 3 As shown, perform the following operations: Step 301: Obtain instructions to perform a consistency check on the first financial data added in the current period in the original data source and the second financial data added in the current period in the backup data source. Step 302: Determine the business categories of the first and second newly added financial data in the current period; Step 303: Based on the business category, query several first business tasks corresponding to the first financial data newly added in the current period, and query several second business tasks corresponding to the second financial data newly added in the current period; Step 304: Determine several comparison groups based on the task attributes of the business tasks. Based on the execution results of the tasks in the comparison groups, determine the consistency verification results of the first financial data added in the current period and the second financial data added in the current period.

[0032] In practical applications, the duration of the first data verification task can be set as needed. If it is set to a shorter duration, the consistency verification of financial data will be more frequent.

[0033] In one exemplary instance, the first data verification task is sent out every hour on the hour; The first financial data refers to the financial data within the verification time range corresponding to each hour in the original data source, and the second financial data refers to the financial data within the verification time range corresponding to each hour in the backup data source. The verification time range corresponding to each hour is determined based on the business activity level of each hour. The verification time range corresponding to the hour with higher business activity is smaller than the verification time range corresponding to the hour with lower business activity.

[0034] For example, if a lot of financial data verification is performed during peak business hours, it may affect the normal operation of the business. Therefore, the verification time range is relatively small. On the other hand, during peak business hours, there is no need to worry about affecting the normal operation of the business, so more financial data verification can be performed accordingly, and the verification time range is relatively large.

[0035] For example, when a full hour is reached, such as Figure 4 As shown, perform the following operations: Step 401: Obtain instructions to perform a consistency check on the first financial data added at the current hour in the original data source and the second financial data added at the current hour in the backup data source. Step 402: Determine the business categories of the first and second financial data added at the current hour. Step 403: Based on the business category, query several first business tasks corresponding to the first financial data added at the current hour, and query several second business tasks corresponding to the second financial data added at the current hour. Step 404: Determine several comparison groups based on the task attributes of the business tasks. Based on the execution results of the tasks in the comparison groups, determine the consistency verification results of the first financial data added at the current hour and the second financial data added at the current hour.

[0036] In one exemplary instance, the time range for data verification at each hour is determined by the following calculation expression: ((each hourly interval + 16) mod 24 + 1) * sampling exponent / 100 * 100%

[0037] The sampling index is a pre-set integer greater than 0 and not greater than 8.

[0038] The hour is displayed in a 24-hour time system.

[0039] Generally, 8 a.m. is the peak business time, and this calculation formula can ensure that the time range for data verification at 8 a.m. is minimized.

[0040] For example, if the sampling index is 2, then the time range for data verification at 8 am is ((8+16)mod 24 + 1)*2 / 100*100%=2%, that is, data verification is performed for a 2% time range starting at 8 am.

[0041] This application embodiment also provides an apparatus for performing consistency verification on financial data, including: a memory and a processor, wherein the memory is used to store an executable program; The processor is used to read and execute the executable program to implement the method for consistency verification of financial data provided in the embodiments of this application as described in any of the above embodiments.

[0042] The apparatus for verifying the consistency of financial data provided in this application query different business tasks based on the business category of the financial data when comparing the data consistency between the source database and the backup database. Then, it determines multiple comparison groups based on the task attributes of the business tasks. Finally, it determines the final comparison result based on the execution results of the tasks in the comparison groups. This avoids the problem of huge amount of verification data when using the Checksum function to verify the database, greatly speeds up the verification of financial data, and improves the efficiency of financial data verification.

[0043] It should be understood that a processor can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor, etc.

[0044] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0045] In implementation, the processing performed by the terminal device can be accomplished through integrated logic circuits in the processor's hardware or through software instructions. That is, the steps of the method disclosed in this application can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method. To avoid repetition, further details are omitted here.

[0046] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0047] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0048] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

Claims

1. A method for consistency verification of financial data, characterized in that, The method includes: Obtain a first data verification task, which instructs to perform a consistency verification between the first financial data in the original data source and the second financial data in the backup data source. Determine the business categories of the first financial data and the second financial data; Based on the business category, query several first business tasks corresponding to the first financial data, and query several second business tasks corresponding to the second financial data; Based on the task attributes of the business tasks, several comparison groups are determined. Based on the execution results of the tasks in the comparison groups, the consistency verification results of the first financial data and the second financial data are determined.

2. The method according to claim 1, characterized in that, The task attributes include business logic attributes and business batch attributes.

3. The method according to claim 2, characterized in that, The business categories include: pending reconciliation categories; The business logic attributes include: accounting reconciliation and / or account balance verification.

4. The method according to claim 1, characterized in that, The determination of the consistency verification result between the first financial data and the second financial data based on the execution results of the tasks in the comparison group includes: If the execution results of the tasks in each comparison group are the same, then the first financial data and the second financial data are determined to be consistent. If there are target comparison groups with different task execution results, then a consistency check is performed on each data item contained in the first financial data and the second financial data to obtain the consistency check result.

5. The method according to claim 1 or 4, characterized in that, The method further includes: monitoring whether new financial data is written to the original data source and monitoring whether new financial data is written to the backup data source; wherein, the first data verification task is issued whenever new financial data is detected being written to the original data source and / or new financial data is detected being written to the backup data source. The first financial data refers to the newly added first financial data, and the second financial data refers to the newly added second financial data.

6. The method according to claim 1 or 4, characterized in that, The method further includes: monitoring the writing status of financial data in the original data source and monitoring the writing status of financial data in the backup data source; wherein, the first data verification task is periodically issued. The first financial data refers to the first financial data added in each period of the original data source, and the second financial data refers to the second financial data added in each period of the backup data source.

7. The method according to claim 1 or 4, characterized in that, The first data verification task is sent out every hour on the hour. The first financial data refers to the financial data within the verification time range corresponding to each hour in the original data source, and the second financial data refers to the financial data within the verification time range corresponding to each hour in the backup data source. The verification time range corresponding to each hour is determined based on the business activity level of each hour. The verification time range corresponding to the hour with higher business activity is smaller than the verification time range corresponding to the hour with lower business activity.

8. The method according to claim 7, characterized in that, The verification time range corresponding to each hour is determined by the following calculation expression: ((each hour + 16) mod 24 + 1) * sampling exponent / 100 * 100%; The sampling index is a pre-set integer greater than 0 and not greater than 8.

9. An apparatus for performing consistency verification on financial data, comprising: A memory and a processor, wherein the memory is used to store an executable program; The processor is used to read and execute the executable program to implement the method for consistency verification of financial data as described in any one of claims 1-8.

10. A readable storage medium, characterized in that, The readable storage medium stores computer-executable commands for performing the method for verifying the consistency of financial data as described in any one of claims 1-8.

Images