Report data verification methods, devices, equipment, systems, media and program products

Abstract

The present application discloses a report data verification method, apparatus, device, system, medium, and program product. The method comprises: receiving an uploaded report file set, the report file set including a report file; scanning the report file, obtaining attribute information of the report file, and writing the attribute information into a database; obtaining a verification rule table template, importing the verification rule table template into the database, the verification rule table template including an expression of a verification rule; filtering the report file to obtain a to-be-verified report file based on the report file set and the attribute information written into the database; calling a fast general calculation engine, and using the expression of the verification rule in the database to verify the to-be-verified report file to obtain a verification result. According to the embodiments of the present application, the efficiency of report data verification can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a report data verification method, device, equipment, system, medium and program product. Background Art

[0002] With the continuous development of big data applications, data mining has become a key component of big data technology. A large portion of the data used in data mining comes from reports, which can be referred to as report data. To accurately and reliably mine using report data, it's crucial to raise the quality requirements for report data, which necessitates verification.

[0003] With the continuous increase in the amount of report data and the number of fields, as well as the increasing complexity of the report data structure, there is an urgent need for a report data verification method with higher verification efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a report data verification method, apparatus, device, system, medium, and program product, which can improve the verification efficiency of report data.

[0005] In a first aspect, an embodiment of the present application provides a report data verification method, including: receiving an uploaded report file set, the report file set including a report file; scanning the report file, obtaining attribute information of the report file, and writing the attribute information into a database; obtaining a verification rule table template, importing the verification rule table template into the database, the verification rule table template including an expression of a verification rule; based on the report file set and the attribute information written into the database, screening the report file to be verified in the report file; calling a fast general computing engine, and using the expression of the verification rule in the database to verify the report file to be verified to obtain a verification result.

[0006] In the second aspect, an embodiment of the present application provides a report data verification device, including: a receiving module for receiving an uploaded report file set, the report file set including a report file; a scanning module for scanning the report file, obtaining the attribute information of the report file, and writing the attribute information into a database; a rule importing module for obtaining a verification rule table template, importing the verification rule table template into the database, the verification rule table template including the expression of the verification rule; a screening module for screening the report file to be verified in the report file according to the report file set and the attribute information written into the database; a verification module for calling a fast general computing engine, and using the expression of the verification rule in the database to verify the report file to be verified to obtain a verification result.

[0007] In a third aspect, an embodiment of the present application provides a report data verification device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the report data verification method of the first aspect is implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a report data verification system, comprising: a scanning storage device for receiving an uploaded report file set, the report file set including a report file, and scanning the report file to obtain attribute information of the report file and write the attribute information into a database device; a database device for receiving the written attribute information, and obtaining a verification rule table template, importing the verification rule table template, the verification rule table template including an expression of a verification rule; the scanning storage device is also used to filter the report file to be verified in the report file according to the report file set and the attribute information written into the database; a computing engine device for verifying the report file to be verified using the expression of the verification rule in the database to obtain a verification result.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, the report data verification method of the first aspect is implemented.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, characterized in that when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the report data verification method of the first aspect.

[0011] The embodiments of the present application provide a report data verification method, apparatus, equipment, system, medium and program product, which can scan the report files in the uploaded report file collection, and first perform a data format review based on the attribute information of the report file collection and the scanned report files, and screen out the report files that meet the data format as the report files to be verified. The verification rules are imported into the database through the verification rule table template, so that a fast general engine can be called to verify the report files to be verified using the verification rules imported into the database. Report files that do not meet the data format are screened out through data format review, avoiding time-consuming verification of report files that do not meet the data format; importing verification rules into the database in the form of a table saves time spent on configuration of verification rules; the fast general computing engine is designed for large-scale data processing, which further speeds up the verification speed; the report data verification method, apparatus, equipment, system, medium and program product provided by the embodiments of the present application can save the time required for verification, speed up the verification speed, and thus improve the verification efficiency of report data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 A schematic diagram of an example of a logical function module for executing a report data verification method provided in an embodiment of the present application;

[0014] Figure 2 A flowchart of a report data verification method provided in one embodiment of the present application;

[0015] Figure 3 A flowchart of a report data verification method provided in another embodiment of the present application;

[0016] Figure 4 A schematic diagram of the structure of a report data verification device provided in one embodiment of the present application;

[0017] Figure 5 A schematic diagram of the structure of a report data verification device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0018] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0019] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0020] With the continuous development of big data applications, data mining has become a key component of big data technology. A significant portion of the data used in data mining is obtained through reports, referred to here as report data. Accurate and stable data mining using report data requires higher quality standards, necessitating verification of report data. However, with the passage of time and increasing data requirements, the volume of report data continues to increase, along with the number of fields in reports and the increasing complexity of report data structures. This reduces the efficiency of report data verification. Therefore, a more efficient report data verification method is urgently needed.

[0021] The embodiments of the present application provide a report data verification method, apparatus, equipment, system, medium and program product, which can scan the report files in the uploaded report file collection, and first perform a data format review based on the attribute information of the report file collection and the scanned report files, and screen out the report files that meet the data format as the report files to be verified. The verification rules are imported into the database through the verification rule table template, so that a fast general engine can be called to verify the report files to be verified using the verification rules imported into the database. Report files that do not meet the data format are screened out through data format review, avoiding time-consuming verification of report files that do not meet the data format; importing the verification rules into the database in the form of a table saves the time spent on configuration of the verification rules; the fast general computing engine is designed for large-scale data processing, which further speeds up the verification speed; the report data verification method, apparatus, equipment, system, medium and program product provided by the embodiments of the present application can save the time required for verification, speed up the verification speed, and thus improve the verification efficiency.

[0022] The following is an introduction to the report data verification method, device, equipment, system, medium and program product provided by this application.

[0023] In a first aspect, the present application provides a report data verification method that can be applied to verify data in reports and improve data quality. The report data verification method can be executed by an apparatus, device, or the like, or by a system comprising multiple apparatuses and devices, without limitation. For ease of understanding, the following example illustrates the logical functional modules for executing the report data verification method in an embodiment of the present application. Figure 1 This is a schematic diagram of an example of a logical function module for executing a report data verification method provided in an embodiment of the present application. Figure 1As shown, the logical function module may include a storage scanning module 11, a database module 12, and a computing engine module 13. A report file uploader can upload report files, etc., to the storage scanning module 11. The storage scanning module 11 can interact with the database module 12, transferring certain information or data to the database module 12 for storage. The database module 12 can also import verification rules from an external source. The computing engine module 13 can obtain report files, etc. from the storage scanning module 11 and verification rules and other information from the database module 12 to verify the report files. Verification of report files includes verification of the data within the report files. After obtaining the verification results, the computing engine module 13 can also feed the verification results back to the storage scanning module 11 and the database module 12. The report file uploader can obtain the verification results from the storage scanning module 11 to determine whether the report file needs to be re-uploaded. The storage scanning module 11, database module 12, and computing engine module 13 can be implemented as modules, units, etc. within a single device or two or more devices, or each can be implemented as a device or equipment to form a system, but this is not a limitation here.

[0024] Figure 2 This is a flow chart of a report data verification method provided in one embodiment of the present application. Figure 2 As shown, the report data verification method may include steps S201 to S205.

[0025] In step S201, an uploaded report file set is received.

[0026] Receives report file collections from uploaders. Report file collections contain report files. Report files display data in a tabular report format. Tabular reports display the content in a tiled format, following the header order, making it easy to view detailed information. Tabular reports have a fixed number of fields, a specified format, and a specified entry method. They can be added, modified, and deleted.

[0027] In addition to report files, the list file collection may also include other files, which are not limited here. In some examples, the report file collection may also include a report status file and / or a report description file. Each time a report file uploader uploads a report file, they may upload a report status file. The report status file includes information representing the report status. For example, the report status file may include, but is not limited to, one or more of the following: a report file uploader identifier, report file data date information, report file upload time information, report file name information, and report file size information. The report file uploader identifier is used to identify the report file uploader. The report file data date information is used to represent the date of the data in the report file. If the data in the report file is data generated during business execution, the report file data date information may represent the date of the business corresponding to the data in the report file. The report file upload time information may represent the time the report file was uploaded. The report file name information is used to represent the name of the report file, the name of the report, etc. The report file size information may represent the amount of storage space occupied by the report file. The report status file may be implemented as an Extensible Markup Language (XML) file.

[0028] A report description file can include descriptive information that matches the report file. For example, the report file uploaded by the report file may contain special circumstances that cause the data in the report file to fail to meet some verification rules. However, the data in the report file does not have quality issues. The report description file can explain this situation. The report description file can include the verification rules that the data cannot meet and the reasons why the data cannot meet the verification rules.

[0029] In step S202, the report file is scanned to obtain attribute information of the report file, and the attribute information is written into the database.

[0030] A scanning program can be set up and called to scan the report file to obtain attribute information. The attribute information represents the attributes of the report file. For example, the attribute information may include the field type and number of fields in the report file, which is not limited here.

[0031] In step S203, a verification rule table template is obtained and imported into the database.

[0032] The validation rule table template is a table structure template that carries validation rules. The types of validation rules are not limited here. For example, validation rule types may include, but are not limited to, null value validation, out-of-range validation, date format and range validation, length validation, illegal character validation, numeric comparison validation, and other types of intra-table and inter-table logic.

[0033] The verification rules can be written into the verification rule template in the form of expressions, that is, the verification rule table template includes the expression of the verification rules. The verification rules implemented in the form of expressions are easy to configure, have a wide range of support, and are highly flexible. The expressions of the verification rules in the verification rule table template can be changed according to needs to implement the configuration, that is, the configuration method of the expression of the verification rules in the verification rule table template can be a custom configuration. The verification rules implemented in the form of expressions are also easy to execute, and some complex verification rule logics can be converted into expressions to be executed more accurately, avoiding the situation where the verification rules are complex and difficult to execute or errors occur in the execution. In some examples, at least some of the expressions of the verification rules in the verification rule table template may include the preconditions of the verification rules. For example, the verification rule is that field A1 is greater than 1, and the precondition is that field A2 is not empty, then the expression of the verification rule may be "field A1>1and field ”.

[0034] In step S204, based on the report file set and the attribute information written into the database, the report files are screened to obtain the report files to be verified.

[0035] Based on the report file collection and the attribute information written into the database, the report file data format can be reviewed to determine whether the report file meets the data format requirements for verification. The report to be verified is the report file that meets the data format requirements.

[0036] In some examples, the report file set also includes a report status file. The specific contents of the report status file can be found in the relevant descriptions of the above embodiments and will not be repeated here. Status information determined by scanning a report file in the report file set can be compared with the report status file to obtain a first comparison result; attribute information written into the database can be compared with standard attribute information to obtain a second comparison result. If the first comparison result indicates that the status information matches the report status file, and the second comparison result indicates that the attribute information matches the standard attribute information, the report file is determined to be a report file to be verified.

[0037] The first comparison result indicates whether the status information is consistent with the report status file. The status information determined by scanning the report file in the report file set is the actual status information of the report file. The report status file includes information that indicates the report status. When the data format of the report file is correct, the status information determined by scanning the report file should be consistent with the report status file. The status information determined by scanning the report file is consistent with the report status file, which may mean that the status information determined by scanning the report file should be consistent with the information in the report status file. If the status information indicated by the first comparison result is inconsistent with the report status file, the report file cannot be used as a report file to be verified. In the case where the status information indicated by the first comparison result is inconsistent with the report status file, the report files that do not meet the data format requirements can be divided into different categories of reports based on the status information that is inconsistent with the report status file. For example, the report file names of the report files in the report file collection can be compared with the report file name information in the report status file to see whether they are consistent, that is, whether the report files recorded in the report status file are all in the report file collection. If they are inconsistent, the report file can be determined as a first-category report file; the sizes of the report files in the report file collection can be compared with the report file size information in the report status file to see whether they are consistent. If they are inconsistent, the report file can be determined as a second-category report file.

[0038] The second comparison result indicates whether the attribute information is consistent with the standard attribute information. The attribute information is obtained by scanning the report files in the report file set. The standard attribute information is pre-set attribute information as a standard. When the data format of the report file is correct, the attribute information should be consistent with the standard attribute information. The attribute information is consistent with the report status file, which may mean that the attribute information should be consistent with the standard attribute information. If the second comparison result indicates that the attribute information is inconsistent with the standard attribute information, the report file cannot be used as a report file to be verified. In the case where the second comparison result indicates that the attribute information is inconsistent with the standard attribute information, the report files that do not meet the data format requirements can be divided into different categories of reports based on the attribute information that is inconsistent with the standard attribute information. For example, a report file whose number of fields obtained by scanning a report file is inconsistent with the number of standard fields can be determined as a third-category report file.

[0039] In some examples, the first comparison result, the second comparison result, and the category of the report file that does not meet the data format review requirements can be written into the database, and this information can also be used as part of the verification result to facilitate subsequent search for specific information on the data format review.

[0040] The first comparison result indicates that the status information is consistent with the report status file, and the second comparison result indicates that the attribute information is consistent with the standard attribute information, indicating that the report file meets the data format review requirements and can proceed to the subsequent verification process.

[0041] If there are report files that are not to be verified, a retransmission instruction can be issued to instruct the re-upload of the report files that do not meet the data format review requirements.

[0042] In step S205, a fast general calculation engine is called to verify the report file to be verified using the expression of the verification rule in the database to obtain a verification result.

[0043] The fast general computing engine can adopt the Spark engine, which can be used for large-scale data processing. It has the characteristics of high speed and strong versatility, and can speed up the verification process.

[0044] The verification rule expression is used to verify the report file to be verified, that is, to detect whether the report file to be verified meets the verification rule expression. The verification result may include relevant information generated during the verification process.

[0045] In some examples, the verification results include one or more of the following: verification statistics, summary report information, and error details files of report files that failed verification.

[0046] Verification statistics may include but are not limited to one or more of the following: verification success flag, retransmission flag, verification failure report file name, the number of matches between verification rules and data in the report file, the number of discrepancies between verification rules and data in the report file, the match rate between verification rules and data in the report file, the discrepancy rate between verification rules and data in the report file, the month-on-month change in the match rate between verification rules and data in the report file, the number of successful verifications of data in the report file, the number of failed verifications of data in the report file, the percentage of successful verifications of data in the report file, and the percentage of failed verifications of data in the report file.

[0047] The verification success flag indicates whether the verification was successful, guaranteeing either a successful or failed verification. A successful verification indicates that the report file to be verified complies with the verification rule expression, while a failed verification indicates that the report file to be verified does not comply with the verification rule expression. The retransmission flag indicates whether the report file uploader needs to re-upload the report file. The failed verification report file name includes the name of the report file that failed verification, that is, the name of the report that failed verification. The number of matches between the verification rules and the data in the report file may include the number of passes for each verification rule. The number of discrepancies between the verification rules and the data in the report file may include the number of failures for each verification rule. The match rate between the verification rules and the data in the report file may include the pass rate for each verification rule. The discrepancy rate between the verification rules and the data in the report file includes the failure rate for each verification rule. The month-on-month change in the match rate between the verification rules and the data in the report file is the month-on-month change in the pass rate for each verification rule. The number of successful verifications of data in the report file is the number of successfully verified report files, and a successfully verified report file refers to a report file that complies with each verification rule. The number of report files with failed verification is the number of report files that failed verification. Failed verification refers to report files that do not meet at least one verification rule. The successful verification percentage of report files is the percentage of report files that succeeded verification. The failed verification percentage of report files is the percentage of report files that failed verification.

[0048] The summary report information may include but is not limited to one or more of the following: the name of the report file to be verified, the name of the report file not to be verified, the number of matches between the verification rules and the data in the report file to be verified, and the number of discrepancies between the verification rules and the data in the report file to be verified.

[0049] Report files not to be verified are report files that do not meet the data review format requirements in the above-mentioned embodiment. The number of matches between verification rules and data in the report files to be verified may include the number of verification rules that each report file to be verified passed. The number of discrepancies between verification rules and data in the report files to be verified may include the number of verification rules that each report file to be verified failed.

[0050] The error detail file may include, but is not limited to, the location of the data that failed verification in the report file to be verified, and / or description information of the verification rule that caused the verification failure.

[0051] Data in the pending report file that failed verification includes data in the pending report file that does not comply with verification rules. Verification rules that caused verification failure include verification rules that do not comply with the data in the pending report file. The location of the data in the pending report file can be specified by row or column number within the report file, and is not limited here.

[0052] The verification results can help the report file uploader to troubleshoot errors and re-upload the report file.

[0053] In some examples, the storage locations of the verification statistics, summary report information, and error detail files of the verification failed report files may be different. For example, the verification statistics may be written to a database, and the summary report information and error detail files may be stored in the storage scanning module 11 in the above embodiment.

[0054] In an embodiment of the present application, the report files in the uploaded report file collection can be scanned, and based on the attribute information of the report file collection and the scanned report files, a data format review is first performed to screen out the report files that meet the data format as the report files to be verified, and the verification rules are imported into the database through the verification rule table template, so that the fast general engine can be called to use the verification rules imported into the database to verify the report files to be verified. The report files that do not meet the data format are screened out through data format review, thereby avoiding time-consuming verification of report files that do not meet the data format; the verification rules are imported into the database in the form of a table, saving the time spent on configuration of the verification rules; the fast general computing engine is designed for large-scale data processing, which further speeds up the verification speed; the report data verification method, device, equipment, system, medium and program product provided by the embodiment of the present application can save the time required for verification, speed up the verification speed, and thus improve the verification efficiency of the report data.

[0055] In the above embodiment, if the report file uploader re-uploads the report file, the verification results obtained from verifying the re-uploaded report file can overwrite the original verification results. The fast general computing engine can implement verification through engine tasks. If the engine task fails, the error report in the fast general computing engine's operation log can be used to investigate the configuration of verification rules and report files.

[0056] In some embodiments, the verification rules may include a first type of verification rule and a second type of verification rule. The first type of verification rule may be considered a strong verification rule; if a report file does not comply with the first type of verification rule, the report file verification fails. The second type of verification rule may be considered a weak verification rule; if a report file does not comply with the second type of verification rule, the report file verification succeeds. However, a prompt message may be provided to the report file uploader, prompting them to adjust the report file to comply with the second type of verification rule before uploading it if they obtain a report file of the same type.

[0057] If the verification result includes a verification success flag, a retransmission flag, and verification details, and if the report file to be verified does not comply with the first type of verification rules, the verification success flag indicates a verification failure, the retransmission flag indicates re-upload of the report file, and the verification details include a first error position and description information of the first type of verification rules that the report file to be verified does not comply with. The first error position includes the position of the data that does not comply with the first type of verification rules in the report file to be verified.

[0058] If the verification result includes a verification success flag, a retransmission flag, and verification details, when the report file to be verified complies with the first type of verification rules but does not comply with the second type of verification rules, the verification success flag indicates that the verification is successful, the retransmission flag indicates that the report file does not need to be re-uploaded, and the verification details include a second error position and description information of the second type of verification rules that the report file to be verified does not comply with. The second error position includes the position of the data that does not comply with the second type of verification rules in the report file to be verified.

[0059] In some examples, the fast general calculation engine may first use the expression of the first type of verification rule to verify the report file to be verified. If the verification of the first type of verification rule fails, the verification result may be generated directly without using the expression of the second type of verification rule to verify the report file to be verified; if the verification of the first type of verification rule is successful, the expression of the second type of verification rule may be used to verify the report file to be verified; regardless of whether the verification of the second type of verification rule fails or succeeds, a verification result is generated.

[0060] In some embodiments, the files in the report file set may be scanned periodically. Figure 3 This is a flowchart of a report data verification method provided in another embodiment of the present application. Figure 3 and Figure 2 The difference is that Figure 2 Step S202 in the above example can be specifically broken down into Figure 3 Steps S2021 to S2023 in Figure 3 The report data verification method shown may further include step S206 and step S207.

[0061] In step S2021, the report file is scanned periodically.

[0062] Report file uploaders can upload report files via a dedicated network. This means that report files uploaded by the uploader can be received via the dedicated network. The storage location for uploaded report files can be pre-defined, and the scanning program can be set to that storage location to reduce the time required to scan report files.

[0063] Since you can't determine how long it takes for a report file to upload, you can schedule a report file scan. The scheduled duration can be set based on the scenario and needs, and is not limited here. For example, if the scheduled duration is 5 minutes, the report file will be scanned every 5 minutes.

[0064] In step S2022, if the sizes of the report files obtained from two consecutive scans are different, the uploaded report files are continued to be received.

[0065] If the sizes of the report files obtained from two consecutive scans are different, it means that the report file has not been uploaded yet. You can continue to receive the uploaded report file. Since the report file has not been uploaded yet, the attribute information of the report file is also incomplete. Therefore, there is no need to obtain the attribute information of the report file.

[0066] In step S2023, if the sizes of the report files obtained by two consecutive scans are the same, the attribute information obtained by scanning the report files is obtained.

[0067] If the report files obtained from two consecutive scans have the same size, it means that the report file has been uploaded, the attribute information of the table file is complete, and the attribute information can be obtained.

[0068] In some examples, the report file may be uploaded in the form of a compressed package. If the report files obtained from two consecutive scans have the same size, the compressed package may be decompressed to extract the report file.

[0069] In step S206, the report status file is scanned.

[0070] The scanning of the message status file may be a timed scanning, and the timed duration may be the same as or different from the timed duration of the scanning report file, which is not limited here.

[0071] In step S207, if the report status files obtained by two consecutive scans are different, the report file is scanned again to obtain the attribute information of the report file, and the attribute information is written into the database to overwrite the original attribute information of the report file in the database.

[0072] Each time a report file uploader re-uploads a report file, a new report status file overwrites the original one. You can determine whether a report file has been re-uploaded by checking whether the report status files obtained from two consecutive scans are the same. If the report status files obtained from two consecutive scans are different, it means the report file has been re-uploaded. The re-uploaded report file differs from the original report file, and accordingly, the attribute information of the re-uploaded report file also differs from the attribute information of the original report file. You need to obtain the attribute information of the re-uploaded report file, write the newly obtained attribute information to the database, re-review the data format, and verify the re-uploaded report file.

[0073] After step S207 , step S204 and step S205 may be executed again, but the attribute information in step S204 is the attribute information newly written into the database, that is, the attribute information that overwrites the attribute information of the original report file.

[0074] If the report status files obtained from two consecutive scans are the same, it means that the report file has not changed and no additional action is required.

[0075] In some embodiments, the report file set also includes a report description file, which may include an ignore verification rule. The ignore verification rule is a verification rule that needs to be ignored, and the ignore verification rule includes a verification rule that the data in the report file does not comply with but does not affect the data quality. Before step S205, the report description file can be scanned to obtain the ignore verification rule, and the expression of the ignore verification rule in the database can be invalidated. The invalidation of the expression of the ignore verification rule in the database can refer to deactivating the expression of the ignore verification rule in the database at the beginning of the verification, so that the expression of the ignore verification rule is invalid, that is, the expression of the ignore verification rule is not used in the process of verifying the report file. After the verification is completed, the expression of the ignore verification rule in the database can be validated to make the expression of the ignore verification rule valid.

[0076] In some examples, the report description file may further include an ignoring reason, where the ignoring reason includes a description of why the report file does not comply with the ignoring verification rules but does not affect the data quality.

[0077] By setting the ignore verification rules in the report description file, you can avoid verification failures caused by non-data quality issues, and further improve the accuracy of verification while eliminating manual verification.

[0078] A second aspect of the present application provides a report data verification device. Figure 4 This is a structural diagram of a report data verification device provided in one embodiment of the present application. Figure 4 As shown, the report data verification device 300 may include a receiving module 301 , a scanning module 302 , a rule importing module 303 , a screening module 304 and a verification module 305 .

[0079] The receiving module 301 may be configured to receive an uploaded report file set, where the report file set includes a report file.

[0080] The scanning module 302 may be used to scan a report file, obtain attribute information of the report file, and write the attribute information into a database.

[0081] The rule import module 303 may be used to obtain a verification rule table template and import the verification rule table template into a database. The verification rule table template includes an expression of a verification rule.

[0082] The screening module 304 may be used to screen the report files to obtain the report files to be verified based on the report file set and the attribute information written into the database.

[0083] The verification module 305 can be used to call a fast general calculation engine, and use the expression of the verification rule in the database to verify the report file to be verified to obtain the verification result.

[0084] In some examples, the verification results include one or more of the following: verification statistics, summary report information, and error details files of report files that failed verification.

[0085] Verification statistics include one or more of the following: verification success flag, retransmission flag, verification failure report file name, number of matches between verification rules and data in report file, number of discrepancies between verification rules and data in report file, match rate between verification rules and data in report file, discrepancy rate between verification rules and data in report file, month-on-month change in match rate between verification rules and data in report file, number of successful verifications in report file, number of failed verifications in report file, percentage of successful verifications in report file, and percentage of failed verifications in report file.

[0086] The summary report information includes one or more of the following: the name of the report file to be verified, the name of the report file not to be verified, the number of matches between the verification rules and the data in the report file to be verified, and the number of discrepancies between the verification rules and the data in the report file to be verified.

[0087] The error detail file includes: the location of the data that failed verification in the report file to be verified, and / or the description information of the verification rule that caused the verification failure.

[0088] In an embodiment of the present application, the report files in the uploaded report file collection can be scanned, and based on the attribute information of the report file collection and the scanned report files, a data format review is first performed to screen out the report files that meet the data format as the report files to be verified, and the verification rules are imported into the database through the verification rule table template, so that the fast general engine can be called to use the verification rules imported into the database to verify the report files to be verified. The report files that do not meet the data format are screened out through data format review, thereby avoiding time-consuming verification of report files that do not meet the data format; the verification rules are imported into the database in the form of a table, saving the time spent on configuration of the verification rules; the fast general computing engine is designed for large-scale data processing, which further speeds up the verification speed; the report data verification method, device, equipment, system, medium and program product provided by the embodiment of the present application can save the time required for verification, speed up the verification speed, and thus improve the verification efficiency of the report data.

[0089] In some embodiments, the verification rules include first-type verification rules and second-type verification rules, and the verification results include a verification success flag, a retransmission flag, and verification details.

[0090] When the report file to be verified does not comply with the first type of verification rules, the verification success flag indicates that the verification has failed, the retransmission flag indicates that the report file is to be uploaded again, and the verification details include the first error position and description information of the first type of verification rules that the report file to be verified does not comply with. The first error position includes the position of the data that does not comply with the first type of verification rules in the report file to be verified.

[0091] When the report file to be verified complies with the first type of verification rules but does not comply with the second type of verification rules, the verification success flag indicates that the verification is successful, the retransmission flag indicates that the report file will not be re-uploaded, and the verification details include the second error position and description information of the second type of verification rules that the report file to be verified does not comply with. The second error position includes the position of the data that does not comply with the second type of verification rules in the report file to be verified.

[0092] In some embodiments, the scanning module 302 can be specifically used to: scan report files at regular intervals; if the sizes of report files obtained by two consecutive scans are different, continue to receive uploaded report files; if the sizes of report files obtained by two consecutive scans are the same, obtain attribute information obtained by scanning the report files.

[0093] In some embodiments, the report file set also includes a report status file.

[0094] The scanning module 302 can also be used to: scan the report status file; if the report status files obtained by two consecutive scans are different, rescan the report file to obtain the attribute information of the report file, write the attribute information into the database, and overwrite the attribute information of the original report file in the database.

[0095] The screening module 304 is further configured to screen the report files to obtain the report files to be verified according to the report file set and the attribute information newly written into the database.

[0096] The verification module 305 is further used to call a fast general calculation engine, verify the report file to be verified using the expression of the verification rule in the database, and obtain the verification result.

[0097] In some examples, the report status file includes one or more of the following: report file uploader identification, report file data date information, report file upload time information, report file name information, and report file size information.

[0098] In some embodiments, the report file set further includes a report description file, and the report description file includes ignoring verification rules.

[0099] The scanning module 302 can also be used to scan the report description file to obtain the ignore verification rules.

[0100] The rule import module 303 can also be used to invalidate expressions in the database that ignore verification rules.

[0101] In some examples, the expression of the verification rule in the verification rule table template is configured in a custom configuration manner.

[0102] In some examples, at least part of the expressions of the verification rules in the verification rule table template include preconditions of the verification rules.

[0103] In some embodiments, the report file set also includes a report status file.

[0104] The screening module 304 can also be used to: compare the status information determined by scanning the report file in the report file set with the report status file to obtain a first comparison result; compare the attribute information written in the database with the standard attribute information to obtain a second comparison result; when the first comparison result indicates that the status information is consistent with the report status file, and the second comparison result indicates that the attribute information is consistent with the standard attribute information, determine that the report file is a report file to be verified.

[0105] The third aspect of the present application provides a report data verification device. Figure 5 This is a structural diagram of a report data verification device provided in one embodiment of the present application. Figure 5 As shown, the report data verification device 400 includes a memory 401 , a processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0106] In an example, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0107] The memory 401 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the report data verification method according to the embodiment of the present application.

[0108] The processor 402 reads the executable program code stored in the memory 401 to run a computer program corresponding to the executable program code, so as to implement the report data verification method in the above embodiment.

[0109] In one example, the report data verification device 400 may further include a communication interface 403 and a bus 404. Figure 5 As shown, the memory 401 , the processor 402 , and the communication interface 403 are connected via a bus 404 and communicate with each other.

[0110] The communication interface 403 is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. Input devices and / or output devices can also be connected through the communication interface 403.

[0111] The bus 404 includes hardware, software, or both, and couples the components of the report data verification device 400 to each other. By way of example, and not limitation, the bus 404 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, the bus 404 may include one or more buses. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0112] A fourth aspect of the present application provides a report data verification system, which may include a scanning storage device, a database device, and a computing engine device.

[0113] The scanning storage device can be used to receive an uploaded report file set, the report file set including a report file, and scan the report file to obtain attribute information of the report file, and write the attribute information into the database device.

[0114] The database device can be used to receive written attribute information, and obtain a verification rule table template and import the verification rule table template, wherein the verification rule table template includes an expression of a verification rule.

[0115] The scanning storage device can also be used to filter the report files to obtain the report files to be verified based on the report file set and the attribute information written into the database.

[0116] The calculation engine device can be used to verify the report file to be verified using the expression of the verification rule in the database to obtain a verification result.

[0117] The scanning storage device can be considered the storage scanning module 11 in the above embodiment, the database device can be considered the database module 12 in the above embodiment, and the computing engine device can be considered the computing engine module 13 in the above embodiment. The steps performed by the scanning storage device, the database device, and the computing engine device can be referred to in the relevant description of the report data verification method in the above embodiment, and can achieve the same technical effects. To avoid repetition, they are not repeated here.

[0118] In a fifth aspect, the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the report data verification method in the above-mentioned embodiment can be implemented, and the same technical effect can be achieved. To avoid repetition, the above-mentioned computer-readable storage medium may include a non-transitory computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which is not limited here.

[0119] In a sixth aspect, the present application provides a computer program product. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device executes the report data verification method in the above embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0120] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For device embodiments, equipment embodiments, system embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description part of the method embodiment. This application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of this application. In addition, for the sake of brevity, a detailed description of known method technologies is omitted here.

[0121] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.

[0122] Those skilled in the art should understand that the above embodiments are illustrative rather than restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, the specification and the claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other devices or steps; the quantifier "one" does not exclude a plurality; the terms "first" and "second" are used to identify names rather than to indicate any specific order. Any figure marks in the claims should not be understood as limiting the scope of protection. The functions of multiple parts appearing in the claims can be implemented by a separate hardware or software module. The fact that certain technical features appear in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A report data verification method, characterized in that: include: Receiving an uploaded report file set, wherein the report file set includes a report file; Scan the report file, obtain attribute information of the report file, and write the attribute information into a database; Obtaining a verification rule table template, and importing the verification rule table template into the database, wherein the verification rule table template includes an expression of a verification rule; According to the report file set and the attribute information written in the database, the report files are screened to obtain the report files to be verified; Invoking a fast general calculation engine to verify the report file to be verified using the expression of the verification rule in the database to obtain a verification result; The verification rules include first-type verification rules and second-type verification rules, and the verification results include a verification success flag, a retransmission flag, and verification details; If the report file to be verified does not comply with the first type of verification rules, the verification success flag indicates a verification failure, the retransmission flag instructs re-uploading of the report file, and the verification details include a first error location and description information of the first type of verification rules that the report file to be verified does not comply with, the first error location including the location of the data in the report file to be verified that does not comply with the first type of verification rules in the report file to be verified; When the report file to be verified complies with the first type of verification rules but does not comply with the second type of verification rules, the verification success identifier indicates that the verification is successful, the retransmission identifier indicates that the report file is not to be re-uploaded, and the verification details information includes a second error position and description information of the second type of verification rules that the report file to be verified does not comply with. The second error position includes the position of the data in the report file to be verified that does not comply with the second type of verification rules in the report file to be verified.

2. The method according to claim 1, characterized in that The scanning of the report file to obtain attribute information of the report file includes: Scanning the report file regularly; If the sizes of the report files obtained from two consecutive scans are different, continue to receive the uploaded report files; If the sizes of the report files obtained by two consecutive scans are the same, the attribute information obtained by scanning the report files is acquired.

3. The method according to claim 1, characterized in that The report file set also includes a report status file, The method further comprises: Scanning the report status file; If the report status files obtained by two consecutive scans are different, rescan the report file to obtain the attribute information of the report file, and write the attribute information into the database to overwrite the original attribute information of the report file in the database; According to the report file set and the attribute information newly written into the database, the report file to be verified is obtained by screening the report files; A fast general calculation engine is called to verify the report file to be verified using the expression of the verification rule in the database to obtain a verification result.

4. The method according to claim 3, characterized in that The report status file includes one or more of the following: Report file uploader ID, report file data date information, report file upload time information, report file name information, and report file size information.

5. The method according to claim 1, wherein The report file set also includes a report description file, and the report description file includes ignoring verification rules, Before calling the fast general calculation engine and verifying the report file to be verified using the expression of the verification rule in the database to obtain the verification result, the method further includes: Scan the report description file to obtain the ignore verification rule; Invalidate the expression in the database that ignores the verification rule.

6. The method according to claim 1, characterized in that The configuration mode of the expression of the verification rule in the verification rule table template is custom configuration; At least part of the expressions of the verification rules in the verification rule table template include the preconditions of the verification rules.

7. The method according to claim 1, characterized in that The report file set also includes a report status file, The step of screening the report files to obtain the report files to be verified based on the report file set and the attribute information written into the database includes: Comparing the status information determined by scanning the report files in the report file set with the report status file to obtain a first comparison result; comparing the attribute information written into the database with the standard attribute information to obtain a second comparison result; When the first comparison result characterizing state information is consistent with the report state file, and the second comparison result characterizing attribute information is consistent with the standard attribute information, the report file is determined to be the report file to be verified.

8. The method according to claim 1, characterized in that The verification results include one or more of the following: Verify statistical data, summary report information, and error details of report files that failed verification; The verification statistics include one or more of the following: a verification success flag, a retransmission flag, a verification failure report file name, the number of matches between the verification rules and the data in the report file, the number of discrepancies between the verification rules and the data in the report file, the match rate between the verification rules and the data in the report file, the discrepancy rate between the verification rules and the data in the report file, the month-on-month change in the match rate between the verification rules and the data in the report file, the number of successful verifications in the report file, the number of failed verifications in the report file, the percentage of successful verifications in the report file, and the percentage of failed verifications in the report file; The summary report information includes one or more of the following: the name of the report file to be verified, the name of the report file not to be verified, the number of matches between the verification rules and the data in the report file to be verified, and the number of discrepancies between the verification rules and the data in the report file to be verified; The error detail file includes: the location of the data that failed verification in the report file to be verified, and / or description information of the verification rule that caused the verification failure.

9. A report data verification device, characterized in that: include: A receiving module, configured to receive an uploaded report file set, wherein the report file set includes a report file; A scanning module, configured to scan the report file, obtain attribute information of the report file, and write the attribute information into a database; A rule import module, configured to obtain a verification rule table template and import the verification rule table template into the database, wherein the verification rule table template includes an expression of a verification rule; A screening module, configured to screen the report files to obtain the report files to be verified based on the report file set and the attribute information written into the database; A verification module, configured to call a fast general calculation engine, verify the report file to be verified using the expression of the verification rule in the database, and obtain a verification result; The verification rules include first-type verification rules and second-type verification rules, and the verification results include a verification success flag, a retransmission flag, and verification details; If the report file to be verified does not comply with the first type of verification rules, the verification success flag indicates a verification failure, the retransmission flag instructs re-uploading of the report file, and the verification details include a first error location and description information of the first type of verification rules that the report file to be verified does not comply with, the first error location including the location of the data in the report file to be verified that does not comply with the first type of verification rules in the report file to be verified; When the report file to be verified complies with the first type of verification rules but does not comply with the second type of verification rules, the verification success identifier indicates that the verification is successful, the retransmission identifier indicates that the report file is not to be re-uploaded, and the verification details information includes a second error position and description information of the second type of verification rules that the report file to be verified does not comply with. The second error position includes the position of the data in the report file to be verified that does not comply with the second type of verification rules in the report file to be verified.

10. A report data verification device, characterized in that: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the report data verification method according to any one of claims 1 to 8 is implemented.

11. A report data verification system, characterized in that: include: A scanning storage device is used to receive an uploaded report file set, wherein the report file set includes a report file, and scan the report file to obtain attribute information of the report file, and write the attribute information into a database device; The database device is configured to receive the written attribute information, obtain a verification rule table template, and import the verification rule table template, wherein the verification rule table template includes an expression of a verification rule; The scanning storage device is further configured to filter the report files to obtain the report files to be verified based on the report file set and the attribute information written into the database; A computing engine device, configured to verify the report file to be verified using the expression of the verification rule in the database to obtain a verification result; The verification rules include first-type verification rules and second-type verification rules, and the verification results include a verification success flag, a retransmission flag, and verification details; If the report file to be verified does not comply with the first type of verification rules, the verification success flag indicates a verification failure, the retransmission flag instructs re-uploading of the report file, and the verification details include a first error location and description information of the first type of verification rules that the report file to be verified does not comply with, the first error location including the location of the data in the report file to be verified that does not comply with the first type of verification rules in the report file to be verified; When the report file to be verified complies with the first type of verification rules but does not comply with the second type of verification rules, the verification success identifier indicates that the verification is successful, the retransmission identifier indicates that the report file is not to be re-uploaded, and the verification details information includes a second error position and description information of the second type of verification rules that the report file to be verified does not comply with. The second error position includes the position of the data in the report file to be verified that does not comply with the second type of verification rules in the report file to be verified.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the report data verification method according to any one of claims 1 to 8.

13. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the report data verification method according to any one of claims 1 to 8.

Images