Audit data quality modeling analysis method and device

Abstract

The present invention relates to an audit data quality modeling and analysis method and device, the method comprising: obtaining a data dictionary of a target database; constructing an audit data rule base; marking the source attribute to which data type information belongs; identifying the named entity information of the audit data element to be evaluated, performing format screening, duplicate checking and key field extraction on the audit data element to be evaluated, and obtaining key information; constructing a data set; generating quality modeling and analysis results of each dimension indicator; calculating the weight of each dimension indicator; constructing an audit data quality modeling and analysis model, and calculating and obtaining the audit data quality modeling and analysis results. The present invention is based on the audit data quality modeling and analysis model, combined with NLP technology, OCR technology, MD5 algorithm and bidirectional long short-term memory network, comprehensively considering factors such as the source attribute of the audit data element, file structure information, named entity information and key information, and effectively improving the quality modeling and analysis effect of the audit data element.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an audit data quality modeling and analysis method and device. Background Art

[0002] Audit data refers to data that is spontaneously generated, collected, maintained, managed, and owned during the long-term operation of the audit object. The data sources involved in the audit mainly include the internal management system, financial system, business system, and external data sources (such as public databases, third-party data providers, etc.) of the audit object, such as enterprises, institutions, and individuals. Audit data types include structured data (such as tables and databases) and unstructured data (such as text, pictures, audio, video, etc.). The content carried by audit data covers the financial records, transaction information, business activities, internal control, and other related information of the audit object.

[0003] Audit data itself presents some new characteristics, such as large data volume (Volume), various data types (Variety), fast processing speed (Velocity), and low value density (Value). Therefore, how to extract high-quality audit data from massive, rapidly changing, and richly sourced big data to support accurate and true audit business work has become an urgent problem that restricts the development of audit business.

[0004] The quality of audit data reflects the extent to which the data can meet the audit business needs, including: (1) ensuring the accuracy of audit results: high-quality data is the basis for ensuring the correctness of audit conclusions; (2) improving audit efficiency: good data quality can reduce data cleaning and verification work during the audit process and improve work efficiency; (3) reducing audit risk: data quality issues are one of the important factors leading to audit risk.

[0005] From a qualitative perspective, the quality of audit data depends on the following factors: (1) Completeness, that is, whether the data is comprehensive and has no missing values; (2) Accuracy, whether the data truly reflects the economic activities of the enterprise; (3) Consistency, whether the data remains consistent at different times and from different sources; (4) Traceability: whether the data can be traced back to its original source; (5) Timeliness: whether the data is within the validity period; (6) Standardization, audit data should comply with certain standards or specifications, such as data format, measurement standards, coding rules, etc., to ensure the stability and comparability of data quality.

[0006] Audit data quality has received more and more attention in practice. With the deepening of theoretical research, some qualitative analysis methods for audit data quality have emerged, and some management system measures for data quality control have been formulated. However, the existing technology still lacks a targeted rule comparison process for different types of audit data elements. The indicator constraint rules are determined only based on the quality modeling analysis indicators, resulting in poor audit data element quality modeling analysis results and poor practicality. Summary of the invention

[0007] Based on this, it is necessary to provide an audit data quality modeling and analysis method and equipment to address the above technical issues.

[0008] In order to achieve the above object, the present invention provides an audit data quality modeling and analysis method, comprising:

[0009] Get the data dictionary of the target database;

[0010] According to the data dictionary of the target database and with reference to the audit standards, an audit data rule base is constructed;

[0011] Obtain the audit data element to be evaluated of the target database, use a hash table to perform a pre-check on the database table of the audit data element to be evaluated, identify the data type information and file structure information contained in the audit data element to be evaluated, and mark the source attribute to which the data type information belongs accordingly;

[0012] The audit data elements to be evaluated are loaded according to the marked source attributes, and the bidirectional long short-term memory network is used to perform named entity recognition on the audit data elements to be evaluated to obtain the named entity information of the audit data elements to be evaluated. Then, the audit data elements to be evaluated are format screened, duplicated, and key fields are extracted in turn to obtain key information.

[0013] Construct data sets based on source attributes, file structure information, named entity information, and key information;

[0014] Based on the data quality modeling and analysis system, the data set is matched and compared with the corresponding types of rules in the audit data rule library to generate the quality modeling and analysis results of each dimension indicator;

[0015] The analytic hierarchy process is used to calculate the weight of each dimension indicator;

[0016] Based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, an audit data quality modeling and analysis model is constructed. The quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator are input into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results.

[0017] Optionally, the data dictionary includes: definition and description information of target database objects, attribute information of target database objects, association relationships of target database objects, target database users and authority information, and parameters and configuration information of the target database.

[0018] Optionally, the audit data rule base includes pre-rules, basic rules, business rules and intelligent rules; the pre-rules include database naming rules and database structure rules, which are used to detect whether the naming of the database and the structure of the database comply with the pre-specifications of the audit data quality modeling and analysis; the basic rules include audit data format rules, which are used to detect format quality problems of audit data elements; the business rules include general rules and audit domain rules, which are used to detect whether the general identification data and domain data of the audit data elements comply with the general rules and audit domain rules established by the audit standards; the intelligent rules include association verification rules, which are used to detect the degree of logical association matching between audit data and users and / or other data and / or business applications.

[0019] Optionally, the method of using a hash table to perform a pre-check of a library table on the audit data element to be evaluated, identifying the data type information and file structure information contained in the audit data element to be evaluated, and correspondingly marking the source attribute to which the data type information belongs, includes:

[0020] NLP technology is used to identify the data type and length of the table header items and the first five non-blank rows of the audit data element to be evaluated, and the data type information contained in the audit data element to be evaluated is obtained. OCR technology is used to perform structured recognition on the audit data element to be evaluated, and the file structure information contained in the audit data element to be evaluated is obtained. The identified content is then concatenated with the file name of the audit data element to be evaluated to obtain concatenated data. The concatenated data is then converted into character format and converted into a character vector or a string scalar. The MD5 algorithm is used to calculate the hash value, and the source attribute to which the data type information belongs is marked by matching and comparing with the hash value of the stored source attribute.

[0021] Optionally, the step of loading the audit data element to be evaluated according to the marked source attribute, and performing named entity recognition on the audit data element to be evaluated using a bidirectional long short-term memory network to obtain the named entity information of the audit data element to be evaluated includes:

[0022] The audit data elements to be evaluated are loaded according to the marked source attributes, and data cleaning is performed on the audit data elements to obtain the cleaned named data information. A bidirectional long short-term memory network is used to perform named entity recognition on the cleaned named data information to obtain the named entity information of the audit data elements to be evaluated.

[0023] Optionally, the data quality modeling and analysis system is based on matching and comparing the data set with the rules of the corresponding type in the audit data rule library to generate the quality modeling and analysis results of each dimension indicator, including:

[0024] According to the marked source attributes, the data set is matched and compared with the corresponding types of rules in the audit data rule library. Based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, the error rate and score of the data information contained in the data set under the six dimensional indicators are calculated to form the quality modeling analysis results of each dimensional indicator.

[0025] Optionally, the calculation formula of the audit data quality modeling analysis model is as follows:

[0026]

[0027] The constraints are: , Q is the audit data quality score, G is the quality modeling analysis result of the standardization dimension, W is the quality modeling analysis result of the integrity dimension, Z is the quality modeling analysis result of the accuracy dimension, Y is the quality modeling analysis result of the consistency dimension, S is the quality modeling analysis result of the timeliness dimension, and K is the quality modeling analysis result of the traceability dimension. is the weight of the normative dimension, is the weight of the completeness dimension indicator, is the weight of the accuracy dimension indicator, is the weight of the consistency dimension indicator, is the weight of the timeliness dimension indicator, It is the weight of the traceability dimension indicator.

[0028] The present invention also provides an audit data quality modeling and analysis device, comprising:

[0029] A data dictionary acquisition module is used to acquire the data dictionary of the target database;

[0030] An audit data rule base construction module is used to construct an audit data rule base based on the data dictionary of the target database and with reference to the audit standards;

[0031] The audit data element acquisition and pre-detection module is used to obtain the audit data element to be evaluated in the target database, use the hash table to perform pre-detection of the database table on the audit data element to be evaluated, identify the data type information and file structure information contained in the audit data element to be evaluated, and mark the source attribute to which the data type information belongs;

[0032] A named entity recognition module is used to load the audit data element to be evaluated according to the marked source attributes, and use a bidirectional long short-term memory network to perform named entity recognition on the audit data element to be evaluated to obtain the named entity information of the audit data element to be evaluated;

[0033] The data processing module is used to perform format screening, duplicate checking and key field extraction on the audit data elements to be evaluated in order to obtain key information;

[0034] A data set construction module is used to construct a data set based on source attributes, file structure information, named entity information and key information;

[0035] The matching and comparison module is used to match and compare the data set with the corresponding types of rules in the audit data rule library based on the data quality modeling and analysis system, and generate the quality modeling and analysis results of each dimension indicator;

[0036] The weight calculation module is used to calculate the weight of each dimension indicator using the hierarchical analysis method;

[0037] The audit data quality modeling and analysis model building module is used to build an audit data quality modeling and analysis model based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability.

[0038] The data quality modeling and analysis module is used to input the quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results.

[0039] The present invention also provides a computer device, the computer device comprising: a processor, the processor is used to implement the steps of any of the above methods when executing a computer program stored in a memory.

[0040] The present invention also provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the steps described in any of the above methods are implemented.

[0041] The advantages and beneficial effects of the present invention are: the present invention provides an audit data quality modeling and analysis method and device. The evaluation method is based on the audit data quality modeling and analysis model, combines NLP technology, OCR technology, MD5 algorithm and bidirectional long short-term memory network, and specifically constructs a data set with source attributes. According to the source attributes of the audit data element to be evaluated, the corresponding types of rules are matched and compared in the audit data rule library, and the quality modeling and analysis results of each dimensional indicator are analyzed from six dimensions: standardization, completeness, accuracy, consistency, timeliness and traceability. According to the quality modeling and analysis results of each dimensional indicator, an overall evaluation is performed on the audit data element to be evaluated, and then the audit data quality modeling and analysis results are obtained. The source attributes of the audit data element, file structure information, named entity information and key information of the audit data element are comprehensively considered. The influence of factors such as the data quality modeling and analysis is effectively improved. The quality modeling and analysis effect of the audit data element is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of an audit data quality modeling and analysis method in an embodiment of the present invention;

[0043] Figure 2 It is a structural schematic diagram of an audit data quality modeling and analysis device in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific implementation methods in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] To facilitate understanding of this embodiment, the embodiment of the present invention is described in detail below.

[0046] Embodiment 1

[0047] Figure 1 A flowchart of an audit data quality modeling and analysis method provided in Example 1 of the present invention.

[0048] refer to Figure 1 , the evaluation method comprises the following steps:

[0049] S101, obtaining a data dictionary of a target database.

[0050] In this embodiment, in order to build an audit data rule base, it is first necessary to obtain the data dictionary of the target database, which mainly contains the following information: definition and description information of target database objects, attribute information of target database objects, association relationships of target database objects, target database users and permission information, parameters and configuration information of the target database, etc.

[0051] It should be understood that the data dictionary is an important component of the target database. It is mainly used to store a collection of various objects in the target database, including the definition and description information of all objects in the target database. The objects here are also audit data elements. The data dictionary can help users understand the various objects and data structures in the target database, as well as the relationships and associations between them.

[0052] The definition and description information of the target database objects include but are not limited to tables, views, indexes, stored procedures, etc.; the attribute information of the target database objects include but are not limited to the name, type, size, creation time, modification time, etc. of the objects; the association relationships of the target database objects include but are not limited to the relationship between tables, the relationship between views and tables, the relationship between indexes and tables, etc.; the target database user and permission information includes but is not limited to the user's name, role, permission, etc.; the target database parameter and configuration information includes but is not limited to the database name, character set, time zone, etc.

[0053] For example, the definition and description information of the target database objects: This may include the specific definitions of the financial tables, transaction record tables, employee information tables, etc. required for the audit, such as the description of the account code, name, amount field, etc. contained in the financial tables. Attribute information of the target database objects: This involves the attributes of each audit object, such as the data type of each account in the financial table (integer, floating point number, string, etc.), whether it is allowed to be empty, field length limit, etc. Association relationship of the target database objects: In auditing, this may refer to the relationship between different tables, such as the association between the sales order table and the accounts receivable table, or the relationship between the employee information table and the salary payment table. Target database user and permission information: This includes which users can access or modify audit data, for example, only users in the finance department can change financial data, or only auditors have the permission to view specific sensitive information. Parameters and configuration information of the target database: In auditing, this may include the frequency of data backup, data retention policy, configuration of system audit logs, etc., which are key factors to ensure data integrity and reliability. If the audit objective is the authenticity of financial statements, the "financial table" object in the data dictionary will describe the structure of the report, including the specific fields and formats of the balance sheet, income statement, etc. If the audit objective is the legality of the transaction, the "Transaction Record Table" object in the data dictionary will detail the fields of each transaction, such as transaction date, amount, information about both parties to the transaction, etc. Audit authority information may include "Salary Payment Table" that can only be accessed by specifically authorized auditors, ensuring confidentiality and security during the audit process.

[0054] S102. Construct an audit data rule base based on the data dictionary of the target database and with reference to the audit standards.

[0055] In this embodiment, an audit data rule base is constructed based on the acquired data dictionary and with reference to the audit standards. For example, a data dictionary of audit data is acquired based on the financial system of a certain enterprise unit (i.e., the target database), and an audit data rule base is established with reference to national standards, industry standards, and local audit standards.

[0056] In this embodiment, the audit data rule library includes pre-rules, basic rules, business rules and intelligent rules; the pre-rules include database naming rules and database structure rules, which are used to detect whether the database naming and database structure comply with the pre-specifications of audit data quality modeling and analysis; the basic rules include audit data format rules, which are used to detect format quality problems of audit data elements; the business rules include general rules and audit domain rules, which are used to detect whether the general identification data and domain data of the audit data elements comply with the general rules and audit domain rules established by the audit standards; the intelligent rules include association verification rules, which are used to detect the degree of logical association matching between audit data and users and / or other data and / or business applications.

[0057] Among them, the basic rules involved in the present invention are mainly used to detect general format quality problems of audit data elements (such as missing data and duplicate data, etc.) without the help of national standards, industry standards and local standards; business rules are data quality rules mainly established according to national standards, industry standards and local standards, and are composed of general rules (such as unified credit codes, organizational codes, ID numbers representing personal identity, mobile phone numbers and email addresses, etc., and verification rules for general identification data) and audit field rules; intelligent rules verify data content with the help of NLP technology through associating verification rules to determine the accuracy of the data.

[0058] S103, obtaining the audit data element to be evaluated of the target database, using a hash table to perform a pre-check on the database table of the audit data element to be evaluated, identifying the data type information and file structure information contained in the audit data element to be evaluated, and correspondingly marking the source attribute to which the data type information belongs.

[0059] In this embodiment, the audit data element involved in the present invention refers to a data element that serves as a basic unit in the target database.

[0060] In this embodiment, the source attribute involved in the present invention is that the audit data element is classified according to the line classification method given in 6.2 of "Basic Principles and Methods of Information Classification and Coding GB / T 7027-2002", that is, it is divided into several levels one by one according to the information source characteristics of the audit data element, and each level is divided into several categories. The same-level categories of the same branch form a parallel relationship, and the categories of different levels form a subordinate relationship.

[0061] It should be understood that there are a large number of audit data elements in the target database involved in the present invention, and the audit data element to be evaluated belongs to only one type.

[0062] In this embodiment, a hash table is used to perform a pre-check of the library table on the audit data element to be evaluated, identify the data type information and file structure information contained in the audit data element to be evaluated, and mark the source attribute to which the data type information belongs, including:

[0063] NLP technology is used to identify the data type and length of the table header items and the first five non-blank rows of the audit data element to be evaluated, and the data type information contained in the audit data element to be evaluated is obtained. OCR technology is used to perform structured recognition on the audit data element to be evaluated, and the file structure information contained in the audit data element to be evaluated is obtained. The identified content is then concatenated with the file name of the audit data element to be evaluated to obtain concatenated data. The concatenated data is then converted into character format and converted into a character vector or a string scalar. The MD5 algorithm is used to calculate the hash value, and the source attribute to which the data type information belongs is marked by matching and comparing with the hash value of the stored source attribute.

[0064] In this embodiment, NLP technology is used to process table header data to identify data type information of different types of audit data elements, and then OCR technology is used to perform structured recognition to obtain file structure information of different types of audit data elements. The data type information, file structure information and file name are then spliced ​​to obtain spliced ​​data, and the spliced ​​data is then converted to a character format that is convenient for MD5 algorithm operations, and then a hash value is calculated. The calculated hash value is matched and compared with the hash value of the stored source attribute to determine and mark the source attribute to which the data type information belongs.

[0065] S104. Load the audit data elements to be evaluated according to the marked source attributes, use a bidirectional long short-term memory network to perform named entity recognition on the audit data elements to be evaluated, obtain the named entity information of the audit data elements to be evaluated, and then perform format screening, duplicate checking and key field extraction on the audit data elements to be evaluated in turn to obtain key information.

[0066] In this embodiment, the audit data element to be evaluated is loaded according to the marked source attribute, and a bidirectional long short-term memory network is used to perform named entity recognition on the audit data element to be evaluated, so as to obtain the named entity information of the audit data element to be evaluated, including:

[0067] The audit data elements to be evaluated are loaded according to the marked source attributes, and data cleaning is performed on the audit data elements to obtain the cleaned named data information. A bidirectional long short-term memory network is used to perform named entity recognition on the cleaned named data information to obtain the named entity information of the audit data elements to be evaluated.

[0068] In this embodiment, the present invention adopts a bidirectional long short-term memory network (BiLSTM) to perform named entity recognition on the named data information of the audit data element, so as to obtain the named entity information of the audit data element to be evaluated, which is used for subsequent matching and comparison with the pre-rules in the audit data rule base.

[0069] In this embodiment, the audit data elements to be evaluated are subjected to format screening, duplication checking and key field extraction to obtain key information, wherein format screening may adopt Excel format screening technology, the purpose of which is to detect incomplete data, erroneous data and duplicate data, duplication checking may adopt hash table duplication checking method, the purpose of which is to detect duplicate data, key field extraction may adopt NLP technology to extract relevant key fields, and the results obtained by the three processing methods are summarized to form key information (for example: unified credit code, organization code, ID number representing personal identity, mobile phone number, email address, audit field information, government industry information, related information, etc.).

[0070] S105: Construct a data set based on source attributes, file structure information, named entity information and key information.

[0071] In this embodiment, source attributes, file structure information, named entity information and key information are integrated, and a data set with source attributes is specifically constructed.

[0072] S106. Based on the data quality modeling and analysis system, the data set is matched and compared with the corresponding types of rules in the audit data rule library to generate quality modeling and analysis results for each dimensional indicator.

[0073] In this embodiment, based on the data quality modeling and analysis system, the data set is matched and compared with the corresponding types of rules in the audit data rule library to generate the quality modeling and analysis results of each dimension indicator, including:

[0074] According to the marked source attributes, the data set is matched and compared with the corresponding types of rules in the audit data rule library. Based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, the error rate and score of the data information contained in the data set under the six dimensional indicators are calculated to form the quality modeling analysis results of each dimensional indicator.

[0075] In this embodiment, according to the source attributes of the audit data elements to be evaluated, the corresponding types of rules are matched and compared in the audit data rule library to obtain the data quality inspection results, and then based on the six indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, the number of qualified data items, the number of abnormal data items, the error rate and the score of the data information contained in the data set under the six dimensional indicators are calculated to form the quality modeling analysis results of each dimensional indicator.

[0076] In this embodiment, the specific steps of constructing the data quality modeling and analysis system of the present invention are:

[0077] According to GB / T36344-2018 Information Technology Data Quality Modeling and Analysis Indicators, standardization, completeness, accuracy, consistency, timeliness and traceability are used as the six dimensions of data quality modeling and analysis to construct a data quality modeling and analysis system (as shown in Table 1 below).

[0078] Table 1 Data quality modeling and analysis system

[0079]

[0080] Among them, the first-level indicators of quality modeling analysis are as follows:

[0081] Normativity: The degree to which data conforms to data standards, data models, business rules, metadata, or authoritative reference data.

[0082] Completeness: The degree to which data elements are assigned values ​​as required by data rules.

[0083] Accuracy: The degree to which data accurately represents the true value of the real entity (actual object) it describes.

[0084] Consistency: The degree to which data do not contradict other data used in a specific context.

[0085] Timeliness: The accuracy of data over time.

[0086] Traceability: The extent to which data can be accessed.

[0087] S107. Use the hierarchical analysis method to calculate the weight of each dimension indicator.

[0088] In this embodiment, the present invention adopts the hierarchical analysis method to determine the weight of each indicator. The basic idea of ​​the hierarchical analysis method is to transform the overall judgment of the weights of multiple dimensions that constitute a complex problem into a "two-by-two comparison" of these dimensions, and then transform it into a ranking judgment of the overall weights of these dimensions, and finally establish the weight of each dimension.

[0089] In this embodiment, the specific steps of using the hierarchical analysis method to calculate the weight of each dimension indicator are as follows:

[0090] (1) Based on each level of indicators in the data quality modeling and analysis system, a hierarchical structure of indicators is constructed.

[0091] (2) Based on the hierarchical structure of indicators, the relative importance of each level of indicators in the data quality modeling and analysis system is assigned, a judgment matrix is ​​constructed, and the judgment matrix is ​​calculated. The maximum eigenvalue and eigenvector of are normalized to obtain the ranking weight of the relative importance of the indicators at the same level to the indicators at the previous level.

[0092] (3) Hierarchical single sorting and its consistency test,

[0093] Due to the influence of various subjective and objective factors, it is difficult for the judgment matrix to be strictly consistent. Therefore, after obtaining the characteristic roots, the consistency of the judgment matrix needs to be tested.

[0094] In order to test the consistency of the judgment matrix, it is necessary to calculate its consistency index CI, which is defined as: , when CI=0, the judgment matrix has complete consistency. Among them, The larger the CI is, the worse the consistency of the judgment matrix is. In order to check whether the judgment matrix has satisfactory consistency, it is also necessary to compare the CI with the average random consistency index RI, where the value of the average random consistency index RI is as shown in Table 2 below.

[0095] Table 2 Values ​​of the average random consistency index RI

[0096]

[0097] Considering that the deviation from consistency may be caused by random reasons, when checking whether the judgment matrix has satisfactory consistency, it is also necessary to compare CI and random consistency index RI to obtain the test coefficient CR, the formula is: ; Generally, if , then the judgment matrix is ​​considered to have passed the consistency test, otherwise it does not have satisfactory consistency. In addition, the pairwise comparisons between specific dimensional indicators are generally obtained by survey interviews and expert consultation methods, and then the judgment matrix is ​​constructed according to the importance of each dimensional indicator for calculation.

[0098] S108. Based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, an audit data quality modeling and analysis model is constructed. The quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator are input into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results.

[0099] In this embodiment, the calculation formula of the audit data quality modeling analysis model is as follows:

[0100]

[0101] The constraints are: , Q is the audit data quality score, G is the quality modeling analysis result of the standardization dimension, W is the quality modeling analysis result of the integrity dimension, Z is the quality modeling analysis result of the accuracy dimension, Y is the quality modeling analysis result of the consistency dimension, S is the quality modeling analysis result of the timeliness dimension, and K is the quality modeling analysis result of the traceability dimension. is the weight of the normative dimension, is the weight of the completeness dimension indicator, is the weight of the accuracy dimension indicator, is the weight of the consistency dimension indicator, is the weight of the timeliness dimension indicator, It is the weight of the traceability dimension indicator.

[0102] The present embodiment provides an audit data quality modeling and analysis method, including: obtaining a data dictionary of a target database; constructing an audit data rule base based on the data dictionary of the target database and taking the audit standard as a reference; obtaining audit data elements to be evaluated of the target database, using a hash table to perform a pre-detection of the database table on the audit data elements to be evaluated, identifying the data type information and file structure information contained in the audit data elements to be evaluated, and correspondingly marking the source attributes to which the data type information belongs; loading the audit data elements to be evaluated according to the marked source attributes, using a bidirectional long short-term memory network to perform named entity recognition on the audit data elements to be evaluated, obtaining the named entity information of the audit data elements to be evaluated, and then sequentially performing format screening, duplicate checking, and key field extraction on the audit data elements to be evaluated to obtain key information; constructing a data set based on the source attributes, file structure information, named entity information, and key information; matching and comparing the data set with the corresponding types of rules in the audit data rule base based on the data quality modeling and analysis system, and generating the quality modeling and analysis results of each dimensional indicator; using the hierarchical analysis method to calculate the weight of each dimensional indicator; and , completeness, accuracy, consistency, timeliness and traceability, and construct an audit data quality modeling and analysis model. The quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator are input into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results. This implementation method is based on the audit data quality modeling and analysis model, combines NLP technology, OCR technology, MD5 algorithm and bidirectional long short-term memory network, and specifically constructs a data set with source attributes. According to the source attributes of the audit data element to be evaluated, the corresponding types of rules are matched and compared in the audit data rule library, and the quality modeling and analysis results of each dimensional indicator are analyzed from the six dimensions of standardization, completeness, accuracy, consistency, timeliness and traceability. According to the quality modeling and analysis results of each dimensional indicator, an overall evaluation is made on the audit data element to be evaluated, and then the audit data quality modeling and analysis results are obtained. The impact of factors such as the source attributes of the audit data element, file structure information, named entity information and key information on the data quality modeling and analysis is comprehensively considered, and the quality modeling and analysis effect of the audit data element is effectively improved.

[0103] Embodiment 2

[0104] This embodiment provides an audit data quality modeling and analysis device 200 based on the above embodiment 1. Figure 2, used to implement the steps of an audit data quality modeling and analysis method described in the above embodiment 1, the device 200 mainly includes: a data dictionary acquisition module 201, an audit data rule base construction module 202, an audit data element acquisition and pre-detection module 203, a named entity recognition module 204, a data processing module 205, a data set construction module 206, a matching comparison module 207, a weight calculation module 208, an audit data quality modeling and analysis model construction module 209 and a data quality modeling and analysis module 210, wherein,

[0105] A data dictionary acquisition module 201 is used to acquire a data dictionary of a target database;

[0106] The audit data rule base construction module 202 is used to construct the audit data rule base according to the data dictionary of the target database and with reference to the audit standard;

[0107] The audit data element acquisition and pre-detection module 203 is used to acquire the audit data element to be evaluated of the target database, use the hash table to perform pre-detection of the database table on the audit data element to be evaluated, identify the data type information and file structure information contained in the audit data element to be evaluated, and mark the source attribute to which the data type information belongs;

[0108] The named entity recognition module 204 is used to load the audit data element to be evaluated according to the marked source attributes, and use a bidirectional long short-term memory network to perform named entity recognition on the audit data element to be evaluated to obtain the named entity information of the audit data element to be evaluated;

[0109] The data processing module 205 is used to sequentially perform format screening, duplicate checking and key field extraction on the audit data elements to be evaluated to obtain key information;

[0110] A data set construction module 206, for constructing a data set based on source attributes, file structure information, named entity information and key information;

[0111] The matching and comparison module 207 is used to match and compare the data set with the corresponding types of rules in the audit data rule library based on the data quality modeling and analysis system, and generate the quality modeling and analysis results of each dimension indicator;

[0112] The weight calculation module 208 is used to calculate the weight of each dimension indicator by using the hierarchical analysis method;

[0113] The audit data quality modeling and analysis model building module 209 is used to build an audit data quality modeling and analysis model based on six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability.

[0114] The data quality modeling and analysis module 210 is used to input the quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results.

[0115] Embodiment 3

[0116] Based on the above embodiment 1, this embodiment further provides a computer device, which includes: a processor, and the processor is used to implement the steps of any of the above methods when executing a computer program stored in a memory.

[0117] Embodiment 4

[0118] The present invention also provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of any of the above methods are implemented. Exemplarily, the computer program can be divided into one or more modules / units, and the one or more modules / units are stored in a memory and executed by a processor to complete the present invention. One or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program in a computer device.

[0119] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, or other computer device. The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the computer device may include more or fewer components, or a combination of certain components, or different components. For example, the computer device may also include an input / output device, a network access device, a bus, and the like.

[0120] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0121] The memory may be an internal storage unit of a computer device, such as a hard disk or memory of a computer device. The memory may also be an external storage device of a computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device. Furthermore, the memory may include both an internal storage unit of a computer device and an external storage device. The memory is used to store computer programs and other programs and data required by the computer device. The memory may also be used to temporarily store data that has been output or is to be output.

[0122] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0123] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0125] In the embodiments provided by the present invention, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0128] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0129] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A method for audit data quality modeling and analysis, characterized in that: include: Get the data dictionary of the target database; According to the data dictionary of the target database and with reference to the audit standards, an audit data rule base is constructed; Obtain the audit data element to be evaluated of the target database, use a hash table to perform a pre-check on the database table of the audit data element to be evaluated, identify the data type information and file structure information contained in the audit data element to be evaluated, and mark the source attribute to which the data type information belongs accordingly; The audit data elements to be evaluated are loaded according to the marked source attributes, and the bidirectional long short-term memory network is used to perform named entity recognition on the audit data elements to be evaluated to obtain the named entity information of the audit data elements to be evaluated. Then, the audit data elements to be evaluated are format screened, duplicated, and key fields are extracted in turn to obtain key information. Construct data sets based on source attributes, file structure information, named entity information, and key information; Based on the data quality modeling and analysis system, the data set is matched and compared with the corresponding types of rules in the audit data rule library to generate the quality modeling and analysis results of each dimension indicator; The analytic hierarchy process is used to calculate the weight of each dimension indicator; Based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, an audit data quality modeling and analysis model is constructed. The quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator are input into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results.

2. The audit data quality modeling and analysis method according to claim 1 is characterized in that: The data dictionary includes: definition and description information of target database objects, attribute information of target database objects, association relationships of target database objects, target database users and authority information, and parameters and configuration information of the target database.

3. The audit data quality modeling and analysis method according to claim 1 is characterized in that: The audit data rule base includes pre-rules, basic rules, business rules and intelligent rules; the pre-rules include database naming rules and database structure rules, which are used to detect whether the database naming and database structure comply with the pre-specifications of audit data quality modeling and analysis; the basic rules include audit data format rules, which are used to detect format quality problems of audit data elements; the business rules include general rules and audit domain rules, which are used to detect whether the general identification data and domain data of audit data elements comply with the general rules and audit domain rules established by the audit standards; the intelligent rules include association verification rules, which are used to detect the degree of logical association matching between audit data and users and / or other data and / or business applications.

4. The audit data quality modeling and analysis method according to claim 1 is characterized in that: The method of using a hash table to perform a pre-check on the library table of the audit data element to be evaluated, identifying the data type information and file structure information contained in the audit data element to be evaluated, and correspondingly marking the source attribute to which the data type information belongs, includes: NLP technology is used to identify the data type and length of the table header items and the first five non-blank rows of the audit data element to be evaluated, and the data type information contained in the audit data element to be evaluated is obtained. OCR technology is used to perform structured recognition on the audit data element to be evaluated, and the file structure information contained in the audit data element to be evaluated is obtained. The identified content is then concatenated with the file name of the audit data element to be evaluated to obtain concatenated data. The concatenated data is then converted into character format and converted into a character vector or a string scalar. The MD5 algorithm is used to calculate the hash value, and the source attribute to which the data type information belongs is marked by matching and comparing with the hash value of the stored source attribute.

5. The audit data quality modeling and analysis method according to claim 1 is characterized in that: The method of loading the audit data element to be evaluated according to the marked source attributes, and using a bidirectional long short-term memory network to perform named entity recognition on the audit data element to be evaluated to obtain the named entity information of the audit data element to be evaluated includes: The audit data elements to be evaluated are loaded according to the marked source attributes, and data cleaning is performed on the audit data elements to obtain the cleaned named data information. A bidirectional long short-term memory network is used to perform named entity recognition on the cleaned named data information to obtain the named entity information of the audit data elements to be evaluated.

6. The audit data quality modeling and analysis method according to claim 1 is characterized in that: The data quality modeling and analysis system is based on matching and comparing the data set with the corresponding types of rules in the audit data rule library to generate the quality modeling and analysis results of each dimension indicator, including: According to the marked source attributes, the data set is matched and compared with the corresponding types of rules in the audit data rule library. Based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability, the error rate and score of the data information contained in the data set under the six dimensional indicators are calculated to form the quality modeling analysis results of each dimensional indicator.

7. The audit data quality modeling and analysis method according to claim 1 is characterized in that: The calculation formula of the audit data quality modeling analysis model is as follows: The constraints are: , Q is the audit data quality score, G is the quality modeling analysis result of the standardization dimension, W is the quality modeling analysis result of the integrity dimension, Z is the quality modeling analysis result of the accuracy dimension, Y is the quality modeling analysis result of the consistency dimension, S is the quality modeling analysis result of the timeliness dimension, and K is the quality modeling analysis result of the traceability dimension. is the weight of the normative dimension, is the weight of the completeness dimension indicator, is the weight of the accuracy dimension indicator, is the weight of the consistency dimension indicator, is the weight of the timeliness dimension indicator, It is the weight of the traceability dimension indicator.

8. An audit data quality modeling and analysis device, characterized in that: include: A data dictionary acquisition module is used to acquire the data dictionary of the target database; An audit data rule base construction module is used to construct an audit data rule base based on the data dictionary of the target database and with reference to the audit standards; The audit data element acquisition and pre-detection module is used to obtain the audit data element to be evaluated in the target database, use the hash table to perform pre-detection of the database table on the audit data element to be evaluated, identify the data type information and file structure information contained in the audit data element to be evaluated, and mark the source attribute to which the data type information belongs; A named entity recognition module is used to load the audit data element to be evaluated according to the marked source attributes, and use a bidirectional long short-term memory network to perform named entity recognition on the audit data element to be evaluated to obtain the named entity information of the audit data element to be evaluated; The data processing module is used to perform format screening, duplicate checking and key field extraction on the audit data elements to be evaluated in order to obtain key information; A data set construction module is used to construct a data set based on source attributes, file structure information, named entity information and key information; The matching and comparison module is used to match and compare the data set with the corresponding types of rules in the audit data rule library based on the data quality modeling and analysis system, and generate the quality modeling and analysis results of each dimension indicator; The weight calculation module is used to calculate the weight of each dimension indicator using the hierarchical analysis method; Audit data quality modeling and analysis model building module, which is used to build audit data quality modeling and analysis models based on the six dimensional indicators of standardization, completeness, accuracy, consistency, timeliness and traceability; The data quality modeling and analysis module is used to input the quality modeling and analysis results of each dimensional indicator and the weight of each dimensional indicator into the audit data quality modeling and analysis model to calculate the audit data quality modeling and analysis results.

9. A computer device, characterized in that: The computer device comprises: a processor, and the processor is used to implement the steps of any method according to claim 1-7 when executing a computer program stored in a memory.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed by a processor, the steps of any method as claimed in claim 1-7 are implemented.

Images