A data auditing method and device, electronic equipment and storage medium

Abstract

The application provides a data auditing method and device, electronic equipment and storage medium. The method comprises: obtaining a data flow graph in a data flow graph library, the data flow graph comprising at least one of a system data flow graph, a system library table structure graph and a system deployment graph, the system data flow graph comprising an inter-system data flow graph or a business line data flow graph, the system library table structure graph representing a data table of at least one business system matching database, and the system deployment graph representing a hardware and software deployment graph corresponding to the at least one business system; determining an auditing target in the at least one business system according to the data flow graph, and determining a preset auditing rule matching the auditing target; performing data auditing processing on the auditing target according to the preset auditing rule to obtain a data auditing result, the data auditing result being used to show abnormal data in the data flow graph and / or analysis results. The application can improve the effect of data auditing.

Description

Technical Field

[0001] This invention relates to the field of computer information management systems, and more particularly to a data auditing method, apparatus, device, and storage medium. Background Technology

[0002] Nowadays, business processes in many industries need to span multiple business systems. As businesses develop and systems operate for a long time, a large amount of business data is generated. However, inconsistencies in data standards and data between business systems can directly affect the accuracy and consistency of information.

[0003] In existing technologies, data issues are addressed through post-event data audits, which involve auditing offline data. This approach helps users identify data problems and determine whether there are any abnormal data in the target business data.

[0004] It is evident that existing technologies have the problem of poor data auditing effectiveness in business systems. Summary of the Invention

[0005] This invention provides a data auditing method, apparatus, electronic device, and storage medium to address the problem of poor data auditing performance in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a data auditing method, including:

[0007] Obtain data flow diagrams from the data flow diagram library. The data flow diagrams include at least one of the following: system data flow diagram, system database table structure diagram, and system deployment diagram. The system data flow diagram includes inter-system data flow diagrams or business line data flow diagrams. The system database table structure diagram represents the data tables of at least one business system matching the database. The system deployment diagram represents the hardware and software deployment diagram corresponding to at least one business system.

[0008] Based on the data flow diagram, determine the audit targets in the at least one business system, and determine the preset audit rules that match the audit targets;

[0009] The audit target is subjected to data audit processing according to the preset audit rules to obtain data audit results. The data audit results are used to display abnormal data and / or analysis results in the data flow diagram.

[0010] Optionally, before obtaining the data flow diagram matching at least one business system, the method further includes:

[0011] Obtain first operation information, which represents the user's selection of a basic element and the selection of the position of the basic element;

[0012] Based on the first operation information, second operation information is obtained. The second operation information is used to add elements corresponding to audit nodes between the basic elements. The audit nodes are used to configure audit rules.

[0013] A first data flow graph is created based on the first operation information and the second operation information, and the first data flow graph is added to the data flow graph library;

[0014] The basic elements include at least one of the following: nodes, edges, and groups, where nodes represent entities in the business system, edges represent associations between multiple nodes, and groups represent a set of similar nodes.

[0015] Optionally, the step of determining the audit target and the preset audit rules matching the audit target based on the data flow diagram includes at least one of the following:

[0016] When the data flow diagram is the inter-system data flow diagram, the first business entity in the first target business system is set as the audit target; the audit rule configured by the user in the first business entity is set as the preset audit rule, which is used to determine the existence of the business entity.

[0017] When the data flow diagram is the business line data flow diagram, the attribute of the second business entity in the second target business system is set as the audit target; the audit rule configured by the user in the second business entity is set as the preset audit rule, which is used to perform consistency judgment processing on the attribute values ​​of the second business entity.

[0018] When the data flow diagram is the system database structure diagram, the metadata information in the matching database of the third target business system is set as the audit target; the database audit rules set by the user are set as the preset audit rules, which are used to perform consistency judgment processing on the attributes of the metadata information;

[0019] When the data flow diagram is the system deployment diagram, the hardware and software architecture corresponding to the fourth target business system is set as the audit target; the deployment audit rules for user equipment are set as the preset audit rules, which are used to judge and process the execution status of hardware and software.

[0020] Wherein, the first target business system, the second target business system, the third target business system and the fourth target business system are business systems in at least one business system, the first business entity is any business entity in the first target business system, and the second business entity is any business entity in the second target business system.

[0021] Optionally, the step of performing data auditing processing on the audit target according to the preset auditing rules to generate data auditing results includes at least one of the following:

[0022] When the data flow diagram is the system's database table structure diagram, and the metadata information is subjected to data auditing processing, the full amount of metadata in the matching database of the business system is collected, and the full amount of metadata is subjected to consistency determination to obtain a first data audit result. The first data audit result is used to display abnormal data and / or analysis results in the data flow diagram.

[0023] When the data flow diagram is the business line data flow diagram, and data auditing processing is performed on the real-time data corresponding to the second business entity, incremental data between multiple business systems is collected, the incremental data is copied to the memory database, and the incremental data in the memory database is audited using a preset auditing model to obtain a second data auditing result. The second data auditing result is used to display abnormal data and / or analysis results in the data flow diagram.

[0024] When the data flow diagram is an inter-system data flow diagram or a business line data flow diagram, and offline data is subjected to data audit processing, an audit script is determined according to the audit rules. The audit script matches the audit rules. The target data of the matching database of the business system is queried according to the audit script, and an audit result file is generated. A third data audit result is generated based on the audit result file. The third data audit result is used to display abnormal data or analysis results in the data flow diagram.

[0025] Optionally, the first data audit result includes a first data audit sub-result or a second data audit sub-result;

[0026] When the data flow diagram is the system's database table structure diagram, and data auditing processing is performed on the metadata information, the full metadata in the matching database of the business system is collected, and a consistency determination is made on the full metadata to obtain a first data audit result, including:

[0027] A consistency determination is performed on the full metadata. If the full metadata meets the first preset condition, the first data audit sub-result is generated. The first data audit sub-result indicates that the full metadata does not contain abnormal data.

[0028] If the full metadata does not meet the first preset condition, the second data audit sub-result is generated, which indicates that the full metadata contains abnormal data.

[0029] The first preset condition indicates that the target attribute of the database table corresponding to the full metadata matches the preset target attribute, and the target attribute includes at least one of the following: database table structure, field type, and field length.

[0030] Optionally, after performing data audit processing on the audit target according to the preset audit rules and generating data audit results, the method further includes:

[0031] Obtain third operation information, which indicates that the user has modified the audit rules matched by the audit node in the data flow diagram;

[0032] The data flow graph is updated based on the third operation information.

[0033] Secondly, embodiments of the present invention provide a data auditing device, comprising:

[0034] The first acquisition module is used to acquire data flow diagrams in the data flow diagram library. The data flow diagrams include at least one of the following: system data flow diagram, system database table structure diagram, and system deployment diagram. The system data flow diagram includes inter-system data flow diagrams or business line data flow diagrams. The system database table structure diagram represents the data tables of at least one business system matching the database. The system deployment diagram represents the hardware and software deployment diagram corresponding to at least one business system.

[0035] The determination module is used to determine the audit targets in the at least one business system based on the data flow diagram, and to determine the preset audit rules that match the audit targets;

[0036] The processing module is used to perform data audit processing on the audit target according to the preset audit rules to obtain data audit results. The data audit results are used to display abnormal data and / or analysis results in the data flow diagram.

[0037] Optionally, the device further includes:

[0038] The second acquisition module is used to acquire first operation information, which represents the user's selection of basic elements and the selection of the position of basic elements.

[0039] The third acquisition module is used to acquire second operation information based on the first operation information. The second operation information is used to add elements corresponding to audit nodes between the basic elements. The audit nodes are used to configure audit rules.

[0040] A creation module is used to create a first data flow graph based on the first operation information and the second operation information, and add the first data flow graph to the data flow graph library;

[0041] The basic elements include at least one of the following: nodes, edges, and groups, where nodes represent entities in the business system, edges represent associations between multiple nodes, and groups represent a set of similar nodes.

[0042] Optionally, the determining module includes:

[0043] The first determining unit is configured to, when the data flow diagram is the inter-system data flow diagram, set the first business entity in the first target business system as the audit target; and set the audit rule configured by the user in the first business entity as the preset audit rule, wherein the preset audit rule is used to determine the existence of the business entity.

[0044] The second determining unit is used to, when the data flow diagram is the business line data flow diagram, set the attribute of the second business entity in the second target business system as the audit target; and set the audit rule configured by the user in the second business entity as the preset audit rule, wherein the preset audit rule is used to perform consistency judgment processing on the attribute values ​​of the second business entity.

[0045] The third determining unit is used to set the metadata information in the matching database of the third target business system as the audit target when the data flow diagram is the system database table structure diagram; and to set the database audit rules set by the user as the preset audit rules, wherein the preset audit rules are used to perform consistency judgment processing on the attributes of the metadata information.

[0046] The fourth determining unit is used to, when the data flow diagram is the system deployment diagram, set the hardware and software architecture corresponding to the fourth target business system as the audit target; and set the deployment audit rules of the user equipment as the preset audit rules, wherein the preset audit rules are used to judge and process the execution status of the hardware and software.

[0047] Wherein, the first target business system, the second target business system, the third target business system and the fourth target business system are business systems in at least one business system, the first business entity is any business entity in the first target business system, and the second business entity is any business entity in the second target business system.

[0048] Optionally, the processing module includes:

[0049] The first processing unit is configured to, when the data flow diagram is the system's database structure diagram and the metadata information is subjected to data audit processing, collect all metadata in the matching database of the business system, and perform consistency determination on the all metadata to obtain a first data audit result. The first data audit result is used to display abnormal data and / or analysis results in the data flow diagram.

[0050] The second processing unit is configured to, when the data flow diagram is the business line data flow diagram and data audit processing is performed on the real-time data corresponding to the second business entity, collect incremental data between multiple business systems, copy the incremental data to an in-memory database, and perform data audit processing on the incremental data in the in-memory database using a preset audit model to obtain a second data audit result. The second data audit result is used to display abnormal data and / or analysis results in the data flow diagram.

[0051] The third processing unit is configured to, when the data flow diagram is the inter-system data flow diagram or the business line data flow diagram, and offline data is subjected to data audit processing, determine an audit script based on the audit rules, wherein the audit script matches the audit rules, query the target data of the matching database of the business system based on the audit script, generate an audit result file, and generate a third data audit result based on the audit result file, wherein the third data audit result is used to display abnormal data or analysis results in the data flow diagram.

[0052] Optionally, the first data audit result includes a first data audit sub-result or a second data audit sub-result;

[0053] The first processing unit includes:

[0054] A consistency determination is performed on the full metadata. If the full metadata meets the first preset condition, the first data audit sub-result is generated. The first data audit sub-result indicates that the full metadata does not contain abnormal data.

[0055] If the full metadata does not meet the first preset condition, the second data audit sub-result is generated, which indicates that the full metadata contains abnormal data.

[0056] The first preset condition indicates that the target attribute of the database table corresponding to the full metadata matches the preset target attribute, and the target attribute includes at least one of the following: database table structure, field type, and field length.

[0057] Optionally, the device further includes:

[0058] The fourth acquisition module is used to acquire third operation information, which indicates that the user has modified the audit rules matched by the audit node in the data flow diagram;

[0059] An update module is used to update the data flow graph based on the third operation information.

[0060] Thirdly, embodiments of the present invention provide an electronic device, including:

[0061] At least one processor; and

[0062] A memory communicatively connected to the at least one processor; wherein,

[0063] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the data auditing method described in the first aspect.

[0064] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer instructions, comprising:

[0065] The computer instructions are used to cause the computer to execute the data auditing method according to the first aspect.

[0066] In this embodiment of the invention, data flow diagrams are first obtained from a data flow diagram library. Based on the business systems and processes involved, the data flow diagrams can be divided into system data flow diagrams, system database table structure diagrams, and system deployment diagrams. Then, based on the data flow diagrams, audit targets in at least one business system are determined, along with preset audit rules matching the audit targets. Finally, data audit processing is performed on the audit targets according to the preset audit rules matching the audit targets, and the data audit results are obtained. This method constructs a data audit system covering the entire business process, enabling supervision of all stages of data auditing before, during, and after the audit. This effectively ensures the integrity of data in the business chain, improves the quality of data auditing, helps users promptly identify data problems, and thus enhances the effectiveness of data auditing.

[0067] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart illustrating a data auditing method provided in an embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of a data audit portal provided in an embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram of an inter-system data flow diagram provided in an embodiment of the present invention;

[0072] Figure 4 This is a schematic diagram of a business line data flow diagram provided in an embodiment of the present invention;

[0073] Figure 5 This is a schematic diagram of a system library table structure provided in an embodiment of the present invention;

[0074] Figure 6 This is a schematic diagram of a system deployment topology provided in an embodiment of the present invention;

[0075] Figure 7 A schematic diagram of the structure of a data auditing device provided in an embodiment of the present invention;

[0076] Figure 8 A block diagram of an electronic device used to implement the data auditing method of the embodiments of the present invention. Detailed Implementation

[0077] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0078] In the embodiments of this invention, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0079] Please see Figure 1 , Figure 1 This is a flowchart illustrating a data auditing method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, it includes the following steps:

[0080] Step 101: Obtain the data flow diagrams from the data flow diagram library. The data flow diagrams include at least one of the following: system data flow diagram, system database table structure diagram, and system deployment diagram. The system data flow diagram includes inter-system data flow diagrams or business line data flow diagrams. The system database table structure diagram represents the data tables of at least one business system matching the database. The system deployment diagram represents the hardware and software deployment diagram corresponding to at least one business system.

[0081] Step 102: Determine the audit targets in the at least one business system based on the data flow diagram, and determine the preset audit rules that match the audit targets;

[0082] Step 103: Perform data audit processing on the audit target according to the preset audit rules to obtain data audit results. The data audit results are used to display abnormal data and / or analysis results in the data flow diagram.

[0083] In this embodiment of the invention, steps 101, 102, 103 and 104 of the above data auditing method can be executed by a computer or a virtual machine, and the present invention does not limit the execution of these steps.

[0084] Furthermore, the database involved in the above data auditing method can be any data warehouse used to store data, such as MySQL, Oracle, SQL Server, and SQLite databases, and this embodiment of the invention does not limit this.

[0085] In step 101, the data flow graph library may store multiple data flow graphs, wherein the data flow graphs may be created and generated by the user. That is, the method provided in this embodiment of the invention may be configured with a graph visualization engine, thereby providing basic graph visualization capabilities such as graph drawing, layout, analysis, interaction, and animation. Based on the graph visualization engine, users can efficiently create the data flow graphs.

[0086] It should be noted that the components of the data flow diagram mentioned above can include nodes, edges, and combos. Users can change the color scheme and combination of nodes, edges, and combos according to the actual business scenario and business process, thereby creating a graph visualization analysis application that is more in line with the business scenario or business system, i.e., the data flow diagram mentioned above.

[0087] In addition, nodes are the basic units that make up a graph and generally represent an entity. In the embodiments of this invention, nodes can represent business nodes, edges can represent the relationship between two nodes, and groups can be understood as node groupings, used to manage a group of similar nodes, such as a group of nodes with the same type or a group of nodes that are relatively close in location. These can be grouped into the same group, thereby effectively reducing visual interference to users.

[0088] It should be understood that users can also create data audit nodes based on the above data flow diagram, that is, display data audit nodes in the above data flow diagram, and select appropriate audit rules to check the data. In some optional implementations, the above data flow diagram can also display the number of audit rules that have been created and the number of missing audit rules, so as to provide users with more information about inter-system data audit.

[0089] In some alternative implementations, users can use the following interactive methods to draw and create the above data flow graph: The graph visualization engine can be configured with relevant interactive styles for users to use directly, such as clicking nodes, clicking edges, scaling the canvas, and dragging the canvas. The interactive styles can be further divided into point interactive styles, edge interactive styles, and group interactive styles.

[0090] Specifically, users can click on an entity or relationship to browse the corresponding data flow diagram at the next level, or click on the audit rule number to view the corresponding audit rules.

[0091] The aforementioned data flow diagram includes at least one of the following: a system data flow diagram, a system database table structure diagram, and a system deployment diagram. These can be understood as data flow diagrams at different levels. For example, the system data flow diagram represents the data flow between at least one business system; the system database table structure diagram represents the data tables of at least one business system matching the database; and the system deployment diagram represents the hardware and software deployment of at least one business system. Through this hierarchical data audit system, guided by the aforementioned data flow diagram, and vertically penetrating the data flow diagram, database table structure diagram, and system deployment diagram, end-to-end data auditing is achieved, improving the effectiveness of data auditing.

[0092] In step 102, the audit targets in at least one of the above business systems are determined based on the above data flow diagram, and the preset audit rules matching the above audit targets are determined. First, the type of the above data flow diagram can be determined. The audit targets, which can also be understood as audit objects, are determined based on the type of the above data flow diagram. Since the above preset audit rules are set by the user in advance, after the above audit targets are determined, data audit processing can be performed according to the above preset audit rules set by the user.

[0093] It should be noted that the above-mentioned preset audit rules are added by the user when creating or editing the above-mentioned data flow diagram. The above-mentioned preset audit rules match the above-mentioned audit targets. Therefore, in the embodiments of the present invention, the specific rules of the above-mentioned preset audit rules and the number of the above-mentioned preset audit rules are not limited.

[0094] In step 103, users can perform data audits on the business system before, during, and after the event. The data audit before the event can be understood as the data audit of the data stored in the database, that is, the data audit processing of metadata. The data audit during the event can be understood as the data audit processing of streaming data or real-time data. The data audit after the event can be understood as the data audit processing after the data is entered into the database.

[0095] The aforementioned data audit results can be used to display abnormal data and / or analysis results in the data flow diagram. For example, after a user performs a pre-audit of the business system, the resulting data audit report can be used to guide relevant managers in re-standardizing and rectifying management processes, thereby ensuring the effective implementation and application of data standards. Similarly, after a user performs a mid-audit of the business system, the resulting data audit results can be directly fed back to the corresponding business system. Relevant managers can then correct business data based on these results and intercept abnormal data, thus improving the efficiency and accuracy of the audit work. Furthermore, after a user performs a post-audit of the business system, the resulting data audit report can also be used to centrally display data quality. Relevant managers can then improve the business system based on the data quality, thereby enhancing the effectiveness of the business system's data audit.

[0096] In this implementation plan, data flow diagrams are first obtained from a data flow diagram library. Based on the business systems and processes involved, the data flow diagrams can be divided into system data flow diagrams, system database table structure diagrams, and system deployment diagrams. Then, based on the data flow diagrams, audit targets in at least one business system are identified, along with preset audit rules matching the audit targets. Finally, data audit processing is performed on the audit targets according to the preset audit rules matching the audit targets, and the data audit results are obtained. Through this method, a data audit system covering the entire business process is constructed to achieve supervision of all stages of data audit before, during, and after the audit, effectively ensuring the integrity of data in the business chain, improving the quality of data audit, helping users to discover data problems in a timely manner, and thus improving the effectiveness of data audit.

[0097] Additionally, in embodiments of the present invention, an interactive module for user use may be provided, such as a data audit portal; please refer to [link to relevant documentation]. Figure 2 , Figure 2This is a schematic diagram of a data audit portal. The data audit portal can open up the above-mentioned data flow diagram editing as a middleware service, as well as the service capabilities for pre-event, in-event, and post-event data audits, namely the service capabilities for metadata audit, real-time data audit, and offline data audit. The program interface implements the service call, and provides multi-level audit rule management, task scheduling, and audit rule sharing for various levels, providing different data audit service views and capabilities to complete a one-stop data audit service.

[0098] The data audit portal integrates the configuration and management of metadata audit, real-time data audit, and offline data audit, as well as data standards, into a single portal for convenient unified management and maintenance. Configuration and management include system management, user management, role management, data security management, and system parameter configuration. By setting up the data audit portal, pre-event, in-event, and post-event data audit service capabilities can be integrated, and metadata audit, real-time data audit, and offline data audit can be configured and managed in a unified manner.

[0099] Optionally, before obtaining the data flow diagram matching at least one business system, the method further includes:

[0100] Obtain first operation information, which represents the user's selection of a basic element and the selection of the position of the basic element;

[0101] Based on the first operation information, second operation information is obtained. The second operation information is used to add elements corresponding to audit nodes between the basic elements. The audit nodes are used to configure audit rules.

[0102] A first data flow graph is created based on the first operation information and the second operation information, and the first data flow graph is added to the data flow graph library;

[0103] The basic elements include at least one of the following: nodes, edges, and groups, where nodes represent entities in the business system, edges represent associations between multiple nodes, and groups represent a set of similar nodes.

[0104] In this implementation scheme, the user's first operation information can be obtained firstly. This first operation information can be understood as the user's determination of the system block diagram, as it represents the selection of basic elements and the selection of their positions. After the system block diagram is confirmed, the user's second operation information is obtained again. This second operation information represents the user adding preset audit rules to the system block diagram. Finally, based on the first and second operation information, the first data flow diagram is created and added to the data flow diagram library. Through this method, users can create data flow diagrams according to the addition of business systems, thereby achieving full-process control of the global business system, ensuring the integrity of data in the business chain, improving data quality, helping data managers to discover data problems in a timely manner, and thus improving the effectiveness of data auditing of the business system.

[0105] It should be noted that the components of the data flow diagram mentioned above can include nodes, edges, and groups. Users can change the color scheme and combination of nodes, edges, and groups according to the actual business scenario and business process, thereby creating a graph visualization analysis application that is more in line with the business scenario or business system, i.e., the data flow diagram mentioned above.

[0106] In addition, nodes are the basic units that make up a graph and generally represent an entity. In the embodiments of this invention, nodes can represent business nodes, edges can represent the relationship between two nodes, and groups can be understood as node groupings, used to manage a group of similar nodes, such as a group of nodes with the same type or a group of nodes that are relatively close in location. These can be grouped into the same group, thereby effectively reducing visual interference to users.

[0107] Optionally, the step of determining the audit target and the preset audit rules matching the audit target based on the data flow diagram includes at least one of the following:

[0108] When the data flow diagram is the inter-system data flow diagram, the first business entity in the first target business system is set as the audit target; the audit rule configured by the user in the first business entity is set as the preset audit rule, which is used to determine the existence of the business entity.

[0109] When the data flow diagram is the business line data flow diagram, the attribute of the second business entity in the second target business system is set as the audit target; the audit rule configured by the user in the second business entity is set as the preset audit rule, which is used to perform consistency judgment processing on the attribute values ​​of the second business entity.

[0110] When the data flow diagram is the system database structure diagram, the metadata information in the matching database of the third target business system is set as the audit target; the database audit rules set by the user are set as the preset audit rules, which are used to perform consistency judgment processing on the attributes of the metadata information;

[0111] When the data flow diagram is the system deployment diagram, the hardware and software architecture corresponding to the fourth target business system is set as the audit target; the deployment audit rules for user equipment are set as the preset audit rules, which are used to judge and process the execution status of hardware and software.

[0112] Wherein, the first target business system, the second target business system, the third target business system and the fourth target business system are business systems in at least one business system, the first business entity is any business entity in the first target business system, and the second business entity is any business entity in the second target business system.

[0113] In this implementation scheme, based on the different types of the aforementioned data flow diagrams, the corresponding audit targets and the preset audit rules corresponding to the audit targets are determined. It should be understood that the aforementioned data flow diagrams represent the logical functions, the logical flow of business data within the business system, and the process of logical transformation in a graphical way from the perspective of data transmission and processing. Through this method, using the aforementioned data flow diagrams as a guide, the data flow diagrams, database table structure diagrams, and system deployment diagrams are vertically penetrated, achieving end-to-end data auditing and improving the data auditing effect and capability of the business system.

[0114] As can be seen from the previous embodiments, the audit targets in at least one of the above business systems are determined based on the above data flow diagrams, and the preset audit rules that match the above audit targets are determined. When the above data flow diagrams are respectively the data flow diagram between the above systems, the business line data flow diagram, the system database table structure diagram and the system deployment diagram, the above audit targets and the corresponding preset audit rules are different. Please refer to Table 1.

[0115] Table 1

[0116] hierarchy Audit Relationship Audit Target L1 Inter-system The existence of key entities across systems remains consistent (such as site addresses, projects, etc.). L2 Inter-system / Intra-system The key attributes of the same entity should have consistent values ​​(such as amount, status, etc.). L3 System library table structure Metadata information for the same entity and the same attribute remains consistent (e.g., length, precision). L4 Global The rationality of software and hardware architecture and system deployment (such as interfaces, replicas, etc.).

[0117] The data flow diagrams described above can be divided into four levels: L1, L2, L3, and L4, which correspond to the inter-system data flow diagram, the business line data flow diagram, the system database table structure diagram, and the system deployment topology diagram, respectively.

[0118] If the above data flow diagram is an inter-system data flow diagram, please refer to [link / reference]. Figure 3 , Figure 3The figure shows a schematic diagram of an inter-system data flow diagram provided in an embodiment of the present invention. The rounded rectangles in the figure represent various business systems, the right-angled rectangles represent entities in the business systems, the circles set on the entities represent data audit rules that have been created, and the numbers inside the circles represent the number of data audit rules that have been created. The diamonds set on the entities represent missing data audit rules, and the numbers inside the diamonds represent the number of missing data audit rules.

[0119] In this context, the data flow diagram represents data auditing between business systems, such as a data audit diagram between the financial management system and the resource asset system, or a data audit diagram between the financial management system and the public database. At this level, the audit targets are the key entities between the various business systems, and the aforementioned preset audit rules are used to determine the existence of the business entities.

[0120] If the above data flow diagram is a business line data flow diagram, please refer to [link / reference]. Figure 4 , Figure 4 The figure shows a schematic diagram of a business line data flow diagram provided in an embodiment of the present invention. The closed rectangles in the figure represent business process nodes, the open rectangles represent business entities, the solid arrows represent business flows, the dashed arrows represent data flows, the circles connecting business entities represent data audit rules that have been created, and the numbers inside the circles represent the number of data audit rules that have been created. The diamonds connecting business entities represent missing data audit rules, and the numbers inside the diamonds represent the number of missing data audit rules.

[0121] The financial management system and the property electricity management system have business interactions, and the payment order (business entity) exists in both business systems. Therefore, users can set up data audit operations between the two payment orders to audit the corresponding data in the payment order, such as the key attributes of the payment order, and check whether the values ​​are consistent.

[0122] If the above data flow diagram is a system library table structure diagram, please refer to [link / reference]. Figure 5 , Figure 5 A schematic diagram of a system library table structure provided in an embodiment of the present invention is shown in the figure. Figure 5 This includes information from the databases corresponding to the financial system and the property electricity management system, respectively.

[0123] In this context, the data flow diagram above represents a data audit of the corresponding databases of the financial system and the property electricity management system, specifically checking whether the metadata of the payment slips in the database is consistent (such as length, precision, etc.).

[0124] If the above data flow diagram is a system deployment topology diagram, please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of a system deployment topology provided in an embodiment of the present invention. As shown in the figure, there is a global deployment of software and hardware. In this case, the above data flow diagram represents the checking of the rationality of the deployment of software and hardware according to the above-mentioned preset audit rules.

[0125] Optionally, the step of performing data audit processing on the audit target according to the preset audit rules to generate data audit results includes at least one of the following:

[0126] When the data flow diagram is the system's database table structure diagram, and the metadata information is subjected to data auditing processing, the full amount of metadata in the matching database of the business system is collected, and the full amount of metadata is subjected to consistency determination to obtain a first data audit result. The first data audit result is used to display abnormal data and / or analysis results in the data flow diagram.

[0127] When the data flow diagram is the business line data flow diagram, and data auditing processing is performed on the real-time data corresponding to the second business entity, incremental data between multiple business systems is collected, the incremental data is copied to the memory database, and the incremental data in the memory database is audited using a preset auditing model to obtain a second data auditing result. The second data auditing result is used to display abnormal data and / or analysis results in the data flow diagram.

[0128] When the data flow diagram is an inter-system data flow diagram or a business line data flow diagram, and offline data is subjected to data audit processing, an audit script is determined according to the audit rules. The audit script matches the audit rules. The target data of the matching database of the business system is queried according to the audit script, and an audit result file is generated. A third data audit result is generated based on the audit result file. The third data audit result is used to display abnormal data or analysis results in the data flow diagram.

[0129] The embodiments of this invention mainly include auditing of three types of data: metadata auditing, real-time data auditing, and offline data auditing. It should be understood that metadata auditing corresponds to pre-audit, real-time data auditing corresponds to in-process auditing, and offline data auditing corresponds to post-audit. These three aspects will be described below:

[0130] (1) Metadata audit

[0131] Metadata auditing is mainly based on dictionary standards to audit the consistency of metadata information in various business systems and data warehouses, check the implementation of data standards, and finally generate the first data audit result mentioned above, namely the metadata audit report, thereby improving the metadata quality of various business systems and data warehouses.

[0132] First, in a multi-source heterogeneous environment, a data acquisition adapter collects full metadata from various business systems and data warehouses, including descriptions of data entities (systems, databases, tables, and fields) and the logic involved in processing these data entities. The acquisition adapter is compatible with data sources in different formats and converts data dictionary information from different database types to meet metadata collection requirements, enabling the collection of mainstream database data dictionaries (Excel) or online direct acquisition.

[0133] After collecting metadata, two data auditing methods can be used: consistency auditing and model comparison. Consistency auditing audits the consistency of key attributes of the objects (database tables) maintained by the system, such as whether the table structure is consistent, whether the field types are consistent, and whether the field lengths are consistent. Model comparison compares the differences between the metadata of the warehouse dimension and the metadata of the actual business database, such as the absence of a data dictionary, the absence of instance objects, and inconsistent fields.

[0134] Finally, based on the aforementioned first data audit results, relevant management personnel can standardize management processes, rectify issues, and track related problems, thereby ensuring the effective implementation of data standards.

[0135] (2) Real-time data auditing

[0136] Real-time data auditing can be understood as copying information to an independent in-memory database in real time through a collector, and then comparing the data in real time using a real-time auditing model to intercept non-compliant data from entering the system.

[0137] First, the collector replicates data from multiple heterogeneous business systems to an in-memory database in real time. When data changes in a business system, the collector synchronizes incremental data to the in-memory database in real time, enabling real-time monitoring and maintenance of multiple heterogeneous databases.

[0138] It should be noted that the data collection method varies depending on the database type. For example: 1) MySQL data collection

[0139] Based on a MySQL database, data is replicated to an in-memory database in real time via a MySQL Binlog real-time data stream, achieving real-time data synchronization. This is achieved by disguising itself as a MySQL slave node and retrieving the Binlog from the master node in real time.

[0140] The data synchronization server is responsible for connecting to different MySQL instances and maintaining an event message queue for each instance to achieve real-time data synchronization.

[0141] Configure a backup MySQL instance in each instance, and periodically update the auxiliary table's heartbeat mechanism to detect the master database's liveness. The heartbeat mechanism inserts a heartbeat record every minute (configurable) into each extracted table and saves the sending time. This heartbeat table is also extracted and follows the entire process, actually following the same logic as the synchronized table. When a heartbeat packet is received, even if there is no data added, deleted, or modified, it can prove that the entire link is working. In a dual-master scenario, this mechanism can automatically select the working data source.

[0142] 2) Oracle Data Acquisition

[0143] Based on an Oracle database, data replication is performed using SLT, and data is obtained by the client creating log tables and triggers in the source system.

[0144] Next, based on the business audit scenario, a data audit model is embedded in the in-memory database, for example:

[0145] ① Centralized review and automatic verification model

[0146] Historical data issues affecting revenue arising from the transfer of fixed assets from operators during the establishment of integrated services will be centrally managed by the company headquarters. Specifically, monthly routine business operations such as lease initiation, maintenance deductions, and non-lump-sum generator power generation fees will be handled by branch offices; service termination and account adjustments will be centrally processed at headquarters; business changes (including retrospective processing) will be categorized based on frequency and impact on revenue, with different data categories authorized to different levels of business support departments and professional departments for review and approval before centralized processing. The centralized audit will be designed as a hierarchical audit system independent of the production process. By synchronizing information required for centralized auditing across various systems in real time, batch or order-by-order automatic audits will be conducted, and the reasons will be fed back, reducing manual auditing workload and improving the efficiency of centralized auditing.

[0147] ② Sales order bulk binding resource audit model

[0148] When creating and initiating sales orders, it's impossible to determine whether the product specifications in the Customer Relationship Management System (CRM) match the corresponding tower and data center resource specifications in the resource system. This can lead to incorrect product types and subsequent cost disputes. Furthermore, it's impossible to batch bind orders and check resource compatibility during actual operation. By calling a real-time audit service for verification, we assist users in batch operations and complete consistency checks, preventing subsequent changes that might be necessary upon discovery.

[0149] ③ Audit Model for Lease Return Orders in Transit

[0150] When an account manager initiates a lease cancellation due to site closure or other reasons, the lack of data exchange between the property management and sales systems prevents operators from promptly ascertaining whether related services and fees have been adjusted. Unilateral cancellation can lead to a mismatch between revenue and expenditure for that site. By utilizing an audit model to synchronize data between the property management and sales systems in real time and verify the in-transit services and fees related to the order's corresponding site, account managers can be promptly alerted to coordinate the completion or transfer of related business.

[0151] ④ Audit Model for Three Types of Key Documents in the Procurement-to-Payment Process

[0152] When materials are procured and received, the receiving slip and purchase order must match perfectly. Similarly, when payments are made, the settlement document and purchase order must also match perfectly. Without verifying project status and payment ratios, operators cannot verify payments for engineering projects, easily leading to risks in material management and payment. By invoking real-time auditing services, the consistency between purchase orders and receiving slips, purchase orders and settlement documents, and payment ratios can be checked in real time, providing operators with timely reminders for corrections.

[0153] By using a real-time data audit model to examine business data, the aforementioned second data audit result is generated. Abnormal data and audit results are then fed back to each business system in real time, allowing relevant management personnel to correct the business data and improve the efficiency and accuracy of the audit work.

[0154] It should be understood that embedding real-time auditing capabilities based on streaming computing into various business processes allows for the interception and alerting of abnormal business data, while simultaneously calculating the abnormal data interception rate to create warning signs at each business point. When the interception rate reaches a certain threshold, an alert is sent to the user, which helps business departments identify risks and problems at each stage and address them promptly, minimizing losses from abnormal data.

[0155] (3) Offline data audit

[0156] Offline data auditing can be understood as adhering to six dimensions of data auditing rules: completeness, uniqueness, timeliness, effectiveness, accuracy, and consistency. Data auditing is carried out through scheduled tasks to identify and rectify data quality issues in various business systems and data warehouses.

[0157] When a user creates an offline data audit rule and begins the audit, each audit method is executed sequentially. The execution of an audit method is essentially the execution of its audit script. If any data is retrieved by these audit methods, a corresponding audit result file (DAT file) will be generated in the Receive directory, along with an XML file of the same name as a readiness identifier for the result file. All these files will be stored in a folder within the Receive directory named after the audit date of this task.

[0158] The audit result collection program uses Quartz jobs for polling. Because the number of files to be collected varies each time, as does the amount of data contained, and other potential interference factors, the time taken for each job will differ. This means that when one job starts, the previous job may not have finished. To avoid resource contention between different job batches, the system adopts a single-job execution method. If the previous job has not finished when the current job starts, the current job automatically ends, and the next job only starts after the previous job has finished.

[0159] The audit result acquisition module collects the audit result files into the database. During the acquisition process, it performs simple summary operations on these audit result data and stores the detailed data and summary data in the result detail table and summary table, respectively. If an exception occurs during this process, a rollback operation is performed on the database tables to avoid incomplete data, and then the file is moved to the Error directory.

[0160] It should be noted that users can define audit rules for different business entities based on audit categories. Each audit rule is a simple description of quality issues from the perspective of the business entity and contains one or more information items. These information items are the specific objects to be audited for each business entity. In this way, we define specific audit methods based on different information items on each audit rule.

[0161] In this implementation scheme, based on the different types of the data flow diagram, the audit targets corresponding to the data flow diagram and the preset audit rules corresponding to the audit targets are determined. Through this method, the data flow diagram is used as a guide to vertically penetrate the data flow diagram, database table structure diagram and system deployment diagram, thereby realizing end-to-end data audit and improving the data audit effect and capability of the business system.

[0162] Optionally, the first data audit result includes a first data audit sub-result or a second data audit sub-result;

[0163] When the data flow diagram is the system's database table structure diagram, and data auditing processing is performed on the metadata information, the full metadata in the matching database of the business system is collected, and a consistency determination is made on the full metadata to obtain a first data audit result, including:

[0164] A consistency determination is performed on the full metadata. If the full metadata meets the first preset condition, the first data audit sub-result is generated. The first data audit sub-result indicates that the full metadata does not contain abnormal data.

[0165] If the full metadata does not meet the first preset condition, the second data audit sub-result is generated, which indicates that the full metadata contains abnormal data.

[0166] The first preset condition indicates that the target attribute of the database table corresponding to the full metadata matches the preset target attribute, and the target attribute includes at least one of the following: database table structure, field type, and field length.

[0167] In this implementation plan, consistency auditing is used to audit the consistency of key attributes of objects (database tables) maintained by business systems, such as whether the table structure, field types, and field lengths are consistent. This method can improve the accuracy of metadata auditing. On the other hand, by checking the implementation of data standards, the first data audit sub-result or the aforementioned second data audit sub-result is generated, thereby improving the metadata quality of various business systems and data warehouses.

[0168] It should be noted that relevant personnel can use the results of the second data audit to rectify the business system, track and modify abnormal data, thereby ensuring the effective implementation of data standards.

[0169] Optionally, after performing data audit processing on the audit target according to the preset audit rules and generating data audit results, the method further includes:

[0170] Obtain third operation information, which indicates that the user has modified the audit rules matched by the audit node in the data flow diagram;

[0171] The data flow graph is updated based on the third operation information.

[0172] In this implementation scheme, the user's third operation information is obtained. The third operation information can be understood as the user's modification operation on the data flow diagram in the data flow diagram library. When the business system operating environment or data changes, it is necessary to replace or modify the audit rules. This method improves the stability of business system data audit.

[0173] It should be noted that if the business process in the business system changes, the user can modify the basic elements corresponding to the business entities in the data flow diagram through the aforementioned third operation information. This embodiment of the invention does not limit this.

[0174] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a data auditing device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the data auditing device 700 includes:

[0175] The first acquisition module 701 is used to acquire data flow diagrams in the data flow diagram library. The data flow diagrams include at least one of the following: system data flow diagrams, system database table structure diagrams, and system deployment diagrams. The system data flow diagrams include inter-system data flow diagrams or business line data flow diagrams. The system database table structure diagrams represent data tables of at least one business system matching the database. The system deployment diagrams represent hardware and software deployment diagrams corresponding to at least one business system.

[0176] The determination module 702 is used to determine the audit targets in the at least one business system based on the data flow diagram, and to determine the preset audit rules that match the audit targets;

[0177] The processing module 703 is used to perform data audit processing on the audit target according to the preset audit rules to obtain data audit results. The data audit results are used to display abnormal data and / or analysis results in the data flow diagram.

[0178] Optionally, the device 700 also includes:

[0179] The second acquisition module is used to acquire first operation information, which represents the user's selection of basic elements and the selection of the position of basic elements.

[0180] The third acquisition module is used to acquire second operation information based on the first operation information. The second operation information is used to add elements corresponding to audit nodes between the basic elements. The audit nodes are used to configure audit rules.

[0181] A creation module is used to create a first data flow graph based on the first operation information and the second operation information, and add the first data flow graph to the data flow graph library;

[0182] The basic elements include at least one of the following: nodes, edges, and groups, where nodes represent entities in the business system, edges represent associations between multiple nodes, and groups represent a set of similar nodes.

[0183] Optionally, the determining module 702 includes:

[0184] The first determining unit is configured to, when the data flow diagram is the inter-system data flow diagram, set the first business entity in the first target business system as the audit target; and set the audit rule configured by the user in the first business entity as the preset audit rule, wherein the preset audit rule is used to determine the existence of the business entity.

[0185] The second determining unit is used to, when the data flow diagram is the business line data flow diagram, set the attribute of the second business entity in the second target business system as the audit target; and set the audit rule configured by the user in the second business entity as the preset audit rule, wherein the preset audit rule is used to perform consistency judgment processing on the attribute values ​​of the second business entity.

[0186] The third determining unit is used to set the metadata information in the matching database of the third target business system as the audit target when the data flow diagram is the system database table structure diagram; and to set the database audit rules set by the user as the preset audit rules, wherein the preset audit rules are used to perform consistency judgment processing on the attributes of the metadata information.

[0187] The fourth determining unit is used to, when the data flow diagram is the system deployment diagram, set the hardware and software architecture corresponding to the fourth target business system as the audit target; and set the deployment audit rules of the user equipment as the preset audit rules, wherein the preset audit rules are used to judge and process the execution status of the hardware and software.

[0188] Wherein, the first target business system, the second target business system, the third target business system and the fourth target business system are business systems in at least one business system, the first business entity is any business entity in the first target business system, and the second business entity is any business entity in the second target business system.

[0189] Optionally, the processing module 703 includes:

[0190] The first processing unit is configured to, when the data flow diagram is the system's database structure diagram and the metadata information is subjected to data audit processing, collect all metadata in the matching database of the business system, and perform consistency determination on the all metadata to obtain a first data audit result. The first data audit result is used to display abnormal data and / or analysis results in the data flow diagram.

[0191] The second processing unit is configured to, when the data flow diagram is the business line data flow diagram and data audit processing is performed on the real-time data corresponding to the second business entity, collect incremental data between multiple business systems, copy the incremental data to an in-memory database, and perform data audit processing on the incremental data in the in-memory database using a preset audit model to obtain a second data audit result. The second data audit result is used to display abnormal data and / or analysis results in the data flow diagram.

[0192] The third processing unit is configured to, when the data flow diagram is the inter-system data flow diagram or the business line data flow diagram, and offline data is subjected to data audit processing, determine an audit script based on the audit rules, wherein the audit script matches the audit rules, query the target data of the matching database of the business system based on the audit script, generate an audit result file, and generate a third data audit result based on the audit result file, wherein the third data audit result is used to display abnormal data or analysis results in the data flow diagram.

[0193] Optionally, the first data audit result includes a first data audit sub-result or a second data audit sub-result;

[0194] The first processing unit includes:

[0195] A consistency determination is performed on the full metadata. If the full metadata meets the first preset condition, the first data audit sub-result is generated. The first data audit sub-result indicates that the full metadata does not contain abnormal data.

[0196] If the full metadata does not meet the first preset condition, the second data audit sub-result is generated, which indicates that the full metadata contains abnormal data.

[0197] The first preset condition indicates that the target attribute of the database table corresponding to the full metadata matches the preset target attribute, and the target attribute includes at least one of the following: database table structure, field type, and field length.

[0198] Optionally, the device 700 also includes:

[0199] The fourth acquisition module is used to acquire third operation information, which indicates that the user has modified the audit rules matched by the audit node in the data flow diagram;

[0200] An update module is used to update the data flow graph based on the third operation information.

[0201] According to embodiments of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0202] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0203] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0204] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0205] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data auditing methods.

[0206] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0207] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0208] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0209] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0210] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0211] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0212] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0213] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data auditing method, characterized in that, include: Obtain data flow diagrams from the data flow diagram library. The data flow diagrams include at least one of the following: system data flow diagram, system database table structure diagram, and system deployment diagram. The system data flow diagram includes inter-system data flow diagrams or business line data flow diagrams. The system database table structure diagram represents the data tables of at least one business system matching the database. The system deployment diagram represents the hardware and software deployment diagram corresponding to at least one business system. Based on the data flow diagram, determine the audit targets in the at least one business system, and determine the preset audit rules that match the audit targets; The audit target is subjected to data audit processing according to the preset audit rules to obtain data audit results. The data audit results are used to display abnormal data and / or analysis results in the data flow diagram. The step of determining the audit target and the preset audit rules for matching the audit target based on the data flow diagram includes at least one of the following: When the data flow diagram is the inter-system data flow diagram, the first business entity in the first target business system is set as the audit target; the audit rule configured by the user in the first business entity is set as the preset audit rule, which is used to determine the existence of the business entity. When the data flow diagram is the business line data flow diagram, the attribute of the second business entity in the second target business system is set as the audit target; The audit rules configured by the user in the second business entity are set as the preset audit rules, which are used to perform consistency judgment processing on the attribute values ​​of the second business entity. When the data flow diagram is the system database structure diagram, the metadata information in the matching database of the third target business system is set as the audit target; the database audit rules set by the user are set as the preset audit rules, which are used to perform consistency judgment processing on the attributes of the metadata information; When the data flow diagram is the system deployment diagram, the hardware and software architecture corresponding to the fourth target business system is set as the audit target; The deployment audit rules for user equipment are set to the preset audit rules, which are used to judge and process the execution status of software and hardware. Wherein, the first target business system, the second target business system, the third target business system and the fourth target business system are business systems in at least one business system, the first business entity is any business entity in the first target business system, and the second business entity is any business entity in the second target business system; The step of performing data auditing processing on the audit target according to the preset auditing rules to generate data auditing results includes at least one of the following: When the data flow diagram is the system's database table structure diagram, and the metadata information is subjected to data auditing processing, the full amount of metadata in the matching database of the business system is collected, and the full amount of metadata is subjected to consistency determination to obtain a first data audit result. The first data audit result is used to display abnormal data and / or analysis results in the data flow diagram. When the data flow diagram is the business line data flow diagram, and data auditing processing is performed on the real-time data corresponding to the second business entity, incremental data between multiple business systems is collected, the incremental data is copied to the memory database, and the incremental data in the memory database is audited using a preset auditing model to obtain a second data auditing result. The second data auditing result is used to display abnormal data and / or analysis results in the data flow diagram. When the data flow diagram is an inter-system data flow diagram or a business line data flow diagram, and offline data is subjected to data audit processing, an audit script is determined according to the audit rules. The audit script matches the audit rules. The target data of the matching database of the business system is queried according to the audit script, and an audit result file is generated. A third data audit result is generated based on the audit result file. The third data audit result is used to display abnormal data or analysis results in the data flow diagram.

2. The data auditing method according to claim 1, characterized in that, Before obtaining a data flow graph matching at least one business system, the method further includes: Obtain first operation information, which represents the user's selection of basic elements and the selection of the position of basic elements; Based on the first operation information, second operation information is obtained. The second operation information is used to add elements corresponding to audit nodes between the basic elements. The audit nodes are used to configure audit rules. A first data flow graph is created based on the first operation information and the second operation information, and the first data flow graph is added to the data flow graph library; The basic elements include at least one of the following: nodes, edges, and groups, where nodes represent entities in the business system, edges represent associations between multiple nodes, and groups represent a set of similar nodes.

3. The data auditing method according to claim 1, characterized in that, The first data audit result includes either a first data audit sub-result or a second data audit sub-result; When the data flow diagram is the system's database table structure diagram, and data auditing processing is performed on the metadata information, the full metadata in the matching database of the business system is collected, and a consistency determination is made on the full metadata to obtain a first data audit result, including: A consistency determination is performed on the full metadata. If the full metadata meets the first preset condition, the first data audit sub-result is generated. The first data audit sub-result indicates that the full metadata does not contain abnormal data. If the full metadata does not meet the first preset condition, the second data audit sub-result is generated, which indicates that the full metadata contains abnormal data. Wherein, the first preset condition indicates that the target attribute of the database table corresponding to the full metadata matches the preset target attribute, and the target attribute includes at least one of the following: database table structure, field type and field length.

4. The data auditing method according to any one of claims 1 to 3, characterized in that, After performing data audit processing on the audit target according to the preset audit rules and generating data audit results, the method further includes: Obtain third operation information, which indicates that the user has modified the audit rules matched by the audit nodes in the data flow diagram; The data flow graph is updated based on the third operation information.

5. A data auditing device, characterized in that, include: The first acquisition module is used to acquire data flow diagrams in the data flow diagram library. The data flow diagrams include at least one of the following: system data flow diagram, system database table structure diagram, and system deployment diagram. The system data flow diagram includes inter-system data flow diagrams or business line data flow diagrams. The system database table structure diagram represents the data tables of at least one business system matching the database. The system deployment diagram represents the hardware and software deployment diagram corresponding to at least one business system. The determination module is used to determine the audit targets in the at least one business system based on the data flow diagram, and to determine the preset audit rules that match the audit targets; The processing module is used to perform data audit processing on the audit target according to the preset audit rules to obtain data audit results. The data audit results are used to display abnormal data and / or analysis results in the data flow diagram. The first determining unit is configured to set the first business entity in the first target business system as the audit target when the data flow diagram is the inter-system data flow diagram; The audit rules configured by the user in the first business entity are set as the preset audit rules, which are used to determine the existence of the business entity. The second determining unit is used to set the attribute of the second business entity in the second target business system as the audit target when the data flow diagram is the business line data flow diagram; The audit rules configured by the user in the second business entity are set as the preset audit rules, which are used to perform consistency judgment processing on the attribute values ​​of the second business entity. The third determining unit is used to set the metadata information in the matching database of the third target business system as the audit target when the data flow diagram is the system database table structure diagram; and to set the database audit rules set by the user as the preset audit rules, wherein the preset audit rules are used to perform consistency judgment processing on the attributes of the metadata information. The fourth determining unit is used to set the hardware and software architecture corresponding to the fourth target business system as the audit target when the data flow diagram is the system deployment diagram; The deployment audit rules for user equipment are set to the preset audit rules, which are used to judge and process the execution status of software and hardware. Wherein, the first target business system, the second target business system, the third target business system and the fourth target business system are business systems in at least one business system, the first business entity is any business entity in the first target business system, and the second business entity is any business entity in the second target business system; The first processing unit is configured to, when the data flow diagram is the system's database structure diagram and the metadata information is subjected to data audit processing, collect all metadata in the matching database of the business system, and perform consistency determination on the all metadata to obtain a first data audit result. The first data audit result is used to display abnormal data and / or analysis results in the data flow diagram. The second processing unit is configured to, when the data flow diagram is the business line data flow diagram and data audit processing is performed on the real-time data corresponding to the second business entity, collect incremental data between multiple business systems, copy the incremental data to an in-memory database, and perform data audit processing on the incremental data in the in-memory database using a preset audit model to obtain a second data audit result. The second data audit result is used to display abnormal data and / or analysis results in the data flow diagram. The third processing unit is configured to, when the data flow diagram is the inter-system data flow diagram or the business line data flow diagram, and offline data is subjected to data audit processing, determine an audit script based on the audit rules, wherein the audit script matches the audit rules, query the target data of the matching database of the business system based on the audit script, generate an audit result file, and generate a third data audit result based on the audit result file, wherein the third data audit result is used to display abnormal data or analysis results in the data flow diagram.

6. The data auditing device according to claim 5, characterized in that, The device further includes: The second acquisition module is used to acquire first operation information, which represents the user's selection of basic elements and the selection of the position of basic elements. The third acquisition module is used to acquire second operation information based on the first operation information. The second operation information is used to add elements corresponding to audit nodes between the basic elements. The audit nodes are used to configure audit rules. A creation module is used to create a first data flow graph based on the first operation information and the second operation information, and add the first data flow graph to the data flow graph library; The basic elements include at least one of the following: nodes, edges, and groups, where nodes represent entities in the business system, edges represent associations between multiple nodes, and groups represent a set of similar nodes.

7. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the data auditing method according to any one of claims 1 to 4.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the data auditing method according to any one of claims 1 to 4.

Images