Data source performance evaluation method and device, electronic equipment and storage medium

Abstract

The invention discloses a data source performance evaluation method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that to-be-inspected data uploaded by a plurality of data sources respectively are acquired, data inspection rules corresponding to the data sources are determined respectively, and the data inspection rules corresponding to different data sources are different; for each data source, based on a data inspection rule corresponding to the data source, performing data inspection on the to-be-inspected data of the data source to obtain a data inspection result of the data source; and based on the data inspection result of the data source, performing performance evaluation on the data source to obtain a performance evaluation result of the data source.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data source performance evaluation method, device, electronic device and storage medium. Background Art

[0002] Data auditing refers to the process of reviewing and verifying the data submitted by each data reporter on a unified management platform. Its purpose is to ensure the accuracy, completeness and compliance of the data. In the context of building a multi-cloud management platform, data auditing is particularly important because it is related to the normal operation of the entire data center and the reliability of data.

[0003] In the related art, the performance evaluation of the data source is usually achieved through a single data audit rule. Due to the identicalness of the data audit rules between different data sources, the accuracy of the data audit is not high, which leads to poor accuracy of the performance evaluation of the data source. Summary of the invention

[0004] In order to solve the technical problems existing in the related technologies, the embodiments of the present application provide a data source performance evaluation method, device, electronic device and computer-readable storage medium.

[0005] To achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a data source performance evaluation method, which is applied to an electronic device, and the method includes:

[0007] Acquire the data to be audited uploaded by multiple data sources, and determine the data audit rules corresponding to each data source, where different data sources correspond to different data audit rules;

[0008] For each of the data sources, based on the data audit rules corresponding to the data source, data audit is performed on the data to be audited of the data source to obtain the data audit result of the data source;

[0009] Based on the data audit result of the data source, a performance evaluation is performed on the data source to obtain a performance evaluation result of the data source.

[0010] In a second aspect, an embodiment of the present application provides a data source performance evaluation device, which is applied to an electronic device, including:

[0011] An acquisition module is used to acquire the data to be audited uploaded by multiple data sources, and to determine the data audit rules corresponding to each data source, where different data sources correspond to different data audit rules;

[0012] A data audit module is used to perform data audit on the data to be audited of each data source based on the data audit rules corresponding to the data source, and obtain the data audit result of the data source;

[0013] The performance evaluation module is used to perform performance evaluation on the data source based on the data audit result of the data source to obtain the performance evaluation result of the data source.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a first memory for storing a computer program that can be run on the processor;

[0015] Wherein, when the processor is used to run the computer program, it executes the steps of the performance evaluation method of the data source on the electronic device side described in the embodiment of the present application.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the performance evaluation method of the data source provided in the embodiment of the present application is implemented.

[0017] The data source performance evaluation method, device, electronic device, and computer-readable storage medium provided in the embodiments of the present application obtain the data to be audited uploaded by multiple data sources, and determine the data audit rules corresponding to each data source. For each data source, based on the data audit rules corresponding to the data source, the data to be audited of the data source is audited to obtain the data audit result of the data source. Based on the data audit result of the data source, the data source is performance evaluated to obtain the performance evaluation result of the data source. Since different data sources correspond to different data audit rules, customized data audits are implemented for different data sources, thereby effectively improving the accuracy of data audits. By performing performance evaluation on the data source based on the data audit result of the data source, the performance evaluation result of the data source is obtained, thereby effectively improving the accuracy of the performance evaluation of the data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The following is a flow chart of the performance evaluation method of the data source of the present application embodiment. Figure 1 ;

[0019] Figure 2 The following is a flow chart of the performance evaluation method of the data source of the present application embodiment. Figure 2 ;

[0020] Figure 3 A schematic diagram of the principle of a method for evaluating the performance of a data source according to an embodiment of the present application;

[0021] Figure 4A schematic diagram of the structure of a data source performance evaluation device according to an embodiment of the present application;

[0022] Figure 5 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0025] According to the overall strategic layout of the integrated big data center collaborative innovation system, more and more places are building multi-cloud management platforms to achieve integrated coordination and dispatch of computing power, network, and security resources. Building a multi-cloud management platform requires unified access and management of heterogeneous resources such as public clouds, private clouds, industry clouds, and intelligent computing in the target area. Each cloud vendor reports data to the unified management platform in accordance with the specifications, and the unified management platform audits the managed data.

[0026] Data audit is the only means to verify the correctness of the data reported by each reporting party. There are usually two ways of auditing: one is manual auditing, which is to manually check the correctness of the data after the data is managed; the other is program auditing, which is to verify each field of the reported data according to the established specifications. Program auditing is the current mainstream auditing method.

[0027] Although the existing means of auditing massive data can audit massive data, they have the following problems: low flexibility, inability to flexibly configure audit rules and audit scope for resource data in each resource pool. Inability to help data reporters improve data reporting quality from the root. Auditable data management scenarios are limited.

[0028] How to audit massive data efficiently, flexibly and adaptably, how to improve the quality of data reporting and how to adapt to various data management scenarios are the pain points that need to be solved urgently in the management of massive data in a multi-cloud environment.

[0029] The embodiments of the present application have improved the shortcomings of the existing audit scheme. The embodiments of the present application abstract the audit rules into a model, which contains information such as audit expressions, audit scopes, and audit time. The audit rule metadata with a relatively fixed storage structure is used in MySql to set the expression responsible for the rule engine execution, and then the plug-in rule engine is extended to realize the plug-in setting of special audit scenarios, and realize flexible configuration of audit rules and audit scopes; the issues audited by the reporter, the triggered audit rules, the unresolved issues, and the total number of resources are aggregated, and an accuracy calculation formula is formulated to assess the accuracy of the data reported by the reporter, and a visual data display system is provided for the reporter to help the reporter improve the reporting quality; a unified import entry is provided for the management of multiple data, and the imported data is aggregated and the accuracy is calculated, and the four management scenarios of configuration data, performance data, alarm data, and capacity quota data are adapted.

[0030] Based on this, an embodiment of the present application provides a data source performance evaluation method, which is applied to an electronic device. Figure 1 The following is a flow chart of the performance evaluation method of the data source of the present application embodiment. Figure 1 ,like Figure 1 As shown, the performance evaluation method of the data source provided in the embodiment of the present application can be Figure 1 Steps 101 to 104 are shown to be implemented.

[0031] In step 101, data to be audited uploaded by multiple data sources are obtained.

[0032] In some embodiments, obtaining the data to be audited uploaded by multiple data sources refers to collecting data sets to be reviewed or audited from different data sources, which may include databases, file systems, external APIs, log files, etc. It is necessary to identify all relevant data sources, which includes understanding the type, storage format, access method and security requirements of each data source. According to the characteristics of the data source, develop or configure the corresponding access program or service so that data can be extracted from each data source. Perform data extraction operations on each data source, which may involve writing SQL queries, calling API interfaces, reading files or listening to message queues, etc. The extracted data should include all key information to be audited, such as transaction records, user behavior logs, system event records, etc. Since the data sources may use different formats or data standards, it is necessary to clean and convert the data to ensure that all data is in a unified format and structure. Before uploading the data, the integrity and accuracy of the data need to be verified. This may involve checking the consistency of the data, eliminating duplicate records, identifying outliers, etc. Data verification helps to ensure the quality of the data to be audited, thereby improving the efficiency and effectiveness of the audit. Once the data is cleaned and verified, it will be uploaded to the audit system. This system may be a dedicated data warehouse or an application with audit functions. The uploaded data needs to be stored securely and be able to support subsequent query and analysis operations.

[0033] In step 102, data audit rules corresponding to each of the data sources are determined respectively.

[0034] In some embodiments, different data sources correspond to different data audit rules.

[0035] In some embodiments, determining the data audit rules corresponding to each data source is a key process involving data governance and risk management. Different data sources may have different data structures, uses, and audit requirements, so specific audit rules need to be formulated for each data source. An in-depth understanding of each data source is required, including the source of the data, data content, data format, data update frequency, data importance and sensitivity, etc. Analyze the usage scenarios of the data and determine which data elements are critical for business decisions or regulatory compliance. Formulate audit rules based on the characteristics of the data source and business needs. These rules should comply with the following principles: Comprehensiveness: The rules should cover all key aspects of the data source to ensure that no important audit points are missed. Targetedness: The rules should target the specific characteristics of the data source and reflect the specific uses and risk points of the data source in the business. Operability: The rules should be able to be effectively implemented and easy to understand and apply. Compliance: The rules should comply with relevant laws, regulations and industry standards.

[0036] In some embodiments, examples of audit rules for the same data source are: Database: For database data sources, audit rules may include frequency of access to specific fields, data change records, data integrity and consistency checks, etc. File system: For file systems, audit rules may focus on file creation, modification, and access time, file permission settings, and compliance checks on file content. External API: For external APIs, audit rules may include API call frequency, response time, data format verification, and whether API usage complies with the terms of service, etc. Log files: For log files, audit rules may focus on log generation time, log level, exception records, and integrity of log content, etc.

[0037] In some embodiments, the audit rules are converted into executable programs or scripts that can automatically extract data from the data source and perform audit operations. Ensure that the implementation of the audit rules does not have a negative impact on the normal operation of the data source. Regularly monitor the effectiveness of the audit rules to ensure that they can detect potential problems in a timely manner. As the business develops or laws and regulations change, the audit rules are updated in a timely manner to adapt to new requirements.

[0038] In some embodiments, see Figure 2 , Figure 2 This is a schematic diagram of the process of the data source performance evaluation method provided in the embodiment of the present application. Figure 1 , Figure 1 The step 102 shown in FIG. 1 can be performed by Figure 2 Steps 1021 to 1023 shown in are implemented.

[0039] In step 1021, a preset audit rule library is obtained, and the following steps 1022 to 1023 are respectively performed for each of the data sources.

[0040] In some embodiments, obtaining a preset audit rule base and analyzing the mapping relationship between the preset audit rules and the preset audit scope is a key process involving data management and compliance monitoring. The audit rule base is a collection of preset audit rules that define how to perform audit tasks on data, including the conditions, parameters, frequency, and expected results of the audit. The audit rule base may be a database, a configuration file, or a dedicated rule management system that allows organizations to centrally manage audit rules. Preset audit rules are sets of rules that are predefined in the audit rule base for automatically performing audit tasks. These rules can be based on business logic, compliance requirements, or industry standards, such as checking the accuracy, completeness, consistency, or legality of data. The preset audit scope defines the data set or data type to which the audit rule should be applied, and the scope may include specific database tables, file types, API interfaces, or system logs, etc.

[0041] In some embodiments, the second mapping relationship refers to a logical relationship that associates a preset audit rule with a preset audit scope. This mapping relationship ensures that the audit rule can be correctly applied to the corresponding data source, thereby improving the pertinence and efficiency of the audit. Obtaining an audit rule base typically involves retrieving rule data from a rule base management system or storage medium. This may include querying a database, reading a configuration file, or calling an API interface to obtain audit rules and mapping relationships. Once the audit rule base is obtained, the audit system can automatically apply the corresponding audit rules according to the preset audit scope. The data within the specified range will be audited, and the audit results will be recorded, including any anomalies, violations, or potential risks found.

[0042] As an example, a financial institution's audit rule base is stored in a dedicated rule management system, which contains a series of preset audit rules. For example, there is a rule that "all transactions over $10,000 must be marked and subject to additional review", and another rule is "any abnormal transaction patterns, such as multiple small transactions in a short period of time, must be monitored". Rules are formulated based on AML regulations and industry best practices to automatically identify and report suspicious transactions. Each rule defines the conditions for the audit (such as transaction amounts exceeding $10,000), parameters (such as specific transaction types or accounts), and expected results (such as generating warnings or reports). The audit scope may include all customers' transaction record databases, electronic banking logs, and API interfaces with third-party payment systems. The scope defines the data set to which the audit rule should be applied, ensuring that the audit task covers all relevant data sources. In the audit rule base, each preset audit rule has a preset audit scope associated with it. For example, the "transactions over $10,000" rule is mapped to the "transaction record database for all customers", and the "abnormal transaction pattern" rule is mapped to the "electronic banking log" and "API interface of the third-party payment system".

[0043] In step 1022, the data audit scope corresponding to the data source is determined.

[0044] In some embodiments, the above-mentioned step 1022 can be implemented as follows: obtain a first mapping relationship between a preset audit scope and a preset data source, and determine a preset data source that is the same as the data source from the first mapping relationship; determine the preset audit scope corresponding to the preset data source that is the same as the data source in the first mapping relationship as the data audit scope corresponding to the data source.

[0045] In some embodiments, the process of obtaining a first mapping relationship between a preset audit scope and a preset data source and determining a preset data source that is the same as the data source is to ensure the accuracy and comprehensiveness of the data audit activity. In an organization's data management system, there is usually a mapping relationship between the preset audit scope and the data source. This mapping relationship defines which audit scopes should be applied to which data sources to ensure that the audit task covers all relevant data. When a new data source is introduced into the system, it is first necessary to determine which preset data source this data source is the same or similar to in the system. By analyzing the structure, content, purpose and sensitivity of the data source to determine whether it belongs to a defined data source category, once the preset data source that is the same as the data source is determined, the audit scope corresponding to this preset data source can be found from the first mapping relationship. This audit scope contains specific audit rules and tasks for this data source. According to the determined audit scope, the corresponding data audit task is performed, which may include running a predefined audit script, analyzing data patterns, checking data integrity or verifying data compliance.

[0046] In some embodiments, the audit results, including any anomalies or issues found, are collected and a report is generated. Based on the audit results, the organization can take corrective actions, such as updating data, notifying relevant personnel, or modifying business processes. Ensure the applicability of audit rules and scope. As data sources change or business needs evolve, mapping relationships and audit scopes may need to be updated to adapt to new situations.

[0047] As an example, suppose an e-commerce company has a data audit system that monitors order data and payment data to ensure the accuracy and compliance of transaction records. In the company's audit system, there is a mapping relationship between preset audit scopes and preset data sources. For example, the mapping relationship may define that the "order data" audit scope should be applied to the "order database" data source, and the "payment data" audit scope should be applied to the "payment processor API" data source. The company decides to introduce a new data source, the "customer relationship management system" (CRM) database, which is used to store customer contact information and purchase preferences. Through analysis, the company determines that the "CRM database" and the "order database" have similarities in structure and purpose, so the "CRM database" can be regarded as the same preset data source as the "order database". According to the mapping relationship, the preset data source that is the same as the "order database" corresponds to the "order data" audit scope. Therefore, the "CRM database" will also be applied to the "order data" audit scope, which means that the same audit rules will be used to check the data in the CRM database. The audit system performs audit tasks for the "CRM database", including checking the integrity and consistency of the data, and verifying the accuracy of customer information. Audit systems might automatically run scripts to check for unusual patterns in the data, such as duplicate customer records or contact details that don’t match regulations.

[0048] In this way, obtaining the first mapping relationship between the preset audit scope and the preset data source, and determining the preset data source that is the same as the data source based on this mapping relationship, and determining the corresponding preset audit scope for the data source, helps to ensure the pertinence and comprehensiveness of data audit activities. The existence and use of this mapping relationship enables organizations to quickly identify which audit rules apply to which data sources, thereby avoiding missing any key data points and improving the efficiency and effectiveness of data audits. By accurately matching the preset audit scope with the data source, organizations can ensure that audit activities not only meet internal business needs, but also meet external compliance requirements, reduce data risks, and improve data accuracy and quality.

[0049] In step 1023, based on the data audit scope corresponding to the data source, a preset audit rule matching the data source is selected from the preset audit rule library as the data audit rule corresponding to the data source.

[0050] In some embodiments, the preset audit rule library includes a second mapping relationship between multiple preset audit rules and preset audit scopes.

[0051] In some embodiments, the data audit scope refers to the data set or data type that the audit activity should cover, which may include specific database tables, file types, API interfaces or system logs, etc. The definition of the scope depends on the business importance, compliance requirements and risk level of the data. The preset audit rule base is a collection of pre-defined audit rules that are used to guide the execution of audit activities. These rules may include checking data integrity, identifying abnormal patterns, verifying compliance, etc. The second mapping relationship exists in the preset audit rule base, which defines the corresponding relationship between each audit rule and the audit scope. This mapping relationship ensures that each audit scope can apply appropriate audit rules, thereby improving the pertinence and effectiveness of the audit activities. According to the data audit scope corresponding to the data source, select those preset audit rules that match the scope from the preset audit rule base. For example, if the data audit scope is "all customer transaction records", the corresponding audit rules may include "checking whether the transaction amount exceeds the legal limit" or "verifying the consistency of transaction records". Apply the selected audit rules to the corresponding data source to automatically or manually perform the audit task.

[0052] In some embodiments, the above-mentioned step 1023 can be implemented in the following manner: from the preset audit scope of the second mapping relationship, the preset audit scope that is the same as the data audit scope corresponding to the data source is determined as the target audit scope; the preset audit rule corresponding to the target audit scope in the second mapping relationship is determined as the data audit rule corresponding to the data source.

[0053] In some embodiments, the second mapping relationship is defined in a preset audit rule base, which associates the preset audit scope with a specific audit rule. For example, the preset audit scope "payment data" may be associated with the audit rule "check whether the payment transaction is successfully completed". Based on the data audit scope corresponding to the data source, the preset audit scope identical to it is identified from the second mapping relationship. For example, if the data source is "payment database", then the target audit scope may be "payment data". Once the target audit scope is determined, the preset audit rule corresponding to the scope can be found from the second mapping relationship. For example, the preset audit rule corresponding to the target audit scope "payment data" may be "check whether the payment transaction is successfully completed". The determined data audit rule is applied to the corresponding data source to perform the audit task. The audit process may include real-time monitoring of payment data, regular inspections, or retrospective analysis of historical data. The audit system records the audit results, including the anomalies, errors, or potential risks found. The audit report usually provides detailed information such as the audit time, data source, rules, anomaly description, and possible solutions. Based on the audit findings, the organization can take corrective actions such as fixing data errors, updating business processes, or enhancing system controls.

[0054] As an example, assume that an online retail company uses a data audit system to monitor data in the inventory management system. The company has defined multiple preset audit scopes and corresponding audit rules, and established a second mapping relationship to associate these rules with specific data sources. The company defines the "Inventory Quantity" audit scope and creates corresponding audit rules, such as: Rule 1: Check whether the inventory quantity is zero to avoid out-of-stock. Rule 2: Verify whether the inventory quantity is consistent with the sales record to ensure data accuracy. The second mapping relationship associates the "Inventory Quantity" audit scope with the "Inventory Database" data source, indicating that these audit rules should be applied to the "Inventory Database". The company decides to introduce a new data source, namely "Supplier Delivery Log", to record supplier delivery status. Through analysis, the company determines that the "Supplier Delivery Log" is related to the "Inventory Quantity" in business logic, so the "Inventory Quantity" audit scope is determined as the target audit scope. According to the second mapping relationship, the company maps the "Supplier Delivery Log" to the "Inventory Quantity" audit scope, so the data audit rules corresponding to the "Supplier Delivery Log" are the audit rules corresponding to the "Inventory Quantity" audit scope, namely Rule 1 and Rule 2. The audit system applies rules 1 and 2 to the "Supplier Delivery Log" to perform audit tasks. For example, the audit system checks the update of inventory quantity after each supplier delivery, ensures that the zero quantity is recorded and reported, and verifies whether the inventory quantity is consistent with the sales record. The audit system generates a report listing any inconsistent or abnormal delivery records and provides it to the inventory management team. The team can take action based on the report, such as contacting the supplier to confirm the delivery or adjusting the inventory records.

[0055] In this way, by determining the target audit scope from the preset audit scope in the second mapping relationship, and determining the preset audit rules corresponding to the target audit scope as the data audit rules of the data source, the effectiveness and pertinence of data audit activities can be ensured. This practice helps to improve data quality, enhance compliance, and reduce potential business risks. By accurately matching the rules in the preset audit rule library with specific data sources, organizations can ensure that audit activities not only meet internal business needs, but also meet external compliance requirements. This will help ensure that data audit activities can effectively discover and solve problems while ensuring that data meets internal and external requirements.

[0056] In step 103, for each of the data sources, based on the data audit rules corresponding to the data source, a data audit is performed on the data to be audited of the data source to obtain a data audit result of the data source.

[0057] In some embodiments, the data audit rules corresponding to the above-mentioned data source include multiple sub-data audit rules, and the above-mentioned step 103 can be implemented in the following manner: for each of the sub-data audit rules, based on the sub-data audit rules, a data audit is performed on the data to be audited of the data source to obtain the sub-data audit results of the data source; and the sub-data audit results are merged to obtain the data audit results of the data source.

[0058] In some embodiments, the sub-data audit rules refer to specific audit standards and processes formulated for specific types or parts of data in the data source. These rules may include requirements for the format, scope, type, integrity, consistency, etc. of the data. Each sub-data audit rule targets a subset of the data source, for example, a specific data field or data table. According to each sub-data audit rule, the data to be audited in the data source is checked, the data to be audited is extracted from the data source, and the sub-data audit rules are applied to the extracted data, which may involve data verification, comparison, calculation and other operations, identify data items that do not comply with the audit rules, and record corresponding error information. After the audit process, each sub-data of the data source will generate an audit result. This result usually includes the conclusion of the audit (such as whether the data is compliant), error list, error description and other information. The audit results of each sub-data are integrated, and the audit results of all sub-data are summarized into a general audit report. If conflicts or repeated errors are found in different sub-data audit results, these conflicts need to be resolved to ensure that the final audit results are consistent, and the integrated data are analyzed as a whole to evaluate the overall quality and compliance of the data source.

[0059] As an example, a data source, which contains user data, order data, and product data of an e-commerce platform, checks the format of user data (such as the format of the email address), the uniqueness of the user ID, the rationality of the registration date, etc. Verify whether the order number is unique, whether the order date is within a reasonable range, whether the order amount is within the expected range, etc. Check the uniqueness of the product number, the completeness of the product description, whether the price is a positive number, etc. Based on the user data audit rules, check the user data. For example, it is found that some users' email addresses are in an incorrect format and some user IDs are repeated. Apply the order data audit rules to audit the order data. For example, it is found that the amounts of some orders are abnormally high, and there may be fraud. According to the product data audit rules, audit the product data. For example, it is found that the prices of some products are set to negative numbers, which is unreasonable. User data audit results: record the specific entries and quantities of incorrect email formats and repeated user IDs. Order data audit results: record the list of orders with abnormal amounts and related details. Product data audit results: record the list of products with incorrect price settings. Summarize the audit results of user data, order data, and product data into a comprehensive report. This report may include the following: The total number and type of errors. The distribution of errors for each sub-dataset. Cross-check results, for example, whether a certain order is associated with a valid user and product. The final data audit result is a comprehensive report that summarizes the quality of the entire data source. For example, the report may show: 5% of the email addresses in the user data have incorrect format and 1% of the user IDs are repeated. 0.1% of the order amounts in the order data are abnormal. 2% of the product prices in the product data are set incorrectly.

[0060] In this way, the accuracy and integrity of the data are ensured, and the overall quality and reliability of the data are improved by identifying and repairing sub-data that do not comply with the rules. The integration of the audit results of each sub-data to obtain a comprehensive data audit report not only helps to find problems in a timely manner and take corrective measures, but also enhances data governance capabilities and improves the security of data use. Such process optimization helps reduce operational risks, improve decision-making efficiency, and ultimately provide solid data support for the sustainable development of the enterprise.

[0061] In some embodiments, the above-mentioned sub-data audit rule carries standard data corresponding to the sub-data audit rule. The above-mentioned data audit is performed on the data to be audited of the data source based on the sub-data audit rule to obtain the sub-data audit result of the data source. It can be achieved in the following way: compare the standard data carried by the sub-data audit rule with the data to be audited to obtain a comparison result; when the comparison result indicates that the standard data does not exist in the data to be audited, determine the sub-data audit result as the data source does not pass the data audit of the sub-data audit rule; when the comparison result indicates that the standard data exists in the data to be audited, determine the sub-data audit result as the data source passes the data audit of the sub-data audit rule.

[0062] In some embodiments, during the data audit process, the standard data (i.e., the expected, correct data sample or template) carried by the audit rules is compared with the data to be audited. This comparison is to verify whether the data to be audited meets the predetermined standards and requirements. If the comparison result shows that the key information or fields in the standard data are missing in the data to be audited, this means that the data to be audited does not meet the requirements of the audit rules. For example, if the audit rule requires that all user records must have a valid email address, and some records in the data to be audited do not have an email address, then these records have not passed the audit. If the comparison result shows that the data to be audited contains information about all the standard data, and this information is correct and complete, this indicates that the data to be audited meets the requirements of the audit rules, and therefore can be considered to have passed the audit. When there is no standard data in the data to be audited, the sub-data audit result is determined to be unsuccessful. This usually means that there are quality problems with the data source or it does not meet the specified standards, and further investigation and repair are required. When there is standard data in the data to be audited, the sub-data audit result is determined to be passed. This indicates that the data source performs well at least in this audit and meets the predetermined data quality standards.

[0063] In this way, it is possible to clearly identify whether the data to be audited meets the established standards and requirements. When the comparison results indicate that the data to be audited is missing standard data, it is promptly marked as failing the audit, which helps to quickly discover and locate data quality problems, so that measures can be taken to repair them and ensure the accuracy and integrity of the data. On the contrary, when the comparison results show that the data to be audited contains standard data, that is, the data source has passed the audit, this can not only confirm the compliance of the data, but also enhance confidence in the data quality, improve the efficiency of data use and the quality of corporate decision-making. Overall, this audit mechanism significantly reduces data-related risks, optimizes data processes, and provides reliable data support for the company's stable operation and sustainable development.

[0064] In some embodiments, the data to be audited includes multiple sub-data to be audited. The above-mentioned comparison of the standard data carried by the sub-data audit rule with the data to be audited to obtain the comparison result can be achieved in the following manner: for each sub-data to be audited, the standard data carried by the sub-data audit rule is compared with the sub-data to be audited to obtain a sub-comparison result; when each sub-comparison result indicates that the standard data does not exist in the sub-data to be audited, the comparison result is determined as the standard data does not exist in the data to be audited; when the sub-comparison result exists, indicating that the standard data exists in the sub-data to be audited, the comparison result is determined as the standard data exists in the data to be audited.

[0065] In some embodiments, for each sub-data to be audited, the standard data (i.e., the expected data pattern or value) carried by the audit rule is compared with the sub-data to be audited one by one. This step is to verify whether each sub-data item meets the specified standards and requirements. For each sub-data to be audited, the comparison operation will generate a sub-comparison result, which indicates whether the sub-data contains the information specified in the standard data. There are two possibilities for the sub-comparison result: There is no standard data in the sub-data: this means that the sub-data does not meet the requirements of the audit rules. There is standard data in the sub-data: this means that the sub-data meets the requirements of the audit rules. If all sub-comparison results indicate that there is no standard data in the sub-data to be audited, it can be determined that there is no standard data in the entire data set to be audited. This indicates that the data to be audited has not passed the audit as a whole, and there may be serious data quality problems or it does not meet the specified standards. If at least one sub-comparison result indicates that there is standard data in the sub-data to be audited, it can be determined that at least a part of the data set to be audited meets the audit rules. This indicates that part of the data to be audited has passed the audit, and further analysis may be required as to which specific sub-data have passed the audit and which have not.

[0066] As an example, there is an e-commerce platform that contains user data, order data, and product data. Each data set has its own audit rules and standard data. The audit rules require that each user's email address must conform to a specific format and the user ID must be unique. The audit found that one email address in the user data was incorrectly formatted and one user ID was repeated. The audit rules require that the order number must be unique, the order date should be within the past 30 days, and the order amount should be greater than 0. The audit found that all order data met the requirements and no anomalies were found. The audit rules require that the product number must be unique, the product description cannot be empty, and the price must be a positive number. Comparison results: The audit found that all product data met the requirements and no anomalies were found. User data sub-comparison results: There are records that do not conform to the rules (wrong email format and repeated user IDs). Order data sub-comparison results: All records conform to the rules. Product data sub-comparison results: All records conform to the rules. Since the user data sub-comparison results indicate that there are records that do not conform to the rules, it can be determined that there is no standard data in the user data. Since both the order data sub-comparison result and the product data sub-comparison result indicate that all records meet the rule, it can be determined that standard data exists in the order data and the product data.

[0067] In this way, errors and non-compliances in the data can be accurately identified, thereby ensuring data quality and accuracy. When all sub-comparison results show that the sub-data to be audited does not meet the standards, corrective measures are taken immediately to prevent potential data problems from having a negative impact on business processes and decisions. When at least one sub-comparison result shows that the data meets the standards, the validity of some or all of the data can be confirmed, enhancing confidence in the data set. This refined audit process not only helps to continuously improve data quality, but also helps to reduce data risks, ensure data consistency and reliability, and provide strong support for the company's data-driven decision-making.

[0068] In step 104, based on the data audit result of the data source, a performance evaluation is performed on the data source to obtain a performance evaluation result of the data source.

[0069] In some embodiments, the data audit rule corresponding to the above data source includes multiple sub-data audit rules, and the above-mentioned data audit result includes sub-data audit results corresponding to each of the sub-data audit rules.

[0070] In some embodiments, the above step 104 can be implemented as follows: when the sub-data audit results of the data source all indicate that the data source has passed the data audit of the sub-data audit rule, the performance evaluation result of the data source is determined as the performance of the data source meets the standard; when the sub-data audit result in the data source indicates that the data source has not passed the data audit of the sub-data audit rule, the data source is performance evaluated based on the sub-data audit result indicating that the data source has not passed the data audit to obtain the performance evaluation result of the data source.

[0071] In some embodiments, the data audit rules corresponding to the data source include multiple sub-data audit rules, which define in detail the specific standards and requirements that each part of the data in the data source should meet. For example, the rules may include data format, numerical range, logical consistency, etc. Each sub-data audit rule corresponds to a sub-data audit result, which records whether each sub-data set has passed the audit. These sub-data audit results are specific evaluations of the quality of each part of the data source. When all sub-data audit results indicate that the data source has passed the corresponding sub-data audit rules, the performance evaluation results of the overall data source can be determined to be up to standard. This means that the data in the data source as a whole meets the predetermined quality standards. If there is any sub-data audit result indicating that the data source has not passed the audit rules, a more detailed analysis is required. Based on these sub-data audit results that indicate that the data source has not passed the audit, an in-depth performance evaluation of the data source is performed to determine the overall performance status of the data source.

[0072] As an example, there is a data source of a bank, which includes customer account information, transaction records, and customer service logs. Each part has a corresponding sub-data audit rule. Customer name, address, and ID number must be complete and in the correct format. Bank account numbers must be unique and conform to a specific format. The amount of each transaction must be a positive number. The transaction date must be within the past two years. The category and description of the service request cannot be empty. The customer service response time must be within the specified time range. Customer account information audit result: All records comply with the rules. Transaction record audit result: All records comply with the rules. Customer service log audit result: There are some records whose service response time exceeds the specified time range. Since all records of these two sub-datasets have passed the audit, it can be determined that the performance of customer account information and transaction records meets the standards. Customer service log performance evaluation: Since there are some records that have not passed the audit (the service response time exceeds the specified range), these failed records need to be analyzed to determine whether the overall performance of the customer service log meets the standards. If the number of failed records is small and does not affect the overall data quality, it can be considered that the performance of the customer service log basically meets the standards, but the service response time still needs to be paid attention to and improved. If the number of failed records is large, more stringent measures may be needed to improve the data quality of the customer service log. The performance of the entire data source was evaluated based on the sub-data audit results to ensure that the overall performance of the data source meets the expected standards, and improvement measures were proposed for the parts that did not meet the standards.

[0073] In this way, by implementing a detailed data audit on the data source and using the sub-data audit rules to evaluate the quality of each part of the data one by one, the performance of the data source can be fully monitored and optimized. When all sub-data audit results show that the data source has passed all the rules, we can determine that the overall performance of the data source meets the standards, thereby providing reliable data support for business decisions. However, if there are sub-data that have not passed the audit, these results will become the key basis for further evaluation and improvement of the performance of the data source, ensuring that data quality is continuously improved. This refined audit process not only enhances data reliability, but also provides a clear direction for data governance.

[0074] In some embodiments, the data to be audited includes a plurality of sub-data to be audited, and the sub-data audit result includes sub-comparison results corresponding to each of the sub-data to be audited.

[0075] In some embodiments, the above-mentioned performance evaluation of the data source based on the sub-data audit result indicating that the data audit has not passed, to obtain the performance evaluation result of the data source, can be achieved in the following way: for each sub-data audit result indicating that the data audit has not passed, the sub-comparison result in the sub-data audit result indicating that the standard data does not exist in the sub-data to be audited is determined as the target comparison result; based on the number of the target comparison results, the performance evaluation of the data source is performed to obtain the performance evaluation result of the data source.

[0076] In some embodiments, the data to be audited is composed of multiple sub-data to be audited, and each sub-data needs to be reviewed according to predefined audit rules. The sub-data audit results include sub-comparison results corresponding to each sub-data to be audited, which indicate whether each sub-data has passed the audit rules. For each sub-data that has not passed the audit, it is necessary to find out from its sub-data audit results those sub-comparison results indicating that there is no standard data in the sub-data, and determine it as the target comparison result. Based on the number of target comparison results, we can perform performance evaluation on the data source. The more target comparison results there are, the more data items that do not meet the standards in the data source, and the worse the performance evaluation results of the data source may be. If the number of target comparison results is small, it may indicate that there are a small number of problems in the data source, but the overall data quality is still high. If the number of target comparison results is large, it means that there are many data items that do not meet the standards in the data source, and the overall performance of the data source may not meet expectations.

[0077] In this way, by carefully analyzing multiple sub-data to be audited in the data to be audited and recording their respective sub-comparison results, we can accurately identify and evaluate the performance of the data source. Taking the sub-comparison results indicating that there is no standard data in the sub-data that failed the audit as the target comparison results, and evaluating the performance of the data source based on the number of these target comparison results, will help us fully understand the quality status of the data source. It can significantly improve the efficiency and accuracy of data management, ensure data quality, and reduce the risks caused by data errors or omissions, thereby providing a solid foundation for the company's data-driven decision-making. At the same time, the number of target comparison results also provides us with a direction for improving the performance of the data source, helping us to continuously improve data quality.

[0078] In some embodiments, after the data source is performance evaluated based on the data audit results of the data source and the performance evaluation results of the data source are obtained, the following processing may be performed: in response to a request for display of the performance evaluation results, the performance evaluation results of the data source are displayed.

[0079] In some embodiments, the quality of the data source is evaluated through the data audit process and performance evaluation results are obtained. These results may include whether the standards are met, the types and number of problems that exist, etc. When there is a request to display the performance evaluation results, the system or tool will provide the corresponding performance evaluation results based on the request. This usually involves presenting the results to the user in some form (such as reports, dashboards, charts, etc.). The purpose of displaying the performance evaluation results is to enable users or decision makers to intuitively understand the performance status of the data source. The content displayed may include the pass / fail ratio, the specific reasons for not meeting the standards, areas that need improvement, etc. The display method should be designed according to the needs and preferences of the user so that the user can quickly understand and take action.

[0080] In this way, by obtaining the data to be audited uploaded by multiple data sources, and determining the data audit rules corresponding to each data source, for each data source, based on the data audit rules corresponding to the data source, the data to be audited of the data source is audited to obtain the data audit result of the data source, and based on the data audit result of the data source, the data source is performance evaluated to obtain the performance evaluation result of the data source. Since different data sources correspond to different data audit rules, customized data audits can be implemented for different data sources, thereby effectively improving the accuracy of data audits, and by performing performance evaluation on the data source based on the data audit result of the data source, the performance evaluation result of the data source is obtained, thereby effectively improving the accuracy of the performance evaluation of the data source.

[0081] The present application is described below in conjunction with application examples.

[0082] Dynamic configuration of data audit rules is achieved through dynamic configuration of multi-dimensional models to solve the problem that the existing audit rule configuration is not flexible enough. The embodiment of the present application realizes the timed triggering of audit actions and the generation of audit reports through custom configuration of audit time, thereby solving the problem that huge data is not conducive to data analysis.

[0083] In some embodiments, see Figure 3 , Figure 3It is a principle diagram of the performance evaluation method of the data source provided by the embodiment of the present application, and dynamically configures the audit rules and audit scope. At present, CMDB is widely used in large IT enterprises by relying on its own advantages in identifying, maintaining and storing configuration information, and is mainly used to manage data information such as enterprise data centers, public clouds, private clouds, and asset equipment. The embodiment of the present application makes full use of the advantages of CMDB and abstracts the audit rule module and the audit scope module from the CMDB metadata. The audit rule module includes rule name, rule key, check field, check rule, check premise and solution. Among them, ID is used to uniquely identify a rule, and the rule ID is bound to the audit scope; the rule name is used to name the rule, and the audit effect of this rule can be roughly seen from the naming; the audit key is the English name of the audit name, which is usually named after the resource type; the check field is used to identify those attributes of the resource type that need to be checked, and supports the configuration of 1-n attribute names; the rule is a detailed description of the audit rule; the premise indicates that a special category among the attributes of the resource type is audited. The embodiment of the present application stores the ID, rule name, rule key, check field, check rule, check premise and solution as attribute fields in the data table.

[0084] In some embodiments, see Figure 3 The audit scope module mainly includes the region, resource pool, and POD to which the resource belongs. ID is used to uniquely identify an audit scope. The embodiment of the present application stores ID, region, resource pool, and POD as attribute fields in the data table. According to the actual audit scenario, the attributes of the audit rule module and the audit scope module are combined into several audit configuration items that meet the audit scenario, and the configuration items are stored. The audited data is audited according to the configuration items, and an audit report is generated. The audit report contains the number of problem data, unresolved data, resolved data, and server scale at the end of the month.

[0085] In some embodiments, a configuration assessment plan is formulated, and a reasonable assessment plan can constrain the data reporting party, thereby improving the reporting quality and management efficiency of data. The assessment timing task module of the embodiment of the present application uses the audit result metadata and calculates the assessment result regularly according to the preset assessment formula.

[0086] Assessment formula: Accuracy = (1-(the cumulative number of problem data items that occurred this month + the number of problem data items that were not resolved last month) / server scale at the end of the month)*100%. The cumulative number of problem data items that occurred this month, the number of problem data items that were not resolved last month, and the server scale at the end of the month are obtained from the audit results of the subsection.

[0087] In some embodiments, see Figure 3, Daily scheduled combined assessment: The scheduled assessment module obtains the audit configuration items from the data table according to the configured time every day. Audit the data according to the audit configuration items, and count the audit results, audit rules, total number of resources, and number of unresolved issues. The assessment-free task queue module batch-exempts the data from the assessment operation, and recalculates the audit results, total number of resources, and total number of unresolved issues.

[0088] In some embodiments, see Figure 3 For the visual audit and assessment results, the front-end operation page module of the embodiment of the present application provides a visual audit and assessment result display function. This module includes a rule configuration page, an audit result display page, a solved problem page, a trend data display page, an accuracy display page, and an audit import page.

[0089] In some embodiments, see Figure 3 The rule configuration page provides the functions of configuring audit rules and audit scope. The rule name, rule key, verification field, verification rule, verification premise and solution support custom configuration, and then associate the rule with POD.

[0090] In some embodiments, see Figure 3 The audit result display page will display the problem data (unresolved problem data) that triggers the audit rules on the page. It supports multi-dimensional queries based on the audit scope, resource type, type attributes, and audit rules, which is convenient for the reporter to check the problem data. The resolved audit result display page will display the problem data that the reporter has resolved on the page. It supports multi-dimensional queries based on the audit scope, resource type, type attributes, and audit rules.

[0091] In some embodiments, see Figure 3 The trend data display page displays the total number of problem data, unresolved issues, and triggered rules in the form of a trend chart. It supports multi-dimensional queries based on audit scope, resource type, and audit time. The accuracy display page displays the accuracy of each audit scope and resource type in a table. It intuitively displays the accuracy of the data reported by each reporter as a basis for evaluating the reporter. The audit import page provides the function of importing problem data.

[0092] In some embodiments, see Figure 3 , adapted to various data management scenarios, the embodiments of the present application are applicable to the audit and assessment requirements of various data management scenarios, such as configuration data audit, capacity quota audit, alarm audit and performance audit. The embodiments of the present application provide a problem data import page, which calculates the accuracy of the imported problem data in real time and displays it on the page.

[0093] In order to implement the performance evaluation method of a data source on the electronic device side of the embodiment of the present application, the embodiment of the present application also provides a performance evaluation device for a data source, Figure 4 The structure diagram of the performance evaluation device for the data source of the embodiment of the present application is shown in FIG. Figure 1 ,like Figure 4 As shown, the performance evaluation device of the data source includes: an acquisition module 61, which is used to acquire the data to be audited uploaded by multiple data sources respectively, and respectively determine the data audit rules corresponding to each of the data sources, and the data audit rules corresponding to different data sources are different; a data audit module 62, which is used to perform data audit on the data to be audited of the data source based on the data audit rules corresponding to the data source for each of the data sources, and obtain the data audit results of the data source; a performance evaluation module 63, which is used to perform performance evaluation on the data source based on the data audit results of the data source, and obtain the performance evaluation results of the data source.

[0094] In some embodiments, the acquisition module 61 is also used to obtain a preset audit rule library, and perform the following processing for each of the data sources: determine the data audit scope corresponding to the data source, and based on the data audit scope corresponding to the data source, select the preset audit rule that matches the data source from the preset audit rule library as the data audit rule corresponding to the data source.

[0095] In some embodiments, the acquisition module 61 is also used to obtain a first mapping relationship between a preset audit range and a preset data source, and determine a preset data source that is the same as the data source from the first mapping relationship; and determine the preset audit range corresponding to the preset data source that is the same as the data source in the first mapping relationship as the data audit range corresponding to the data source.

[0096] In some embodiments, the preset audit rule library includes a second mapping relationship between multiple preset audit rules and preset audit ranges; the acquisition module 61 is also used to determine, from the preset audit range of the second mapping relationship, a preset audit range that is the same as the data audit range corresponding to the data source as the target audit range; and determine the preset audit rules corresponding to the target audit range in the second mapping relationship as the data audit rules corresponding to the data source.

[0097] In some embodiments, the data audit rules corresponding to the data source include multiple sub-data audit rules, and the data audit module 62 is further used to perform data audit on the data to be audited of the data source based on each sub-data audit rule to obtain the sub-data audit result of the data source; and merge the sub-data audit results to obtain the data audit result of the data source.

[0098] In some embodiments, the data audit module 62 is further used to compare the standard data carried by the sub-data audit rule with the data to be audited to obtain a comparison result; when the comparison result indicates that the standard data does not exist in the data to be audited, the sub-data audit result is determined to be that the data source has not passed the data audit of the sub-data audit rule; when the comparison result indicates that the standard data exists in the data to be audited, the sub-data audit result is determined to be that the data source has passed the data audit of the sub-data audit rule.

[0099] In some embodiments, the data audit module 62 is also used to compare the standard data carried by the sub-data audit rule with the sub-data to be audited for each sub-data to be audited, to obtain a sub-comparison result; when each of the sub-comparison results indicates that the standard data does not exist in the sub-data to be audited, the comparison result is determined as the standard data does not exist in the data to be audited; when the sub-comparison result indicates that the standard data exists in the sub-data to be audited, the comparison result is determined as the standard data exists in the data to be audited.

[0100] In some embodiments, the data audit rules corresponding to the data source include multiple sub-data audit rules, and the data audit results include sub-data audit results corresponding to each of the sub-data audit rules; the performance evaluation module 63 is also used to determine the performance evaluation result of the data source as the performance of the data source meeting the standard when each of the sub-data audit results of the data source indicates that the data source has passed the data audit of the sub-data audit rules; when the sub-data audit results in the data source indicate that the data source has not passed the data audit of the sub-data audit rules, the data source is performance evaluated based on the sub-data audit results indicating that the data source has not passed the data audit to obtain the performance evaluation result of the data source.

[0101] In some embodiments, the data to be audited includes multiple sub-data to be audited, and the sub-data audit results include sub-comparison results corresponding to each sub-data to be audited; the performance evaluation module 63 is also used to determine, for each sub-data audit result indicating that the data audit has not passed, the sub-comparison result in the sub-data audit result indicating that there is no standard data in the sub-data to be audited, as the target comparison result; based on the number of the target comparison results, the data source is performance evaluated to obtain the performance evaluation result of the data source.

[0102] In some embodiments, the performance evaluation module 63 is further configured to display the performance evaluation result of the data source in response to a display request for the performance evaluation result.

[0103] In actual application, the acquisition unit 61 and the execution unit 63 can be implemented by a processor in the performance evaluation device of the data source; the query unit 62 can be implemented by a communication interface in the performance evaluation device of the data source.

[0104] It should be noted that: the performance evaluation device for the data source provided in the above embodiment only uses the division of the above program modules as an example when performing the performance evaluation of the data source. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the performance evaluation device for the data source provided in the above embodiment and the performance evaluation method embodiment of the data source on the electronic device side belong to the same concept. The specific implementation process is detailed in the performance evaluation method embodiment of the data source on the electronic device side, which will not be repeated here.

[0105] Based on the hardware implementation of the above program modules, and in order to implement the performance evaluation method of the data source on the electronic device side of the embodiment of the present application, the embodiment of the present application also provides an electronic device, Figure 5 Schematic diagram of the hardware structure of the electronic device of the embodiment of the present application. Figure 5 As shown, the electronic device 80 includes:

[0106] The first communication interface 81 is capable of exchanging information with other devices (such as the second client);

[0107] The processor 82 is connected to the first communication interface 81 to implement information interaction with other devices (such as a second client) and is used to execute the performance evaluation method of the data source on the electronic device side provided above when running a computer program, and the computer program is stored in the first memory 83.

[0108] Specifically, the processor 82 is used to obtain the data to be audited uploaded by multiple data sources, and determine the data audit rules corresponding to each data source, respectively. Different data sources correspond to different data audit rules.

[0109] The first communication interface 81 is used to perform data audit on the data to be audited of each data source based on the data audit rules corresponding to the data source, and obtain the data audit result of the data source;

[0110] The processor 82 is further configured to perform a performance evaluation on the data source based on the data audit result of the data source to obtain a performance evaluation result of the data source.

[0111] In one embodiment, the first communication interface 81 is also used to obtain a preset audit rule library in the processor 82, and perform the following processing for each of the data sources: determine the data audit range corresponding to the data source, and based on the data audit range corresponding to the data source, select a preset audit rule that matches the data source from the preset audit rule library as the data audit rule corresponding to the data source.

[0112] In one embodiment, the first communication interface 81 is specifically used for:

[0113] Obtaining a first mapping relationship between a preset audit scope and a preset data source, and determining a preset data source that is the same as the data source from the first mapping relationship;

[0114] The preset audit scope corresponding to the preset data source that is the same as the data source in the first mapping relationship is determined as the data audit scope corresponding to the data source.

[0115] In one embodiment, the first communication interface 81 is specifically used for:

[0116] From the preset audit scopes of the second mapping relationship, determine the preset audit scope that is the same as the data audit scope corresponding to the data source as the target audit scope;

[0117] The preset audit rule corresponding to the target audit scope in the second mapping relationship is determined as the data audit rule corresponding to the data source.

[0118] In one embodiment, the processor 82 is further configured to:

[0119] For each of the sub-data audit rules, based on the sub-data audit rules, data audit is performed on the data to be audited of the data source to obtain the sub-data audit result of the data source;

[0120] The sub-data audit results are merged to obtain the data audit result of the data source.

[0121] In one embodiment, the processor 82 is further configured to:

[0122] Compare the standard data carried by the sub-data audit rule with the data to be audited to obtain a comparison result;

[0123] When the comparison result indicates that the standard data does not exist in the data to be audited, the sub-data audit result is determined as the data source failing the data audit by the sub-data audit rule;

[0124] When the comparison result indicates that the standard data exists in the data to be audited, the sub-data audit result is determined as the data audit of the data source passing the sub-data audit rule.

[0125] In one embodiment, the processor 82 is further configured to:

[0126] For each of the sub-data to be audited, the standard data carried by the sub-data audit rule is compared with the sub-data to be audited to obtain a sub-comparison result;

[0127] When each of the sub-comparison results indicates that the standard data does not exist in the sub-data to be audited, determining the comparison result as that the standard data does not exist in the data to be audited;

[0128] When the sub-comparison result indicates that the standard data exists in the sub-data to be audited, the comparison result is determined as the standard data exists in the data to be audited.

[0129] In one embodiment, the processor 82 is further configured to:

[0130] When each of the sub-data audit results of the data source indicates that the data source has passed the data audit of the sub-data audit rule, the performance evaluation result of the data source is determined as the performance of the data source meeting the standard;

[0131] When the sub-data audit result exists in the data source and indicates that the data source has not passed the data audit of the sub-data audit rule, a performance evaluation is performed on the data source based on the sub-data audit result indicating that the data source has not passed the data audit to obtain a performance evaluation result of the data source.

[0132] In one embodiment, the processor 82 is further configured to:

[0133] For each sub-data audit result indicating that the data audit has not passed, a sub-comparison result in the sub-data audit result indicating that the standard data does not exist in the sub-data to be audited is determined as a target comparison result;

[0134] Based on the number of the target comparison results, a performance evaluation is performed on the data source to obtain a performance evaluation result of the data source.

[0135] In one embodiment, the processor 82 is further configured to:

[0136] In response to a display request for the performance evaluation result, the performance evaluation result of the data source is displayed.

[0137] It should be noted that the specific processing process of the first communication interface 81 and the processor 82 can be understood by referring to the performance evaluation method of the data source on the electronic device side.

[0138] Of course, in actual application, the various components in the electronic device 80 are coupled together through the first bus system 84. It is understandable that the first bus system 84 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 84 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 In the figure, various buses are labeled as a first bus system 84 .

[0139] The first memory 83 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 80. Examples of such data include: any computer program used to operate on the electronic device 80.

[0140] The performance evaluation method of the data source on the electronic device side disclosed in the above embodiment of the present application can be applied to the processor 82, or implemented by the processor 82. The processor 82 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the performance evaluation method of the data source on the electronic device side can be completed by the hardware integrated logic circuit or software instructions in the processor 82. The above-mentioned processor 82 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 82 can implement or execute the performance evaluation method, steps and logic block diagram of the data source on each electronic device side disclosed in the embodiment of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the performance evaluation method of the data source on the electronic device side disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in the first memory 83. The processor 82 reads the information in the first memory 83 and completes the steps of the performance evaluation method of the data source on the electronic device side in combination with its hardware.

[0141] In an exemplary embodiment, the electronic device 80 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the performance evaluation method of the data source on the electronic device side mentioned above.

[0142] It can be understood that the memory (including the first memory 83) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories (including memory 83) described in the embodiments of the present application are intended to include but are not limited to these and any other suitable types of memories.

[0143] In an exemplary embodiment, the embodiment of the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory 83 for storing a computer program, and the above-mentioned computer program can be executed by a processor 82 in an electronic device 80 to complete the steps of the performance evaluation method of the data source on the electronic device side described in the aforementioned embodiment of the present application, or, wherein the computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0144] It should be noted that: "first", "second", "third", etc. are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0145] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data source performance evaluation method, characterized in that: The method comprises: Acquire the data to be audited uploaded by multiple data sources, and determine the data audit rules corresponding to each data source, where different data sources correspond to different data audit rules; For each of the data sources, based on the data audit rules corresponding to the data source, data audit is performed on the data to be audited of the data source to obtain the data audit result of the data source; Based on the data audit result of the data source, a performance evaluation is performed on the data source to obtain a performance evaluation result of the data source.

2. The method according to claim 1, characterized in that The respectively determining the data audit rules corresponding to each of the data sources comprises: Obtain the preset audit rule library, and perform the following processing for each of the data sources: Determine the data audit scope corresponding to the data source, and based on the data audit scope corresponding to the data source, select the preset audit rule that matches the data source from the preset audit rule library as the data audit rule corresponding to the data source.

3. The method according to claim 2, characterized in that Determining the data audit scope corresponding to the data source includes: Obtaining a first mapping relationship between a preset audit scope and a preset data source, and determining a preset data source that is the same as the data source from the first mapping relationship; The preset audit scope corresponding to the preset data source that is the same as the data source in the first mapping relationship is determined as the data audit scope corresponding to the data source.

4. The method according to claim 2, characterized in that: The preset audit rule library includes a second mapping relationship between a plurality of preset audit rules and preset audit scopes; The data audit scope corresponding to the data source is based on, and a preset audit rule matching the data source is selected from the preset audit rule library as the data audit rule corresponding to the data source, including: From the preset audit scopes of the second mapping relationship, determine the preset audit scope that is the same as the data audit scope corresponding to the data source as the target audit scope; The preset audit rule corresponding to the target audit scope in the second mapping relationship is determined as the data audit rule corresponding to the data source.

5. The method according to claim 1, characterized in that The data audit rule corresponding to the data source includes a plurality of sub-data audit rules. Based on the data audit rule corresponding to the data source, data audit is performed on the data to be audited of the data source to obtain the data audit result of the data source, including: For each of the sub-data audit rules, based on the sub-data audit rules, data audit is performed on the data to be audited of the data source to obtain the sub-data audit result of the data source; The sub-data audit results are merged to obtain the data audit result of the data source.

6. The method according to claim 5, characterized in that The sub-data audit rule carries standard data corresponding to the sub-data audit rule. Based on the sub-data audit rule, data audit is performed on the data to be audited of the data source to obtain the sub-data audit result of the data source, including: Compare the standard data carried by the sub-data audit rule with the data to be audited to obtain a comparison result; When the comparison result indicates that the standard data does not exist in the data to be audited, the sub-data audit result is determined as the data source failing the data audit by the sub-data audit rule; When the comparison result indicates that the standard data exists in the data to be audited, the sub-data audit result is determined as the data audit of the data source passing the sub-data audit rule.

7. The method according to claim 6, characterized in that The data to be audited includes a plurality of sub-data to be audited, and the standard data carried by the sub-data audit rule is compared with the data to be audited to obtain a comparison result, including: For each of the sub-data to be audited, the standard data carried by the sub-data audit rule is compared with the sub-data to be audited to obtain a sub-comparison result; When each of the sub-comparison results indicates that the standard data does not exist in the sub-data to be audited, determining the comparison result as that the standard data does not exist in the data to be audited; When the sub-comparison result indicates that the standard data exists in the sub-data to be audited, the comparison result is determined as the standard data exists in the data to be audited.

8. The method according to claim 1, characterized in that The data audit rule corresponding to the data source includes a plurality of sub-data audit rules, and the data audit result includes sub-data audit results corresponding to each of the sub-data audit rules one by one; The data audit result based on the data source is used to perform a performance evaluation on the data source to obtain a performance evaluation result of the data source, including: When each of the sub-data audit results of the data source indicates that the data source has passed the data audit of the sub-data audit rule, the performance evaluation result of the data source is determined as the performance of the data source meeting the standard; When the sub-data audit result exists in the data source and indicates that the data source has not passed the data audit of the sub-data audit rule, a performance evaluation is performed on the data source based on the sub-data audit result indicating that the data source has not passed the data audit to obtain a performance evaluation result of the data source.

9. The method according to claim 8, characterized in that The data to be audited includes a plurality of sub-data to be audited, and the sub-data audit result includes sub-comparison results corresponding to each of the sub-data to be audited; The performing performance evaluation on the data source based on the sub-data audit result indicating failure to pass the data audit to obtain the performance evaluation result of the data source includes: For each sub-data audit result indicating that the data audit has not passed, a sub-comparison result in the sub-data audit result indicating that the standard data does not exist in the sub-data to be audited is determined as a target comparison result; Based on the number of the target comparison results, a performance evaluation is performed on the data source to obtain a performance evaluation result of the data source.

10. The method according to claim 1, characterized in that After performing performance evaluation on the data source based on the data audit result of the data source and obtaining the performance evaluation result of the data source, the method further includes: In response to a display request for the performance evaluation result, the performance evaluation result of the data source is displayed.

11. A data source performance evaluation device, characterized in that: include: An acquisition module is used to acquire the data to be audited uploaded by multiple data sources, and to determine the data audit rules corresponding to each data source, where different data sources correspond to different data audit rules; A data audit module is used to perform data audit on the data to be audited of each data source based on the data audit rules corresponding to the data source, and obtain the data audit result of the data source; The performance evaluation module is used to perform performance evaluation on the data source based on the data audit result of the data source to obtain the performance evaluation result of the data source.

12. An electronic device, characterized in that: include: a processor and a first memory for storing a computer program executable on said processor; Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images