Data Quality Verification System for Data Transactions Based on Data Valuation Models

Abstract

The present invention is a data quality verification system for data trading based on a data value assessment model, comprising: a meta-information collection module (11) that collects basic information of a target data set and tracks the data processing history; a data history tracking module (12) that tracks and records the data processing process; a data error verification module (13) that verifies the integrity, validity, and errors of the data; a forgery verification module (14) that examines whether the data has been tampered with; a quality evaluation module (15) that evaluates the timeliness, accuracy, and sufficiency of the data; and a transaction suitability module (16) that evaluates transaction suitability based on an evaluation index.

Description

Technology Field

[0001] The present invention relates to a data quality verification system for data development based on a data valuation model. Background Technology

[0002] The present invention relates to a data quality verification method for data trading based on a data valuation model. Data has established itself as a crucial asset in the modern economy, and data quality significantly impacts data-driven decision-making, the performance of artificial intelligence models, and the reliability of data trading. To revitalize the data trading market, a systematic system capable of guaranteeing data quality and objectively verifying it is required. A data quality verification system can play an important role in developing a data valuation model and establishing a foundation for operating a data trading platform in the long term.

[0003] Currently, the demand for data for various services in both the public and private sectors is skyrocketing due to the utilization of AI and other technologies. Furthermore, with the activation of personal data (My Data), we are entering an era of true data sovereignty. Consequently, the distribution of safe and highly reliable data is essential for data trading, and the development of data quality verification technology is required as a prerequisite for data trading through platforms. Through this, trust must be established between data providers and consumers, and the utilization value of data must be maximized. In the long term, a data valuation model should be established based on data secured through quality verification, and a data bank should be operated based on this model to revitalize data trading. The problem to be solved

[0004] The purpose is to identify quality elements necessary for data quality verification and define indicators for verifying them in order to solve the aforementioned challenges. Furthermore, data quality verification is made possible including unstructured data, and based on integrity, validity, and value error verification, it enables the verification of falsification or alteration of large volumes of data, verification of data timeliness, and verification of data value through AI algorithms, thereby allowing for the determination of transaction suitability. means of solving the problem

[0005] In order to achieve the above-mentioned purpose, a data quality verification system for data trading based on a data value assessment model is provided, which may include: a meta-information collection module (11) that collects basic information of a target data set and tracks the data processing history; a data history tracking module (12) that tracks and records the data processing process; a data error verification module (13) that verifies the integrity, validity, and errors of the data; a forgery verification module (14) that examines whether the data has been tampered with; a quality evaluation module (15) that evaluates the timeliness, accuracy, and sufficiency of the data; and a transaction suitability module (16) that evaluates transaction suitability based on an evaluation index.

[0006] It may further include an authentication module (17) that provides a mark identifying that the metadata has been authenticated when the transaction suitability is above a certain level.

[0007] The information collected by the meta-information collection module (11) may include the time when the data was created, the source of the data, ownership information, file format, and data structure.

[0008] The data error verification module (13) can check the suitability of the data type and whether there is a value within the allowable range. Effects of the invention

[0009] By the configuration described above, quality elements necessary for data quality verification are identified and indicators for verifying them are defined, thereby having the effect of maintaining the basic quality of data for data trading. Brief explanation of the drawing

[0010] FIGS. 1 and 2 are drawings illustrating an embodiment according to the present invention. Specific details for implementing the invention

[0011] The present invention will be described in detail below with reference to the attached drawings. FIG. 1 is a drawing illustrating an embodiment according to the present invention. To revitalize the data trading market, the present invention requires a systematic system capable of guaranteeing data quality and objectively verifying it, and the data quality verification system provides a data value assessment model. In order to become a subject of trading on a data trading platform, source data undergoes a data cleansing process and a data verification process. To this end, the process includes collecting metadata of the dataset, verifying various errors, confirming forgery or tampering, and classifying the data through quality evaluation. Furthermore, reliability is enhanced by operating a data certification system, and all these processes can be automated and processed and managed in real time by AI.

[0012] The metadata collection module (11) performs the function of collecting basic information of the dataset and tracking the history of data processing, and identifies the overall characteristics of the data. It determines the timeliness of the data by recording the time when the data was created or modified, and prevents legal issues by clarifying the source and ownership information of the data. In addition, it improves processing efficiency by identifying the file format, size, and internal structure of the dataset. Additionally, it can verify changes during the processing process by tracking the history of data changes.

[0013] The data history tracking module (12) tracks and records the entire data processing process, records information on personnel at each data processing stage, stores information on algorithms and parameters used, ensures the continuity of the data processing process, and provides verifiability.

[0014] The data error verification module (13) provides functions for checking data integrity, validity, and value errors, such as checking the completeness of the data structure, verifying integrity constraints, verifying the validity of the relationship between the primary key and foreign key, and checking whether required field values ​​exist. In addition, it can check the suitability of the data type, check for the existence of values ​​within the allowed range, verify the consistency of date and time formats, and check for pattern matching based on regular expressions. For value error verification, it can detect outliers using statistical methods, identify abnormal values ​​based on Z-scores, detect deviations from the trend of time series data, and verify domain-specific rules.

[0015] The tamper-proof verification module (14) can verify whether data has been tampered with using a hash function such as SHA-256, store the hash value of the original data, and verify it periodically. Additionally, it can verify digital signatures, secure data reliability through public key-based electronic signatures, and verify the authenticity of the signature and the signer's authority through linkage with a certification authority. Furthermore, it can perform blockchain-based history management, store verification results on the blockchain to ensure transparency of data change history, perform automatic verification through smart contracts, and maintain data integrity by utilizing a distributed ledger.

[0016] The quality evaluation module (15) evaluates the accuracy, sufficiency, completeness, validity, timeliness, and reliability of the data in various ways. For the accuracy evaluation, it calculates the outlier ratio and measures the accuracy of the data relative to the reference value. For the completeness evaluation, it analyzes the ratio of missing values ​​and the satisfaction rate of required values, and checks the data density. For the validity evaluation, it evaluates the format compliance rate and the constraint satisfaction rate, and verifies whether the standard code is used.

[0017] In addition, for recency evaluation, it is possible to verify the last update date of the data, analyze the satisfaction of real-time requirements and the appropriateness of the update cycle, and for reliability evaluation, calculate the ratio of duplicate data and assess the reliability of the data source.

[0018] For tradable data that has passed the above evaluation, meta-information is stored in the tradable data DB (21), and the transaction suitability module (16) calculates the transaction suitability based on the evaluation index.

[0019] The authentication module (17) can assign an authentication mark to the dataset. Additionally, for tradable data, the identifier assignment module (18) may assign an identifier according to categories such as the format or content of the data for evaluation, and allow the valuation module (19) to evaluate the value. The valuation module calculates the grade of the data based on the quality score and evaluates the market value.

[0020] The aforementioned module operates independently and manages results in an integrated manner through a centralized database. This enables the entire data quality verification process to be traceable and verified in real time.

[0021] The judgment of specific data quality may vary from case to case, and an example is described below.

[0022] The above weights can be assigned to Accuracy (35%), Completeness (25%), Validity (20%), Recency (10%), and Reliability (10%), respectively, where Accuracy is z-score-based outlier detection,

[0023] Completeness can be assessed by calculating the null value ratio, validity by format verification for each data type, recency by the number of days elapsed since the reference date, and reliability by considering the ratio of records. High weighting can be assigned.

[0024] A comprehensive score is calculated by multiplying the score of each element by a weight and summing them, and the final score is normalized to within 100 points. These weights may be applied separately as industry-specific weights and can be adjusted to suit the characteristics of the data used and business requirements. Explanation of the symbols

[0025] 11: Meta-information collection module 12: Data History Tracking Module 13: Data Error Verification Module 14: Tamper-proof Verification Module 15: Quality Evaluation Module 16: Transaction Fit Module 17: Authentication Module

Claims

Claim 1 A data quality verification system for data transactions based on a data valuation model, comprising: a meta-information collection module (11) that collects basic information of a target data set and tracks data processing history; a data history tracking module (12) that tracks and records data processing processes; a data error verification module (13) that verifies the integrity, validity, and errors of the data; a forgery verification module (14) that examines whether the data has been tampered with; a quality evaluation module (15) that evaluates the timeliness, accuracy, and sufficiency of the data; and a transaction suitability module (16) that evaluates transaction suitability based on an evaluation index. Claim 2 A data quality verification system for data trading based on a data valuation model, further comprising, in claim 1, an authentication module (17) that provides a mark identifying that the metadata has been authenticated when the transaction suitability is above a certain level. Claim 3 In paragraph 2, the information collected by the meta-information collection module (11) includes the time of data creation, the source of the data, ownership information, file format, and data structure, in a data quality verification system for data creation based on a data value assessment model. Claim 4 In paragraph 3, the data error verification module (13) checks the suitability of the data type and whether there is a value within the allowable range, a data quality verification system for data development based on a data value assessment model.

Images