Corroborated Data Trust Scoring for Synthetic Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer-implemented services are compromised by the inclusion of synthetic data, which reduces the quality and trustworthiness of datasets, failing to meet the needs of data consumers and downstream consumers.
Innovation Solution
A corroborated data database is populated with non-synthetic data corroborated using other trusted data sources, assigning a level of trust based on a corroboration schema, and providing it to consumers to ensure the data is usable for computer-implemented services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If synthetic data is included in datasets, then data quantity and diversity are improved, but data quality and trustworthiness deteriorate
Solution Approach 1:
The system performs corroboration of data before it is used in computer-implemented services. By verifying data provenance and authenticity in advance through multiple data sources and corroboration schemas, the system ensures that only trusted non-synthetic data is provided to consumers, preventing the inclusion of synthetic data while maintaining data availability.
2Reliability
If retroactive synthetic data detection is performed, then data quality is improved, but computational resources are consumed
Solution Approach 1:
Instead of performing retroactive detection of synthetic data after it has been included in datasets (which consumes significant computational resources), the system establishes corroboration mechanisms in advance. By verifying data authenticity and provenance before data is consumed, the system eliminates the need for resource-intensive retroactive detection while maintaining data quality.
Solution Approach 2:
The system introduces corroboration schemas and data provenance tracking as intermediary mechanisms between data sources and consumers. These intermediaries verify data authenticity and provide trust information, eliminating the need for consumers to perform expensive retroactive synthetic data detection themselves.
3Reliability
If data corroboration is performed using multiple data sources, then data trustworthiness is improved, but system complexity increases
Solution Approach 1:
The system implements a universal corroboration framework that can verify data from multiple sources using standardized corroboration schemas. This multi-functional approach allows the same corroboration mechanism to handle different data types and sources, managing system complexity through generalization rather than requiring separate verification processes for each data source.
Data Source
AI summary
Methods and systems for managing data used to provide computer-implemented services are disclosed. To manage the data, a request for corroborated data may be obtained from a data consumer indicating a desired information content and a threshold level of trust. Based on the request, the corroborated data may be obtained from a corroborated data database which has the desired information content and a level of trust that meets the level of trust threshold. The corroborated data may be obtained from a first data source and may be corroborated using at least one other data source adapted to measure a similar information content to the desired information content which the first data source may be adapted to measure. The level of trust may be based on a corroboration schema. At least a portion of the corroborated data may be provided to the data consumer to facilitate provisioning of the computer-implemented services.