Semantic Network Data Validation and Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data systems are vulnerable to data errors and inconsistencies due to the integration of data from disparate sources, leading to incomplete and invalid data, which complicates strategic and operational decision-making in businesses. These systems often require skilled analysts and are prone to data gaps or mistakes, making it difficult to ensure data accuracy across business units.
Innovation Solution
A computer-implemented method using a semantic network with nodes and links to validate data by applying transformation functions and rules, ensuring data correctness through syntactic, semantic, and network validation, and revising data accordingly to maintain data integrity across the network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is integrated from multiple disparate sources, then the quantity and coverage of data increases, but data accuracy and consistency deteriorate
Solution Approach 1:
The patent introduces a semantic network as an intermediary layer between disparate data sources and analysis systems. This semantic network standardizes data representation through unified schemas, data type definitions, and relationship models, enabling data integration while maintaining accuracy through consistent semantic interpretation across all sources
Solution Approach 2:
The system transforms raw data from multiple sources into standardized parameters within the semantic network framework. By changing the representation parameters of incoming data to match predefined schemas and data types, the system maintains data accuracy while integrating diverse sources with different formats and structures
2Reliability
If traditional data validation methods are used, then data correctness can be verified, but system complexity and resource requirements increase
Solution Approach 1:
The semantic network performs self-validation through its structured schema definitions, data type constraints, and predefined relationship rules. The system automatically verifies data correctness by checking against these built-in constraints without requiring external validation systems or complex manual verification processes
Solution Approach 2:
The system implements continuous feedback loops where data validation results are fed back into the semantic network to maintain consistency. Validation errors and corrections are propagated through the network, enabling automatic detection and resolution of data issues while maintaining overall data integrity
3Reliability
If skilled analysts manually verify data, then data accuracy improves, but time consumption and operational costs increase
Solution Approach 1:
The patent replaces manual mechanical verification processes with automated computational validation. The semantic network's structured schemas, data type constraints, and validation rules automatically perform verification functions that previously required skilled analysts, dramatically reducing time and cost while maintaining accuracy
Solution Approach 2:
The system performs self-validation through its built-in semantic constraints and validation mechanisms, eliminating the need for external human verification. The semantic network automatically ensures data correctness through its structured framework, reducing reliance on manual analyst intervention
4Loss of time
If data updates occur frequently, then data freshness and relevance improve, but data validity and completeness deteriorate
Solution Approach 1:
The semantic network implements continuous feedback validation where each data update is automatically checked against schema constraints and relationship rules. This feedback mechanism ensures that frequent updates maintain validity by verifying consistency with the established semantic model before accepting changes
Solution Approach 2:
The system performs preliminary validation checks before accepting data updates. By pre-defining valid data types, schemas, and relationship constraints, the system ensures that incoming updates meet validity requirements before being integrated, preventing corrupt data from compromising the network
Data Source
AI summary
Disclosed herein are techniques and tools for verifying data for semantic correctness and/or verifying data for network correctness. In one respect, a method includes receiving input defining a validation point, the validation point comprising at least two or more validation functions applicable to (i) raw data and (ii) other data stored within a semantic network comprising nodes and links, importing source data; applying one or more transformations to the source data, populating the source data into one or more of the nodes and links comprising the semantic network, executing the validation point with respect the source data, based on the executing, determining one or more rules associated with the validation point are not satisfied, and based on the determining, revising either the source data or the other data stored within the semantic network.