SQL Data Quality Metrics for Geophysical Exploration Hierarchies

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Geophysical explorations generate vast and disparate borehole data that is often incomplete, inaccurate, or duplicated across multiple databases, posing challenges in data quality management and integration.

Innovation Solution

A computer-implemented method that accesses metadata tables to characterize exploration data assets, applies data quality rules using SQL statements, identifies defects, calculates quality metrics, and monitors data quality over time, utilizing a dynamic schema to maintain and update data attributes dynamically.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data is stored in multiple disparate databases across vast geographic areas, then data quantity and coverage are improved, but data quality and consistency deteriorate due to duplication and distribution

Engineering Contradiction:

Improvedata quantityVSAvoiddata quality

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system segments data quality management into hierarchical levels (project, well, measurement, attribute) and implements distributed data storage across multiple databases while maintaining centralized quality control through a unified data model and metadata standards

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A centralized data quality management system acts as an intermediary between disparate databases, using standardized metadata tables, data models, and quality rules to coordinate data across distributed systems while maintaining consistency and enabling quality assessment

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If data is distributed and duplicated across multiple databases, then data availability is improved, but data accuracy and completeness worsen due to disparate formats and loading errors

Engineering Contradiction:

Improvedata availabilityVSAvoiddata accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The system implements a universal data model and standardized metadata schema that can represent multiple data types and formats uniformly, enabling the same data structure to serve multiple purposes across different databases while maintaining accuracy through consistent validation rules

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system changes the state of data from raw, unvalidated formats to standardized, validated formats through automated quality rules that transform data parameters (completeness, consistency, accuracy) while maintaining the underlying data values across distributed systems

Inventive Principle:

Principle #35Parameter changes

3Reliability

If comprehensive data quality rules are applied to vast quantities of exploration data, then data quality assessment is improved, but computational complexity and processing time worsen

Engineering Contradiction:

Improvedata quality assessmentVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system segments comprehensive data quality rules into hierarchical categories (project-level, well-level, measurement-level, attribute-level rules) that can be applied incrementally, reducing computational complexity while maintaining thorough quality assessment across vast datasets

Inventive Principle:

Principle #1Segmentation

4Ease of manufacture

If static data quality rules are used, then implementation simplicity is improved, but adaptability to changing data requirements worsens

Engineering Contradiction:

Improveimplementation simplicityVSAvoidadaptability

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The system implements dynamic data quality rules stored in metadata tables that can be modified, added, or removed without changing the underlying system structure, allowing rules to adapt to evolving data requirements while maintaining a simple standardized implementation framework

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11816077B2Measuring data quality in a structured database through SQL

Publication Date: 2023.11.14 SAUDI ARABIAN OIL CO

AI summary

Some implementations of the present disclosure provide a method that include: accessing a plurality of tables that store (i) metadata that characterize a hierarch of exploration data assets, (ii) metadata that characterize a set of data quality rules, (iii) metadata that characterize defects identifiable as data records in the hierarchy of exploration data assets that fail to comply with the set of data quality rules; querying the hierarch of exploration data assets according to one or more data quality rules from the set of data quality rules; identifying instances of data records that fail to meet the one or more data quality rules; based on analyzing the instances of data records, calculating one or more data quality metrics for the hierarchy of exploration data assets; and monitoring the hierarchy of exploration data assets based on the calculated one or more data quality metrics.