Data Rule Binding via Domain Signatures

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In complex IT systems, manually defining data quality rules is difficult and time-consuming due to the large number of tables and columns with unclear semantics, making it challenging to apply the right rules to the correct columns effectively.

Innovation Solution

A system and method that automatically suggest data rules by associating logical variables with columns using precomputed domain signatures, allowing for the comparison and binding of characteristics to identify suitable rule applications across different data sources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual definition of data rules is used, then rule accuracy can be ensured, but time consumption and difficulty increase significantly

Engineering Contradiction:

Improverule accuracyVSAvoidtime consumption

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary data profiling and analysis to pre-compute domain signatures, data characteristics, and column metadata before rule definition. This preliminary action enables automatic rule suggestion by having the necessary information ready in advance, reducing the time required for manual rule definition while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system enables self-service by automatically suggesting data rules based on analyzed data characteristics and domain signatures. The automatic rule suggestion mechanism allows the system to serve itself in identifying appropriate rules without requiring extensive manual intervention, thereby reducing time consumption while preserving rule accuracy through expert-system-based recommendations.

Inventive Principle:

Principle #25Self-service

2Reliability

If manual definition of data rules is used, then rule quality can be maintained, but operational difficulty increases

Engineering Contradiction:

Improverule qualityVSAvoidoperational difficulty

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The system introduces an intermediary layer consisting of domain signatures and data characteristics that mediate between the raw data and the data rules. This intermediary enables automatic rule suggestion by providing structured information about data domains and characteristics, making the rule definition process easier while maintaining rule quality through the intermediary's structured analysis.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system enables self-service by automatically suggesting data rules based on analyzed data characteristics and domain signatures. The automatic rule suggestion mechanism allows the system to serve itself in identifying appropriate rules without requiring extensive manual intervention, thereby reducing operational difficulty while preserving rule quality through expert-system-based recommendations.

Inventive Principle:

Principle #25Self-service

3Measurement precision

If comprehensive data analysis is performed to understand data semantics, then rule application accuracy improves, but system complexity increases

Engineering Contradiction:

Improverule application accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system segments the comprehensive data analysis into distinct components: data profiling, domain signature computation, characteristic extraction, and rule suggestion. This segmentation reduces system complexity by breaking down the complex analysis process into manageable, modular components while maintaining rule application accuracy through the cumulative effect of each segment.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary data profiling and analysis to pre-compute domain signatures and data characteristics before rule definition. This preliminary action simplifies the overall system by having the complex analysis work done in advance, making the subsequent rule application more accurate without requiring the full complexity to be present during rule execution.

Inventive Principle:

Principle #10Preliminary action

4Productivity

If data rules are automatically suggested, then productivity increases, but measurement precision may decrease

Engineering Contradiction:

Improverule definition efficiencyVSAvoidrule accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system incorporates feedback mechanisms where data profiling results and domain signature analysis feed into the automatic rule suggestion process. This feedback loop ensures that automatically suggested rules are based on actual data characteristics and domain knowledge, maintaining measurement precision while achieving high productivity through automation.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS8949166B2Creating and processing a data rule for data quality

Publication Date: 2015.02.03 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A data rule is created and processed by receiving an expression defining a logic of a rule and at least one logical variable, creating a rule definition including the expression and the at least one logical variable for binding each logical variable of the rule with at least one column, associating a characteristic enabling comparison of columns with a first logical variable of the rule definition, and storing the characteristic as part of the rule definition.