Data Segment Clustering for Inheritance Source Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying relevant data in large-scale databases with billions of records is challenging due to the computational infeasibility of comparing numerous datasets without a concrete strategy, and datasets associated with different individuals or events are often not connected without proper determination of their relationship.
Innovation Solution
A computer-implemented method involving receiving a target data instance, identifying matched data segments, defining data string ranges based on overlapping segments, and applying an iterative clustering algorithm to group these ranges using a similarity metric, attributing them to a data inheritance source.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a large-scale database with billions of data records is examined to identify relevant data, then the completeness of data analysis is improved, but the computational complexity and time required increase significantly
Solution Approach 1:
The patent segments the data analysis process into multiple stages: first identifying candidate datasets through initial filtering, then applying clustering algorithms to group similar datasets, and finally performing detailed analysis only on relevant clusters. This segmentation reduces the computational burden by avoiding exhaustive comparison of all billions of records while maintaining analysis completeness through systematic sampling and clustering.
Solution Approach 2:
The patent introduces intermediary clustering structures and similarity metrics as mediators between the raw data and the final analysis. By using clustering algorithms to create intermediate groupings of similar datasets, the system can efficiently navigate and filter through the large database without directly comparing every record, thus reducing computational complexity while maintaining relevant data identification.
2Adaptability or versatility
If datasets are compared without a concrete strategy to determine relationships, then the flexibility of analysis is improved, but the reliability of identified connections decreases
Solution Approach 1:
The patent employs parameter changes through dynamic similarity thresholds and adjustable clustering criteria. The system can adapt similarity metrics and comparison parameters based on the specific analysis context, allowing flexible identification of relevant datasets while maintaining reliability through statistically sound comparison methods. This enables the system to adjust sensitivity and specificity parameters to balance flexibility and accuracy.
3Loss of information
If the entire database is searched for related datasets, then the comprehensiveness of data inheritance identification is improved, but the time required for analysis increases
Solution Approach 1:
The patent performs preliminary actions by pre-computing similarity metrics, pre-grouping datasets into clusters based on initial criteria, and pre-identifying candidate inheritance paths before the main analysis. This preliminary processing creates structured intermediaries that accelerate the final analysis, reducing time requirements while maintaining comprehensive coverage of potential data inheritances through systematic search strategies.
Data Source
AI summary
A computing device may receive a target data instance. The computing device may identify a plurality of matched segments that match to the target data instance for at least a threshold length. The computing device may define, based on overlapping of the matched segments, the target data instance as a plurality of data string ranges, wherein each divided data string is matched to a set of overlapping matched segments. The computing device may apply an iterative clustering algorithm to group the plurality of data string ranges based on values of a similarity metric among data string ranges that are assigned to a given group. The computing device may attribute a first set of data string ranges that are assigned to a first group to a first inheritance.