Data Record Grouping via Multi-Metric Similarity Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in grouping and predicting data records with variations in descriptors, leading to difficulties in identifying trends and predicting future data, which affects computer memory management and data retrieval efficiency.
Innovation Solution
A system comprising a data record textual match analysis module, a data record grouping module, and a quality control module that determines textual and temporal similarity, clusters similar records, and identifies recurring patterns to provide accurate predictions and improve data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data records are grouped using automated computerised systems, then data retrieval speed and memory management improve, but variations in data descriptors cause related records to be excluded from grouping
Solution Approach 1:
The system changes the parameter of textual similarity threshold and applies multiple similarity metrics (edit distance, cosine distance, word2vec) to dynamically adjust grouping criteria. This allows the system to adapt to variations in data descriptors while maintaining accurate grouping of related records.
Solution Approach 2:
The patent introduces an intermediary quality control module that mediates between automated grouping and final record association. This module performs additional similarity analysis on borderline cases, ensuring that related records are not incorrectly excluded due to descriptor variations.
2Measurement precision
If strict textual matching is applied to group data records, then grouping precision improves, but related records with descriptor variations are excluded
Solution Approach 1:
The system employs multiple textual similarity metrics (edit distance, cosine distance, word2vec analysis) with adjustable thresholds. This allows the system to maintain precision while adapting to descriptor variations by selecting appropriate metrics and threshold levels for different data scenarios.
Solution Approach 2:
The patent combines multiple similarity assessment methods (edit distance, cosine distance, word2vec) into a composite evaluation system. This composite approach leverages the strengths of each method to achieve both precision and adaptability in handling descriptor variations.
3Reliability
If multiple similarity metrics are applied to all data records, then grouping accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The system segments the data processing into multiple stages: initial filtering with one metric, intermediate analysis with additional metrics for borderline cases, and final quality control review. This segmentation applies multiple similarity metrics only where necessary, reducing overall processing time while maintaining accuracy.
Solution Approach 2:
The patent applies multiple similarity metrics partially - not to all data records uniformly, but selectively to records that require further analysis after initial screening. This partial application of excessive action ensures high accuracy for critical cases while minimizing unnecessary computational overhead.
Data Source
AI summary
A system and computer-implemented method of grouping data records for subsequent data record searching. A level of textual similarity of data records in a group of data records is determined using matching and identifying a textual similarity metric between pairs of data records in the group of data records, and clustering the data records to form groups of textually similar data records. The groups of textually similar data records are provided to a computerised log of textually-matched data records. Further, temporally-repeating data records in the computerised log are identified. If all the data records in a group of temporally-repeating textually similar data records have a level of temporal and textual similarity above an overall similarity threshold, they are provided to a computerised log of temporally and textually-matched data records, for example for searching or future forecasting.