Dataset Linkage via Embedding Vectors for Disjoint Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data linkage techniques struggle to effectively link disjointed datasets without common attributes, limiting cross-modality analysis and performance improvement, especially in low-constrained settings where resources are limited.
Innovation Solution
The system projects datasets into an equal-dimensional feature space using dimension reduction algorithms, computes a linkage matrix with distance metrics, and identifies close neighbors across datasets to enable cross-modality analysis and performance improvement without requiring common features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional data linkage techniques are used to link datasets, then common attributes are required for successful linkage, but this limits the ability to link disjointed datasets without common features
Solution Approach 1:
The patent introduces an intermediary representation called 'embedding vectors' that mediate between disparate datasets. These vectors transform data from different sources into a common mathematical space, enabling linkage without requiring common attributes. The embedding layer acts as a mediator that translates heterogeneous data formats and features into a unified representation that can be processed by the linkage algorithm.
Solution Approach 2:
The patent changes the fundamental parameters of data representation by transforming raw data features into embedding vectors through neural network layers. This parameter transformation allows datasets with completely different feature types and dimensions to be compared and linked by projecting them into a shared vector space where similarity can be measured using standard distance metrics.
2Adaptability or versatility
If dimension reduction algorithms are applied to project datasets into equal-dimensional feature space, then cross-modality analysis is enabled, but computational resources are consumed
Solution Approach 1:
The patent performs preliminary dimensionality reduction by projecting all datasets into a common equal-dimensional feature space before executing the linkage algorithm. This pre-processing step, performed through embedding layers and projection mechanisms, transforms heterogeneous high-dimensional data into a standardized lower-dimensional representation, enabling subsequent linkage operations to be computationally efficient and avoiding repeated heavy computations during the actual linkage process.
3Measurement precision
If baseline predictions are generated for each database to measure performance difference, then linkage accuracy is improved, but additional processing time is required
Solution Approach 1:
The patent implements a feedback mechanism by generating baseline predictions for each database and comparing them against actual performance measurements. This feedback loop allows the system to quantify the improvement achieved through linkage by measuring the difference between baseline (unlinked) performance and actual (linked) performance, thereby providing a precise metric for linkage quality and enabling iterative optimization of the linkage process.
Data Source
AI summary
Embodiments of the present invention provide a computer system, a computer program product, and a method that comprises generating a baseline prediction for each database in a plurality of databases, wherein the generated baseline prediction includes datapoints for each performance feature of a respective database in the plurality of databases; generating a linkage matrix using a dimension reduction algorithm in an equal-dimensional feature space based on a measured difference between actual performance datapoints of each respective database and respective datapoints associated with generated baseline prediction for each database in the plurality of databases; linking one or more databases in the plurality of databases based on the measured difference within the generated linkage matrix reaching a predetermined threshold of measured performance; and generating a design using the linked databases for a computer system capable of being displayed through a user interface.