Part Catalogue Auto-Encoder for Missing and Noisy Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Industrial product catalogues suffer from insufficient data quality, leading to incorrect or missing search results, poor performance of machine learning models, and high costs for human expert intervention due to the difficulty in identifying data quality issues like wrong units of measure or inconsistent parameter values.
Innovation Solution
An auto-encoder model is used to perform entity resolution and auto-completion on co-occurrence graphs representing part specifications, enabling automated correction and completion of datasets by capturing correlations between part properties, values, and units of measure without requiring labeled training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If traditional entity resolution systems with multiple steps (data pre-processing, blocking, clustering, matching) are used, then domain expert involvement is required and the process is complex, but automation extent is limited and time consumption is high
Solution Approach 1:
The patent combines multiple entity resolution steps (blocking, clustering, matching) into a single auto-encoder neural network model. The encoder processes input entities and the decoder generates matched entities, integrating what were previously separate manual or rule-based steps into one automated deep learning system, thereby increasing automation while reducing process complexity
Solution Approach 2:
The patent replaces traditional mechanical entity resolution methods (regular expression text matching, fuzzy string matching, and manual expert intervention) with a neural network-based auto-encoder system. This substitution enables fully automated processing without requiring domain experts to guide each step, significantly improving automation extent
2Measurement precision
If graph embedding methods are used for entity resolution, then data representation is enhanced, but seed alignments are required which are expensive and require expertise
Solution Approach 1:
The auto-encoder model is trained in a self-supervised manner where the model learns to map entities to themselves through the encoder-decoder architecture. The training process automatically learns meaningful representations without requiring pre-labeled seed alignments, making the system self-sufficient and eliminating the need for expensive expert annotation
Solution Approach 2:
The patent performs preliminary training of the auto-encoder model on available data to learn entity representations before actual entity resolution is performed. This preliminary learning phase enables the model to achieve high matching precision without requiring seed alignments at runtime, saving significant time and expert resources
3Ease of manufacture
If regular expression and fuzzy string matching are used for entity resolution, then implementation is simple, but performance is poor on missing values or noisy data
Solution Approach 1:
The patent transforms entity resolution from simple string matching to a parameter-based neural network approach. The auto-encoder learns optimal parameter representations of entities and performs matching in this transformed parameter space, which is much more robust to missing values, noise, and variations than direct string comparison, while maintaining ease of implementation through standard deep learning frameworks
Data Source
AI summary
An auto-encoder model processes a datasets describing a physical part from a part catalogue in the form of a property co-occurrence graph is provided, and performs entity resolution and auto-completion on the co-occurrence graph in order to compute a corrected and/or completed dataset. The encoder includes a recurrent neural network and a graph attention network. The decoder contains a linear decoder for numeric values and a recurrent neural network decoder for strings. The auto-encoder model provides an automated end-to-end solution that can auto-complete missing information as well as correct data errors such as misspellings or wrong values. The auto-encoder model is capable of auto-completion for highly unaligned part specification data with missing values.