Tensor Decomposition for Handling Missing Data in ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning on data with varying formats and missing values faces challenges in maintaining distinction accuracy due to the introduction of 'NA' values, which can obscure the graph structure and reduce generalization in classification tasks.
Innovation Solution
A learning device generates integrated records with tensor decomposition, allowing for deep learning on unified data formats by appropriately complementing blank fields with 'NA' values, thereby improving generalization and maintaining classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If integrated records are generated by setting NA values for missing data fields, then data unification is achieved, but distinction accuracy degrades due to the introduction of NA as a new feature value
Solution Approach 1:
The patent extracts and removes NA values from the integrated records before performing machine learning. By eliminating these artificial feature values that indicate missing data, the system prevents NA from being interpreted as meaningful feature information, thereby maintaining distinction accuracy while still achieving data unification through the integrated record structure.
2Adaptability or versatility
If NA values are set in integrated records to handle missing data, then data integration is completed, but graph structure learning is obstructed by the added feature value
Solution Approach 1:
The patent removes NA values from the integrated records to prevent them from interfering with graph structure learning. By extracting these problematic feature values, the system maintains clean graph structures that accurately represent the underlying data relationships without being confounded by artificial missing data indicators.
3Adaptability or versatility
If conversion is performed to unify input data formats, then machine learning can be performed, but processing complexity increases
Solution Approach 1:
The patent performs preliminary integration of records with different formats into a unified integrated record structure before the machine learning process. This preliminary action consolidates format conversion and NA value handling into a single preprocessing step, reducing the overall processing complexity during the actual machine learning execution.
Data Source
AI summary
A non-transitory computer-readable recording medium stores a program that causes a computer to execute a process including: inputting input data including one or more records that have one of a plurality of formats, each of the plurality of formats including a plurality of items; generating conversion data by generating an integrated record having an integrated format from the one or more records; and causing a learner to execute a learning process using the conversion data as input tensor, the learner performing deep learning by performing tensor decomposition on input tensor.