Semantic Query Feature Specification for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning approaches require users to specify columns for each feature and assume one row per sample, which becomes inefficient with large datasets, especially in life sciences where thousands of features are involved, leading to data complexity and challenges in analysis.
Innovation Solution
A method using semantic queries and entity feature maps to simplify feature specification, allowing users to select thousands of features in a few columns, and enabling automated detection of unique IDs and feature-measurement pairs, suitable for both supervised and unsupervised machine learning methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users specify one column for each feature and one row for each sample, then machine learning can be applied to data cubes and spreadsheets, but the approach scales poorly when input data has hundreds or thousands of features per row
Solution Approach 1:
The patent introduces an intermediary processing layer that automatically transforms complex data structures into machine learning-compatible formats. This intermediary system handles the conversion of nested data, hierarchical structures, and unstructured data into standardized feature matrices, eliminating the need for users to manually specify each column and row mapping.
Solution Approach 2:
The system performs self-service by automatically detecting data patterns, inferring feature relationships, and generating appropriate data transformations without requiring explicit user specification. The machine learning tool autonomously identifies suitable features and samples from the raw data structure.
2Adaptability or versatility
If the data is stored as thousands of measurement events associated with each sample, then life sciences data can be represented, but machine learning on the data sets becomes challenging
Solution Approach 1:
The patent segments the complex measurement event data into distinct hierarchical layers: sample-level metadata, feature-level measurements, and value-level observations. This segmentation allows the system to process different portions of the data structure independently and combine them into a unified machine learning format.
Solution Approach 2:
The system transforms the data from a complex multi-dimensional measurement event structure into a standardized two-dimensional feature matrix suitable for machine learning. This dimensionality transformation converts nested hierarchical data into flat tabular format while preserving all relevant information.
3Adaptability or versatility
If the data is sparse or the available features are not known a priori to the user, then real-world scenarios can be handled, but the traditional approach becomes inefficient
Solution Approach 1:
The system performs preliminary actions by automatically exploring and characterizing the data structure before machine learning model training. It pre-identifies potential features, detects data sparsity patterns, and prepares appropriate handling strategies in advance, eliminating the need for users to pre-specify features.
Solution Approach 2:
The system incorporates feedback mechanisms that automatically analyze data characteristics and adjust the data transformation process accordingly. When encountering sparse data or unknown features, the system receives feedback about data properties and dynamically modifies its transformation strategy to optimize for the specific data type.
Data Source
AI summary
Technology is described that includes a method of feature specification via semantic queries. The method can include the operation of obtaining a data set having an identifier for each data row and a plurality of data features for each data row. A semantic query can be received that can be applied to the dataset that is usable by a machine learning tool. A entity feature map can be supplied that has entities and associated features for use by the machine learning tool. Further, a query structure can be analyzed using the entity feature map to identify input from the dataset for the machine learning tool.