Automated Feature Engineering System for Real-Time ML Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning algorithms face challenges in generating accurate event-based predictions due to the difficulty in computing event-based feature vectors, which requires accessing event-based data that is often not readily available, and the labor-intensive process of creating these features, leading to inefficiencies in both training and application stages.
Innovation Solution
A feature engineering system that ingests data from various sources, computes features in real-time, and allows users to define feature selection and configuration through a user-friendly interface, enabling the generation of event-based feature vectors and training examples at arbitrary points in time, thereby facilitating accurate point-in-time predictions and scalable model implementation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If event-based feature vectors are computed manually, then prediction accuracy can be improved, but the process becomes labor-intensive and inefficient
Solution Approach 1:
The system enables automated self-service feature generation where the computing device automatically computes event-based feature vectors without requiring manual data extraction or complex coding by users. The system handles the entire process from event data ingestion to feature vector generation autonomously
Solution Approach 2:
The system performs preliminary computation of event-based feature vectors in advance, storing them for later use in machine learning models. This pre-computation approach eliminates the need for manual feature engineering at the time of model training or prediction
2Measurement precision
If event-based data is accessed for feature computation, then accurate feature vectors can be generated, but data availability becomes a limiting factor
Solution Approach 1:
The system acts as an intermediary between event data sources and machine learning models, automatically accessing and processing event-based data. It bridges the gap between unavailable raw event data and the need for accurate feature vectors by performing automated data retrieval and computation
3Adaptability or versatility
If complex coding is required for feature creation, then customized features can be generated, but the process becomes difficult to implement and maintain
Solution Approach 1:
The system provides self-service capabilities that allow users to define custom features through simple configurations rather than complex coding. Users can specify feature parameters and the system automatically generates the corresponding feature vectors without requiring programming expertise
Data Source
AI summary
A system for generating machine learning feature vectors or examples is disclosed herein. The system comprises at least one database configured to store data indicative of events associated with a plurality of entities, an application programming interface (API) server configured to receive a user query from at least one user device, and at least one computing node in communication with the API server and the at least one database. The at least one computing node is configured at least to receive, from the API server and at a first time, a first indication of the user query. The at least one computing node is configured to generate, based at least on the data indicative of events and the first indication of the user query, results associated with the user query, wherein the results comprise one or more feature vectors or examples for use with a machine learning algorithm. The at least one computing node is configured to cause storage of data indicative of the results in the at least one database.