Modular Computational Framework for Real-Time ML Inferencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current batch-based machine-learning and artificial-intelligence processes in financial institutions are inflexible and unable to support real-time inferencing requests, as they rely on static, hard-coded scripts that cannot transition from pre-scheduled inferencing to request-by-request and customer-by-customer scenarios without significant modifications.
Innovation Solution
A modular and configurable computational framework that allows real-time application of trained machine-learning or artificial-intelligence processes to input datasets through a generalized framework with customizable feature-engineering operations, executed by a distributed computing system without modifying underlying code, enabling real-time inferencing and output generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If batch-based machine-learning processes with static hard-coded scripts are used, then system stability and ease of maintenance are improved, but real-time inferencing capability and adaptability deteriorate
Solution Approach 1:
The system is segmented into distinct modular components: a framework layer that handles request routing and configuration management, feature engineering modules that can be dynamically selected and configured, and model inference components. This segmentation allows each module to be independently configured and modified without affecting the entire system, enabling real-time adaptability while maintaining overall system stability through standardized interfaces.
Solution Approach 2:
The system transitions from static hard-coded scripts to dynamic configuration-driven processing. Configuration files and metadata schemas allow the system behavior to be modified at runtime without code changes. The feature engineering pipeline can be dynamically adjusted based on input data characteristics and business requirements, enabling real-time adaptability while the core framework remains stable.
2Ease of operation
If static hard-coded scripts are used for batch processing, then ease of operation is improved, but productivity and response time deteriorate
Solution Approach 1:
The framework is designed as a universal platform that can handle multiple types of inferencing requests (batch, real-time, interactive) through a common architecture. The same core components serve different processing modes by accepting different configuration parameters, eliminating the need for separate codebases for each operation mode and improving both ease of operation and productivity.
Solution Approach 2:
The system uses parameter-driven configuration where processing behavior is controlled by configurable parameters rather than hard-coded logic. By changing parameters in configuration files or metadata schemas, the system can switch between batch and real-time processing modes, adjust feature engineering pipelines, and modify model inference parameters without altering the underlying code, thus improving productivity while maintaining ease of operation.
3Device complexity
If batch-based processes are used, then device complexity is reduced, but adaptability to customer-specific needs deteriorates
Solution Approach 1:
Configuration files and metadata schemas serve as intermediaries between the simple core framework and complex customer-specific requirements. These intermediaries allow customization of feature engineering operations, model selection, and processing parameters without modifying the framework code itself. This approach maintains low system complexity while enabling high adaptability to customer needs through configurable parameters and modular components.
Data Source
AI summary
The disclosed embodiments include modular and configurable computer-implemented processes and systems that facilitate real-time inferencing in distributed computing environments. For example, an apparatus may receive a real-time inferencing request from a computing system, and, based on the inferencing request, the apparatus may obtain configuration data associated with a trained machine-learning process. The apparatus may also perform operations that (i) generate the input dataset based on an application of one or more feature-engineering operations to the elements of payload data in accordance with a first portion of the configuration data, and (ii) generate output data based on the application, in real-time, of the trained machine-learning process to the input dataset. The apparatus may transmit, to the computing system, a response to the real-time inferencing request that includes at least at least a portion of the output data.