ML Inference Framework for Cross-Language Model Deployment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The deployment of machine learning models in production environments is challenging due to differences in programming languages, leading to increased development time, resource usage, and compatibility issues.

Innovation Solution

A system and method for real-time selection and deployment of machine learning models using a machine learning model framework, which includes receiving requests, selecting appropriate models, transforming data configurations, and generating inferences, thereby minimizing language discrepancies and reducing development and redesign time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If machine learning models are deployed into production environment with different programming languages, then the models can generate inferences in production, but compatibility issues arise requiring further development work and redesign

Engineering Contradiction:

Improvemodel accuracyVSAvoiddevelopment work

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary layer (inference service system with translation framework) between the machine learning model and production environment. This intermediary handles the programming language translation automatically, eliminating the need for manual development work and redesign when deploying models to different production environments.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The inference service system is designed with universal compatibility to work with multiple programming languages and production environments. The system can automatically adapt to different target environments without requiring specific customization, making the deployment process language-agnostic and environment-independent.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Reliability

If machine learning models undergo further development work and redesign for compatibility, then accuracy can be maintained, but deployment time increases

Engineering Contradiction:

Improvemodel accuracyVSAvoiddeployment time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary translation and compatibility preparation automatically during the deployment process. By pre-configuring the translation framework and having it ready to automatically translate models to target programming languages, the system eliminates the need for time-consuming manual development work and redesign during deployment.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces manual mechanical development work and redesign processes with an automated translation framework. The system automatically translates machine learning models from one programming language to another, substituting the need for manual coding and adaptation work, thereby significantly reducing deployment time while maintaining accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If manual development work and redesign are performed for language compatibility, then model accuracy can be ensured, but computing resources are tied up

Engineering Contradiction:

Improvemodel accuracyVSAvoidcomputing resources

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The translation framework is designed to be self-service, automatically translating machine learning models to target programming languages without requiring manual intervention. The system self-manages the translation process, including loading models, translating code, and deploying to target environments, thereby freeing up computing resources that would otherwise be consumed by manual development work.

Inventive Principle:

Principle #25Self-service

4Productivity

If a generic machine learning model framework is implemented for real-time selection and deployment, then deployment time and computing resources are reduced, but system complexity increases

Engineering Contradiction:

Improvedeployment speedVSAvoidframework complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the inference service system into distinct modular components: model loading module, translation framework, and deployment module. Each component has a specific function and can be independently managed. This segmentation reduces the perceived complexity by organizing the system into manageable, well-defined parts that work together seamlessly.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12572831B2System, method, and computer program product for generating an inference using a machine learning model framework

Publication Date: 2026.03.10 VISA INTERNATIONAL SERVICE ASSOCIATION

AI summary

Provided is a system for generating an inference based on real-time selection of a machine learning model using a machine learning model framework that includes at least one processor programmed or configured to receive a request for inference, wherein the request includes a payload, select a machine learning model of a plurality of machine learning models based on the request for inference, determine an aggregation of data based on the machine learning model and the payload of the request, transform the aggregation of data into inference data, wherein the inference data has a configuration that is capable of being processed by the machine learning model, and generate an inference based on the inference data using the machine learning model. Methods and computer program products are also provided.