AI Pipeline Rule Engine for Secure LLM Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprises face challenges in securely using large language models (LLMs) due to concerns over sensitive data exposure and inefficiencies in model routing, leading to hesitation in adopting AI technologies.
Innovation Solution
A rule engine that combines data loss prevention (DLP) and AI model routing, allowing enterprises to define granular management rules for secure AI service usage, with a gateway for easy integration, and a rules engine that applies these rules to inputs and outputs, routing queries to appropriate models based on context and sensitivity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If enterprises use third-party AI services, then AI functionality is improved, but sensitive data exposure risk increases
Solution Approach 1:
The patent introduces a gateway as an intermediary component between enterprise applications and AI service providers. This gateway enforces management rules that prevent sensitive data from being transmitted to external AI services, thereby enabling AI functionality while protecting against data exposure risks through the mediating layer
Solution Approach 2:
The system performs preliminary evaluation of inputs against management rules before they are sent to AI services. By proactively identifying and blocking sensitive data in advance, the system prevents data exposure before it can occur, rather than reacting after the fact
2Device complexity
If a single AI model is used, then device complexity is reduced, but productivity decreases due to inappropriate model handling
Solution Approach 1:
The patent implements dynamic model routing that automatically selects appropriate AI models based on the characteristics of each input and enterprise-defined rules. This dynamic approach allows the system to optimize query handling efficiency without requiring manual configuration or complex static routing logic
Solution Approach 2:
The gateway automatically evaluates inputs against management rules and routes queries to appropriate models without requiring manual intervention. The system self-manages the complexity of model selection and routing, improving productivity while keeping the user interface simple
3Reliability
If granular management rules are enforced, then data security is improved, but device complexity increases
Solution Approach 1:
The patent segments management rules into distinct, granular categories (e.g., PII detection, financial data protection, healthcare data rules) that can be independently configured and enforced. This segmentation allows enterprises to implement comprehensive security without overwhelming complexity, as each rule module handles a specific aspect of data protection
4Productivity
If AI services are adopted without restrictions, then productivity increases, but loss of information increases due to sensitive data leakage
Solution Approach 1:
The system implements feedback loops where outputs from AI services are evaluated against management rules before being returned to applications. This feedback mechanism ensures that sensitive information is not leaked in responses, while still allowing productive use of AI services through continuous monitoring and enforcement
Data Source
AI summary
The invention provides a rules engine that evaluates, modifies, and dynamically routes inputs to AI models based on provider restrictions and management rules. Provider restrictions are stored. Based on management rules and the detected provider restrictions, the input can be modified with contextual placeholders and prompts can be injected for input into the destination model. Reversal of the modifications can occur to the outputs. The system can notify the user, an administrator, and a supervisor regarding the security evaluation and remedial actions. The evaluations can be logged for auditing purposes.