LLM Endpoint Routing for Ambiguous Backend Service Requests

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems face challenges in effectively handling ambiguous natural language user requests due to hard-coded communication methods between services, which restrict user interactions and are impractical for diverse queries, leading to inefficient and time-consuming coding.

Innovation Solution

A routing service utilizes a large language model (LLM) to translate user queries into proper requests, routing them to appropriate backend services, and employs a request processor to generate prompts for the LLM to ensure accurate responses, including schema-based prompts to prevent hallucinations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If hard-coded communication methods are used between services, then service interaction is deterministic and reliable, but user interaction flexibility is restricted and the system cannot handle ambiguous natural language requests

Engineering Contradiction:

Improveuser interaction flexibilityVSAvoidcoding complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a natural language processing intermediary layer between the user and the service system. This intermediary translates ambiguous natural language requests into structured service calls, allowing flexible user interaction without requiring hard-coded handling for each possible request format. The intermediary acts as a mediator that converts unstructured input into actionable service parameters.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes the parameter representation from rigid structured formats to flexible natural language. By accepting variable parameter formats (natural language) and transforming them into standardized service parameters dynamically, the system achieves both flexibility in user interaction and reliability in service execution without hard-coding each scenario.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If hard-coded communication methods are used for all potential user requests, then service interactions are reliable, but the system becomes impractical due to the vast and ambiguous nature of natural language

Engineering Contradiction:

Improveservice interaction reliabilityVSAvoidhandling of ambiguous natural language

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The natural language processing intermediary preserves reliability by ensuring that regardless of how ambiguous the user input is, it gets translated into a standardized, reliable service call format. The intermediary maintains the reliability of service execution while adapting to handle any natural language variation.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system uses self-service through automated natural language processing instead of requiring pre-programmed handling for every possible request. The NLP system automatically adapts to new types of requests without requiring manual hard-coding, maintaining reliability through consistent translation to standardized service parameters.

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If every potential use case is hard-coded, then all user requests can be handled, but the development time and effort become excessive and impractical

Engineering Contradiction:

Improvecoverage of user requestsVSAvoiddevelopment efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The NLP system provides self-service by automatically learning and adapting to handle new types of user requests without requiring manual intervention to hard-code each use case. This dramatically improves development efficiency while maintaining comprehensive coverage of user requests.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent implements a universal NLP layer that handles multiple types of requests through a single flexible mechanism. Instead of creating separate hard-coded paths for each use case, one universal NLP system handles diverse request types, improving productivity while maintaining broad adaptability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Ease of operation

If hard-coded communication methods are used, then service interactions follow predefined ways, but user options are restricted

Engineering Contradiction:

Improveease of user interactionVSAvoiduser interaction options

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system changes the interaction paradigm from fixed predefined options to flexible natural language parameters. Users can interact in their own words rather than being constrained to predefined commands, greatly improving ease of operation while the NLP layer ensures reliable translation to service parameters.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20260064670A1Natural language endpoint manipulation with a large language model

Publication Date: 2026.03.05 PALO ALTO NETWORKS INC

AI summary

A user query that includes a natural language description for access to a backend service of a plurality of backend services is received from a client device. The user query is routed to the backend service based in part on an large language model understanding of the natural language description. A response is received from the backend service. The response is provided to the client device.