Context Profile Selection for Latency-Bounded Foundation Models

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional foundation models face limitations in providing specialized domain knowledge and require significant computing resources, leading to inefficiencies in computing environments with limited processing capacity and strict latency requirements.

Innovation Solution

A context generation system that iteratively analyzes multiple context profiles to determine the most relevant response within a latency budget, using a combination of text similarity metrics, vector embeddings, and plugins, and selectively employs a compressed foundation model to reduce resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional foundation models are used to process queries with specialized domain knowledge, then response accuracy is improved, but computing resource consumption increases significantly

Engineering Contradiction:

Improveresponse accuracyVSAvoidcomputing resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system segments the query processing into two distinct stages: first, a compressed foundation model generates an initial response using fewer computing resources; second, if the initial response does not meet the relevance threshold, a full foundation model is selectively applied to generate a refined response. This segmentation allows the system to achieve high response accuracy while minimizing computing resource consumption by using the resource-intensive full model only when necessary.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies partial action by using the compressed foundation model for the initial processing step, which consumes fewer resources than the full model. The full foundation model is then applied partially or excessively only when the initial response fails to meet the relevance threshold, thereby optimizing the balance between response accuracy and computing resource consumption.

Inventive Principle:

Principle #16Partial or excessive action

2Measurement precision

If context generation is performed to improve specialized knowledge accuracy, then response quality is improved, but processing time increases

Engineering Contradiction:

Improveresponse qualityVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary action by using the compressed foundation model to generate an initial response before applying the full foundation model. This preliminary step filters out queries that do not require extensive context generation, thereby reducing the overall processing time while maintaining response quality for queries that do need specialized knowledge.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by evaluating the relevance of the initial response generated by the compressed foundation model against a relevance threshold. Based on this feedback, the system determines whether to proceed with generating a refined response using the full foundation model, thereby optimizing processing time while ensuring response quality when necessary.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If multiple context profiles are analyzed to improve response relevance, then response accuracy is improved, but computing resource utilization increases

Engineering Contradiction:

Improveresponse relevanceVSAvoidresource utilization efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system segments the context profile analysis into two phases: first, the compressed foundation model analyzes context profiles to generate an initial response; second, if the initial response does not meet the relevance threshold, the full foundation model is selectively applied to perform a more thorough context profile analysis. This segmentation improves response relevance while maintaining efficient resource utilization by avoiding unnecessary full-model processing.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies partial action by using the compressed foundation model for the initial context profile analysis, which consumes fewer computing resources. The full foundation model is then applied partially or excessively only when the initial response fails to meet the relevance threshold, thereby optimizing the balance between response relevance and resource utilization efficiency.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12393618B2Generating and implementing context profiles in processing queries using foundation models

Publication Date: 2025.08.19 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A context analysis system receives a query from a user. The context analysis system generates one or multiple context profiles and generates a prompt for a foundation model for each of the context profiles. The context analysis system analyzes each of the context profiles and generates a relevancy score. The context analysis system selects one of the context profiles based on the relevancy score. In some examples, the context analysis system iteratively determines predicted latencies and relevancies of processing a query in conjunction with a generated context and, based on the predicted latencies and/or relevancies, processes the query using a foundation model, such as a large language model (LLM).