LLM Artifact Management With Verification Tests for GenAI Apps

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing software applications utilizing generative artificial intelligence (AI) lack the necessary capabilities for key users to quickly customize instructions to Large Language Models (LLMs) and adjust the scope of AI functionality to meet specific organizational or industry needs, leading to potential disruptions and inefficiencies due to rapid LLM changes and varying user inputs.

Innovation Solution

A system enabling key users to manage and customize LLM artifacts, including prompts, configurations, and verification tests, allowing for test-driven development and A/B testing to ensure consistent and optimized AI functionality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If key users are provided with capabilities to quickly customize instructions to LLMs and adjust AI functionality, then adaptability and responsiveness to organizational needs are improved, but device complexity and system configuration requirements increase

Engineering Contradiction:

Improveadaptability to organizational needsVSAvoidsystem configuration complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments AI functionality into modular artifacts (prompts, parameters, configurations) that can be independently customized and managed. This allows key users to adjust specific components without affecting the entire system, thereby improving adaptability while managing complexity through organized modularity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system enables key users to perform self-service customization of AI artifacts without requiring vendor intervention for every change. Users can directly modify prompts, adjust parameters, and reconfigure AI functionality according to organizational needs, reducing dependency on external support while maintaining system complexity through user-friendly interfaces.

Inventive Principle:

Principle #25Self-service

2Reliability

If verification tests are continuously executed to monitor LLM responses, then reliability and quality control are improved, but loss of time and computational resources increase

Engineering Contradiction:

Improvequality control of AI responsesVSAvoidtime for test execution

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent implements preliminary verification tests that are pre-configured and automatically executed when LLM responses are generated. This allows quality control to occur proactively before responses are finalized, ensuring reliability while minimizing time loss through automated, background test execution rather than manual verification.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system establishes continuous feedback loops where verification test results are automatically fed back to adjust and improve LLM responses. This automated feedback mechanism maintains high reliability by continuously monitoring quality while optimizing time efficiency through intelligent test selection and automated response adjustment based on test outcomes.

Inventive Principle:

Principle #23Feedback

3Ease of operation

If key users can independently manage and customize LLM artifacts, then ease of operation and responsiveness are improved, but manufacturing precision and consistency of AI functionality may deteriorate

Engineering Contradiction:

Improveease of customizationVSAvoidconsistency of AI functionality

Core Design Contradiction:

Ease of operationVSManufacturing precision

Solution Approach 1:

The patent creates universal artifact templates and standardized configurations that can be customized by key users while maintaining core consistency. These universal structures ensure that regardless of user customization, fundamental AI functionality remains precise and consistent across different organizational contexts, balancing ease of operation with manufacturing precision.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system introduces an intermediary layer of artifact management that sits between user customization actions and the underlying LLM functionality. This intermediary layer enforces consistency rules, validates customizations, and ensures that user-friendly customization does not compromise the precision and reliability of core AI operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20260030145A1Generative ai application artifact management enabling safeguarding by key users

Publication Date: 2026.01.29 SAP SE

AI summary

A computer-implemented method includes customizing artifacts of a large language model (LLM). A generative artificial intelligence (AI) (genAI) function of a software application that interfaces with the LLM and is instrumented for customer-side management is activated, as a genAI application. A verification test is activated to determine whether interactions between the LLM and genAI application generate expected results. Based on a result of the verification test, either adjusting the artifacts of the LLM or updating the genAI function of the software application. A new verification test is defined to determine whether interactions between the LLM and genAI application generate expected results. The new verification test is activated.