LLM Prompt Generation for Bias-Resistant IVA Stress Testing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for testing interactive voice assistants (IVAs) are inadequate due to inherent biases in engineers who design the systems, leading to insufficient stress testing and suboptimal user experiences.

Innovation Solution

Utilizing a large language model (LLM) to assume the persona of a naïve user and simulate service requests to test IVAs, logging interactions for analysis and debugging to improve system performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If engineers design and test IVAs themselves, then they understand the system capabilities, but they exhibit bias leading to insufficient stress testing

Engineering Contradiction:

Improvestress testing qualityVSAvoidengineer bias

Core Design Contradiction:

ReliabilityVSObject-generated harmful factors

Solution Approach 1:

The patent creates a simulated user persona that copies the characteristics of a naïve user, including limited knowledge of the IVA system and typical user questioning behavior. This persona generates test queries that reflect real user interactions rather than engineer-designed tests, thereby eliminating bias while maintaining comprehensive stress testing coverage.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The simulated user persona acts as an intermediary between the testing system and the IVA. This persona translates complex testing requirements into natural user queries and interprets IVA responses through the lens of a typical user, providing an unbiased assessment of system performance that neither the engineers nor the IVA itself could produce alone.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If comprehensive stress testing is performed, then system weaknesses are identified, but testing complexity and time increase

Engineering Contradiction:

ImproveIVA performanceVSAvoidtesting duration

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The simulated user persona performs self-service testing by autonomously generating queries, evaluating IVA responses, and identifying weaknesses without requiring manual intervention from engineers. The persona continuously refines its testing based on IVA performance, creating a self-directed stress testing process that is both comprehensive and efficient.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The testing process maintains continuous interaction between the simulated user persona and the IVA, with the persona adapting its queries based on previous responses. This continuous dialogue allows comprehensive stress testing to be performed efficiently, as the persona builds upon previous interactions rather than repeating tests independently, thereby reducing overall testing time while maintaining thoroughness.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS20250217657A1Prompt generator for testing

Publication Date: 2025.07.03 CX360 INC

AI summary

There is disclosed a computer-implemented method of training an interactive voice assistant (IVA), including prompting an artificial intelligence (AI) to assume a persona and to seek to fill a service request by a service provider that operates the IVA; receiving, from the AI, a query for the IVA; providing the query to the IVA; receiving, from the IVA, a response to the query; continuing to permit the AI to query the IVA until the AI is satisfied that the IVA has filled the service request or that the IVA is unable to fill the service request; logging the queries and responses between the AI and the IVA; and based on the logging, debugging or improving the IVA.