Spoken LLM Conversation Interface With Sentiment-Driven Voice Style

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing large language models primarily interact through text inputs, limiting user engagement and application potential, especially for users without a technical background, and restricting interactions to text-based contexts.

Innovation Solution

Implementing a natural language interface that enables spoken conversations with large language models by analyzing user speech input for sentiment, using a prompt engine to determine sentiment and style cues, and generating responses with appropriate emotional inflection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If text input interface is used for large language models, then interaction precision is maintained, but ease of operation deteriorates and adaptability is limited

Engineering Contradiction:

Improveease of operationVSAvoidadaptability

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system enables the large language model to accept multiple input modalities (text and speech) and produce multiple output modalities (text and speech), transforming a single-function text-based interface into a multi-functional communication system that adapts to different user preferences and contexts

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If text-only interface is implemented, then device complexity is reduced, but adaptability and user accessibility deteriorate

Engineering Contradiction:

ImproveadaptabilityVSAvoiddevice complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system introduces speech-to-text conversion and text-to-speech conversion as intermediary components that bridge the gap between speech input/output and the large language model's text-based processing, enabling speech interaction without requiring fundamental changes to the model's core architecture

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The interaction system is divided into distinct functional modules: speech-to-text conversion module, large language model processing module, and text-to-speech conversion module, allowing each component to be optimized independently while maintaining overall system adaptability

Inventive Principle:

Principle #1Segmentation

3Adaptability or versatility

If speech input is added to large language model interface, then ease of operation and adaptability improve, but device complexity increases

Engineering Contradiction:

ImproveadaptabilityVSAvoiddevice complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250363978A1Real-time system for spoken natural stylistic conversations with large language models

Publication Date: 2025.11.27 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

The techniques disclosed herein enable systems for spoken natural stylistic conversations with large language models. In contrast to many existing modalities for interacting with large language models that are limited to text, the techniques presented herein enable users to carry a fully spoken conversation with a large language model. This is accomplished by converting a user speech audio input to text and utilizing a prompt engine to analyze a sentiment expressed by the user. A large language model, having been trained on example conversations, by generating a text response as well as a style cue to express emotion in response to the sentiment expressed by speech audio input. A text-to-speech engine can subsequently interpret the text response and style cue to generate an audio output which emulates the sensation of human conversation.