Multilingual Speech-to-Speech Translation With LLM Refinement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing multilingual speech-to-text translation models struggle with handling multiple languages concurrently and lack the capability to produce fluent or professional translations.

Innovation Solution

A customized prompt is generated for a selected Large Language Model (LLM) to perform idiomatic and polished translations by concatenating a system instruction, output language indication, and input content, enabling the LLM to handle multilingual inputs and produce natural, concise translations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional speech-to-text translation models are used, then they can handle single language input, but they fail to handle multiple languages mixed together

Engineering Contradiction:

Improvecapability to handle multiple languagesVSAvoidtranslation accuracy

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system segments the multilingual speech input by first transcribing it to text, then using an LLM to identify and separate different languages within the text, processing each language segment appropriately before generating the final translation output

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary LLM component that acts as a mediator between the speech-to-text transcription and the translation model. This intermediary identifies language boundaries and structures the multilingual text, enabling the translation model to handle mixed languages effectively

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If literal translation is provided, then translation speed is maintained, but the translation lacks fluency and professionalism

Engineering Contradiction:

Improvetranslation speedVSAvoidtranslation fluency

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The system incorporates feedback mechanisms where the LLM evaluates and refines the initial translation output, adjusting for fluency and professionalism while maintaining reasonable translation speed through iterative improvement

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20250335725A1System and method for multilingual speech-to-speech translation with speech refinement using combined machine learning models

Publication Date: 2025.10.30 SANAS AI INC

AI summary

Methods and systems are provided for multilingual idiomatic translation using large language model. In one novel aspect, customized prompt is generated for a selected large language model (LLM) to generate an idiomatic translation. In one embodiment, the input for the idiomatic translation is multilingual, which contains mixed multiple languages. In one embodiment, the computer system generates a customized prompt for a selected LLM, wherein the customized prompt concatenates a system instruction, an output language indication, and an input content, wherein the customized prompt is dynamically generated for an idiomatic translation of the text input. In one embodiment, the system instruction contains one or more elements comprising a direct instruction for multilingual detection for the input, a direct instruction for output text format, and an indication customized for translation. In another embodiment, the computer system performs an LLM selection procedure using an LLM selection prompt.