Speech Translation Continuous Decoding for Higher Prediction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The prediction performance of current end-to-end speech translation models does not meet desired requirements, and existing technologies face issues such as long delay, parameter redundancy, error accumulation, and speech feature loss.
Innovation Solution
A speech translation method involving an encoder and decoder mechanism that extracts semantic features, decodes source language text, and splits the sequence to obtain a target language text, using a continuous decoding mechanism that first predicts a simple source language text and then predicts a complex target language text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a traditional end-to-end speech translation model is used, then the system structure is simple, but the prediction performance does not meet desired requirements
Solution Approach 1:
The patent segments the speech translation task into two distinct decoding processes: first decoding the source language text from semantic features, then decoding the target language text from the source language text. This segmentation allows each decoding stage to be optimized independently, improving overall prediction performance while maintaining a manageable system structure through modular design.
2Manufacturing precision
If direct decoding from semantic feature to target language text is performed, then the decoding process is simple, but errors accumulate and prediction performance deteriorates
Solution Approach 1:
The patent applies preliminary action by first decoding the source language text from semantic features before proceeding to decode the target language text. This intermediate step establishes an accurate reference that guides the subsequent target language decoding, preventing error accumulation that would occur in direct decoding approaches.
Solution Approach 2:
The source language text serves as an intermediary between the semantic features and the target language text. This intermediary structure allows the model to maintain accurate semantic representations while facilitating accurate translation, as the source text provides a structured bridge that prevents direct error propagation from semantic features to target text.
3Productivity
If the model directly maps speech to target language text, then the translation process is efficient, but speech feature loss occurs
Solution Approach 1:
The patent extracts the source language text as a separate intermediate representation from the semantic features before generating the target language text. This extraction process preserves critical speech features and semantic information that would otherwise be lost in direct mapping, while maintaining translation efficiency through the structured two-stage decoding approach.
Data Source
AI summary
Provided are a speech translation method, a device, and a storage medium. The method includes: extracting, through an encoder of an end-to-end speech translation model, the semantic feature of a to-be-processed speech; decoding, through a decoder of the end-to-end speech translation model, a source language text corresponding to the semantic feature from the semantic feature; decoding, through the decoder of the end-to-end speech translation model, the semantic feature according to the source language text to obtain a text sequence corresponding to the semantic feature; and splitting the text sequence to obtain a target language text corresponding to the to-be-processed speech.