End-to-end speech input conversational large language model
By combining end-to-end processing of audio embedding and text embedding in the speech recognition system, the problem of direct audio processing in the prior art is solved, and more accurate and consistent speech response generation is achieved, especially when dealing with complex emotions and rare words.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- META PLATFORMS INC
- Filing Date
- 2024-11-08
- Publication Date
- 2026-06-26
AI Technical Summary
Existing speech recognition systems cannot directly process audio input, and large language models may produce outputs that are inconsistent with user preferences, resulting in hallucinations, useless or harmful content.
By receiving audio and generating audio embedding sequences, the system uses a trained Large Language Model (LLM) combined with text embedding sequences to generate text responses associated with the audio. The system can directly process audio input without intermediate steps of converting audio to text and uses conversation history to guide response generation.
It achieves end-to-end speech processing that directly handles audio input, reducing cascading errors and improving the accuracy and consistency of responses. It performs particularly well when processing difficult and rare words, reducing the error rate.
Smart Images
Figure CN122295718A_ABST