A speech-text joint pre-training method and system
Through multiple iterative training and closed-loop speech chain mechanism, combined with the continuous cumulative distribution mechanism, the loss function is calculated using non-paired and paired data, the problem of insufficient data utilization and unmined alignment relationship in speech text joint pre-training is solved, and the problem of insufficient data utilization and unmined alignment relationship in speech text joint pre-training is improved. Model performance.
Patent Information
- Application Number
- CN202210346308.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-31
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-03-31
AI Technical Summary
The problems of insufficient utilization of non-paired data and insufficient mining of paired data alignment relationships in existing speech and text joint pre-training techniques.
The speech text joint pre-training method with multiple iteration training is adopted, combined with the closed-loop speech chain mechanism and the continuous accumulation distribution mechanism, and the initial, update and final loss totals are calculated through non-paired speech data, non-paired text data and paired speech text data. Function, train joint models to improve model performance.
It effectively solves the problems of insufficient utilization of non-paired data and insufficient learning of paired data alignment relationships, and improves the performance of speech text joint pre-trained models, especially in speech recognition and synthesis capabilities.
Smart Images

Figure CN114842834B_ABST
