Speech recognition method and system based on large model and speech synthesis engine

Abstract

The invention relates to the technical field of speech recognition, provides a speech recognition method and system based on a large model and a speech synthesis engine, and greatly improves the recognition accuracy of ASR recognition by combining a large language model language with a new generation of speech synthesis engine. A large language model is utilized to generate a large batch of high-quality corpora, and a speech synthesis engine is utilized to obtain a large amount of high-quality speech training data for training an automatic speech recognition model and checking and correcting a recognition result at the same time. A training closed loop of generating high-quality text training data, synthesizing natural speech, recognizing natural speech, finding errors, correcting the errors and generating training data is realized, so that an automatic speech recognition model can continuously, professionally and automatically reinforce learning at low cost; and furthermore, a high-accuracy and high-quality speech recognition result is output under semantic errors and complex scenes.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and more specifically, to a speech recognition method and system based on a large model and a speech synthesis engine. Background Technology

[0002] Automatic Speech Recognition (ASR) technology, as a key entry point for human-computer interaction, has been widely applied in many fields such as voice assistants, real-time captioning, conference transcription, intelligent customer service, and audio and video content production. However, despite decades of development, traditional ASR systems still face a series of severe challenges in practical implementation that have not been completely resolved, seriously restricting the improvement of user experience and the expansion of application boundaries.

[0003] Traditional ASR systems suffer from the following common problems: (1) their recognition accuracy drops significantly in complex scenarios such as accents, dialects, technical terms, mixed Chinese and English, background noise, and multi-person conversations; (2) they rely heavily on acoustic models and language models with limited context, making it difficult to effectively distinguish homophones and near-homophones, leading to semantic errors; (3) their model optimization loop is highly dependent on manpower and has high annotation and training costs; (4) their model response is slow and iteration cycles are long for newly emerging online terms, personal names, place names, product names, and other new words; and (5) they lack high-quality speech that matches industry terminology and language style. Therefore, traditional ASR optimization models are trapped in a dilemma of "high investment, long cycle, and slow results," especially for semantic errors and complex scenarios, where the recognition accuracy of traditional SR systems urgently needs to be significantly improved. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention provides a speech recognition method and system based on a large language model and a speech synthesis engine. By using a large language model (LLM) combined with a new generation of text-to-speech (TTS) engine, the recognition accuracy of ASR can be significantly improved.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A speech recognition method based on a large model and a speech synthesis engine includes the following steps: S100, deploy local large language models, speech synthesis engines and automatic speech recognition models; S200: Obtain the first training data and input it into the large language model to obtain the first generated text; S300: Input the first generated text into the speech synthesis engine to generate a synthesized speech stream based on context awareness and semantic disambiguation technology; S400. The first generated text is used as the label of the corresponding speech sample in the synthesized speech stream to form the second training data, and is input into the automatic speech recognition model for speech recognition training to obtain the recognized text. S500. Verify the recognized text based on the second training data: If there is an incorrect recognition, the incorrect recognition is labeled for training, and the corresponding second training data is re-input into the automatic speech recognition model for speech recognition training, and step S500 is repeated; if there is no incorrect recognition, steps S200 to S500 are repeated until the number of iterations reaches a preset threshold. S600. Input the speech stream to be recognized into the automatic speech recognition model to obtain the speech recognition result.

[0006] As a preferred embodiment, the speech synthesis engine generates synthesized speech streams based on context-aware and semantic disambiguation technologies, including the following steps: S310. The first generated text input is segmented and annotated with pinyin. S320. Based on context-aware and semantic disambiguation technology, perform semantic disambiguation on word segmentation results and / or phonetic transcription; S330. The first generated text after semantic disambiguation is converted into the target speech waveform to obtain a synthesized speech stream.

[0007] As a preferred embodiment, converting the semantically disambiguated first generated text into a target speech waveform includes the following steps: S331. Perform lexical analysis, syntactic structure recognition, and prosodic boundary prediction on the first generated text after semantic disambiguation to generate structured text features; S332, Retrieve a timbre sample from the timbre pool; S333, Randomly generate synthesis parameters; the synthesis parameters include one or more of accent, speech rate, and signal-to-noise ratio; S334. Based on the structured text features, timbre samples, and synthesis parameters, the speech synthesis engine predicts the spectral features of the target speech based on a neural network acoustic model; S335. The spectral features are converted into target speech waveforms using a neural vocoder, and the naturalness of the speech is further enhanced based on reverberation, timbre control and prosody adjustment to generate a synthesized speech stream that conforms to the characteristics of real speech.

[0008] As a preferred embodiment, the large language model generates a first generated text based on the first training text, and further includes the following steps: Obtain external input commands; Based on the input instructions and the first training text, the large language model is trained a second time; or... Based on the input instructions and the first training text, the first generated text is generated a second time.

[0009] As a preferred embodiment, the verification of the recognized text based on the second training data includes the following steps: S510, Recognition trigger, the second training data with the first generated text as the label is used as the recognition verification input; S520. Perform character-level, word-level, and / or phoneme-level difference analysis on the recognition and verification input, calculate the recognition error index by replacing, inserting, and / or deleting difference types, and determine whether there is a recognition error: if there is a recognition error, proceed to step S530; otherwise, proceed to step S560. S530. Construct training annotations for speech samples with identification errors, wherein the training annotations include error location annotations, error type annotations, and speech-text alignment information; S540. Input the second training data with the added training annotations into the training module of the automatic speech recognition model, and repeatedly execute the incremental training or fine-tuning process to update the parameters of the acoustic model, language model or end-to-end recognition model in the automatic speech recognition model to correct the errors that occurred in the previous round of recognition; when the recognized text is consistent with the annotated text, the system determines that the training and verification of this round has converged and executes step S550; otherwise, return to step S520. S550. Increase the iteration count. When the cumulative number of iterations reaches the preset threshold, stop the loop and execute step S600; if the threshold is not reached, return to step S200.

[0010] As a preferred embodiment, the method further includes the following steps: The first generated text volume of the large language model, and / or the recognition accuracy, training parameters, number of iterations and number of verifications of the automatic speech recognition model are monitored, and preset warning thresholds are configured for each; when there is an object that exceeds the corresponding warning threshold, a warning signal is output.

[0011] Furthermore, this invention also proposes a speech recognition system based on a large model and a speech synthesis engine, applied to the speech recognition method proposed in this invention. This system includes: The corpus generation module is configured to generate the first generated text using a pre-trained large language model and the first training text it introduces. The speech synthesis module is configured to use a pre-trained speech synthesis engine to take the first generated text as a label for the synthesized speech stream to form second training data. The speech recognition module is configured to train the second training data using an automatic speech recognition model to obtain the recognized text; and is configured to perform speech recognition on the input speech stream to be recognized and output the speech recognition result. The verification module is configured to verify the recognized text using the second training data. If a recognition error exists, the error recognition is used for training annotation, and the corresponding second training data is re-inputted into the automatic speech recognition model for speech recognition training.

[0012] As a preferred embodiment, the system further includes: a monitoring module configured to monitor the first generated text volume of the corpus generation module, the recognition accuracy, training parameters, and iteration count of the speech recognition module, and / or the verification count of the verification module, and configure preset warning thresholds for each; when there is an object exceeding the corresponding warning threshold, a warning signal is output.

[0013] Furthermore, the present invention also proposes an apparatus comprising a memory and a processor, wherein the memory stores computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the processor causes the processor to perform all or part of the steps of the speech recognition method proposed in the present invention.

[0014] Furthermore, the present invention also proposes a storage medium storing computer-readable instructions thereon, wherein the computer-readable instructions, when executed by a processor, implement all or part of the steps of the speech recognition method proposed in the present invention.

[0015] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention utilizes a large language model to generate a large amount of high-quality corpus and a speech synthesis engine to obtain a large amount of high-quality speech training data for training an automatic speech recognition model. At the same time, it verifies and corrects the recognition results, realizing a training closed loop of "generating high-quality text training data - synthesizing natural speech - recognizing natural speech - discovering and correcting errors - generating training data". This enables the automatic speech recognition model to continuously, cost-effectively, and professionally perform automated reinforcement learning, and further achieve high accuracy and high quality speech recognition results in the face of semantic errors and complex scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a speech recognition method according to an embodiment of the present invention.

[0017] Figure 2 This is an architectural diagram of a speech recognition system according to an embodiment of the present invention.

[0018] Figure 3This is an architecture diagram of a seat assistance system that applies a speech recognition system according to an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0022] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Example

[0023] This embodiment proposes a speech recognition method based on a large model and a speech synthesis engine, such as... Figure 1 The diagram shown is a flowchart of the speech recognition method in this embodiment.

[0024] The speech recognition method proposed in this embodiment includes the following steps: S100, deploy local large language models, speech synthesis engines and automatic speech recognition models; S200: Obtain the first training data and input it into the large language model to obtain the first generated text; S300: Input the first generated text into the speech synthesis engine to generate a synthesized speech stream based on context awareness and semantic disambiguation technology; S400. The first generated text is used as the label of the corresponding speech sample in the synthesized speech stream to form the second training data, and is input into the automatic speech recognition model for speech recognition training to obtain the recognized text. S500. Verify the recognized text based on the second training data: If there is an incorrect recognition, the incorrect recognition is labeled for training, and the corresponding second training data is re-input into the automatic speech recognition model for speech recognition training, and step S500 is repeated; if there is no incorrect recognition, steps S200 to S500 are repeated until the number of iterations reaches a preset threshold. S600. Input the speech stream to be recognized into the automatic speech recognition model to obtain the speech recognition result.

[0025] This embodiment uses a large language model to generate initial text training data, namely, the first generated text. By connecting the large language model to a speech synthesis engine interface, a large-scale, high-quality speech recognition training corpus that conforms to industry context and covers professional terminology and common expressions can be obtained, namely, the second training data. This second training data includes a large number of synthesized speech stream samples labeled with the first generated text. When the large language model is deployed, it also includes deep understanding and training using the acquired large-scale target industry text, namely, the first training data. During its training process, the large language model learns from massive amounts of text data, with extensive coverage of new words, hot words, professional terms, and entity names. Furthermore, by utilizing the generalization and deep learning capabilities of the large language model, it can correct and refine the acquired text data. Using the above-mentioned large language model to train an automatic speech recognition model can effectively compensate for the shortcomings of traditional automatic speech recognition models in terms of slow knowledge updates. Thus, this embodiment realizes a training closed loop of "generating high-quality text training data - synthesizing natural speech - recognizing natural speech - discovering and correcting errors - generating training data", enabling the automatic speech recognition model to continuously, cost-effectively, and professionally perform automated reinforcement learning.

[0026] For example, in the specific implementation process, the first training data in the corresponding field can be obtained according to the application scenario, including: knowledge base, business system name, business processing flow, user company name, historical work order text, etc. A large language model is used to deeply understand industry knowledge, generating massive amounts of high-quality text corpora that conform to industry context and cover professional terminology and common expressions.

[0027] In one alternative implementation, the large language model can be retrained and / or regenerated using instruction tuning to adapt the first generated text to a specific task or human instruction. The specific steps include: Obtain external input commands; Based on the input instructions and the first training text, the large language model is trained a second time; or... Based on the input instructions and the first training text, the first generated text is generated a second time.

[0028] In the above optional implementation, by creating data in the format of "instruction-input (optional)-output", the pre-trained large language model is retrained so that it can understand and execute natural language instructions, rather than relying solely on the text completion ability in the pre-training stage.

[0029] In one alternative implementation, the large language model can obtain a more desirable first generated text through prompt engineering. For example, by structuring the text, the large language model can be guided to output the desired result to accomplish different types of tasks without updating the model weights.

[0030] In one optional implementation, the speech synthesis engine employs context-aware and semantic disambiguation techniques when generating synthesized speech streams, enabling it to intelligently identify and accurately handle pronunciation differences of homophones in different words and contexts. The specific steps include: S310. The first generated text input is segmented and annotated with pinyin. S320. Based on context-aware and semantic disambiguation technology, perform speech disambiguation on word segmentation results and / or phonetic transcription; S330. The first generated text after speech disambiguation is converted into the target speech waveform to obtain a synthesized speech stream.

[0031] In the above optional implementation, by deeply analyzing the semantics, grammatical roles and contextual relationships of target words, it is possible to identify the pronunciation of polyphonic characters, thereby generating natural speech that highly conforms to language habits and contextual requirements. In specific scenarios, the accuracy and reliability of synthesized speech can reach over 99%.

[0032] As an example, speech synthesis technology is an artificial intelligence technology that can convert any text into natural and fluent speech in real time. It achieves "making machines speak" by simulating human vocal organs or training on large-scale corpora. This includes processes such as text processing, acoustic feature conversion, and vocode conversion, ultimately realizing the function of converting text into speech streams. It is particularly suitable for fields such as accessible reading, intelligent customer service, navigation, education, and healthcare.

[0033] Furthermore, in an optional implementation, when converting the first generated text after speech disambiguation into the target speech waveform, it is possible to use speaker models of different ages, genders, and timbres for synthesis to simulate a real user group. Additionally, various accents, speech rates, noise, and other variables are gradually and selectively added to obtain high-quality speech recognition training data, i.e., the second training data. The specific steps include: S331. Perform lexical analysis, syntactic structure recognition, and prosodic boundary prediction on the first generated text after semantic disambiguation to generate structured text features; S332, Retrieve a timbre sample from the timbre pool; S333, Randomly generate synthesis parameters; the synthesis parameters include one or more of accent, speech rate, and signal-to-noise ratio; S334. Based on the structured text features, timbre samples, and synthesis parameters, the speech synthesis engine predicts the spectral features of the target speech based on a neural network acoustic model; S335. The spectral features are converted into target speech waveforms using a neural vocoder, and the naturalness of the speech is further enhanced based on reverberation, timbre control and prosody adjustment to generate a synthesized speech stream that conforms to the characteristics of real speech.

[0034] In the above optional implementation, the speech recognition training corpus does not require manual identification and annotation, which can effectively improve training efficiency and reduce training costs.

[0035] In one optional implementation, after the automatic speech recognition model completes speech recognition, it further includes verification and correction of the recognized text to improve the recognition accuracy of the automatic speech recognition model. The specific steps include: S510, Recognition trigger, the second training data with the first generated text as the label is used as the recognition verification input; S520. Perform character-level, word-level, and / or phoneme-level difference analysis on the recognition and verification input, calculate the recognition error index by replacing, inserting, and / or deleting difference types, and determine whether there is a recognition error: if there is a recognition error, proceed to step S530; otherwise, proceed to step S560. S530. Construct training annotations for speech samples with identification errors, wherein the training annotations include error location annotations, error type annotations, and speech-text alignment information; S540. Input the second training data with the added training annotations into the training module of the automatic speech recognition model, and repeatedly execute the incremental training or fine-tuning process to update the parameters of the acoustic model, language model or end-to-end recognition model in the automatic speech recognition model to correct the errors that occurred in the previous round of recognition; when the recognized text is consistent with the annotated text, the system determines that the training and verification of this round has converged and executes step S550; otherwise, return to step S520. S550. Increase the iteration count. When the cumulative number of iterations reaches the preset threshold, stop the loop and execute step S600; if the threshold is not reached, return to step S200.

[0036] In the above optional implementation, the automatic speech recognition model not only performs recognition training on the second training data containing the training corpus, but also performs a review and verification of speech recognition, and performs a second review and recognition for the results of recognition errors, thereby achieving a functional closed loop of "error detection - automatic learning - model improvement".

[0037] As an example, a typical automatic speech recognition model includes modules such as an acoustic model, a language model, and a decoder. The acoustic model converts speech signals into probability distributions of phonemes or word units; the language model uses statistical methods to predict the probability and semantics of word sequences to understand the context and improve recognition accuracy; the decoder, or recognition model, combines the outputs of the acoustic and language models to generate the most probable recognized text. Specifically, the automatic speech recognition model in this embodiment also includes a training module for iterative incremental training or model parameter fine-tuning based on recognition verification results to correct erroneous recognitions and further improve model recognition accuracy.

[0038] Furthermore, in an optional implementation, the method further includes monitoring each node of the automatic speech recognition model's training process and setting up anomaly warnings. The specific steps include: The first generated text volume of the large language model, and / or the recognition accuracy, training parameters, number of iterations and number of verifications of the automatic speech recognition model are monitored, and preset warning thresholds are configured for each; when there is an object that exceeds the corresponding warning threshold, a warning signal is output.

[0039] In the above optional implementation, the monitored data includes, but is not limited to, the amount of data generated, the recognition accuracy, the training status, the verification status, etc. The corresponding monitoring objects can also be set according to the application scenario and task requirements, and corresponding warning thresholds can be set for different monitoring objects. Example

[0040] This embodiment proposes a speech recognition system based on a large model and a speech synthesis engine, applying the speech recognition method proposed in Embodiment 1 above. For example... Figure 2 The diagram shown is an architecture diagram of the speech recognition system in this embodiment.

[0041] The speech recognition system based on a large model and a speech synthesis engine proposed in this embodiment includes: The corpus generation module is configured to generate the first generated text using a pre-trained large language model and the first training text it introduces. The speech synthesis module is configured to use a pre-trained speech synthesis engine to take the first generated text as a label for the synthesized speech stream to form second training data. The speech recognition module is configured to train the second training data using an automatic speech recognition model to obtain the recognized text; and is configured to perform speech recognition on the input speech stream to be recognized and output the speech recognition result. The verification module is configured to verify the recognized text using the second training data. If a recognition error exists, the error recognition is used for training annotation, and the corresponding second training data is re-inputted into the automatic speech recognition model for speech recognition training.

[0042] In the speech recognition system proposed in this embodiment, a training module for the automatic speech recognition model is constructed by configuring a large language model and a speech synthesis engine to form a training closed loop of "high-quality text - natural speech generation - speech recognition training - proofreading - secondary training".

[0043] In an optional embodiment, the speech recognition system further includes a monitoring module configured to monitor the first generated text volume of the corpus generation module, the recognition accuracy, training parameters, and iteration count of the speech recognition module, and / or the verification count of the verification module, and configure preset warning thresholds for each; when there is an object exceeding the corresponding warning threshold, a warning signal is output.

[0044] The speech recognition model proposed in this embodiment will become a unique and continuously value-added data asset for enterprises, forming a strong competitive moat and creating a technological barrier in the industry. More accurate and faster automatic speech recognition capabilities make it possible to implement applications in more niche scenarios that were previously inaccessible due to accuracy issues, thereby empowering business innovation and greatly expanding business boundaries. The following will provide illustrative examples of applicable application scenarios.

[0045] Please refer to Figure 3 An exemplary agent assistance system applying the AusunASR speech recognition model proposed in this embodiment is provided, comprising: (1) Knowledge Q&A Module: Based on the business knowledge base, the large model LLM is used to understand and analyze the questions of users or agents, which can more accurately locate the questions and answer them with smooth language organized in the professional knowledge base, thereby improving the user experience.

[0046] (2) Intelligent IVR Acceptance Module: By combining the large model LLM with the AusunASR model, the intelligent IVR automatically understands the customer's intent through natural voice dialogue and completes the creation, classification and initial information filling of work orders by itself, which greatly saves the manpower cost of the initial agent communication.

[0047] (3) Long document summary module: The long text summary function provides core tool support for enterprises to quickly process a large number of internal documents (such as project reports and meeting minutes) and build a knowledge base.

[0048] (4) Dialogue summary and automatic work order generation module: By combining the large model LLM with the high-quality AusunASR model, a clear and complete dialogue summary is automatically generated after the call or chat ends. The summary includes user problems, solutions, and to-do items. The large model LLM automatically extracts key information from the dialogue (such as user information, problem classification, urgency) and pre-populates it into the customer service work order system. Agents only need to confirm to submit, saving a lot of paperwork time.

[0049] (5) Emotion Recognition and Early Warning Module: By combining the large model LLM with the AusunASR model, the module analyzes the user's text emotions (such as anger, anxiety, satisfaction) in real time. When negative emotions are detected to be escalating, an early warning is issued to the human agent in a timely manner, and a soothing message is recommended or the user is advised to transfer to a senior customer service representative, which helps to prevent complaints from escalating.

[0050] (6) Full-volume dialogue quality inspection module: Automatically inspects all customer service dialogues (after voice-to-text conversion) using the large model LLM, analyzes whether there are problems such as service violations, non-standard language, and incorrect answers, and generates a quality inspection report.

[0051] (7) Customer feedback module: Through the large model LLM, it automatically analyzes massive dialogue data, clusters and summarizes the issues that users care about most and complain about most recently, and provides data insights for product improvement, operation strategy and knowledge base optimization. Example

[0052] This embodiment proposes a device including a memory and a processor. The memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform all or part of the steps of the speech recognition method proposed in Embodiment 1. Example

[0053] This embodiment proposes a storage medium storing computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, they implement all or part of the steps of the speech recognition method proposed in Embodiment 1.

[0054] By way of example, the storage medium includes, but is not limited to, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media capable of storing program code.

[0055] By way of example, the instructions, programs, code sets, or instruction sets may be implemented using conventional programming languages.

[0056] By way of example, the processor includes, but is not limited to, smartphones, personal computers, servers, network devices, etc., for performing all or part of the steps of the speech recognition method described in Example 1.

[0057] The terminology used in the accompanying drawings is for illustrative purposes only and should not be construed as limiting the invention. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A speech recognition method based on a large model and a speech synthesis engine, characterized in that, Includes the following steps: S100, deploy local large language models, speech synthesis engines and automatic speech recognition models; S200: Obtain the first training data and input it into the large language model to obtain the first generated text; S300: Input the first generated text into the speech synthesis engine to generate a synthesized speech stream based on context awareness and semantic disambiguation technology; S400. The first generated text is used as the label of the corresponding speech sample in the synthesized speech stream to form the second training data, and is input into the automatic speech recognition model for speech recognition training to obtain the recognized text. S500. Verify the recognized text based on the second training data: If there is an incorrect recognition, the incorrect recognition is labeled for training, and the corresponding second training data is re-input into the automatic speech recognition model for speech recognition training, and step S500 is repeated; if there is no incorrect recognition, steps S200 to S500 are repeated until the number of iterations reaches a preset threshold. S600. Input the speech stream to be recognized into the automatic speech recognition model to obtain the speech recognition result.

2. The speech recognition method based on a large model and a speech synthesis engine according to claim 1, characterized in that, The speech synthesis engine generates synthesized speech streams based on context-aware and semantic disambiguation technologies, including the following steps: S310. The first generated text input is segmented and annotated with pinyin. S320. Based on context-aware and semantic disambiguation technology, perform semantic disambiguation on word segmentation results and / or phonetic transcription; S330. The first generated text after semantic disambiguation is converted into the target speech waveform to obtain a synthesized speech stream.

3. The speech recognition method based on a large model and a speech synthesis engine according to claim 2, characterized in that, The process of converting the semantically disambiguated first generated text into a target speech waveform includes the following steps: S331. Perform lexical analysis, syntactic structure recognition, and prosodic boundary prediction on the first generated text after semantic disambiguation to generate structured text features; S332, Retrieve a timbre sample from the timbre pool; S333, Randomly generate synthesis parameters; the synthesis parameters include one or more of accent, speech rate, and signal-to-noise ratio; S334. Based on the structured text features, timbre samples, and synthesis parameters, the speech synthesis engine predicts the spectral features of the target speech based on a neural network acoustic model; S335. The spectral features are converted into target speech waveforms using a neural vocoder, and the naturalness of the speech is further enhanced based on reverberation, timbre control and prosody adjustment to generate a synthesized speech stream that conforms to the characteristics of real speech.

4. The speech recognition method based on a large model and a speech synthesis engine according to claim 1, characterized in that, The large language model generates a first generated text based on the first training text, and further includes the following steps: Obtain external input commands; Based on the input instructions and the first training text, the large language model is trained a second time; or... Based on the input instructions and the first training text, the first generated text is generated a second time.

5. The speech recognition method based on a large model and a speech synthesis engine according to claim 1, characterized in that, The verification of the recognized text based on the second training data includes the following steps: S510, Recognition trigger, the second training data with the first generated text as the label is used as the recognition verification input; S520. Perform character-level, word-level, and / or phoneme-level difference analysis on the recognition verification input, calculate the recognition error index by replacing, inserting, and / or deleting difference types, and determine whether there is a recognition error: if there is a recognition error, proceed to step S530; otherwise, proceed to step S550. S530. Construct training annotations for speech samples with identification errors, wherein the training annotations include error location annotations, error type annotations, and speech-text alignment information; S540. Input the second training data with the added training annotations into the training module of the automatic speech recognition model, and repeatedly execute the incremental training or fine-tuning process to update the parameters of the acoustic model, language model or end-to-end recognition model in the automatic speech recognition model to correct the errors that occurred in the previous round of recognition; when the recognized text is consistent with the annotated text, the system determines that the training and verification of this round has converged and executes step S550; otherwise, return to step S520. S550. Increase the iteration count. When the cumulative number of iterations reaches the preset threshold, stop the loop and execute step S600; if the threshold is not reached, return to step S200.

6. The speech recognition method based on a large model and a speech synthesis engine according to claim 5, characterized in that, The method further includes the following steps: The first generated text volume of the large language model, and / or the recognition accuracy, training parameters, number of iterations and number of verifications of the automatic speech recognition model are monitored, and preset warning thresholds are configured for each; when there is an object that exceeds the corresponding warning threshold, a warning signal is output.

7. A speech recognition system based on a large model and a speech synthesis engine, applied to the speech recognition method according to any one of claims 1 to 6, characterized in that, include: The corpus generation module is configured to generate the first generated text using a pre-trained large language model and the first training text it introduces. The speech synthesis module is configured to use a pre-trained speech synthesis engine to take the first generated text as a label for the synthesized speech stream to form second training data. The speech recognition module is configured to train the second training data using an automatic speech recognition model to obtain the recognized text; and is configured to perform speech recognition on the input speech stream to be recognized and output the speech recognition result. The verification module is configured to verify the recognized text using the second training data. If a recognition error exists, the error recognition is used for training annotation, and the corresponding second training data is re-inputted into the automatic speech recognition model for speech recognition training.

8. The speech recognition system based on a large model and a speech synthesis engine according to claim 7, characterized in that, The system also includes: The monitoring module is configured to monitor the first generated text volume of the corpus generation module, the recognition accuracy, training parameters, and iteration count of the speech recognition module, and / or the verification count of the verification module, and configure preset warning thresholds for each; when there is an object that exceeds the corresponding warning threshold, a warning signal is output.

9. A device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that, When the computer-readable instructions are executed by the processor, the processor performs all or part of the steps of the speech recognition method as described in any one of claims 1 to 6.

10. A storage medium having computer-readable instructions stored thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement all or part of the steps of the speech recognition method as described in any one of claims 1 to 6.

Images