A speech recognition method and system

Abstract

The embodiment of the present invention provides a speech recognition method and system, belonging to the field of speech recognition, the method comprising: extracting corresponding sample speech feature vectors from the preprocessed sample speech respectively through a feature extraction module to be trained, extracting the feature representation and the temporal dependency of the speech from the sample speech feature vector through the speech module to be trained; inputting the mapping representation and the temporal dependency of the speech into a language module to be trained to form a plurality of probability factor sequences; for each probability factor sequence, the probability factor sequence with the highest matching degree is used as the language text of the sample speech signal and outputted, until the outputted language text is consistent with the marked text corresponding to the sample speech signal, to obtain a trained speech recognition model, the speech recognition model is used to receive the speech signal to be recognized and output the corresponding language text, so as to improve the speech recognition accuracy and generalization ability.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition, and in particular to a speech recognition method and system. Background Art

[0002] Traditional speech recognition methods often have difficulty achieving ideal recognition effects in changing and complex speech environments (such as telephone communications, smart homes, smart cars, etc.) due to the presence of environmental noise, which limits the widespread application of speech recognition technology. Or due to user habits (non-standard Mandarin, dialects, and daily communication with obvious oral characteristics), there are unknown words, which will affect the accuracy of speech recognition and often make it difficult to achieve ideal recognition effects, which limits the widespread application of speech recognition technology.

[0003] In the process of implementing the present invention, the applicant discovered that there are at least the following problems in the prior art: In traditional speech recognition methods, the system relies on predefined rules and dictionaries for matching, which performs poorly when dealing with unknown vocabulary or changing and complex speech environments. At the same time, traditional methods also have limitations in the extraction of speech features and model training, resulting in limited recognition accuracy and generalization capabilities. Summary of the invention

[0004] The embodiments of the present invention provide a speech recognition method and system, which can solve the technical problem that the traditional methods in the prior art have limitations in the extraction of speech features and the training of models, resulting in limited recognition accuracy and generalization ability.

[0005] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a speech recognition method, comprising: Step 11: for each preprocessed sample speech, extract a corresponding sample speech feature vector from the preprocessed sample speech through a feature extraction module to be trained, wherein the sample speech feature vector retains the speech features of the speech generator, and the sample speech feature vector includes the spectral features of the speech, the time sequence information of the speech, and the contextual connection of the speech; Step 12: extracting a feature representation and a temporal dependency of speech from the sample speech feature vector through the speech module to be trained, wherein the feature representation includes Mel-frequency cepstral coefficients of speech, temporal features of speech, and temporal dependency of speech; mapping the spectral features of speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; Step 13: input the mapping representation and the temporal dependency of the speech into a language module to be trained, and the language module to be trained combines and sorts the phonemes or words according to grammatical rules and semantic relationships according to the temporal dependency of the speech to form a plurality of probability factor sequences; Step 14: input the temporal dependency of the speech and the plurality of probability factor sequences into the decoding module to be trained; for each probability factor sequence, determine the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech through a dynamic programming algorithm in the decoding module to be trained; and output the probability factor sequence with the highest matching degree as the language text of the sample speech signal; the sample speech signal is the original speech of the preprocessed sample speech; the sample speech signal refers to a speech stream generated by a speech generator in any language that can express meaning; Step 15: According to the matching degree between the output language text and the sample speech signal, adjust the parameters of the modules in each step from step 11 to step 14, and continue training using the preprocessed sample speech until the output language text matches the marked text corresponding to the sample speech signal, thereby obtaining a trained speech recognition model; the speech recognition model is used to receive the speech signal to be recognized and output the corresponding language text.

[0006] In a second aspect, an embodiment of the present invention provides a speech recognition system, including: A feature extraction module training unit is used to extract corresponding sample speech feature vectors from the preprocessed sample speech through the feature extraction module to be trained for each preprocessed sample speech, wherein the sample speech feature vector retains the speech features of the speech generator, and the sample speech feature vector includes the spectral features of the speech, the time sequence information of the speech, and the contextual connection of the speech; A speech module training unit is used to extract a feature representation and a temporal dependency of speech from the sample speech feature vector through the speech module to be trained, wherein the feature representation includes a Mel-frequency cepstral coefficient of speech, a temporal feature of speech and a temporal dependency of speech; and map the spectral features of speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; A language module training unit, used for inputting the mapping representation and the temporal dependency of speech into the language module to be trained, and combining and sorting the phonemes or words according to grammatical rules and semantic relationships through the language module to be trained according to the temporal dependency of speech to form a plurality of probability factor sequences; A decoding module training unit, used for inputting the temporal dependency of speech and a plurality of the probability factor sequences into the decoding module to be trained, for each probability factor sequence, judging the matching degree between the probability factor sequence and the sample speech signal according to the temporal dependency of speech through a dynamic programming algorithm in the decoding module to be trained, and taking the probability factor sequence with the highest matching degree as the language text of the sample speech signal and outputting it; the preprocessed sample speech is obtained based on preprocessing the sample speech signal; Among them, according to the matching degree between the output language text and the sample voice signal, the parameters of each module to be trained are adjusted, and the pre-processed sample voice is used to continue training through the feature extraction module training unit, the voice module training unit, the language module training unit, and the decoding module training unit in sequence, until the output language text is consistent with the marked text corresponding to the sample voice signal, and a trained voice recognition model is obtained; the voice recognition model is used to receive the voice signal to be recognized and output the corresponding language text.

[0007] In a third aspect, an embodiment of the present invention provides a computer device, including: A processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the aforementioned speech recognition method.

[0008] The above technical solution has the following beneficial effects: the sample speech feature vector not only contains the spectral characteristics of the sample speech signal, but also contains the timing information and contextual relationship of the sample speech signal, which provides a basis for the modeling of the subsequent speech module. At the same time, the sample speech feature vector also contains user feature information and accent dialect, etc., which provides a basis for personalized training. It can automatically learn the speech features of different speakers and different speech environments, thereby learning the complex features in the speech signal and enhancing the generalization ability of the model. And in the language module to be trained, it can be mapped to the corresponding phoneme or word label, so as to achieve accurate recognition of the speech signal. The recognized phonemes or words can be combined and sorted according to grammatical rules and semantic relationships to form meaningful sentences or paragraphs. Thereby optimizing and correcting the recognition results, improving the precision, accuracy and fluency of speech recognition. The process of decoding and predicting the probability factor sequence to generate the language text model training reduces the computational complexity, improves the training efficiency and improves the generalization ability. The speech recognition method of the embodiment of the present invention can adapt to the speech recognition problem in a complex speech environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 is a flow chart of a speech recognition method according to an embodiment of the present invention; Figure 2 is a logical structure diagram of a speech recognition system according to an embodiment of the present invention; Figure 3It is a flow chart of the application of the speech recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] like Figure 1 As shown, in combination with an embodiment of the present invention, a speech recognition method is provided, comprising: Step 11: for each preprocessed sample speech, extract a corresponding sample speech feature vector from the preprocessed sample speech through a feature extraction module to be trained, wherein the sample speech feature vector retains the speech features of the speech generator, and the sample speech feature vector includes the spectral features of the speech, the time sequence information of the speech, and the contextual connection of the speech; Step 12: extracting a feature representation and a temporal dependency of speech from the sample speech feature vector through the speech module to be trained, wherein the feature representation includes Mel-frequency cepstral coefficients of speech, temporal features of speech, and temporal dependency of speech; mapping the spectral features of speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; Step 13: input the mapping representation and the temporal dependency of the speech into a language module to be trained, and the language module to be trained combines and sorts the phonemes or words according to grammatical rules and semantic relationships according to the temporal dependency of the speech to form a plurality of probability factor sequences; Step 14: input the temporal dependency of the speech and the plurality of probability factor sequences into the decoding module to be trained; for each probability factor sequence, determine the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech through a dynamic programming algorithm in the decoding module to be trained; and output the probability factor sequence with the highest matching degree as the language text of the sample speech signal; the sample speech signal is the original speech of the preprocessed sample speech; the sample speech signal refers to a speech stream generated by a speech generator in any language that can express meaning; Step 15: According to the matching degree between the output language text and the sample speech signal, adjust the parameters of the modules in each step from step 11 to step 14, and continue training using the preprocessed sample speech until the output language text matches the marked text corresponding to the sample speech signal, thereby obtaining a trained speech recognition model; the speech recognition model is used to receive the speech signal to be recognized and output the corresponding language text.

[0013] The feature vector not only contains the spectral characteristics of the sample speech signal, but also contains the temporal information and contextual relationship of the sample speech signal, which provides a basis for the modeling of subsequent speech modules. At the same time, the feature vector also contains user feature information and accents, etc., which provides a basis for personalized training.

[0014] It can automatically learn the speech features of different speakers and different speech environments, thereby learning the complex features in the speech signal, enhancing the generalization ability of the model. And in the language module to be trained, it can map it to the corresponding phoneme or word label, thereby achieving accurate recognition of the speech signal.

[0015] It can combine and sort the recognized phonemes or words according to grammatical rules and semantic relationships to form meaningful sentences or paragraphs, thereby optimizing and correcting the recognition results and improving the precision, accuracy and fluency of speech recognition.

[0016] The process of decoding and predicting the probability factor sequence to generate the language text model training reduces the computational complexity, improves the training efficiency, and improves the generalization ability.

[0017] The speech recognition method of the embodiment of the present invention can adapt to the speech recognition problem in a complex speech environment. In practical applications, speech signals are often interfered by various factors, such as background noise, echo, multi-talker interference, etc., which can seriously affect the accuracy of speech recognition.

[0018] Preferably, step 11 comprises: Extracting spectral features from the preprocessed sample speech by using a mixed Mel-frequency cepstral coefficient feature extraction algorithm; The first deep learning model is used to extract the temporal information of the speech from the preprocessed sample speech. The temporal information of the speech refers to the law of the evolution of the previous phoneme to the next phoneme over time. The first deep learning model is a convolutional neural network (CNN) with an attention mechanism; the temporal information refers to the characteristics of the speech signal changing over time. For example, in a continuous speech recognition task, understanding how one phoneme evolves to another phoneme over time can help the speech recognition model better predict the next possible sound.

[0019] Extracting speech contextual connections (contextual information, especially long-distance contextual information) from the preprocessed sample speech through a second deep learning model, wherein the speech contextual connections refer to the connections between a current speech frame and its adjacent speech frames, and the second deep learning model is a long short-term memory network LSTM; The sample speech feature vector corresponding to the sample speech signal is formed by the spectral feature, the timing information of the speech and the context of the speech. The context relationship refers to the relationship between the current speech frame (phoneme) and its adjacent frames (phonemes) before and after it. Since speech is continuous streaming media data, the current speech feature (phoneme) is often affected by the speech features (phonemes) before and after it.

[0020] By using the Hybrid Mel Frequency Cepstral Coefficients (MFCC) feature extraction algorithm and the first deep learning model (Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM)), the sample speech is preprocessed to extract feature vectors containing rich speech information, so that rich feature vectors can be automatically learned from the original sample speech signal. Among them, CNN is responsible for extracting static features, which are local features of the speech signal, such as spectral features. LSTM is used to extract dynamic features, which refer to time-varying features, time series information (time series features), that is, the changes of speech signals in time. When learning, static features and dynamic features are mixed for learning. The feature vector not only contains the spectral characteristics of the sample speech signal (the spectral characteristics are the performance of the speech signal in the frequency domain. The spectrum diagram shows the energy distribution of the speech signal at different frequencies, which can reflect the pitch, timbre and other information of the speech), but also contains the time series information and contextual relationship of the sample speech signal, which provides a basis for the modeling of subsequent speech modules. At the same time, the feature vector also contains user feature information and accents, etc., which provides a basis for personalized training and can more effectively process complex speech signals (preprocessing and feature extraction).

[0021] Preferably, step 12 comprises: Constructing a speech module to be trained, wherein the speech module to be trained includes a third deep learning model and a fourth deep learning model; Extracting a feature representation from the sample speech feature vector through the third deep learning model, the feature representation includes Mel-frequency cepstral coefficients of the speech and the temporal features of the speech, the temporal features of the speech represent the frequency characteristics of the speech changing over time, and mapping the spectral features of the speech to corresponding phoneme labels or word labels in sequence according to the feature representation, wherein the third deep learning model is a convolutional neural network (CNN) with an attention mechanism; The temporal dependency of speech in the sample speech feature vector is captured by the fourth deep learning model, and the fourth deep learning model is a long short-term memory network LSTM.

[0022] Deep learning models include: speech modeling (acoustic modeling) and language modeling. Speech modeling (acoustic modeling) uses a deep neural network structure, and constructs a speech module by introducing a convolutional neural network (CNN) and a long short-term memory network (LSTM) with an attention mechanism. The CNN network has a strong feature learning capability and can extract richer feature representations from the input feature vector: spectral features (mainly MFCC, the abbreviation of Mel frequency cepstral coefficients), temporal features (time-frequency features), and map the feature representation to the corresponding phoneme or word label, that is, convert the phoneme or word corresponding to the feature representation into its corresponding label; while the LSTM network can capture the temporal dependency in the speech signal, which is particularly effective for processing continuous speech signals. It can learn the probability distribution of various speech features, further reducing the probability of misrecognition and missed recognition.

[0023] During training, feature representation and temporal relationship information are used to correspond to the text of the speech. After training, feature representation and temporal relationship information are used as model input to predict the corresponding text. For example, a sound is "The weather is so nice today", and the information features of the speech obtained during training are ABCDEF (not necessarily one-to-one correspondence). When the speech module is trained and can be recognized, the ABCDEF features correspond to "The weather is so nice today"; if the speech signal to be recognized is "The weather is so nice tomorrow" with a little noise; the extracted features should be zBCDEF, because the feature representation of "tomorrow" in the speech recognition model is zB, and the language text "The weather is so nice tomorrow" will be extracted in the end.

[0024] The feature vectors extracted by the feature extraction module to be trained are used for voice modeling to achieve accurate classification of voice signals. Classification is to map each time segment (usually a short time frame) in the voice signal to a specific category, such as "b", "p", "m" in Chinese. Classification is to convert continuous voice signals into discrete language symbols (phonemes) so that computers can understand and process human language. Through classification, the system can gradually build a complete voice transcription result, and finally achieve accurate recognition of the voice content.

[0025] These deep learning models can automatically learn the speech features of different speakers and different speech environments through a large number of sample language signals, and thus learn the complex features in the speech signals, such as different timbres and tones. The complexity of the language itself enhances the generalization ability of the model. And in the language module to be trained, it can be mapped to the corresponding phoneme or word label, so as to achieve accurate recognition of the speech signal. In the process of deep learning, words are not directly processed as information, but in a digital way, similar to the form of labels and characters.

[0026] Preferably, step 13 comprises: Constructing a language module to be trained, wherein the language module to be trained includes an N-gram model and a fifth deep learning model, wherein the fifth deep learning model is a transformer model; Through the N-gram based on the context grammatical rules, according to the temporal dependencies of speech whose grammatical strength is lower than the preset value in the temporal dependencies of speech, the corresponding phonemes or words are combined and sorted according to the grammatical rules and semantic relationships, and the temporal dependencies of speech whose grammatical strength is not lower than the preset value in the temporal dependencies of speech are captured through the self-attention mechanism of the fifth deep learning model, and the corresponding phonemes or words are combined and sorted according to the grammatical rules and semantic relationships to form a plurality of probability factor sequences, each probability factor sequence including one of the following: meaningful words, meaningful paragraphs, and meaningful sentences.

[0027] In terms of language modeling, the output of the speech module to be trained is used as the input of the language model to be trained. The language model to be trained adopts the traditional N-gram model and the more advanced deep learning model Transformer. N-gram is a language model based on probability statistics. It captures basic local context grammatical rules through probability distribution. It is suitable for simple and concise grammar and captures simple grammar with low computational complexity and fast speed. The Transformer model can capture the grammatical rules and semantic relations of the global context in the sentence through the self-attention mechanism. It has strong grammatical comprehension ability and is suitable for capturing complex grammar, but has high computational complexity and slow speed. By combining the two, the best result of the two is taken to improve the overall accuracy and fluency.

[0028] Because both language models can learn the grammatical rules and semantic relationships of the language, they can combine and sort the recognized phonemes or words according to the grammatical rules and semantic relationships to form meaningful sentences or paragraphs. This optimizes and corrects the recognition results, improves the accuracy of speech recognition and the fluency of the language. Because there are many possibilities between speech signals and texts (noise, polyphones, accents, etc.), the language module to be trained outputs a probability sequence with multiple.

[0029] By learning both the static and dynamic features of speech signals, the accuracy of speech recognition is improved. By introducing technologies such as the attention mechanism, the robustness and recognition accuracy of the language module to be trained are further improved.

[0030] Preferably, step 14 comprises: For each probability factor sequence, a dynamic programming decoding method based on the Viterbi algorithm is used to determine the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech, and the probability factor sequence with the highest matching degree is used as the probability factor sequence matching the sample speech signal; A pre-trained language model is used to capture the grammatical and semantic structures in the matched sequence of probability factors to form a language text.

[0031] The use of language modules can capture the probability factor sequence that best conforms to language rules (grammar, semantic structure) and context rules, providing additional constraints for the dynamic programming decoding method based on the Viterbi algorithm, helping it to select text sequences that are more in line with language rules. The dynamic programming decoding method based on the Viterbi algorithm will comprehensively consider the probabilities of the speech module and the language module, find the probability factor sequence with the largest joint probability, and use the result with the highest probability to ensure the accuracy of recognition.

[0032] Pre-trained language models are models that have been trained in related fields. They can further accelerate the training process of speech recognition models and improve the performance of speech recognition models. Pre-trained language models can be existing pre-trained speech modules (HuBert, etc.) and language modules (Bert, GPT, etc.). By training and fine-tuning the data model on the training data set, the purpose of quickly generating a new model can be achieved.

[0033] Preferably, the speech recognition method further comprises: Step 16, before step 12, for each of the sample speech signals, the sample speech signal stream is subjected to noise reduction processing, echo removal processing and pre-emphasis processing in sequence through the pre-processing module to be trained to obtain an initial processed speech signal; the pre-emphasis processing is to reduce the influence of low-frequency noise and make the spectrum of the sample speech signal flatter.

[0034] Endpoint detection is performed on the initially processed voice signal to detect the starting position and the ending position of the initially processed voice signal; voice pre-emphasis processing is performed on the initially processed voice signal from the starting position to the ending position to obtain a corresponding pre-processed sample voice to improve the quality of the voice signal and provide a sufficiently clear voice signal for the subsequent process.

[0035] Through preprocessing, complex speech signals can be processed more effectively.

[0036] Preferably, the speech recognition method further comprises: Step 17, obtain multiple original voice signals, transform and expand each original voice signal to obtain a new voice signal, the transformation and expansion method includes at least one of the following: adding noise, changing the playback speed, changing the pitch of the voice signal, adding echo, and disturbing the filter characteristics of the voice signal, the sample voice signal refers to the voice stream generated by the voice generator in any language that can express meaning. Use data enhancement technology to transform and expand the original voice data to generate more training data to improve the generalization ability of the model.

[0037] Both the original speech signal and the new speech signal are used as sample speech signals to construct a sample speech signal set; the sample speech signal set is relatively large in size to ensure that the speech recognition model can learn sufficiently rich speech features.

[0038] Wherein, step 17 is performed before step 16.

[0039] Preferably, the speech recognition method, step 17, further includes: For application scenarios with user feature requirements, the original voice signal is formed by obtaining the natural language recording formed by the user reading a text for a certain length of time to meet the needs of special application scenarios. The accuracy and adaptability of voice recognition are comprehensively improved to meet more extensive and complex application needs.

[0040] In summary, the speech recognition method of the embodiment of the present invention can adapt to the speech recognition problems in complex speech environments. In practical applications, speech signals are often interfered by various factors, such as background noise, echo, multi-speaker interference, etc., which will seriously affect the accuracy of speech recognition. By introducing deep learning technology, this embodiment can automatically learn the deep features in the speech signal, effectively distinguish the target speech from the background noise, and thus significantly improve the recognition effect in complex environments. The realization of this goal will greatly promote the application of speech recognition technology in complex environments such as noisy environments and outdoor scenes, and bring users a more convenient and efficient interactive experience. Among them, deep features refer to: phoneme-level features, which are the smallest pronunciation units in speech signals; word-level features, which indicate how these phonemes are combined into words and capture the grammatical and semantic relationships between words; sentence-level features, which can understand the semantics of the entire sentence.

[0041] Because in actual applications, speech signals are often interfered by various environmental noises, such as background noise, wind, echo, etc. Different users' pronunciation habits, speaking speed and volume will also affect the speech recognition effect. Through the adaptive ability of deep learning, these changes can be learned during the training process, and the model parameters can be adjusted to adapt to different speech environments, maintaining a high recognition accuracy and enhancing the robustness of speech recognition.

[0042] By retaining user feature information during preprocessing, such as personal accents and dialect features, and including this feature information during feature extraction, personalized training of the speech recognition model can be achieved. In this way, the model can not only better understand the user's voice commands, but also make intelligent adjustments based on the user's habits and needs to improve recognition accuracy and user experience. This improves the personalization level of speech recognition and can meet the unique needs of different users. Therefore, it can promote the popularization of speech recognition technology in personalized application scenarios such as smart homes and smart cars, and bring more intimate and personalized services to users.

[0043] Because it can handle complex language structures, pronunciation rules, and vocabulary changes, it can accurately recognize multiple languages ​​and dialects. For example, in scenarios such as multinational companies, international tourism, or cultural exchanges, users communicate in different languages ​​or dialects. The multi-language and multi-dialect recognition capabilities of this embodiment enable the system to automatically adapt to and understand the user's voice commands, thereby providing a more personalized and intelligent service experience.

[0044] By optimizing the network structure and training strategy of the deep learning model, rapid processing and accurate recognition of continuous speech signals are achieved. At the same time, the efficient decoding algorithm further improves the real-time performance and efficiency of speech recognition, meeting the needs of efficient interaction. It can promote the development of speech recognition technology in efficient application scenarios such as real-time interaction and online recognition, and bring users a smoother and more efficient interactive experience.

[0045] Through deep learning methods, the voice signal is subjected to fine feature extraction and complex modeling processes. The deep neural network can learn high-level features in the voice signal, which are essential for distinguishing different voice contents and improve the accuracy of voice recognition. For application scenarios that require high-precision voice recognition, such as smart home control and in-car voice assistants, it can bring fewer misoperations, higher user satisfaction and more reliable system performance.

[0046] The lightweight network structure (LSTM and Transformer can be set relatively small) and efficient computing algorithms are adopted to reduce the computing resources and storage space occupied. In the inference stage, it can quickly process the input voice signal and output the recognition result, which improves the real-time performance and response speed of voice recognition. This low resource consumption and high efficiency feature makes the patent have outstanding advantages in resource-constrained environments (such as mobile devices, embedded systems, etc.).

[0047] By dividing and encoding the feature sequence of the sample speech signal, a feature representation (target speech feature block) is obtained, multiple probability factor sequences are generated, and the probability factor sequences are decoded and predicted to generate language text. This processing method simplifies the model training process, reduces computational complexity, and improves training efficiency. At the same time, the model can be iteratively optimized based on the feedback of the predicted text sequence. This processing method makes the speech recognition model no longer rely on the input of the entire sentence, but can process streaming voice data in real time, and is effectively applied to scenarios that require instant feedback, such as real-time communication and online speech transcription. Enterprises and developers can use the tools and frameworks provided in this application to quickly build and deploy speech recognition systems.

[0048] Preferably, the speech recognition method further comprises: Step 21, preprocessing the speech signal to be recognized by a preprocessing module to obtain the corresponding speech to be recognized; wherein the speech signal to be recognized refers to a speech stream generated by a speech generator in any language that can express meaning; Step 22: extracting a feature vector of the speech to be recognized from the speech to be recognized by a feature extraction module, wherein the feature vector of the speech to be recognized includes a spectral feature of the speech, a time sequence information of the speech, and a contextual connection of the speech, wherein the spectral feature contains a speech feature of a speech generator; Step 23: extracting a feature representation and a temporal dependency of the speech from the feature vector of the speech to be recognized through the speech module, wherein the feature representation includes the Mel-frequency cepstral coefficient of the speech, the temporal features of the speech, and the temporal dependency of the speech; mapping the spectral features of the speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; Step 24, inputting the mapping representation and the temporal dependency of the speech into a language module, and the language module combines and sorts the phonemes or words according to the grammatical rules and semantic relationships according to the temporal dependency of the speech to form multiple probability factor sequences; Step 25: Input the temporal dependency of speech and multiple probability factor sequences into the module to be decoded. For each probability factor sequence, determine the degree of matching between the probability factor sequence and the speech signal to be recognized based on the temporal dependency of speech through the dynamic programming algorithm in the decoding module, and output the probability factor sequence with the highest matching degree as the language text of the speech signal to be recognized.

[0049] like Figure 2 As shown, in combination with an embodiment of the present invention, a speech recognition system is provided, comprising: The feature extraction module training unit 301 is used to extract corresponding sample speech feature vectors from the preprocessed sample speech through the feature extraction module to be trained for each preprocessed sample speech, wherein the sample speech feature vector retains the speech features of the speech generator, and the sample speech feature vector includes the spectral features of the speech, the time sequence information of the speech, and the contextual connection of the speech; The speech module training unit 302 is used to extract a feature representation and a temporal dependency of speech from the sample speech feature vector through the speech module to be trained, wherein the feature representation includes a Mel-frequency cepstral coefficient of speech, a temporal feature of speech and a temporal dependency of speech; and map the spectral features of speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; A language module training unit 303 is used to input the mapping representation and the temporal dependency relationship of the speech into the language module to be trained, and the language module to be trained combines and sorts the phonemes or words according to the grammatical rules and semantic relationships according to the temporal dependency relationship of the speech to form a plurality of probability factor sequences; The decoding module training unit 304 is used to input the temporal dependency of speech and the plurality of probability factor sequences into the decoding module to be trained, and for each probability factor sequence, determine the matching degree between the probability factor sequence and the sample speech signal according to the temporal dependency of speech through the dynamic programming algorithm in the decoding module to be trained, and output the probability factor sequence with the highest matching degree as the language text of the sample speech signal; the preprocessed sample speech is obtained based on preprocessing the sample speech signal; Among them, according to the matching degree between the output language text and the sample voice signal, the parameters of each module to be trained are adjusted, and the pre-processed sample voice is used to continue training through the feature extraction module training unit, the voice module training unit, the language module training unit, and the decoding module training unit in sequence, until the output language text is consistent with the marked text corresponding to the sample voice signal, and a trained voice recognition model is obtained; the voice recognition model is used to receive the voice signal to be recognized and output the corresponding language text.

[0050] The feature vector not only contains the spectral characteristics of the sample speech signal, but also contains the temporal information and contextual relationship of the sample speech signal, which provides a basis for the modeling of subsequent speech modules. At the same time, the feature vector also contains user feature information and accents, etc., which provides a basis for personalized training.

[0051] It can automatically learn the speech features of different speakers and different speech environments, thereby learning the complex features in the speech signal, enhancing the generalization ability of the model. And in the language module to be trained, it can map it to the corresponding phoneme or word label, thereby achieving accurate recognition of the speech signal.

[0052] It can combine and sort the recognized phonemes or words according to grammatical rules and semantic relationships to form meaningful sentences or paragraphs, thereby optimizing and correcting the recognition results and improving the precision, accuracy and fluency of speech recognition.

[0053] The process of decoding and predicting the probability factor sequence to generate the language text model training reduces the computational complexity, improves the training efficiency, and improves the generalization ability.

[0054] The speech recognition method of the embodiment of the present invention can adapt to the speech recognition problem in a complex speech environment. In practical applications, speech signals are often interfered by various factors, such as background noise, echo, multi-talker interference, etc., which can seriously affect the accuracy of speech recognition.

[0055] Preferably, the feature extraction module training unit 301 is specifically used for: Extracting spectral features from the preprocessed sample speech by using a mixed Mel-frequency cepstral coefficient feature extraction algorithm; The first deep learning model is used to extract the temporal information of the speech from the preprocessed sample speech. The temporal information of the speech refers to the law of the evolution of the previous phoneme to the next phoneme over time. The first deep learning model is a convolutional neural network (CNN) with an attention mechanism; the temporal information refers to the characteristics of the speech signal changing over time. For example, in a continuous speech recognition task, understanding how one phoneme evolves to another phoneme over time can help the speech module better predict the next possible sound.

[0056] Extracting speech contextual connections (contextual information, especially long-distance contextual information) from the preprocessed sample speech through a second deep learning model, wherein the speech contextual connection refers to the connection between a current speech frame and its adjacent speech frames, and the second deep learning model is a long short-term memory network LSTM; The sample speech feature vector corresponding to the sample speech signal is formed by the spectral feature, the timing information of the speech and the contextual connection of the speech. The contextual connection refers to the connection between the current speech frame (phoneme) and its adjacent frames (phonemes) before and after it. Since speech is continuous streaming media data, the current speech feature (phoneme) is often affected by the speech features (phonemes) before and after it.

[0057] By mixing the Mel Frequency Cepstral Coefficient (MFCC) feature extraction algorithm and the first deep learning model (CNN and LSTM), the sample speech is preprocessed to extract feature vectors containing rich speech information, so that rich feature vectors can be automatically learned from the original sample speech signal. Among them, CNN is responsible for extracting static features, which are local features of the speech signal, such as spectral features. LSTM is responsible for extracting dynamic features, which refer to time-varying features, timing information (timing features), that is, the changes of the speech signal in time. When learning, static features and dynamic features are mixed for learning. The feature vector not only contains the spectral characteristics of the sample speech signal (the spectral characteristics are the performance of the speech signal in the frequency domain. The spectrum diagram shows the energy distribution of the speech signal at different frequencies, which can reflect the pitch, timbre and other information of the speech), but also contains the timing information and contextual relationship of the sample speech signal, which provides a basis for the modeling of the subsequent speech module. At the same time, the feature vector also contains user feature information and accent dialects, which provides a basis for personalized training and can more effectively process complex speech signals (preprocessing and feature extraction).

[0058] Preferably, the speech module training unit 302 is specifically used for: Constructing a speech module to be trained, wherein the speech module to be trained includes a third deep learning model and a fourth deep learning model; Extracting a feature representation from the sample speech feature vector through the third deep learning model, the feature representation includes Mel-frequency cepstral coefficients of the speech and the temporal features of the speech, the temporal features of the speech represent the frequency characteristics of the speech changing over time, and mapping the spectral features of the speech to corresponding phoneme labels or word labels in sequence according to the feature representation, wherein the third deep learning model is a convolutional neural network (CNN) with an attention mechanism; The temporal dependency of speech in the sample speech feature vector is captured by the fourth deep learning model, and the fourth deep learning model is a long short-term memory network LSTM.

[0059] The deep learning model includes: speech modeling (acoustic modeling) and language modeling, which are the core parts. The speech modeling (acoustic modeling) adopts advanced deep neural network structure, and constructs the speech module by introducing convolutional neural network (CNN) and long short-term memory network (LSTM) with attention mechanism. CNN network has powerful feature learning ability, and can extract richer feature representation from the input feature vector: spectrum feature (mainly MFCC, the abbreviation of Mel frequency cepstral coefficient), time series feature (time-frequency feature), and map the feature representation to the corresponding phoneme or word label, that is, convert the phoneme or word corresponding to the feature representation into its corresponding label; while LSTM network can capture the time series dependency in speech signal, which is particularly effective for processing continuous speech signal, and can learn the probability distribution of various speech features, further reducing the probability of misrecognition and missed recognition.

[0060] During training, feature representation and temporal relationship information are used to correspond to the text of the speech. After training, feature representation and temporal relationship information are used as model input to predict the corresponding text. For example, a sound is "The weather is so nice today", and the information features of the speech obtained during training are ABCDEF (not necessarily one-to-one correspondence). When the speech module is trained and can be recognized, the ABCDEF features correspond to "The weather is so nice today"; if the speech signal to be recognized is "The weather is so nice tomorrow" with a little noise; the extracted features should be zBCDEF, because the feature representation of "tomorrow" in the speech recognition model is zB, and the language text "The weather is so nice tomorrow" will be extracted in the end.

[0061] The feature vectors extracted by the feature extraction module to be trained are used for voice modeling to achieve accurate classification of voice signals. Classification is to map each time segment (usually a short time frame) in the voice signal to a specific category, such as "b", "p", "m" in Chinese. Classification is to convert continuous voice signals into discrete language symbols (phonemes) so that computers can understand and process human language. Through classification, the system can gradually build a complete voice transcription result, and finally achieve accurate recognition of the voice content.

[0062] These deep learning models can automatically learn the speech features of different speakers and different speech environments through a large number of sample language signals, thereby learning the complex features in the speech signals, such as different timbres and tones. The complexity of the language itself enhances the generalization ability of the model. And in the language module to be trained, it can be mapped to the corresponding phoneme or word label, so as to achieve accurate recognition of the speech signal. In the process of deep learning, words are not directly processed as information, but in a digital way, similar to the form of labels and characters.

[0063] Preferably, the language module training unit 303 is specifically used for: Constructing a language module to be trained, wherein the language module to be trained includes an N-gram model and a fifth deep learning model, wherein the fifth deep learning model is a transformer model; Through the N-gram based on the context grammatical rules, according to the temporal dependencies of speech whose grammatical strength is lower than the preset value in the temporal dependencies of speech, the corresponding phonemes or words are combined and sorted according to the grammatical rules and semantic relationships, and the temporal dependencies of speech whose grammatical strength is not lower than the preset value in the temporal dependencies of speech are captured through the self-attention mechanism of the fifth deep learning model, and the corresponding phonemes or words are combined and sorted according to the grammatical rules and semantic relationships to form a plurality of probability factor sequences, each probability factor sequence including one of the following: meaningful words, meaningful paragraphs, and meaningful sentences.

[0064] In terms of language modeling, the output of the speech module to be trained is used as the input of the language model to be trained. The language model to be trained adopts the N-gram model and the deep learning model Transformer. N-gram is a language model based on probability statistics. It captures basic local context grammatical rules through probability distribution. It is suitable for simple and concise grammar and captures simple grammar. Its computational complexity is low and its speed is fast. The Transformer model can capture the grammatical rules and semantic relationships of the global context in the sentence through the self-attention mechanism. It has strong grammatical comprehension ability and is suitable for capturing complex grammar, but its computational complexity is high and its speed is slow. By combining the two, the best result of the two is taken to improve the overall accuracy and fluency.

[0065] Because both language models can learn the grammatical rules and semantic relationships of the language, they can combine and sort the recognized phonemes or words according to the grammatical rules and semantic relationships to form meaningful sentences or paragraphs. This optimizes and corrects the recognition results, improves the accuracy of speech recognition and the fluency of the language. Because there are many possibilities between speech signals and texts (noise, polyphones, accents, etc.), the language module to be trained outputs a probability sequence with multiple.

[0066] By learning both the static and dynamic features of speech signals, the accuracy of speech recognition is improved. By introducing technologies such as the attention mechanism, the robustness and recognition accuracy of the language module to be trained are further improved.

[0067] Preferably, the decoding module training unit 304 is specifically used for: For each probability factor sequence, a dynamic programming decoding method based on the Viterbi algorithm is used to determine the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech, and the probability factor sequence with the highest matching degree is used as the probability factor sequence matching the sample speech signal; A pre-trained language model is used to capture the grammatical and semantic structures in the matched sequence of probability factors to form a language text.

[0068] The use of language modules can capture the probability factor sequence that best conforms to language rules (grammar, semantic structure) and context rules, providing additional constraints for the dynamic programming decoding method based on the Viterbi algorithm, helping it to select text sequences that are more in line with language rules. The dynamic programming decoding method based on the Viterbi algorithm will comprehensively consider the probabilities of the speech module and the language module, find the probability factor sequence with the largest joint probability, and use the result with the highest probability to ensure the accuracy of recognition.

[0069] Pre-trained language models are models that have been trained in related fields. They can further accelerate the training process of speech recognition models and improve the performance of speech recognition models. Pre-trained language models can be existing pre-trained speech modules (HuBert, etc.) and language modules (Bert, GPT, etc.). By training and fine-tuning the data model on the training data set, the purpose of quickly generating a new model can be achieved.

[0070] Preferably, the speech recognition system further comprises: The preprocessing unit 306 is used to perform noise reduction processing, echo removal processing and pre-emphasis processing on the sample voice signal stream in sequence through the preprocessing module to be trained for each of the sample voice signals to obtain an initial processed voice signal; the pre-emphasis processing is to reduce the influence of low-frequency noise and make the frequency spectrum of the sample voice signal flatter.

[0071] Endpoint detection is performed on the initially processed voice signal to detect the starting position and the ending position of the initially processed voice signal; voice pre-emphasis processing is performed on the initially processed voice signal from the starting position to the ending position to obtain a corresponding pre-processed sample voice to improve the quality of the voice signal and provide a sufficiently clear voice signal for the subsequent process.

[0072] Through preprocessing, complex speech signals can be processed more effectively.

[0073] Preferably, the speech recognition method further comprises: The expansion unit 307 is used to obtain multiple original voice signals, transform and expand each original voice signal to obtain a new voice signal, and the transformation and expansion method includes at least one of the following: adding noise, changing the playback speed, changing the pitch of the voice signal, adding echo, and disturbing the filter characteristics of the voice signal. The sample voice signal refers to the voice stream generated by the voice generator in any language that can express meaning. The data enhancement technology is used to transform and expand the original voice data to generate more training data to improve the generalization ability of the model.

[0074] Both the original speech signal and the new speech signal are used as sample speech signals to construct a sample speech signal set; the sample speech signal set is relatively large in size to ensure that the speech recognition model can learn sufficiently rich speech features.

[0075] Preferably, the speech recognition method, the expansion unit 307, is further used for: For application scenarios with user feature requirements, the original voice signal is formed by obtaining the natural language recording formed by the user reading a text for a certain length of time to meet the needs of special application scenarios. The accuracy and adaptability of voice recognition are comprehensively improved to meet more extensive and complex application needs.

[0076] Preferably, the speech recognition system further includes a speech recognition unit to be processed, and the speech recognition unit to be processed includes: The preprocessing module is used to preprocess the speech signal to be recognized to obtain the corresponding speech to be recognized; wherein the speech signal to be recognized refers to the speech stream generated by the speech generator in any language that can express meaning; A feature extraction module is used to extract a feature vector of a speech to be recognized from the speech to be recognized, wherein the feature vector of the speech to be recognized includes a spectral feature of the speech, a time sequence information of the speech, and a contextual connection of the speech, wherein the spectral feature contains a speech feature of a speech generator; The speech module is used to extract feature representation and speech temporal dependency from the feature vector of the speech to be recognized, wherein the feature representation includes the Mel-frequency cepstral coefficient of the speech, the temporal feature of the speech and the temporal dependency of the speech; and map the spectral features to the corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; A language module, which is used to input the mapping representation and the temporal dependency of the speech into the language module, and the language module combines and sorts the phonemes or words according to the grammatical rules and semantic relationships according to the temporal dependency of the speech to form multiple probability factor sequences; The module to be decoded is used to input the temporal dependency of speech and multiple probability factor sequences into the module to be decoded. For each probability factor sequence, the dynamic programming algorithm in the decoding module is used to determine the degree of matching between the probability factor sequence and the speech signal to be recognized based on the temporal dependency of speech, and the probability factor sequence with the highest matching degree is output as the language text of the speech signal to be recognized.

[0077] In addition, a speech recognition device is also provided, which includes the aforementioned speech recognition system and a microphone array. The microphone array is used to collect speech signals. When there are application scenarios with user feature requirements, natural language recordings can be obtained by having the user read text of a certain length.

[0078] like Figure 3 The following is a flowchart of the training and application of the speech recognition model. The brief process is as follows: 1. Use microphone array to collect and input voice signals.

[0079] 2. Preprocess the sample speech signal, which includes: echo removal, noise reduction, pre-emphasis, and endpoint detection to increase the amplitude of the high-frequency signal and improve the spectral characteristics of the speech signal. Among them, the echo removal step processes the echo component in the speech signal to reduce its impact on speech recognition. The speech signal is subjected to noise reduction and other processing. The noise in the signal can be further reduced through noise reduction processing, making the speech signal clearer. Then, filtering is performed to improve the signal quality. Enter the endpoint detection step to determine the starting and ending positions of the speech signal to obtain the preprocessed sample speech.

[0080] 3. Enter the feature extraction step to extract features useful for speech recognition, namely feature vectors, from the preprocessed sample speech.

[0081] 4. For the feature vector corresponding to each sample speech signal in the sample speech signal set, a neural network model is used for model training to obtain a speech recognition model.

[0082] Among them, the test data set is used to test the trained speech recognition model, so that the speech recognition model can recognize the text content in the speech signal.

[0083] 5. In the step of performing speech recognition on the speech signal to be recognized, multiple probability factor sequences are obtained. The probability factor sequences are calculated with probability distribution to evaluate the accuracy of the recognition result. The most matching probability factor sequence is determined by the decoding module (also called decoding unit) and presented in the form of language text, completing the workflow of the recognition of the sample speech signal.

[0084] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0085] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0086] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0087] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0088] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps mentioned above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0089] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0090] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0091] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and Blu-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. Combinations of the above may also be included in computer-readable media.

[0092] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A speech recognition method, characterized in that: include: Step 11: for each preprocessed sample speech, extract a corresponding sample speech feature vector from the preprocessed sample speech through a feature extraction module to be trained, wherein the sample speech feature vector retains the speech features of the speech generator, and the sample speech feature vector includes the spectral features of the speech, the time sequence information of the speech, and the contextual connection of the speech; Step 12: extracting a feature representation and a temporal dependency of speech from the sample speech feature vector through the speech module to be trained, wherein the feature representation includes Mel-frequency cepstral coefficients of speech, temporal features of speech, and temporal dependency of speech; mapping the spectral features of speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; Step 13: input the mapping representation and the temporal dependency of the speech into a language module to be trained, and the language module to be trained combines and sorts the phonemes or words according to grammatical rules and semantic relationships according to the temporal dependency of the speech to form a plurality of probability factor sequences; Step 14: input the temporal dependency of the speech and the plurality of probability factor sequences into the decoding module to be trained; for each probability factor sequence, determine the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech through a dynamic programming algorithm in the decoding module to be trained; and output the probability factor sequence with the highest matching degree as the language text of the sample speech signal; the sample speech signal is the original speech of the preprocessed sample speech; the sample speech signal refers to a speech stream generated by a speech generator in any language that can express meaning; Step 15: According to the matching degree between the output language text and the sample speech signal, adjust the parameters of the modules in each step from step 11 to step 14, and continue training using the preprocessed sample speech until the output language text matches the marked text corresponding to the sample speech signal, thereby obtaining a trained speech recognition model; the speech recognition model is used to receive the speech signal to be recognized and output the corresponding language text.

2. The speech recognition method according to claim 1, characterized in that: Step 11 includes: Extracting spectral features from the preprocessed sample speech by using a mixed Mel-frequency cepstral coefficient feature extraction algorithm; Extracting speech timing information from the preprocessed sample speech through a first deep learning model, wherein the speech timing information refers to the law of evolution of a previous phoneme to a subsequent phoneme over time, and the first deep learning model is a convolutional neural network with an attention mechanism; Extracting a speech contextual connection from the preprocessed sample speech through a second deep learning model, wherein the speech contextual connection refers to a connection between a current speech frame and its adjacent speech frames, and the second deep learning model is a long short-term memory network; A sample speech feature vector corresponding to the sample speech signal is formed through the frequency spectrum feature, the time sequence information of the speech and the context of the speech.

3. The speech recognition method according to claim 1, characterized in that: Step 12 includes: Constructing a speech module to be trained, wherein the speech module to be trained includes a third deep learning model and a fourth deep learning model; Extracting a feature representation from the sample speech feature vector through the third deep learning model, the feature representation includes Mel-frequency cepstral coefficients of the speech and the time series features of the speech, the time series features of the speech represent the frequency features of the speech changing over time, and mapping the spectral features of the speech to corresponding phoneme labels or word labels in sequence according to the feature representation, wherein the third deep learning model is a convolutional neural network with an attention mechanism; The temporal dependency of speech in the sample speech feature vector is captured by the fourth deep learning model, and the fourth deep learning model is a long short-term memory network.

4. The speech recognition method according to claim 1, characterized in that: Step 13 includes: Constructing a language module to be trained, wherein the language module to be trained includes an N-gram model N-gram and a fifth deep learning model, wherein the fifth deep learning model is a transformer model Transformer; Through the N-gram based on the context grammatical rules, according to the temporal dependencies of speech whose grammatical strength is lower than the preset value in the temporal dependencies of speech, the corresponding phonemes or words are combined and sorted according to the grammatical rules and semantic relationships, and the temporal dependencies of speech whose grammatical strength is not lower than the preset value in the temporal dependencies of speech are captured through the self-attention mechanism of the fifth deep learning model, and the corresponding phonemes or words are combined and sorted according to the grammatical rules and semantic relationships to form a plurality of probability factor sequences, each probability factor sequence including one of the following: meaningful words, meaningful paragraphs, and meaningful sentences.

5. The speech recognition method according to claim 1, characterized in that: Step 14 includes: For each probability factor sequence, a dynamic programming decoding method based on the Viterbi algorithm is used to determine the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech, and the probability factor sequence with the highest matching degree is used as the probability factor sequence matching the sample speech signal; A pre-trained language model is used to capture the grammatical and semantic structures in the matched sequence of probability factors to form a language text.

6. The speech recognition method according to claim 1, characterized in that: Also includes: Step 16, before step 12, for each of the sample speech signals, the sample speech signal stream is subjected to noise reduction processing and echo removal processing in sequence by the preprocessing module to be trained to obtain an initial processed speech signal; Endpoint detection is performed on the initially processed speech signal to detect the starting position and the ending position of the initially processed speech signal; speech pre-emphasis processing is performed on the initially processed speech signal from the starting position to the ending position to obtain a corresponding pre-processed sample speech.

7. The speech recognition method according to claim 1, characterized in that: Also includes: Step 17, obtaining multiple original voice signals, transforming and expanding each original voice signal to obtain a new voice signal, wherein the transformation and expansion method includes at least one of the following: adding noise, changing the playback speed, changing the pitch of the voice signal, adding echo, and disturbing the filter characteristics of the voice signal, wherein the sample voice signal refers to a voice stream generated by a voice generator in any language that can express meaning; Taking the original speech signal and the new speech signal as sample speech signals, a sample speech signal set is constructed; Wherein, step 17 is performed before step 16.

8. The speech recognition method according to claim 7, characterized in that: Step 17 also includes: For application scenarios with user feature requirements, the original voice signal is formed by obtaining the natural language recording formed by the user reading a text for a certain length of time.

9. A speech recognition system, characterized in that: include: A feature extraction module training unit is used to extract corresponding sample speech feature vectors from the preprocessed sample speech through the feature extraction module to be trained for each preprocessed sample speech, wherein the sample speech feature vector retains the speech features of the speech generator, and the sample speech feature vector includes the spectral features of the speech, the time sequence information of the speech, and the contextual connection of the speech; A speech module training unit is used to extract a feature representation and a temporal dependency of speech from the sample speech feature vector through the speech module to be trained, wherein the feature representation includes a Mel-frequency cepstral coefficient of speech, a temporal feature of speech and a temporal dependency of speech; and map the spectral features of speech to corresponding phoneme labels or word labels in sequence according to the feature representation to obtain a mapping representation; A language module training unit, used for inputting the mapping representation and the temporal dependency of the speech into the language module to be trained, and combining and sorting the phonemes or words according to grammatical rules and semantic relationships through the language module to be trained according to the temporal dependency of the speech to form a plurality of probability factor sequences; A decoding module training unit, used for inputting the temporal dependency of the speech and the plurality of probability factor sequences into a decoding module to be trained, and for each probability factor sequence, judging the degree of matching between the probability factor sequence and the sample speech signal according to the temporal dependency of the speech through a dynamic programming algorithm in the decoding module to be trained, and taking the probability factor sequence with the highest matching degree as the language text of the sample speech signal and outputting it; The preprocessed sample speech is obtained based on preprocessing the sample speech signal; Among them, according to the matching degree between the output language text and the sample voice signal, the parameters of each module to be trained are adjusted, and the pre-processed sample voice is used to continue training through the feature extraction module training unit, the voice module training unit, the language module training unit, and the decoding module training unit in sequence until the output language text matches the marked text corresponding to the sample voice signal, thereby obtaining a trained voice recognition model; the voice recognition model is used to receive the voice signal to be recognized and output the corresponding language text.

10. A computer device, characterized in that: include: processor; And, a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the speech recognition method according to any one of claims 1 to 8.

Images