Method for training a speech recognition model and speech recognition method

Abstract

The application relates to a method for training a speech recognition model, comprising: providing a speech training data set comprising a plurality of speech data and speech labels corresponding to each speech data; providing a speech recognition model to be trained, the speech recognition model comprising a convolutional neural network, a first fully connected network, a recurrent neural network and a second fully connected network coupled in cascade, wherein each network comprises one or more network layers with a parameter matrix; wherein the speech recognition model is used to process speech data to generate corresponding speech recognition results; and training the speech recognition model using the speech training data set, so that after training, the parameter matrices of at least two adjacent network layers in the speech recognition model satisfy a predetermined constraint condition; and so that the accuracy of the speech recognition results of the speech recognition model on the speech data calculated using at least one loss function satisfies a predetermined recognition target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech recognition, and more particularly, to a method for training a speech recognition model and a speech recognition method based on the trained speech recognition model. BACKGROUND

[0002] Automatic speech recognition (ASR) technology emerged in the 1950s and is one of the cross fields of linguistics and computer science, which aims to automatically convert speech signals into corresponding texts by using a computing device, so as to recognize the speech commands and text contents of a speaker. In the development process of several decades, ASR technology has experienced from simple digit and syllable recognition to systematic theoretical method establishment, from original pattern matching algorithm to representative statistical algorithm of Hidden Markov Model (HMM) combined with Gaussian Mixture Model (GMM), and then to deep learning algorithm represented by neural network.

[0003] The pattern matching algorithm can only recognize simple monosyllables, and the construction process of the GMM-HMM speech recognition model is too complex and lacks automatic alignment capability. Therefore, people proposed an end-to-end speech recognition model based on deep neural network (DNN), which greatly improved the recognition accuracy while simplifying the construction process. Since each layer in the DNN uses a full connection (FC) network, the parameter scale is large and the local structure features in the speech feature space cannot be fully utilized. Therefore, people turned to the convolutional neural network (CNN) widely used in image recognition. The CNN uses a small scale convolution kernel combined with a pooling operation with down-sampling function, so that the convolution operation has translational invariance, overcoming the complex changes of speech. The cascade of multi-layer CNNs fully excavates the multi-scale structure features in the speech feature space. After the CNN processes the input features and performs preliminary classification, the recurrent neural network (RNN) models the semantic association of context through recursive relationship, further improving the recognition accuracy of the neural network model.

[0004] The currently widely used end-to-end model mainly includes two categories, one is a small and medium parameter scale model using CNN, RNN combined with connectionist temporal classification (CTC), and the other is a large scale model represented by attention mechanism. The attention model is only suitable for cloud recognition, and the CNN-LSTM-CTC model suitable for embedded devices involves a large amount of data reading and writing and operation due to the need for multiple residual network (resnet) connections and normalization operations, and the processing delay is difficult to satisfy.

[0005] Therefore, it is necessary to provide an improved method for training a speech recognition model. SUMMARY

[0006] The purpose of the present application is to provide a lightweight method for training a speech recognition model.

[0007] According to an aspect of the present application, a method for training a speech recognition model is provided, comprising: providing a speech training data set, the speech training data set comprising a plurality of speech data and speech labels corresponding to each speech data; providing a speech recognition model to be trained, the speech recognition model comprising a convolutional neural network, a first fully connected network, a recurrent neural network and a second fully connected network coupled in cascade, wherein each network comprises one or more network layers with a parameter matrix; wherein the speech recognition model is used to process speech data to generate corresponding speech recognition results; and training the speech recognition model using the speech training data set, so that after training, the parameter matrices of at least two adjacent network layers in the speech recognition model satisfy a predetermined constraint condition as follows: L cond is less than a first threshold, wherein the condition number constraint loss function wherein σmax and σmin represent the maximum and minimum singular values of the parameter matrix respectively; and the accuracy of the speech recognition results of the speech recognition model calculated using at least one loss function meets a predetermined recognition target.

[0008] In another aspect of the present application, a speech recognition method is also provided, comprising the following steps: providing a speech recognition model, the speech recognition model comprising a convolutional neural network, a first fully connected network, a recurrent neural network and a second fully connected network coupled in cascade, wherein each network comprises one or more network layers with a parameter matrix; training the speech recognition model by the following steps: providing a speech training data set, the speech training data set comprising a plurality of speech data and speech labels corresponding to each speech data; and processing the speech data in the speech training data set using the speech recognition model to generate corresponding speech recognition results; so that after training, the parameter matrices of at least two adjacent network layers in the speech recognition model satisfy a predetermined constraint condition as follows: L cond is less than a first threshold, wherein the condition number constraint loss function wherein σmax and σmin represent the maximum and minimum singular values of the parameter matrix respectively; and the accuracy of the speech recognition results of the speech recognition model calculated using at least one loss function meets a predetermined recognition target; receiving a speech input signal to be processed and pre-processing the speech input signal to generate a corresponding speech feature vector; and processing the speech feature vector using the trained speech recognition model to generate a corresponding speech recognition result.

[0009] In yet another aspect of the present application, a non-transitory computer storage medium having stored thereon one or more executable instructions that, when executed by a processor, perform the method of the above aspect is also provided.

[0010] The above is a summary of the present application, which can be simplified, generalized and omitted details, so those skilled in the art should recognize that this part is only illustrative and is not intended to limit the scope of the present application in any way. This summary part is neither intended to determine the key features or essential features of the claimed subject matter, nor intended to serve as an auxiliary means to determine the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other features of the present application will become more fully understood from the following detailed description and the accompanying drawings, in which: It is to be understood that the drawings are only schematic and are therefore not to scale, and that they are merely intended to conceptual illustrate the application and to provide a conceptual understanding of the application. The combination of features shown in the drawings is intended to explain the principles of the application and the application is not limited to the combination shown in the drawings. In the drawings:

[0012] Figure 1 A speech recognition model according to an embodiment of the present application is shown;

[0013] Figure 2 A flowchart of vector processing using a convolutional subnetwork with cross-layer connections using a residual network is shown;

[0014] Figure 3 A method for training a speech recognition model according to an embodiment of the present application is shown;

[0015] Figure 4 A comparison of recognition error rates of a speech recognition model trained using multiple loss functions and a speech recognition model trained using a single loss function is shown. DETAILED DESCRIPTION

[0016] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments can be utilized, and other changes can be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present application, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.

[0017] Figure 1A block diagram of a speech recognition model 100 according to one embodiment of this application is shown. In some embodiments, the speech recognition model 100 may be implemented in hardware, software, or a combination of both. The hardware portion may be implemented using dedicated logic; the software portion may be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the apparatus, devices, and methods of this application may be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier.

[0018] like Figure 1 As shown, the speech recognition model 100 includes a feature extraction module 101, which preprocesses the original speech input signal and generates a corresponding speech feature vector. For example, the feature extraction module 101 may be coupled to a microphone or similar audio acquisition or input device to receive the acquired or input speech input signal. In some embodiments, the feature extraction module 101 may include an analog-to-digital converter (ADC) for converting the received analog-formatted speech input signal into a digital signal for subsequent processing; in other embodiments, the feature extraction module 101 may not include an ADC, for example, when the original speech input signal is in digital format. In some embodiments, the analog-formatted speech input signal may be sampled at a predetermined sampling rate (e.g., 16 kHz); and the sampled speech data points may be framed according to a predetermined frame length (e.g., 10 milliseconds) to generate multiple speech data frames with a predetermined frame length. These consecutive multiple speech data frames constitute the speech feature vector, which reflects the feature information of the original speech input signal. Each speech data frame may include a corresponding number of speech data points. For example, when acquiring a speech input signal at a sampling rate of 16kHz and a frame length of 10 milliseconds, each speech data frame can have 160 speech data points. It is understood that in the aforementioned example, the frame length is expressed as a time length; in other cases, the frame length can also be expressed as the number of speech data points, such as a frame length of 160 or 256 speech data points. In this case, the sampling rate of the speech data points and the number of speech data points per frame correspond to the frame length expressed as a time length. In some optional embodiments, the framing processing of the speech signal acquisition does not involve back-to-back segmentation of speech data points, but rather overlapping segmentation, i.e., introducing frame shift between different speech data frames. It is understood that overlapping segmentation of speech data points can increase the correlation between adjacent or nearby speech data frames.

[0019] In some embodiments, each speech data frame is a time-domain signal, and the speech data points included therein are sampled in time sequence. In other embodiments, each speech data frame can also be subjected to a fast Fourier transform process to obtain a signal in frequency domain format.

[0020] Still referring to Figure 1 As shown, the speech recognition model 100 further includes a convolutional neural network 103, a first fully connected network 105, a recurrent neural network 107, and a second fully connected network 109 coupled in cascade, wherein the convolutional neural network 103 is coupled to the feature extraction module 101 to receive the speech feature vector therefrom. In this way, the speech feature vector can be sequentially processed by the aforementioned networks to generate a speech recognition result corresponding to the speech feature vector.

[0021] It can be appreciated that for the convolutional neural network 103, the first fully connected network 105, the recurrent neural network 107, and the second fully connected network 109 in the speech recognition model 100, each of the networks can respectively include different weight coefficient matrices, and the speech recognition model 100 can be trained using a predetermined speech training data set to determine the values of the weight coefficient matrices of the networks respectively. For example, the convolutional neural network 103 can include one or more convolution kernel matrices, each of which can be subjected to a convolution operation with the input speech feature vector; wherein the convolution kernel matrix can be configured as a weight coefficient matrix composed of a plurality of weight coefficients. The fully connected network and the recurrent neural network can also have similar weight coefficient matrices. In this way, the trained speech recognition model 100 can be used to process unknown speech input signals and generate speech recognition results corresponding to the speech in the speech input signals.

[0022] The convolutional neural network 103 can include one or more convolutional sub-networks. In some embodiments, the convolutional sub-networks can include a convolutional layer, a non-linear activation layer, and a pooling layer. Each convolutional layer includes a convolution kernel matrix, which can be used to perform a point-by-point convolution operation on the input speech feature vector (usually a two-dimensional time-frequency vector) to extract feature information of the speech feature vector. Different convolution kernel matrices can have the same or different dimensions. Then, the non-linear activation layer can perform a non-linear operation on the convolution-processed speech feature vector, thereby setting all elements less than 0 in the speech feature vector to zero. Finally, the pooling layer can further perform a pooling operation on the non-linearly activated speech feature vector to reduce its dimension while retaining most of the important information therein (some unimportant samples in the speech feature vector can be removed). It can be appreciated that in some embodiments, one or more convolutional sub-networks can also only include a convolutional layer without, for example, a non-linear activation layer and a pooling layer.

[0023] Figure 2A flowchart of vector processing using a convolution subnetwork with cross-layer connection of a residual network (resnet) is shown. As shown in Figure 2 When the current convolution processing is performed, the input feature vector needs to be saved in the memory, and after the nonlinear processing is completed, the sum operation is performed with the current operation result, and then the output feature vector is generated after the sum and pooling processing. As can be seen, this vector processing needs to write and read the input feature vector in and out of the memory before each convolution processing, and the input two-dimensional speech feature vector often has a large dimension, which will cause a large amount of data reading and writing.

[0024] In order to avoid the problems caused by the cross-layer connection of the residual network, in some embodiments of the present application, when processing the input two-dimensional speech feature vector, such as recognition or training, the two-dimensional speech feature vector can be converted by a convolution operation with a step of 1, and the converted vector is stored in the memory. This can make the two-dimensional speech feature vector stored in the memory be stored continuously. In addition, since the cross-layer connection is not used, the data after the current operation does not need to be stored in the memory, which can avoid a large amount of data reading and writing operation, thereby improving the operation efficiency. Wherein, the step refers to the amount of window movement of the convolution kernel matrix relative to the speech feature vector.

[0025] Still referring to Figure 1 , the convolutional neural network 103 is coupled to the first fully connected network 105, which is used to make global association of information at different times / frequencies. In some embodiments, the first fully connected network 105 can include one or more fully connected layers. The more the number of fully connected layers, the higher the precision of information processing, but the more the amount of weight parameters. In actual application, a balance can be made between the information processing precision and the processing capacity demand, and an appropriate number of fully connected layers is selected. For example, in some embedded devices or other devices with limited processing / computing resources, the first fully connected network 105 can include a small number of fully connected layers, for example, 1 layer.

[0026] The first fully connected network 105 is coupled with a recurrent neural network 107. For sequence vectors like speech feature vectors, using a recurrent neural network such as a gated recurrent neural network helps to utilize the state of the previous time step when solving the state of the current time step, because in the model of the recurrent neural network, the state of the current time step is determined by the input of the current time step and the state of the previous time step. In some other embodiments, the recurrent neural network 107 can be a long short-term memory based or a gated recurrent unit based network, or can be other types of recurrent neural networks. For a gated recurrent unit based recurrent neural network, a single unit thereof can control both reset and update at the same time; where the update function is used to control the degree to which the state information of the previous time step is brought into the current state, and the reset function controls how much information of the state information of the previous time step is written onto the current candidate set.

[0027] The recurrent neural network 107 is coupled with a second fully connected network 109, which further processes the feature vectors after the previous stage of processing, and provides processed speech recognition results corresponding to the speech feature vectors to an output layer of the speech recognition model 100. For the application scenario of Chinese speech recognition, the speech recognition results are, for example, Chinese pinyin. It can be understood that for application scenarios for processing other languages, the speech recognition results can also be, for example, phonetic or orthographic transcriptions of other languages.

[0028] In actual applications, Figure 1 The illustrated speech recognition model 100 needs to be trained to determine the values of the various weight parameter matrices therein. In some embodiments of the present application, in order to train the speech recognition model 100, Figure 1 The illustrated speech recognition model 100 is trained by providing a training method as illustrated in Figure 3 The illustrated speech recognition model 100 is trained by providing a training method as illustrated in

[0029] As illustrated in Figure 3 In step 302, a speech training data set can be provided, which includes a plurality of speech data and speech labels corresponding to each speech data. The speech data can have a similar format to the speech feature vectors to be recognized, that is, it is also a two-dimensional time-frequency vector. In some embodiments, open source speech data with speech labels can be used as the speech training data set, such as the Aishell data set. Those skilled in the art can understand that different speech training data sets can be used to train the speech recognition model 100 according to different recognition objects.

[0030] Then, in step 304, the speech recognition model 100 as illustrated in Figure 1 is trained using the speech training data set. Specifically, the convolutional neural network 103, the first fully connected network 105, and the second fully connected network 109 in the speech recognition model 100 can be subjected to a condition number constraint condition.

[0031] In particular, for a matrix A, the condition number in p-norm is defined by equation (1):

[0032] cond p (A) = ||A||p p ||A -1 ||p p (1)

[0033] where p is a positive integer. For example, in 2-norm, the condition number can be calculated by singular value decomposition as shown in equation (2):

[0034]

[0035] where σ max and σ min are the maximum and minimum singular values of the weight parameter matrix, respectively. The condition number L cond The constraint loss function is determined by equation (3) as follows:

[0036]

[0037] Experimental results show that the constraint loss function of the condition number L cond tends to 1, which can ensure that the dynamic range of each element in the matrix A remains consistent.

[0038] In addition, in order to avoid normalization processing, i.e., to ensure that the dynamic range of each parameter matrix remains consistent, it can be known from inequality (4) as follows:

[0039] σ min (A)·||x||2≤||Ax||2≤σ max (A)·||x||2 (4)

[0040] where if the input matrix A is a matrix of dimension m*n, then x is any input vector of dimension n*1, and the output matrix Ax is not equal to 0, i.e., x is any vector not belonging to the null space of the matrix A.

[0041] The sufficient condition for the dynamic range of the input matrix A and the output matrix Ax to remain consistent is shown in equation (5) as follows:

[0042] σ min →1←σ max (5)

[0043] That is, the maximum and minimum singular values of the matrix A should tend to 1, for example, the difference between the maximum and minimum singular values of the matrix A and 1 is less than 0.2. Therefore, the modified condition number L cond The constraint loss function is shown in equation (6) as follows:

[0044]

[0045] In some embodiments, the modified condition number constrained loss function can have a value less than a first threshold, for example, less than 20%, preferably, less than 5%.

[0046] Therefore, in some embodiments, the modified condition number constrained loss function as shown in equation (6) can be applied to each layer in the convolutional neural network 103, the first fully connected network 105 and the second fully connected network 109 during the training process in step 304, so that the dynamic range of the effective parameters in each convolution kernel matrix and parameter matrix of the fully connected layer in the trained speech recognition model 100 is kept within a consistent range. After being trained in this way, unnecessary normalization operations on the parameter matrix can be avoided, and the processing resources required for repeated reading and writing of data can be reduced, so that the trained speech recognition model 100 is convenient for deployment on embedded devices or other devices with limited computing / processing resources. If the above method is not used, the dynamic range of the effective parameters in the convolution kernel matrix and the parameter matrix in the trained speech recognition model will not be constrained, and will be widely distributed, which makes it difficult to deploy on embedded devices.

[0047] Still referring to Figure 1 As shown, in order to evaluate the accuracy of the speech recognition result output by the speech recognition model 100, in some embodiments, the speech recognition model 100 can include at least one loss function for measuring or calibrating the accuracy of the speech recognition result. In Figure 1 In the embodiment shown in

[0048] Specifically, in the process of training the speech recognition model 100 using the training method shown in Figure 3 During the training of the speech recognition model 100 using the training method shown in

[0049] Correspondingly, in step 306 shown in Figure 3

[0050] ​It can be understood that each loss function corresponds to a speech training data set. For example, for the application of Chinese speech recognition, the loss function can be a pinyin tone loss function, i.e., the accuracy of the speech recognition result is calculated based on the label including the pinyin tone (the probability that the speech recognition result conforms to the predetermined pinyin tone label). For another example, for the application of Chinese speech recognition, the loss function can also be a toneless pinyin loss function, i.e., the accuracy of the speech recognition result is calculated based on the label including the toneless pinyin (the probability that the speech recognition result conforms to the predetermined toneless pinyin label). In some embodiments, a tone loss function can also be provided, i.e., the accuracy of the speech recognition result is calculated based on the label of the tone (the probability that the speech recognition result conforms to the predetermined tone label). Preferably, these loss functions can be used together to improve the training speed of the speech recognition model in the form of multi-task calculation.

[0051] In some embodiments, the loss function can be a connectionist temporal classification (CTC) loss function. The CTC loss function has the characteristics of dealing with syllable length and reasonably removing redundant judgments, and is particularly suitable for speech recognition. In one embodiment, the CTC loss function CTC(x) can be constructed by the following method:

[0052] For an input sequence x = (x1, x2, …, x T ) of a given length T and the corresponding output label sequence l, the CTC loss function CTC(x) is defined as the negative logarithm of the conditional probability of l given x:

[0053]

[0054] where B represents the mapping relationship from the path to the label sequence, represents the probability of observing the label π t in the t-th frame; the label refers to the Chinese pinyin corresponding to a certain speech feature vector in the speech training data set (taking the Chinese speech training data set as an example).

[0055] For example, Figure 1In the illustrated embodiment, the pinyin tone loss function CTC1(x), the tone loss function CTC2(x), and the toneless pinyin loss function CTC3(x) are used together for training of the speech recognition model. These loss functions can all take the form of the CTC(x) loss function shown in equation (7), with the only difference being the speech feature labels used in training (depending on the corresponding speech training dataset). The pinyin tone loss function CTC1(x) can be calculated based on the speech recognition result output by the second fully connected network and the pinyin tone labels in the speech training dataset, and corresponds to the pinyin with tone that has the highest output probability, i.e., the loss function associated with this pinyin with tone. Similarly, the tone loss function CTC2(x) corresponds to the tone that has the highest output probability, i.e., the loss function associated with this tone (as the label); and the toneless pinyin loss function CTC3(x) corresponds to the toneless pinyin that has the highest output probability, i.e., the loss function associated with this toneless pinyin (as the label). For example, a predetermined recognition target can be set for each loss function, such as a speech recognition accuracy of no less than 85%, 90%, or 95%, etc. Alternatively, the loss functions can be weighted, and the speech recognition accuracy calculated using the weighted loss functions can be made to satisfy the predetermined recognition target. For example, the speech recognition accuracy CTC'(x) can be calculated in the manner shown in equation (8):

[0056] CTC'(x) = a*CTC1(x) + b*CTC2(x) + c*CTC3(x) (8)

[0057] where a, b, c are predetermined weighting coefficients, which can be determined according to actual application. For example, in some embodiments, the sum of a, b, and c can be 1. For example, a can be 0.5, b can be 0.25, and c can be 0.25. Accordingly, the predetermined recognition target can be set to a recognition accuracy of no less than 90%, 95%, etc. In the case where multiple loss functions are provided to evaluate the speech recognition model, the training method of the speech recognition model can take full advantage of the acoustic rules of Chinese speech, set the training process as multi-task training, so that the speech recognition model converges quickly when the parameter size is small, and improves the accuracy of speech recognition.

[0058] Still referring to Figure 1To provide the speech recognition result to different loss functions, the second fully connected network 109 can include a fully connected sub-network 109A, a fully connected sub-network 109B, and a fully connected sub-network 109C. Among them, the fully connected sub-network 109A is cascaded with the tone loss function CTC2(x) 111, the fully connected sub-network 109B is cascaded with the pinyin tone loss function CTC1(x) 113, and the fully connected sub-network 109C is cascaded with the toneless pinyin loss function CTC3(x) 115. By using three different fully connected networks to connect with respective loss functions, the second fully connected network 109 can output a one-dimensional vector corresponding to each loss function (speech recognition result, which can be pinyin tone, toneless pinyin, or tone) to each loss function.

[0059] It can be understood that when training the speech recognition model by using the training method shown in Figure 3 It can be understood that when training the speech recognition model by using the training method shown in

[0060] Figure 4 A comparison diagram of the recognition error rate of the speech recognition model trained by using multiple loss functions and the recognition error rate of the speech recognition model trained by using a single loss function is shown. Among them, the curve 402 shows the result of training by using 3 loss functions, and the curve 404 shows the result of training by using a single pinyin tone loss function. From Figure 4 It can be seen that in the case of fewer training rounds, the recognition error rate using multiple loss functions is significantly lower than that using a single loss function.

[0061] In some embodiments, the present application also provides some computer program products, which include a non-transitory computer readable storage medium. The non-transitory computer readable storage medium includes computer executable code for implementing the speech recognition method and the method for training the speech recognition model described in the embodiments of the present application. In some embodiments, the computer program product can be stored in a hardware device, such as an audio device or other computing device.

[0062] Embodiments of the application can be implemented in hardware, software, or a combination of both. Hardware portions can be implemented in dedicated logic; software portions can be stored in memory and executed by suitable instruction execution systems including microprocessors or dedicated design hardware. Those of ordinary skill in the art will appreciate the foregoing apparatus and method can be implemented using computer executable instructions and / or in processor control code, for example provided on a carrier medium such as a disk, CD or DVD ROM, programmable memory such as read-only memory (firmware) or data carrier such as an optical or electrical signal carrier. The apparatus of the application and its modules can be implemented by hardware circuitry such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, and the like, by software executed by various types of processors, or by a combination of both, such as firmware.

[0063] Those of ordinary skill in the art will appreciate the foregoing disclosure and practice the disclosed implementations as understood by one of ordinary skill in the art upon inspection of the drawings, disclosure and appended claims. In the claims, the article "a" is not excluded from the definition of "one"; and the indefinite articles "a" and "an" are not limited to the singular. In the practical application of the application, one part can perform the function of several technical features cited in the claims. Any reference signs in the claims should not be understood as limiting the scope.

Claims

1. A method for training a speech recognition model, the method comprising: The method comprises: providing a speech training dataset, the speech training dataset comprising a plurality of speech data and speech labels corresponding to each speech data; providing a speech recognition model to be trained, the speech recognition model comprising a convolutional neural network, a first fully connected network, a recurrent neural network and a second fully connected network coupled in cascade, wherein each network comprises one or more network layers with parameter matrices; wherein the speech recognition model is configured to process speech data to generate corresponding speech recognition results; and training the speech recognition model using the speech training dataset, such that after training, the parameter matrices of at least two adjacent network layers in the speech recognition model satisfy a predetermined constraint condition as follows: L cond less than the first threshold, wherein the condition number constraint loss function satisfies the equation where σ max and σ min respectively represent the maximum singular value and the minimum singular value of the parameter matrix; and such that the accuracy of the speech recognition results of the speech recognition model calculated using at least one loss function satisfies a predetermined recognition target.

2. The method of claim 1, wherein, The speech training dataset is a Chinese speech training dataset, and the at least one loss function comprises a pinyin tone loss function, a tone loss function and a toneless pinyin loss function.

3. The method of claim 2, wherein, such that the accuracy of the speech recognition results of the speech recognition model calculated using at least one loss function satisfies a predetermined recognition target comprises: such that the accuracy calculated using each loss function exceeds a predetermined recognition threshold.

4. The method of claim 2, wherein, such that the accuracy of the speech recognition results of the speech recognition model calculated using at least one loss function satisfies a predetermined recognition target comprises: such that the weighted value of the accuracy calculated using each loss function exceeds a predetermined recognition threshold.

5. The method according to any one of claims 2 to 4, characterized in that, The at least one loss function is a connectionist temporal classification loss function.

6. The method according to any one of claims 2 to 4, characterized in that, The second fully connected network comprises at least one fully connected subnetwork, and each fully connected subnetwork is configured to couple a corresponding loss function to provide speech recognition results to the loss function.

7. The method of claim 1, wherein, The recurrent neural network comprises a gated recurrent neural network.

8. The method of claim 1, wherein, The convolutional neural network comprises a convolution kernel matrix, and the window moving step of the convolution kernel matrix is 1 when processing speech data.

9. A voice recognition method, characterized by, The speech recognition method comprises the following steps: providing a speech recognition model, the speech recognition model comprising a convolutional neural network, a first fully connected network, a recurrent neural network and a second fully connected network coupled in cascade, wherein each network comprises one or more network layers with parameter matrices; training the speech recognition model using the following steps: providing a speech training dataset, the speech training dataset comprising a plurality of speech data and speech labels corresponding to each speech data; and processing the speech data in the speech training dataset using the speech recognition model to generate corresponding speech recognition results; such that after training, the parameter matrices of at least two adjacent network layers in the speech recognition model satisfy a predetermined constraint condition as follows: L cond less than the first threshold, wherein the condition number constraint loss function satisfies the equation where σ max and σ min respectively represent the maximum singular value and the minimum singular value of the parameter matrix; and such that the accuracy of the speech recognition results of the speech recognition model calculated using at least one loss function satisfies a predetermined recognition target; receiving a speech input signal to be processed and pre-processing the speech input signal to generate a corresponding speech feature vector; and processing the speech feature vectors with the trained speech recognition model to generate corresponding speech recognition results.

10. The voice recognition method of claim 9, wherein, The speech training dataset is a Chinese speech training dataset, and the at least one loss function includes a pinyin tone loss function, a tone loss function, and a toneless pinyin loss function.

11. The voice recognition method of claim 10, wherein, The accuracy of the speech recognition results of the speech data by the speech recognition model calculated using the at least one loss function meets a predetermined recognition target, including: The accuracy calculated using each loss function exceeds a predetermined recognition threshold.

12. The voice recognition method of claim 10, wherein, The accuracy of the speech recognition results of the speech data by the speech recognition model calculated using the at least one loss function meets a predetermined recognition target, including: The weighted value of the accuracy calculated using each loss function exceeds a predetermined recognition threshold.

13. The speech recognition method of any one of claims 10-12, wherein, The at least one loss function is a connectionist temporal classification loss function.

14. The speech recognition method of any one of claims 10-12, wherein, The second fully connected network includes at least one fully connected subnetwork, and each fully connected subnetwork is coupled to a corresponding loss function to provide the speech recognition results to the loss function.

15. The voice recognition method of claim 9, wherein, The recurrent neural network includes a gated recurrent neural network.

16. The voice recognition method of claim 9, wherein, The convolutional neural network includes a convolution kernel matrix, and a window moving step of the convolution kernel matrix is 1 when processing the speech data.

17. A non-transitory computer storage medium having stored thereon one or more executable instructions that, when executed by a processor, perform the following steps: providing a speech training dataset, the speech training dataset including a plurality of speech data and a speech label corresponding to each speech data; A speech recognition model to be trained is provided, the speech recognition model to be trained comprising a convolutional neural network, a first fully connected network, a recurrent neural network and a second fully connected network coupled in cascade, wherein each network comprises one or more network layers with a matrix of parameters; wherein the speech recognition model is configured to process the speech data to generate corresponding speech recognition results; and training the speech recognition model using the speech training dataset, such that after training, parameter matrices of at least two adjacent network layers in the speech recognition model meet a predetermined constraint condition as follows: L cond less than the first threshold, wherein the condition number constraint loss function satisfies the equation wherein σ max and σ min respectively represent the maximum singular value and the minimum singular value of the parameter matrix; and The accuracy of the speech recognition results of the speech data by the speech recognition model calculated using the at least one loss function meets a predetermined recognition target.

18. A non-transitory computer storage medium having stored thereon one or more executable instructions that, when executed by a processor, perform the following steps: providing a speech recognition model, the speech recognition model including a convolutional neural network, a first fully connected network, a recurrent neural network, and a second fully connected network coupled in cascade, wherein each network includes one or more network layers having parameter matrices; training the speech recognition model using the following steps: A voice training dataset is provided, the voice training dataset including a plurality of voice data and a voice label corresponding to each voice data; and processing the speech data in the speech training dataset using the speech recognition model to generate corresponding speech recognition results; such that after training, parameter matrices of at least two adjacent network layers in the speech recognition model meet a predetermined constraint condition as follows: L cond less than the first threshold, wherein the condition number constrains the loss function to satisfy the equation where σ max and σ min respectively represent the maximum and minimum singular values of the parameter matrix; and The accuracy of the speech recognition results of the speech data by the speech recognition model calculated using the at least one loss function meets a predetermined recognition target. receiving a speech input signal to be processed and pre-processing the speech input signal to generate corresponding speech feature vectors; and processing the speech feature vectors with a trained speech recognition model to generate corresponding speech recognition results.

Images