Chinese long speech recognition method, device and equipment and storage medium

Abstract

The present application relates to the technical field of artificial intelligence, and is applied to the field of smart government affairs, and provides a Chinese long speech recognition method, device and equipment and storage medium, which are used for improving the accuracy of mandarin low-delay speech recognition in a long time period. The Chinese long speech recognition method comprises the following steps: calculating the mel-frequency cepstrum coefficient of the segmented Chinese long speech to be processed to obtain a spectrogram; performing character recognition on the spectrogram by using a target deep speech recognition model to obtain a text sequence to be processed, the target deep speech recognition model is an end-to-end model based on connection time sequence classification, and the target deep speech recognition model comprises two convolution layers, four bidirectional long short-term memory networks, one look-ahead layer and one fully connected layer; estimating the probability of the text sequence to be processed by using a language model in the target deep speech recognition model, and determining a target recognition text sequence based on score data. In addition, the present application also relates to the technical field of block chain, and the target recognition text sequence can be stored in a block chain.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of speech semantics of artificial intelligence, and in particular to a Chinese long speech recognition method, device and equipment and storage medium. BACKGROUND

[0002] Language is the most important means of human communication, and its complexity and expressive possibilities allow for extensive exchange of information. In recent years, applications that recognize and understand human language have become increasingly popular. Both applications that recognize and understand human language use Automatic Speech Recognition (ASR) to convert human speech into text before performing subsequent work. Prior to the development of deep learning, most ASR systems used Hidden Markov Models (HMM) or variants thereof. Although these methods achieved good word error rates, they were very slow for recognizing large vocabularies and could not be used for real-time speech recognition.

[0003] At present, the speech recognition method based on deep learning can achieve high accuracy, but in the actual application process, most of the speech scenes are in the presence of noise, and the noise interference will affect the accuracy of recognition, thereby resulting in low accuracy of long period of time Mandarin low delay speech recognition. SUMMARY

[0004] The present application provides a Chinese long speech recognition method, device, equipment and storage medium, which is used to improve the accuracy of long period of time Mandarin low delay speech recognition.

[0005] The first aspect of the present application provides a Chinese long speech recognition method, comprising:

[0006] Obtaining a segmented Chinese long speech to be processed, calculating the mel-frequency cepstral coefficient of the segmented Chinese long speech to be processed, and obtaining a spectrogram;

[0007] Performing character recognition on the spectrogram through a pre-trained target deep speech recognition model to obtain a text sequence to be processed, the pre-trained target deep speech recognition model being an end-to-end model based on connection time sequence classification, and the target deep speech recognition model comprising two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer and one fully connected layer;

[0008] Performing probability estimation on the text sequence to be processed through a language model in the target deep speech recognition model to obtain score data;

[0009] Determining a target recognition text sequence based on the score data.

[0010] Optionally, in the first implementation manner of the first aspect, the pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, and the target deep speech recognition model comprises two convolution layers, four bidirectional long short-term memory networks, one look-ahead layer and one fully connected layer, and the two convolution layers, the four bidirectional long short-term memory networks, the one look-ahead layer and the one fully connected layer all adopt batch normalization; the character recognition on the spectrogram is performed by using the pre-trained target deep speech recognition model to obtain the to-be-processed text sequence, comprising:

[0011] the spectrogram is subjected to convolution processing by using the two convolution layers in the pre-trained target deep speech recognition model to obtain a filter mapping sequence;

[0012] the filter mapping sequence is subjected to time sequence semantic feature extraction by using the four bidirectional long short-term memory networks to obtain sequence semantic features;

[0013] the sequence semantic features are subjected to neuron activation learning by using the one look-ahead layer to obtain context-based text information;

[0014] the context-based text information is subjected to vector projection by using the one fully connected layer to obtain an initial text vector;

[0015] the initial text vector is subjected to data processing to obtain the to-be-processed text sequence.

[0016] Optionally, in the second implementation manner of the first aspect, the pre-trained target deep speech recognition model comprises a connection time sequence classification network, and the data processing on the initial text vector to obtain the to-be-processed text sequence comprises:

[0017] the initial text vector is subjected to normalization processing to obtain a normalized text vector;

[0018] the normalized text vector is subjected to operation by using the connection time sequence classification network to obtain the to-be-processed text sequence.

[0019] Optionally, in the third implementation manner of the first aspect, the probability estimation on the to-be-processed text sequence by using the language model in the target deep speech recognition model to obtain the score data comprises:

[0020] the number of words in the to-be-processed text sequence, a target weight and a preference parameter are acquired, the target weight is used to indicate the adjustment of the weights of the language model and the connection time sequence classification network in the target deep speech recognition model, and the preference parameter is used to control the preference of a decoder to a smaller number of long phrases or a larger number of short words;

[0021] The language model in the target deep speech recognition model is used to perform calculation based on the number of words in the to-be-processed text sequence, the target weight, the preference parameter and the to-be-processed text sequence, to obtain score data.

[0022] Optionally, in a fourth implementation manner of the first aspect, the obtaining of the segmented to-be-processed Chinese long speech and the obtaining of the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech to obtain a spectrogram comprises:

[0023] The original Chinese long speech is obtained, and data preprocessing is performed on the original Chinese long speech to obtain an initial Chinese long speech.

[0024] Frame cutting is performed on the initial Chinese long speech to obtain a segmented initial Chinese long speech.

[0025] Signal processing is performed on the segmented initial Chinese long speech to obtain a segmented to-be-processed Chinese long speech.

[0026] The mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech is calculated to obtain a spectrogram.

[0027] Optionally, in a fifth implementation manner of the first aspect, the signal processing performed on the segmented initial Chinese long speech to obtain a segmented to-be-processed Chinese long speech comprises:

[0028] The segmented initial Chinese long speech is denoised through a preset single-channel audio noise reduction algorithm to obtain a preprocessed Chinese long speech.

[0029] The preprocessed Chinese long speech is subjected to high-frequency signal enhancement through a preset high-pass filtering algorithm to obtain a segmented to-be-processed Chinese long speech.

[0030] Optionally, in a sixth implementation manner of the first aspect, before the obtaining of the segmented to-be-processed Chinese long speech and the obtaining of the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech to obtain a spectrogram, the method further comprises:

[0031] An initial deep speech recognition model is constructed, the initial deep speech recognition model comprising two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, one fully connected layer, a connection time sequence classification network and a speech model, the two convolutional layers, the four bidirectional long short-term memory networks, the one look-ahead layer and the one fully connected layer all adopting batch normalization, and the language model being used for scoring.

[0032] A target speech training sample set is obtained, the target speech training sample set being a preprocessed set of Mandarin low-delay speeches in various scenes and long time periods.

[0033] The initial deep speech recognition model is trained end-to-end through the target speech training sample set and a connection time sequence classification loss function of the connection time sequence classification network, to obtain a target deep speech recognition model.

[0034] The second aspect of the present application provides a Chinese long speech recognition device, comprising:

[0035] A calculation module is configured to obtain a segmented Chinese long speech to be processed, calculate the mel-frequency cepstrum coefficient of the segmented Chinese long speech to be processed, and obtain a spectrogram;

[0036] An identification module is configured to perform character recognition on the spectrogram through a pre-trained target deep speech recognition model to obtain a text sequence to be processed, wherein the pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, and the target deep speech recognition model comprises two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer;

[0037] An estimation module is configured to perform probability estimation on the text sequence to be processed through a language model in the target deep speech recognition model to obtain score data;

[0038] A determination module is configured to determine a target recognition text sequence based on the score data.

[0039] Optionally, in the first implementation manner of the second aspect of the present application, the pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, the target deep speech recognition model comprises two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer, the two convolutional layers, the four bidirectional long short-term memory networks, the one look-ahead layer, and the one fully connected layer all adopt batch normalization, and the identification module comprises:

[0040] A convolution processing unit is configured to perform convolution processing on the spectrogram through the two convolutional layers in the pre-trained target deep speech recognition model to obtain a filter mapping sequence;

[0041] An extraction unit is configured to perform time sequence semantic feature extraction on the filter mapping sequence through the four bidirectional long short-term memory networks to obtain sequence semantic features;

[0042] A learning unit is configured to perform neuron activation learning on the sequence semantic features through the one look-ahead layer to obtain context-based text information;

[0043] A mapping unit is configured to perform vector projection on the context-based text information through the one fully connected layer to obtain an initial text vector;

[0044] The first data processing unit is configured to perform data processing on the initial text vector to obtain a to-be-processed text sequence.

[0045] Optionally, in a second implementation manner of the second aspect, the pre-trained target deep speech recognition model comprises a connection time sequence classification network, and the first data processing unit is specifically configured to:

[0046] perform normalization processing on the initial text vector to obtain a normalized text vector;

[0047] perform operation on the normalized text vector through the connection time sequence classification network to obtain the to-be-processed text sequence.

[0048] Optionally, in a third implementation manner of the second aspect, the estimation module is specifically configured to:

[0049] obtain a number of words in the to-be-processed text sequence, a target weight and a preference parameter, the target weight being used to indicate adjustment of weights of a language model and the connection time sequence classification network in the target deep speech recognition model, and the preference parameter being used to control preference of a decoder to a smaller number of long phrases or a larger number of short words;

[0050] perform operation on the to-be-processed text sequence based on the number of words in the to-be-processed text sequence, the target weight, the preference parameter and the to-be-processed text sequence through the language model in the target deep speech recognition model to obtain score data.

[0051] Optionally, in a fourth implementation manner of the second aspect, the calculation module comprises:

[0052] The second data processing unit is configured to obtain an original Chinese long speech, and perform data preprocessing on the original Chinese long speech to obtain an initial Chinese long speech.

[0053] The cutting unit is configured to perform frame cutting on the initial Chinese long speech to obtain a cut initial Chinese long speech.

[0054] The signal processing unit is configured to perform signal processing on the cut initial Chinese long speech to obtain a segmented to-be-processed Chinese long speech.

[0055] The calculation unit is configured to calculate a mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech to obtain a spectrogram.

[0056] Optionally, in a fifth implementation manner of the second aspect, the signal processing unit is specifically configured to:

[0057] The preprocessed Chinese long speech is obtained by denoising the cut initial Chinese long speech through a preset single-channel audio denoising algorithm.

[0058] The segmented Chinese long speech to be processed is obtained by strengthening high-frequency signals of the preprocessed Chinese long speech through a preset high-pass filtering algorithm.

[0059] Optionally, in a sixth implementation manner of the second aspect of the present application, the Chinese long speech recognition device further comprises:

[0060] The constructing module is configured to construct an initial deep speech recognition model, the initial deep speech recognition model comprising two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, one fully connected layer, a connection time sequence classification network and a speech model, the two convolutional layers, the four bidirectional long short-term memory networks, the one look-ahead layer and the one fully connected layer all adopting batch normalization, and the language model being configured to score.

[0061] The obtaining module is configured to obtain a target speech training sample set, the target speech training sample set being a preprocessed set of Mandarin low-delay speech in each scene for a long time period.

[0062] The training module is configured to perform end-to-end training on the initial deep speech recognition model through the target speech training sample set and a connection time sequence classification loss function of the connection time sequence classification network, to obtain a target deep speech recognition model.

[0063] The third aspect of the present application provides a Chinese long speech recognition device, comprising a memory and at least one processor, the memory storing a computer program; the at least one processor invokes the computer program in the memory, so that the Chinese long speech recognition device executes the above-mentioned Chinese long speech recognition method.

[0064] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, when the computer program is run on a computer, the computer program makes the computer execute the above-mentioned Chinese long speech recognition method.

[0065] The technical scheme provided by the present application comprises the following steps: obtaining segmented Chinese long speech to be processed, calculating the mel-frequency cepstral coefficient of the segmented Chinese long speech to be processed to obtain a spectrogram, performing character recognition on the spectrogram through a pre-trained target deep speech recognition model to obtain a text sequence to be processed, the pre-trained target deep speech recognition model being an end-to-end model based on connection time sequence classification, the target deep speech recognition model comprising two convolution layers, four bidirectional long short-term memory networks, one look-ahead layer and one fully connected layer, performing probability estimation on the text sequence to be processed through a language model in the target deep speech recognition model to obtain score data, and determining a target recognition text sequence based on the score data. In the embodiment of the present application, the segmented Chinese long speech to be processed is recognized through the combination of the target deep speech recognition model and the language model for additional scoring, so that the noise interference can be eliminated, the accuracy of the predicted text is improved, and the accuracy of the long-period Mandarin low-delay speech recognition is improved. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 An embodiment diagram of the method for recognizing Chinese long speech in the embodiment of the present application;

[0067] Figure 2 Another embodiment diagram of the method for recognizing Chinese long speech in the embodiment of the present application;

[0068] Figure 3 An embodiment diagram of the device for recognizing Chinese long speech in the embodiment of the present application;

[0069] Figure 4 Another embodiment diagram of the device for recognizing Chinese long speech in the embodiment of the present application;

[0070] Figure 5 An embodiment diagram of the equipment for recognizing Chinese long speech in the embodiment of the present application. DETAILED DESCRIPTION

[0071] The method, device, equipment and storage medium for recognizing Chinese long speech provided by the embodiment of the present application improve the accuracy of long-period Mandarin low-delay speech recognition.

[0072] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-described drawings, if any, are used to distinguish between similar objects and are not necessarily used to describe a particular sequential or chronological order. It is to be understood that the use of the terms so-termed, where appropriate, can be interchanged with each other to the extent that embodiments described herein can be carried out in sequences other than those illustrated or described herein. Furthermore, the terms "comprise" or "have" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that includes a list of steps or units not necessarily limited to those explicitly listed, but can include other steps or units not expressly listed or inherent to such processes, methods, products or apparatus.

[0073] Embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (artificial intelligence, AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0074] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. Several major directions.

[0075] For ease of understanding, the specific process of the embodiments of the present application is described below. Please refer to Figure 1 An embodiment of the method for recognizing Chinese long speech in the embodiments of the present application includes:

[0076] 101, obtain the segmented Chinese long speech to be processed, calculate the mel-frequency cepstrum coefficient of the segmented Chinese long speech to be processed, and obtain a spectrogram.

[0077] It can be understood that the execution subject of the present application can be a Chinese long speech recognition device, and can also be a terminal or a server, which is not limited here. The embodiments of the present application take the server as an example to illustrate the execution subject. The server can be an independent server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (content delivery network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0078] The server acquires original sound data, which is a long period of Mandarin low-delay speech. Based on a preset spectrum entropy-based voice endpoint detection algorithm, the original sound data is detected for a starting point and an ending point, obtaining a voice starting point and a voice ending point. Based on the voice starting point and the voice ending point, the original sound data is segmented, thereby obtaining segmented long Chinese speech to be processed. Alternatively, the server slides the original sound data according to a preset size window, thereby obtaining the segmented long Chinese speech to be processed.

[0079] The segmented long Chinese speech to be processed is subjected to windowing processing, obtaining windowed speech data. The windowed speech data is subjected to fast Fourier transform, obtaining transformed speech data. The transformed speech data is subjected to triangular band-pass filter processing, obtaining filtered speech data. The filtered speech data is subjected to calculation of logarithmic energy of each filter bank output, obtaining speech data to be processed. The speech data to be processed is subjected to discrete cosine transform, obtaining mel-frequency cepstral coefficients. The mel-frequency cepstral coefficients are subjected to exponential normalization, obtaining a spectrogram. The spectrogram can be an exponential normalized speech piece logarithmic spectrum sequence.

[0080] 102. The spectrogram is subjected to character recognition by a pre-trained target deep speech recognition model, obtaining a text sequence to be processed. The pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification. The target deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer.

[0081] The pre-trained target deep speech recognition model is used to predict a character sequence from input audio. The input is an exponential normalized speech piece logarithmic spectrum sequence, and the output is a sequence composed of each Chinese character, such as {I, love, language, sound,}. The entire system of the target deep speech recognition model is similar to a recurrent neural network (RNN), which has one or more convolutional input layers, followed by multiple recurrent (unidirectional or bidirectional) RNN layers, a fully connected layer before the normalization softmax layer, and finally uses a connection time sequence classification (CTC) loss function to train the network end-to-end.

[0082] The target deep speech recognition model in the long Chinese speech recognition method is similar to the existing end-to-end deep speech automatic recognition network Deep Speech2 in the aspect of architecture, but there are also differences. The target deep speech recognition model in the long Chinese speech recognition method differs from the existing end-to-end deep speech automatic recognition network Deep Speech2 in the number of layers and the number of parameters, and is scaled down to match the size of the data. That is, the target deep speech recognition model includes two convolutional layers (the target deep speech recognition model has one less convolutional layer than the existing Deep Speech2), four bidirectional long short-term memory networks (the number of bidirectional long short-term memory networks of the target deep speech recognition model is reduced to four compared with the existing Deep Speech2), one lookahead layer and one fully connected layer.

[0083] The target deep speech recognition model is used to perform audio character recognition on the spectrogram, so as to obtain a to-be-processed text sequence. The to-be-processed text sequence includes each Chinese character or each blank symbol predicted by the RNN of the target deep speech recognition model.

[0084] 103. The language model in the target deep speech recognition model is used to perform probability estimation on the to-be-processed text sequence, so as to obtain score data.

[0085] An additional language model (commonly referred to as a scorer) is used to improve the speech text recognition performance of the target deep speech recognition model, that is, to improve the accuracy of the predicted text. The language model is a scorer used to calculate the possibility of a word or character sequence in the output to guide the decoder to obtain a more likely result. When performing inference, the CTC is combined with the language model trained on a large amount of text, and a preset search function is used to find the maximum result, that is, the score data. The score data can be used to indicate the most likely word or character sequence in the to-be-processed text sequence, or the score data can be used to indicate the maximum possibility of the word or character sequence in the to-be-processed text sequence.

[0086] The language model in the target deep speech recognition model is used to perform probability estimation on the to-be-processed text sequence based on a preset search function, so as to obtain score data. The search function is as follows: Q(y)=log(p RNN (y|x))+αlog(p FC (y))+βwc(y), Q(y) represents the score data, p RNN (y|x) represents the output of the RNN neural network, that is, the to-be-processed text sequence, p FC(y) represents the output result calculated by the fully connected layer, i.e., the initial text vector. wc(y) represents the number of characters in the transcribed text. α is used to adjust the weights of the language model and the connection time classification network. β represents the decoder's preference for a smaller number of long phrases or a larger number of short words. It should be noted that adjusting the size of the β parameter can allow more characters than in a vocabulary system.

[0087] 104. Determine the target recognition text sequence based on the scoring data.

[0088] The server determines the target recognition text sequence based on the scoring data. This means that the target recognition text sequence is obtained by correcting the text sequence to be processed using the most likely word or character sequence in the text sequence to be processed, or by matching the most likely word or character sequence in the text sequence to the corresponding word or character sequence, and then correcting the text sequence to be processed using the matched word or character sequence.

[0089] It should be noted that the Chinese long speech recognition method provided by this invention can be applied to the recognition of Chinese long speech in government systems in the field of smart government.

[0090] In this embodiment of the invention, by combining a target deep speech recognition model and a language model for additional scoring, speech recognition is performed on segmented long Chinese speech, which eliminates noise interference, improves the accuracy of predicted text, and thus improves the accuracy of low-latency Mandarin speech recognition over long time periods. This solution can be applied to the field of smart government, thereby promoting the construction of smart cities.

[0091] Please see Figure 2 Another embodiment of the Chinese long speech recognition method in this invention includes:

[0092] 201. Obtain the segmented long Chinese speech to be processed, calculate the Mel-frequency cepstral coefficients of the segmented long Chinese speech to be processed, and obtain the spectrogram.

[0093] Specifically, the server acquires the original long Chinese speech, performs data preprocessing on the original long Chinese speech to obtain the initial long Chinese speech; performs frame segmentation on the initial long Chinese speech to obtain the segmented initial long Chinese speech; performs signal processing on the segmented initial long Chinese speech to obtain the segmented long Chinese speech to be processed; and calculates the Mel-frequency cepstral coefficients of the segmented long Chinese speech to be processed to obtain the spectrogram.

[0094] The server obtains an original long Chinese speech by receiving a long Chinese speech sent by a preset system or a voice collector, and performs pre-emphasis processing and windowing processing on the original long Chinese speech to realize data preprocessing of the original long Chinese speech, thereby obtaining an initial long Chinese speech. The initial long Chinese speech is frame cut according to a preset size of frame and a preset moving interval size, thereby obtaining a cut initial long Chinese speech. It should be noted that there is 10 ms of overlap between every two frames of the initial long Chinese speech, and therefore, the preset moving interval size in the embodiment of the application is 10 ms, and the preset size of frame is 20 ms, that is, one frame is taken every 20 ms, and then the frame is moved by 10 ms to take another frame.

[0095] The cut initial long Chinese speech is signal processed to obtain a segmented long Chinese speech to be processed. Specifically, the server performs signal processing on the cut initial long Chinese speech to obtain the segmented long Chinese speech to be processed, including: performing denoising on the cut initial long Chinese speech by a preset single-channel audio denoising algorithm to obtain a preprocessed long Chinese speech; and performing high-frequency signal strengthening on the preprocessed long Chinese speech by a preset high-pass filtering algorithm to obtain the segmented long Chinese speech to be processed. As an example but not limitation, the single-channel audio denoising algorithm in the embodiment of the application is IMCRA (Improved Minima Controlled Recursive Averaging), the background noise and other noises in the cut initial long Chinese speech are denoised by the IMCRA algorithm, and the high-frequency signals in the preprocessed long Chinese speech are strengthened by the preset high-pass filtering algorithm, thereby obtaining the segmented long Chinese speech to be processed. The high-frequency resolution of the segmented long Chinese speech to be processed is enhanced, the noise interference in the segmented long Chinese speech to be processed is avoided, and the quality of the segmented long Chinese speech to be processed is ensured.

[0096] The server performs triangular band-pass filter processing on the segmented long Chinese speech to be processed to obtain first data, calculates the logarithmic energy output by each filter group based on the first data to obtain second data, and performs discrete cosine transform on the second data to obtain a mel-frequency cepstrum coefficient, that is, a spectrogram.

[0097] The data preprocessing, frame cutting, signal processing and mel-frequency cepstrum coefficient calculation on the original long Chinese speech avoid the noise interference in the segmented long Chinese speech to be processed, ensure the quality of the segmented long Chinese speech to be processed, and facilitate subsequent text character recognition.

[0098] Specifically, before the server obtains the segmented Chinese long speech to be processed and calculates the mel-frequency cepstral coefficient of the segmented Chinese long speech to be processed to obtain the spectrogram, the method further includes: constructing an initial deep speech recognition model, the initial deep speech recognition model including two convolutional layers, four bidirectional long short-term memory networks, a look-ahead layer, a fully connected layer, a connectionist temporal classification network, and a language model, the two convolutional layers, the four bidirectional long short-term memory networks, the look-ahead layer, and the fully connected layer all adopting batch normalization, and the language model being used for scoring; obtaining a target speech training sample set, the target speech training sample set being a set of preprocessed long-period Mandarin low-delay speeches in various scenarios; and performing end-to-end training on the initial deep speech recognition model through a connectionist temporal classification loss function of the connectionist temporal classification network, to obtain a target deep speech recognition model.

[0099] The initial deep speech recognition model is constructed in advance, and the initial deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, a look-ahead layer, a fully connected layer, a connectionist temporal classification (CTC) network, and a language model, so as to fine-tune a general Deep Speech2 Chinese model. The two convolutional layers and the four bidirectional long short-term memory networks are used to avoid overfitting caused by an excessively deep deep neural network. The two convolutional layers, the four bidirectional long short-term memory networks, the look-ahead layer, and the fully connected layer all adopt batch normalization, so that the model converges faster. Since the pronunciation of a short phrase may need more than one window size (for example, 20 ms), the result character sequence recognized by the neural network is usually repetitive, and therefore, the initial deep speech recognition model is trained using a connectionist temporal classification (CTC) loss.

[0100] The target speech training sample set is a set of preprocessed long-period Mandarin low-delay speeches in various scenarios. Preferably, as an example but not limitation, the long period is 30 hours, so as to obtain a better model result. The initial speech training sample set is obtained, data preprocessing, frame cutting, signal processing (denoising through a preset single-channel audio denoising algorithm and high-frequency signal strengthening through a preset high-pass filtering algorithm), and mel-frequency cepstral coefficient calculation are performed on the initial speech training sample set, to obtain the target speech training sample set. Further refined training is performed on the target speech training sample set, so as to obtain a more robust general model. The initial deep speech recognition model is trained through the target speech training sample set, the initial deep speech recognition model after training is adjusted in weight through a CTC loss function, and all layers of the initial deep speech recognition model are updated through gradient descent, so as to obtain the target deep speech recognition model. Through the above training, the accuracy of the target deep speech recognition model is improved.

[0101] 202、the pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, the target deep speech recognition model comprises two convolutional layers, four bidirectional long short-term memory networks, a look-ahead layer and a fully connected layer, the two convolutional layers, the four bidirectional long short-term memory networks, the look-ahead layer and the fully connected layer all adopt batch normalization; the two convolutional layers in the pre-trained target deep speech recognition model are used for convolution processing of the spectrogram to obtain a filter mapping sequence.

[0102] The two convolutional layers in the pre-trained target deep speech recognition model are two convolutional layers with 32 filters, the kernel size of the first convolutional layer is (41, 11) and the step size is (2, 2), and the kernel size of the second convolutional layer is (21, 11) and the step size is (2, 1). The server performs convolution processing on the spectrogram through the first convolutional layer in the target deep speech recognition model to obtain a primary convolution sequence, and performs convolution processing on the primary convolution sequence through the second convolutional layer to obtain a filter mapping sequence.

[0103] 203、the four bidirectional long short-term memory networks are used for time sequence semantic feature extraction on the filter mapping sequence to obtain a sequence semantic feature.

[0104] The four bidirectional long short-term memory networks are four bidirectional long short-term memory networks (Bi-LSTM) with 768 units, which are a first bidirectional long short-term memory network, a second bidirectional long short-term memory network, a third bidirectional long short-term memory network and a fourth bidirectional long short-term memory network, and the first bidirectional long short-term memory network, the second bidirectional long short-term memory network, the third bidirectional long short-term memory network and the fourth bidirectional long short-term memory network are sequentially connected. The filter mapping sequence is the input of the first bidirectional long short-term memory network. The bidirectional long short-term memory network Bi-LSTM is reduced to four layers to avoid overfitting caused by a too deep deep neural network.

[0105] The server performs time sequence semantic feature extraction on the filter mapping sequence through the first bidirectional long short-term memory network to obtain a first semantic feature, performs time sequence semantic feature extraction on the first semantic feature through the second bidirectional long short-term memory network to obtain a second semantic feature, performs time sequence semantic feature extraction on the second semantic feature through the third bidirectional long short-term memory network to obtain a third semantic feature, and performs time sequence semantic feature extraction on the third semantic feature through the fourth bidirectional long short-term memory network to obtain a sequence semantic feature.

[0106] 204、the look-ahead layer is used for neuron activation learning on the sequence semantic feature to obtain context-based text information.

[0107] In the method, the target deep speech recognition model adds a lookahead convolution for learning a linear combination of weights of activation time steps of each neuron, so as to control the amount of required context. The lookahead convolution is defined by a parameter matrix of a previous layer of neural network. The lookahead convolution is connected before a fully connected layer, so as to make the subsequent calculation more accurate.

[0108] The server learns neuron activation of the sequence semantic feature based on a preset calculation formula through a lookahead convolution, to obtain context-based text information. The preset calculation formula is specifically as follows: r t,i represents activation of a next layer at an activation time step, i.e. the context-based text information, W i，j represents a parameter matrix of a previous layer of neural network, h t+j-1 represents a hidden state of the previous layer of neural network, d represents a number of neurons in the previous layer, which is 768 in the embodiment, t represents a time step of each output, and τ represents an activation time step of each neuron. The context-based text information is a sequence

[0109] 205. The server projects the context-based text information into a vector through a fully connected layer, to obtain an initial text vector.

[0110] The server projects each output in the context-based text information into a vector through a fully connected layer, to map the learned distributed feature representation to a sample label space, so as to obtain an initial text vector.

[0111] 206. The server performs data processing on the initial text vector, to obtain a to-be-processed text sequence.

[0112] Specifically, the pre-trained target deep speech recognition model includes a connection time sequence classification network. The server performs normalization processing on the initial text vector, to obtain a normalized text vector. The server performs operation on the normalized text vector through the connection time sequence classification network, to obtain the to-be-processed text sequence.

[0113] The server normalizes the output result of the full connection layer (i.e., the initial text vector) by using a softmax function to obtain a normalized text vector, and inputs the normalized text vector into a connection time classification network (CTC loss function) to obtain a to-be-processed text sequence by performing an operation on the normalized text vector through the connection time classification network. The operation on the normalized text vector through the connection time classification network can refer to an existing CTC loss function operation, and will not be described here again. The connection time classification CTC loss function folds repeated characters (without a blank index) in a sequence output by a neural model (i.e., an initial deep speech recognition model and a target deep speech recognition model) to obtain all possible alignments of a real text.

[0114] 207. The server estimates a probability of the to-be-processed text sequence by using a language model in the target deep speech recognition model to obtain score data.

[0115] Specifically, the server obtains a number of words in the to-be-processed text sequence, a target weight, and a preference parameter. The target weight is used to indicate an adjustment of weights of the language model and the connection time classification network in the target deep speech recognition model. The preference parameter is used to control a preference of a decoder for a smaller number of long phrases or a larger number of short words. The server performs an operation on the number of words in the to-be-processed text sequence, the target weight, the preference parameter, and the to-be-processed text sequence based on the language model in the target deep speech recognition model to obtain the score data.

[0116] In an actual inference problem, the probability estimation of the language model is taken as a factor. For example, the server performs an operation on the number of words in the to-be-processed text sequence, the target weight, the preference parameter, and the to-be-processed text sequence based on a preset probability estimation formula by using the language model in the target deep speech recognition model to obtain a score factor. The probability estimation formula is specifically as follows: W * W represents the score factor, W represents the to-be-processed text sequence, X represents each word or each character sequence in the to-be-processed text sequence, P represents a probability, a represents the target weight, and β represents the preference parameter. a and β are both determined according to experience by tuning a data set. W W represents the number of words in the to-be-processed text sequence. The score factor can be determined as the score data, or the score data can be determined based on the score factor. The score data can be used to indicate a word or a character sequence with maximum possibility in the to-be-processed text sequence, or the score data can be used to indicate a maximum possibility of a word or a character sequence in the to-be-processed text sequence.

[0117] The probability estimation of the to-be-processed text sequence by using the additional language model improves the accuracy of the predicted text.

[0118] 208、determine the target recognized text sequence based on the score data.

[0119] The server determines the target recognized text sequence based on the score data, that is, the target recognized text sequence is obtained by correcting the to-be-processed text sequence through the maximum likelihood of the word or character sequence in the to-be-processed text sequence, or the target recognized text sequence is obtained by correcting the to-be-processed text sequence through the word or character sequence corresponding to the maximum likelihood matching of the word or character sequence in the to-be-processed text sequence.

[0120] It is worth noting that through experiments, the accuracy of the Mandarin end-to-end automatic speech Chinese recognition system based on the Deep Speech2 architecture applied in the Chinese long speech recognition method in the present application can reach 99%, and the average delay is within 1s. The running time delay and overall performance of this system make it suitable for deployment in business project applications, such as business projects for human-computer interaction.

[0121] It should be noted that the Chinese long speech recognition method provided by the present application can be applied to the recognition of Chinese long speech in the government system in the field of smart government affairs.

[0122] In the embodiment of the present application, by combining the target deep speech recognition model and the language model for additional scoring, the segmented to-be-processed Chinese long speech is subjected to speech recognition, which can eliminate noise interference, improve the accuracy of the predicted text, and further improve the accuracy of long-period Mandarin low-delay speech recognition. The present scheme can be applied in the field of smart government affairs, thereby promoting the construction of smart cities.

[0123] The Chinese long speech recognition method in the embodiment of the present application is described above, and the Chinese long speech recognition device in the embodiment of the present application is described below. Please refer to Figure 3 An embodiment of the Chinese long speech recognition device in the embodiment of the present application includes:

[0124] The computing module 301 is configured to obtain the segmented to-be-processed Chinese long speech, calculate the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech, and obtain a spectrogram.

[0125] The recognition module 302 is configured to perform character recognition on the spectrogram through a pre-trained target deep speech recognition model to obtain a to-be-processed text sequence. The pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification. The target deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer.

[0126] The estimation module 303 is configured to perform probability estimation on the to-be-processed text sequence by using a language model in the target deep speech recognition model, to obtain score data.

[0127] The determination module 304 is configured to determine the target recognition text sequence based on the score data.

[0128] The functions of the modules in the above Chinese long speech recognition device correspond to the steps in the above Chinese long speech recognition method embodiments, and the functions and implementation processes will not be repeated here.

[0129] In the embodiment of the application, by combining the target deep speech recognition model and the language model for additional scoring, the segmented to-be-processed Chinese long speech is subjected to speech recognition, which can eliminate noise interference and improve the accuracy of the predicted text, thereby improving the accuracy of the long-period Mandarin low-delay speech recognition. The scheme can be applied to the field of smart government, thereby promoting the construction of smart cities.

[0130] Referring to Figure 4 Another embodiment of the Chinese long speech recognition device in the embodiment of the application includes:

[0131] The calculation module 301 is configured to obtain the segmented to-be-processed Chinese long speech, calculate the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech, and obtain a spectrogram;

[0132] The recognition module 302 is configured to perform character recognition on the spectrogram by using a pre-trained target deep speech recognition model to obtain a to-be-processed text sequence. The pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification. The target deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer.

[0133] The pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification. The target deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer. The two convolutional layers, the four bidirectional long short-term memory networks, the one look-ahead layer, and the one fully connected layer all use batch normalization. The recognition module 302 specifically includes:

[0134] The convolution processing unit 3021 is configured to perform convolution processing on the spectrogram by using the two convolutional layers in the pre-trained target deep speech recognition model to obtain a filter mapping sequence.

[0135] The extraction unit 3022 is configured to perform time sequence semantic feature extraction on the filter mapping sequence by using the four bidirectional long short-term memory networks to obtain sequence semantic features.

[0136] The learning unit 3023 is configured to learn neuron activation of the sequence semantic features through a front-view layer to obtain context-based text information.

[0137] The mapping unit 3024 is configured to perform vector projection on the context-based text information through a full connection layer to obtain an initial text vector.

[0138] The first data processing unit 3025 is configured to perform data processing on the initial text vector to obtain a to-be-processed text sequence.

[0139] The estimation module 303 is configured to perform probability estimation on the to-be-processed text sequence through a language model in the target deep speech recognition model to obtain score data.

[0140] The determination module 304 is configured to determine a target recognition text sequence based on the score data.

[0141] Optionally, the pre-trained target deep speech recognition model comprises a connection time sequence classification network, and the first data processing unit 3025 can be specifically configured to:

[0142] perform normalization processing on the initial text vector to obtain a normalized text vector;

[0143] perform operation on the normalized text vector through the connection time sequence classification network to obtain the to-be-processed text sequence.

[0144] Optionally, the estimation module 303 can be specifically configured to:

[0145] obtain a number of words in the to-be-processed text sequence, a target weight and a preference parameter, the target weight being used to indicate an adjustment of weights of the language model and the connection time sequence classification network in the target deep speech recognition model, and the preference parameter being used to control a preference of a decoder to a smaller number of long phrases or a larger number of short words;

[0146] perform operation on the to-be-processed text sequence based on the number of words in the to-be-processed text sequence, the target weight, the preference parameter and the to-be-processed text sequence through the language model in the target deep speech recognition model to obtain the score data.

[0147] Optionally, the calculation module 301 comprises:

[0148] The second data processing unit 3011 is configured to obtain an original Chinese long speech, and perform data preprocessing on the original Chinese long speech to obtain an initial Chinese long speech.

[0149] The cutting unit 3012 is configured to perform frame cutting on the initial Chinese long speech to obtain a cut initial Chinese long speech.

[0150] The signal processing unit 3013 is configured to perform signal processing on the segmented initial Chinese long speech to obtain the segmented to-be-processed Chinese long speech.

[0151] The computing unit 3014 is configured to calculate the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech to obtain a spectrogram.

[0152] Optionally, the signal processing unit 3013 can be specifically configured to:

[0153] perform denoising on the segmented initial Chinese long speech by using a preset single-channel audio denoising algorithm to obtain a preprocessed Chinese long speech.

[0154] perform high-frequency signal enhancement on the preprocessed Chinese long speech by using a preset high-pass filtering algorithm to obtain the segmented to-be-processed Chinese long speech.

[0155] Optionally, the Chinese long speech recognition device further comprises:

[0156] The constructing module 305 is configured to construct an initial deep speech recognition model, the initial deep speech recognition model comprising two convolutional layers, four bidirectional long short-term memory networks, a look-ahead layer, a fully connected layer, a connection time sequence classification network and a language model, the two convolutional layers, the four bidirectional long short-term memory networks, the look-ahead layer and the fully connected layer all using batch normalization, and the language model being used for scoring.

[0157] The obtaining module 306 is configured to obtain a target speech training sample set, the target speech training sample set being a preprocessed set of long-time period Mandarin low-delay speech in each scene.

[0158] The training module 307 is configured to perform end-to-end training on the initial deep speech recognition model by using the target speech training sample set and a connection time sequence classification loss function of the connection time sequence classification network to obtain a target deep speech recognition model.

[0159] The functions of the modules and units in the above Chinese long speech recognition device correspond to the steps in the above Chinese long speech recognition method, and the functions and implementation processes will not be repeated here.

[0160] In the embodiment, by combining the target deep speech recognition model and the language model used for additional scoring, the segmented to-be-processed Chinese long speech is subjected to speech recognition, noise interference is eliminated, the accuracy of the predicted text is improved, and the accuracy of the long-time period Mandarin low-delay speech recognition is improved. The scheme can be applied to the field of smart government affairs, thereby promoting the construction of smart cities.

[0161] The above Figure 3 and Figure 4The recognition device for long Chinese speech in the embodiment of the present application is described in detail from the perspective of a modular functional entity. The recognition device for long Chinese speech in the embodiment of the present application is described in detail from the perspective of hardware processing.

[0162] Figure 5 Fig. 1 is a structural schematic diagram of a recognition device for long Chinese speech provided by the embodiment of the present application. The recognition device for long Chinese speech 500 can be quite different due to different configurations or performances, and can include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. The memory 520 and the storage media 530 can be temporary storage or persistent storage. The programs stored in the storage media 530 can include one or more modules (not shown in the figure), each of which can include a series of computer program operations in the recognition device for long Chinese speech 500. Furthermore, the processor 510 can be configured to communicate with the storage media 530 and execute a series of computer program operations in the storage media 530 on the recognition device for long Chinese speech 500.

[0163] The recognition device for long Chinese speech 500 can also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that the recognition device for long Chinese speech 500 can include more or fewer components than those shown in the figure, or some components can be combined, or different components can be arranged. Figure 5 The recognition device for long Chinese speech structure shown in the figure does not constitute a limitation on the recognition device for long Chinese speech, and can include more or fewer components than those shown in the figure, or some components can be combined, or different components can be arranged.

[0164] The present application also provides a recognition device for long Chinese speech, which includes a memory and at least one processor, the memory storing a computer program, and the memory and the at least one processor being interconnected by a circuit; the at least one processor invoking the computer program in the memory to enable the recognition device for long Chinese speech to perform the steps of the recognition method for long Chinese speech. The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium, and the computer readable storage medium stores a computer program, which, when running on a computer, enables the computer to perform the steps of the recognition method for long Chinese speech.

[0165] Further, the computer readable storage medium can mainly include a storage program area and a storage data area, wherein the storage program area can store an operating system, an application program required by at least one function, and the like; and the storage data area can store data created according to the use of the blockchain node, and the like.

[0166] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods, each of which contains the information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, and the like.

[0167] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0168] If the integrated units are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of computer programs for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0169] The above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recognizing a Chinese long speech, characterized in that, The method for recognizing the Chinese long speech comprises the following steps: obtaining the segmented Chinese long speech to be processed, calculating the mel-frequency cepstrum coefficient of the segmented Chinese long speech to be processed, and obtaining a spectrogram; performing character recognition on the spectrogram by using a pre-trained target deep speech recognition model to obtain a text sequence to be processed, wherein the pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, and the target deep speech recognition model comprises two convolution layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer; The language model in the target deep speech recognition model performs probability estimation on the to-be-processed text sequence based on a preset search function to obtain score data; the search function is as follows: , denotes score data, denotes the output of the RNN neural network, that is, the to-be-processed text sequence, denotes the output result calculated through the full connection layer, that is, the initial text vector, denotes the number of characters in the transcribed text, is used to adjust the weights of the language model and the connection time sequence classification network, denotes the preference of the control decoder for a smaller number of long phrases or a larger number of short words. determining a target recognition text sequence based on the score data; performing probability estimation on the text sequence to be processed based on a preset search function by using a language model in the target deep speech recognition model to obtain score data, comprising the following steps: obtaining the number of words in the text sequence to be processed, a target weight, and a preference parameter, wherein the target weight is used to indicate the adjustment of the weight of the language model and the connection time sequence classification network in the target deep speech recognition model, and the preference parameter is used to control the preference of the decoder to a small number of long words or a large number of short words; and performing calculation on the number of words in the text sequence to be processed, the target weight, the preference parameter, and the text sequence to be processed by using the language model in the target deep speech recognition model to obtain score data; obtaining the segmented Chinese long speech to be processed, calculating the mel-frequency cepstrum coefficient of the segmented Chinese long speech to be processed, and obtaining a spectrogram, comprising the following steps: obtaining an original Chinese long speech, performing data preprocessing on the original Chinese long speech to obtain an initial Chinese long speech; performing frame cutting on the initial Chinese long speech to obtain a cut initial Chinese long speech; performing signal processing on the cut initial Chinese long speech to obtain a segmented Chinese long speech to be processed; and calculating the mel-frequency cepstrum coefficient of the segmented Chinese long speech to be processed to obtain a spectrogram.

2. The method for recognizing a Chinese long speech according to claim 1, wherein, The pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, and the target deep speech recognition model comprises two convolution layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer, wherein the two convolution layers, the four bidirectional long short-term memory networks, the one look-ahead layer, and the one fully connected layer all adopt batch normalization; performing character recognition on the spectrogram by using a pre-trained target deep speech recognition model to obtain a text sequence to be processed, comprising the following steps: performing convolution processing on the spectrogram by using the two convolution layers in the pre-trained target deep speech recognition model to obtain a filter mapping sequence; performing time sequence semantic feature extraction on the filter mapping sequence by using the four bidirectional long short-term memory networks to obtain sequence semantic features; performing neuron activation learning on the sequence semantic features by using the one look-ahead layer to obtain context-based text information; performing vector projection on the context-based text information by using the one fully connected layer to obtain an initial text vector; Data processing is performed on the initial text vector to obtain a to-be-processed text sequence.

3. The method for recognizing a long Chinese speech according to claim 2, characterized in that, The pre-trained target deep speech recognition model includes a connection time sequence classification network, and the data processing performed on the initial text vector to obtain a to-be-processed text sequence includes: The initial text vector is normalized to obtain a normalized text vector. The normalized text vector is operated through the connection time sequence classification network to obtain a to-be-processed text sequence.

4. The method for recognizing a long Chinese speech according to claim 1, characterized in that, The signal processing performed on the cut initial Chinese long speech to obtain a segmented to-be-processed Chinese long speech includes: The cut initial Chinese long speech is denoised through a preset single-channel audio noise reduction algorithm to obtain a preprocessed Chinese long speech. The preprocessed Chinese long speech is subjected to high-frequency signal enhancement through a preset high-pass filtering algorithm to obtain a segmented to-be-processed Chinese long speech.

5. The method for recognizing a Chinese long speech according to any one of claims 1-4, characterized in that, Before the segmented to-be-processed Chinese long speech is obtained and the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech is calculated to obtain a spectrogram, the method further includes: An initial deep speech recognition model is constructed, the initial deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, one fully connected layer, a connection time sequence classification network, and a speech model, the two convolutional layers, the four bidirectional long short-term memory networks, the one look-ahead layer, and the one fully connected layer all adopt batch normalization, and the language model is used for scoring; A target speech training sample set is obtained, the target speech training sample set is a preprocessed set of Mandarin low-delay speech in various scenes and long time periods; The initial deep speech recognition model is trained end to end through the target speech training sample set and a connection time sequence classification loss function of the connection time sequence classification network to obtain a target deep speech recognition model.

6. A Chinese long speech recognition apparatus, characterized by comprising: The Chinese long speech recognition device includes: A calculation module is configured to obtain a segmented to-be-processed Chinese long speech, calculate a mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech, and obtain a spectrogram. An identification module is configured to perform character recognition on the spectrogram through a pre-trained target deep speech recognition model to obtain a to-be-processed text sequence, the pre-trained target deep speech recognition model is an end-to-end model based on connection time sequence classification, and the target deep speech recognition model includes two convolutional layers, four bidirectional long short-term memory networks, one look-ahead layer, and one fully connected layer. An estimation module is configured to perform probability estimation on the text sequence to be processed based on a preset search function through a language model in the target deep speech recognition model, to obtain score data; the search function is as follows: , represents score data, represents the output of the RNN neural network, i.e., the text sequence to be processed, represents the output result calculated through the full connection layer, i.e., the initial text vector, represents the number of characters in the transcribed text, is used to adjust the weights of the language model and the connection time sequence classification network, represents the preference of the decoder for a smaller number of long phrases or a larger number of short words. A determination module is configured to determine a target recognition text sequence based on the scoring data. The language model in the target deep speech recognition model is used to perform probability estimation on the to-be-processed text sequence based on a preset search function, to obtain score data, including: obtaining a number of words in the to-be-processed text sequence, a target weight, and a preference parameter, the target weight being used to indicate an adjustment of a weight of the language model and a connection time sequence classification network in the target deep speech recognition model, and the preference parameter being used to control a preference of a decoder to a smaller number of long phrases or a larger number of short words; and the language model in the target deep speech recognition model is used to perform calculation on the number of words in the to-be-processed text sequence, the target weight, the preference parameter, and the to-be-processed text sequence, to obtain the score data. The obtaining of the segmented to-be-processed Chinese long speech and the calculation of the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech to obtain a spectrogram include: obtaining an original Chinese long speech, performing data preprocessing on the original Chinese long speech to obtain an initial Chinese long speech; performing frame cutting on the initial Chinese long speech to obtain a cut initial Chinese long speech; performing signal processing on the cut initial Chinese long speech to obtain a segmented to-be-processed Chinese long speech; and calculating the mel-frequency cepstrum coefficient of the segmented to-be-processed Chinese long speech to obtain a spectrogram.

7. A Chinese long speech recognition device, characterized by, The Chinese long speech recognition device includes a memory and at least one processor, and the memory stores a computer program. The at least one processor invokes the computer program in the memory, so that the Chinese long speech recognition device performs the Chinese long speech recognition method in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the Chinese long speech recognition method in any one of claims 1-5.

Images