Speech recognition method, device, computer equipment and storage medium

Abstract

The present application relates to a speech recognition method, apparatus, computer equipment and storage medium. The method comprises: obtaining a current frame of a speech to be recognized and a historical frame before the current frame, and taking the current frame and the historical frame before the current frame as a first block, and then simulating a future frame corresponding to the first block according to the first block and a target synthesis predictor, thereby determining a speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model. The use of this method can reduce the delay time of recognition.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a speech recognition method, apparatus, computer equipment and storage medium. Background Art

[0002] Automatic speech recognition (ASR) refers to the technology of transcribing speech into text. In actual production and living environments, a well-functioning speech recognition system needs not only a high recognition accuracy, but also a low recognition delay, that is, the speech recognition system needs to recognize the user's speech synchronously, rather than waiting until the user finishes speaking before starting recognition.

[0003] In order to achieve synchronous recognition, current speech recognition systems all use a block model, that is, a sentence is divided into multiple blocks for recognition. In order to improve the accuracy of block recognition, current speech recognition methods will add certain historical frames and future frames to each block to provide contextual information for the block.

[0004] However, in the existing speech recognition method based on the block model, it is necessary to wait until a certain number of future frames arrive before starting recognition, thereby increasing the delay time of recognition. Summary of the invention

[0005] Based on this, it is necessary to provide a speech recognition method, apparatus, computer device and storage medium that can reduce the recognition delay time in order to solve the above technical problems.

[0006] In a first aspect, the present application provides a speech recognition method. The method comprises:

[0007] Acquire a current frame of the speech to be recognized and a history frame before the current frame, and use the current frame and the history frame before the current frame as a first block;

[0008] Simulating a future frame corresponding to the first block according to the first block and the target synthesis predictor;

[0009] A speech recognition result of the speech to be recognized is determined according to the first block, the future frame and the target acoustic model.

[0010] In one embodiment, simulating a future frame corresponding to the first block according to the first block and the target synthesis predictor includes:

[0011] Inputting the first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block;

[0012] The second block is input into the target synthesis predictor to simulate a future frame corresponding to the first block.

[0013] In one embodiment, determining the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model includes:

[0014] Performing splicing processing on the first block and the future frame to obtain a splicing result;

[0015] The speech recognition result is determined according to the concatenation result and the target acoustic model.

[0016] In one of the embodiments, the first block is at least one block obtained after performing block processing on the speech to be recognized, and the length of each first block is determined according to a preset uniform distribution interval.

[0017] In one embodiment, the method further comprises:

[0018] Obtain a training sample set;

[0019] According to the training sample set, the initial speech recognition model is trained to obtain a target speech recognition model, and the target speech recognition model includes the target synthesis predictor, the target acoustic model and the synthesis encoder.

[0020] In one embodiment, the initial speech recognition model includes an initial synthesis predictor, an initial acoustic model and an initial synthesis encoder, and the initial speech recognition model is trained according to the training sample set to obtain a target speech recognition model, including:

[0021] Determine non-streaming loss and streaming loss according to the training sample set and the initial acoustic model;

[0022] Determining a simulation loss based on the training sample set, the initial synthesis encoder, and the initial synthesis predictor;

[0023] The initial speech recognition model is trained according to the non-streaming loss, the streaming loss and the simulated loss to obtain the target speech recognition model.

[0024] In one embodiment, obtaining a training sample set includes:

[0025] determining a feature vector for each audio signal sample;

[0026] Performing mean processing and variance normalization processing on the feature vectors of each audio signal sample to obtain a target feature vector;

[0027] The target feature vector of each audio signal sample is processed at a double speed to obtain the training sample set.

[0028] In a second aspect, the present application also provides a speech recognition device. The device comprises:

[0029] A first acquisition module, used for acquiring a current frame of the speech to be recognized and a history frame before the current frame, and taking the current frame and the history frame before the current frame as a first block;

[0030] A simulation module, configured to simulate a future frame corresponding to the first block according to the first block and a target synthesis predictor;

[0031] A determination module is used to determine a speech recognition result of the speech to be recognized based on the first block, the future frame and the target acoustic model.

[0032] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0034] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0035] The above-mentioned speech recognition method, apparatus, computer equipment and storage medium obtain the current frame of the speech to be recognized and the historical frames before the current frame, and use the current frame and the historical frames before the current frame as the first block, and then simulate the future frame corresponding to the first block according to the first block and the target synthesis predictor, so as to determine the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model. Since the speech recognition result of the speech to be recognized is determined according to the historical frame, the current frame and the simulated future frame of the speech to be recognized in the present application, that is, the speech recognition method in the present application does not need to wait for a certain real future frame but directly uses the simulated future frame, thus avoiding the situation in the traditional technology that must wait until a certain number of future frames arrive before starting recognition, solving the problem of waiting for real future frames in the traditional technology leading to increased recognition delay time, and reducing the recognition delay time. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is an application environment diagram of the speech recognition method in the embodiment of the present application;

[0037] Figure 2A flow chart of a speech recognition method in an embodiment of the present application;

[0038] Figure 3 This is a schematic diagram of a process of simulating a future frame in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of a process for determining a speech recognition result in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the principle of the speech recognition method in the embodiment of the present application;

[0041] Figure 6 A schematic diagram of a process for obtaining a target speech recognition model in an embodiment of the present application;

[0042] Figure 7 This is another flow chart of obtaining a target speech recognition model in an embodiment of the present application;

[0043] Figure 8 This is a comparison diagram of the principles of CTC and CTC-CTR;

[0044] Fig. 9 A schematic diagram of a process for obtaining a training sample set in an embodiment of the present application;

[0045] Fig.10 This is a structural block diagram of a speech recognition device in an embodiment of the present application;

[0046] Fig.11 Schematic diagram of the internal structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. The serial numbers in the present application, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0048] Figure 1 FIG. 1 is an application environment diagram of the speech recognition method in the embodiment of the present application, such as Figure 1 As shown, the speech recognition method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0049] Figure 2 The flowchart of the speech recognition method in the embodiment of the present application is shown in FIG. Figure 1 In the server shown, in one embodiment, as Figure 2 As shown, the following steps are included:

[0050] S201, obtaining a current frame of the speech to be recognized and historical frames before the current frame, and taking the current frame and the historical frames before the current frame as a first block.

[0051] In this embodiment, the server obtains the current frame of the speech to be recognized and the historical frames before the current frame. The server may obtain the speech to be recognized sent by the terminal, or may obtain the speech to be recognized from a data storage system.

[0052] In a streaming scenario, that is, in an application scenario where speech recognition needs to be performed synchronously during the process of acquiring the speech to be recognized, the server acquires each speech frame in the speech to be recognized in chronological order, for example, the server acquires the speech frame of the speech to be recognized at the first moment, the speech frame of the speech to be recognized at the second moment, ... until the server acquires the speech frame of the speech to be recognized at the last moment. Among them, the speech frame can be a frame corresponding to the processed speech to be recognized obtained after the server performs preset processing on the speech to be recognized, for example, the server acquires the feature vector of the speech to be recognized, and performs mean processing and variance normalization processing on the feature vector of the speech to be recognized to obtain the processed speech to be recognized, and then acquires each speech frame of the processed speech to be recognized.

[0053] The server includes a data buffer. Specifically, the data buffer in the server receives speech frames in the speech to be recognized at each moment. When the data buffer receives a certain number of speech frames, the certain number of speech frames are used as the first block of the speech to be recognized, that is, the current frame of the speech to be recognized and the historical frames before the current frame are obtained, and the current frame and the historical frames before the current frame are used as the first block, thereby completing the block processing of the speech to be recognized. Of course, the server can also obtain multiple first blocks of the speech to be recognized. For example, the speech to be recognized has a total of 15 moments of speech frames 1 to 15, then the server determines the first block 1, the first block 2 and the first block 3, wherein the first block 1 includes speech frames 1 to 5, the first block 2 includes speech frames 6 to 10, and the first block 3 includes speech frames 11 to 15.

[0054] It can be understood that historical frames and current frames are relative concepts. The speech frame corresponding to the largest moment in the first block is the current frame, and the rest are historical frames before the current frame. For example, the speech to be recognized has five moments of speech frame 1, speech frame 2, speech frame 3, ..., speech frame 5. The server data buffer receives the speech frames of the five moments in chronological order. When receiving speech frame 5 at the fifth moment, the speech frames at the moments of speech frame 1 to speech frame 5 are taken as the first block 1. For the first block 1, speech frame 5 is the current frame, and speech frames 1 to speech frames 4 are the historical frames before the current frame.

[0055] S202: simulate a future frame corresponding to the first block according to the first block and the target synthesis predictor.

[0056] In this embodiment, the server simulates a future frame corresponding to the first block according to the first block and the target synthesis predictor. The target synthesis predictor is a trained one-layer feedforward neural network, such as a fully connected neural network (Multi-Layer Perception, MLP). It is understandable that the server simulates at least one future frame, and generally simulates a certain number of future frames to ensure the subsequent recognition effect.

[0057] Continuing with the example of S201 above, the first block 1 includes speech frames 1 to 5. The server simulates the future frames corresponding to the first block 1 according to the first block 1 and the target synthesis predictor, for example, speech frame 6 at the 6th moment and speech frame 7 at the 7th moment. One possible implementation method is that the server directly uses the first block 1 as the input of the target synthesis predictor to obtain the future frames output by the target synthesis predictor, namely, speech frame 6 and speech frame 7.

[0058] S203: Determine a speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model.

[0059] In this embodiment, the server determines the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model.

[0060] Among them, the trained conformer network is used as the target acoustic model in this embodiment. Of course, the target acoustic model can also be other neural networks, such as a bidirectional long short-term memory network (Long Short-Term Memory, LSTM), a transformer network, a recurrent neural network transformer (Recurrent Neural Network Transducer, RNN-T), and an attention model based on encoder and decoder Attention based encoder decoder, which is not limited in this embodiment.

[0061] The Conformer network is a time series model that combines the Transformer network and the Convolutional Neural Network (CNN). Transformer is a time series model based on the self-attention mechanism, which is good at capturing content-based global interactions, while CNN can effectively utilize local features, so the Conformer network can effectively combine long-term global interaction information and local features. Therefore, the Conformer network uses a convolution-enhanced multi-head self-attention structure, which includes a convolutional layer and a multi-head self-attention layer. Among them, the convolutional layer can better extract local information in the audio signal, while the multi-head self-attention layer can better focus on the global information of the sentence. Specifically, a 12-layer Conformer network is used in this embodiment, and the number of attention heads, attention dimension, and feedforward layer dimension of each layer are 4, 256, and 1024, respectively.

[0062] Furthermore, since the current frame and the historical frames before the current frame are used as the first block, the server can determine the speech recognition result of the speech to be recognized based on the historical frames, current frames, future frames and target acoustic model of the speech to be recognized. Specifically, the server uses the first block and future frames of the speech to be recognized as the input of the target acoustic model, obtains the text information corresponding to the speech to be recognized output by the target acoustic model, and the server can directly use the text information as the speech recognition result of the speech to be recognized, and also starts to further process the text information to determine the speech recognition result of the speech to be recognized.

[0063] The speech recognition method provided in this embodiment obtains the current frame of the speech to be recognized and the historical frames before the current frame, and uses the current frame and the historical frames before the current frame as the first block, and then simulates the future frame corresponding to the first block according to the first block and the target synthesis predictor, so as to determine the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model. Since the speech recognition result of the speech to be recognized is determined according to the historical frame, the current frame and the simulated future frame of the speech to be recognized in the present application, that is, the speech recognition method in the present application does not need to wait for a certain real future frame but directly uses the simulated future frame, thus avoiding the situation in the traditional technology that must wait until a certain number of future frames arrive before starting recognition, solving the problem of waiting for real future frames in the traditional technology leading to increased recognition delay time, and reducing the recognition delay time.

[0064] Figure 3 This is a schematic diagram of a process of simulating future frames in an embodiment of the present application, referring to Figure 3 This embodiment relates to an optional implementation method of how to simulate the future frame corresponding to the first block. Based on the above embodiment, the above S202, based on the first block and the target synthesis predictor, simulates the future frame corresponding to the first block, including the following steps:

[0065] S301, inputting a first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block.

[0066] In this embodiment, the server inputs the first block into a target synthesis encoder to obtain a second block. The target synthesis encoder is a trained gated recurrent unit (GRU), for example, a 3-layer unidirectional GRU is used in this application, and the hidden layer dimension is 256.

[0067] Continuing with the example of S202 above, the server inputs the first block 1 to the target synthesis encoder to obtain the second block 1 output by the target synthesis encoder for encoding the first block 1. Specifically, the hidden state of the target synthesis encoder maps the feature sequence corresponding to the first block 1 to a hidden state sequence to determine the second block 1.

[0068] S302: Input the second block into a target synthesis predictor to simulate a future frame corresponding to the first block.

[0069] In this embodiment, the server inputs the second block into the target synthesis predictor to simulate the future frame corresponding to the first block. For example, the last hidden state of the target synthesis encoder outputs the second block 1, and the server inputs the second block 1 into the target synthesis predictor to obtain the future frame output by the target synthesis predictor.

[0070] Among them, the target synthesis encoder and the target synthesis predictor introduce very few additional parameters and are a lightweight network. In addition, the target synthesis encoder and the target synthesis predictor can be trained in a self-supervised manner without the need for additional annotations, thus not increasing the computational and memory overhead of speech recognition.

[0071] In this embodiment, the first block is input into the target synthesis encoder to obtain the second block, wherein the second block is the block obtained after encoding the first block, and then the second block is input into the target synthesis predictor to simulate the future frame corresponding to the first block. Therefore, the server does not need to wait for the real future frame, but can directly determine the speech recognition result of the speech to be recognized based on the first block, the simulated future frame and the target acoustic model, thereby reducing the recognition delay time.

[0072] Figure 4 This is a flow chart of determining a speech recognition result in an embodiment of the present application, referring to Figure 4 This embodiment relates to an optional implementation method of how to determine the speech recognition result of the speech to be recognized. Based on the above embodiment, the above S203, according to the first block, the future frame and the target acoustic model, determines the speech recognition result of the speech to be recognized, including the following steps:

[0073] S401, performing splicing processing on the first block and the future frame to obtain a splicing result.

[0074] In this embodiment, the server uses the target acoustic model to perform splicing processing on the first block and the future frame to obtain a splicing result. For example, the server uses the cat operation provided by pytorch to perform splicing processing on the first block and the future frame to obtain a splicing result, where PyTorch is an open source Python machine learning library.

[0075] Specifically, the server performs splicing processing in a time sequence, that is, a splicing sequence of "historical frame + current frame + future frame" to obtain a splicing result. Continuing with the example of S202 above, the first block 1 includes speech frames 1 to 5, and the future frames corresponding to the first block 1 include speech frames 6 to 7. The server obtains a splicing result 1 in the order of speech frames 1 to 7.

[0076] S402, determining a speech recognition result according to the concatenation result and the target acoustic model.

[0077] In this embodiment, the server determines the speech recognition result according to the splicing result and the target acoustic model. It should be noted that the server may splice the first block and the future frame before or after the first block and the future frame are input to the target acoustic model.

[0078] Specifically, in this embodiment, the server inputs the splicing result to the target acoustic model, the multi-head self-attention layer in the target acoustic model encodes the splicing result, and the convolution layer in the target acoustic model decodes the encoded splicing result to determine the speech recognition result.

[0079] In this embodiment, the target acoustic model is used to splice the first block and the future frame to obtain a splicing result, and the speech recognition result is determined based on the splicing result. Therefore, the server does not need to wait for the real future frame, but can directly determine the speech recognition result of the speech to be recognized based on the splicing result of the first block, the simulated future frame and the target acoustic model, thereby reducing the recognition delay time.

[0080] Optionally, the first block is at least one block obtained after performing block processing on the speech to be recognized, and the length of each first block is determined according to a preset uniform distribution interval.

[0081] In this embodiment, the first block is at least one block obtained after the speech to be recognized is processed by block processing. It can be understood that in a streaming scenario, in order to achieve synchronous speech recognition, the speech to be recognized is generally divided into multiple blocks.

[0082] Specifically, in combination with S201, the data buffer in the server receives voice frames at various times. When the data buffer receives a certain number of voice frames, the certain number of voice frames are used as the first block of the voice to be recognized. The number of voice frames contained in the first block is the length of the first block. The length of each first block is obtained by random sampling by the server from a preset uniform distribution interval U(CA, C+A). Among them, U and A can be set according to the actual situation, for example, C=5, A=2, then the length of each first block is between 3 frames and 7 frames.

[0083] Furthermore, in order to improve the accuracy of speech recognition, when the server processes the speech to be recognized in blocks, there will be some overlapping frames between each first block, and the number of overlapping frames can be a specified value or a random value within a preset number range. For example, the speech to be recognized includes speech frame 1, speech frame 2, speech frame 3, ..., speech frame 10 at 10 moments, then the first block 1 includes speech frames 1 to 5, speech frame 5 is the current frame corresponding to the first block 1, and speech frames 1 to 4 are the historical frames corresponding to the first block 1; the first block 2 includes speech frames 5 to 8, speech frame 8 is the current frame corresponding to the first block 2, and speech frames 5 to 7 are the historical frames corresponding to the first block 2; the first block 3 includes speech frames 7 to 10, speech frame 10 is the current frame corresponding to the first block 3, and speech frames 7 to 9 are the historical frames corresponding to the first block 3. It can be understood that there are overlapping frames between the first block 1 and the first block 2: speech frame 5, and there are overlapping frames between the first block 2 and the first block 3: speech frame 7 and speech frame 8.

[0084] Therefore, the server obtains three first blocks of the speech to be recognized, namely, the first block 1, the first block 2, and the second block 3, and simulates the future frames corresponding to each first block based on each first block and the target synthesis predictor, and then determines the speech recognition result of the speech to be recognized according to each first block, the future frames corresponding to each first block, and the target acoustic model. One way to achieve this is that the server determines the text information 1 corresponding to the first block 1, determines the text information 2 of the first block 2, and determines the text information 3 of the first block 3, and reorganizes the text information 1, text information 2, and text information 3 into a sentence sequence, thereby determining the speech recognition result of the speech to be recognized.

[0085] In this embodiment, the first block is at least one block obtained after the speech to be recognized is processed by block processing, and the length of each first block is determined according to a preset uniform distribution interval. Since the speech to be recognized is divided into multiple blocks, the server determines the speech recognition result faster than directly performing speech recognition based on the entire speech to be recognized. In addition, the length of each first block is determined according to a preset uniform distribution interval, so the accuracy of the speech recognition result can also be improved.

[0086] In order to more clearly explain the speech recognition method in this application, Figure 5 To explain, Figure 5 The following is a schematic diagram of the principle of the speech recognition method in the embodiment of the present application. In the streaming scenario, each speech frame in the speech to be recognized arrives at the data buffer in the server frame by frame in time sequence. Figure 5, the data buffer of the server obtains each speech frame in the speech to be recognized, and each speech frame enters the data buffer in the form of a queue in chronological order, that is, speech frame 1 corresponding to the first moment, speech frame 2 corresponding to the second moment, speech frame 3 corresponding to the third moment, and speech frame 4 corresponding to the fourth moment. At the fifth moment, the data buffer of the server obtains speech frame 5 corresponding to the fifth moment, and uses the speech frames of these five moments as the first block 1. It can be understood that the first block 1 is generated based on the fifth moment, so the speech frame 5 corresponding to the fifth moment is the current frame of the first block 1, and the speech frames 1 to 4 corresponding to the first four moments are the historical frames of the first block 1. It can also be understood that speech frame 5 is the current frame of the speech to be recognized, and speech frames 1 to 4 are the historical frames before the current frame of the speech to be recognized. Similarly, the data buffer of the server can continue to determine the first block 2, the first block 3, etc., and this process will not be repeated.

[0087] Further, taking the first block 1 as an example, the server inputs the first block 1 into the target synthesis encoder to obtain the second block 1 output by the target synthesis encoder after encoding the first block 1. Then the server inputs the second block 1 into the target synthesis predictor to obtain the simulated future frame output by the target synthesis predictor, thereby simulating the future frame corresponding to the first block 1.

[0088] Furthermore, the server determines the speech recognition result of the speech to be recognized based on the first block 1 and the simulated future frame. Specifically, the server performs splicing processing on the first block 1 and the simulated future frame in the order of "historical frame + current frame + future frame" to obtain a splicing result, and then the server inputs the splicing result into the target acoustic model, and determines the speech recognition result of the speech to be recognized based on the output of the target acoustic model.

[0089] It is understandable that the server can determine to obtain at least one first block of the speech to be recognized. If the speech to be recognized has only one first block, the server can directly use the text information corresponding to the first block output by the target acoustic model as the speech recognition result of the speech to be recognized; if the speech to be recognized has multiple first blocks, the server can combine the text information corresponding to each first block output by the target acoustic model into a sentence sequence to obtain the speech recognition result.

[0090] Figure 6 This is a flow chart of obtaining a target speech recognition model in an embodiment of the present application, referring to Figure 6 This embodiment relates to an optional implementation method of how to train the initial speech recognition model to obtain the target speech recognition model. Based on the above embodiment, the above speech recognition method includes the following steps:

[0091] S601, obtaining a training sample set.

[0092] In this embodiment, the server needs to train the initial speech recognition model to obtain the target speech recognition model to obtain the target synthesis predictor, target acoustic model and synthesis encoder used in the end. Therefore, the server first needs to obtain a training sample set, which can be derived from audio signals in various scenarios, and the training sample set can include the actual speech results of each audio signal. The specific content and specific acquisition method of the training sample set are not limited in the embodiments of the present application.

[0093] S602, training the initial speech recognition model according to the training sample set to obtain a target speech recognition model, wherein the target speech recognition model includes a target synthesis predictor, a target acoustic model and a synthesis encoder.

[0094] In this embodiment, the server trains the initial speech recognition model according to the training sample set to obtain the target speech recognition model. Specifically, the server obtains each first block of the training sample set, and then determines the speech recognition result corresponding to the training sample set according to the first block and the initial speech recognition model. Then, the server updates the parameters of the initial speech recognition model according to the difference between the speech recognition result corresponding to the training sample set and the actual speech result of the training sample set, until the difference between the speech recognition result corresponding to the training sample set and the actual speech result of the training sample set is less than a preset difference threshold, the server can stop training, thereby determining the target speech recognition model, which includes a target synthesis predictor, a target acoustic model and a synthesis encoder.

[0095] In this embodiment, a target speech recognition model is obtained by obtaining a training sample set and training an initial speech recognition model according to the training sample set. The target speech recognition model includes a target synthesis predictor, a target acoustic model and a synthesis encoder. Since the target synthesis predictor, the target acoustic model and the synthesis encoder are obtained by training the initial speech recognition model by obtaining a training sample set in this application, the server can determine the speech recognition result of the speech to be recognized and realize speech synchronous recognition.

[0096] Figure 7 This is another flow chart of obtaining a target speech recognition model in an embodiment of the present application, referring to Figure 7 This embodiment relates to an optional implementation method of how to train the initial speech recognition model to obtain the target speech recognition model. Based on the above embodiment, the above S602, according to the training sample set, trains the initial speech recognition model to obtain the target speech recognition model, including the following steps:

[0097] S701, determining a non-streaming loss and a streaming loss according to a training sample set and an initial acoustic model.

[0098] In this embodiment, the initial speech recognition model includes an initial synthesis predictor, an initial acoustic model and an initial synthesis encoder. The process of training the initial speech recognition model to obtain the target speech recognition model is the process of training the initial synthesis predictor, the initial acoustic model and the initial synthesis encoder to obtain the target synthesis predictor, the target acoustic model and the synthesis encoder.

[0099] Among them, in this embodiment, the initial speech recognition model is trained by a training method in which the streaming model and the non-streaming model share parameters. The streaming model and the non-streaming model share parameters, which means that the same speech recognition model can be used in both streaming scenarios and non-streaming scenarios. In other words, speech recognition can be performed synchronously during the process of acquiring the speech to be recognized, or speech recognition can be performed after acquiring the speech to be recognized. The streaming model and the non-streaming model share parameters, which can not only reduce the memory and parameter overhead of the target speech recognition model, but also help improve the recognition accuracy of the target speech recognition model.

[0100] Specifically, the server first determines the non-streaming loss L according to the training sample set and the initial acoustic model. non-streaming and flow loss L streaming That is to say, during the training process, the server obtains the speech recognition results corresponding to the training sample set in two parallel ways, thereby determining the non-streaming loss L non-streaming and flow loss L streaming .

[0101] Method 1: The server does not perform block processing and future frame simulation on the training sample set. The server directly inputs the training sample set into the initial acoustic model to obtain the speech recognition result corresponding to the training sample set output by the initial acoustic model, thereby determining the non-streaming loss L between the speech recognition result corresponding to the training sample set and the actual speech result corresponding to the training sample set. non-streaming .

[0102] Method 2: The server processes the training sample set into blocks to obtain blocks of the training sample set, and adds certain real historical frames and real future frames to each block and inputs them into the initial acoustic model to obtain the speech recognition result corresponding to the training sample set output by the initial acoustic model, thereby determining the streaming loss L between the speech recognition result corresponding to the training sample set and the actual speech result corresponding to the training sample set. streaming .

[0103] Optionally, when training in mode 2, the block size is jittered when the training sample set is divided into blocks. Specifically, the length of each block is determined according to a preset uniform distribution interval, and there will be some overlapping frames between each block to improve the generalization performance of the initial acoustic model. The principle that the length of each block is determined according to a preset uniform distribution interval and there will be some overlapping frames between each block is the same as the content recorded in the introduction of the first block, and will not be repeated here.

[0104] Furthermore, in the embodiment of the present application, the non-streaming loss L is determined by using the connectionist temporal classification (CTC)-conditional random field (CRF) method. non-streaming and flow loss L streaming Among them, CTC-CRF uses the same topology as CTC, please refer to Figure 8 , Figure 8 This is a comparison diagram of the principles of CTC and CTC-CTR. Figure 8 As shown in Figure 1, in CTC, CTC will output multiple states for the input X, and the states are conditionally independent, and the dependency between the states cannot be determined. CTC-CRF solves the unreasonable state independence dependency in CTC by introducing edge potential energy into the random field potential function based on CTC. Therefore, CTC-CRF is used to determine the non-streaming loss L non-streaming and flow loss L streaming , which can further improve the performance of the target speech recognition model.

[0105] S702, determining a simulation loss according to the training sample set, the initial synthesis encoder and the initial synthesis predictor.

[0106] In this embodiment, the server determines the simulation loss L according to the training sample set, the initial synthesis encoder and the initial synthesis predictor. simu Specifically, the server processes the training sample set into blocks to obtain blocks of the training sample set, inputs each block of the training sample set into the initial synthesis encoder, and further inputs the output of the initial synthesis encoder into the initial synthesis predictor to simulate the future frames corresponding to each block of the training sample set. Then, the server determines the 1-norm distance (L1 loss) between the simulated future frame and the real future frame of each block of the training sample set as the simulation loss L simu .

[0107] S703, training the initial speech recognition model according to the non-streaming loss, the streaming loss and the simulation loss to obtain the target speech recognition model.

[0108] In this embodiment, the server calculates the non-streaming loss L non-streaming and flow loss L streaming and the simulated loss L simu , train the initial speech recognition model to obtain the target speech recognition model. Specifically, the server determines the total loss function loss = L non-streaming +L streaming +αL simu , where α is the simulation loss L simu The weighting coefficient of , α is an integer greater than 0, for example, α is set to 100.

[0109] Furthermore, the server trains the initial speech recognition model according to the total loss function back propagation to obtain the target speech recognition model. When the total loss function is less than a preset stop threshold, the training is stopped and the target speech recognition model is obtained.

[0110] Optionally, the server can also use the Adam optimizer and train the initial speech recognition model according to the transformer learning rate adjustment strategy to obtain the target speech recognition model to update the model parameters of the initial speech recognition model. Furthermore, the parameters of the five models with the best performance on the training sample set will be averaged to obtain the final target speech recognition model.

[0111] This embodiment determines the non-streaming loss and the streaming loss based on the training sample set and the initial acoustic model, and determines the simulation loss based on the training sample set, the initial synthesis encoder and the initial synthesis predictor, so as to train the initial speech recognition model according to the non-streaming loss, the streaming loss and the simulation loss to obtain the target speech recognition model. Therefore, the target speech recognition model finally trained can determine the speech recognition result of the speech to be recognized, and since the simulated future frames are used in the process of obtaining the speech recognition results instead of waiting for the real future frames, the recognition delay time can be reduced.

[0112] Fig. 9 This is a flow chart of obtaining a training sample set in an embodiment of the present application, referring to Fig. 9 This embodiment relates to an optional implementation method of how to obtain a training sample set. Based on the above embodiment, the above S601 includes the following steps:

[0113] S901, determining a feature vector of each audio signal sample.

[0114] In this embodiment, the server uses the commonly used Chinese speech recognition data set aishell-1 to determine each audio signal sample. Further, for each audio signal sample, the server extracts 80-dimensional log filter bank features and 3-dimensional pitch features as feature vectors of each audio signal sample with a window length of 25 ms and a frame rate of 10 ms per frame.

[0115] S902 , performing mean processing and variance normalization processing on the feature vector of each audio signal sample to obtain a target feature vector.

[0116] In this embodiment, the server also performs averaging and variance normalization on the feature vectors of each audio signal sample to obtain a target feature vector after averaging and variance normalization. The process of averaging and variance normalization will not be described in detail here.

[0117] S903, performing double-speed processing on the target feature vector of each audio signal sample to obtain a training sample set.

[0118] In this embodiment, the server processes the target feature vector of each audio signal sample at a multiple speed to obtain a training sample set. Specifically, the server processes the target feature vector of each audio signal sample at a speed of 0.9 and 1.1. Of course, the server may also select other multiple speeds for multiple speed processing, which is not limited in this embodiment. The target feature vector processed at multiple speeds and the target feature vector of each audio signal sample together form a training sample set to increase the number of training sample sets and further improve the performance of the target speech recognition model.

[0119] This embodiment determines the feature vector of each audio signal sample, performs mean value processing and variance normalization processing on the feature vector of each audio signal sample to obtain a target feature vector, and performs speed processing on the target feature vector of each audio signal sample to obtain a training sample set, that is, the training sample set includes audio signals at different speech rates. Therefore, this embodiment increases the number of training sample sets, thereby being able to improve the performance of the target speech recognition model obtained from the training sample sets.

[0120] In summary, this embodiment uses the simulated future information module to enable the model to perform low-latency streaming recognition without relying on real future information. In addition, this embodiment adopts a training strategy that shares parameters between the streaming model and the non-streaming model, which narrows the performance gap between the streaming model and the non-streaming model.

[0121] If the traditional speech recognition method does not rely on contextual information, that is, the historical frames and future frames of the speech to be recognized, the recognition accuracy will decrease; if it relies on contextual information, it is necessary to wait for the arrival of the real future frames, which will increase the recognition delay. The training and speech recognition test of this embodiment on the commonly used Chinese speech recognition data set aishell-1, the final test results show that compared with the traditional speech recognition method, the low-latency speech recognition method based on block segmentation, future prediction, and decoding proposed in this application obtains the best speech recognition results with minimal delay.

[0122] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0123] Based on the same inventive concept, the embodiment of the present application also provides a speech recognition device for implementing the speech recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more speech recognition device embodiments provided below can refer to the limitations on the speech recognition method above, and will not be repeated here.

[0124] Fig.10 is a structural block diagram of a speech recognition device in an embodiment of the present application. In an embodiment of the present application, Fig.10 As shown, a speech recognition device 1000 is provided, comprising: a first acquisition module 1001, a simulation module 1002 and a determination module 1003, wherein:

[0125] A first acquisition module 1001 is used to acquire a current frame of the speech to be recognized and a history frame before the current frame, and use the current frame and the history frame before the current frame as a first block;

[0126] A simulation module 1002, configured to simulate a future frame corresponding to the first block according to the first block and a target synthesis predictor;

[0127] The determination module 1003 is used to determine the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model.

[0128] The speech recognition device provided in this embodiment obtains the current frame of the speech to be recognized and the historical frames before the current frame, and uses the current frame and the historical frames before the current frame as the first block, and then simulates the future frame corresponding to the first block according to the first block and the target synthesis predictor, thereby determining the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model. Since the speech recognition result of the speech to be recognized is determined according to the historical frames, the current frame and the simulated future frames of the speech to be recognized in this application, that is, the speech recognition device in this application does not need to wait for a certain real future frame but directly uses the simulated future frame, thus avoiding the situation in the traditional technology that must wait until a certain number of future frames arrive before starting recognition, solving the problem of waiting for real future frames in the traditional technology leading to increased recognition delay time, and reducing the recognition delay time.

[0129] Optionally, the simulation module 1002 includes:

[0130] The encoding unit is used to input the first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block.

[0131] A simulation unit is used to input the second block into the target synthesis predictor to simulate a future frame corresponding to the first block.

[0132] Optionally, the determining module 1003 includes:

[0133] A splicing unit is used to use the target acoustic model to perform splicing processing on the first block and the future frame to obtain a splicing result.

[0134] The first determining unit is used to determine the speech recognition result according to the splicing result.

[0135] Optionally, the first block is at least one block obtained after block processing of the speech to be recognized, and the length of each of the first blocks is determined according to a preset uniform distribution interval.

[0136] Optionally, the speech recognition device 1000 further includes:

[0137] The second acquisition module is used to acquire a training sample set.

[0138] A training module is used to train an initial speech recognition model according to the training sample set to obtain a target speech recognition model, wherein the target speech recognition model includes the target synthesis predictor, the target acoustic model and the synthesis encoder.

[0139] Optional training modules include:

[0140] The second determining unit is used to determine the non-streaming loss and the streaming loss according to the training sample set and the initial acoustic model.

[0141] The third determination unit is used to determine the simulation loss according to the training sample set, the initial synthesis encoder and the initial synthesis predictor.

[0142] The third determination unit is used to train the initial speech recognition model according to the non-streaming loss, the streaming loss and the simulated loss to obtain the target speech recognition model.

[0143] Optionally, the second acquisition module includes:

[0144] The fourth determining unit is used to determine the feature vector of each audio signal sample.

[0145] The first processing unit is used to perform mean processing and variance normalization processing on the feature vector of each audio signal sample to obtain a target feature vector.

[0146] The second processing unit is used to perform double-speed processing on the target feature vector of each of the audio signal samples to obtain the training sample set.

[0147] Each module in the above-mentioned speech recognition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0148] Fig.11 : is an internal structure diagram of a computer device in an embodiment of the present application. In an embodiment of the present application, a computer device is provided. The computer device may be a server. The internal structure diagram thereof may be as shown in FIG. Fig.11 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a speech recognition method is implemented.

[0149] Those skilled in the art will understand that Fig.11The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0151] Acquire a current frame of the speech to be recognized and a historical frame before the current frame, and use the current frame and the historical frame before the current frame as a first block;

[0152] Simulating a future frame corresponding to the first block according to the first block and a target synthesis predictor;

[0153] A speech recognition result of the speech to be recognized is determined according to the first block, the future frame and the target acoustic model.

[0154] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0155] Inputting the first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block;

[0156] The second block is input into the target synthesis predictor to simulate a future frame corresponding to the first block.

[0157] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0158] Performing splicing processing on the first block and the future frame to obtain a splicing result;

[0159] The speech recognition result is determined according to the splicing result and the target acoustic model.

[0160] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0161] The first block is at least one block obtained after performing block processing on the speech to be recognized, and the length of each of the first blocks is determined according to a preset uniform distribution interval.

[0162] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0163] Obtain a training sample set;

[0164] According to the training sample set, the initial speech recognition model is trained to obtain a target speech recognition model, wherein the target speech recognition model includes the target synthesis predictor, the target acoustic model and the synthesis encoder.

[0165] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0166] Determining non-streaming loss and streaming loss according to the training sample set and the initial acoustic model;

[0167] Determining a simulation loss based on the training sample set, the initial synthesis encoder, and the initial synthesis predictor;

[0168] The initial speech recognition model is trained according to the non-streaming loss, the streaming loss and the simulated loss to obtain the target speech recognition model.

[0169] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0170] determining a feature vector for each audio signal sample;

[0171] Performing mean processing and variance normalization processing on the feature vectors of each of the audio signal samples to obtain a target feature vector;

[0172] The target feature vector of each of the audio signal samples is processed at a double speed to obtain the training sample set.

[0173] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0174] Acquire a current frame of the speech to be recognized and a historical frame before the current frame, and use the current frame and the historical frame before the current frame as a first block;

[0175] Simulating a future frame corresponding to the first block according to the first block and a target synthesis predictor;

[0176] A speech recognition result of the speech to be recognized is determined according to the first block, the future frame and the target acoustic model.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0178] Inputting the first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block;

[0179] The second block is input into the target synthesis predictor to simulate a future frame corresponding to the first block.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0181] Performing splicing processing on the first block and the future frame to obtain a splicing result;

[0182] The speech recognition result is determined according to the splicing result and the target acoustic model.

[0183] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0184] The first block is at least one block obtained after performing block processing on the speech to be recognized, and the length of each of the first blocks is determined according to a preset uniform distribution interval.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0186] Obtain a training sample set;

[0187] According to the training sample set, the initial speech recognition model is trained to obtain a target speech recognition model, wherein the target speech recognition model includes the target synthesis predictor, the target acoustic model and the synthesis encoder.

[0188] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0189] Determining non-streaming loss and streaming loss according to the training sample set and the initial acoustic model;

[0190] Determining a simulation loss based on the training sample set, the initial synthesis encoder, and the initial synthesis predictor;

[0191] The initial speech recognition model is trained according to the non-streaming loss, the streaming loss and the simulated loss to obtain the target speech recognition model.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] determining a feature vector for each audio signal sample;

[0194] Performing mean processing and variance normalization processing on the feature vectors of each of the audio signal samples to obtain a target feature vector;

[0195] The target feature vector of each of the audio signal samples is processed at a double speed to obtain the training sample set.

[0196] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0197] Acquire a current frame of the speech to be recognized and a historical frame before the current frame, and use the current frame and the historical frame before the current frame as a first block;

[0198] Simulating a future frame corresponding to the first block according to the first block and a target synthesis predictor;

[0199] A speech recognition result of the speech to be recognized is determined according to the first block, the future frame and the target acoustic model.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] Inputting the first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block;

[0202] The second block is input into the target synthesis predictor to simulate a future frame corresponding to the first block.

[0203] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0204] Performing splicing processing on the first block and the future frame to obtain a splicing result;

[0205] The speech recognition result is determined according to the splicing result and the target acoustic model.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0207] The first block is at least one block obtained after performing block processing on the speech to be recognized, and the length of each of the first blocks is determined according to a preset uniform distribution interval.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0209] Obtain a training sample set;

[0210] According to the training sample set, the initial speech recognition model is trained to obtain a target speech recognition model, wherein the target speech recognition model includes the target synthesis predictor, the target acoustic model and the synthesis encoder.

[0211] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0212] Determining non-streaming loss and streaming loss according to the training sample set and the initial acoustic model;

[0213] Determining a simulation loss based on the training sample set, the initial synthesis encoder, and the initial synthesis predictor;

[0214] The initial speech recognition model is trained according to the non-streaming loss, the streaming loss and the simulated loss to obtain the target speech recognition model.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0216] determining a feature vector for each audio signal sample;

[0217] Performing mean processing and variance normalization processing on the feature vectors of each of the audio signal samples to obtain a target feature vector;

[0218] The target feature vector of each of the audio signal samples is processed at a double speed to obtain the training sample set.

[0219] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0220] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0221] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A speech recognition method, characterized in that: The method comprises: Acquire a current frame of the speech to be recognized and a historical frame before the current frame, and use the current frame and the historical frame before the current frame as a first block; Simulating a future frame corresponding to the first block according to the first block and a target synthesis predictor; Determining a speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model; Among them, the target speech recognition model includes the target synthesis predictor and the target acoustic model. The target speech recognition model is a model obtained by training the initial speech recognition model according to non-streaming loss, streaming loss and simulation loss. The simulation loss is determined according to the training sample set, the initial synthesis encoder and the initial synthesis predictor. The non-streaming loss and streaming loss are both determined according to the training sample set and the initial acoustic model.

2. The method according to claim 1, characterized in that The simulating a future frame corresponding to the first block according to the first block and the target synthesis predictor includes: Inputting the first block into a target synthesis encoder to obtain a second block, wherein the second block is a block obtained after encoding the first block; The second block is input into the target synthesis predictor to simulate a future frame corresponding to the first block.

3. The method according to claim 1, characterized in that The step of determining the speech recognition result of the speech to be recognized according to the first block, the future frame and the target acoustic model includes: Performing splicing processing on the first block and the future frame to obtain a splicing result; The speech recognition result is determined according to the splicing result and the target acoustic model.

4. The method according to claim 1, characterized in that: The first block is at least one block obtained after performing block processing on the speech to be recognized, and the length of each of the first blocks is determined according to a preset uniform distribution interval.

5. The method according to claim 2, characterized in that: The method further comprises: Obtaining the training sample set; The initial speech recognition model is trained according to the training sample set to obtain the target speech recognition model, and the target speech recognition model also includes the synthesis encoder.

6. The method according to claim 5, characterized in that The obtaining of the training sample set comprises: determining a feature vector for each audio signal sample; Performing mean processing and variance normalization processing on the feature vectors of each of the audio signal samples to obtain a target feature vector; The target feature vector of each of the audio signal samples is processed at a double speed to obtain the training sample set.

7. A speech recognition device, characterized in that: The device comprises: A first acquisition module, used for acquiring a current frame of the speech to be recognized and a historical frame before the current frame, and taking the current frame and the historical frame before the current frame as a first block; A simulation module, configured to simulate a future frame corresponding to the first block according to the first block and a target synthesis predictor; A determination module, configured to determine a speech recognition result of the speech to be recognized according to the first block, the future frame and a target acoustic model; Among them, the target speech recognition model includes the target synthesis predictor and the target acoustic model. The target speech recognition model is a model obtained by training the initial speech recognition model according to non-streaming loss, streaming loss and simulation loss. The simulation loss is determined according to the training sample set, the initial synthesis encoder and the initial synthesis predictor. The non-streaming loss and streaming loss are both determined according to the training sample set and the initial acoustic model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images