Speech recognition method and device, electronic equipment and medium

Abstract

The invention provides a voice recognition method and device, electronic equipment and a medium, and the method comprises the steps: obtaining a to-be-recognized audio in response to a voice recognition request of a user, sampling the to-be-recognized audio to obtain initial audio data, determining the spectrum feature data of the initial audio data, and storing the spectrum feature data of the initial audio data; processing the initial audio data according to the frequency spectrum feature data to obtain human voice audio data, splitting the human voice audio data to obtain target audio data, inputting the target audio data into a pre-trained voice recognition model for parallel recognition to obtain an initial text sequence output by the voice recognition model, and determining starting and ending time corresponding to words in the initial text sequence according to the human voice audio data, sorting the words in the initial text sequence according to the starting and ending time to obtain a target text sequence, taking the target text sequence as a voice recognition result, and displaying the voice recognition result to a user. The method can improve the recognition accuracy while guaranteeing the recognition speed, and improves the user experience.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a speech recognition method, device, electronic device and medium. Background Art

[0002] Existing intelligent customer service systems generally use speech recognition technology to improve service efficiency and user experience. Traditional automatic speech recognition mostly uses neural network models under the DNN (Deep Neural Network) framework. In recent years, the Transformer model based on the self-attention mechanism has made significant progress in the fields of natural language processing (NLP) and speech processing. Whisper, as a new ASR (speech recognition) system, takes advantage of the Transformer model and is trained on large-scale data sets to perform high-quality speech recognition and translation. Compared with the RNN (Recurrent Neural Network)-based method in the prior art, Transformer can better capture long-distance dependencies when processing sequence data, and has higher training efficiency, which means that private training can be performed for industry or customer scenario knowledge, and the recognition rate can be greatly improved after training.

[0003] The ASR algorithm based on the RNN model recognizes speech information quickly, but the accuracy is not high. The Whisper model has high accuracy but slow processing of speech information when the context information is long or the parameters are large. The existing speech recognition model cannot take into account both speech recognition efficiency and accuracy. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application propose a speech recognition method, device, electronic device and medium.

[0005] In a first aspect of the implementation of the present application, a speech recognition method is first provided, characterized in that the method comprises:

[0006] In response to a user's speech recognition request, obtaining audio to be recognized, and sampling the audio to be recognized to obtain initial audio data;

[0007] Determine frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain human voice audio data;

[0008] Splitting the human voice audio data to obtain at least one target audio data;

[0009] Inputting at least one of the target audio data into a pre-trained speech recognition model for parallel recognition to obtain at least one initial text sequence output by the speech recognition model;

[0010] Determine the start and end times corresponding to the words in the initial text sequence according to the human voice audio data;

[0011] Sort the words in the initial text sequence according to the start and end time to obtain a target text sequence;

[0012] The target text sequence is used as a speech recognition result, and the speech recognition result is displayed to the user.

[0013] Optionally, determining the frequency spectrum feature data of the initial audio data, and processing the initial audio data according to the frequency spectrum feature data to obtain the human voice audio data includes:

[0014] Determine the frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain candidate audio data;

[0015] Power spectrum density data of the candidate audio data is determined, and the candidate audio data is processed according to the power spectrum density data to obtain the human voice audio data.

[0016] Optionally, the determining the frequency spectrum feature data of the initial audio data, and processing the initial audio data according to the frequency spectrum feature data to obtain candidate audio data includes:

[0017] Determining linear frequency data and power spectrum data of the initial audio data;

[0018] Convert the linear frequency data into Mel frequency according to a preset frequency conversion formula;

[0019] Obtain a preset filter set, and determine frequency response data of any filter in the filter set;

[0020] Determine the frequency spectrum characteristic data according to the frequency response data and the power spectrum data;

[0021] The initial audio data is processed according to the frequency spectrum feature data to obtain the candidate audio data.

[0022] Optionally, determining power spectrum density data of the candidate audio data, and processing the candidate audio data according to the power spectrum density data to obtain the human voice audio data includes:

[0023] Performing Fourier transform on the candidate audio data to obtain frequency domain audio data;

[0024] Determine power spectrum density data of the candidate audio data according to the frequency domain audio data;

[0025] The candidate audio data is processed according to the power spectrum density data to obtain the human voice audio data.

[0026] Optionally, the human voice audio data includes an audio frame sequence, and determining the start and end times corresponding to the words in the initial text sequence according to the human voice audio data includes:

[0027] Determine, according to the words in the initial text sequence and the audio frame sequence, attention weight data between any of the words in the initial text sequence and the audio frame in the audio frame sequence;

[0028] Constructing an attention weight matrix according to the attention weight data; wherein any row of data in the attention weight matrix corresponds to any word in the initial text sequence, and any column of data in the attention weight matrix corresponds to the audio frame sequence;

[0029] For any of the words in the initial text sequence, determining a target audio frame corresponding to the word according to the attention weight matrix;

[0030] Obtaining a sampling rate of the human voice audio data, and determining a start and end time of the target audio frame according to a position of the target audio frame in the attention weight matrix and the sampling rate;

[0031] The start and end times of the target audio frame are used as the start and end times corresponding to the word.

[0032] Optionally, the speech recognition model is trained in the following manner:

[0033] Acquire training samples and verification samples; the training samples include training voice data and training text data; the verification samples include verification voice data and verification text data;

[0034] Using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using the training samples to train the speech recognition model to obtain a speech recognition model to be verified;

[0035] Inputting the verification voice data into the speech recognition model to be verified to obtain sample text data output by the speech recognition model to be verified;

[0036] Calculate the loss function corresponding to the speech recognition model according to the sample text data and the verification text data;

[0037] Obtaining an initial weight matrix of the speech recognition model to be verified;

[0038] Decomposing the initial weight matrix to obtain a low-rank weight matrix of the speech recognition model to be verified;

[0039] The loss function corresponding to the speech recognition model is used to adjust the low-rank weight matrix of the speech recognition model to be verified, and the steps of using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using training samples to train the speech recognition model are re-executed until a preset training stop condition is reached to obtain a trained speech recognition model.

[0040] Optionally, the speech recognition model includes an initial recognition model and a target recognition model, the initial recognition model includes at least one linear layer and an activation function layer, the target recognition model includes a fusion layer, and the inputting at least one of the target audio data into a pre-trained speech recognition model for parallel recognition to obtain at least one initial text sequence output by the speech recognition model includes:

[0041] Acquire initial floating-point precision data of the low-rank weight matrix in the initial recognition model, and adjust the initial floating-point precision data to obtain target floating-point precision data;

[0042] Obtaining the linear layer and the activation function layer in the initial recognition model;

[0043] Fusing the linear layer and the activation function layer to obtain the fused layer;

[0044] Constructing the target recognition model according to the target floating-point precision data and the fusion layer;

[0045] Respectively obtain the audio duration of at least one of the target audio data;

[0046] Sort at least one of the target audio data in descending order according to the audio duration to obtain an initial audio data sequence;

[0047] Filling the target audio data in the audio data sequence with preset filling data to obtain a target audio data sequence;

[0048] The target audio data in the target audio data sequence is input into the target recognition model for parallel recognition to obtain at least one initial text sequence output by the target recognition model.

[0049] In a second aspect of the present application, a speech recognition device is provided, wherein the device comprises:

[0050] An initial audio acquisition module, used to respond to a user's speech recognition request, acquire the audio to be recognized, and sample the audio to be recognized to obtain initial audio data;

[0051] A human voice audio acquisition module, used to determine frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain human voice audio data;

[0052] A human voice audio splitting module, used for splitting the human voice audio data to obtain at least one target audio data;

[0053] A human voice audio recognition module, used for inputting at least one of the target audio data into a pre-trained speech recognition model for parallel recognition, and obtaining at least one initial text sequence output by the speech recognition model;

[0054] A time determination module, used to determine the start and end times corresponding to the words in the initial text sequence according to the human voice audio data;

[0055] A target text acquisition module, used for sorting the words in the initial text sequence according to the start and end time to obtain a target text sequence;

[0056] The target text display module is used to use the target text sequence as a speech recognition result and display the speech recognition result to the user.

[0057] In the third aspect of the implementation of the present application, there is also provided an electronic device, characterized in that it includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, it implements the speech recognition method as described above.

[0058] In the fourth aspect of the implementation of the present application, a computer-readable storage medium is also provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the speech recognition method as described above is implemented.

[0059] The embodiments of the present application have the following advantages:

[0060] In an embodiment of the present application, in response to a user's speech recognition request, the audio to be recognized is obtained, the audio to be recognized is sampled to obtain initial audio data, the spectral feature data of the initial audio data is determined, the initial audio data is processed according to the spectral feature data to obtain human voice audio data, the human voice audio data is split to obtain at least one target audio data, at least one target audio data is input into a pre-trained speech recognition model for parallel recognition, at least one initial text sequence output by the speech recognition model is obtained, the start and end times corresponding to the words in the initial text sequence are determined according to the human voice audio data, the words in the initial text sequence are sorted according to the start and end times, the target text sequence is obtained, the target text sequence is used as the speech recognition result, and the speech recognition result is displayed to the user. The present application achieves efficient and accurate speech recognition through a series of steps such as sampling the audio to be recognized, extracting spectral features, processing human voice audio data, splitting the target audio data, parallel recognition, determining the start and end times, and sorting the text sequence. The present application makes full use of the advantages of spectral feature data and pre-trained speech recognition models, and can improve the recognition accuracy while ensuring the recognition speed. Parallel recognition can further improve recognition speed and accuracy, and ultimately present the target text sequence that conforms to the original speech timing to the user as the speech recognition result, thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0062] Figure 1 is a flowchart of a speech recognition method provided by an embodiment of the present application;

[0063] Figure 2 is a flowchart of the steps of a model acceleration method provided by an embodiment of the present application;

[0064] Figure 3 This is a schematic diagram of speech recognition effect comparison provided by an embodiment of the present application;

[0065] Figure 4 is a structural diagram of an attention weight matrix provided in one embodiment of the present application;

[0066] Figure 5 It is a flowchart of the steps of fine-tuning the Whisper model using the LoRA method provided in one embodiment of the present application;

[0067] Figure 6 This is a schematic diagram of the convergence curve of the amount of labeled data and the accuracy provided in an embodiment of the present application.

[0068] Figure 7This is a schematic diagram of the speech recognition model recognition result data provided by an embodiment of the present application

[0069] Figure 8 This is a schematic diagram of the original Whisper model recognition result data provided by an embodiment of the present application

[0070] Fig. 9 It is a structural diagram of a speech recognition device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0071] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not constitute any limitation, and the various embodiments can be combined with each other and referenced to each other without contradiction.

[0072] The embodiment of the present application uses transformer-based speech recognition technology to build a model with low video memory consumption that can quickly recognize and display two-channel dialogues in an intelligent customer service question-and-answer system, thereby promoting the implementation of AI robot projects in call center telephone service centers.

[0073] Reference Figure 1 , shows a flowchart of a method for calibrating parameters of a digital Bluetooth key provided by an embodiment of the present application, which is applied to a terminal device. The method may specifically include the following steps:

[0074] S101: In response to a user's speech recognition request, obtain audio to be recognized, and sample the audio to be recognized to obtain initial audio data.

[0075] In the embodiment of the present application, in response to a user's speech recognition request, the audio to be recognized can be obtained, and the audio to be recognized can be sampled to obtain initial audio data. The initial audio data is obtained by directly sampling the audio to be recognized.

[0076] In the specific implementation, the pydub library can be used to load the voice file to achieve the sampling of the audio to be recognized. The corresponding file has a sampling rate of 16K.

[0077] It should be noted that pydub is a Python library for processing audio files, which provides a simple and easy-to-use interface to operate audio data. Pydub supports a variety of audio formats and can be used in conjunction with ffmpeg or libav to provide more powerful functions.

[0078] S102: Determine frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain human voice audio data.

[0079] In the embodiment of the present application, the frequency spectrum feature data of the initial audio data can be determined, and the initial audio data can be processed according to the frequency spectrum feature data to obtain human voice audio data, wherein the human voice audio data is audio data that conforms to the auditory characteristics of the human ear and has been filtered out of non-human voice data.

[0080] S103: Split the human voice audio data to obtain at least one target audio data.

[0081] In an embodiment of the present application, human voice audio data may be split to obtain at least one target audio data.

[0082] S104: Input at least one of the target audio data into a pre-trained speech recognition model for parallel recognition to obtain at least one initial text sequence output by the speech recognition model.

[0083] In an embodiment of the present application, at least one target audio data may be input into a pre-trained speech recognition model for parallel recognition to obtain at least one initial text sequence output by the speech recognition model. The speech recognition model may be a Whisper model. The initial text sequence is a text sequence generated based on the target audio data.

[0084] It should be noted that the Whisper model is an automatic speech recognition (ASR) model developed by OpenAI. The Whisper model is based on the Transformer architecture and is specifically designed to handle multilingual speech recognition tasks. It performs well in processing multiple languages ​​and dialects, and can adapt to different audio quality and background noise.

[0085] S105: Determine the start and end times corresponding to the words in the initial text sequence according to the human voice audio data.

[0086] In an embodiment of the present application, the start and end times corresponding to the words in the initial text sequence can be determined according to the human voice audio data. The start and end times corresponding to the words can be the start and end times of the words in the initial text sequence in the human voice audio data.

[0087] S106: Sort the words in the initial text sequence according to the start and end time to obtain a target text sequence.

[0088] In an embodiment of the present application, the words in the initial text sequence can be sorted according to the start and end time to obtain a target text sequence, wherein the words in the target text sequence match the words in the human voice audio data, and the words in the target text sequence are arranged in the time sequence of the words in the human voice audio data.

[0089] S107: Using the target text sequence as a speech recognition result, and displaying the speech recognition result to the user.

[0090] In an embodiment of the present application, the target text sequence can be used as a speech recognition result, and the speech recognition result can be displayed to the user.

[0091] In the specific implementation, you can refer to Figure 2 Speed ​​up the process of model processing data. Figure 2 , showing a step flow chart of a model acceleration method provided in an embodiment of the present application.

[0092] First, the audio is loaded and preprocessed, and then the audio is decoded, encoded, normalized, and other operations are performed. Then, the mel feature is extracted, and the long speech segments are segmented, and then the mel filter is filtered. Then VAD (Voice Activity Detection) processing is performed, that is, noise processing and speech segment processing are performed. Finally, the Whisper model is used to recognize the speech to obtain text, and multiple working processes can be set. After the recognition is completed, multiple texts are assembled into a complete ASR text.

[0093] In the specific implementation, you can refer to Figure 3 To see the speech recognition results of this application, refer to Figure 3 , showing a schematic diagram of speech recognition effect comparison provided by an embodiment of the present application.

[0094] Figure 3 The left side in the middle is the speech recognition result of the prior art. It can be seen that its recognition accuracy for some words is not high. The right side is the speech recognition result of the present application, which can accurately recognize various contents and improve the user experience.

[0095] In an embodiment of the present application, in response to a user's speech recognition request, the audio to be recognized is obtained, the audio to be recognized is sampled to obtain initial audio data, the spectral feature data of the initial audio data is determined, the initial audio data is processed according to the spectral feature data to obtain human voice audio data, the human voice audio data is split to obtain at least one target audio data, at least one target audio data is input into a pre-trained speech recognition model for parallel recognition, at least one initial text sequence output by the speech recognition model is obtained, the start and end times corresponding to the words in the initial text sequence are determined according to the human voice audio data, the words in the initial text sequence are sorted according to the start and end times, the target text sequence is obtained, the target text sequence is used as the speech recognition result, and the speech recognition result is displayed to the user. The present application achieves efficient and accurate speech recognition through a series of steps such as sampling the audio to be recognized, extracting spectral features, processing human voice audio data, splitting the target audio data, parallel recognition, determining the start and end times, and sorting the text sequence. The present application makes full use of the advantages of spectral feature data and pre-trained speech recognition models, and can improve the recognition accuracy while ensuring the recognition speed. Parallel recognition can further improve recognition speed and accuracy, and ultimately present the target text sequence that conforms to the original speech timing to the user as the speech recognition result, thus improving the user experience.

[0096] In an optional embodiment of the present application, step S102 further includes the following sub-steps:

[0097] S11: determining the frequency spectrum feature data of the initial audio data, and processing the initial audio data according to the frequency spectrum feature data to obtain candidate audio data;

[0098] S12: Determine power spectrum density data of the candidate audio data, and process the candidate audio data according to the power spectrum density data to obtain the human voice audio data.

[0099] In an embodiment of the present application, the frequency spectrum feature data of the initial audio data may be determined first, the initial audio data may be processed according to the frequency spectrum feature data to obtain candidate audio data, the power spectrum density data of the candidate audio data may be determined, the candidate audio data may be processed according to the power spectrum density data to obtain human voice audio data. Among them, the frequency spectrum feature data may be feature data that can characterize the auditory characteristics of the human ear. The candidate audio data may be audio data that can reflect the auditory characteristics of the human ear. The human voice audio data may be audio data that can reflect the auditory characteristics of the human ear and contains only valid audio after filtering out the interrupted or speech-free parts of the audio.

[0100] In an optional embodiment of the present application, step S11 further includes the following sub-steps:

[0101] S21: Determine linear frequency data and power spectrum data of the initial audio data;

[0102] S22: converting the linear frequency data into Mel frequency according to a preset frequency conversion formula;

[0103] S23: Obtain a preset filter set, and determine frequency response data of any filter in the filter set;

[0104] S24: Determine the frequency spectrum characteristic data according to the frequency response data and the power spectrum data;

[0105] S25: Processing the initial audio data according to the frequency spectrum feature data to obtain the candidate audio data.

[0106] In an embodiment of the present application, the linear frequency data and power spectrum data of the initial audio data can be determined, the linear frequency data can be converted into Mel frequency according to a preset frequency conversion formula, a preset filter set can be obtained, and for any filter in the filter set, the frequency response data of the filter can be determined, and the frequency spectrum feature data can be determined based on the frequency response data and the power spectrum data. The initial audio data can be processed based on the frequency spectrum feature data to obtain candidate audio data.

[0107] In the specific implementation, the use of Mel filtering for spectrum representation is more in line with the auditory characteristics of the human ear. The human ear's perception of frequency is not linear, but is more sensitive to low frequencies and relatively insensitive to high frequencies. The Mel filter is designed based on this feature.

[0108] The Mel filter bank consists of a set of triangular filters that are evenly distributed on the Mel frequency axis. The core idea is to map the frequencies on the linear frequency axis to the Mel frequency axis through a nonlinear conversion function.

[0109] The specific step is frequency conversion, using the formula to convert the linear frequency f to the Mel frequency Mel(f):

[0110]

[0111] Filter bank design, design a set of triangular filters whose center frequencies are evenly distributed on the Mel frequency axis. For each filter, calculate the power spectrum of the audio signal and perform weighted summation to obtain the output of each filter.

[0112] In an optional embodiment of the present application, step S12 further includes the following sub-steps:

[0113] S31: Perform Fourier transform on the candidate audio data to obtain frequency domain audio data;

[0114] S32: Determine power spectrum density data of the candidate audio data according to the frequency domain audio data;

[0115] S33: Process the candidate audio data according to the power spectrum density data to obtain the human voice audio data.

[0116] In an embodiment of the present application, the candidate audio data can be Fourier transformed to obtain frequency domain audio data, the power spectrum density data of the candidate audio data can be determined based on the frequency domain audio data, and the candidate audio data can be processed based on the power spectrum density data to obtain human voice audio data.

[0117] In a specific implementation, after filtering, the VAD model can be applied to filter the intermittent voiceless parts, and the Silero VAD (Voice Activity Detection) model can be used. After Fourier transforming the sampled data, the energy (amplitude) and spectrum (frequency) characteristics per unit time are calculated, such as the power spectral density (PSD). It is used to detect the voice activity part in the audio signal and filter out the silent and voiceless parts. In an embodiment of the present application, the power spectral density parameters can be: the sampling rate is 160000, the Fourier frequency is 1024, and then normalization and single-window power averaging are performed. And after actual testing, in different scenarios, with different voice strengths and speeds, the present application can correctly handle silence and abnormal noises.

[0118] In the VAD model, Fourier transform can be used to perform spectrum analysis on sampled data and calculate the energy (amplitude) and spectrum characteristics per unit time, such as power spectral density (PSD). The Fourier transform formula is:

[0119]

[0120] Among them, X(f) represents the frequency domain signal, which represents the component with frequency f, x(t) is the time domain signal, e -j2πft is a complex exponential function, j is an imaginary unit,

[0121] Power spectral density can be used to measure the power distribution of a signal per unit frequency and is usually used to identify active parts of speech. The calculation of power spectral density is based on the Fourier transformed signal and can be expressed as:

[0122]

[0123] Among them, P xx (f) is the power spectral density, X T (f) is the Fourier transform result of the signal in the time window T, |X T (f)| 2is the square of the modulus of the Fourier transform.

[0124] In an optional embodiment of the present application, step S105 further includes the following sub-steps:

[0125] S41: Determine attention weight data between any word in the initial text sequence and an audio frame in the audio frame sequence according to the words in the initial text sequence and the audio frame sequence;

[0126] S42: constructing an attention weight matrix according to the attention weight data; wherein any row of data in the attention weight matrix corresponds to any word in the initial text sequence, and any column of data in the attention weight matrix corresponds to the audio frame sequence;

[0127] S43: for any of the words in the initial text sequence, determining a target audio frame corresponding to the word according to the attention weight matrix;

[0128] S44: Obtaining the sampling rate of the human voice audio data, and determining the start and end time of the target audio frame according to the position of the target audio frame in the attention weight matrix and the sampling rate;

[0129] S45: Taking the start and end time of the target audio frame as the start and end time corresponding to the word.

[0130] In an embodiment of the present application, attention weight data between any word in the initial text sequence and the audio frame in the audio frame sequence can be determined based on the words in the initial text sequence and the audio frame sequence. Then, an attention weight matrix can be constructed based on the attention weight data, wherein any row of data in the attention weight matrix corresponds to any word in the initial text sequence, and any column of data in the attention weight matrix corresponds to the audio frame sequence. For any word in the initial text sequence, the target audio frame corresponding to the word is determined based on the attention weight matrix, the sampling rate of the human voice audio data is obtained, the start and end times of the target audio frame are determined based on the position and sampling rate of the target audio frame in the attention weight matrix, and the start and end times of the target audio frame are used as the start and end times corresponding to the word.

[0131] In the specific implementation, in order to make up for the text without timestamps after parallel processing so that the ASR dialogue can be displayed in normal order, the speech needs to be re-processed, and the DTW (Dynamic Time Warping) model is used to calculate the correlation between the audio and words, and then the timestamps are added to the text sequence.

[0132] Calculate the attention weights for the human voice audio data and the initial text sequence.

[0133] For each output word, the model calculates the attention weights between it and each audio frame to form a matrix. Each row of the matrix represents a word and the column represents an audio frame. Figure 4 , which shows a schematic diagram of the structure of an attention weight matrix provided in an embodiment of the present application. In the embodiment of the present application, reference can be made to Figure 4 Construct the attention weight matrix.

[0134] Then, the most relevant audio frame can be found. For each word, by finding the position with the largest attention weight, the most relevant audio frame for the word can be determined. For example, if the maximum value in a row is in the 10th column, it means that the word is most relevant to the 10th frame. Calculate the timestamp: The time corresponding to the frame can be calculated by the index of the audio frame and the frame interval time, where the frame interval time is determined by the sampling rate and hop_length (the number of samples from the beginning of one frame to the beginning of the next frame on the timeline).

[0135] Assuming the sampling rate is 16000Hz and hop_length is 256, the time interval of each frame is: The time interval t is calculated by dividing hop_length by the sampling rate. In this example, t can be 0.016 seconds. If a word is most relevant to the 10th frame, its timestamp is: 10×0.016=0.16 seconds, so the frame timestamp corresponding to the original audio can be added before each word.

[0136] The following is an example of data after restoration and transformation using the DTW model:

[0137]

[0138]

[0139] In an optional embodiment of the present application, the speech recognition model is trained in the following manner:

[0140] S51: Acquire training samples and verification samples; the training samples include training voice data and training text data; the verification samples include verification voice data and verification text data;

[0141] S52: using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using the training samples to train the speech recognition model to obtain a speech recognition model to be verified;

[0142] S53: inputting the verification voice data into the voice recognition model to be verified, and obtaining sample text data output by the voice recognition model to be verified;

[0143] S54: Calculating a loss function corresponding to the speech recognition model according to the sample text data and the verification text data;

[0144] S55: Obtaining an initial weight matrix of the speech recognition model to be verified;

[0145] S56: Decomposing the initial weight matrix to obtain a low-rank weight matrix of the speech recognition model to be verified;

[0146] S57: Use the loss function corresponding to the speech recognition model to adjust the low-rank weight matrix of the speech recognition model to be verified, and re-execute the steps of using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using training samples to train the speech recognition model until the preset training stop condition is reached, thereby obtaining a trained speech recognition model.

[0147] In an embodiment of the present application, training samples and verification samples can be obtained; the training samples include training voice data and training text data, and the verification samples include verification voice data and verification text data. The training voice data is used as the input of the speech recognition model, and the training text data is used as the output of the speech recognition model. The speech recognition model is trained using the training samples to obtain a speech recognition model to be verified, and the verification voice data is input into the speech recognition model to be verified to obtain sample text data output by the speech recognition model to be verified. The loss function corresponding to the speech recognition model is calculated based on the sample text data and the verification text data.

[0148] Then, the initial weight matrix of the speech recognition model to be verified can be obtained, the initial weight matrix can be decomposed to obtain the low-rank weight matrix of the speech recognition model to be verified, the loss function corresponding to the speech recognition model can be used to adjust the low-rank weight matrix of the speech recognition model to be verified, and the steps of using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using the training samples to train the speech recognition model can be re-executed until the preset training stop condition is reached to obtain a trained speech recognition model.

[0149] In the specific implementation, the LoRA (Low-Rank Adaptation of Large Language Models) method can be used to fine-tune the Whisper model. Figure 5 , shows a flowchart of the steps of fine-tuning the Whisper model using the LoRA method provided in an embodiment of the present application.

[0150] First load the LoRA configuration and data, then load the basic Whisper model to get the Whisper matrix. Then load the LoRA model to get the LoRA training data matrix. Use the low-rank matrix of the LoRA training data matrix to transpose the Whisper matrix for training. Then transpose the decoding matrix and save the fine-tuned Whisper model.

[0151] In the specific implementation, you can load the LoRA configuration first, which is for the introduction of the low-rank matrix.

[0152] Assume that a matrix W can be approximated as the product of two low-rank matrices: W ≈ U × V

[0153] Where U and V are low-rank matrices. This decomposition method makes the number of parameters of the original matrix O(n 2 ) to a lower rank matrix O(nk) where n is the dimension of the original matrix and k is the rank of the matrix.

[0154] Then the basic Whisper model can be loaded. In the embodiment of the present application, whisper large v2 is used, the bare model is about 1.6G in size, and the memory used is about 11.3G.

[0155] Load the LoRA model. LoRa technology reduces the complexity and computation of model parameters by introducing low-rank matrices in the neural network. It decomposes the weight matrix into two low-rank matrices, thereby reducing the computational burden during model training and inference. During model training, only the low-rank matrices U and V are updated instead of the entire weight matrix.

[0156] The matrix of the basic model is transposed by the multiplication of the basic model to generate the reduced-dimensional attention model matrix. The reduced-dimensional attention model is used to train the labeled data. The model parameters after training are less than 1M in size.

[0157] The generated LoRa adjusted low-rank matrix is ​​merged with the original Whisper model. The final bare model is 1.6G in size and the memory usage does not exceed 30G.

[0158] This application has trained the annotation of 1,000 voice ASR data in the seat scene data of the cultural and tourism industry, and cut each voice into 30 seconds for annotation training. The trained model verifies the actual data. See Table 1 below for details. Table 1 shows the spatial and temporal resources and accuracy of model training with different training quantities. The word error rate is all wrong words (including punctuation marks) / total number of words (including punctuation marks).

[0159] Table 1 Training effect convergence data statistics

[0160]

[0161] In addition, you can refer to Figure 6 , which shows a schematic diagram of the convergence curve of the amount of labeled data and the accuracy provided by an embodiment of the present application. Figure 6 The convergence curve of the amount of labeled data and the accuracy is shown, and the convergence limit is reached from 229 to 250. This part of the invention fully proves the training effect. The accuracy of the test data is increased from 64% to 86%, which means that the accuracy of the speech recognition model of this application has reached the requirement of replacing manual work.

[0162] In an optional embodiment of the present application, the speech recognition model includes an initial recognition model and a target recognition model, the initial recognition model includes at least one linear layer and an activation function layer, the target recognition model includes a fusion layer, and step S104 further includes the following sub-steps:

[0163] S61: Acquire initial floating-point precision data of the low-rank weight matrix in the initial recognition model, and adjust the initial floating-point precision data to obtain target floating-point precision data;

[0164] S62: Obtain the linear layer and the activation function layer in the initial recognition model;

[0165] S63: Fusing the linear layer and the activation function layer to obtain the fused layer;

[0166] S64: constructing the target recognition model according to the target floating-point precision data and the fusion layer;

[0167] S65: Obtaining the audio duration of at least one of the target audio data respectively;

[0168] S66: Sort at least one of the target audio data in descending order according to the audio duration to obtain an initial audio data sequence;

[0169] S67: Fill the target audio data in the audio data sequence with preset filling data to obtain a target audio data sequence;

[0170] S68: Input the target audio data in the target audio data sequence into the target recognition model for parallel recognition, and obtain at least one initial text sequence output by the target recognition model.

[0171] In an embodiment of the present application, initial floating-point precision data of a low-rank weight matrix in an initial recognition model can be obtained, the initial floating-point precision data can be adjusted to obtain target floating-point precision data, the linear layer and the activation function layer in the initial recognition model can be obtained, the linear layer and the activation function layer can be fused to obtain a fusion layer, a target recognition model can be constructed based on the target floating-point precision data and the fusion layer, the audio duration of at least one target audio data can be obtained respectively, at least one target audio data can be sorted in descending order according to the audio duration to obtain an initial audio data sequence, the target audio data in the audio data sequence can be filled with preset filling data to obtain a target audio data sequence, the target audio data in the target audio data sequence can be input into the target recognition model for parallel recognition, and at least one initial text sequence output by the target recognition model can be obtained.

[0172] In the specific implementation, whisper can be accelerated by quantization and parallelism. (1) Transformer can reduce the size of GPU memory occupied by the model during runtime by modifying the progress of weight parameters. (2) Through layer fusion (LayerFusion): multiple Transformer layers are merged into fewer calculation steps to reduce memory access and accelerate calculations. Two linear layers plus activation functions are used to fuse linear transformations and activation functions together, and two linear transformations and activation functions are completed directly in one calculation. (3) Batch reordering (Batch Reordering): Dynamic memory management: Dynamically adjust memory allocation according to the request size, and use a cache allocator to optimize memory usage. For example, the input is a sequence list, and each sequence has different lengths. It can be sorted according to the sequence length, with the longer ones in front, and then the batches are reordered, and the reordered sequences are converted into tensors of the same length (Tensor), that is, padding (Padding). Dynamically adjust the order of input batches to ensure that parallel processing capabilities are fully utilized during execution, especially when reasoning on inputs of different lengths. At the same time, it is necessary to support parallel execution of multiple GPU cards.

[0173] In the specific implementation, 8int can be used to convert whisper large2 from float16 to int8 and then process in parallel. Using the communication industry and cultural and tourism industry sample test data, a total of 500, 100-500 seconds of agent conversations can be tested. Refer to Table 2, which is a comparison of the recognition result data of the speech recognition model of this application and the original Whisper model.

[0174] Table 2 Comparison of recognition results between the speech recognition model and the original Whisper model

[0175] Model Accuracy Average recording time Identify time GPU Memory Original Whisper Model 16-bit floating point 225s 77.8s 11.3G This application speech recognition model 16-bit floating point 225 s 19.5s 6.8G

[0176] Reference Figure 7, which shows a schematic diagram of the speech recognition model recognition result data provided by an embodiment of the present application, with reference to Figure 8 , which shows a schematic diagram of the original Whisper model recognition result data provided by an embodiment of the present application. Figure 7 and Figure 8 It takes more than 9 seconds to load the speech recognition model of this application, and more than 19 seconds to load the original Whisper model. The speech recognition model of this application takes 6.2 seconds to recognize, and the original Whisper model takes 16 seconds to recognize, which is about 3 times faster.

[0177] Reference Fig. 9 , shows a schematic diagram of the structure of a speech recognition device provided by an embodiment of the present application, the device comprising:

[0178] The initial audio acquisition module 901 is used to obtain the audio to be recognized in response to the user's speech recognition request, and to sample the audio to be recognized to obtain initial audio data;

[0179] A human voice audio acquisition module 902 is used to determine frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain human voice audio data;

[0180] A human voice audio splitting module 903, used to split the human voice audio data to obtain at least one target audio data;

[0181] A human voice audio recognition module 904, configured to input at least one of the target audio data into a pre-trained speech recognition model for parallel recognition, and obtain at least one initial text sequence output by the speech recognition model;

[0182] A time determination module 905 is used to determine the start and end times corresponding to the words in the initial text sequence according to the human voice audio data;

[0183] A target text acquisition module 906 is used to sort the words in the initial text sequence according to the start and end time to obtain a target text sequence;

[0184] The target text display module 907 is used to use the target text sequence as a speech recognition result and display the speech recognition result to the user.

[0185] Optionally, the human voice audio acquisition module 902 includes:

[0186] A candidate audio data acquisition module, used to determine the frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain candidate audio data;

[0187] The human voice audio data acquisition submodule is used to determine the power spectrum density data of the candidate audio data, and process the candidate audio data according to the power spectrum density data to obtain the human voice audio data.

[0188] Optionally, the candidate audio data acquisition module includes:

[0189] A frequency power spectrum acquisition module, used to determine the linear frequency data and power spectrum data of the initial audio data;

[0190] A Mel frequency acquisition module, used for converting the linear frequency data into Mel frequency according to a preset frequency conversion formula;

[0191] A frequency response acquisition module, used to acquire a preset filter set, and determine the frequency response data of any filter in the filter set;

[0192] A spectrum feature acquisition module, used to determine the spectrum feature data according to the frequency response data and the power spectrum data;

[0193] The spectrum processing module is used to process the initial audio data according to the spectrum feature data to obtain the candidate audio data.

[0194] Optionally, the human voice audio data acquisition submodule includes:

[0195] A frequency domain audio acquisition module, used for performing Fourier transform on the candidate audio data to obtain frequency domain audio data;

[0196] A power spectrum density acquisition module, used to determine the power spectrum density data of the candidate audio data according to the frequency domain audio data;

[0197] A power spectrum density processing module is used to process the candidate audio data according to the power spectrum density data to obtain the human voice audio data.

[0198] Optionally, the time determination module 905 includes:

[0199] An attention weight acquisition module, used to determine attention weight data between any word in the initial text sequence and an audio frame in the audio frame sequence according to the words in the initial text sequence and the audio frame sequence;

[0200] An attention weight matrix acquisition module is used to construct an attention weight matrix according to the attention weight data; wherein any row data in the attention weight matrix corresponds to any word in the initial text sequence, and any column data in the attention weight matrix corresponds to the audio frame sequence;

[0201] A target audio frame acquisition module, configured to determine, for any word in the initial text sequence, a target audio frame corresponding to the word according to the attention weight matrix;

[0202] A target audio frame start and end time acquisition module, used to obtain the sampling rate of the human voice audio data, and determine the start and end time of the target audio frame according to the position of the target audio frame in the attention weight matrix and the sampling rate;

[0203] The word start and end time acquisition module is used to use the start and end time of the target audio frame as the start and end time corresponding to the word.

[0204] Optionally, the speech recognition model is trained in the following manner:

[0205] Acquire training samples and verification samples; the training samples include training voice data and training text data; the verification samples include verification voice data and verification text data;

[0206] Using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using the training samples to train the speech recognition model to obtain a speech recognition model to be verified;

[0207] Inputting the verification voice data into the speech recognition model to be verified to obtain sample text data output by the speech recognition model to be verified;

[0208] Calculate the loss function corresponding to the speech recognition model according to the sample text data and the verification text data;

[0209] Obtaining an initial weight matrix of the speech recognition model to be verified;

[0210] Decomposing the initial weight matrix to obtain a low-rank weight matrix of the speech recognition model to be verified;

[0211] The loss function corresponding to the speech recognition model is used to adjust the low-rank weight matrix of the speech recognition model to be verified, and the steps of using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using training samples to train the speech recognition model are re-executed until a preset training stop condition is reached to obtain a trained speech recognition model.

[0212] Optionally, the speech recognition model includes an initial recognition model and a target recognition model, the initial recognition model includes at least one linear layer and an activation function layer, the target recognition model includes a fusion layer, and the human voice audio recognition module 904 includes:

[0213] A target floating-point precision data acquisition module is used to acquire initial floating-point precision data of the low-rank weight matrix in the initial recognition model, and adjust the initial floating-point precision data to obtain target floating-point precision data;

[0214] A layer acquisition module, used to acquire the linear layer and the activation function layer in the initial recognition model;

[0215] A fusion module, used for fusing the linear layer and the activation function layer to obtain the fusion layer;

[0216] A construction module, used to construct the target recognition model according to the target floating-point precision data and the fusion layer;

[0217] An audio duration acquisition module, used to respectively acquire the audio duration of at least one of the target audio data;

[0218] A descending sorting module, used to sort at least one of the target audio data in descending order according to the audio duration to obtain an initial audio data sequence;

[0219] A target audio data sequence module, used to fill the target audio data in the audio data sequence with preset filling data to obtain a target audio data sequence;

[0220] The parallel recognition module is used to input the target audio data in the target audio data sequence into the target recognition model for parallel recognition to obtain at least one initial text sequence output by the target recognition model.

[0221] An embodiment of the present application also provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the speech recognition method as described above is implemented.

[0222] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the speech recognition method as described above is implemented.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0224] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0225] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0226] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0227] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0228] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0229] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0230] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0231] The digital Bluetooth key parameter calibration method and device provided above are introduced in detail. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​this application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A speech recognition method, characterized in that: The method comprises: In response to a user's speech recognition request, obtaining audio to be recognized, and sampling the audio to be recognized to obtain initial audio data; Determine frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain human voice audio data; Splitting the human voice audio data to obtain at least one target audio data; Inputting at least one of the target audio data into a pre-trained speech recognition model for parallel recognition to obtain at least one initial text sequence output by the speech recognition model; Determine the start and end times corresponding to the words in the initial text sequence according to the human voice audio data; Sort the words in the initial text sequence according to the start and end time to obtain a target text sequence; The target text sequence is used as a speech recognition result, and the speech recognition result is displayed to the user.

2. The method according to claim 1, characterized in that: The determining of the frequency spectrum feature data of the initial audio data, and processing the initial audio data according to the frequency spectrum feature data to obtain human voice audio data, comprises: Determine the frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain candidate audio data; Power spectrum density data of the candidate audio data is determined, and the candidate audio data is processed according to the power spectrum density data to obtain the human voice audio data.

3. The method according to claim 2, characterized in that The determining the frequency spectrum feature data of the initial audio data, and processing the initial audio data according to the frequency spectrum feature data to obtain candidate audio data, includes: Determining linear frequency data and power spectrum data of the initial audio data; Convert the linear frequency data into Mel frequency according to a preset frequency conversion formula; Obtain a preset filter set, and determine frequency response data of any filter in the filter set; Determine the frequency spectrum characteristic data according to the frequency response data and the power spectrum data; The initial audio data is processed according to the frequency spectrum feature data to obtain the candidate audio data.

4. The method according to claim 2, characterized in that: The determining the power spectrum density data of the candidate audio data, and processing the candidate audio data according to the power spectrum density data to obtain the human voice audio data, comprises: Performing Fourier transform on the candidate audio data to obtain frequency domain audio data; Determine power spectrum density data of the candidate audio data according to the frequency domain audio data; The candidate audio data is processed according to the power spectrum density data to obtain the human voice audio data.

5. The method according to claim 1, characterized in that The human voice audio data includes an audio frame sequence, and determining the start and end times corresponding to the words in the initial text sequence according to the human voice audio data includes: Determine, according to the words in the initial text sequence and the audio frame sequence, attention weight data between any of the words in the initial text sequence and the audio frame in the audio frame sequence; Constructing an attention weight matrix according to the attention weight data; wherein any row of data in the attention weight matrix corresponds to any word in the initial text sequence, and any column of data in the attention weight matrix corresponds to the audio frame sequence; For any of the words in the initial text sequence, determining a target audio frame corresponding to the word according to the attention weight matrix; Obtaining a sampling rate of the human voice audio data, and determining a start and end time of the target audio frame according to a position of the target audio frame in the attention weight matrix and the sampling rate; The start and end times of the target audio frame are used as the start and end times corresponding to the word.

6. The method according to claim 1, characterized in that The speech recognition model is trained in the following way: Acquire training samples and verification samples; the training samples include training voice data and training text data; the verification samples include verification voice data and verification text data; Using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using the training samples to train the speech recognition model to obtain a speech recognition model to be verified; Inputting the verification voice data into the speech recognition model to be verified to obtain sample text data output by the speech recognition model to be verified; Calculate the loss function corresponding to the speech recognition model according to the sample text data and the verification text data; Obtaining an initial weight matrix of the speech recognition model to be verified; Decomposing the initial weight matrix to obtain a low-rank weight matrix of the speech recognition model to be verified; The loss function corresponding to the speech recognition model is used to adjust the low-rank weight matrix of the speech recognition model to be verified, and the steps of using the training speech data as the input of the speech recognition model, using the training text data as the output of the speech recognition model, and using training samples to train the speech recognition model are re-executed until a preset training stop condition is reached to obtain a trained speech recognition model.

7. The method according to claim 6, characterized in that The speech recognition model includes an initial recognition model and a target recognition model, the initial recognition model includes at least one linear layer and an activation function layer, the target recognition model includes a fusion layer, and the at least one target audio data is input into a pre-trained speech recognition model for parallel recognition to obtain at least one initial text sequence output by the speech recognition model, including: Acquire initial floating-point precision data of the low-rank weight matrix in the initial recognition model, and adjust the initial floating-point precision data to obtain target floating-point precision data; Obtaining the linear layer and the activation function layer in the initial recognition model; Fusing the linear layer and the activation function layer to obtain the fused layer; Constructing the target recognition model according to the target floating-point precision data and the fusion layer; Respectively obtain the audio duration of at least one of the target audio data; Sort at least one of the target audio data in descending order according to the audio duration to obtain an initial audio data sequence; Filling the target audio data in the audio data sequence with preset filling data to obtain a target audio data sequence; The target audio data in the target audio data sequence is input into the target recognition model for parallel recognition to obtain at least one initial text sequence output by the target recognition model.

8. A speech recognition device, characterized in that: The device comprises: An initial audio acquisition module, used to respond to a user's speech recognition request, acquire the audio to be recognized, and sample the audio to be recognized to obtain initial audio data; A human voice audio acquisition module, used to determine the frequency spectrum feature data of the initial audio data, and process the initial audio data according to the frequency spectrum feature data to obtain human voice audio data; A human voice audio splitting module, used for splitting the human voice audio data to obtain at least one target audio data; A human voice audio recognition module, used for inputting at least one of the target audio data into a pre-trained speech recognition model for parallel recognition, and obtaining at least one initial text sequence output by the speech recognition model; A time determination module, used to determine the start and end times corresponding to the words in the initial text sequence according to the human voice audio data; A target text acquisition module, used for sorting the words in the initial text sequence according to the start and end time to obtain a target text sequence; The target text display module is used to use the target text sequence as a speech recognition result and display the speech recognition result to the user.

9. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the speech recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the speech recognition method according to any one of claims 1 to 7 is implemented.

Images