A method and apparatus for end-to-end speech recognition in the vertical domain

Abstract

The application provides a vertical field end-to-end speech recognition method and device, and relates to the technical field of intelligent speech, and comprises the following steps: receiving a to-be-recognized speech signal, and extracting acoustic features of the to-be-recognized speech signal; inputting the acoustic features into a trained end-to-end speech recognition model to obtain a recognition result; extracting high-level acoustic representations of the speech signal through an encoder, and obtaining context acoustic representations by fusing the context acoustic representations with a context perception matrix; mapping context phrases into fixed-dimension word embedding representations through a context encoder; extracting final word embedding representations of text corresponding to the audio through the encoder combined with a decoder, and fusing the context acoustic representations through an attention mechanism, and outputting a decoding result. The context encoder considers both word sequences and pronunciation sequences of the context phrases, which is beneficial to extracting more distinctive word embedding representations and relieving crosstalk between similar phrases caused by the order of magnitude of the context phrases.

Description

Technical Field

[0001] The present invention relates to the field of intelligent voice technology, and in particular to an end-to-end voice recognition method and device for a vertical field. Background Art

[0002] As one of the key means of human-computer interaction, automatic speech recognition (ASR) technology has been widely used in smart terminal devices such as mobile phones, speakers, large screens, and car computers. Currently, speech recognition solutions can be divided into traditional solutions and end-to-end solutions. Traditional solutions consist of independent modules such as the acoustic model, language model, pronunciation lexicon, and decoder. End-to-end solutions integrate these independent modules into a neural network model consisting of an encoder and decoder for joint optimization, directly converting the input sound signal into the corresponding text sequence. In recent years, with the development of end-to-end speech recognition technology and its superiority over traditional solutions, end-to-end speech recognition solutions have become the mainstream solution used in the industry. In general fields, end-to-end speech recognition has achieved good recognition results. However, in some vertical fields (such as medicine and finance), due to the large number of proper nouns and their low frequency in training sets, these proper nouns are often recognized as common words with similar pronunciations, resulting in poor recognition performance of ASR models for such proper nouns.

[0003] In order to improve the recognition effect of end-to-end speech recognition models on proper nouns in vertical fields, existing end-to-end recognition solutions generally dynamically incorporate contextual information into the recognition process to increase the model's ability to perceive context, and then improve the recognition effect based on the provided vertical context phrases (such as drug names in the medical field, proper nouns in the financial field, etc.). The specific approach is to first randomly extract several consecutive indefinite-length texts from the current batch of text data as the context phrases of the current batch data during model training, and use a context encoder to map each phrase into a fixed-dimensional word embedding. Then, the attention mechanism is used to calculate the attention score for the word embedding. At each step of the model output prediction, the provided context phrase list is weighted averaged according to the attention score calculated by attention, to focus on the available context information and perform targeted enhancement on the target text. In the inference stage, the recognition results are corrected and enhanced based on the provided vertical field context phrase list.

[0004] To address the problem that end-to-end speech recognition solutions have poor recognition effects in vertical fields, existing solutions generally add a context encoder to the end-to-end speech recognition model to map the provided context phrases into word embedding features of fixed dimensions. During the model decoding process, the attention mechanism is used to calculate the attention score of the current audio for each context phrase and weight the word embedding features corresponding to the context phrases based on the attention score. The weighted word embedding features are then integrated into the model's decoding process, so that the model can output the correct decoding results based on the current context information. However, on the one hand, existing schemes generally only fuse the context information output by the context encoder in the decoder, and do not fully utilize the audio acoustic representation extracted by the encoder, resulting in a low recall rate of context phrases. In addition, the context encoder in the existing scheme only models the word sequence corresponding to the context phrase. For context phrases containing polyphones, it is impossible to generate a discriminative word embedding representation, which leads to a low recall rate of such context phrases. On the other hand, when the magnitude of context phrases in the vertical field is large (1000), the attention score distribution of the end-to-end speech recognition model calculated by the attention mechanism for the context phrase list tends to be smooth, and the model's discrimination between the target context phrase and other irrelevant context phrases is weakened, resulting in the model being unable to decode the correct context phrase, which in turn leads to a decrease in the recall rate of the context phrase. Summary of the Invention

[0005] The present invention provides an end-to-end speech recognition method and device for a vertical field, aiming to solve the shortcomings existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an end-to-end speech recognition method for a vertical field, comprising the following steps:

[0007] receiving a speech signal to be recognized, and extracting acoustic features of the speech signal to be recognized;

[0008] Build an end-to-end speech recognition model;

[0009] Obtaining the acoustic features and inputting them into a trained end-to-end speech recognition model to obtain recognition results;

[0010] The end-to-end speech recognition model is composed of an encoder, a context encoder, and a decoder, and obtaining the acoustic features and inputting them into the trained end-to-end speech recognition model to obtain recognition results includes the following steps:

[0011] The encoder extracts the acoustic representation of the speech signal and obtains the annotated text representation and context phrases corresponding to the acoustic features;

[0012] Calculate the similarity matrix of acoustic representation, annotated text representation and context phrases, sort the similarity matrix in multiple dimensions, select the topk values ​​with the largest value for softmax normalization, and set the elements in other positions to zero;

[0013] The context-aware matrix is ​​obtained by weighting the word embeddings of the context phrases through the normalized similarity matrix;

[0014] The contextual acoustic representation X is obtained by fusing it with the context-aware matrix context ;

[0015] The context phrase is mapped into a fixed-dimensional word embedding representation and a phoneme embedding representation through a context encoder, and the word embedding representation and the phoneme embedding representation are concatenated to form the final word embedding representation;

[0016] The decoder extracts the final word embedding representation of the audio corresponding to the text and combines it with the context acoustic representation X context The fusion is performed through the attention mechanism and the recognition results are output.

[0017] Preferably, the acoustic representation of the speech signal extracted by the encoder is fused with the context perception matrix to obtain the context acoustic representation X context , specifically including the following steps:

[0018] Constructing the encoder through an acoustic encoding module and a first context attention module;

[0019] The acoustic features are fed into an acoustic coding module for calculation to obtain an acoustic representation of the speech signal;

[0020] Utilize the first contextual attention module to calculate the correlation between the acoustic representation and the word embedding representation of the context phrase, using the acoustic representation as the query vector Q and the word embedding representation of the context phrase as the key vector K and the value vector V;

[0021] The context perception matrix H and acoustic representation X are calculated based on the query vector Q, key vector K and value vector V;

[0022] The context-aware matrix H is fused with the acoustic representation X to obtain the context-aware acoustic representation X context .

[0023] Preferably, the context perception matrix H and the acoustic representation X are calculated according to the query vector, the key vector and the value vector as follows:

[0024] Q1=σ(X T W q +1(b q ) T )

[0025] K1=σ(C T W k +1(b k ) T )

[0026] V1=σ(C T W v +1(b v ) T )

[0027]

[0028] H=a1V1

[0029] X context =X+H or X context =[X,H]

[0030] Where X=[h1,...,h T ]∈R dxT is the output of the acoustic coding module and T is the number of frames of audio extracted by the acoustic coding module, d is the dimension of the acoustic representation of each frame; C = [c1, ..., c n ]∈R k×n is the word embedding representation generated by the context encoder, n is the number of context phrases, k is the word embedding dimension of the context phrase; σ(·) is the activation function, W q 、W k 、W v 、b q 、b k 、b v Are the relevant weights and bias parameters; a∈R T×n is the similarity matrix between the acoustic representation and the context phrase.

[0031] Preferably, mapping the context phrase into a fixed-dimensional word embedding representation and a phoneme embedding representation by a context encoder, and concatenating the word embedding representation and the phoneme embedding representation to form a final word embedding representation, specifically includes the following steps:

[0032] The context encoder is formed by a word encoding module and a phoneme encoding module;

[0033] Segment the context phrase into subwords or characters;

[0034] Input the subword or word into the word encoding module to generate a fixed-dimensional representation vector;

[0035] Convert context phrases into corresponding pronunciation sequences;

[0036] Inputting the pronunciation sequence into a phoneme encoding module to generate a phoneme representation vector of fixed dimension;

[0037] The representation vectors output by the word encoding module and the phoneme encoding module are spliced to form the final word embedding representation of the context phrase.

[0038] Preferably, the final word embedding representation of the corresponding text of the audio is extracted by the decoder, and is combined with the context acoustic representation X context The fusion is performed through the attention mechanism, and the recognition result is output, specifically including the following steps:

[0039] The decoder is composed of a decoding module and a second context attention module;

[0040] The decoder calculates the correlation between the current labeled text and the context acoustic representation through the attention mechanism;

[0041] The context acoustic representation and the embedding representation of the current labeled text are fused according to the correlation coefficient, and the decoding result is finally output;

[0042] The second context attention module calculates the correlation between the embedding representation of the current text of the audio and the word embedding representation of the context phrase.

[0043] Preferably, the end-to-end speech recognition model is constructed, and the end-to-end speech recognition model is trained, including the following steps:

[0044] The context phrase list during training is constructed;

[0045] The similarity matrix between the acoustic representation and the labeled text representation and the context phrase list is calculated, and the end-to-end speech recognition model is trained through the similarity matrix.

[0046] Preferably, the context phrase list during training is constructed, including the following steps:

[0047] For the current batch of training data, the corresponding text annotation is segmented and tagged with parts of speech;

[0048] For each piece of audio corresponding to the labeled text, it is determined whether to select consecutive characters from the current labeled text as a context phrase with a probability p1;

[0049] It is determined whether the randomly generated probability is greater than p1;

[0050] When it is greater than P1, consecutive characters are selected from the current labeled text as a context phrase, and for multiple word groups in the current labeled text, it is determined which word group is selected as a context phrase with a probability p2, and all word groups in the current labeled text are traversed;

[0051] When the randomly generated probability is greater than p2, the current word group is selected to be added to the context phrase list;

[0052] If the current phrase is a noun, the current noun is added to the context phrase list when the probability of random generation is greater than p2 / 2;

[0053] The no-bias phrase is added to the context phrase list to adapt to the situation where the current audio has no corresponding context phrase.

[0054] Preferably, the step of calculating a similarity matrix between the acoustic representation and the annotated text representation and the context phrase list comprises the following steps:

[0055] The acoustic representation and the annotated text representation are used as the query vector, and the word embedding representation corresponding to the context phrase list is used as the key vector and value vector;

[0056] The similarity matrix between the acoustic representation and the annotated text representation and the context phrase list is calculated separately through the attention mechanism.

[0057] A speech recognition device, comprising:

[0058] A first acquisition module, configured to receive a speech signal to be recognized and obtain acoustic features of the speech signal to be recognized;

[0059] The second acquisition module is connected to the first acquisition module and is used to input the acoustic features into the trained end-to-end speech recognition model to obtain a recognition result.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention considers that the context encoder considers both the word sequence and pronunciation sequence of the context phrase, which is conducive to extracting more discriminative word embedding representations. At the same time, when calculating the correlation between the acoustic representation and the text representation and the context phrase, the present invention first takes the topk and then performs softmax normalization, and sets other positions to zero. This not only eliminates the interference of irrelevant context phrases on the decoding process, but also increases the distinctiveness between similar context phrases, which is conducive to alleviating the crosstalk between similar phrases caused by the increase in the magnitude of context phrases. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of the end-to-end speech recognition model of the present invention;

[0063] Figure 2 This is a flow chart of the end-to-end speech recognition model training process of the present invention;

[0064] Figure 3 This is a diagram of the speech recognition device of the present invention. DETAILED DESCRIPTION

[0065] The following embodiments of the present invention are further described in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0066] like Figure 1-3 As shown, the present application provides an end-to-end speech recognition method, a model training method and related devices in a vertical field to improve the recall rate of existing end-to-end speech recognition models for keywords and reduce the crosstalk rate between keywords.

[0067] To solve the above technical problems, a technical solution adopted in this application is to provide an end-to-end speech recognition method for a vertical field, including:

[0068] S1: Receive a speech signal to be recognized through a first acquisition module, and extract acoustic features of the speech signal to be detected.

[0069] S3: Build an end-to-end speech recognition model.

[0070] S3: Acquire acoustic features and input them into the trained end-to-end speech recognition model to obtain recognition results.

[0071] The end-to-end speech recognition model consists of an encoder, a context encoder, and a decoder. Acquiring acoustic features and inputting them into the trained end-to-end speech recognition model to obtain recognition results includes the following steps:

[0072] S31: Extract the acoustic representation of the acoustic feature through the encoder, and obtain the annotated text representation and context phrase corresponding to the acoustic feature.

[0073] S32: Calculate the similarity matrix of the acoustic representation, the annotated text representation, and the context phrase, and perform multi-dimensional sorting on the similarity matrix. Select the topk values ​​with the largest values ​​for softmax normalization, and set the elements in other positions to zero.

[0074] S33: The context-aware matrix is ​​obtained by weighting the word embeddings of the context phrases through the normalized similarity matrix.

[0075] S34: The context-aware matrix is ​​fused with the acoustic representation to obtain the context acoustic representation X context .

[0076] S35: Map the context phrase into a fixed-dimensional word embedding representation and a phoneme embedding representation through a context encoder, and concatenate the word embedding representation and the phoneme embedding representation to form the final word embedding representation.

[0077] S36: Extract the final word embedding representation of the audio corresponding to the text through the decoder and combine it with the context acoustic representation X contextThe fusion is performed through the attention mechanism and the recognition results are output.

[0078] like Figure 1 As shown, the encoder consists of an acoustic coding module and a first contextual attention module. The acoustic coding module is used to extract high-level acoustic representations of speech signals. It can be composed of a stacked multi-layer convolutional neural network (CNN), a long short-term memory network (LSTM), a self-attention transformer layer and other neural network structures. The acoustic features are fed into the acoustic coding module for calculation to obtain a high-level acoustic representation of the speech signal. The first contextual attention module is used to calculate the correlation between the audio acoustic representation and the word embedding representation of the context phrase. The acoustic representation is used as the query vector (Q), and the word embedding representation of the context phrase is used as the key vector (K) and the value vector (V). The specific calculation process is as follows:

[0079] Q1=σ(X T W q +1(b q ) T )

[0080] K1=σ(C T W k +1(b k ) T )

[0081] V1=σ(C T W v +1(b v ) T )

[0082]

[0083] H=a1V1

[0084] X context =X+H or X context =[X,H]

[0085] Where X=[h1,...,h T ]∈R dxT is the output of the acoustic coding module and T is the number of frames of audio extracted by the acoustic coding module, d is the dimension of the acoustic representation of each frame; C = [c1, ..., c n ]∈R k×n is the word embedding representation generated by the context encoder, n is the number of context phrases, k is the word embedding dimension of the context phrase; σ(·) is the activation function, W q 、Wk , W v , b q , b k , b v are the relevant weights and bias parameters; a e R T×n is the similarity matrix between the acoustic representation and the context phrase, which needs to be sorted along the n dimension before softmax normalization, only the topk values with larger values are processed by softmax normalization, and other positions are set to zero, then the word embedding of the context phrase is weighted and summed according to the similarity matrix to obtain the context-aware matrix, finally the context-aware matrix H is fused with the acoustic representation X, the specific fusion method can be realized by matrix addition or splicing, and finally the acoustic representation X fused by the context-aware fusion is obtained context .

[0086] The context encoder is composed of a word encoding module and a phoneme encoding module. The word encoding module is used to map the context phrase into a fixed-dimensional word embedding representation. The context phrase is first segmented into subwords or words, and then input into the word encoding module to generate a fixed-dimensional representation vector. The word encoding module can be composed of neural network structures such as stacked unidirectional long short-term memory network, bidirectional long short-term memory network, and self-attention layer. The phoneme encoding module is used to map the context phrase into a fixed-dimensional phoneme embedding representation. The context phrase is first converted into a corresponding pronunciation sequence (Chinese can be initials and finals, and English can be phonemes), and then input into the phoneme encoding module to generate a fixed-dimensional phoneme representation vector. The phoneme encoding module can be composed of neural network structures such as stacked unidirectional long short-term memory network, bidirectional long short-term memory network, and self-attention layer. Finally, the representation vectors output by the word encoding module and the phoneme encoding module are spliced to form the final word embedding representation of the context phrase.

[0087] The decoder includes a decoding module and a second context attention module. The decoding module is used to extract the word embedding representation of the text corresponding to the audio and fuse it with the context acoustic representation X context through the attention mechanism. The neural network structure can be composed of stacked multi-layer convolutional neural network (CNN), long short-term memory (LSTM), and self-attention transformer layer. The specific method is to take the word embedding representation of the annotated text as the query vector, and the context acoustic representation X contextAs the key vector and value vector, the attention mechanism is used to calculate the correlation between the current annotated text and the context acoustic representation, and the context acoustic representation is fused with the embedding representation of the current annotated text according to the correlation coefficient, and finally the decoding result is output; the second context attention module is used to calculate the correlation between the embedding representation of the current audio text and the word embedding representation of the context phrase, and the embedding representation of the current annotated text is used as the query vector (Q), and the word embedding representation of the context phrase is used as the key vector (K) and the value vector (V). The specific calculation process is the same as the first context attention module.

[0088] In order to solve the above technical problems, another technical solution adopted by this application is: Figure 2 As shown, a model training method is provided, including:

[0089] 1. Construct a list of context phrases for training. The specific steps are as follows: First, for the current batch of training data, perform word segmentation and part-of-speech tagging on the corresponding text annotations. Then, for the annotated text corresponding to each audio, use probability p1 to determine whether to select consecutive characters from the current annotated text as context phrases. If the randomly generated probability is greater than p1, select consecutive characters from the current annotated text as context phrases. For multiple phrases in the current annotated text, use probability p2 to determine which phrase to select as the context phrase. Traverse all phrases in the current annotated text. If the randomly generated probability is greater than p2, select the current phrase to be added to the context phrase list. If the current phrase is a noun, then when the randomly generated probability is greater than p2 / 2, the current noun will be added to the context phrase list. This can increase the probability of nouns being selected, which is consistent with actual usage scenarios. Matching; finally, the no-bias phrase is added to the context phrase list to adapt to the situation where the current audio has no corresponding context phrase; 2. Calculate the similarity matrix between the acoustic representation and the annotated text representation and the context phrase list. The specific steps are: first, use the acoustic representation and the annotated text representation as the query vector, and the word embedding representation corresponding to the context phrase list as the key vector and the value vector. The similarity matrix between the acoustic representation and the annotated text representation and the context phrase list is calculated respectively through the attention mechanism. When performing softmax on the similarity matrix, only the topk elements with larger values ​​are normalized, and the correlation parameter is normalized to between 0 and 1. The elements in other positions are set to zero. This can reduce the interference of irrelevant context phrases on the one hand, and increase the distinguishability between similar context phrases on the other hand, which is conducive to the correct recall of the corresponding context phrases.

[0090] In order to solve the above technical problems, another technical solution adopted by this application is: Figure 3As shown, a speech recognition device is provided, including: a first acquisition module for receiving a speech signal to be recognized and obtaining the acoustic features of the speech signal to be recognized; a second acquisition module, connected to the first acquisition module, for inputting the acoustic features into a trained end-to-end speech recognition model to obtain a recognition result.

[0091] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method in any of the above embodiments.

[0092] In order to solve the above technical problems, another technical solution adopted in the present application is: providing a storage device storing program instructions that can be executed by a processor, and the program instructions are used to implement the method in any of the above embodiments.

[0093] The above-described embodiments are only preferred specific implementation methods of the present invention, and the protection scope of the present invention is not limited thereto. Any simple changes or equivalent replacements of the technical solutions that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention fall within the protection scope of the present invention.

Claims

1. A vertical field end-to-end speech recognition method, characterized in that: The steps include: receiving a speech signal to be recognized, and extracting acoustic features of the speech signal to be recognized; Build an end-to-end speech recognition model; Acquiring the acoustic features and inputting them into an end-to-end speech recognition model to obtain recognition results; The end-to-end speech recognition model is composed of an encoder, a context encoder, and a decoder, and obtaining the acoustic features and inputting them into the trained end-to-end speech recognition model to obtain recognition results includes the following steps: The encoder extracts the acoustic representation of the acoustic features and obtains the annotated text representation and context phrases corresponding to the acoustic features; Calculate the similarity matrix of acoustic representation, annotated text representation and context phrases, sort the similarity matrix in multiple dimensions, select the topk values ​​with the largest value for softmax normalization, and set the elements in other positions to zero; The context-aware matrix is ​​obtained by weighting the word embeddings of the context phrases through the normalized similarity matrix; The context perception matrix is ​​fused with the acoustic representation to obtain the context acoustic representation ; The context phrase is mapped into a fixed-dimensional word embedding representation and a phoneme embedding representation through a context encoder, and the word embedding representation and the phoneme embedding representation are concatenated to form the final word embedding representation; The decoder extracts the final word embedding representation of the audio corresponding to the text and combines it with the context acoustic representation The fusion is performed through the attention mechanism and the recognition results are output.

2. The end-to-end speech recognition method for vertical fields according to claim 1, characterized in that: The context perception matrix is ​​fused with the acoustic representation to obtain the context acoustic representation , specifically including the following steps: Constructing the encoder through an acoustic encoding module and a first context attention module; The acoustic features are fed into an acoustic coding module for calculation to obtain an acoustic representation of the speech signal; Utilize the first contextual attention module to calculate the correlation between the acoustic representation and the word embedding representation of the context phrase, using the acoustic representation as the query vector Q and the word embedding representation of the context phrase as the key vector K and the value vector V; The context perception matrix H and acoustic representation X are calculated based on the query vector Q, key vector K and value vector V; The context-aware matrix H is fused with the acoustic representation X to obtain the context-aware acoustic representation .

3. The end-to-end speech recognition method for vertical fields according to claim 2, characterized in that: The formula for calculating the context perception matrix H and the acoustic representation X based on the query vector, key vector and value vector is as follows: in, is the output of the acoustic coding module, T is the number of frames of audio extracted by the acoustic coding module, and d is the dimension of the acoustic representation of each frame; is the word embedding representation generated by the context encoder, n is the number of context phrases, and k is the word embedding dimension of the context phrase; is the activation function, 、 、 、 、 、 are the relevant weight and bias parameters; is the similarity matrix between the acoustic representation and the context phrase.

4. The end-to-end speech recognition method for vertical fields according to claim 1, characterized in that: The context phrase is mapped into a fixed-dimensional word embedding representation and a phoneme embedding representation by a context encoder, and the word embedding representation and the phoneme embedding representation are concatenated to form a final word embedding representation, specifically including the following steps: The context encoder is formed by a word encoding module and a phoneme encoding module; Segment the context phrase into subwords or characters; Input the subword or word into the word encoding module to generate a fixed-dimensional representation vector; Convert context phrases into corresponding pronunciation sequences; Inputting the pronunciation sequence into a phoneme encoding module to generate a phoneme representation vector of fixed dimension; The representation vectors output by the character encoding module and the phoneme encoding module are concatenated to form the final word embedding representation of the context phrase.

5. The end-to-end speech recognition method for vertical fields according to claim 1, characterized in that: The decoder extracts the final word embedding representation of the audio corresponding to the text and combines it with the context acoustic representation The recognition results are output through the attention mechanism, which includes the following steps: The decoder is constructed by a decoding module and a second context attention module; The decoder calculates the correlation between the current annotation text and the context acoustic representation through the attention mechanism; The contextual acoustic representation is fused with the embedded representation of the current annotated text according to the correlation coefficient, and the decoding result is finally output; The second context attention module calculates the correlation between the embedding representation of the current audio text and the word embedding representation of the context phrase.

6. The end-to-end speech recognition method for a vertical field according to claim 2, characterized in that: The step of building an end-to-end speech recognition model and training the end-to-end speech recognition model includes the following steps: Build a list of context phrases for training; Calculate the similarity matrix between the acoustic representation and the annotated text representation and the context phrase list, and use the similarity matrix to train the end-to-end speech recognition model.

7. The end-to-end speech recognition method for vertical fields according to claim 6, characterized in that: The step of constructing a context phrase list for training includes the following steps: For the current batch of training data, perform word segmentation and part-of-speech tagging on the corresponding text annotations; For each annotated text corresponding to an audio, determine whether to select consecutive characters from the current annotated text as a context phrase with probability p1, and construct a context phrase list; Determine whether the probability of random generation is greater than probability p1; When the probability is greater than p1, consecutive characters are selected from the current annotated text as context phrases. For multiple phrases in the current annotated text, the probability p2 is used to determine which phrase to select as the context phrase, and all phrases in the current annotated text are traversed. When the probability of random generation is greater than p2, the current phrase is selected and added to the context phrase list; If the current phrase is a noun, the current noun is added to the context phrase list when the probability of random generation is greater than p2 / 2; The no-bias phrase is added to the context phrase list to adapt to the situation where the current audio has no corresponding context phrase.

8. The end-to-end speech recognition method for vertical fields according to claim 6, characterized in that: The calculating of the similarity matrix between the acoustic representation, the annotated text representation and the context phrase list comprises the following steps: The acoustic representation and the annotated text representation are used as the query vector, and the word embedding representation corresponding to the context phrase list is used as the key vector and value vector; The similarity matrix between the acoustic representation and the annotated text representation and the context phrase list is calculated separately through the attention mechanism.

9. A speech recognition device, characterized in that: include: A first acquisition module, configured to receive a speech signal to be recognized and obtain acoustic features of the speech signal to be recognized; a second acquisition module, connected to the first acquisition module, for constructing an end-to-end speech recognition model, and for inputting the acoustic features into the trained end-to-end speech recognition model to obtain a recognition result; The end-to-end speech recognition model is composed of an encoder, a context encoder, and a decoder, and obtaining the acoustic features and inputting them into the trained end-to-end speech recognition model to obtain recognition results includes the following steps: The encoder extracts the acoustic representation of the acoustic features and obtains the annotated text representation and context phrases corresponding to the acoustic features; Calculate the similarity matrix of acoustic representation, annotated text representation and context phrases, sort the similarity matrix in multiple dimensions, select the topk values ​​with the largest value for softmax normalization, and set the elements in other positions to zero; The context-aware matrix is ​​obtained by weighting the word embeddings of the context phrases through the normalized similarity matrix; The context perception matrix is ​​fused with the acoustic representation to obtain the context acoustic representation ; The context phrase is mapped into a fixed-dimensional word embedding representation and a phoneme embedding representation through a context encoder, and the word embedding representation and the phoneme embedding representation are concatenated to form the final word embedding representation; The decoder extracts the final word embedding representation of the audio corresponding to the text and combines it with the context acoustic representation The fusion is performed through the attention mechanism and the recognition results are output.

Images