Speech recognition method and system based on domain classification and hot word prefix tree beam search

Abstract

The present invention belongs to the field of speech recognition technology, and particularly relates to a speech recognition method and system based on domain classification and hot word prefix tree beam search, which uses a hot word list in a known domain to construct a hot word prefix tree of a corresponding domain type, and trains a corresponding hot word language model; for target speech data, a trained end-to-end model is used for one decoding, and a trained domain classification model is used to obtain the domain type to which the target speech data belongs; the target speech data is decoded twice using the end-to-end model, and a beam search algorithm is used and the trained hot word language model is used to output the text corresponding to the target speech data, wherein, in each time step search of the beam search algorithm, the hot word prefix tree of the domain type to which the target speech data belongs is combined to enhance the hot word probability. The present invention uses a domain hot word prefix tree to appropriately enhance the hot words in the speech data, improve the speech recognition effect, and facilitate application in actual scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech recognition, and in particular relates to a speech recognition method and system based on domain classification and hot word prefix tree beam search. Background Art

[0002] With the development of artificial intelligence technology, speech recognition technology has been widely used in various industries, and speech recognition technology based on traditional modeling methods is relatively mature. In recent years, with the development of end-to-end speech recognition, speech recognition systems have gradually overcome the traditional modular design and independence assumptions, and have the advantages of joint optimization and easy deployment. When the front-end to end speech recognition model mainly relies on deep learning technology and uses an integrated neural network for modeling, in the actual application of speech recognition, the system has a good recognition effect on general fields, but when it comes to user-specific information or proper nouns in specific fields, the effect is always unsatisfactory. However, these words often contain important information that users are concerned about, so correctly identifying these rare words is crucial to the user experience. , Summary of the invention

[0003] To this end, the present invention provides a speech recognition method and system based on domain classification and hot word prefix tree cluster search, which uses the domain hot word prefix tree to appropriately enhance the hot words in the speech data, improve the speech recognition effect, and facilitate application in actual scenarios.

[0004] According to the design scheme provided by the present invention, a speech recognition method based on field classification and hot word prefix tree beam search is provided, which includes the following contents:

[0005] Use the hot word list in the known field to build a hot word prefix tree of the corresponding field type, and train the corresponding hot word language model;

[0006] For the target speech data, use the trained end-to-end model to perform decoding once, and use the trained domain classification model to obtain the domain type to which the target speech data belongs;

[0007] The target speech data is decoded twice using an end-to-end model, and the text corresponding to the target speech data is output using a beam search algorithm and a trained hot word language model. In each time step search of the beam search algorithm, the hot word prefix tree of the field type to which the target speech data belongs is combined to enhance the hot word probability.

[0008] As a speech recognition method based on field classification and hot word prefix tree cluster search in the present invention, further, in constructing the hot word prefix tree, the known hot word list data in each field is stored as a hot word prefix tree structure of the corresponding field type, in which the root node does not store character information, and each node except the root node stores a character information, and the character string formed by connecting all the characters on the path from the root node to other target nodes is the hot word corresponding to the target node.

[0009] As a speech recognition method based on domain classification and hot word prefix tree beam search in the present invention, further, in an end-to-end model, a Transformer model is used to convert the target speech data sequence into a text sequence, wherein the Transformer model adopts an encoder-decoder neural network structure based on a self-attention mechanism, and the encoder and decoder in the neural network structure are respectively composed of a stack of several identical self-attention modules, the input of the encoder is a speech data feature sequence, and the input of the decoder is a text label embedding vector sequence.

[0010] As a speech recognition method based on domain classification and hot word prefix tree beam search in the present invention, further, in a decoding using an end-to-end model, the speech data is first decoded using a greedy algorithm, and the word with the highest probability is selected as the sequence element in the predicted text at each time step, thereby obtaining a predicted text sequence of a complete path; the predicted text sequence is input into a trained domain classification model, and the domain classification model is used to obtain the domain type to which the speech data belongs.

[0011] As a speech recognition method based on domain classification and hot word prefix tree beam search of the present invention, further, the domain classification model adopts a TextCNN network structure, which includes: an input layer for converting each character in a text sequence into a corresponding word vector, a convolutional layer for extracting text features of the word vector, a pooling layer for performing a pooling operation on the extracted text feature vector to obtain a global feature vector, and a fully connected layer for classifying the global feature vector through a softmax function to obtain the corresponding domain type.

[0012] As a speech recognition method based on domain classification and hot word prefix tree beam search of the present invention, further, in the secondary decoding using the end-to-end model, first, the beam width hyperparameter B is set, and the beam search algorithm is used to select B candidate characters with the highest current probability in each time step search, and the prefix tree state and node depth corresponding to the beam candidate character in the current time step are determined based on the candidate character results predicted by each beam in the previous time step and the relationship between the beam candidate characters in the current time step; then, the hot word score is adjusted starting from the child node of the root node according to the node depth, so that the hot word obtains an additional score reward before reaching the leaf node; then, the prefix tree is pruned, and the pruned prefix tree is used to obtain the text sequence corresponding to the speech data through the path between the nodes.

[0013] As a speech recognition method based on field classification and hot word prefix tree beam search of the present invention, further, in determining the prefix tree state and node depth corresponding to the beam candidate character in the current time step, if the beam candidate character in the current time step is a child node of the prefix tree of the candidate character predicted in the previous time step, the prefix tree state of the candidate character in the current time step is set to the candidate character node in the current time step, and the node depth of the current time step is increased on the basis of the prefix tree of the candidate character predicted in the previous time step; if the beam candidate character in the current time step is not a child node of the prefix tree of the candidate character predicted in the previous time step, and is not a child node of the root node of the prefix tree, the prefix tree state of the candidate character in the current time step is set to the root node, and the node depth is set to 0; if the beam candidate character in the current time step is a child node of the root node, the prefix tree state of the candidate character in the current time step is set to the child node, and the node depth is set to 1.

[0014] As a speech recognition method based on field classification and hot word prefix tree beam search of the present invention, further, the process of adjusting the hot word score according to the node depth is expressed as follows: Among them, ω k Indicates the bias score of the hot word enhancement, the b-th beam candidate character at the current time step t The corresponding prefix tree state It is used when the depth is greater than or equal to 2, i is the label of the candidate character; α k represents the pre-enhancement score of the hot word. Used when depth equals 1; β k Indicates the bonus score for short hot word length, Used when the depth is less than or equal to 3 and reaches a leaf node, n t-1,b Indicates the prefix tree state corresponding to the prediction result of the b-th beam at time step t-1.

[0015] As the speech recognition method based on field classification and hot word prefix tree beam search of the present invention, further, the final output text corresponding to the target speech data is expressed as:

[0016] Where x represents a given set of acoustic sequences (x1, x2, ..., x m ), y represents the E2E model selection that can maximize the posterior probability P E2E The text sequence (y1,y2,...,y n ),y * represents the total sequence score, P E2E (y|x) represents the score of the end-to-end model, P C (y) represents the score of the language model, λ C S is a hyperparameter that represents the influence of the language model on the overall model score. t,b (y|H) represents the prefix tree score, H represents the hot word prefix string, λ K is the hyperparameter for the prefix tree score.

[0017] Furthermore, the present invention also provides a speech recognition system based on domain classification and hot word prefix tree beam search, comprising: a prefix tree construction module, a first decoding module and a second decoding module, wherein:

[0018] The prefix tree construction module is used to construct a hot word prefix tree of the corresponding field type using a hot word list in a known field, and train the corresponding hot word language model;

[0019] A first decoding module is used to decode the target speech data once using the trained end-to-end model, and to obtain the domain type to which the target speech data belongs using the trained domain classification model;

[0020] The second decoding module is used to perform secondary decoding on the target voice data using an end-to-end model, and use a beam search algorithm and a trained hot word language model to output the text corresponding to the target voice data, wherein the hot word probability is enhanced in each time step search of the beam search algorithm by combining the hot word prefix tree of the field type to which the target voice data belongs.

[0021] Beneficial effects of the present invention:

[0022] The present invention uses a hot word prefix tree before cluster search pruning, uses a two-side decoding method, uses a greedy algorithm to determine the voice content field in the first decoding, uses a second algorithm to use a cluster search, uses hot word score pre-enhancement and length reward, enhances short hot words, and improves the accuracy of speech recognition. In the end-to-end speech recognition decoding reasoning, the domain hot word prefix tree is used to enhance the hot word. By constructing the hot word list into a hot word prefix tree, the hot word prefix tree is used to improve the score before the cluster search pruning, effectively ensuring that the hot word string is retained in the cluster to avoid being pruned; and further, through hot word pre-enhancement and length reward, the recognition effect of short hot words in Chinese is effectively improved; when performing a cluster search, if the hot word prefix tree is always in an active state, when recognizing speech without hot words, excessive hot word bias will occur, which will affect the general field recognition performance. The domain classification model is used to determine whether it is necessary to use the hot word prefix tree for enhancement, reduce the impact of hot words on the general field recognition effect, and facilitate application in actual scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the speech recognition process based on field classification and hot word prefix tree beam search in the embodiment;

[0024] Figure 2 This is a schematic diagram of the prefix tree structure in the embodiment;

[0025] Figure 3 The figure is a schematic diagram of the model structure in the embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.

[0027] In view of the unsatisfactory recognition effect of the end-to-end speech recognition system on hot words, the present invention embodiment, see Figure 1 As shown, a speech recognition method based on domain classification and hot word prefix tree beam search is provided, comprising:

[0028] S101, using the hot word list in the known field to construct a hot word prefix tree of the corresponding field type, and train the corresponding hot word language model;

[0029] S102, decoding the target speech data once using the trained end-to-end model, and obtaining the domain type to which the target speech data belongs using the trained domain classification model;

[0030] S103. Use an end-to-end model to perform secondary decoding on the target speech data, and use the beam search algorithm and the trained hotword language model to output the text corresponding to the target speech data. Among them, in the search at each time step of the beam search algorithm, the hotword probability is enhanced by combining the hotword prefix tree of the domain type to which the target speech data belongs.

[0031] In the embodiments of this case, the hotword list is constructed as a prefix tree for score enhancement during decoding inference; in the first pass of decoding, the greedy algorithm is used by the end-to-end model to quickly recognize the speech signal, and then the obtained predicted text is input into the domain classification model to determine which domain the speech content belongs to; then the beam search algorithm is used for the second pass of decoding, and the hotword neural language model and the hotword prefix tree of the corresponding domain are used for score improvement before pruning, so that the paths containing hotwords are retained in the beam, effectively improving the recognition rate of hotwords. At the same time, aiming at the poor recognition effect of short Chinese hotwords, pre-enhancement and length reward are performed on the hotwords.

[0032] As a preferred embodiment, further, in constructing the hotword prefix tree, the known hotword list data in each domain is stored as a hotword prefix tree tree structure of the corresponding domain type. In this tree structure, the root node does not store character information, and each node except the root node stores one character information, and the string formed by connecting all the characters on the path from the root node to other target nodes is the hotword corresponding to the target node.

[0033] The prefix tree is a tree structure, mainly applied to the statistics and sorting of strings, which can minimize the comparison of invalid strings. To construct a hotword prefix tree, assume that a set of hotword lists are known in advance and the hotword lists are stored as a tree structure. The root node of the prefix tree does not store character information, and each node except the root node can only store the information of one character. Each node can contain multiple child nodes. The string formed by connecting all the characters on the path from the root node to a certain node is the hotword corresponding to the node. As Figure 2 shown, to construct a hotword prefix tree, starting from the root node, select its child node "Zheng", then select the child node "Zhou" of "Zheng", and finally select the leaf node "Shi" to get the hotword "Zhengzhou City". It can be seen from the figure that the strings represented by the descendant nodes of a certain node all have the same prefix as the node. The gray nodes represent the leaf nodes (lead node), indicating the end flag of the hotword. Among them, the depth of the root node is 0, and the longer the string obtained as the depth increases.

[0034] Furthermore, in an embodiment of the present case, a Transformer model is used as an end-to-end model to convert a target speech data sequence into a text sequence, wherein the Transformer model adopts an encoder-decoder neural network structure based on a self-attention mechanism, and the encoder and decoder in the neural network structure are respectively composed of a stack of several identical self-attention modules, the input of the encoder is a speech data feature sequence, and the input of the decoder is a text label embedding vector sequence.

[0035] The end-to-end speech recognition model can directly convert the speech sequence into a text sequence. In the embodiment of this case, the Transformer model is used as the end-to-end speech recognition model. Transformer is an encoder-decoder neural network structure based entirely on the attention mechanism, which does not rely on any loop and convolution operations. The encoder and decoder of Transformer are respectively stacked by several identical self-attention modules. The input of the encoder is a speech feature sequence, and the input of the decoder is an embedded vector sequence of text labels. Before the speech feature input is encoded, it will be downsampled through a convolutional network to reduce the amount of calculation of the subsequent neural network. Each encoder module consists of two sub-networks: a multi-head self-attention layer and a feedforward neural layer. The inputs Q, K, and V of the self-attention are all from the output of the previous module; each module normalizes the input, and residual connections are added between modules. The decoder structure is similar to the encoder structure, but there are two differences: 1. A mask operation is added to the self-attention to ensure that only the input of the current and previous moments can be seen during training; 2. A multi-head attention sublayer is added between the self-attention and the feedforward neural network, whose input Q comes from the output of the previous module, and K, V come from the last layer output of the encoder, thus forming an attention mechanism between the encoder and the decoder.

[0036] As a preferred embodiment, further, in a decoding using an end-to-end model, the speech data is first decoded using a greedy algorithm, and a predicted text sequence of a complete path is obtained by selecting the word with the highest probability as a sequence element in the predicted text at each time step; the predicted text sequence is input into a trained domain classification model, and the domain classification model is used to obtain the domain type to which the speech data belongs. Further, the domain classification model adopts a TextCNN network structure, which includes: an input layer for converting each character in the text sequence into a corresponding word vector, a convolutional layer for extracting text features of the word vector, a pooling layer for performing a pooling operation on the extracted text feature vector to obtain a global feature vector, and a fully connected layer for classifying the global feature vector through a softmax function to obtain the domain type to which it belongs.

[0037] According to the field to which the speech content belongs, the corresponding hot word language model and hot word prefix tree can be selected for hot word enhancement. The greedy algorithm can be used to quickly decode the speech signal. Although the result of the first decoding cannot correctly identify the hot word information, it has little impact on the field to which the sentence belongs. It can be directly input into the field classifier to determine the field to which the text belongs. The text classification model used is TextCNN, which has good performance in short text classification. Its network structure is simple and mainly consists of 4 layers: the first layer is the input layer. Given a text sequence of length n, each character is converted into a corresponding k-dimensional word vector. The second layer is the convolution layer, which realizes the extraction of local features of the text to obtain text features, and uses h convolution kernels of different sizes. The convolution feature value is obtained by the convolution kernel at position i:

[0038] S i =f(w·T i:i+h-1 +b),w∈R h×k

[0039] Among them, w represents the weight matrix, the dimension is h×k, T i:i+h-1 represents the sliding window from the i-th row to the i+h-1th row of the input matrix. b represents the bias function. The third layer is the pooling layer, which can reduce the feature dimension while retaining useful feature information to obtain the global feature vector:

[0040] S=max(S i )

[0041] Finally, a fully connected layer is used and the Softmax function is used for field classification.

[0042] Since the hot word data is sparse, the hot word language model can use a neural network language model to effectively alleviate the data sparsity problem. The neural network learns the vector representation of the word and maps the word to a continuous space. In the embodiment of this case, the long short-term memory (LSTM) network can be used as the hot word language model to train the hot words. The LSTM network is a special recurrent neural network (RNN). Its structure adds input gates, forget gates, and output gates on the basis of RNN, which respectively control the input, the historical information of the node at the previous moment, and the output information. A double-layer LSTM network structure is used when constructing the LSTM language model. The stacked LSTM can increase the depth of the network, thereby improving the efficiency of training.

[0043] As a preferred embodiment, further, in the secondary decoding using the end-to-end model, first, the beam width hyperparameter B is set, and the beam search algorithm is used to select the B candidate characters with the highest current probability in each time step search, and the prefix tree state and node depth corresponding to the beam candidate character in the current time step are determined based on the candidate character results predicted by each beam in the previous time step and the relationship between the beam candidate characters in the current time step; then, the hot word score is adjusted starting from the child node of the root node according to the node depth, so that the hot word obtains an additional score reward before reaching the leaf node; then, the prefix tree is pruned, and the pruned prefix tree is used to obtain the text sequence corresponding to the voice data through the path between nodes.

[0044] The greedy algorithm takes the word with the highest probability as the prediction result at each time step t, and finally obtains a complete path. Although this approach is computationally efficient, it only seeks the current optimal path and cannot guarantee that the entire path will obtain the global optimal. The beam search algorithm is an improvement on the greedy search algorithm. By setting a beam width hyperparameter B, when searching at each time step, the B candidate characters with the highest current probability are selected. The specific approach is to select the B characters with the highest conditional probability in the first time step. They will be the first characters of the B candidate output sequences, that is, find y 11 ,y 12 ,...,y 1B In the next time step, based on the B candidate y in the previous time step 1b , continue to select B candidate characters with the highest conditional probability, that is, find y 21 ,y 22 ,...,y 2B . In this way, there are B*B candidate sequences, and then the highest B results are selected according to the bar probability, and so on, until the final end character is output.

[0045] In this embodiment, hot word enhancement is performed on B*B candidate sequences at each time step. Assuming that n t-1,b represents the prediction result y of the bth beam at time step t-1 t-1,b At decoding time step t, the bth beam obtained at the previous time step will generate B candidate characters. With y t-1,b To get the current candidate character The corresponding prefix tree state Where i is the number of the candidate character. There are three cases: Yes t-1,b , then the state Set as At the same time, let the node depth be If the candidate character Not t-1,b is not a child node of the prefix tree root node, then the state Set it as root node n0 and set the node depth to 0; if the candidate character is a child node of the root node, then the state Set it as the child node, and set the node depth equal to 1. The detailed process can be shown in Algorithm 1.

[0046] After obtaining the prefix tree state and node depth corresponding to each candidate character at time step t, before pruning, it is necessary to improve the score of each candidate character, that is, by increasing the score of the corresponding child node, to avoid hot words being pruned because the posterior probability score of the acoustic model is too low. Because Chinese does not have clear word boundaries and the length of Chinese hot words is short (for example, contact names, which usually consist of two or three characters), the hot word score improvement starts from the child nodes of the root node n0, so that the hot words can also get additional score rewards before reaching the leaf nodes, avoiding short hot words from being pruned prematurely. According to The depth of the score is used to improve the score. The specific formula is as follows:

[0047]

[0048] Among them, ω k represents the bias score of hot word enhancement, Used when depth is greater than or equal to 2; α k represents the pre-enhancement score of the hot word. Used when depth equals 1; β k Indicates the bonus score for short hot word length, Used when the depth is less than or equal to 3 and reaches a leaf node. The depth is less than or equal to n t-1,b When the depth is less than , it indicates that the hot word match fails, and the score enhancement needs to be interrupted. The fallback method is adopted to subtract the previously accumulated enhanced score. The hot word prefix tree is used to improve the score of the corresponding output sequence, and the formula is obtained:

[0049]

[0050] Among them, P E2E (y|x) represents the score of the E2E model, P C (y) represents the score of the language model, λ C is a hyperparameter that indicates the influence of the language model on the overall model score. t,b (y|H) represents the score of the prefix tree, H represents the hot word prefix string, λ K is a hyperparameter of the prefix tree score.

[0051]

[0052] Furthermore, based on the above method, an embodiment of the present invention also provides a speech recognition system based on field classification and hot word prefix tree beam search, comprising: a prefix tree construction module, a first decoding module and a second decoding module, wherein:

[0053] The prefix tree construction module is used to construct a hot word prefix tree of the corresponding field type using a hot word list in a known field, and train the corresponding hot word language model;

[0054] A first decoding module is used to decode the target speech data once using the trained end-to-end model, and to obtain the domain type to which the target speech data belongs using the trained domain classification model;

[0055] The second decoding module is used to perform secondary decoding on the target speech data using an end-to-end model, and use a beam search algorithm and a trained language model to output the text corresponding to the target speech data. In each time step search of the beam search algorithm, the hot word prefix tree of the field type to which the target speech data belongs is combined to enhance the hot word probability.

[0056] In order to verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0057] See also Figure 2 As shown, a known hot word list is used to construct hot word prefix trees in different fields respectively; the speech signal is input into an end-to-end speech recognition model for decoding; the end-to-end speech recognition model is used for the first decoding, a greedy search algorithm is used for fast decoding, and a predicted text is output; the output predicted text is input into a field classification model to determine which field the speech content belongs to; the end-to-end speech recognition model is used for the second decoding, a beam search algorithm is used, and the field language model and hot word prefix tree determined by the first decoding are used in the decoding process to output the enhanced text.

[0058] The above steps are used to implement a speech recognition solution based on domain classification and hot word prefix tree beam search algorithm. In the experiment, real audio data sets and speech synthesis (TTS) data can be used for testing. The test set of AIshell-2 consists of a 5,000 sentence set. This test set is used as a general domain test set and does not contain hot word information. At the same time, a hot word test set is constructed based on the training set of the MAGICDATA Mandarin corpus. The training set is mainly recording data of mobile terminals, mainly including interactive questions and answers, music search, home control, etc. The transcribed text of the training set is screened using the Tsinghua University Chinese Lexical Analysis Package to obtain corpus about the field of music search, including rich singer names, song names, and TV series names (for example: play XXX's songs), etc., as a song test set. At the same time, TTS is used to construct a contact test set, which contains a large number of communication requests with names (for example: call XXX). The hot words in the test set appear less than 10 times in the training set.

[0059] The end-to-end speech recognition model consists of 12 encoder and 6 decoder blocks. The dimension of the multi-head attention in each encoder and decoder layer is 256, the number of heads is 4, and the dimension of the feedforward neural network is 2048. A 2D convolutional layer is used in the front end, and the RELU activation function is used with a step size of 2 and a convolution kernel of 3. The Adam optimizer with a training warm-up of 12000 steps is used, and SpecAugment is used for data augmentation. This experiment uses 80-dimensional FBank (Filter Banks) as input features, with a frame length of 25ms and a frame shift of 10ms. The Adam optimizer is used during training, and the learning rate adaptation strategy is used, with a warm-up step of 12K and a maximum learning rate of 0.001. The TextCNN model uses 1 network layer, a word vector dimension of 300, Relu as the activation function, 1_max pooling as the pooling strategy, and a dropout ratio of 0.5, a maximum text sequence length of 32, and a learning rate of 0.001. The LSTM language model uses two hidden layers, each with 1024 LSTM units, and uses the Adam gradient update strategy.

[0060] The experimental results, as shown in Table 1, show the experimental performance of the original prefix tree method and the improved prefix tree method. Experiment E0 represents the baseline system, which only uses the E2E model for decoding. Experiment E1 represents the use of the hot word prefix tree after beam search pruning, which shows good performance on both test sets. Experiment E2 represents the use of the hot word prefix tree after pruning, and the use of the language model for shallow fusion can further improve the performance. Experiments E3 and E4 are our improvements to the prefix tree hot word enhancement method, which has outperformed the performance of previous methods without using a language model. After using the language model, the word error rate was reduced by 1.3% to 4.8%, and the hot word recall rate was increased by 6% to 17%.

[0061] Table 1 Comparison of prefix tree method before and after improvement

[0062]

[0063] In order to further study the impact of various factors on system performance, ablation experiments were conducted by operating the following experimental conditions: Experiment E0 represents the baseline system of the system, using only the E2E model for decoding output. Experiment E5 represents the shallow fusion using only the language model. As shown in Table 2, the word error rate and hot word recall rate did not improve much on the two test sets; Experiment E6 represents the prefix tree method without short hot word enhancement. On the two test sets, the word error rate was improved by 4.6% to 6.4% compared with the baseline system, and the hot word recall rate was improved by about 30% compared with the baseline system; for the hot words with a string length of less than 3 that often appear in Chinese hot words, Experiment E7 represents the prefix tree method using short hot word enhancement, which has a significant improvement on the contact test set, effectively solving the problem of poor recognition of short hot words.

[0064] Table 2. Improved performance of prefix tree method

[0065]

[0066] In the process of beam search, if the hot word prefix tree is always active, it will not only reduce the decoding speed, but also cause false triggers, resulting in a decrease in general domain performance. The hot word prefix tree is constructed using song hot words to explore the impact of the prefix tree method on general domain performance. As shown in Table 3, when the domain model is not used, the performance of the general domain test is significantly reduced; after using the domain classification model, the hot word prefix tree has little effect on the performance of the general domain.

[0067] Table 3 Field classification performance test

[0068]

[0069] The experimental results show that Table 4 shows examples with and without hot word prefix tree enhancement. The speech recognition solution in this embodiment can solve most of the problems related to the difficulty in recognizing rare words such as song names and contact names, and can effectively solve the recognition errors of specific words caused by homophones. However, due to the large number of Chinese characters, the probability of rare characters increases, and the recognition effect for particularly rare characters is average.

[0070] Table 4 Some examples of test sets

[0071]

[0072] The above experiments further verified that the solution in this case effectively ensures that the hot word string is retained in the cluster to avoid pruning through hot word enhancement during end-to-end speech recognition decoding inference; and through hot word pre-enhancement and length reward, it can improve the recognition effect of short hot words in Chinese; using the domain classification model to determine whether the hot word prefix tree is needed for enhancement can reduce the impact of hot words on the recognition effect of general domains.

[0073] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0074] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0075] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0076] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0077] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A speech recognition method based on domain classification and hot word prefix tree beam search, characterized in that: Contains the following: Use the hot word list in the known field to build a hot word prefix tree of the corresponding field type, and train the corresponding hot word language model; For the target speech data, a decoding is performed using the trained end-to-end model, and the trained domain classification model is used to obtain the domain type to which the target speech data belongs; wherein, in the decoding using the end-to-end model, the speech data is firstly decoded using a greedy algorithm, and the word with the highest probability is selected as the sequence element in the predicted text at each time step, thereby obtaining a predicted text sequence of a complete path; the predicted text sequence is input into the trained domain classification model, and the domain classification model is used to obtain the domain type to which the speech data belongs; The target speech data is decoded twice using an end-to-end model, and the text corresponding to the target speech data is output using a beam search algorithm and a trained hot word language model. In each time step search of the beam search algorithm, the hot word prefix tree of the field type to which the target speech data belongs is combined to enhance the hot word probability.

2. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 1 is characterized in that: In constructing the hot word prefix tree, the known hot word list data in each field is stored as a hot word prefix tree structure of the corresponding field type. The root node in the tree structure does not store character information, and each node except the root node stores a character information, and the string formed by connecting all the characters on the path from the root node to other target nodes is the hot word corresponding to the target node.

3. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 1 is characterized in that: In the end-to-end model, the Transformer model is used to convert the target speech data sequence into a text sequence. The Transformer model adopts an encoder-decoder neural network structure based on the self-attention mechanism. The encoder and decoder in this neural network structure are respectively composed of several identical self-attention modules stacked together. The input of the encoder is the speech data feature sequence, and the input of the decoder is the text label embedding vector sequence.

4. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 1 is characterized in that: The domain classification model adopts the TextCNN network structure, which includes: an input layer for converting each character in a text sequence into a corresponding word vector, a convolutional layer for extracting text features of the word vector, a pooling layer for pooling the extracted text feature vector to obtain a global feature vector, and a fully connected layer for classifying the global feature vector through a softmax function to obtain the domain type to which it belongs.

5. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 1 is characterized in that: In the secondary decoding using the end-to-end model, first, the beam width hyperparameter B is set, and the beam search algorithm is used to select the B candidate characters with the highest current probability in each time step search, and the prefix tree state and node depth corresponding to the B candidate characters of the beam in the current time step are determined based on the candidate character results predicted by each beam in the previous time step and the relationship between the B candidate characters of the beam in the current time step; then, the hot word score is adjusted starting from the child nodes of the root node according to the node depth, so that the hot words before reaching the leaf nodes obtain additional score rewards; then, the prefix tree is pruned, and the pruned prefix tree is used to obtain the text sequence corresponding to the speech data through the paths between nodes.

6. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 5 is characterized in that: Determine the prefix tree state and node depth corresponding to the beam candidate character in the current time step. If the beam candidate character in the current time step is a child node of the prefix tree of the candidate character predicted in the previous time step, the prefix tree state of the candidate character in the current time step is set to the candidate character node in the current time step, and the node depth of the current time step is increased based on the prefix tree of the candidate character predicted in the previous time step; if the beam candidate character in the current time step is not a child node of the prefix tree of the candidate character predicted in the previous time step, and is not a child node of the root node of the prefix tree, the prefix tree state of the candidate character in the current time step is set to the root node, and the node depth is set to 0; if the beam candidate character in the current time step is a child node of the root node, the prefix tree state of the candidate character in the current time step is set to the child node, and the node depth is set to 1.

7. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 5 is characterized in that: The process of adjusting the hot word score according to the node depth is expressed as: Among them, ω k Indicates the bias score of the hot word enhancement, the b-th beam candidate character at the current time step t The corresponding prefix tree state It is used when the depth is greater than or equal to 2, i is the label of the candidate character; α k represents the pre-enhancement score of the hot word. Used when depth equals 1; β k Indicates the bonus score for short hot word length, Used when the depth is less than or equal to 3 and reaches a leaf node, n t-1,b Indicates the prefix tree state corresponding to the prediction result of the b-th beam at time step t-1.

8. The speech recognition method based on domain classification and hot word prefix tree beam search according to claim 1 is characterized in that: The final output text corresponding to the target speech data is expressed as: Where x represents a given set of acoustic sequences (x1, x2, ..., x m ), y represents the E2E model selection that can maximize the posterior probability P E2E The text sequence (y1,y2,...,y n ),y * represents the total sequence score, P E2E (y|x) represents the score of the end-to-end model, P C (y) represents the score of the language model, λ C S is a hyperparameter that represents the influence of the language model on the overall model score. t,b (y|H) represents the prefix tree score, H represents the hot word prefix string, λ K is the hyperparameter for the prefix tree score.

9. A speech recognition system based on domain classification and hot word prefix tree beam search, characterized in that: It includes: a prefix tree construction module, a first decoding module and a second decoding module, wherein: The prefix tree construction module is used to construct a hot word prefix tree of the corresponding field type using a hot word list in a known field, and train the corresponding hot word language model; The first decoding module is used to decode the target speech data once using the trained end-to-end model, and to obtain the domain type to which the target speech data belongs using the trained domain classification model; wherein, in the decoding using the end-to-end model, the speech data is firstly decoded using a greedy algorithm, and a predicted text sequence of a complete path is obtained by selecting the word with the highest probability at each time step as a sequence element in the predicted text; the predicted text sequence is input into the trained domain classification model, and the domain classification model is used to obtain the domain type to which the speech data belongs; The second decoding module is used to perform secondary decoding on the target voice data using an end-to-end model, and use a beam search algorithm and a trained hot word language model to output the text corresponding to the target voice data, wherein the beam search algorithm enhances the hot word probability in each time step search by combining the hot word prefix tree of the field type to which the target voice data belongs.

Images