Hainan dialect speech recognition method based on excitation CTC and sub-word decoding

Abstract

The invention discloses a Hainan dialect speech recognition method based on excitation CTC and sub-word decoding, and belongs to the technical field of speech recognition, and the method comprises the following steps: obtaining a Hainan dialect speech signal, and processing the Hainan dialect speech signal to extract acoustic features; obtaining text data, and processing the text data to extract language features; the method comprises the steps that a speech recognition model is constructed, the speech recognition model comprises an acoustic model, an RNN language model, an improved CTC loss function and a decoder, in the training process, parameter optimization is conducted on the acoustic model and the RNN language model based on the improved CTC loss function, and finally the trained speech recognition model is obtained; introducing a sub-word model, constructing a TSLG decoding graph, and carrying out Hainan dialect recognition by adopting the trained speech recognition model based on the TSLG decoding graph to obtain a Hainan dialect recognition result; through the method, an accurate voice recognition result can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and in particular to a Hainan dialect speech recognition method based on excitation CTC and subword decoding. Background Art

[0002] Automatic speech recognition (ASR) technology can convert human speech into text, thus providing support for applications such as voice search and voice assistants. ASR technology has been widely used. ASR technology has made significant progress in processing mainstream languages ​​such as Mandarin, but the speech data resources of Hainan dialect are relatively scarce, which limits the application and development of ASR technology in this field.

[0003] Hainan dialect speech recognition technology can provide service support for local tourism, medical care, elderly care and other industries, improve service quality and efficiency, and promote the informatization construction and development of Hainan. However, the lack of sufficient training data has limited the performance of the speech recognition system, and its recognition ability for the Hainan dialect is relatively weak.

[0004] Compared with the scarce Hainan dialect audio-text paired data, the cost of obtaining plain text data in actual production is indeed lower, and the amount of plain text data available is often several or even dozens of orders of magnitude more than audio-text paired data. Therefore, in the absence of audio-text paired data, the existing large amount of plain text data can be used to assist in the training of speech recognition models.

[0005] Using pure text data to improve speech recognition accuracy is the key to end-to-end ASR research. A common method is to integrate an external language model (ELM) and combine the ASR system score with the ELM score through shallow fusion, thereby effectively utilizing text information and improving recognition performance. However, current language model fusion usually integrates external language models during the decoding stage, which can easily lead to conflicts between the output of the external language model and the output of the acoustic model, causing information confusion, thereby affecting the final speech recognition results. Summary of the invention

[0006] In view of the above-mentioned prior art, the present invention provides a Hainan dialect speech recognition method based on excitation CTC and subword decoding, which mainly solves the technical problems existing in the above-mentioned background technology.

[0007] To achieve the above purpose, the technical solution of the embodiment of the present invention is implemented as follows:

[0008] A Hainan dialect speech recognition method based on excitation CTC and subword decoding, the method comprising the following steps:

[0009] Step S1: obtaining a Hainan dialect speech signal, processing the Hainan dialect speech signal to extract acoustic features; obtaining text data, processing the text data to extract language features;

[0010] Step S2: constructing a speech recognition model, wherein the speech recognition model includes an acoustic model, an RNN language model, an improved CTC loss function, and a decoder. During the training process, the acoustic model and the RNN language model are optimized based on the improved CTC loss function to finally obtain a trained speech recognition model;

[0011] Step S3: introducing a subword model, constructing a TSLG decoding graph, and performing Hainan dialect recognition using the trained speech recognition model based on the TSLG decoding graph to obtain a Hainan dialect recognition result.

[0012] Optionally, during the training process, parameters of the acoustic model and the RNN language model are optimized based on the improved CTC loss function, specifically including inputting the acoustic features and the text data into the acoustic model, inputting the language features into the RNN language model, and adjusting the parameters of the acoustic model and the RNN language model based on the improved CTC loss function, so that the speech recognition model implicitly learns the dependency relationship between the acoustic features and the corresponding text labels.

[0013] Optionally, the sub-word model is introduced to construct a TSLG decoding graph, and the trained speech recognition model is used to perform Hainan dialect recognition based on the TSLG decoding graph to obtain a Hainan dialect recognition result, specifically including: inputting the Hainan dialect speech signal into the acoustic model to obtain a phoneme sequence function, introducing the sub-word model, constructing a TSLG decoding graph, performing sub-word decoding on the phoneme sequence function through the TSLG decoding graph in the decoder to obtain N-best candidate results, and re-scoring based on the RNN language model, and obtaining a Hainan dialect recognition result based on the scoring result.

[0014] Optionally, the expression of the improved CTC loss function is specifically:

[0015]

[0016] in, represents the standard CTC loss function, represents the RNN language model loss function, represents the activation loss function, α and β are weight factors.

[0017] Optionally, the specific formula of the RNN language model loss function is:

[0018]

[0019] Among them, w k represents the kth tag, K is the length of the tag sequence, It is defined as the language model state encoded by the RNN, which integrates the historical information up to the k-1th label.

[0020] Optionally, the specific formula of the incentive loss function is:

[0021]

[0022] in, is the hidden state, is the label history information, γ t (k) is the label w k The probability of alignment with time t, K is the length of the label sequence, and T is the length of the time series.

[0023] Optionally, the specific expression of the introduced subword model is:

[0024] s∈Ψ(W)

[0025]

[0026] Where W is the word sequence obtained based on the n-gram language model, X is the acoustic feature sequence obtained based on the acoustic model, s is the subword sequence obtained based on the subword model, Ψ is a function that converts the word sequence W into a set of possible subword sequences s, P r (s|X) is the acoustic model probability, P r (W) is the probability of the n-gram language model, α is the scaling factor of the acoustic model, P r (s) is the probability of the subword model, β is the scaling factor of the subword model, P r (W|s) is the posterior probability of the word sequence W given the subword sequence s, P r (s|W) is the transition probability from word to subword.

[0027] Optionally, the step of introducing the subword model and constructing the TSLG decoding graph includes:

[0028] A subword model, an external n-gram language model and a dictionary are introduced, and WFST directed cyclic graphs corresponding to the subword model, the dictionary and the n-gram language model are respectively constructed. The WFST directed cyclic graphs of the acoustic model, the dictionary and the n-gram language model are compositely compiled to obtain a TLG-based decoding graph; the WFST directed cyclic graph corresponding to the subword model is compositely compiled with the TLG-based decoding graph to obtain a TSLG-based decoding graph.

[0029] Optionally, the specific expression for constructing the TSLG decoding graph is:

[0030]

[0031]

[0032] Among them, TLG is a decoding graph based on TLG, G is an n-gram language model, L is a dictionary, T is a modeling unit, TSLG is a decoding graph based on TSLG, β is a scaling factor, and S is a subword model.

[0033] The beneficial effects of the present invention are as follows: the extracted acoustic features are trained through an acoustic model, the extracted language features and acoustic features are trained through an RNN language model, the parameters of the acoustic model and the RNN language model are optimized based on an improved CTC loss function, and then decoding is performed based on a constructed TSLG decoding graph to obtain N-best candidate results, and the trained RNN language model is used for re-scoring, and finally a speech recognition result is obtained; by combining an external RNN language model in the encoding stage, the RNN language model can be used to assist the acoustic model, and by improving the unified optimization of the CTC loss function, the problem of information confusion caused by introducing an external language model in the decoding stage is avoided;

[0034] In the training phase, the improved CTC loss function is used to combine the acoustic model with the RNN language model, which can effectively capture the complex relationship between speech and language.

[0035] In the decoding stage, the TSLG decoding graph constructed by the sub-word model is introduced to decode and obtain the N-best candidate results. The trained RNN language model is used for re-scoring to obtain accurate speech recognition results. This can effectively avoid the problem of information confusion caused by the fusion of the acoustic model and the language model in the decoding stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a Hainan dialect speech recognition method based on excitation CTC and subword decoding provided in an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the training process of the Hainan dialect speech recognition method based on excitation CTC and subword decoding provided in an embodiment of the present invention.

[0038] Figure 3 It is a schematic diagram of the decoding process of the Hainan dialect speech recognition method based on excitation CTC and subword decoding provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further elaborated in detail below in conjunction with the drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. In the following description, the expression "some embodiments" is related to a subset of all possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0040] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0041] It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments proposed herein. On the contrary, providing these embodiments will make the disclosure thorough and complete, and the scope of the present invention will be fully conveyed to those skilled in the art. And the purpose of the terms used herein is only to describe specific embodiments and is not intended to be a limitation of the present invention. When used herein, the singular forms of "one", "one" and "said / the" are also intended to include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "compose" and / or "include" when used in this specification determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0042] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only implementation method.

[0043] In order to fully understand the present invention, a detailed structure will be proposed in the following description to illustrate the technical solution proposed by the present invention. The optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementations.

[0044] Example

[0045] Please refer to the attached Figure 1 , Figure 2 and Figure 3 The present invention provides a Hainan dialect speech recognition method based on excitation CTC and subword decoding, the method comprising the following steps:

[0046] Step S1: obtaining a Hainan dialect speech signal, processing the Hainan dialect speech signal to extract acoustic features; obtaining text data, processing the text data to extract language features;

[0047] Audio data of Hainan dialect are collected through professional broadcast recordings, book and article recordings, free conversation recordings, etc. The collected Hainan dialect speech signals are processed. The processing steps include audio format conversion, resampling, noise reduction, sound enhancement, and acoustic features are extracted from the processed speech signals.

[0048] Collect text data and clean the collected text data, including removing irrelevant symbols, unifying character encoding, correcting typos, and extracting language features from the processed text data;

[0049] Step S2: constructing a speech recognition model, which includes an acoustic model, an RNN language model, an improved CTC loss function, and a decoder. During the training process, the acoustic model and the RNN language model are optimized based on the improved CTC loss function to finally obtain a trained speech recognition model.

[0050] The acoustic model uses the Transformer architecture to train acoustic features and text data, and enhances the training data during the training process to effectively improve the generalization ability of the model. In addition, when processing input features with higher complexity, the acoustic model using the Transformer architecture can more flexibly capture contextual information and has stronger generalization ability.

[0051] Step S3: introduce the subword model, construct a TSLG decoding graph, and use the trained speech recognition model based on the TSLG decoding graph to perform Hainan dialect recognition to obtain the Hainan dialect recognition result.

[0052] Specifically, the acoustic features, language features, and text data are input into the speech recognition model for training, and the improved CTC loss function is used for iterative update. The language model is used to assist the acoustic model in implicitly learning the dependency relationship between the acoustic features and the corresponding text labels. The N-best candidate results are obtained by constructing a TSLG decoding graph for subword decoding, and the trained RNN language model is used for re-scoring to obtain the speech recognition results.

[0053] As an optional implementation, during the training process, the parameters of the acoustic model and the RNN language model are optimized based on the improved CTC loss function, specifically including: inputting the acoustic features and text data into the acoustic model, inputting the language features into the RNN language model, calculating the loss based on the improved CTC loss function, adjusting the parameters of the acoustic model and the RNN language model, so that the speech recognition model implicitly learns the dependency relationship between the acoustic features and the corresponding text labels;

[0054] By improving the joint optimization of the CTC loss function, it is possible to learn acoustic features and language features at the same time, which can more effectively capture the complex relationship between speech and language. It can also help the model more accurately learn the correspondence between acoustic features and language features, thereby significantly improving the performance of the speech recognition system and further improving the accuracy of recognition.

[0055] As an optional implementation, a subword model is introduced, a TSLG decoding graph is constructed, and a trained speech recognition model is used to perform Hainan dialect recognition based on the TSLG decoding graph to obtain a Hainan dialect recognition result, specifically including: inputting the Hainan dialect speech signal into the acoustic model to obtain a phoneme sequence function, introducing a subword model, constructing a TSLG decoding graph, performing subword decoding on the phoneme sequence function through the TSLG decoding graph in the decoder to obtain N-best candidate results, and re-scoring based on the RNN language model, and obtaining the Hainan dialect recognition result based on the scoring result.

[0056] As an optional implementation, the expression of the improved CTC loss function is specifically:

[0057]

[0058] in, represents the standard CTC loss function, represents the RNN speech model loss function, represents the incentive loss function, α and β are weight factors, which can be adjusted to weigh the importance of different loss terms;

[0059] Specifically, the mean square error loss term, the standard CTC loss term, and the standard cross entropy loss term of the RNN language model in the incentive learning strategy are weighted and summed to form an improved CTC loss function. The acoustic model and the RNN language model are trained based on the improved CTC loss function, and the parameters of the acoustic model and the RNN language model are updated to obtain the optimized results.

[0060] Since the acoustic features and text data are input into the acoustic model, and the language features are input into the language model, the acoustic model and the language model will output a probability matrix to the CTC loss function respectively; the input of the CTC loss function includes the output of the acoustic model and the real label sequence;

[0061] Specifically, the output of the acoustic model is a probability matrix normalized by softmax, with a dimension of N×T, where N is the number of categories (such as the number of English letters plus a space character) and T is the number of speech frames; the true label sequence, that is, the text information corresponding to the speech; this true label sequence is used to compare with the output of the acoustic model to calculate the loss, and this loss value is used to optimize the acoustic model during the training process so that its output is closer to the true label sequence;

[0062] The predicted output of the RNN language model is the probability distribution of the next word predicted under all acoustic features and historical information. These probability distributions are calculated through the hidden layer state of the RNN and normalized by the softmax function to ensure that the sum of the probabilities of all possible outputs is 1; the real label sequence, that is, the text transcription corresponding to the input speech signal, the difference between the predicted output and the real label sequence is the loss value. By calculating the loss value, the smaller the loss value, the closer the model's prediction result is to the real value, and the better the model's performance is;

[0063] Therefore, during the training process, the improved CTC loss function is used to calculate the loss value between the output of the acoustic model and the real label sequence, and the loss value between the output of the RNN language model and the real label sequence, and the excitation loss function is used to assist in balancing the loss value between the output of the acoustic model and the real label sequence, and the loss value between the output of the RNN language model and the real label sequence; in addition to obtaining the loss value, the loss function also outputs gradient information, which is used to guide the model training process, that is, how to adjust the model parameters to reduce the loss value. The gradient information is obtained by calculating the partial derivative of the loss value with respect to the model parameters, which indicates the impact of changes in the model parameters in each direction on the loss value;

[0064] The back-propagation algorithm is used to calculate the gradient information, and the model parameters are updated through the gradient descent method or other optimization algorithms. This process will be iterated continuously until the loss value of the model reaches an acceptable range or the preset number of iterations is reached, that is, to better learn the dependency between the acoustic features and the corresponding text labels, and then update the acoustic model and language parameters. Through continuous iterative updates, the trained speech recognition model is finally obtained.

[0065] The specific formula of the RNN language model loss function is:

[0066]

[0067] Among them, w k represents the kth tag, K is the length of the tag sequence, It is defined as the language model state encoded by the RNN, which integrates the historical information up to the k-1th label.

[0068] The specific formula of the incentive loss function is:

[0069]

[0070] in, is the hidden state, is the label history information, γ t (k) is the label w k The probability of alignment with time t, K is the length of the label sequence, and T is the length of the time series;

[0071] During the training process, an incentive learning strategy is designed, which introduces an additional mean squared error loss term in the training process, aiming to minimize the hidden state History information with tags The difference between

[0072] The incentive loss term is a key part, which aims to promote the recognition model to implicitly learn higher-level abstract concepts by strengthening the consistency between the state trajectory of the recurrent neural network inside the recognition model and the state trajectory of the auxiliary RNN language model, and use the auxiliary RNN as a guide for language representation learning, so that the main recognition model can gradually approach a more complex language feature space during the optimization process.

[0073] As an optional implementation, the specific expression of introducing the subword model is:

[0074] s∈Ψ(W)

[0075]

[0076] P r (s|W)=1

[0077]

[0078] Where W is the word sequence obtained based on the n-gram language model, X is the acoustic feature sequence obtained based on the acoustic model, s is the subword sequence obtained based on the subword model, Ψ is a function that converts the word sequence W into a set of possible subword sequences s, P r (s|X) is the acoustic model probability, P r (W) is the probability of the n-gram language model, α is the scaling factor of the acoustic model, P r (s) is the probability of the subword model, β is the scaling factor of the subword model, P r (W|s) is the posterior probability of the word sequence W given the subword sequence s, Pr (s|W) is the transition probability from word to subword.

[0079] The specific expression for constructing the TSLG decoding graph is:

[0080]

[0081]

[0082] Among them, TLG is a decoding graph based on TLG, G is an n-gram language model, L is a dictionary, T is an acoustic model, TSLG is a decoding graph based on TSLG, β is a scaling factor for proper scaling between the acoustic model and the language model, and S is a subword model.

[0083] The steps of introducing the subword model and constructing the TSLG decoding graph include:

[0084] The sub-word model, external n-gram language model and dictionary are introduced, and the WFST directed cyclic graphs corresponding to the sub-word model, dictionary and n-gram language model are constructed respectively. The WFST directed cyclic graphs of the acoustic model, dictionary and n-gram language model are compositely compiled to obtain a TLG-based decoding graph; the WFST directed cyclic graph corresponding to the sub-word model is compositely compiled with the TLG-based decoding graph to obtain a TSLG-based decoding graph.

[0085] It should be noted that the subword model decomposes words into smaller units, such as phonemes or letter combinations. When faced with a large number of words, the introduction of the subword model can help improve the accuracy of recognition. After constructing the TSLG decoding graph, the decoding graph is used to decode to obtain the N-best candidate results, and the trained RNN language model is used to re-score, and the value with the highest probability is found as the final predicted output, and continuous repetitions and blank blocks are removed to obtain the final recognition result.

[0086] The subword model is a model that divides text into subword units for processing. It uses subwords as input units and splits text into multiple subwords. Common word segmentation methods include rule-based word segmentation and statistical-based word segmentation. Each subword is mapped to a vector representation and then input into a neural network for processing. The subword level model can better capture the complex language structure and word meaning information of the text;

[0087] Construct a decoding graph to combine the information of each part of T, S, L and G, where the modeling unit is represented by T, the subword model is represented by S, the dictionary is represented by L, and the language model is represented by G; the acoustic model is converted into the representation of WFST to form T, the words or sentences are split into modeling units to form the dictionary L, the N-gram language model is converted into the representation of WFST to form G, and the subword model is converted into the representation of WFST to form S. WFST is a decoding form commonly used in the field of speech recognition, and the N-gram language model / subword model / acoustic model can be represented in the form of a graph;

[0088] Through the composite operation of WFST, the four WFST directed cycle graphs T, S, L, and G are merged into a TSLG decoding graph. In this process, the output of T becomes the input of S, the output of S becomes the input of L, and the output of L becomes the input of G, finally forming a static decoding network containing all necessary information;

[0089] Specifically, the input of T is the joint optimization result, which includes the trained speech feature vector sequence, and the output is the phoneme sequence;

[0090] The input of S is the phoneme sequence output by T, and the output is the phoneme sequence;

[0091] The input of L is the phoneme sequence output by S, and the output is a word sequence. Based on the pronunciation rules and vocabulary in the dictionary, L maps the phoneme sequence output by S to the corresponding word or phrase sequence;

[0092] The input of G is the word sequence output by L, and it outputs the optimal word sequence, i.e. the recognition result. G scores and sorts the input word sequence according to the statistical information of the n-gram language model, and selects the sequence with the highest score as the final recognition result. In this process, G will consider the dependency relationship and contextual information between words.

[0093] Acoustic model, subword model and external n-gram language model, integrate the information of the three models, the decoding graph contains all possible sequences and the probabilities of each word in the sequence, the decoding graph uses the information integrated by the three models, and then uses a search strategy to search on the decoding graph. In the present invention, the search strategy is a beam search algorithm, which searches for the optimal path, finds the most likely word sequence N-best candidate result, and uses the trained RNN language model to re-score to obtain the text sequence that is most likely to correspond to the input voice information, that is, to obtain the corresponding voice recognition result;

[0094] Since the acoustic model and the language model differ in processing granularity, the acoustic model is usually built based on subword-level units, while the language model is generally modeled at the word level. In order to better balance the acoustic model score and the language model score, the introduction of the subword model can balance the differences in processing granularity between the acoustic model and the language model. While maintaining the fine-grained features of the acoustic model, the contextual information of the language model is incorporated to achieve more accurate speech recognition results.

[0095] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A Hainan dialect speech recognition method based on excitation CTC and subword decoding, characterized in that: The method comprises the following steps: Step S1: obtaining a Hainan dialect speech signal, processing the Hainan dialect speech signal to extract acoustic features; obtaining text data, processing the text data to extract language features; Step S2: constructing a speech recognition model, wherein the speech recognition model includes an acoustic model, an RNN language model, an improved CTC loss function, and a decoder. During the training process, the acoustic model and the RNN language model are optimized based on the improved CTC loss function to finally obtain a trained speech recognition model; Step S3: introducing a subword model, constructing a TSLG decoding graph, and performing Hainan dialect recognition using the trained speech recognition model based on the TSLG decoding graph to obtain a Hainan dialect recognition result.

2. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 1 is characterized in that: During the training process, the parameters of the acoustic model and the RNN language model are optimized based on the improved CTC loss function, specifically including: inputting the acoustic features and the text data into the acoustic model, inputting the language features into the RNN language model, and adjusting the parameters of the acoustic model and the parameters of the RNN language model based on the improved CTC loss function, so that the speech recognition model implicitly learns the dependency relationship between the acoustic features and the corresponding text labels.

3. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 1 is characterized in that: The method introduces a subword model, constructs a TSLG decoding graph, and uses a trained speech recognition model based on the TSLG decoding graph to perform Hainan dialect recognition to obtain a Hainan dialect recognition result. Specifically, the method includes inputting a Hainan dialect speech signal into an acoustic model to obtain a phoneme sequence function, introducing a subword model, constructing a TSLG decoding graph, performing subword decoding on the phoneme sequence function through the TSLG decoding graph in the decoder to obtain N-best candidate results, and re-scoring based on the RNN language model, and obtaining a Hainan dialect recognition result based on the scoring result.

4. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 2 is characterized in that: The expression of the improved CTC loss function is specifically: in, represents the standard CTC loss function, represents the RNN language model loss function, represents the activation loss function, α and β are weight factors.

5. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 4 is characterized in that: The specific formula of the RNN language model loss function is: Among them, w k represents the kth tag, K is the length of the tag sequence, It is defined as the language model state encoded by the RNN, which integrates the historical information up to the k-1th label.

6. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 4 is characterized in that: The specific formula of the incentive loss function is: in, is the hidden state, is the label history information, γ t (k) is the label w k The probability of alignment with time t, K is the length of the label sequence, and T is the length of the time series.

7. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 3 is characterized in that: The specific expression of the introduced subword model is: s∈Ψ(W) Where W is the word sequence obtained based on the n-gram language model, X is the acoustic feature sequence obtained based on the acoustic model, s is the subword sequence obtained based on the subword model, Ψ is a function that converts the word sequence W into a set of possible subword sequences s, P r (s|X) is the acoustic model probability, P r (W) is the probability of the n-gram language model, α is the scaling factor of the acoustic model, P r (s) is the probability of the subword model, β is the scaling factor of the subword model, P r (W|s) is the posterior probability of the word sequence W given the subword sequence s, P r (s|W) is the transition probability from word to subword.

8. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 7 is characterized in that: The step of introducing the subword model and constructing the TSLG decoding graph includes: A subword model, an external n-gram language model and a dictionary are introduced, and WFST directed cyclic graphs corresponding to the subword model, the dictionary and the n-gram language model are respectively constructed. The WFST directed cyclic graphs of the acoustic model, the dictionary and the n-gram language model are compositely compiled to obtain a TLG-based decoding graph; the WFST directed cyclic graph corresponding to the subword model is compositely compiled with the TLG-based decoding graph to obtain a TSLG-based decoding graph.

9. The Hainan dialect speech recognition method based on excitation CTC and subword decoding according to claim 8 is characterized in that: The specific expression for constructing the TSLG decoding graph is: Among them, TLG is a decoding graph based on TLG, G is an n-gram language model, L is a dictionary, T is a modeling unit, TSLG is a decoding graph based on TSLG, β is a scaling factor, and S is a subword model.

Images