Burmese speech recognition error correction method and device

By using a BERT model with shared parameters and a multi-head attention mechanism, combined with a phoneme context extractor and a Transformer decoder, a Burmese speech recognition error correction device was constructed. This device solved the problems of spaces and homophones in the Burmese speech recognition system, improving recognition accuracy and error correction effect.

CN117059072BActive Publication Date: 2026-11-17KUNMING UNIV OF SCI & TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311148154.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-07
Publication Date
2026-11-17
Estimated Expiration
2043-09-07

AI Technical Summary

Technical Problem

Existing Burmese speech recognition systems suffer from recognition errors in practical applications, particularly incorrect space placement and homophony errors, which affect the accuracy of downstream tasks. Existing error correction methods are also unable to effectively capture subtle differences in language.

Method used

A Burmese speech recognition error correction device is constructed by using a BERT model with shared parameters to extract text and phoneme features, combined with a multi-head attention mechanism and a Transformer decoder. The device automatically corrects speech recognition errors through a phoneme context extractor and a label predictor.

Benefits of technology

It significantly improved the accuracy of Burmese speech recognition, especially by 2% in recall and F1 score, and achieved an error correction rate of 79.15%, effectively correcting space errors and homophone errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117059072B_ABST
    Figure CN117059072B_ABST
Patent Text Reader

Abstract

This invention relates to a method and apparatus for Burmese speech recognition error correction, belonging to the field of natural language processing technology. The invention includes the following steps: acquiring text features and phoneme features of data through a text encoder and a phoneme encoder, respectively; using a label predictor to determine the labels of characters in the transcribed text, achieving label prediction; employing a phoneme context extractor to obtain phoneme context feature information; and using a multi-head attention mechanism and a Transformer decoder to decode the text and phoneme hidden states, constructing a Burmese speech recognition text error correction apparatus. The two feature extraction processes of this invention share the parameters of the BERT model. In this way, the relevant features of the transcribed text and phonemes are simultaneously acquired, thereby more accurately capturing the correlation between text and phonemes and improving the model's performance and accuracy. This invention achieves good error correction results, with an F1 score of 79.15% for correcting transcribed text.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a method and apparatus for Burmese speech recognition and error correction, belonging to the field of natural language processing technology. Background Technology

[0002] With the widespread application of Automatic Speech Recognition (ASR) technology in scenarios such as real-time captioning for human-computer interaction and conference transcription, ASR is playing an increasingly important role in multiple fields. However, due to factors such as differences in user pronunciation, environmental noise, and language complexity, ASR still suffers from recognition errors in practical applications. These erroneous texts impact downstream tasks such as intent recognition and speech translation. Text correction for ASR can effectively improve ASR performance and prevent the accumulation of errors in downstream tasks, making it a crucial method for enhancing ASR system performance.

[0003] Early research primarily focused on speech recognition error detection methods, with relatively little research on speech recognition error correction methods. With technological advancements, three main types of methods have emerged for ASR text correction: First, rule-based methods use manually defined rules to detect and correct text errors, requiring definition based on the grammatical rules of the specific language; second, statistical methods (such as N-Gram, HMM, etc.), but these often fail to capture subtle differences specific to a particular language; and third, methods based on pre-trained language models, such as BERT (Bidirectional Encoder Representations from Transformers), use a mask-prediction approach for error correction. Summary of the Invention

[0004] This invention provides a Burmese speech recognition error correction method and apparatus to improve error correction performance and accuracy, and reduce word error rate.

[0005] The technical solution of the present invention is as follows: Firstly, the present invention provides a Burmese speech recognition error correction method, the specific steps of which are as follows:

[0006] Step 1: Corpus Collection: Collect training and testing corpora of correct texts and transcribed texts of several types of errors;

[0007] Step 2: Extracting text features: Based on the text feature space of the transcribed text, feature values ​​are automatically extracted through a text encoder to obtain the text features of the data;

[0008] Step 3: Extract phoneme features: Based on the phoneme feature space of the transcribed text, feature values ​​are automatically extracted through a phoneme encoder to obtain the phoneme features of the data;

[0009] Step 4: Use a label predictor to determine the labels of characters in the transcribed text to achieve label prediction;

[0010] Step 5: Use a phoneme context extractor to obtain phoneme context feature information;

[0011] Step 6: Using a multi-head attention mechanism and a Transformer decoder, the text and phoneme hidden states are decoded to construct a Burmese speech recognition text correction device.

[0012] Further, in Step 1, a Burmese speech recognition text error correction dataset is constructed, collecting four types of errors: space errors, homophone errors, deletion errors, and word omission errors. First, using Microsoft speech recognition technology and the Python programming language, the collected Burmese text audio is input into the Burmese speech recognition interface for processing. Second, by replacing homophones, randomly adding or deleting spaces, and adding randomly generated characters, error text is simulated using Burmese monolingual text to simulate the recognition errors output by the speech recognition system. Finally, the obtained transcribed text and its corresponding correct text are combined into a tuple SEN = (RAW, ASR), where RAW represents the correct text and ASR represents the Burmese speech recognition transcribed text. This process aims to simulate and evaluate the performance and accuracy of the speech recognition system.

[0013] Using appropriate tools, SEN is converted to PH = (RAW_phome, ASR_phome), where RAW_phome represents the original phoneme data and ASR_phome represents the phoneme result of automatic speech recognition. Through this process, error-correcting corpus corresponding to the text phonemes is obtained.

[0014] Furthermore, in Step 2, the text features of the transcribed text are extracted using a pre-trained Burmese BERT model. At the same time, in Step 3, the corresponding phoneme features are obtained. These two feature extraction processes share the parameters of the BERT model. In this way, the relevant features of the transcribed text and phonemes are obtained simultaneously, thereby more accurately capturing the relationship between text and phonemes and improving the performance and accuracy of the model.

[0015] Furthermore, the specific steps of Step 2 are as follows: For a Burmese speech recognition text sequence to be detected, it is represented as W = {w1, w2, ..., w...} n}, where n is the length of the text sequence, W i By modeling the subwords and passing them through the embedding layer, we obtain the BERT input sequence X = {x1, x2, x3, ..., x...} n The text encoding process is shown in equation (1):

[0016] E = BERT TextEncoder (WE(X)+PE(X)) (1)

[0017] E = {h1, h2, h3, ..., h n} is the encoded text feature sequence, h t ∈R b×l×d Let X be the encoding representation of the t-th feature, l be the length of the text sequence, d be the feature dimension, and WE and PE be the word embedding and position embedding functions, respectively.

[0018] Furthermore, the specific steps of Step 3 are as follows:

[0019] The phoneme encoder converts Burmese speech recognition text into a corresponding phoneme sequence M = {m1, m2, m3, ... m}. n}, and then map it to a continuous real vector space through an embedding layer to obtain Y = {y1, y2, y3, ..., y}. n Then, the phoneme embedding input encoder obtains the phoneme feature encoding; the process of obtaining the phoneme encoding is represented by equation (2);

[0020] P = BERT PhonemeEncoder (WE(Y)+PE(Y)) (2)

[0021] In equation (2), P = {p1, p2, p3..., p...} m} represents phoneme coding features, while P t ∈R b×l×dim Let represent the encoded feature information of the t-th phoneme in P, l be the length of the text sequence, and d be the phoneme feature dimension. The phoneme encoder and the text encoder adopt a shared parameter method to realize the information sharing between characters and phonemes in the input text sequence. WE and PE represent word embedding and position embedding functions, respectively.

[0022] Furthermore, the label prediction in Step 4 includes:

[0023] A fully connected layer is used as input, which is the encoded text feature sequence, and the prediction result is represented as T. In the prediction result, "K" represents a correct character, "C" represents a space position error, substitution error, or deletion error, and the symbol "D" represents an insertion error. To train the label prediction model and achieve label prediction, a label predictor is constructed. The label predictor is used to automatically determine the label of characters in the transcribed text, ultimately achieving the goal of label prediction. The specific process of label prediction is as follows:

[0024]

[0025] In equation (3), It is a predictive operation, Wd and b d These are learnable parameters, and the operational loss is defined as minimizing the log-likelihood estimate.

[0026] Furthermore, in Step 5, the phoneme context extractor consists of a bidirectional long short-term memory network (BiLSTM) and a linear layer. The BiLSTM is used to capture contextual information in the phoneme sequence and generate feature representations with contextual relevance, while the linear layer maps the hidden state output by the BiLSTM to a low-dimensional vector space and serves as the input to the multi-head attention mechanism.

[0027] This invention captures the contextual information of phonemes in transcribed text using a bidirectional LSTM network and generates context-dependent feature representations. Then, in Step 6, a multi-head attention mechanism is used to fuse the phoneme features with the output labeled text features. This fusion mechanism effectively correlates phoneme features with text features, further improving the model's performance and accuracy. Step 6 also receives the text features output from Step 4, thereby achieving automatic error correction for Burmese speech. By fusing phoneme features with text features, the model can comprehensively consider the relationship between phonemes and text, achieving automatic error correction for Burmese speech and improving the accuracy and effectiveness of the error correction function.

[0028] Furthermore, in Step 6, the construction process of the Burmese speech recognition text correction device is as follows:

[0029] The Transformer decoder is used to generate the corrected text. The Transformer decoder uses a self-attention mechanism to calculate the dependencies between different positions in the input sequence, thereby better capturing the contextual information of the text. In addition, a multi-head cross-attention mechanism is introduced to capture the association features between phoneme information and text, further extracting phoneme information and assisting the decoder in correcting errors in space position and homophones.

[0030] Furthermore, in Step 6, during the error correction operation, the input data of the label predictor is first represented in the form of a tuple SEN = (RAW, ASR), where RAW represents the original data and ASR represents the automatic speech recognition result. In addition, the input data of the phoneme feature extractor is represented in the form of a tuple PH = (RAW_phoneme, ASR_phoneme), where RAW_phoneme represents the original phoneme data and ASR_phoneme represents the phoneme result of automatic speech recognition. By automatically extracting features from these two tuples using the model, the required input data is finally obtained.

[0031] In a second aspect, the present invention also provides a Burmese speech recognition and error correction apparatus, which includes a module for performing the method described in the first aspect.

[0032] The beneficial effects of this invention are:

[0033] 1. The error detection mechanism using shared parameters in this invention exhibits significant advantages. It outperforms schemes without shared parameters in all metrics, particularly in recall and F1 score, where the shared parameter method improves performance by more than 2%.

[0034] 2. The method of this invention corrects errors such as whitespace, homophones, deletions, and omissions. Several Burmese error-correction corpora were collected, with whitespace and homophone errors being the most common types. An encoder-decoder model architecture was used to establish a device for predicting transcribed text labels and correcting errors. This device can automatically correct errors in the transcribed text after speech recognition in Burmese. Experimental results show that this method achieves good error correction performance and is the optimal technical solution obtained during the experiment, with an F1 score of 79.15% for correcting transcribed text. Attached Figure Description

[0035] Figure 1 This is a flowchart from the present invention;

[0036] Figure 2 This is a schematic diagram illustrating an example of an error in this invention. Detailed Implementation

[0037] Example 1: As Figures 1-2 As shown, the Burmese speech recognition error correction method has the following specific steps:

[0038] Step 1: Construct a Burmese speech recognition text correction dataset, collecting four types of errors: whitespace errors, homophone errors, deletion errors, and word omission errors. First, using Microsoft speech recognition technology and the Python programming language, 5000 real error correction data points were generated, and 30,000 error correction data points were created based on Burmese language features to form a training dataset. The collected Burmese text audio was input into a Burmese speech recognition interface for processing to obtain Burmese speech recognition text. Second, by replacing homophones, randomly adding or deleting spaces, and adding randomly generated characters, error text was simulated using Burmese monolingual text to simulate the recognition errors output by the speech recognition system, ensuring that the distribution of simulated errors is consistent with real errors. This process aims to simulate and evaluate the performance and accuracy of the speech recognition system. Furthermore, to verify the effectiveness of the text method on an English dataset that also uses spaces for word segmentation, experiments were conducted using the English public dataset ATIS. This dataset contains user voice recordings composed of flight information, and the transcribed text corresponding to the ATIS dataset voice was obtained using Amazon Polly.

[0039] Finally, the transcribed text and its corresponding correct text are combined into a tuple SEN = (RAW, ASR), where RAW represents the correct text and ASR represents the Burmese speech recognition transcribed text;

[0040] The transcribed text was phoneme-converted using a tool called Epitran (https: / / github.com / dmort27 / epitran).

[0041] A total of 35,000 data points were collected, and each pair of error-correcting sentences was organized according to the rules of binary pairs. Simultaneously, appropriate tools were used to convert SEN to PH = (RAW_phome, ASR_phome), where RAW_phome represents the original phoneme data, and ASR_phome represents the phoneme result from automatic speech recognition. Through this process, error-correcting corpus corresponding to the text phonemes was obtained.

[0042] Typically, errors in Burmese speech recognition transcripts mainly consist of homophones and incorrect space placement. Analysis of Burmese transcripts obtained using Microsoft speech recognition technology reveals a large number of space placement errors and homophone errors (such as...). Figure 2 (The errors shown). These errors have a certain impact on the accuracy and reliability of Burmese speech recognition.

[0043] Step 2: Extract text features from the transcribed text using a pre-trained Burmese BERT model. Simultaneously, obtain the corresponding phoneme features in Step 3. These two feature extraction processes share the parameters of the BERT model. In this way, the relevant features of the transcribed text and phonemes can be obtained at the same time, thereby more accurately capturing the relationship between text and phonemes and improving the performance and accuracy of the model.

[0044] The specific steps of Step 2 are as follows: For a Burmese speech recognition text sequence to be detected, it is represented as W = {w1, w2, ..., w...} n}, where n is the length of the text sequence, W i By modeling the subwords and passing them through the embedding layer, we obtain the BERT input sequence X = {x1, x2, x3, ..., x...} n The text encoding process is shown in equation (1):

[0045] E = BERT TextEncoder (WE(X)+PE(X)) (1)

[0046] E = {h1, h2, h3, ..., h n} is the encoded text feature sequence, h t ∈R b×l×d Let X be the encoding representation of the t-th feature, l be the length of the text sequence, d be the feature dimension, and WE and PE represent the word embedding and position embedding functions, respectively. The text encoder structure is as follows: Figure 2 As shown in ①.

[0047] Step 3: Extract phoneme features: Based on the phoneme feature space of the transcribed text, feature values ​​are automatically extracted through a phoneme encoder to obtain the phoneme features of the data;

[0048] The specific steps of Step 3 are as follows:

[0049] The phoneme encoder converts Burmese speech recognition text into a corresponding phoneme sequence M = {m1, m2, m3, ... m}. n}, and then map it to a continuous real vector space through an embedding layer to obtain Y = {y1, y2, y3, ..., y}. n Then, the phoneme embedding input encoder obtains the phoneme feature encoding; the process of obtaining the phoneme encoding is represented by equation (2);

[0050] P = BERT PhonemeEncoder (WE(Y)+PE(Y)) (2)

[0051] In equation (2), P = {p1, p2, p3..., p...} m} represents phoneme coding features, while P t ∈Rb×l×dim Let represent the encoded feature information of the t-th phoneme in P, l be the length of the text sequence, and d be the phoneme feature dimension. The phoneme encoder and the text encoder adopt a shared parameter method to realize the information sharing between characters and phonemes in the input text sequence. WE and PE represent word embedding and position embedding functions, respectively.

[0052] Step 4: Use a tag predictor to determine the tags of characters in the transcribed text to achieve tag prediction; the tag prediction in Step 4 includes:

[0053] A fully connected layer is used as input, which is the encoded text feature sequence, and the prediction result is represented as T. In the prediction result, "K" represents a correct character, "C" represents a space position error, substitution error, or deletion error, and the symbol "D" represents an insertion error. To train the label prediction model and achieve label prediction, a label predictor is constructed. The label predictor is used to automatically determine the label of characters in the transcribed text, ultimately achieving the goal of label prediction. The specific process of label prediction is as follows:

[0054]

[0055] In equation (3), It is a predictive operation, W d and b d These are learnable parameters, and the operational loss is defined as minimizing the log-likelihood estimate:

[0056]

[0057] Step 5: Use a phoneme context extractor to obtain phoneme context feature information;

[0058] The phoneme context extractor aims to learn the relationships and patterns between phonemes to better capture their semantic information, thereby improving the performance of the error correction model. The phoneme context extractor consists of a Bidirectional Long Short-Term Memory (BiLSTM) network and a linear layer. The BiLSTM captures contextual information in the phoneme sequence and generates context-dependent feature representations, while the linear layer maps the hidden states output by the BiLSTM to a low-dimensional vector space, which serves as input to a multi-head attention mechanism. This design aims to effectively introduce phoneme information into the model and further optimize the feature extraction process through multi-head attention to improve the performance of the error correction model.

[0059] In summary, this invention employs a feature selection method comprising two main strategies. First, it uses a shared parameter approach to extract textual and phoneme features to improve the model's efficiency and accuracy. Second, to capture contextual information within the phoneme sequence, it utilizes a BiLSTM+Linear approach for feature selection, thereby further optimizing model performance.

[0060] Step 6: Employing a multi-head attention mechanism and a Transformer decoder, the text and phoneme latent states are decoded to construct a Burmese speech recognition text correction device. The construction process of the Burmese speech recognition text correction device is as follows: A Transformer decoder is used to generate the corrected text. The Transformer decoder utilizes a self-attention mechanism to calculate the dependencies between different positions in the input sequence, thereby better capturing the contextual information of the text. Furthermore, a multi-head cross-attention mechanism is introduced to capture the association features between phoneme information and text, further extracting phoneme information to assist the decoder in correcting errors in space placement and homophones.

[0061] For the position k that needs to be changed, the corresponding calculation formula for each latent vector is as follows:

[0062]

[0063] In equation (5), WE and PE are the same word embedding and position embedding functions as in equation (1), h t W is the encoded text feature. p These are trainable parameters. Then, the transformer's decoder is used to fuse all the information, a process that can be represented by equation (6):

[0064]

[0065] Where K is the output of the phoneme context extractor, and Q and V are the outputs of the text encoder, E.

[0066]

[0067] Among them W y and b y These are all parameters obtained after training. The loss of the error correction model is defined as:

[0068]

[0069] The total loss is as follows, where α is a hyperparameter in equation (9):

[0070] loss = αloss oper +loss dec (9)

[0071] During error correction, the input data of the label predictor is first represented as a pair SEN = (RAW, ASR), where RAW represents the original data and ASR represents the automatic speech recognition result. Additionally, the input data of the phoneme feature extractor is represented as a pair PH = (RAW_phoneme, ASR_phoneme), where RAW_phoneme represents the original phoneme data and ASR_phoneme represents the phoneme result of the automatic speech recognition. By automatically extracting features from these two pairs using the model, the required input data is finally obtained.

[0072] According to the concept of the present invention, the present invention also provides a Burmese speech recognition and error correction device, which includes the following integrated modules:

[0073] Corpus collection module: used to collect training and testing corpora of correct texts and transcribed texts with several types of errors;

[0074] Text feature extraction module: This module is used to automatically extract feature values ​​from the text feature space of transcribed text through a text encoder, thereby obtaining the text features of the data.

[0075] Phoneme feature extraction module: This module is used to automatically extract feature values ​​from the phoneme feature space of the transcribed text through a phoneme encoder, thereby obtaining the phoneme features of the data.

[0076] Tag predictor: Used to determine the tags of characters in transcribed text, thus achieving tag prediction;

[0077] Acquisition module: Used to acquire phoneme context feature information using a phoneme context extractor;

[0078] Decoding and Building Module: Used to decode text and phoneme hidden states using a multi-head attention mechanism and a Transformer decoder, and to build a Burmese speech recognition text correction device.

[0079] To verify the effectiveness of the method of the present invention, three types of phoneme feature fusion methods, attention, cat(dim=1), and cat(dim=0), were selected to train and construct different error correction devices. The error correction effects of the three methods were compared on the dataset to verify the optimal error correction method and fusion method.

[0080] Table 1 Comparison of shared parameters and phonemes fused using different methods.

[0081]

[0082] Based on the data analysis in Table 1, the error detection mechanism using shared parameters demonstrates significant advantages. It outperforms the scheme without shared parameters across all metrics, particularly in recall and F1 score, where the shared parameter method improves performance by over 2%. However, it still lags behind the multi-head attention module proposed in this invention, which directly integrates phoneme information into the decoder. Compared to other fusion methods, the multi-head attention mechanism more accurately introduces phoneme information into the decoder, better establishes the association between phonemes and text, and precisely selects appropriate characters for correction. The method employing parameter sharing and cross-attention mechanisms reduces the word error rate.

[0083] This invention employs a text-based BERT and a phoneme-based BERT with shared parameters to construct an error detection module, while a phoneme context extractor and decoder serve as the error correction module. This device can automatically detect and correct errors in transcribed text. Experimental results show that this error correction method is an optimal solution obtained during the experiment, and the constructed error correction device achieves good error correction performance, with an F1 score of 79.15% for correcting transcribed text.

[0084] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for Myanmar speech recognition error correction, characterized in that: The specific steps of the method are as follows: Step 1, Corpus Collection: Collect training and testing corpora of correct texts and transcribed texts of several types of errors; Step 2: Extracting text features: Based on the text feature space of the transcribed text, feature values ​​are automatically extracted through a text encoder to obtain the text features of the data; In Step 2, the text features of the transcribed text are extracted using a pre-trained Burmese BERT model. At the same time, the corresponding phoneme features are obtained in Step 3. These two feature extraction processes share the parameters of the BERT model. In this way, the relevant features of the transcribed text and phonemes are obtained at the same time, so as to more accurately capture the relationship between the text and phonemes and improve the performance and accuracy of the model. Step 3: Extract phoneme features: Based on the phoneme feature space of the transcribed text, feature values ​​are automatically extracted through a phoneme encoder to obtain the phoneme features of the data; Step 4: Use a label predictor to determine the labels of characters in the transcribed text to achieve label prediction; Step 5: Use a phoneme context extractor to obtain phoneme context feature information; Step 6: Using a multi-head attention mechanism and a Transformer decoder, the text and phoneme hidden states are decoded to construct a Burmese speech recognition text correction device.

2. The Myanmar speech recognition correction method of claim 1, wherein: In Step 1, a Burmese speech recognition text error correction dataset is constructed, and four types of errors are collected, including space errors, homophone errors, deletion errors, and word omission errors. First, using Microsoft speech recognition technology and the Python programming language, the collected Burmese text audio is input into the Burmese speech recognition interface for processing; Secondly, by replacing homophones, randomly adding or deleting spaces, and adding randomly generated characters, erroneous text is generated using Burmese monolingual text to simulate the recognition errors output by the speech recognition system. Finally, the transcribed text and its corresponding correct text are combined into a tuple SEN=(RAW,ASR), where RAW represents the correct text and ASR represents the Burmese speech recognition transcribed text. This process aims to simulate and evaluate the performance and accuracy of the speech recognition system. Using appropriate tools, SEN is converted to PH=(RAW_phome,ASR_phome), where RAW_phome represents the original phoneme data and ASR_phome represents the phoneme result of automatic speech recognition. Through this process, error-correcting corpus corresponding to the text phonemes is obtained.

3. The Burmese speech recognition error correction method according to claim 1, characterized in that: The specific steps of Step 2 are as follows: for a Myanmar speech recognition text sequence to be detected, represented as , n is the length of the text sequence, , the sub-word is modeled, and the input sequence of BERT is obtained after the embedding layer , the text encoding process is shown in formula (1): (1); is the encoded text feature sequence, is the th feature encoding representation, l is the length of the text sequence, is the feature dimension, and and represent the word embedding and position embedding functions, respectively.

4. The method of Myanmar speech recognition correction as claimed in claim 1, characterized in that: The specific steps of Step 3 are as follows: A phoneme encoder converts Burmese speech recognition text into a corresponding phoneme sequence and maps it to a continuous real-valued vector space by an embedding layer, resulting in ; Then, the phoneme embedding input encoder obtains the phoneme feature encoding; The process of obtaining phoneme encoding is represented by equation (2); (2); In equation (2), This represents phoneme coding features, while express No. The encoded feature information of each phoneme, where l is the length of the text sequence. The phoneme feature dimension is used; the phoneme encoder and text encoder adopt a shared parameter method to achieve information sharing between characters and phonemes in the input text sequence. and These represent word embedding and position embedding functions, respectively.

5. The Burmese speech recognition error correction method according to claim 1, characterized in that: The label prediction in Step 4 includes: A fully connected layer is used as input, which is the encoded text feature sequence, and the prediction result is represented as T. In the prediction result, "K" represents a correct character, "C" represents a space position error, substitution error, or deletion error, and the symbol "D" represents an insertion error. To train the label prediction model and achieve label prediction, a label predictor is constructed. The label predictor is used to automatically determine the label of characters in the transcribed text, ultimately achieving the goal of label prediction. The specific process of label prediction is as follows: (3); In formula (3), is a prediction operation, and are learnable parameters, and the operation loss is defined as minimizing a log-likelihood estimate.

6. The method of Myanmar speech recognition correction as claimed in claim 1, characterized in that: In Step 5, the phoneme context extractor consists of a bidirectional long short-term memory network (BiLSTM) and a linear layer. The BiLSTM is used to capture contextual information in the phoneme sequence and generate feature representations with contextual relevance, while the linear layer maps the hidden state output by the BiLSTM to a low-dimensional vector space and serves as the input to the multi-head attention mechanism.

7. The method of Myanmar speech recognition correction as claimed in claim 1, characterized in that: In Step 6, the construction process of the Burmese speech recognition text correction device is as follows: The Transformer decoder is used to generate the corrected text; the Transformer decoder uses a self-attention mechanism to compute the dependencies between different positions in the input sequence, thereby better capturing the contextual information of the text; In addition, a multi-head cross-attention mechanism is introduced to capture the correlation features between phoneme information and text, further extract phoneme information, and assist the decoder in correcting errors in space position and homophones.

8. The Burmese speech recognition error correction method according to claim 1, characterized in that: In Step 6, during the error correction operation, the input data of the label predictor is first represented in the form of a tuple SEN=(RAW,ASR), where RAW represents the original data and ASR represents the automatic speech recognition result. Furthermore, the input data of the phoneme feature extractor is represented in the form of a tuple PH=(RAW_phoneme,ASR_phoneme), where RAW_phoneme represents the original phoneme data and ASR_phoneme represents the phoneme result of the automatic speech recognition. By automatically extracting features from these two tuples using the model, the required input data is finally obtained.

9. A Burmese speech recognition and error correction device, characterized in that, Includes a module for performing the method as described in any one of claims 1-8.

Citation Information

Patent Citations

  • Speech recognition method and device, electronic equipment and computer readable storage medium

    CN115132175A