Speech recognition correction method and apparatus, electronic device, and storage medium

Abstract

The application provides a speech recognition error correction method and device, electronic equipment and a storage medium, wherein the method comprises: determining the recognized text of the speech data to be corrected; determining the acoustic features corresponding to each character in the recognized text based on the alignment position of each character in the speech data; and correcting the recognized text based on the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text. The speech recognition error correction method and device, electronic equipment and storage medium provided by the application not only use the semantic features of each character in the recognized text, but also use the acoustic features corresponding to each character. Compared with the related art which only considers semantic features, the application can capture the acoustic and semantic features of each character, fully utilizes multiple features to enhance the representation ability of the recognized text to be corrected, thereby improving the accuracy of error positioning and error correction.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a speech recognition error correction method, apparatus, electronic device, and storage medium. Background Technology

[0002] The accuracy of speech recognition is crucial for voice-based products and scenarios, such as voice input methods, meeting transcription, speech emotion recognition, and translation systems. Current speech recognition systems inevitably contain some recognition errors. Therefore, a robust error correction system is of great significance for the entire voice-based application landscape.

[0003] However, most current speech recognition error correction methods use text for modeling, and use text with errors as training data. The information available is relatively limited, making it difficult to accurately judge and correct errors, resulting in poor error correction performance. Summary of the Invention

[0004] This invention provides a speech recognition error correction method, device, electronic device, and storage medium to address the shortcomings of existing technologies that use text modeling, which makes accurate error judgment and correction difficult and results in poor error correction performance.

[0005] This invention provides a speech recognition error correction method, comprising:

[0006] Determine the recognized text of the speech data to be corrected;

[0007] Based on the alignment position of each character in the recognized text in the speech data, the acoustic features corresponding to each character in the recognized text are determined;

[0008] Based on the acoustic features corresponding to each character in the identified text and the semantic features of each character in the identified text, the identified text is corrected.

[0009] According to the speech recognition error correction method provided by the present invention, the step of correcting the recognized text based on the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text includes:

[0010] Based on the alignment position of each character in the recognized text in the speech data, the positional features of each character in the recognized text are determined;

[0011] The recognized text is corrected based on the acoustic features, positional features, and semantic features corresponding to each character in the recognized text.

[0012] According to the speech recognition error correction method provided by the present invention, the step of correcting the recognized text based on the acoustic features corresponding to each character in the recognized text, the positional features, and the semantic features includes:

[0013] The positional features of each character in the identified text are added to the semantic features to obtain the positional semantic features of each character in the identified text;

[0014] The positional semantic features of each character in the identified text are concatenated with the acoustic features to obtain the concatenated features of each character in the identified text.

[0015] Based on the splicing features of each character in the identified text, the identified text is corrected.

[0016] According to the speech recognition error correction method provided by the present invention, determining the recognition text of the speech data to be corrected includes:

[0017] Determine the initial recognized text of the speech data and display the initial recognized text;

[0018] Align the initial recognition text with the candidate recognition text corresponding to the speech data, determine the aligned initial recognition text as the recognition text of the speech data to be corrected, and display the recognition text;

[0019] The step of correcting errors in the identified text based on the acoustic features corresponding to each character and the semantic features of each character in the identified text includes:

[0020] In response to the user's selection of characters in the identified text, the character to be corrected is determined from the identified text;

[0021] Based on the acoustic features and semantic features of the character to be corrected, the character to be corrected is corrected.

[0022] The speech recognition error correction method provided by the present invention further includes:

[0023] When the character to be corrected is a special symbol without semantic meaning, the character to be corrected is corrected based on its alignment position in the candidate recognition text.

[0024] According to the speech recognition error correction method provided by the present invention, determining the acoustic features corresponding to each character in the recognized text based on the alignment position of each character in the speech data includes:

[0025] Acoustic features are extracted from the speech data to obtain the acoustic features of each speech frame.

[0026] Align the recognized text with the predicted text of each speech frame of the speech data to determine the alignment position of each character in the recognized text in the speech data;

[0027] From the acoustic features of each speech frame of the speech data, the acoustic features at the alignment position of each character in the recognized text in the speech data are selected as the acoustic features corresponding to each character in the recognized text.

[0028] According to the speech recognition error correction method provided by the present invention, the step of correcting the recognized text based on the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text includes:

[0029] Based on the speech recognition error correction model, the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text are applied to correct the errors in the recognized text.

[0030] The speech recognition error correction model is trained based on sample speech data, standard recognition text of the sample speech data, and candidate sample recognition text.

[0031] According to the speech recognition error correction method provided by the present invention, the sample speech data is obtained by speech synthesis of the standard recognition text, and the candidate sample recognition text is obtained by speech recognition of the sample speech data.

[0032] According to the speech recognition error correction method provided by the present invention, the candidate sample recognition text is obtained by adding perturbation to the standard recognition text, and the perturbation includes at least one of character replacement, insertion or deletion;

[0033] The replacement character is determined based on the similar pronunciation of each character in the text identified by the standard.

[0034] According to the speech recognition error correction method provided by the present invention, the speech recognition error correction model is trained based on the following steps:

[0035] Based on the contextual information of the sample text, the initial model is pre-trained to obtain a pre-trained model;

[0036] The pre-trained model is trained based on sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition text to obtain the speech recognition error correction model.

[0037] The present invention also provides a speech recognition error correction device, comprising:

[0038] The text recognition determination unit is used to determine the recognition text of the speech data to be corrected;

[0039] An acoustic feature determination unit is used to determine the acoustic features corresponding to each character in the recognized text based on the alignment position of each character in the speech data.

[0040] The error correction unit is used to correct errors in the identified text based on the acoustic features corresponding to each character in the identified text and the semantic features of each character in the identified text.

[0041] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the speech recognition error correction method as described above.

[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the speech recognition error correction method as described above.

[0043] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the speech recognition error correction method as described above.

[0044] The speech recognition error correction method, device, electronic device, and storage medium provided by this invention, when correcting errors in recognized text, not only use the semantic features of each character in the recognized text, but also the acoustic features corresponding to each character. Compared with related technologies that only consider semantic features, it can capture both acoustic and semantic features of each character, making full use of multiple features to enhance the representation ability of the recognized text to be corrected, thereby improving the accuracy of error location and error correction. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is one of the flowcharts of the speech recognition error correction method provided by the present invention;

[0047] Figure 2 This is a flowchart illustrating step 130 in the speech recognition error correction method provided by the present invention;

[0048] Figure 3 This is the second flowchart of the speech recognition error correction method provided by the present invention;

[0049] Figure 4 This is a flowchart illustrating step 120 in the speech recognition error correction method provided by the present invention;

[0050] Figure 5 This is a flowchart illustrating the speech recognition model training method provided by the present invention;

[0051] Figure 6 This is a schematic diagram of the structure of the speech recognition error correction model provided by the present invention;

[0052] Figure 7 This is a schematic diagram of the structure of the speech recognition error correction device provided by the present invention;

[0053] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] Automatic Speech Recognition (ASR), or simply speech recognition, is a technology that involves a computer processor receiving speech signals, processing them, and converting them into text information that humans can understand. This technology is widely used in mobile phone voice assistants, input method software, in-vehicle navigation, and various AI-powered wearable devices, demonstrating significant application value. Natural Language Processing (NLP), as a branch of artificial intelligence, uses techniques such as deep learning and machine learning to process and interpret textual data, typically encompassing areas like natural language understanding and natural language generation.

[0056] The accuracy of speech recognition is crucial for voice-based products and scenarios, such as voice input methods, meeting transcription, speech emotion recognition, and translation systems. However, current speech recognition systems inevitably contain some recognition errors. If a key word in a sentence is misidentified, it can lead to errors in the entire semantics, resulting in more serious errors in downstream tasks such as translation and sentiment analysis. Therefore, a robust error correction system is of great significance for all voice-based applications.

[0057] However, most current speech recognition error correction methods use text for modeling, and use text with errors as training data. The information available is relatively limited, making it difficult to accurately judge and correct errors, resulting in poor error correction performance.

[0058] Based on this, embodiments of the present invention provide a speech recognition error correction method to correct the text transcribed by the speech recognition system, thereby improving the accuracy of the speech recognition results and enabling speech recognition-based application products to have better robustness.

[0059] The application scenarios of the speech recognition error correction method provided in this invention include, but are not limited to, mobile phone voice assistants, input method software, in-vehicle navigation, and translation devices. It should be noted that, in addition to Chinese, this method is also applicable to other languages ​​such as Japanese, Korean, English, and Latin.

[0060] Figure 1 This is one of the flowcharts illustrating the speech recognition error correction method provided by the present invention. The execution entity for each step of this method can be a speech recognition error correction device, which can be implemented through software and / or hardware. This device can be integrated into electronic devices, including but not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, and in-vehicle terminals. Figure 1 As shown, the speech recognition error correction method may include the following steps:

[0061] Step 110: Determine the text to be recognized from the speech data to be corrected.

[0062] Specifically, the recognized text of the speech data to be corrected is the text obtained by performing speech recognition on the speech data and that needs to be corrected. For example, a piece of speech data labeled "iFlytek" may output recognized text such as "iFlytek Xunfei", "iFlytek Xunfei", or "Kodak iFlytek".

[0063] A voice acquisition device can be installed on or around a smart device to collect voice data. This voice acquisition device may include a microphone. After the voice data is acquired via a microphone array, it can be amplified and noise-reduced; however, this embodiment of the invention does not specifically limit the specific actions taken in this regard.

[0064] The recognized text of voice data can be obtained offline and / or online. For example, a voice recognition device can be pre-set on a smart device, and the device can output the recognized text of the voice data, thus achieving offline voice recognition. The voice recognition device may include a voice recognition model.

[0065] For example, a smart device can establish a network connection with a server, sending voice data to the server. The server's voice recognition device then outputs the recognized text, which the server can send back to the smart device, thus enabling online voice recognition. This server can include a cloud server.

[0066] It should be noted that the recognized text of the speech data to be corrected here can be the recognized text obtained directly from speech recognition of the speech data, or it can be obtained by further processing the recognized text obtained directly from speech recognition, such as the aligned text obtained after aligning the candidate recognized text corresponding to the speech recognition. This embodiment of the invention does not specifically limit this.

[0067] Step 120: Based on the alignment position of each character in the recognized text in the speech data, determine the acoustic features corresponding to each character in the recognized text.

[0068] Specifically, existing speech recognition error correction methods only use text, typically based on the semantic relationship between the erroneous recognized text and the standard text of the speech data. However, Encoder-Decoder based ASR models often produce outputs with significant acoustic differences from the speech data. For example, speech data labeled "functional department" might be incorrectly identified as "functional part." Without acoustic information, even if the error correction model can output a semantically reasonable new sentence, it still cannot resolve the problem of ASR recognition errors.

[0069] Considering the poor performance of error correction for this type of problem, the method provided in this embodiment of the invention aligns each character in the identified text with the speech data to obtain the alignment position of each character in the identified text in the speech data, thereby determining the acoustic features corresponding to each character in the identified text. The acoustic features corresponding to each character in the identified text obtained in this way can represent the acoustic features of the speech frame at the alignment position in the speech data.

[0070] Among them, recognizing the alignment position of each character in the text in the speech data can be achieved through audio-text alignment algorithms, such as Connectionist Temporal Classification (CTC) alignment and Hidden Markov Model (HMM) alignment.

[0071] Step 130: Correct errors in the identified text based on the acoustic features corresponding to each character in the identified text and the semantic features of each character in the identified text.

[0072] Specifically, the semantic features of each character can also be represented by word vectors or word embeddings, which are quantified representations of character information in vector form. Optionally, a word representation vector generation network can be used to generate a word representation vector for each character in the recognized text, i.e., the semantic features of each character.

[0073] The acoustic features and semantic features of each character in the identified text can be input into a pre-trained speech recognition error correction model. The trained speech recognition error correction model performs feature fusion on the acoustic and semantic features and performs error correction based on the fusion result, thereby outputting the corrected text. Alternatively, the acoustic features and semantic features of each character can be applied separately for error correction, and the corrected text can be determined by combining the error correction results obtained based on the acoustic features and the error correction results obtained based on the semantic features. This embodiment of the invention does not specifically limit the approach.

[0074] Furthermore, text correction can be performed on each character in the text, or on one or more characters. For example, the user can select a character in the text, and the correction will be performed on that character based on the user's selection. For characters that the user has not selected, no correction is required.

[0075] The speech recognition error correction method provided in this invention not only uses the semantic features of each character in the recognized text when correcting errors, but also uses the acoustic features corresponding to each character. Compared with related technologies that only consider semantic features, it can capture both acoustic and semantic features of each character, making full use of multiple features to enhance the representation ability of the recognized text to be corrected, thereby improving the accuracy of error location and error correction.

[0076] Based on the above embodiments, Figure 2 This is a flowchart illustrating step 130 of the speech recognition error correction method provided by the present invention, as follows: Figure 2 As shown, step 130 specifically includes:

[0077] Step 131: Based on the alignment position of each character in the recognized text in the speech data, determine the positional features of each character in the recognized text;

[0078] Step 132: Correct errors in the identified text based on the acoustic features, positional features, and semantic features corresponding to each character in the identified text.

[0079] Specifically, the positional features of each character in the text can be identified using a positional encoding network. The positional encoding network generates positional features for each character in the identified text based on their alignment position in the speech data. Optionally, the positional features can be quantized using position vectors to represent the positional information.

[0080] Based on the positional features of each character in the identified text, error correction can be performed on the identified text based on the acoustic features, positional features, and semantic features corresponding to each character. For example, the acoustic features, positional features, and semantic features corresponding to each character can be fused through a feature fusion network. The fusion weight parameters in the feature fusion network can be updated and learned, and error correction can be performed based on the fused features. Alternatively, feature fusion can be performed by vector addition or concatenation, and error correction can be performed based on the fused features. This embodiment of the invention does not specifically limit the specific methods used.

[0081] The speech recognition error correction method provided in this embodiment of the invention not only considers the acoustic and semantic features corresponding to each character when correcting the recognized text, but also combines the positional features of each character, making full use of multiple features to enhance the representation ability of the recognized text to be corrected, thereby further improving the accuracy of error location and error correction.

[0082] Based on any of the above embodiments Figure 3 This is the second flowchart of the speech recognition error correction method provided by the present invention, as shown below. Figure 3 As shown, step 132 specifically includes:

[0083] Step 132-1: Add the positional features and semantic features of each character in the identified text to obtain the positional semantic features of each character in the identified text;

[0084] Step 132-2: Concatenate the positional semantic features and acoustic features of each character in the identified text to obtain the concatenated features of each character in the identified text;

[0085] Step 132-3: Correct errors in the identified text based on the splicing features of each character in the identified text.

[0086] Specifically, to correct errors in the recognized text, the positional features and semantic features of each character in the recognized text are first added together to obtain the positional semantic features of each character. These positional semantic features provide both positional and semantic information for each character during text correction. Then, the positional semantic features of each character are concatenated with acoustic features to obtain the concatenated features of each character in the recognized text. Finally, errors are corrected based on these concatenated features.

[0087] Suppose the text to be identified contains n characters, where the semantic feature representation vector of the i-th character is W. i The position feature vector of the i-th character is P. i The acoustic feature representation vector corresponding to the i-th character is A. i n is an integer greater than 1, and i is a positive integer less than or equal to n.

[0088] Then the positional semantic feature representation vector of the i-th character is W. i +P i The vector addition method requires that the semantic feature representation vector W i and position feature representation vector P i The vector dimensions are the same, which facilitates calculation.

[0089] Based on this, the positional semantic feature representation vector W of the i-th character is... i +P i The acoustic feature representation vector A corresponding to the i-th character i If concatenation is performed, the concatenation feature representation vector of the i-th character is [W]. i +P i A i The vector concatenation method does not require the positional semantic feature representation vector W to be valid. i +P i Acoustic feature representation vector A i The vectors have the same dimension.

[0090] After obtaining the splicing features, the corrected text is calculated using a non-autoregressive method of Transformer.

[0091] The method provided in this invention fuses the acoustic features, positional features, and semantic features corresponding to each character in the identified text by using feature addition and concatenation based on the dimension of the feature representation vector. This method requires less computation and improves both the accuracy of error localization and error correction, while also increasing efficiency.

[0092] Based on any of the above embodiments, this embodiment of the invention provides a speech recognition error correction method, wherein step 110 specifically includes:

[0093] Step 111: Determine the initial recognition text of the speech data and display the initial recognition text.

[0094] Step 112: Align the initial recognition text with the candidate recognition text corresponding to the speech data, determine the aligned initial recognition text as the recognition text of the speech data to be corrected, and display the recognition text.

[0095] Accordingly, step 130 specifically includes:

[0096] Step 133: In response to the user's selection of characters in the recognized text, determine the characters to be corrected from the recognized text.

[0097] Step 134: Correct the character to be corrected based on the acoustic features and semantic features of the character to be corrected.

[0098] Specifically, considering that correcting every character in the recognized text may increase the probability that a correctly recognized character will be corrected as an incorrect character, it is possible to selectively correct one or more characters in the recognized text. For example, the user can choose to correct certain positions, while other unselected characters are left uncorrected.

[0099] After obtaining the voice data, the initial recognized text can be determined. In scenarios where the user inputs text through a terminal, the user inputs a segment of voice data, the voice data is recognized to obtain the initial recognized text, and this initial recognized text is displayed on the user's terminal.

[0100] The initial recognized text may contain recognition errors. After confirming that the user needs to correct these errors, the initial recognized text and the corresponding candidate recognized texts from the speech data are aligned. The purpose of aligning the initial recognized text with the corresponding candidate recognized texts is to confirm whether there are any missing characters in the initial recognized text. For example, if the initial recognized text contains 6 characters, while multiple candidate recognized texts contain 7 characters, then a missing character error is confirmed in the initial recognized text. The length of the aligned initial recognized text will become 7, and missing characters can be padded using a special symbol, such as "#". Of course, besides missing characters, other substitution errors may also exist.

[0101] Furthermore, the initial recognized text and multiple candidate recognized texts can be aligned in the time dimension using a minimum edit distance algorithm at both the character level and pronunciation level. Specifically, the longest text is selected as the anchor point. The remaining texts are used to calculate the minimum edit distance at the character level with the anchor text, and the edit path is recorded. If multiple edit paths exist, the edit path at the pronunciation level (converting the text to pronunciation) for each alignment method is calculated. The edit path with the smallest pronunciation level is selected for alignment. Finally, each alignment result is merged to obtain the aligned initial recognized text.

[0102] Then, the aligned initial recognition text is identified as the recognition text of the speech data to be corrected, and the recognition text is displayed.

[0103] After viewing the aligned recognized text on the terminal, the user can decide which characters need to be modified. For example, the user can manually click or perform other operations to select one or more characters in the recognized text. The speech recognition error correction device responds to the user's selection of characters in the recognized text and determines the characters to be corrected from the recognized text.

[0104] Based on this, errors can be corrected using the acoustic and semantic features of the character to be corrected. The acoustic features of the character to be corrected can be determined based on its position in the recognized text.

[0105] The method provided in this invention, by displaying the recognized text and determining the characters to be corrected based on the user's selection of characters in the recognized text, can reduce the negative impact that may be caused by correcting each character of the recognized text one by one, give the choice to the user, enhance the human-computer interaction experience, and further improve the accuracy of speech recognition error correction.

[0106] Based on any of the above embodiments, the speech recognition error correction method provided by the present invention further includes:

[0107] When the character to be corrected is a special symbol without semantic meaning, the character to be corrected is corrected based on its alignment position in the candidate recognition text.

[0108] Specifically, as described in the above embodiments, the identified text may include a special symbol without semantic meaning. This special symbol is used to fill in missing characters. When the user selects this special symbol, the character to be corrected is this special symbol, and the special symbol needs to be corrected.

[0109] In this case, the character to be corrected can be corrected based on its alignment position within multiple candidate texts. If a corresponding character exists at the alignment position in the candidate text, the special character can be corrected to the character at that alignment position. Of course, it's also possible that the field will be empty.

[0110] For example, a voice data segment labeled "How is the weather today?" will result in the initial recognized text "How is the weather today?", while other candidate recognized texts will be "How is the weather today?", "How is the weather today?", and "How is the weather today?".

[0111] The initial identified text is aligned with multiple candidate identified texts. The identified text after alignment is "Today's # Weather is fine", and this identified text is then sent back to the user's terminal. Here, "#" is a special symbol without semantic meaning.

[0112] At this time, the user can click on "#", and then, based on other candidate recognized texts, correct "#" to the correct character "天". If the user believes that "莫" may be incorrect, the user can also click on and select "莫", and then, based on the acoustic features and semantic features corresponding to "莫", correct "莫" to the correct character "么".

[0113] For the correct character "样" in the recognized text, instead of using automatic error correction, the user is allowed to decide whether to correct at this position, thus avoiding the possible situation of modifying the correct "样" to the incorrect "杨".

[0114] In the method provided by the embodiment of the present invention, in the case where the character to be error-corrected is a special symbol without semantics, based on the alignment position of the character to be error-corrected in multiple candidate recognized texts, error correction is performed on the character to be error-corrected, realizing error correction for a single character specified by the user, avoiding the negative impacts that may be caused by error-correcting each character in the recognized text one by one, leaving the option to the user to decide, enhancing the human-computer interaction experience while further improving the accuracy of speech recognition error correction.

[0115] Based on any of the above embodiments Figure 4 is a schematic flowchart of step 120 in the speech recognition error correction method provided by the present invention, as Figure 4 shown, step 120 specifically includes:

[0116] Step 121, extract acoustic features from the speech data to obtain the acoustic features of each speech frame of the speech data;

[0117] Step 122, align the recognized text with the predicted text of each speech frame of the speech data to determine the alignment position of each character in the recognized text in the speech data;

[0118] Step 123, select the acoustic features at the alignment positions of each character in the recognized text in the acoustic features of each speech frame of the speech data as the acoustic features corresponding to each character in the recognized text.

[0119] Specifically, extracting acoustic features from the speech data can be implemented by a trained ASR model based on the Encoder-Decoder structure. Use the output of the Encoder end as the acoustic features of each speech frame of the speech data.

[0120] Before executing step 121, a trained ASR model based on the Encoder-Decoder structure can be obtained in advance. Figure 5 is a schematic flowchart of the speech recognition model training method provided by the present invention, as Figure 5As shown, when training the ASR model, a loss function constraint needs to be added to the encoder. That is, the output of the encoder needs to go through a classification layer, and then the predicted text output by the classification layer and the labeled text of the speech data are used to calculate a CTC loss function and participate in the update of the model parameters.

[0121] Figure 5 In (a), the loss function at the Decoder end is no different from that of a conventional Encoder-Decoder based ASR model, using the cross-entropy loss function as a constraint. The addition of the CTC loss function at the Encoder end makes the acoustic features of the Encoder output more concentrated; that is, the acoustic features corresponding to each word in the speech data are concentrated in a small area, and the corresponding classification layer output has a distinct peak shape, such as... Figure 5 As shown in (b).

[0122] After obtaining the acoustic features of each speech frame in the speech data, the CTC alignment algorithm is used to align the recognized text with the predicted text of each speech frame in the speech data, determining the alignment position of each character in the recognized text within the speech data. Then, from the acoustic features of each speech frame in the speech data, the acoustic features at the alignment positions of each character in the recognized text within the speech data are selected as the corresponding acoustic features for each character in the recognized text. This can be achieved through the following steps:

[0123] 1) Calculate the Classification output at the Encoder end to obtain the T×V matrix enc_out, which can be represented in the following form:

[0124] enc_out = F·E(x)

[0125] Where F represents the classification layer, E represents the encoder, x represents the speech data, T represents the frame length of the speech data, and V represents the number of categories, i.e., the vocabulary size.

[0126] 2) Align the recognized text sequentially on enc_out. For a text sequence of length m, place it sequentially on enc_out of length T. Each character's position corresponds to a vector of size V. The value of the character's category at the corresponding position in that vector is the character's score at that position, denoted as Score. ij The final score for the entire recognized text sequence can be expressed in the following form:

[0127]

[0128] Where, pos i-1 Score represents the position of the (i-1)th character in the recognized text within the aligned speech data. ijdenoted by , m represents the score of the i-th character in the recognized text at the j-th position in the speech data, m represents the length of the recognized text, and T represents the frame length of the speech data.

[0129] Iterate through all valid permutations and combinations, and select the combination with the highest overall score as the CTC alignment method. This alignment includes recognizing the most accurate acoustic pronunciation position of the text. The resulting alignment position can be denoted as Pos = {pos1, pos2, ..., pos...} m}

[0130] Next, from the acoustic features of each speech frame in the speech data, the acoustic features at the alignment positions of each character in the recognized text are selected as the acoustic features corresponding to each character in the recognized text. The acoustic features corresponding to each character in the recognized text can be denoted as A = {A1, A2, ..., A...}. m The alignment process does not consider the placeholder "#".

[0131] The speech recognition error correction method provided in this invention provides a foundation for improving the accuracy of error correction by selecting the acoustic features of the alignment position of each character in the recognized text in the speech data from the acoustic features of each speech frame of the speech data, and using these features as the acoustic features corresponding to each character in the recognized text.

[0132] Based on any of the above embodiments, step 130 specifically includes:

[0133] Based on the speech recognition error correction model, the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text are applied to correct errors in the recognized text;

[0134] The speech recognition error correction model is trained based on sample speech data, standard recognition text of the sample speech data, and candidate sample recognition text.

[0135] Specifically, error localization and correction of recognized text can be achieved based on a speech recognition error correction model. Figure 6 This is a schematic diagram of the structure of the speech recognition error correction model provided by the present invention, as shown below. Figure 6 As shown, the speech recognition error correction model can be selected as a Transformer structure with 12 layers, a hidden layer size of 1024, and 12 heads for multi-head attention. The speech recognition error correction model may include a word encoding layer, a positional encoding layer, a feature addition layer, a feature concatenation layer, a linear layer, a high-dimensional feature extraction layer, and a classification layer.

[0136] The text encoding layer encodes the characters in the text to be corrected, resulting in a semantic feature representation vector. The positional encoding layer encodes the alignment of each character in the text within the speech data, resulting in a positional feature representation vector. Both the character encoding layer and the positional encoding layer are learnable and have the same encoding dimension, which can be 512 dimensions.

[0137] The feature addition layer adds the semantic feature representation vector output by the character encoding layer to the position feature representation vector output by the position encoding layer. The feature concatenation layer concatenates the character encoding information with added position information along the feature dimension (512) to obtain a 1024-dimensional vector. This vector contains the character information of each character in the recognized text, corresponding to the position information and acoustic information in the speech data.

[0138] The high-dimensional feature extraction layer can be a Transformer network, which can extract high-dimensional information from the recognized text. Then, autoregressive or non-autoregressive methods are used to calculate this high-dimensional information. The autoregressive approach primarily uses an attention mechanism to perform a weighted average calculation of the high-dimensional information, and then uses a classifier to classify the result. The non-autoregressive approach adds a classification layer at the output of the Transformer to directly classify the output at that location, hoping to obtain the correct classification result.

[0139] Due to the similarity between the input and output of the error correction task (a good ASR recognition model has a low error rate), autoregressive error correction schemes have difficulty learning contextual information. The model often learns a one-to-one mapping relationship, which makes it impossible to truly achieve the error correction effect.

[0140] Therefore, this invention uses a non-autoregressive method based on Transformer for error correction. After calculation by Transformer, the text is then classified by a classification layer to obtain the corrected text.

[0141] Prior to this, a speech recognition error correction model can be pre-trained. For example, the speech recognition error correction model can be trained as follows: First, acquire a large amount of sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition text of the sample speech data. The candidate sample recognition text can be one or more, and this embodiment of the invention does not specifically limit this. Then, train an initial model based on the sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition text, thereby obtaining the speech recognition error correction model.

[0142] like Figure 6 As shown, the candidate sample identifies 4 texts, pos 11This indicates the position of the first character in the text identified by the first candidate sample after alignment, corresponding to the position after CTC alignment. 23 This indicates the position of the third character in the text identified by the second candidate sample after alignment, corresponding to the position after CTC alignment. A 12 A represents the acoustic feature corresponding to the second character in the text identified by the first candidate sample. 35 This represents the acoustic feature corresponding to the fifth character in the text identified by the third candidate sample. This information is a 512-dimensional vector.

[0143] To enable the Transformer model to be trained, the four 1024-dimensional vectors at each time step are fused together and transformed through a linear layer from 4096 to 1024. The transformed 1024-dimensional vector contains word information, position information, and acoustic information of the text recognized by the four candidate samples.

[0144] Existing techniques typically involve first collecting parallel data containing speech data and corresponding standard recognition text, then using a pre-trained ASR model to transcribe the speech data, obtaining erroneous recognition text, which is then used as training data samples, while the standard recognition text serves as the label for supervised learning. However, the sample data only contains erroneous recognition text, and the model training process lacks acoustic information references.

[0145] The method provided in this invention trains an initial model based on sample speech data, standard recognition text of the sample speech data, and candidate sample recognition text. The resulting speech recognition error correction model can capture both acoustic and semantic features of each character in the candidate sample recognition text, making full use of multiple features to enhance the representation ability of the recognition text to be corrected, thereby improving the accuracy of error localization and error correction.

[0146] Based on any of the above embodiments, the sample speech data is obtained by speech synthesis of standard recognition text, and the candidate sample recognition text is obtained by speech recognition of sample speech data.

[0147] Specifically, considering the high cost of acquiring parallel audio-text data and the limited overall training data, the sample speech data provided in this embodiment of the invention is obtained by synthesizing standard recognition text. A large amount of plain text data is pre-acquired and used as the standard recognition text. A pre-trained TTS (text-to-speech) model and the standard recognition text are used to synthesize corresponding audio. The synthesized audio is then used as sample speech data, thereby increasing the amount of sample data and improving the accuracy of error correction.

[0148] The prior art usually corrects the optimal solution of ASR. However, sometimes the correct decoding result exists in the candidate solutions of ASR, and the existing error correction schemes lack the utilization of such information. Therefore, when obtaining the sample recognition text, Beam Search decoding is performed on the sample speech data to obtain n-best candidate texts (n represents the size of the candidate set). Thus, n candidate sample recognition texts are obtained.

[0149] The ASR Beam Search decoding algorithm is different from the greedy algorithm and expands the search space. For example, when the space size is set to n, at the initial moment, the optimal n results are selected as the optimal path at the current time step, and then n*n solutions are searched from the current optimal n states, and n optimal solutions are obtained by sorting. The above process is repeated until the search end conditions are met for all n states. For example, for a voice labeled as "iFlytek", the results obtained by using 4-best Beam Search decoding may be: "iFlytek", "iFlyxunfei", "iFlyxunfei", "Kodak iFlytek".

[0150] The method provided in the embodiment of the present invention synthesizes the standard recognition text into sample speech data, and performs speech recognition on the sample speech data to obtain candidate sample recognition texts, thereby conveniently obtaining a large amount of sample data and improving the accuracy of speech recognition error correction.

[0151] Based on any of the above embodiments, the candidate sample recognition text is obtained by adding perturbations to the standard recognition text. The added perturbations include at least one of character replacement, insertion, or deletion; wherein, the replacement characters are determined based on the similar pronunciations of the characters in the standard recognition text.

[0152] Specifically, the candidate sample recognition text can be obtained not only by decoding the n-best candidate text through ASR, but also by adding perturbations to the standard recognition text. The added perturbations include at least one of replacing, inserting, or deleting characters in the standard recognition text. Among them, the replacement characters are determined based on the similar pronunciations of the characters in the standard recognition text.

[0153] Adding perturbations to the standard recognition text can be achieved through the following steps:

[0154] 1) Collect the decoding results of the test set of the pre-trained ASR model, and count the proportions of substitution errors, insertion errors, and deletion errors therein.

[0155] 2) Construct a table of similar pronunciation rules. Taking Chinese as an example, Table 1 is the table of similar pronunciation rules. As shown in Table 1, the initials b and p belong to similar pronunciations. The initial of a Chinese character containing the initial b in the standard recognition text can be replaced with p to form another Chinese character. For another example, the finals ai and ei belong to similar pronunciations. The final of a Chinese character containing the final ai in the standard recognition text can be replaced with ei to form another Chinese character.

[0156] Table 1

[0157]

[0158]

[0159] 3) Construct substitution errors according to the similar pronunciation rules. For example, the pronunciation of "您" is "nin", which contains the initial "n" and the final "in". By replacing the initial "n" with "l", "您" can be replaced with "林", "临" and other Chinese characters with similar pronunciations; or by replacing the final "in" with "ing", "您" can be replaced with "宁", etc. This construction of substitution errors can, to a certain extent, simulate the incorrect recognition of similar acoustic information by ASR.

[0160] 4) Construct some deletion and insertion errors by randomly deleting and inserting random characters. The construction ratios of the three types of errors all refer to the real ratios statistically obtained in the aforementioned 1).

[0161] The method provided by the embodiment of the present invention obtains the candidate sample recognition text by performing at least one of character substitution, deletion or insertion on the standard recognition text, increasing the data volume of the samples, and thus improving the error correction accuracy of the speech recognition error correction model.

[0162] Based on any of the above embodiments, in the case of multiple candidate sample recognition texts, the multiple candidate sample recognition texts can be further aligned.

[0163] For the n candidate sample recognition texts obtained by ASR decoding, the minimum edit distance algorithm can be used to align the n candidate sample recognition texts in the time dimension. The specific operation is as follows: Select the longest text in the candidate sample recognition texts as the anchor point, calculate the minimum edit distance between the remaining n - 1 candidate sample recognition texts and the anchor point text and record the edit path, and finally merge each alignment result to obtain the aligned n texts. For example, for the sample speech data labeled as "在职能部门", the 4-best decoding results are, in order, "职工部分", "职能部门", "在职能部门", "职能部们". The alignment result of these four candidate sample recognition texts can be shown as in Table 2, where "#" represents the placeholder for padding.

[0164] Table 2

[0165] talent Job work department point # Job able department Door exist Job able department Door # Job able department They

[0166] For the n candidate sample recognition texts obtained by adding perturbations to the standard recognition text, construct substitution errors according to the similar pronunciation rule table, and at the same time construct some random insertion errors and deletion errors. The insertion errors can be to copy the previous character or randomly insert a certain character, and then align the n texts. For the above standard recognition text "in the functional department", the result after aligning the 4 candidate sample recognition texts constructed can be shown as Table 3.

[0167] Table 3

[0168] vegetable Job able # Door exist Job able No Door exist Job Job department Door # Job able department Door

[0169] Based on any of the above embodiments, the speech recognition error correction model is trained based on the following steps:

[0170] Based on the context information of the sample text, pre-train the initial model to obtain a pre-trained model;

[0171] Based on the sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition texts, train the pre-trained model to obtain a speech recognition error correction model.

[0172] Specifically, considering that the speech recognition error correction model corrects a character at a certain position, in addition to using the character information, position information, and acoustic information at that position, sometimes it is also necessary to have a good understanding of the context information. For example, in "at this moment of thunderstorm", "class" is misrecognized as "moment" by ASR. At this time, the error correction of the model requires good context analysis ability. Therefore, before training the speech recognition error correction model, pre-train the initial model. In order to reduce the training cost, pure text is used during pre-training, and the sample text is not specifically limited and can be any text. Based on the context information of the sample text, pre-train the initial model. Further, the pre-training can be performed in the way of Mask Language Model, and the obtained pre-trained model has good context understanding and analysis ability.

[0173] Finally, based on the sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition texts, train the pre-trained model to obtain a speech recognition error correction model.

[0174] Based on any of the above embodiments, a speech recognition error correction method is provided, including:

[0175] S1, determine the recognition text of the speech data to be error-corrected.

[0176] S2, perform acoustic feature extraction on the speech data to obtain the acoustic features of each speech frame of the speech data; align the recognized text with the predicted text of each speech frame of the speech data to determine the alignment position of each character in the recognized text in the speech data; select the acoustic features at the alignment position of each character in the recognized text in the speech data from the acoustic features of each speech frame of the speech data as the acoustic features corresponding to each character in the recognized text.

[0177] S3. Based on the speech recognition error correction model, the alignment position of each character in the recognized text in the speech data is applied to determine the positional features of each character in the recognized text; the acoustic features corresponding to each character in the recognized text, the semantic features of each character in the recognized text, and the positional features are applied to correct errors in the recognized text.

[0178] The speech recognition error correction model is trained based on sample speech data, standard recognition text of the sample speech data, and candidate sample recognition text.

[0179] The sample speech data is obtained by speech synthesis from the standard recognition text, while the candidate sample recognition text is obtained by speech recognition from the sample speech data.

[0180] The candidate sample recognition text is obtained by adding perturbations to the standard recognition text. The perturbations include at least one of character replacement, insertion, or deletion; wherein, the replacement character is determined based on the similar pronunciation of each character in the standard recognition text.

[0181] Based on any of the above embodiments, another embodiment of the present invention provides a speech recognition error correction method, including:

[0182] Determine the initial recognized text from the speech data and display the initial recognized text;

[0183] Align the initial recognition text with the candidate recognition text corresponding to the speech data, determine the aligned initial recognition text as the recognition text of the speech data to be corrected, and display the recognition text.

[0184] Based on the alignment position of each character in the recognized text in the speech data, the acoustic features corresponding to each character in the recognized text are determined; in response to the user's selection operation of characters in the recognized text, the characters to be corrected are determined from the recognized text.

[0185] Based on the acoustic features and semantic features of the character to be corrected, the character to be corrected is corrected.

[0186] When the character to be corrected is a special symbol without semantic meaning, the character to be corrected is corrected based on its alignment position in multiple candidate recognition texts.

[0187] The speech recognition error correction device provided by the present invention is described below. The speech recognition error correction device described below can be referred to in correspondence with the speech recognition error correction method described above.

[0188] Figure 7 This is a schematic diagram of the speech recognition error correction device provided by the present invention, as shown below. Figure 7 As shown, the speech recognition error correction device includes a text recognition determination unit 710, an acoustic feature determination unit 720, and an error correction unit 730.

[0189] The text recognition unit is used to determine the text to be recognized in the speech data to be corrected.

[0190] An acoustic feature determination unit is used to determine the acoustic features corresponding to each character in the recognized text based on the alignment position of each character in the speech data.

[0191] The error correction unit is used to correct errors in the identified text based on the acoustic features corresponding to each character in the identified text and the semantic features of each character in the identified text.

[0192] The speech recognition error correction device provided in this embodiment of the invention not only uses the semantic features of each character in the recognized text when correcting errors, but also uses the acoustic features corresponding to each character. Compared with related technologies that only consider semantic features, it can capture both acoustic and semantic features of each character, making full use of multiple features to enhance the representation ability of the recognized text to be corrected, thereby improving the accuracy of error location and error correction.

[0193] Based on any of the above embodiments, the error correction unit is further configured to:

[0194] Based on the alignment position of each character in the recognized text in the speech data, the positional features of each character in the recognized text are determined;

[0195] The recognized text is corrected based on the acoustic features, positional features, and semantic features corresponding to each character in the recognized text.

[0196] Based on any of the above embodiments, the error correction unit is further configured to:

[0197] The positional features of each character in the identified text are added to the semantic features to obtain the positional semantic features of each character in the identified text;

[0198] The positional semantic features of each character in the identified text are concatenated with the acoustic features to obtain the concatenated features of each character in the identified text.

[0199] Based on the splicing features of each character in the identified text, the identified text is corrected.

[0200] Based on any of the above embodiments, the text recognition and determination unit is further configured to:

[0201] Determine the initial recognized text of the speech data and display the initial recognized text;

[0202] Align the initial recognition text with the candidate recognition text corresponding to the speech data, determine the aligned initial recognition text as the recognition text of the speech data to be corrected, and display the recognition text;

[0203] Accordingly, the error correction unit is further used for:

[0204] In response to the user's selection of characters in the identified text, the character to be corrected is determined from the identified text;

[0205] Based on the acoustic features and semantic features of the character to be corrected, the character to be corrected is corrected.

[0206] Based on any of the above embodiments, a character error correction unit is further included, for:

[0207] When the character to be corrected is a special symbol without semantic meaning, the character to be corrected is corrected based on its alignment position in the multiple candidate recognition texts.

[0208] Based on any of the above embodiments, the acoustic feature determination unit is further configured to:

[0209] Acoustic features are extracted from the speech data to obtain the acoustic features of each speech frame.

[0210] Align the recognized text with the predicted text of each speech frame of the speech data to determine the alignment position of each character in the recognized text in the speech data;

[0211] From the acoustic features of each speech frame of the speech data, the acoustic features at the alignment position of each character in the recognized text in the speech data are selected as the acoustic features corresponding to each character in the recognized text.

[0212] Based on any of the above embodiments, the error correction unit is further configured to:

[0213] Based on the speech recognition error correction model, the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text are applied to correct the errors in the recognized text.

[0214] The speech recognition error correction model is trained based on sample speech data, standard recognition text of the sample speech data, and candidate sample recognition text.

[0215] Based on any of the above embodiments, the sample speech data is obtained by speech synthesis of the standard recognition text, and the candidate sample recognition text is obtained by speech recognition of the sample speech data.

[0216] Based on any of the above embodiments, the candidate sample recognition text is obtained by adding perturbation to the standard recognition text, and the perturbation includes at least one of character replacement, insertion, or deletion; wherein, the replacement character is determined based on the similar pronunciation of each character in the standard recognition text.

[0217] Based on any of the above embodiments, the speech recognition error correction device further includes a model training unit, used for:

[0218] Based on the contextual information of the sample text, the initial model is pre-trained to obtain a pre-trained model;

[0219] The pre-trained model is trained based on sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition text to obtain the speech recognition error correction model.

[0220] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a speech recognition error correction method. This method includes: determining the recognition text of the speech data to be corrected; determining the acoustic features corresponding to each character in the recognition text based on the alignment position of each character in the speech data; and correcting the recognition text based on the acoustic features corresponding to each character in the recognition text and the semantic features of each character in the recognition text.

[0221] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0222] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the speech recognition error correction method provided by the above methods. The method includes: determining the recognition text of the speech data to be corrected; determining the acoustic features corresponding to each character in the recognition text based on the alignment position of each character in the speech data; and correcting the recognition text based on the acoustic features corresponding to each character in the recognition text and the semantic features of each character in the recognition text.

[0223] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the speech recognition error correction method provided by the above methods. The method includes: determining the recognition text of the speech data to be corrected; determining the acoustic features corresponding to each character in the recognition text based on the alignment position of each character in the speech data; and correcting the recognition text based on the acoustic features corresponding to each character in the recognition text and the semantic features of each character in the recognition text.

[0224] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0225] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0226] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A speech recognition error correction method, characterized in that, include: Determine the recognized text of the speech data to be corrected; Based on the alignment position of each character in the recognized text in the speech data, the acoustic features corresponding to each character in the recognized text are determined; Based on the acoustic features corresponding to each character in the identified text and the semantic features of each character in the identified text, the identified text is corrected. The step of correcting errors in the identified text based on the acoustic features corresponding to each character and the semantic features of each character in the identified text includes: Based on the alignment position of each character in the recognized text in the speech data, the positional features of each character in the recognized text are determined; The positional features of each character in the identified text are added to the semantic features to obtain the positional semantic features of each character in the identified text; The positional semantic features of each character in the identified text are concatenated with the acoustic features to obtain the concatenated features of each character in the identified text. Based on the splicing features of each character in the identified text, the identified text is corrected.

2. The speech recognition error correction method according to claim 1, characterized in that, The text to be identified from the speech data to be corrected includes: Determine the initial recognized text of the speech data and display the initial recognized text; Align the initial recognition text with the candidate recognition text corresponding to the speech data, determine the aligned initial recognition text as the recognition text of the speech data to be corrected, and display the recognition text; The step of correcting errors in the identified text based on the acoustic features corresponding to each character and the semantic features of each character in the identified text includes: In response to the user's selection of characters in the identified text, the character to be corrected is determined from the identified text; Based on the acoustic features and semantic features of the character to be corrected, the character to be corrected is corrected.

3. The speech recognition error correction method according to claim 2, characterized in that, Also includes: When the character to be corrected is a special symbol without semantic meaning, the character to be corrected is corrected based on its alignment position in the candidate recognition text.

4. The speech recognition error correction method according to any one of claims 1-3, characterized in that, The step of determining the acoustic features corresponding to each character in the recognized text based on the alignment position of each character in the speech data includes: Acoustic features are extracted from the speech data to obtain the acoustic features of each speech frame. Align the recognized text with the predicted text of each speech frame of the speech data to determine the alignment position of each character in the recognized text in the speech data; From the acoustic features of each speech frame of the speech data, the acoustic features at the alignment position of each character in the recognized text in the speech data are selected as the acoustic features corresponding to each character in the recognized text.

5. The speech recognition error correction method according to claim 1, characterized in that, The step of correcting errors in the identified text based on the acoustic features corresponding to each character and the semantic features of each character in the identified text includes: Based on the speech recognition error correction model, the acoustic features corresponding to each character in the recognized text and the semantic features of each character in the recognized text are applied to correct the errors in the recognized text. The speech recognition error correction model is trained based on sample speech data, standard recognition text of the sample speech data, and candidate sample recognition text.

6. The speech recognition error correction method according to claim 5, characterized in that, The sample speech data is obtained by speech synthesis from the standard recognition text, and the candidate sample recognition text is obtained by speech recognition from the sample speech data.

7. The speech recognition error correction method according to claim 5, characterized in that, The candidate sample recognition text is obtained by adding perturbation to the standard recognition text, and the perturbation includes at least one of character replacement, insertion, or deletion; The replacement character is determined based on the similar pronunciation of each character in the text identified by the standard.

8. The speech recognition error correction method according to any one of claims 5-7, characterized in that, The speech recognition error correction model is trained based on the following steps: Based on the contextual information of the sample text, the initial model is pre-trained to obtain a pre-trained model; The pre-trained model is trained based on sample speech data, the standard recognition text of the sample speech data, and the candidate sample recognition text to obtain the speech recognition error correction model.

9. A speech recognition error correction device, characterized in that, include: The text recognition determination unit is used to determine the recognition text of the speech data to be corrected; An acoustic feature determination unit is used to determine the acoustic features corresponding to each character in the recognized text based on the alignment position of each character in the speech data. The error correction unit is used to correct errors in the identified text based on the acoustic features corresponding to each character in the identified text and the semantic features of each character in the identified text. The device is also used for: Based on the alignment position of each character in the recognized text in the speech data, the positional features of each character in the recognized text are determined; The positional features of each character in the identified text are added to the semantic features to obtain the positional semantic features of each character in the identified text; The positional semantic features of each character in the identified text are concatenated with the acoustic features to obtain the concatenated features of each character in the identified text. Based on the splicing features of each character in the identified text, the identified text is corrected.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the speech recognition error correction method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the speech recognition error correction method as described in any one of claims 1 to 8.

Images