Speech recognition error correction method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a speech recognition error correction method, which comprises the following steps of: training and generating a target field Chinese character text understanding model by collecting dialogue Chinese character texts of a target field; collecting pinyin data, and training to generate a pinyin understanding model; receiving to-be-detected voice data and converting the to-be-detected voice data into to-be-detected Chinese character texts and pinyin texts Inputting the Chinese character text and the pinyin text to corresponding understanding models to generate semantic features and pinyin features; combining the semantic features and the pinyin features to form joint features; inputting the joint features into a speech recognition error detection model, generating error probability distribution of each word, and outputting an error tag; and correcting the Chinese character text through a template error correction module based on the error tag. According to the method, more comprehensive feature representation is formed by combining semantic features and pinyin features, and the accuracy of speech recognition error detection is improved; and a template error correction module is utilized to effectively correct identification errors.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a speech recognition error correction method, device, equipment and storage medium. Background Art

[0002] With the continuous development of smart home technology, the application of voice dialogue systems has gradually expanded to multiple fields, such as home-based elderly care, financial services and other scenarios. As a voice control device, the home-based elderly care speaker dialogue system can help elderly users easily control speakers and other smart sensor devices, and realize the convenience of voice interaction. However, in the process of voice-to-text conversion, due to the accent of elderly users, the decline in voice quality and other problems, the voice recognition system often cannot accurately identify the user's intention, resulting in the system being unable to complete the user's desired operation. This problem seriously affects the experience of elderly users and reduces the actual application effect of the system.

[0003] In financial scenarios, speech recognition systems also face many challenges, especially in applications such as customer service and voice authentication. Financial services usually involve functions such as customer identity authentication and transaction inquiries. The recognition accuracy of the voice dialogue system is directly related to the user's transaction security and service experience. However, factors such as differences in user pronunciation, background noise, and limitations of voice devices often lead to recognition errors, which in turn affect the accuracy and satisfaction of customer service.

[0004] At present, the recognition accuracy of speech recognition systems in ideal environments has been significantly improved, but there are still some challenges that cannot be ignored in practical applications. For example, due to factors such as insufficient vocabulary coverage, poor user pronunciation quality, and environmental noise interference, speech recognition systems are prone to misrecognition. Especially among the elderly, the error rate of speech recognition is significantly increased due to factors such as unclear pronunciation, changes in speech speed, and fluctuations in pitch. These misrecognitions directly affect the accuracy of the intention recognition of the voice dialogue system, further limiting the practicality of the system in elderly care scenarios. Similarly, in financial scenarios, dialect accents or non-standard pronunciations can also cause system misrecognition, which in turn affects the accuracy of identity authentication and may even lead to potential security risks.

[0005] Traditional automatic speech recognition (ASR) error correction methods mainly rely on statistical language models, dictionary matching, and rule sets to perform corrections by analyzing the text context. Although these methods have improved the error correction effect to a certain extent, they still have significant limitations when dealing with complex contexts or accents. With the development of deep learning technology, end-to-end ASR error correction methods based on deep learning have gradually emerged and achieved good results in general scenarios. However, such methods face the problem of data mismatch in elderly care scenarios and financial service scenarios, especially the lack of adaptability to the voice features unique to elderly users or the financial service field.

[0006] In addition, these methods often rely only on semantic context for error correction, and do not fully utilize speech pronunciation information, resulting in limited error correction capabilities in the case of specific accents and pronunciation errors. Therefore, when dealing with the problem of speech recognition for the elderly, existing technologies still have a lot of room for optimization and technical requirements. Summary of the invention

[0007] The main purpose of the present invention is to provide a speech recognition error correction method, device, equipment and storage medium, aiming to solve the technical problem that the prior art fails to effectively combine semantic features and phonetic features for error correction, resulting in insufficient error correction ability when processing pronunciation errors.

[0008] To achieve the above object, the present invention provides a speech recognition error correction method, comprising:

[0009] Collecting dialogue Chinese texts in a target domain, and training and generating a target domain Chinese text understanding model through the dialogue Chinese texts;

[0010] Collecting pinyin data, and training and generating a pinyin understanding model through the pinyin data;

[0011] Receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected;

[0012] Converting the Chinese character text to be detected into a pinyin text to be detected;

[0013] Inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features;

[0014] Inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features;

[0015] Merging the semantic feature and the phonetic feature to generate a joint feature;

[0016] Input the joint feature into a speech recognition error detection model, generate an error probability distribution for each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and output a corresponding error label;

[0017] Based on the error label, the Chinese character text to be detected is corrected by a template error correction module.

[0018] Furthermore, to achieve the above object, the present invention provides a speech recognition error correction device, comprising:

[0019] A Chinese character text understanding module is used to collect dialogue Chinese character texts in a target domain and generate a target domain Chinese character text understanding model through training of the dialogue Chinese character texts;

[0020] A pinyin understanding module, used to collect pinyin data and generate a pinyin understanding model through training of the pinyin data;

[0021] A voice data conversion module, used for receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected;

[0022] A Chinese character to pinyin conversion module, used for converting the Chinese character text to be detected into a pinyin text to be detected;

[0023] A semantic feature extraction module, used for inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features;

[0024] A pinyin feature extraction module, used for inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features;

[0025] A feature merging module, used to merge the semantic feature and the phonetic feature to generate a joint feature;

[0026] An error detection module, used for inputting the joint feature into a speech recognition error detection model, generating an error probability distribution of each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and outputting a corresponding error label;

[0027] The template error correction module is used to correct the Chinese character text to be detected based on the error label through the template error correction module.

[0028] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a speech recognition error correction program stored in the memory and executable on the processor, and the speech recognition error correction program, when executed by the processor, implements the steps of the speech recognition error correction method as described above.

[0029] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a speech recognition error correction program is stored, and when the speech recognition error correction program is executed by a processor, the steps of the speech recognition error correction method as described above are implemented.

[0030] Beneficial effect: The present invention relates to the field of artificial intelligence technology, and discloses a speech recognition error correction method, which collects dialogue Chinese character texts in a target field, trains and generates a target field Chinese character text understanding model; collects pinyin data, trains and generates a pinyin understanding model; receives speech data to be detected and converts it into Chinese character text and pinyin text to be detected; inputs Chinese character text and pinyin text into corresponding understanding models, generates semantic features and pinyin features; merges semantic features and pinyin features to form joint features; inputs joint features into a speech recognition error detection model, generates an error probability distribution for each word, and outputs an error label; based on the error label, corrects the Chinese character text through a template error correction module. The present invention forms a more comprehensive feature representation by combining semantic features and pinyin features, thereby improving the accuracy of speech recognition error detection; and using a template error correction module, it is possible to effectively correct recognition errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0032] Figure 1 A schematic diagram of an application environment of a speech recognition error correction method in an embodiment of the present invention;

[0033] Figure 2 A flow chart of an embodiment of a speech recognition error correction method of the present invention;

[0034] Figure 3 A schematic diagram of functional modules of a preferred embodiment of the speech recognition error correction device of the present invention;

[0035] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0036] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0037] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0038] The speech recognition error correction method provided by the embodiment of the present invention can be applied in Figure 1In the application environment, the user terminal communicates with the server terminal through the network. The server terminal can collect the dialogue Chinese character text of the target field through the user terminal, train and generate the target field Chinese character text understanding model; collect pinyin data, train and generate the pinyin understanding model; receive the voice data to be detected and convert it into the Chinese character text and pinyin text to be detected; input the Chinese character text and the pinyin text into the corresponding understanding model, generate semantic features and pinyin features; merge the semantic features and the pinyin features to form a joint feature; input the joint feature into the speech recognition error detection model, generate the error probability distribution of each word, and output the error label; based on the error label, correct the Chinese character text through the template error correction module. The present invention combines semantic features and pinyin features to form a more comprehensive feature representation, thereby improving the accuracy of speech recognition error detection; using the template error correction module, the recognition error can be effectively corrected. Among them, the user terminal can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server terminal can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0039] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of a speech recognition error correction method provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0040] like Figure 2 As shown, the speech recognition error correction method proposed by the present invention comprises the following steps:

[0041] S10, collecting dialogue Chinese texts in the target domain, and training and generating a target domain Chinese text understanding model through the dialogue Chinese texts;

[0042] In this embodiment, the user's dialogue text data is obtained from a variety of data sources in the field of elderly care. These text data cover the expressions and interaction methods commonly used by the elderly in daily life to ensure the representativeness and pertinence of the data. These dialogue text data are the basis for subsequent model training and directly affect the model's ability to understand the language in the field of elderly care. Data sources can include service records of elderly care centers, user dialogue logs of home-based elderly care speakers, text dialogue data of online health consultation platforms, etc. After data collection, data preprocessing is required, including removing duplicate text, removing meaningless characters, and cleaning noise data in the corpus to ensure the data quality of model training.

[0043] The collected text data in the field of elderly care are trained using deep learning technology (such as the BERT-Tiny model), so that the model can learn and understand the semantics, syntactic structure and language features in the target field text. The trained model has the ability to recognize and understand the commonly used sentences of the elderly group.

[0044] The conversation texts in the elderly care field are divided into training sets and validation sets to ensure that the model can learn and verify its understanding ability. BERT-Tiny is used as a pre-trained model and fine-tuned. The fine-tuning process uses the Masked Language Model (MLM) task, which randomly masks certain words in the text and lets the model predict the masked words, thereby learning the language structure and semantic features of the text.

[0045] During the training process, optimizers (such as Adam) are used to adjust model parameters, and the semantic understanding ability of the model is improved through multiple rounds of training. The loss function can use the cross entropy loss function to measure the difference between the model's prediction and the actual label.

[0046] The Chinese character text understanding model generated by training with Chinese character texts of conversations in the target domain enables the model to more accurately capture the semantic features in the elderly care field. This improves the model's ability to understand and respond to specific language expressions of the elderly, and reduces the error rate in speech recognition and intent understanding.

[0047] S20, collecting pinyin data, and generating a pinyin understanding model through training with the pinyin data;

[0048] In this embodiment, a pinyin data set covering a variety of pronunciation features and pinyin combinations is obtained to train a pinyin understanding model so that it can accurately extract and understand pinyin features. Pinyin data should cover all initials, finals, tones and their combinations in Chinese to ensure that the model has sufficient generalization ability when recognizing different pronunciations. Pinyin data can be collected from the following channels: Public speech-pinyin data sets: Use public Chinese character and pinyin mapping data sets, which usually contain comparison tables of standard Chinese characters and corresponding pinyin. Speech-to-pinyin tool: Use a high-quality speech-to-pinyin tool to convert standard Chinese character text data to generate pinyin data. The accuracy of the pinyin annotations should be proofread during the conversion process to ensure data quality. Diversified collection: Collect pronunciation data from different dialect areas and convert them into standard pinyin format to enhance the model's adaptability to accent changes.

[0049] Use deep learning technology (such as the BERT-Tiny model) to train the Pinyin understanding model so that it can learn the acoustic features and word vector representation of Pinyin. The training process aims to enable the model to master the pronunciation rules and spelling structure of Pinyin, so as to provide effective Pinyin feature support in speech recognition tasks.

[0050] Preprocess the collected pinyin data to ensure that the data format is consistent. The processing steps include cleaning invalid pinyin, removing duplicates, and marking tone information. Load BERT-Tiny as a pre-trained model and fine-tune it. The fine-tuning task uses the Masked Language Model (MLM), that is, some characters in the pinyin data are randomly masked during the training process, and the model is allowed to predict the masked positions to learn the structure and pronunciation pattern of pinyin. Use an optimizer (such as Adam) to adjust the model parameters and iterate through multiple rounds of training. Use the cross-entropy loss function to evaluate the performance of the model in pinyin feature learning, and adjust the training strategy based on the results on the validation set. After training, the model can accurately generate word vector expressions of pinyin for subsequent speech recognition and error correction tasks.

[0051] Example description: In the home-based elderly care speaker dialogue system, the user may have a dialect accent or unclear pronunciation, such as saying "It's so cold." Because the Pinyin understanding model has been trained with diverse Pinyin data, it can correctly generate Pinyin features to assist the system in identifying and correcting recognition errors caused by pronunciation deviations, thereby correctly understanding the user's semantic intent and making appropriate responses.

[0052] The Pinyin understanding model generated by training Pinyin data enables the system to accurately extract and understand Pinyin features. This Pinyin understanding model provides higher fault tolerance and error correction efficiency when dealing with speech recognition errors caused by pronunciation, especially when dealing with accents and inaccurate pronunciation.

[0053] S30, receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected;

[0054] In this embodiment, the system receives the user's voice data through the voice input module. These voice data usually contain the daily interactions of elderly users, such as commands, inquiries, etc., and may have changes in speech speed, accent differences, or background noise. The quality and clarity of the voice data directly affect the subsequent voice recognition effect. During implementation, a microphone device or other voice acquisition device can be used to receive voice data, and the voice signal can be sampled and digitized and converted into an audio format (such as PCM, WAV, etc.). Noise reduction and audio enhancement technology can be used during the transmission of audio data to improve the quality of the voice signal and reduce the impact of environmental noise.

[0055] The received speech data is converted into corresponding Chinese text using an automatic speech recognition (ASR) model. This process involves extracting speech features from the audio signal and mapping these features into Chinese character sequences through a deep learning model. Since elderly users may have accents and voice quality issues when speaking, the ASR model needs to be highly adaptable and robust.

[0056] First, the audio signal is processed, such as endpoint detection and noise removal, and then speech features such as Mel-frequency cepstral coefficients (MFCC) or Mel-frequency spectrum features are extracted. These features are used to represent the audio information of the speech and are passed to the ASR model as input features. The extracted speech features are input into a pre-trained ASR model (such as a speech recognition model based on a deep neural network or an end-to-end Transformer model). The ASR model analyzes the features of the audio signal and maps it to the corresponding Chinese text. The recognition process may be optimized multiple times based on the output of the language model and the acoustic model to improve the conversion accuracy. The converted Chinese text can be post-processed, including correcting common ASR errors, formatting, and adding punctuation marks to optimize the readability and accuracy of the text.

[0057] Example: In the home-based elderly care speaker dialogue system, an elderly user may say "Help me turn on the lights in the living room". The system receives this voice data through the voice input module and uses the ASR model to convert the voice into the Chinese text of "Help me turn on the lights in the living room". Even if the user's pronunciation is not standard or there is background noise, the optimized ASR model can still accurately recognize and convert it, thus ensuring that subsequent operations can be executed correctly.

[0058] By receiving and converting the voice data to be detected into Chinese text, the user's voice input can be converted into text form, providing basic data support for subsequent semantic analysis and error detection.

[0059] S40, converting the Chinese character text to be detected into a pinyin text to be detected;

[0060] In this embodiment, the Chinese character text obtained by speech recognition conversion is further converted into the corresponding pinyin text. The pinyin text is used for the auxiliary matching process of subsequent pinyin feature extraction and error detection. The conversion process needs to accurately map each Chinese character to its correct pinyin form, including initial consonants, finals and tones, and pay special attention to the processing of homophones and polyphones to ensure the accuracy of pinyin conversion.

[0061] Use an efficient Chinese character to pinyin mapping library or dictionary to convert each Chinese character into its standard pinyin form. The mapping library can use an existing Chinese pinyin dictionary with built-in polyphonic solutions for common words. For polyphonic characters, the most appropriate pinyin can be selected based on semantic analysis of the context.

[0062] To more accurately handle the problem of polyphonic characters, the Chinese character text to be detected can be segmented first, and then natural language processing techniques can be used to analyze the context to determine the correct pronunciation of polyphonic characters. For example, "行" has different pinyins in "旅行" (lǚxíng) and "银行" (yín háng).

[0063] The generated pinyin text should contain complete pinyin information, including initials, finals, and tones. The pinyin text can be output in a standard format, such as "lǚxíng" (travel) or "yín háng" (bank), to ensure that subsequent models can correctly use pinyin features.

[0064] By converting the Chinese character text to be detected into pinyin text, accurate data input can be provided for subsequent pinyin feature extraction. In particular, dealing with the problem of polyphonic characters through word segmentation and context analysis can significantly improve the accuracy and robustness of pinyin conversion.

[0065] S50, input the Chinese character text to be detected into the Chinese character text understanding model in the target field to generate semantic features;

[0066] In this embodiment, the Chinese character text to be detected is used as input and passed to the Chinese character text understanding model in the target field to extract the semantic features of the text. Semantic features are used to represent the deep meaning of the text, including context semantics, lexical relationships, and sentence structure information. By understanding the semantics of the text, the model can perform error detection and correction more accurately, especially in adapting to the special language expressions of elderly users in the elderly care field.

[0067] First, load the pre-trained Chinese character text understanding model in the target field (such as BERT-Tiny). This model has been fine-tuned with a large amount of Chinese character text in the elderly care field and can understand the language characteristics of specific fields.

[0068] Necessary preprocessing is performed on the Chinese character text to be detected, such as removing extra spaces, converting to a standard format, etc., to ensure that the input meets the requirements of the model. Word segmentation or punctuation processing can also be performed to make the text format more conducive to model processing.

[0069] Input the preprocessed Chinese character text into the BERT-Tiny model. The encoder of the model converts the text into a word vector representation and extracts the semantic features of the text through a multi-layer attention mechanism. The semantic features can be represented as vectors or tensors for subsequent feature fusion and error detection.

[0070] Example description: In the home-based elderly care speaker dialogue system, when the user says "help me turn on the TV", and after converting it into Chinese text, the system inputs the text into the trained BERT-Tiny model. The model extracts the semantic features of the sentence "help me turn on the TV", understands that the user's intention is to turn on home appliances, and provides accurate feature representation for subsequent error detection and response. Even if the user uses similar expressions, such as "turn on the TV", the model can still recognize and generate corresponding semantic features.

[0071] By inputting the Chinese characters to be detected into the target field Chinese character text understanding model, semantic features with specific field adaptability are generated, enabling the system to analyze the text content more accurately. The deep representation of semantic features can effectively improve the accuracy of speech recognition error detection, especially when dealing with expressions unique to the elderly care field, significantly enhancing the robustness of the model.

[0072] S60, inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features;

[0073] In this embodiment, the converted phonetic text is used as input and passed to the phonetic text understanding model to extract feature information from the phonetic text. Phonetic features are used to capture the pronunciation rules and phonetic structures in the text, helping the model to more accurately identify and correct speech recognition errors caused by pronunciation errors. The extraction of phonetic features is a key link in the speech recognition system, especially when phonetic and semantics are closely related, it can enhance the accuracy of error detection.

[0074] Load the pre-trained BERT-Pinyin model, which has been trained on a large-scale Pinyin dataset and can understand and extract Pinyin features. The model has learned the initial consonants, finals, and tone features in Pinyin and has the ability to handle diverse pronunciations.

[0075] The pinyin text to be tested is formatted and standardized, such as removing extra spaces or punctuation marks, to ensure that the input data format is consistent with the input requirements of the model. Character separation and pinyin serialization can be performed according to the characteristics of pinyin.

[0076] The preprocessed Pinyin text is input into the BERT-Pinyin model. The model encodes the Pinyin text through its embedding layer and multi-layer self-attention mechanism, and extracts the feature information related to Pinyin. These Pinyin features are represented as high-dimensional vectors that can capture the pronunciation details and structural information of Pinyin for use in subsequent steps.

[0077] Example description: In the home-based elderly care speaker dialogue system, when the user says "turn on the air conditioner", it is converted into the pinyin text "dǎkāi kōng tiáo", and the system inputs the pinyin text into the BERT-Pinyin model. The model extracts the pinyin features and can capture the initial consonant, final vowel and tone information. Even if the user has slight deviations in pronunciation, such as inaccurate tones, the system can still understand and process it.

[0078] By inputting the pinyin text to be detected into the pinyin text understanding model, the extracted pinyin features can provide rich pronunciation information support for speech recognition error detection. This improves the system's fault tolerance and correction capabilities for user voices with non-standard pronunciation or accents, which is particularly suitable for elderly users or scenarios with pronunciation errors.

[0079] S70, merging the semantic feature and the phonetic feature to generate a joint feature;

[0080] In this embodiment, the semantic features extracted from the Chinese character text understanding model are fused with the phonetic features extracted from the phonetic text understanding model to generate a joint feature representation. The joint feature combines both semantic and phonetic information, which helps to understand the text more comprehensively, especially when dealing with speech recognition problems caused by pronunciation errors, which can effectively improve the accuracy of error detection.

[0081] First, the semantic features and phonetic features are standardized separately. The purpose of standardization is to normalize the numerical range of different features so that the subsequent feature fusion operation can be carried out more smoothly and avoid the impact of large numerical differences on model performance. Methods such as mean normalization or maximum and minimum value normalization can be used.

[0082] Assign weights based on the importance of features. For a specific task, the impact of semantic features and phonetic features may be different, so different weights can be assigned to each feature. The weights can be optimized through experiments or dynamically adjusted during model training. The weighted features are linearly weighted to highlight the feature information that is more important for error detection.

[0083] The standardized and weighted semantic features and phonetic features are concatenated on the feature dimension to form a joint feature vector. The concatenation operation can simply connect the two vectors so that the joint feature contains both semantic and pronunciation information for subsequent models to use. The concatenated vector dimension increases, but it can comprehensively express multiple information of the text.

[0084] The concatenated joint feature vector is regularized to reduce the risk of collinearity and data overfitting between features. Regularization methods can include L2 regularization or use regularization layers to make feature expression more stable and efficient.

[0085] Example: In the home-based elderly care speaker dialogue system, when the user says "turn on the air conditioner", the semantic feature captures the associated meaning of "turn on" and "air conditioner", while the pinyin feature provides accurate capture of the pronunciation. The system combines the two features to generate a joint feature. Even if the user causes an ASR error due to pronunciation problems, the joint feature can still help the system accurately identify the user's intention and ensure that the correct operation is performed.

[0086] By combining semantic features and phonetic features to generate joint features, the system can comprehensively consider the semantic and pronunciation information of the text, thereby improving the accuracy and robustness of speech recognition error detection. Especially in the case of phonetic errors and semantic ambiguity, the joint features can effectively improve the model's fault tolerance and understanding ability.

[0087] S80, inputting the joint feature into a speech recognition error detection model, generating an error probability distribution of each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and outputting a corresponding error label;

[0088] In this embodiment, the speech recognition error detection model is used to analyze the joint features to evaluate whether there is an error in each word in the Chinese text to be detected. The speech recognition error detection model jointly decodes the semantic and phonetic features through a deep learning algorithm, outputs a probability distribution for predicting the error probability of each word, and marks the error label. This can help the system automatically detect and identify errors that may occur during speech recognition, providing a basis for subsequent error correction.

[0089] Load a trained speech recognition error detection model. This model can be based on a deep neural network structure such as Transformer, BERT, or BiLSTM, which has been trained on joint features to understand and process input high-dimensional feature vectors. When the model is initialized, the pre-trained weights are loaded and the input interface is prepared to receive the joint feature vector.

[0090] The concatenated and regularized joint features are input to the speech recognition error detection model. The input layer of the model parses the joint feature vector and processes it through a multi-layer neural network. Each layer of the network may use different activation functions (such as ReLU, GELU, etc.) and regularization techniques to improve the ability of feature extraction and pattern recognition.

[0091] The model uses a fully connected layer or other classification layer to generate an error probability distribution for each word. This probability distribution represents the probability of each word being recognized as an error, with a value between 0 and 1. When generating the probability distribution, the model comprehensively considers semantic and phonetic features, and further enhances the understanding of the association between words through the self-attention mechanism or convolutional layer.

[0092] Based on the generated error probability distribution, the model assigns an error label to each word through a threshold judgment mechanism. If the error probability of a word exceeds the set threshold, the model marks the word as wrong and outputs the corresponding label. The label can be a simple "wrong" or "correct", or it can be further refined into specific error types, such as "pronunciation error", "lexical error" or "semantic error".

[0093] Example: In the home-based elderly care speaker dialogue system, when the user says "turn on the air conditioner", the system may recognize it as "check in the air conditioner" due to the accent. After the joint features are input into the speech recognition error detection model, the model analyzes the error probability of each word and finds that the error probability of the word "check in" is high, and then marks it as an error. The system then triggers the error correction mechanism based on this label and corrects it to "turn on", ensuring that the user's intention is correctly understood and executed.

[0094] By inputting the joint features into the speech recognition error detection model and generating error probability distribution and error labels, the system can automatically and accurately detect errors generated during speech recognition. Combined with semantic and phonetic feature analysis, the model performs better when processing complex semantic structures and inaccurate pronunciation, helping to improve the overall accuracy of speech recognition and user experience.

[0095] S90, based on the error label, correct the Chinese character text to be detected by a template error correction module.

[0096] In this embodiment, a template error correction module is used to correct the Chinese text to be detected based on the error labels output by the previous speech recognition error detection model. The template error correction module is a fast error correction method based on preset rules, dictionaries and regular expressions, which can efficiently and accurately replace words marked as errors. The correction process is combined with natural language processing technology to ensure that the error correction results are consistent with the semantic context and improve the accuracy and readability of the text.

[0097] The template error correction module receives the error label information and identifies the words marked as errors in the Chinese text to be detected. The error labels indicate which words may have phonetic errors, semantic errors or other types of errors.

[0098] The module uses a preset error correction template library to select appropriate error correction rules based on the error type. The template library contains common ASR (automatic speech recognition) errors and their corresponding correction methods, such as replacing phonetically similar words or adjusting semantically similar words. Regular expressions are used to quickly match and replace incorrect words.

[0099] The module can utilize domain-specific dictionaries or collections of hot words to ensure that the corrected words are consistent with the text context. Especially in the application in the elderly care field, the dictionary can cover vocabulary related to daily communication with the elderly to improve the error correction effect.

[0100] According to the matched error correction templates and dictionary replacement rules, the marked incorrect words are replaced. The module also checks the overall text to ensure grammatical and semantic coherence. After the correction is completed, the updated text is output for subsequent processing or execution.

[0101] Example illustration: In the intelligent voice customer service system of a bank, users often perform operations such as account inquiries, transaction confirmations, and identity verifications through voice interactions. However, due to factors such as the customer's dialect accent, unclear pronunciation, and noise interference, the speech recognition system may make errors in actual applications, thus affecting the user experience and even causing potential transaction risks.

[0102] For example, a user from the south called the intelligent voice customer service system of the bank and wanted to inquire about the recent account balance and transfer situation. This user's Mandarin had a strong dialect accent and was slightly lacking in speech rate and pronunciation clarity. The user uttered some requests like "查查余额 (check the balance)" and "查看转账记录 (check the transfer record)" during the call. However, the intelligent voice customer service system misrecognized "余额 (balance)" as "月额 (monthly amount)" and "转账 (transfer)" as "张三 (Zhang San)" during the recognition process, resulting in the system misunderstanding the user's intention and returning incorrect query results.

[0103] The system converts the user's voice input into Chinese character text to be detected, forming texts such as "查查月额 (check the monthly amount)" and "查看张三记录 (check the Zhang San record)".

[0104] The system extracts the semantic features of these texts through a Chinese character text understanding model to obtain the preliminary meaning of the sentences.

[0105] At the same time, the system converts the text into pinyin text (such as using "yue e" instead of "余额 (balance)" and "zhang san" instead of "张三 (Zhang San)"), and extracts pinyin features through a pinyin understanding model.

[0106] The system combines the semantic features and pinyin features to generate a joint feature vector with voice pronunciation and semantic information. This joint feature ensures that the system takes into account pronunciation errors on the basis of semantic analysis and enhances the accuracy of error correction.

[0107] The system inputs the joint feature into a speech recognition error detection model, and calculates the error probability distribution of each word in the text to be detected through the model. The model marks the word "月额 (monthly amount)" with a relatively high error probability, and at the same time, "张三 (Zhang San)" does not match the semantic context, so it is recognized as an error.

[0108] Based on the error label and template error correction module, the system replaces "monthly amount" with "balance" and "Zhang San" with "transfer", ensuring that the text after error correction is consistent with the user's intention. Figure 1 To.

[0109] Finally, the system confirms the correct query request to the user and prompts the user to query the detailed information of the balance and transfer record. The system also further optimizes the model based on user feedback to improve the error correction effect for similar dialect accents.

[0110] Correcting the Chinese character text to be detected through the template error correction module can effectively improve the accuracy of the speech recognition system, especially when dealing with recognition errors caused by pronunciation differences or speech ambiguity.

[0111] The present invention relates to the field of artificial intelligence technology, and discloses a speech recognition error correction method, which collects dialogue Chinese character texts in a target field, trains and generates a target field Chinese character text understanding model; collects pinyin data, trains and generates a pinyin understanding model; receives speech data to be detected and converts it into a Chinese character text and a pinyin text to be detected; inputs the Chinese character text and the pinyin text into a corresponding understanding model, generates semantic features and pinyin features; merges the semantic features and the pinyin features to form a joint feature; inputs the joint feature into a speech recognition error detection model, generates an error probability distribution for each word, and outputs an error label; based on the error label, corrects the Chinese character text through a template error correction module. The present invention forms a more comprehensive feature representation by combining semantic features and pinyin features, thereby improving the accuracy of speech recognition error detection; and using the template error correction module, it is possible to effectively correct recognition errors.

[0112] In one embodiment, the above S50 includes:

[0113] S501, extracting keywords from the Chinese text of the conversation, and constructing a keyword set based on the keywords;

[0114] S502, matching the Chinese character text to be detected with the keyword set to determine the target keyword in the Chinese character text to be detected;

[0115] S503, inputting the target keyword and the Chinese text to be detected into the target domain Chinese text understanding model, and generating the corresponding semantic features of the target keyword and the semantic features of the Chinese text to be detected respectively;

[0116] S504, combining the semantic features of the target keyword and the semantic features of the Chinese text to be detected to generate a complete semantic feature.

[0117] In this embodiment, commonly used or important keywords are extracted from a large amount of collected dialogue Chinese text data through natural language processing technology. These keywords can reflect the core semantic content of the target field and help the model better understand the user's semantic expression. The extraction method can use TF-IDF, word frequency analysis or other statistical methods to ensure that the keyword set is representative. First, the Chinese text is segmented and the word frequency is counted, and the algorithm is used to calculate the weight of each word, and the words with higher weights are screened out as keywords. Subsequently, these keywords are stored in a keyword set, and the set is updated regularly to maintain the real-time and adaptability of the dialogue scene.

[0118] Match the Chinese text to be detected input by the user with the keyword set word by word to identify the target keywords contained therein. These target keywords can help the model extract semantic features more accurately, especially in the special expressions of elderly users, the target keywords can make up for the understanding bias caused by recognition errors. Use the matching algorithm to traverse each word in the Chinese text to be detected and compare it with the keyword set. If the match is successful, mark the word as the target keyword. Matching can use simple string matching or fuzzy matching technology to increase tolerance for spelling errors or deformed words.

[0119] The matched target keywords and the entire Chinese text to be detected are input into the Chinese text understanding model to generate two types of semantic features. The semantic feature extraction of the target keywords can help the model better understand the key information in the text, while the overall semantic features of the Chinese text to be detected provide the global semantic background of the text. The target keywords and Chinese text are input into the model (such as the BERT model or other deep learning models) in an appropriate format. The model converts the input into a word vector and extracts its semantic features through a multi-layer self-attention mechanism. The output contains the semantic features of the target keywords and the overall semantic features of the text, which are used for subsequent feature fusion.

[0120] The semantic features of the target keyword and the overall semantic features of the Chinese text to be detected are fused to generate a complete semantic feature representation. The fusion process can use feature weighting, splicing or attention mechanism to ensure that the joint features can take into account both the local focus and the overall semantics of the text. The feature fusion technology is used to combine the two types of semantic features into a comprehensive feature vector. Fusion can be performed by linear weighting or feature splicing, or the attention mechanism can be used to adjust the importance of different features according to the context. The generated complete semantic features are used for further speech recognition error detection and text understanding.

[0121] This embodiment can enhance the model's ability to understand the text to be detected by matching and extracting target keywords and combining their semantic features with the overall text semantic features. The complete semantic features integrate local and global semantic information, improve the accuracy of speech recognition error detection and correction, especially in scenarios with polysemous words and complex semantic expressions.

[0122] In one embodiment, the above S502 includes:

[0123] S5021, generating a corresponding pinyin variant representation for each keyword, and forming a pinyin set for each keyword based on the pinyin variant representation;

[0124] S5022, performing word segmentation processing on the Chinese character text to be detected to obtain a word sequence after word segmentation;

[0125] S5023, converting the words in the word sequence into pinyin representation;

[0126] S5024, matching the pinyin representation of the word in the Chinese character text to be detected with the pinyin set of the keyword to generate an auxiliary matching result;

[0127] S5025, combining the auxiliary matching result, matching the Chinese character text to be detected with the keyword set, and determining the target keyword in the Chinese character text to be detected.

[0128] In this embodiment, for each keyword in the keyword set, all possible pinyin variant representations are generated. Since there are problems such as homophones, polyphones and tone differences in Chinese, creating a pinyin set for each keyword helps to improve the fault tolerance of matching. The pinyin set contains possible pinyin representations of the keywords and their different tone combinations, covering various pronunciation variants. Using a pinyin conversion tool or a pinyin dictionary, multiple pinyin combinations are generated for each keyword and stored in the pinyin set. For example, the pinyin set of the keyword "bank" may include "yín háng" and "yìn háng".

[0129] Segment the Chinese text to be detected and decompose the continuous Chinese text into meaningful word sequences. Segmentation helps improve the accuracy of subsequent pinyin matching and semantic analysis, especially when the text contains multi-words or phrases, it can effectively identify the text structure. Use a word segmentation tool (such as jieba or HanLP) to segment the text to be detected. The word sequence after segmentation can contain single words or multi-words, and is arranged in the natural order of the text for subsequent matching.

[0130] Convert each word after segmentation into a pinyin representation, ensuring that the pinyin form can match the pinyin set of the keyword. This conversion process needs to take into account the polyphonic problem of Chinese characters and use the context to determine the correct pinyin pronunciation. By calling the pinyin conversion library, convert each word into the corresponding pinyin representation. For polyphonic characters, use the context analysis algorithm to select the most appropriate pinyin. The conversion result should contain initial consonants, finals, and tone information.

[0131] Compare the pinyin representation of words in the text with the pinyin set of keywords to identify words with high pinyin similarity. This process generates an auxiliary matching result to improve the accuracy of keyword matching. Auxiliary matching can identify words with subtle differences in pinyin pronunciation and provide support for subsequent matching. Calculate the matching degree between the pinyin representation of the word and the pinyin set of keywords. Pinyin similarity algorithms such as edit distance or tone matching algorithms can be used to generate an auxiliary matching result to mark possible keyword matching candidates.

[0132] With the support of auxiliary matching results, the Chinese text to be detected is finally matched with the keyword set to determine the target keywords in the text. Target keywords refer to keywords that actually appear in the text or have similar pronunciations. These keywords are helpful for further semantic analysis and error detection. According to the auxiliary matching results, the candidate words are matched with the keyword set, and strict or loose matching strategies are adopted to finally determine the target keywords. After the matching is completed, the system outputs the identified target keywords for subsequent semantic feature extraction and error correction operations.

[0133] This embodiment can effectively solve the problem of inaccurate keyword matching caused by pronunciation differences or pinyin errors by generating pinyin variants and assisting matching. Combining pinyin and semantic features, the system can more accurately identify target keywords, thereby improving the overall accuracy of speech recognition and providing a reliable data basis for subsequent semantic analysis and error correction.

[0134] In one embodiment, the above S90 includes:

[0135] S901, based on the error label, identifying the erroneous words in the Chinese character text to be detected by the template error correction module;

[0136] S902, according to the erroneous word, searching for a corresponding error correction template from a preset error correction template database;

[0137] S903: Based on the error correction template, replace the erroneous words with correct words in the Chinese character text to be detected.

[0138] In this embodiment, a template error correction module is used to locate problematic words in the Chinese text to be detected based on the error labels output by the speech recognition error detection model. The error labels provide clear indications of which words may have recognition errors, such as misrecognition caused by similar pinyin, semantic mismatch, etc. The template error correction module will parse the error labels and mark all words detected as possible errors in the text to be detected. The module uses a specific algorithm, combined with the contextual information of the words, to ensure that the misrecognized words can be accurately identified.

[0139] Once an incorrect word is identified, the system will retrieve the error correction template related to the word and its context from the preset error correction template database. The error correction template contains common errors and their correction methods, and is designed for specific error scenarios for quick replacement or adjustment. The template database stores a large number of error correction templates classified based on error types. The module searches the database based on the characteristics of the incorrect word (such as spelling errors, typos, etc.) to find the most suitable correction solution for the current error situation. The retrieval process can use an indexing mechanism to speed up queries and improve error correction efficiency.

[0140] Use the correction template retrieved from the correction template database to replace the incorrect words in the Chinese text to be detected. The replacement process not only corrects the incorrect words, but also ensures the overall semantics and text coherence to avoid new semantic problems caused by the correction. The module applies the replacement rules of the correction template to replace the incorrect words with the correct words. The system also performs a simple grammatical and semantic check to ensure that the replaced text conforms to the logic of natural language expression. The replaced text will be output as the correction result for system execution or user viewing.

[0141] In this embodiment, the system can efficiently correct speech recognition errors through the template error correction module, especially when dealing with common spelling or typos. The error correction template provides a specific correction solution for each common error, which not only improves the error correction speed, but also significantly improves the user experience and ensures that the voice command is correctly understood and executed.

[0142] In one embodiment, the above S70 includes:

[0143] S701, performing standardization processing on the semantic features and phonetic features;

[0144] S702, assigning corresponding weights to the semantic features and the phonetic features according to their importance, and performing linear weighted processing on the semantic features and the phonetic features based on the weights;

[0145] S703, concatenating the semantic features and phonetic features after the linear weighting process in the feature dimension to form a joint feature vector;

[0146] S704, performing regularization processing on the joint feature vector to generate a final joint feature.

[0147] In the present embodiment, the numerical ranges of semantic features and phonetic features may be different, so they need to be standardized before merging so that their distribution is within the same numerical range. Standardization can make the numerical values ​​of different features consistent, avoiding unstable model performance due to scale differences between features. Commonly used standardization methods include mean normalization (each feature minus the mean and divided by the standard deviation) and minimum-maximum normalization (scaling the feature value to a range of 0 to 1). Select a suitable standardization method and convert the features into standardized form according to the data characteristics to facilitate subsequent weighting and fusion operations.

[0148] Since semantic features and phonetic features may have different importance in speech recognition error detection, it is necessary to assign weights to each feature. The weight assignment can be determined based on experimental data of a specific task or a dynamic adjustment strategy to ensure that the weight assignment helps improve the performance of the model. Weights are assigned through experiments or using adaptive mechanisms. For example, if phonetic features are more critical in certain tasks, the model will automatically adjust to increase the weight of phonetic features. Linear weighted processing is to multiply the standardized features by their respective weights and sum them to generate a weighted feature vector.

[0149] The processed semantic features and phonetic features are combined through feature concatenation to form a joint feature vector. Concatenation means connecting two feature vectors in the feature dimension, retaining all feature information, so that the joint feature vector has both semantic and pronunciation information. Directly concatenate two feature vectors in the feature dimension to generate a new high-dimensional feature vector. The concatenated joint feature vector will have richer information content than a single feature vector, which can be processed and analyzed by subsequent models.

[0150] Regularization can reduce the risk of overfitting of the model and enhance the generalization ability of features. Regularization can control the distribution of feature vectors within a certain range to reduce the problem of collinearity between features and improve the robustness of the model to features. Use L2 regularization or other regularization techniques to constrain the joint feature vector so that the norm of the feature is less than a certain threshold. The regularized joint feature is more stable and can provide more reliable input for the speech recognition error detection model.

[0151] This embodiment uses standardization, weighting, concatenation and regularization to generate joint features that can fully integrate semantic and phonetic information, combining the advantages of both, and improving the overall performance of the speech recognition error detection model. Regularization can also prevent overfitting of the model and improve the generalization ability and stability of the model.

[0152] In one embodiment, before the above S80, the method further includes:

[0153] S801, obtaining training joint features and corresponding true labels;

[0154] S802, inputting the training joint features into a basic classification model to obtain a predicted error label result;

[0155] S803, determining the error between the predicted erroneous label result and the true label by using a cross entropy loss function;

[0156] S804, optimizing and adjusting the parameters of the basic classification model based on the error to obtain the speech recognition error detection model.

[0157] In this embodiment, in order to train the speech recognition error detection model, it is necessary to construct a labeled data set containing training joint features and true labels. The training joint features are formed by combining semantic features and phonetic features, and the true label indicates whether there is a recognition error for each word in the text (such as a correct or wrong label). A large number of training joint features are obtained from the labeled data set, and each joint feature is paired with its corresponding true label. The true label is manually labeled or generated by pre-set rules to ensure the accuracy and reliability of the training data.

[0158] The training joint features are input into the basic classification model, which generates the predicted error label results for each word by analyzing the feature information. The basic classification model can be a neural network based on deep learning, such as BERT, LSTM, or other models suitable for feature classification. The model receives the training joint features and performs forward propagation to calculate the error probability distribution of each word. The output of the model is the predicted error label for each word, indicating whether the model believes that the word is wrong. The prediction results will be used for subsequent error calculation and model optimization.

[0159] The cross entropy loss function is a commonly used loss function that quantifies the difference between the predicted wrong label result and the true label. The larger the value of the loss function, the greater the deviation between the predicted result and the true label, and vice versa. The predicted wrong label result is compared with the true label to calculate the cross entropy loss value. The loss value represents the prediction error of the model and is used to measure the current performance of the model. Through the calculation of the loss function, the parameter adjustment of the model can be guided to gradually reduce the error.

[0160] The parameters of the base classification model are adjusted based on the calculated error values ​​using an optimization algorithm such as Adam or SGD to optimize model performance. The optimization process updates the model weights through backpropagation so that the model gradually learns to accurately predict the error labels. The calculated error values ​​are passed to the optimizer, which performs backpropagation to update the model parameters. After each update, the model is adjusted to get closer to the distribution of the true labels. After multiple rounds of training, the model gradually converges to obtain the final speech recognition error detection model, which can efficiently detect and mark text errors in the inference stage.

[0161] In this embodiment, by using the training joint features and the true labels for supervised learning, the speech recognition error detection model can accurately identify incorrect words in the text.

[0162] In one embodiment, after the above S80, the method further includes:

[0163] S805, obtaining feedback information of the error label;

[0164] S806, determining the error between the erroneous label and the feedback information through a loss function;

[0165] S807, adjusting the speech recognition error detection model based on the error, and optimizing and updating the parameters of the speech recognition error detection model.

[0166] In this embodiment, after the speech recognition error detection model generates an error label, the system collects feedback information for further optimizing the model performance. The feedback information can come from user feedback, the results of automatic detection by the system, or reference information generated by other error correction modules. The feedback information provides additional guidance for the model to improve the accuracy of the model in error detection. A feedback mechanism is set up so that users can provide feedback data when they find that the model's error label is incorrect or the error correction is incorrect. The system automatically collects this feedback information and organizes it into a meaningful data structure for subsequent error calculation and model optimization.

[0167] Use the loss function to quantify the difference between the error label and the feedback information to guide the further optimization of the model. The choice of loss function can be determined according to the task requirements. For example, the cross entropy loss function is suitable for classification tasks and is used to measure the difference between the error label and the feedback information. Input the error label and feedback information into the loss function to calculate the error value. The larger the error value, the greater the deviation between the model output and the feedback information, and the model needs a greater adjustment. The calculated error value will be used as the optimization target to help the model learn more accurately.

[0168] The error value is used to adjust the model parameters to improve the model's performance on the error detection task. The optimization process is implemented through the back-propagation algorithm, which passes the error value back to each layer of the model, updates the model weights and bias parameters, and makes the model more accurate in future predictions. Choose a suitable optimizer (such as Adam, SGD, etc.) for parameter update. The optimizer calculates the gradient of the parameters based on the error value and updates the model parameters to minimize the error. The adjustment process can be gradually convergent, and the model will gradually reach the optimal performance after multiple iterations.

[0169] By introducing feedback information, this embodiment enables the model to self-optimize and learn in actual usage scenarios, continuously improving the accuracy of error detection. The loss function provides a clear optimization direction. Combined with the parameter adjustment of the optimizer, the speech recognition error detection model can dynamically adapt to user needs, reduce the error rate, and improve user experience and system intelligence.

[0170] In one embodiment, a speech recognition error correction device is provided, and the speech recognition error correction device corresponds one-to-one to the speech recognition error correction method in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of the speech recognition error correction device of the present invention. Chinese character text understanding module 10, pinyin understanding module 20, speech data conversion module 30, Chinese character to pinyin conversion module 40, semantic feature extraction module 50, pinyin feature extraction module 60, feature merging module 70, error detection module 80 and template error correction module 90. The functional modules are described in detail as follows:

[0171] A Chinese character text understanding module 10 is used to collect dialogue Chinese character texts in a target domain and generate a target domain Chinese character text understanding model through training of the dialogue Chinese character texts;

[0172] The pinyin understanding module 20 is used to collect pinyin data and generate a pinyin understanding model through training of the pinyin data;

[0173] The voice data conversion module 30 is used to receive the voice data to be detected and convert the voice data to be detected into Chinese character text to be detected;

[0174] A Chinese character to pinyin conversion module 40, used for converting the Chinese character text to be detected into a pinyin text to be detected;

[0175] A semantic feature extraction module 50, used for inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features;

[0176] The pinyin feature extraction module 60 is used to input the pinyin text to be detected into the pinyin text understanding model to generate pinyin features;

[0177] A feature merging module 70, used to merge the semantic feature and the phonetic feature to generate a joint feature;

[0178] An error detection module 80, used for inputting the joint feature into a speech recognition error detection model, generating an error probability distribution of each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and outputting a corresponding error label;

[0179] The template error correction module 90 is used to correct the Chinese character text to be detected based on the error label through the template error correction module.

[0180] In one embodiment, the semantic feature extraction module 50 is specifically used for:

[0181] Extracting keywords from the Chinese text of the conversation, and constructing a keyword set based on the keywords;

[0182] Matching the Chinese character text to be detected with the keyword set to determine the target keyword in the Chinese character text to be detected;

[0183] Inputting the target keyword and the Chinese character text to be detected into the target domain Chinese character text understanding model, and generating the corresponding semantic features of the target keyword and the semantic features of the Chinese character text to be detected respectively;

[0184] The semantic features of the target keyword and the semantic features of the Chinese character text to be detected are combined to generate a complete semantic feature.

[0185] In one embodiment, the semantic feature extraction module 50 is specifically used for:

[0186] Generate a corresponding pinyin variant representation for each keyword, and form a pinyin set for each keyword based on the pinyin variant representation;

[0187] Perform word segmentation on the Chinese character text to be detected to obtain a word sequence after word segmentation;

[0188] Converting the words in the word sequence into phonetic representation;

[0189] Matching the pinyin representation of the words in the Chinese text to be detected with the pinyin set of the keywords to generate an auxiliary matching result;

[0190] In combination with the auxiliary matching result, the Chinese character text to be detected is matched with the keyword set to determine the target keyword in the Chinese character text to be detected.

[0191] In one embodiment, the template error correction module 90 is specifically used to:

[0192] Based on the error labels, identifying the erroneous words in the Chinese character text to be detected by the template error correction module;

[0193] According to the erroneous word, a corresponding error correction template is retrieved from a preset error correction template database;

[0194] Based on the error correction template, the erroneous words are replaced with correct words in the Chinese character text to be detected.

[0195] In one embodiment, the feature merging module 70 is specifically configured to:

[0196] Standardizing the semantic features and phonetic features;

[0197] According to the importance of the semantic features and the phonetic features, corresponding weights are assigned to the semantic features and the phonetic features, and linear weighted processing is performed on the semantic features and the phonetic features based on the weights;

[0198] The semantic features and phonetic features after linear weighting are concatenated in the feature dimension to form a joint feature vector;

[0199] Regularization is performed on the joint feature vector to generate a final joint feature.

[0200] In one embodiment, the error detection module 80 is specifically configured to:

[0201] Get the training joint features and the corresponding true labels;

[0202] Inputting the training joint features into a basic classification model to obtain a predicted error label result;

[0203] Determine the error between the predicted error label result and the true label by a cross entropy loss function;

[0204] The parameters of the basic classification model are adjusted based on the error optimization to obtain the speech recognition error detection model.

[0205] In one embodiment, the error detection module 80 is specifically configured to:

[0206] Obtaining feedback information of the error label;

[0207] Determine the error between the error label and the feedback information by using a loss function;

[0208] The speech recognition error detection model is adjusted based on the error, and the parameters of the speech recognition error detection model are optimized and updated.

[0209] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal via a network connection. When the computer program is executed by the processor, it implements a function or step on the server side of a speech recognition error correction method.

[0210] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a user-side side of a speech recognition error correction method

[0211] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0212] Collecting dialogue Chinese texts in a target domain, and training and generating a target domain Chinese text understanding model through the dialogue Chinese texts;

[0213] Collecting pinyin data, and training and generating a pinyin understanding model through the pinyin data;

[0214] Receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected;

[0215] Converting the Chinese character text to be detected into a pinyin text to be detected;

[0216] Inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features;

[0217] Inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features;

[0218] Merging the semantic feature and the phonetic feature to generate a joint feature;

[0219] Input the joint feature into a speech recognition error detection model, generate an error probability distribution for each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and output a corresponding error label;

[0220] Based on the error label, the Chinese character text to be detected is corrected by a template error correction module.

[0221] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0222] Collecting dialogue Chinese texts in a target domain, and training and generating a target domain Chinese text understanding model through the dialogue Chinese texts;

[0223] Collecting pinyin data, and training and generating a pinyin understanding model through the pinyin data;

[0224] Receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected;

[0225] Converting the Chinese character text to be detected into a pinyin text to be detected;

[0226] Inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features;

[0227] Inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features;

[0228] Merging the semantic feature and the phonetic feature to generate a joint feature;

[0229] Input the joint feature into a speech recognition error detection model, generate an error probability distribution for each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and output a corresponding error label;

[0230] Based on the error label, the Chinese character text to be detected is corrected by a template error correction module.

[0231] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0232] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0233] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0234] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A speech recognition error correction method, characterized in that: The following steps are involved: Collecting dialogue Chinese texts in a target domain, and training and generating a target domain Chinese text understanding model through the dialogue Chinese texts; Collecting pinyin data, and training and generating a pinyin understanding model through the pinyin data; Receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected; Converting the Chinese character text to be detected into a pinyin text to be detected; Inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features; Inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features; Merging the semantic feature and the phonetic feature to generate a joint feature; Input the joint feature into a speech recognition error detection model, generate an error probability distribution for each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and output a corresponding error label; Based on the error label, the Chinese character text to be detected is corrected by a template error correction module.

2. The speech recognition error correction method according to claim 1, characterized in that: Inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features, including: Extracting keywords from the Chinese text of the conversation, and constructing a keyword set based on the keywords; Matching the Chinese character text to be detected with the keyword set to determine the target keyword in the Chinese character text to be detected; Input the target keyword and the Chinese character text to be detected into the target field Chinese character text understanding model, and generate the corresponding semantic features of the target keyword and the semantic features of the Chinese character text to be detected respectively; The semantic features of the target keyword and the semantic features of the Chinese character text to be detected are combined to generate a complete semantic feature.

3. The speech recognition error correction method according to claim 2, characterized in that: Matching the Chinese character text to be detected with the keyword set to determine the target keyword in the Chinese character text to be detected includes: Generate a corresponding pinyin variant representation for each keyword, and form a pinyin set for each keyword based on the pinyin variant representation; Perform word segmentation on the Chinese character text to be detected to obtain a word sequence after word segmentation; Converting the words in the word sequence into phonetic representation; Matching the pinyin representation of the words in the Chinese text to be detected with the pinyin set of the keywords to generate an auxiliary matching result; In combination with the auxiliary matching result, the Chinese character text to be detected is matched with the keyword set to determine the target keyword in the Chinese character text to be detected.

4. The speech recognition error correction method according to claim 1, characterized in that: Based on the error label, the Chinese character text to be detected is corrected by a template error correction module, including: Based on the error labels, identifying the erroneous words in the Chinese character text to be detected by the template error correction module; According to the erroneous word, a corresponding error correction template is retrieved from a preset error correction template database; Based on the error correction template, the erroneous words are replaced with correct words in the Chinese character text to be detected.

5. The speech recognition error correction method according to claim 1, characterized in that: The semantic feature and the phonetic feature are combined to generate a joint feature, including: Standardizing the semantic features and phonetic features; According to the importance of the semantic features and the phonetic features, corresponding weights are assigned to the semantic features and the phonetic features, and linear weighted processing is performed on the semantic features and the phonetic features based on the weights; The semantic features and phonetic features after linear weighting are concatenated in the feature dimension to form a joint feature vector; Regularization is performed on the joint feature vector to generate a final joint feature.

6. The speech recognition error correction method as claimed in claim 1, characterized in that: The method further includes: inputting the joint feature into a speech recognition error detection model, generating an error probability distribution of each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and outputting a corresponding error label. Get the training joint features and the corresponding true labels; Inputting the training joint features into a basic classification model to obtain a predicted error label result; Determine the error between the predicted error label result and the true label by a cross entropy loss function; The parameters of the basic classification model are adjusted based on the error optimization to obtain the speech recognition error detection model.

7. The speech recognition error correction method according to claim 1, characterized in that: After inputting the joint feature into a speech recognition error detection model, generating an error probability distribution of each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and outputting a corresponding error label, the method further includes: Obtaining feedback information of the error label; Determine the error between the error label and the feedback information by using a loss function; The speech recognition error detection model is adjusted based on the error, and the parameters of the speech recognition error detection model are optimized and updated.

8. A speech recognition error correction device, characterized in that: The speech recognition error correction device comprises: A Chinese character text understanding module is used to collect dialogue Chinese character texts in a target domain and generate a target domain Chinese character text understanding model through training of the dialogue Chinese character texts; A pinyin understanding module, used to collect pinyin data and generate a pinyin understanding model through training of the pinyin data; A voice data conversion module, used for receiving voice data to be detected, and converting the voice data to be detected into Chinese character text to be detected; A Chinese character to pinyin conversion module, used for converting the Chinese character text to be detected into a pinyin text to be detected; A semantic feature extraction module, used for inputting the Chinese character text to be detected into the target domain Chinese character text understanding model to generate semantic features; A pinyin feature extraction module, used for inputting the pinyin text to be detected into the pinyin text understanding model to generate pinyin features; A feature merging module, used to merge the semantic feature and the phonetic feature to generate a joint feature; An error detection module, used for inputting the joint feature into a speech recognition error detection model, generating an error probability distribution of each word in the Chinese character text to be detected according to the joint feature through the speech recognition error detection model, and outputting a corresponding error label; The template error correction module is used to correct the Chinese character text to be detected based on the error label through the template error correction module.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a speech recognition error correction program stored in the memory and executable on the processor. When the speech recognition error correction program is executed by the processor, the steps of the speech recognition error correction method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a speech recognition error correction program, which, when executed by a processor, implements the steps of the speech recognition error correction method according to any one of claims 1 to 7.

Images