A speech recognition method, apparatus, storage medium, and device

Abstract

The application discloses a speech recognition method and device, a storage medium and equipment. The method comprises the following steps: first, generating the audio features of the target speech, and then inputting the audio features into a pre-constructed speech recognition model to recognize the first entity vocabulary and the position information thereof in the phoneme recognition result and the initial text recognition result of the target speech; then, decoding the phoneme recognition result of the target speech by using a pre-constructed domain professional entity vocabulary list to obtain the second entity vocabulary; determining the similar entity vocabulary corresponding to the first entity vocabulary and the second entity vocabulary respectively, and scoring the similar entity vocabulary, the first entity vocabulary and the second entity vocabulary by using a pre-set language model to obtain a scoring result; and finally, judging whether the first entity vocabulary needs to be corrected by using the second entity vocabulary or any similar entity vocabulary according to the scoring result, and if yes, determining the final recognition result of the target speech after correction to improve the speech recognition effect.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a speech recognition method, apparatus, storage medium and device. Background Technology

[0002] With the continuous breakthroughs in artificial intelligence technology and the increasing prevalence of various smart terminal devices, human-computer interaction is occurring more and more frequently in people's daily work and life. Speech recognition technology has also been widely used, covering all areas of human-computer interaction. The core challenge of domain-specific speech recognition lies in the large number of domain-specific entity words. Accurate detection and error correction of these entity words in speech recognition to maintain a high accuracy rate is therefore crucial.

[0003] Existing methods for entity word detection and correction in domain-specific speech recognition typically fall into two categories: one is the detection and correction of specialized entity words in pure text. This approach uses a speech recognition model to obtain the recognition result, and then employs a natural language processing model such as BERT to detect entity nouns in the recognized sentence and correct unreasonable errors in the entity nouns. However, this approach does not actually consider acoustic pronunciation features and is disconnected from speech recognition. As a result, the final recognition result, while often obtaining seemingly reasonable corrections, may not match the actual pronunciation, leading to overcorrection. The other approach is to directly correct the speech recognition model. This approach requires updating and learning the speech recognition model, making zero-shot learning impossible. Therefore, the entire process is time-consuming, labor-intensive, and costly, and the improvement in recognition accuracy is highly dependent on the constructed training corpus. However, for unconstructed contextual statements, the improvement in recognition accuracy is usually very limited. In addition, updating and learning the speech recognition model carries the risk of incremental learning, meaning it is difficult to guarantee that the updated speech recognition model will not decrease in recognition accuracy for existing domain-specific words. Since existing speech recognition models are updated based on new corpora, they will more or less forget the previously trained data. To avoid this problem, it is often necessary to use a method of synchronous training with a large amount of previous data, which will also be time-consuming and laborious. Summary of the Invention

[0004] The main objective of this application is to provide a speech recognition method, apparatus, storage medium, and device that can effectively improve the efficiency and accuracy of speech recognition.

[0005] This application provides a speech recognition method, including:

[0006] Acquire the target speech to be identified and generate the audio features of the target speech;

[0007] The audio features of the target speech are input into a pre-built speech recognition model to identify the phoneme recognition results of the target speech and the first entity words and their location information in the initial text recognition results;

[0008] Using a pre-built list of domain-specific entity words, the phoneme recognition results of the target speech are decoded to obtain a second entity word;

[0009] Identify the similar entity words corresponding to the first entity word and the second entity word respectively, and use a preset language model to score the first entity word, the second entity word, and the similar entity words corresponding to the first entity word and the second entity word respectively, and obtain the scoring results.

[0010] Based on the scoring results, it is determined whether it is necessary to use the second entity word or the similar entity words corresponding to the first entity word and the second entity word to perform error correction processing on the first entity word. If so, the final recognition result corresponding to the target speech is determined based on the obtained error correction result.

[0011] In one possible implementation, the audio features of the target speech are the amplitude spectrum features of the target speech.

[0012] In one possible implementation, the speech recognition model includes an encoding layer, a phoneme recognition layer, a decoding layer, and a text entity word detection layer; the step of inputting the audio features of the target speech into the pre-constructed speech recognition model to identify the first entity word and its location information in the phoneme recognition result and the initial text recognition result of the target speech includes:

[0013] The audio features of the target speech are input into the coding layer of the speech recognition model for encoding processing to obtain a high-dimensional speech representation vector of the target speech;

[0014] The high-dimensional speech representation vector of the target speech is input into the phoneme recognition layer of the speech recognition model for recognition processing to obtain the phoneme recognition result of the target speech;

[0015] The high-dimensional speech representation vector of the target speech is input into the decoding layer of the speech recognition model for decoding processing to obtain the initial text recognition result of the target speech;

[0016] The initial text recognition result of the target speech is input into the text entity word detection layer of the speech recognition model for detection processing to obtain the first entity word and its position information in the initial text recognition result.

[0017] In one possible implementation, after obtaining the phoneme recognition result of the target speech and the first entity word and its location information in the initial text recognition result, the method further includes:

[0018] Based on the location information and the marker symbol of the first entity word, the target domain classification to which the target speech belongs is determined.

[0019] In one possible implementation, the step of decoding the phoneme recognition results of the target speech using a pre-built domain-specific entity vocabulary list to obtain a second entity vocabulary includes:

[0020] Align the domain-specific entity words of the target domain classification with the phoneme recognition results of the target speech to obtain the pronunciation sequence corresponding to the first entity word in the phoneme recognition results;

[0021] The pre-constructed list of domain-specific entity words for the target domain classification is packaged into a decoding resource; and the decoding resource is used to decode the pronunciation sequence corresponding to the first entity word to obtain the second entity word.

[0022] In one possible implementation, determining the similar entity words corresponding to the first entity word and the second entity word respectively includes:

[0023] The first entity word and the second entity word are expanded by similar sounds, and the similar entity words corresponding to the first entity word and the second entity word are determined according to the expansion results.

[0024] In one possible implementation, the step of determining whether to use the second entity word or similar entity words corresponding to the first entity word and the second entity word for error correction based on the scoring result, and if so, determining the final recognition result corresponding to the target speech based on the obtained error correction result, includes:

[0025] Select the highest score and the second highest score from the scores of the first entity word, the second entity word, and the scores of the similar entity words corresponding to the first entity word and the second entity word respectively.

[0026] Calculate the difference between the first high score and the second score, and determine whether the difference is higher than a preset threshold. If so, use the entity words corresponding to the first high score to perform error correction processing on the first entity words to obtain the final recognition result corresponding to the target speech.

[0027] This application also provides a voice recognition device, including:

[0028] An acquisition unit is used to acquire the target speech to be identified and generate the audio features of the target speech;

[0029] The recognition unit is used to input the audio features of the target speech into a pre-built speech recognition model to recognize the phoneme recognition results of the target speech and the first entity words and their location information in the initial text recognition results;

[0030] The decoding unit is used to decode the phoneme recognition results of the target speech using a pre-built list of domain-specific entity words to obtain a second entity word;

[0031] The scoring unit is used to determine the similar entity words corresponding to the first entity word and the second entity word respectively, and to score the first entity word, the second entity word and the similar entity words corresponding to the first entity word and the second entity word respectively using a preset language model to obtain the scoring results.

[0032] The error correction unit is used to determine, based on the scoring result, whether it is necessary to use the second entity word or the similar entity words corresponding to the first entity word and the second entity word to perform error correction processing on the first entity word. If so, the final recognition result corresponding to the target speech is determined based on the obtained error correction result.

[0033] In one possible implementation, the audio features of the target speech are the amplitude spectrum features of the target speech.

[0034] In one possible implementation, the speech recognition model includes an encoding layer, a phoneme recognition layer, a decoding layer, and a text entity word detection layer; the recognition unit includes:

[0035] The first input subunit is used to input the audio features of the target speech into the coding layer of the speech recognition model for encoding processing to obtain a high-dimensional speech representation vector of the target speech;

[0036] The second input subunit is used to input the high-dimensional speech representation vector of the target speech into the phoneme recognition layer of the speech recognition model for recognition processing, so as to obtain the phoneme recognition result of the target speech.

[0037] The third input subunit is used to input the high-dimensional speech representation vector of the target speech into the decoding layer of the speech recognition model for decoding processing, so as to obtain the initial text recognition result of the target speech;

[0038] The fourth input subunit is used to input the initial text recognition result of the target speech into the text entity word detection layer of the speech recognition model for detection processing, so as to obtain the first entity word and its position information in the initial text recognition result.

[0039] In one possible implementation, the device further includes:

[0040] The determining unit is configured to determine the target domain classification to which the target speech belongs based on the location information of the first entity word and the tag symbol of the first entity word.

[0041] In one possible implementation, the decoding unit includes:

[0042] The alignment subunit is used to align the domain-specific entity words of the target domain classification with the phoneme recognition results of the target speech to obtain the pronunciation sequence corresponding to the first entity word in the phoneme recognition results.

[0043] The decoding subunit is used to package the pre-constructed list of domain-specific entity words of the target domain classification into a decoding resource; and use the decoding resource to decode the pronunciation sequence corresponding to the first entity word to obtain the second entity word.

[0044] In one possible implementation, the scoring unit is specifically used for:

[0045] The first entity word and the second entity word are expanded by similar sounds, and the similar entity words corresponding to the first entity word and the second entity word are determined according to the expansion results.

[0046] In one possible implementation, the error correction unit includes:

[0047] The selection subunit is used to select the first high score and the second score from the scores of the first entity word, the scores of the second entity word, and the scores of the similar entity words corresponding to the first entity word and the second entity word respectively.

[0048] The error correction subunit is used to calculate the difference between the first high score and the second score, and to determine whether the difference is higher than a preset threshold. If so, the first entity word is corrected using the entity word corresponding to the first high score, so as to obtain the final recognition result corresponding to the target speech.

[0049] This application also provides a voice recognition device, including: a processor, a memory, and a system bus;

[0050] The processor and the memory are connected via the system bus;

[0051] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the speech recognition method.

[0052] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the speech recognition method.

[0053] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described implementations of the speech recognition method.

[0054] This application provides a speech recognition method, apparatus, storage medium, and device. First, it acquires the target speech to be recognized and generates audio features of the target speech. Then, it inputs the audio features of the target speech into a pre-built speech recognition model to identify the phoneme recognition result of the target speech and the first entity words and their location information in the initial text recognition result. Next, it decodes the phoneme recognition result of the target speech using a pre-built domain-specific entity word list to obtain second entity words. Then, it determines the similar entity words corresponding to the first and second entity words, and uses a preset language model to score the first entity words, the second entity words, and the similar entity words corresponding to the first and second entity words, obtaining a scoring result. Based on the scoring result, it determines whether error correction processing of the first entity word is needed using the second entity word or the similar entity words corresponding to the first and second entity words. If so, it determines the final recognition result corresponding to the target speech based on the obtained error correction result.

[0055] As can be seen, this application not only achieves high efficiency in text entity word detection, but also adds the advanced feature of correcting entity words based on phoneme recognition results, while not requiring updates to existing speech recognition models, thus effectively improving the efficiency and accuracy of speech recognition. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a speech recognition method provided in an embodiment of this application;

[0058] Figure 2 This is a schematic diagram of the structure of the speech recognition model provided in the embodiments of this application;

[0059] Figure 3 A schematic diagram illustrating the entity word correction process provided in the embodiments of this application;

[0060] Figure 4 This is a schematic diagram illustrating the composition of a speech recognition device provided in an embodiment of this application. Detailed Implementation

[0061] With the development of artificial intelligence and deep learning, speech recognition technology has been widely used, covering various fields of human-computer interaction. The core challenge of domain-specific speech recognition lies in the large number of domain-specific entity words. These entity words, especially lower-frequency words, typically appear less frequently in the training data of speech recognition models. Furthermore, domain-specific vocabulary is often open-ended, with new words constantly emerging; for example, in voice navigation applications, new company names and location names are constantly appearing. These characteristics of domain-specific entity words mean that in practical applications, if entity words in speech recognition can be detected and corrected, high accuracy in domain-specific speech recognition can be maintained, leading to a better user experience.

[0062] Traditional methods for improving the recognition rate of domain-specific vocabulary involve adding hot word stimuli to the language model. This method is suitable for traditional schemes where the acoustic and language models are modeled independently. However, the mainstream approach in current speech recognition systems is end-to-end modeling, including encoder-decoder and neural transducer methods. In end-to-end modeling, although the speech recognition decoding process typically uses an external language model to fuse scores with the end-to-end model, the contribution of the language model is usually small. Therefore, adding hot word stimuli to the language model alone has a very limited effect. To meet the recognition rate requirements of newly emerging domain-specific vocabulary, existing methods usually require recording or synthesizing sentences containing domain-specific vocabulary to update the speech recognition model, which is time-consuming and labor-intensive.

[0063] Currently, the methods for entity word detection and error correction in the domain of speech recognition typically include the following two:

[0064] The first approach involves detecting and correcting specialized entity words in plain text. This method uses a speech recognition model to obtain the recognition results, then employs natural language processing models such as BERT to detect entity words in the recognized sentence and correct unreasonable errors. However, this approach does not actually consider acoustic pronunciation features; it is separate from speech recognition. This leads to a situation where, although the final recognition result may seem to provide reasonable corrections, it may not accurately reflect the actual pronunciation, resulting in overcorrection.

[0065] The second approach involves directly correcting the speech recognition model. When dealing with domain-specific vocabulary, it's often necessary to utilize rules or a pre-trained context extension model to construct a large number of different contextual texts based on the words of the current domain entities. For example, after a new song A appears, phrases like "Give me a song A" or "I want to hear a new song A" need to be constructed. Next, a pre-trained speech synthesis model is used to synthesize the corresponding speech, and data enhancement operations such as adding noise, reverb, and timbre conversion are applied. Finally, the current speech recognition model is updated and iterated using the corpus. The resulting new model typically improves the recognition accuracy for newly added domain vocabulary. However, updating the speech recognition model cannot achieve zero-shot learning, making the entire process time-consuming, labor-intensive, and costly. Furthermore, the improvement in recognition accuracy is highly dependent on the constructed training corpus. For unconstructed contextual statements, the improvement in recognition accuracy is usually very limited.

[0066] Furthermore, updating speech recognition models carries the risk of incremental learning; that is, it's difficult to guarantee that the updated speech recognition model won't experience a decrease in accuracy when recognizing vocabulary from existing domains. Since existing speech recognition models are updated based on new corpora, they inevitably forget some of the previously trained data. To avoid this problem, it's often necessary to use synchronous training with a large amount of previous data, which is also time-consuming and labor-intensive.

[0067] To address the aforementioned shortcomings, this application provides a speech recognition method. First, the target speech to be recognized is acquired, and its audio features are generated. Then, the audio features are input into a pre-constructed speech recognition model to identify the phoneme recognition results of the target speech and the first entity words and their location information in the initial text recognition results. Next, using a pre-constructed domain-specific entity word list, the phoneme recognition results of the target speech are decoded to obtain second entity words. Then, similar entity words corresponding to the first and second entity words are determined, and a preset language model is used to score the first entity words, the second entity words, and their respective similar entity words to obtain scoring results. Based on the scoring results, it is determined whether error correction processing of the first entity word is needed using the second entity word or its corresponding similar entity words. If so, the final recognition result corresponding to the target speech is determined based on the obtained error correction result.

[0068] As can be seen, this application not only achieves high efficiency in text entity word detection, but also adds the advanced feature of correcting entity words based on phoneme recognition results, while not requiring updates to existing speech recognition models, thus effectively improving the efficiency and accuracy of speech recognition.

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] First Embodiment

[0071] See Figure 1 This is a flowchart illustrating a speech recognition method provided in this embodiment. The method includes the following steps:

[0072] S101: Obtain the target speech to be recognized and generate the audio features of the target speech.

[0073] In this embodiment, any speech to be recognized is defined as the target speech. It should be noted that this embodiment does not limit the language type of the target speech; for example, the target speech can be composed of Chinese or English. Furthermore, this embodiment does not limit the length of the target speech; for example, the target speech can be a sentence or a paragraph.

[0074] Understandably, the target speech can be obtained through recording or other means as needed. For example, telephone conversations in people's daily lives or recordings of conference systems can be used as target speech. After obtaining the target speech, existing or future feature extraction methods are used to extract the audio features of the target speech, which can then be used in subsequent steps S102 to achieve effective recognition of the target speech.

[0075] It should be noted that the embodiments of this application do not limit the method for extracting audio features of the target speech, nor do they limit the specific extraction process. Appropriate extraction methods and corresponding feature extraction operations can be selected according to the actual situation.

[0076] One optional implementation is that the audio features of the target speech can be the amplitude spectrum features of the target speech, such as Log Filter Bank Energy (LFBE). It should be noted that the embodiments of this application do not limit the method for generating the amplitude spectrum features of the target speech, nor do they limit the specific generation process. Appropriate generation methods and corresponding amplitude spectrum feature generation operations can be selected according to the actual situation.

[0077] S102: Input the audio features of the target speech into the pre-built speech recognition model to identify the phoneme recognition results of the target speech and the first entity words and their location information in the initial text recognition results.

[0078] In this embodiment, after generating the audio features (such as amplitude spectrum features) of the target speech in step S101, in order to effectively improve the recognition effect of the target speech, the audio features of the target speech can be further input into the pre-built speech recognition model to identify the first entity words and their position information in the target speech morpheme recognition result and the initial text recognition result, so as to execute the subsequent steps S103-S105.

[0079] It should be noted that, in order to improve the recognition effect of the target speech, this application pre-constructs a speech recognition model, such as... Figure 2 As shown, the model includes an encoding layer, a phoneme recognition layer, a decoding layer, and a text entity word detection layer. Thus, an optional implementation method is that the specific implementation process of step S102 may include the following steps S1021-S1024:

[0080] Step S1021: Input the audio features of the target speech into the coding layer of the speech recognition model for encoding processing to obtain the high-dimensional speech representation vector of the target speech.

[0081] In this implementation, after generating the audio features (such as amplitude spectrum features) of the target speech in step S101, the audio features of the target speech can be further input into the coding layer of the speech recognition model for encoding processing to obtain a high-dimensional speech representation vector of the target speech, which is then used to execute subsequent steps S1022 and S1023.

[0082] It should be noted that the encoding layer of the speech recognition model is used to extract the high-dimensional speech representation vector of the target speech. The specific structure is not limited in this application; for example, it can be composed of a convolutional neural network, a long short-term memory (LSTM) network, or a Transformer. The encoding layer can be represented as h = f(x), where h = [h1, h2, ..., h...]. T [x1, x2, ..., x] represents the high-dimensional speech representation vector output by the coding layer; x = [x1, x2, ..., x] T ] represents the input features of the coding layer, i.e., the audio features of the target speech (such as amplitude spectrum features); T represents the total number of speech frames of the target speech; f() represents the coding network function of the coding layer.

[0083] For example: Figure 2As shown, the audio features [x1,x2,…,x6] of the target speech are input into the coding layer of the speech recognition model for encoding processing, and the high-dimensional speech representation vector of the target speech is [h1,h2,…,h6].

[0084] Step S1022: Input the high-dimensional speech representation vector of the target speech into the phoneme recognition layer of the speech recognition model for recognition processing to obtain the phoneme recognition result of the target speech.

[0085] In this implementation, after obtaining the high-dimensional speech representation vector of the target speech through step S1021, the high-dimensional speech representation vector of the target speech can be further input into the phoneme recognition layer of the speech recognition model for recognition processing to obtain the phoneme recognition result of the target speech.

[0086] It should be noted that the phoneme recognition layer of the speech recognition model is used to obtain the phoneme sequence representing the pronunciation of the target speech. The phoneme recognition result of the target speech is typically the triphone classification result of each frame output by the coding layer. The specific structure is not limited in this application; for example, it can be composed of a convolutional neural network, LSTM, or Transformer. The phoneme recognition layer can be represented as c = m(h), where c = [c1, c2, ..., c...]. T ] represents the phoneme classification result of each frame of target speech, mainly to prepare for entity word correction in subsequent steps; h represents the high-dimensional speech representation vector output by the coding layer; T represents the total number of speech frames of the target speech; m() represents the network recognition function of the phoneme recognition layer.

[0087] For example: Figure 2 As shown, the high-dimensional speech representation vector of the target speech, [h1,h2,…,h6], is input into the phoneme recognition layer of the speech recognition model for recognition processing, and the triphone phoneme features of the target speech [c1,c2,…,c6] can be obtained.

[0088] Step S1023: Input the high-dimensional speech representation vector of the target speech into the decoding layer of the speech recognition model for decoding processing to obtain the initial text recognition result of the target speech.

[0089] In this implementation, after obtaining the high-dimensional speech representation vector of the target speech in step S1021, the high-dimensional speech representation vector of the target speech can be further input into the decoding layer of the speech recognition model for decoding processing to obtain the initial text recognition result of the target speech, which is then used to execute the subsequent step S1024.

[0090] It should be noted that the decoding layer of the speech recognition model is used to decode the high-dimensional speech representation vector of the target speech to obtain the preliminary text recognition result corresponding to the target speech. The specific structure of the decoding layer is not limited in this application; for example, it can be constructed using a Transformer or LSTM with attention and autoregression mechanisms. Specifically, the decoding layer can be represented as z i =g(z) i-1 z = [z1, z2, ..., zn], where h represents the high-dimensional speech representation vector output by the coding layer; z = [z1, z2, ..., zn] u The image represents the character recognition result output by the decoding layer (i.e., the preliminary text recognition result). This application takes autoregressive decoding layer modeling as an example, requiring the output character z from the previous time step to be... i-1 As the input condition at the current moment, the output character z at the current moment is obtained together with the output h of the coding layer. i T represents the total number of speech frames in the target speech; g() represents the decoding network function of the decoding layer.

[0091] For example: Figure 2 As shown, the high-dimensional speech representation vector of the target speech, [h1,h2,…,h6], is input into the decoding layer of the speech recognition model for decoding processing, and the initial text recognition result of the target speech, "Navigation Zhanghuang Town", can be obtained.

[0092] Step S1024: Input the initial text recognition result of the target speech into the text entity word detection layer of the speech recognition model for detection processing to obtain the first entity word and its position information in the initial text recognition result.

[0093] In this implementation, after obtaining the initial text recognition result of the target speech through step S1023, the initial text recognition result of the target speech can be further input into the text entity word detection layer of the speech recognition model for entity word detection processing to obtain the entity words (here defined as the first entity word) and their location information in the initial text recognition result.

[0094] It should be noted that the text entity word detection layer of the speech recognition model is used to detect the first entity word and its position information from the initial text recognition result. This application does not limit the specific structure of the text entity word detection layer; for example, it can use the main structure of a pre-trained BERT model, and then add an entity classification layer for entity result judgment for each character. The text entity word detection layer can be represented as e = n(z), where z represents the initial text recognition result output by the decoding layer, and e = [e1, e2, ..., e...]. uThe symbol ] represents the entity recognition result for each character. Each domain's entity vocabulary has a unique symbol. For example, the symbol for the entity vocabulary "Zhang Tao" in the personal name domain can be represented as "". <b-per> <i-per>The marker for the entity term "Anhui Province" in the field of place names can be represented as " <b-loc> <i-loc> <i-loc>Non-entity words are uniformly represented as " <o>".

[0095] For example: as Figure 2 shown, the initial text recognition result of the target speech "navigate to Zhanghuang Town" is input into the text entity word detection layer of the speech recognition model for detection processing, so that the first entity word, its position information and mark symbol in the initial text recognition result can be obtained.

[0096] Next, this embodiment will introduce the training process of the speech recognition model. It should be noted that in the present application, the speech recognition part (including the encoding layer, the phoneme recognition layer, and the decoding layer) and the entity word detection part (including the text entity word detection layer) of the speech recognition model are trained separately, so that single-modal corpus and pre-trained models can be used for each part respectively, no parallel data is required, existing data can be utilized to the greatest extent, and the labeling cost is not increased.

[0097] Specifically, for the training of the speech recognition part (including the encoding layer, the phoneme recognition layer, and the decoding layer) in the speech recognition model, pre-collected labeled texts and Triphone phoneme labels can be used as training data, and during the training process, the Cross Entropy criterion is adopted for training until a preset condition is met, for example, when the value of the cross-entropy loss constraint function is very small and basically unchanged, the update of the parameters of the speech recognition part (including the encoding layer, the phoneme recognition layer, and the decoding layer) in the speech recognition model is stopped, and the training of this part of the model is completed.

[0098] For the training of the entity word detection part (that is, the text entity word detection layer), the training data used can be plain text. Since it is a relatively independently trained part from the speech recognition part (including the encoding layer, the phoneme recognition layer, and the decoding layer), the available training corpus is not limited to speech-labeled texts, and plain text corpus without speech can also be used, as long as there are corresponding entity word position labels. To ensure the effect of entity word detection, the present application can use the text pre-trained model BERT for initialization, and at the same time perform some similar phonetic perturbations on the entity words, for example, perturb "Zhanghuang Town" (with a different Chinese character for Huang) into "Zhanghuang Town" (the target recognized word), so as to increase the robustness of the entity word detection part (that is, the text entity word detection layer). The specific training process is consistent with the existing manner, which will not be repeated here.

[0099] S103: Decode the phoneme recognition result of the target speech by using the pre-constructed domain professional entity word list to obtain a second entity word.

[0100] In this embodiment, after the phoneme recognition result of the target speech is obtained through identification in step S102, in order to improve the recognition effect on the target speech, the pre-constructed domain-specific professional entity vocabulary list may further be used to perform secondary decoding on the phoneme recognition result of the target speech, so as to obtain an entity vocabulary (which is defined herein as the second entity vocabulary), for executing the subsequent step S104 to realize effective recognition of the target speech.

[0101] Specifically, an optional implementation manner is that after the first entity vocabulary and the position information thereof in the phoneme recognition result and the initial text recognition result of the target speech are obtained through identification, the target domain classification to which the target speech belongs needs to be determined first according to the position information of the first entity vocabulary and the mark symbol of the first entity vocabulary. For example, for the first entity vocabulary "Zhang Tao", the corresponding mark symbol thereof is " <b-per> <i-per>The target domain of the target speech can be determined by the "PER" in the marker symbol, which indicates that the target speech belongs to the personal name domain, i.e., the "person" domain; or, for the first entity word "Anhui Province", the corresponding marker symbol is " <b-loc> <i-loc> <i-loc>If the target speech belongs to the local domain, it can be determined by the "LOC" symbol in the marker.

[0102] Then, existing or future alignment methods, such as an improved Viterbi algorithm, can be used to align the domain-specific entity words of the target domain classification with the phoneme recognition results of the target speech, obtaining the pronunciation sequence corresponding to the first entity word in the phoneme recognition results. For example: Figure 3 As shown, the pronunciation sequence aligned with the first entity word "Zhanghuangzhen" is "c3,c4,c5,c6".

[0103] Next, a pre-built list of domain-specific entity words categorized by the target domain can be packaged into decoding resources, such as a Weighted Finite State Transducer (WFST) resource. This decoding resource is then used to perform secondary decoding on the pronunciation sequences corresponding to the first entity words, yielding the second entity words. Compared to the first entity words, there's a preference for decoding these target domain entity words rather than a generic term. For example, for the place name "Zhanghuang Town," if a user says "Navigate Zhanghuang Town," the initial text recognition result from the decoding layer is very likely to be recognized as "Navigate Zhanghuang Town," because the character "huang" is more common than "huang." However, if the corpus used when packaging the WFST resource includes "Zhanghuang Town," then the probability of successfully decoding this entity word will be higher. Finally, through this decoding step, several second entity words can be obtained as candidate words to correct errors in the first entity words.

[0104] S104: Determine the similar entity words corresponding to the first entity word and the second entity word respectively, and use the preset language model to score the first entity word, the second entity word, and the similar entity words corresponding to the first entity word and the second entity word respectively, and obtain the scoring results.

[0105] In this embodiment, after obtaining the first entity word and the second entity word through steps S102 and S103 respectively, in order to effectively improve the recognition effect of the target speech, the first entity word and the second entity word can be further expanded by similar sounds, and the similar entity words corresponding to the first entity word and the second entity word can be determined according to the expansion results. For example, for the entity word "Zhanghuang Town", it can be expanded into "Zhanghuang Town", "Zhanghuang Town", and so on, and a series of its corresponding similar entity words.

[0106] Then, a preset language model can be used to score the first entity word, the second entity word, and the similar entity words corresponding to the first entity word and the second entity word respectively, and the scoring results can be used to execute the subsequent step S105.

[0107] The pre-defined language model can be obtained by statistically analyzing a list of entity words in a specific field, but this application categorizes it more precisely. For example, the language model for personal names is obtained solely from statistics on personal names, and the language model for place names is obtained solely from statistics on place names, etc.

[0108] S105: Based on the scoring results, determine whether it is necessary to use the second entity word or the similar entity words corresponding to the first entity word and the second entity word to perform error correction processing on the first entity word. If so, determine the final recognition result corresponding to the target speech based on the obtained error correction result.

[0109] In this embodiment, step S104 uses a preset language model to score the first entity word, the second entity word, and the similar entity words corresponding to the first and second entity words. After obtaining the scoring results, in order to effectively improve the recognition effect of the target speech, a first high score and a second score can be selected from the scores of the first entity word, the second entity word, and the similar entity words corresponding to the first and second entity words. Then, the difference between the first high score and the second score is calculated, and it is determined whether the difference is higher than a preset threshold. If so, the first entity word is corrected using the entity word corresponding to the first high score to obtain the final recognition result of the target speech. The specific value of the preset threshold can be set according to the actual situation and experience. This application does not limit it, for example, it can be set to -10.

[0110] It should be noted that for different types of entity words, after obtaining numerous candidate entity words such as second entity words and similar entity words, only the language model of the corresponding category is used for scoring. This can minimize interference from different entity types. After scoring, the saliency of the language model is set, which can be calculated as: W = Score top1 -Score top2 Only when the score of the first-ranked entity term among all candidate entity terms is significantly higher than that of the second-ranked term (i.e., the value of W is higher than a preset threshold) will the first entity term be corrected; otherwise, no action is taken. This is because if the specialized entity term has never appeared in the entity term list, the scores of its candidates will be relatively similar, resulting in lower significance, and further correction may lead to overcorrection.

[0111] In summary, the speech recognition method provided in this embodiment first acquires the target speech to be recognized, then detects the start frame and end frame of the target speech through real-time speech activity endpoint detection, and determines the effective audio segments in the target speech based on the start frame and end frame; next, it extracts the audio features of the effective audio segments in the target speech; and decodes the target speech based on the audio features to obtain a preliminary recognition result; then, it adaptively and dynamically adjusts a preset pruning threshold based on the preliminary recognition result; and prunes the target speech and the preliminary recognition result based on the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech.

[0112] As can be seen, this application first uses real-time speech activity endpoint detection to more accurately detect the start and end frames of the target speech, and then adaptively and dynamically adjusts the preset pruning threshold based on the preliminary audio recognition results. This not only prunes the truly effective audio segments in the target speech, but also reduces false triggering of noise pruning, thereby effectively improving the accuracy of the recognition results and enhancing the user experience and recognition effect.

[0113] Second Embodiment

[0114] This embodiment will introduce a speech recognition device; please refer to the above method embodiment for related content.

[0115] See Figure 4 This is a schematic diagram of the composition of a voice recognition device provided in this embodiment. The device 400 includes:

[0116] The acquisition unit 401 is used to acquire the target speech to be identified and generate the audio features of the target speech;

[0117] The recognition unit 402 is used to input the audio features of the target speech into a pre-built speech recognition model to recognize the phoneme recognition result of the target speech and the first entity word and its position information in the initial text recognition result;

[0118] Decoding unit 403 is used to decode the phoneme recognition result of the target speech using a pre-built domain-specific entity vocabulary list to obtain a second entity vocabulary;

[0119] The scoring unit 404 is used to determine the similar entity words corresponding to the first entity word and the second entity word respectively, and to score the first entity word, the second entity word and the similar entity words corresponding to the first entity word and the second entity word respectively using a preset language model to obtain the scoring result.

[0120] The error correction unit 405 is used to determine, based on the scoring result, whether it is necessary to use the second entity word or the similar entity words corresponding to the first entity word and the second entity word to perform error correction processing on the first entity word. If so, the final recognition result corresponding to the target speech is determined based on the obtained error correction result.

[0121] In one implementation of this embodiment, the audio feature of the target speech is the amplitude spectrum feature of the target speech.

[0122] In one implementation of this embodiment, the speech recognition model includes an encoding layer, a phoneme recognition layer, a decoding layer, and a text entity vocabulary detection layer; the recognition unit 402 includes:

[0123] The first input subunit is used to input the audio features of the target speech into the coding layer of the speech recognition model for encoding processing to obtain a high-dimensional speech representation vector of the target speech;

[0124] The second input subunit is used to input the high-dimensional speech representation vector of the target speech into the phoneme recognition layer of the speech recognition model for recognition processing, so as to obtain the phoneme recognition result of the target speech.

[0125] The third input subunit is used to input the high-dimensional speech representation vector of the target speech into the decoding layer of the speech recognition model for decoding processing, so as to obtain the initial text recognition result of the target speech;

[0126] The fourth input subunit is used to input the initial text recognition result of the target speech into the text entity word detection layer of the speech recognition model for detection processing, so as to obtain the first entity word and its position information in the initial text recognition result.

[0127] In one implementation of this embodiment, the apparatus further includes:

[0128] The determining unit is configured to determine the target domain classification to which the target speech belongs based on the location information of the first entity word and the tag symbol of the first entity word.

[0129] In one implementation of this embodiment, the decoding unit 403 includes:

[0130] The alignment subunit is used to align the domain-specific entity words of the target domain classification with the phoneme recognition results of the target speech to obtain the pronunciation sequence corresponding to the first entity word in the phoneme recognition results.

[0131] The decoding subunit is used to package the pre-constructed list of domain-specific entity words of the target domain classification into a decoding resource; and use the decoding resource to decode the pronunciation sequence corresponding to the first entity word to obtain the second entity word.

[0132] In one implementation of this embodiment, the scoring unit 404 is specifically used for:

[0133] The first entity word and the second entity word are expanded by similar sounds, and the similar entity words corresponding to the first entity word and the second entity word are determined according to the expansion results.

[0134] In one implementation of this embodiment, the error correction unit 405 includes:

[0135] The selection subunit is used to select the first high score and the second score from the scores of the first entity word, the scores of the second entity word, and the scores of the similar entity words corresponding to the first entity word and the second entity word respectively.

[0136] The error correction subunit is used to calculate the difference between the first high score and the second score, and to determine whether the difference is higher than a preset threshold. If so, the first entity word is corrected using the entity word corresponding to the first high score, so as to obtain the final recognition result corresponding to the target speech.

[0137] Furthermore, embodiments of this application also provide a voice recognition device, including: a processor, a memory, and a system bus;

[0138] The processor and the memory are connected via the system bus;

[0139] The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform any of the above-described implementations of the speech recognition method.

[0140] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the speech recognition method.

[0141] Furthermore, this application embodiment also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described implementation methods of the speech recognition method.

[0142] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0143] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0144] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. < / i-loc> < / b-loc> < / b-per> < / o> < / i-loc> < / b-loc> < / b-per>

Claims

1. A speech recognition method, characterized in that, include: Acquire the target speech to be identified and generate the audio features of the target speech; The audio features of the target speech are input into a pre-built speech recognition model to identify the phoneme recognition results of the target speech and the first entity words and their location information in the initial text recognition results; the phoneme recognition results are the triphone classification results of each frame of target speech; Based on the location information and the marker symbol of the first entity word, the target domain classification to which the target speech belongs is determined; Align the domain-specific entity words of the target domain classification with the phoneme recognition results of the target speech to obtain the pronunciation sequence corresponding to the first entity word in the phoneme recognition results; The pre-constructed list of domain-specific entity words for the target domain classification is packaged into a decoding resource; and the decoding resource is used to decode the pronunciation sequence corresponding to the first entity word to obtain the second entity word. Identify the similar entity words corresponding to the first entity word and the second entity word respectively, and use a preset language model corresponding to the category of the target domain classification to score the first entity word, the second entity word and the similar entity words corresponding to the first entity word and the second entity word respectively, and obtain the scoring results. Based on the scoring results, it is determined whether it is necessary to use the second entity word or the similar entity words corresponding to the first entity word and the second entity word to perform error correction processing on the first entity word. If so, the final recognition result corresponding to the target speech is determined based on the obtained error correction result.

2. The method according to claim 1, characterized in that, The audio features of the target speech are the amplitude spectrum features of the target speech.

3. The method according to claim 1, characterized in that, The speech recognition model includes an encoding layer, a phoneme recognition layer, a decoding layer, and a text entity word detection layer; the step of inputting the audio features of the target speech into the pre-constructed speech recognition model to identify the phoneme recognition result of the target speech and the first entity word and its location information in the initial text recognition result includes: The audio features of the target speech are input into the coding layer of the speech recognition model for encoding processing to obtain a high-dimensional speech representation vector of the target speech; The high-dimensional speech representation vector of the target speech is input into the phoneme recognition layer of the speech recognition model for recognition processing to obtain the phoneme recognition result of the target speech; The high-dimensional speech representation vector of the target speech is input into the decoding layer of the speech recognition model for decoding processing to obtain the initial text recognition result of the target speech; The initial text recognition result of the target speech is input into the text entity word detection layer of the speech recognition model for detection processing to obtain the first entity word and its position information in the initial text recognition result.

4. The method according to claim 1, characterized in that, The step of determining the similar entity words corresponding to the first entity word and the second entity word includes: The first entity word and the second entity word are expanded by similar sounds, and the similar entity words corresponding to the first entity word and the second entity word are determined according to the expansion results.

5. The method according to any one of claims 1-4, characterized in that, The step of determining whether to use the second entity word or similar entity words corresponding to the first entity word and the second entity word for error correction based on the scoring result includes: Select the highest score and the second highest score from the scores of the first entity word, the second entity word, and the scores of the similar entity words corresponding to the first entity word and the second entity word respectively. Calculate the difference between the first high score and the second score, and determine whether the difference is higher than a preset threshold. If so, use the entity words corresponding to the first high score to perform error correction processing on the first entity words to obtain the final recognition result corresponding to the target speech.

6. A voice recognition device, characterized in that, include: An acquisition unit is used to acquire the target speech to be identified and generate the audio features of the target speech; The recognition unit is used to input the audio features of the target speech into a pre-built speech recognition model to identify the first entity words and their location information in the phoneme recognition result and the initial text recognition result of the target speech; the phoneme recognition result is the triphone classification result of each frame of target speech; The determining unit is configured to determine the target domain classification to which the target speech belongs based on the location information of the first entity word and the marker symbol of the first entity word; The decoding unit is used to decode the phoneme recognition results of the target speech using a pre-built list of domain-specific entity words to obtain a second entity word; The scoring unit is used to determine the similar entity words corresponding to the first entity word and the second entity word respectively, and to score the first entity word, the second entity word and the similar entity words corresponding to the first entity word and the second entity word respectively using a preset language model corresponding to the category of the target domain classification, so as to obtain the scoring result. The error correction unit is used to determine, based on the scoring result, whether it is necessary to use the second entity word or the similar entity words corresponding to the first entity word and the second entity word to perform error correction processing on the first entity word. If so, the final recognition result corresponding to the target speech is determined based on the obtained error correction result. The decoding unit includes: The alignment subunit is used to align the domain-specific entity words of the target domain classification with the phoneme recognition results of the target speech to obtain the pronunciation sequence corresponding to the first entity word in the phoneme recognition results. The decoding subunit is used to package the pre-constructed list of domain-specific entity words of the target domain classification into a decoding resource; and use the decoding resource to decode the pronunciation sequence corresponding to the first entity word to obtain the second entity word.

7. A voice recognition device, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-5.

Images