Speech recognition method and device, intelligent device and storage medium

Abstract

The application is suitable for the technical field of intelligent devices, and provides a voice recognition method and device, an intelligent device and a storage medium. The method comprises the following steps: inputting a to-be-recognized voice into a trained voice recognition model, and obtaining a feature probability spectrum of the to-be-recognized voice; decoding and searching the feature probability spectrum to obtain a first search result; decoding and searching the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model used to provide a scene grammar rule; and determining and outputting a recognition result of the to-be-recognized voice according to a fusion result of the first search result and the second search result. The application can effectively exclude the interference of a non-scene high-probability path on a search result, improve the accuracy of voice recognition in a real-time voice recognition scene, improve the performance of voice recognition, and thus enhance the user experience.

Description

Technical Field

[0001] This application relates to the field of smart device technology, and in particular to a voice recognition method, apparatus, smart device, and storage medium. Background Technology

[0002] With the continuous advancement of deep learning research, speech recognition, as an important branch of artificial intelligence, has made significant progress in recent years. The introduction of end-to-end methods has led to a qualitative leap in speech recognition performance and user experience. However, unlike laboratory tests, real-world speech recognition services face more diverse and complex noise environments and device interference. Furthermore, their audience is much broader, encompassing people with different accents and pronunciation habits, as well as various chat scenarios across different fields. This results in speech recognition performance being significantly lower than in laboratory tests.

[0003] Real-time speech recognition applications require services to immediately provide feedback on recognition results and respond accordingly. Improving the accuracy, performance, and user experience of speech recognition in real-time scenarios is a pressing issue that needs to be addressed. Summary of the Invention

[0004] This application provides a speech recognition method, apparatus, smart device, and storage medium, which can improve the accuracy of speech recognition in real-time speech recognition scenarios, enhance speech recognition performance, and improve user experience.

[0005] In a first aspect, embodiments of this application provide a speech recognition method, including:

[0006] The speech to be recognized is input into a pre-trained speech recognition model to obtain the feature probability spectrum of the speech to be recognized;

[0007] Decode and search the feature probability spectrum to obtain the first search result;

[0008] The feature probability spectrum is decoded and searched based on the scene word graph model to obtain the second search result. The scene word graph model is a language model used to provide scene grammar rules.

[0009] Based on the fusion result of the first search result and the second search result, the recognition result of the speech to be recognized is determined and output.

[0010] In one possible implementation of the first aspect, the scene word graph model includes a first word graph model, and the step of decoding and searching the feature probability spectrum based on the scene word graph model to obtain a second search result includes:

[0011] Based on the first word graph model, the feature probability spectrum is decoded and searched to obtain the first sub-search result. The first word graph model is a language model trained on short word texts and used to provide short word grammar rules.

[0012] The step of determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result includes:

[0013] Based on the fusion result of the first search result and the first sub-search result, the recognition result of the speech to be recognized is determined and output.

[0014] In one possible implementation of the first aspect, the scene word graph model further includes a second word graph model, and the step of decoding and searching the feature probability spectrum based on the scene word graph model to obtain the second search result further includes:

[0015] Based on the second word graph model, the feature probability spectrum is decoded and searched to obtain the second sub-search result. The second word graph model is a language model trained on text in a specific domain and used to provide grammatical rules for that specific domain.

[0016] The step of determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result includes:

[0017] Based on the fusion result of the first search result, the first sub-search result, and the second sub-search result, the recognition result of the speech to be recognized is determined and output.

[0018] In one possible implementation of the first aspect, the search results include a search path and its corresponding path score; determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result, and the second sub-search result includes:

[0019] Determine the target search path, which is the search path corresponding to the highest path score among the first search result, the first sub-search result, and the second sub-search result;

[0020] Based on the target search path, the recognition result of the speech to be recognized is determined and output.

[0021] In one possible implementation of the first aspect, determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result, and the second sub-search result includes:

[0022] Obtain the audio duration of the speech to be recognized;

[0023] If the audio duration exceeds a preset duration threshold, the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result and the second sub-search result.

[0024] If the audio duration is less than or equal to the preset duration threshold, the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result, the first sub-search result, and the second sub-search result.

[0025] In one possible implementation of the first aspect, before decoding and searching the feature probability spectrum based on the first word graph model to obtain the first sub-search result, the following is included:

[0026] A specified number of preset short word test sets are sequentially input into the trained speech recognition model to obtain pseudo-labels;

[0027] The scene samples are filtered based on the preset audio duration and the pseudo-tags to obtain target short word samples;

[0028] The language model is trained using the target short word samples to obtain the first word graph model.

[0029] In one possible implementation of the first aspect, determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result includes:

[0030] If the speech to be recognized has contextual knowledge, then the relevance between the search path in the first search result and the second search result and the contextual knowledge is calculated respectively;

[0031] The first target path score is determined based on the degree of relevance between the search path in the first search result and the contextual knowledge.

[0032] The second target path score is determined based on the degree of relevance between the search path in the second search result and the contextual knowledge.

[0033] Determine the target search path, which is the search path corresponding to the higher path score between the first target path score and the second target path score;

[0034] Based on the target search path, the recognition result of the speech to be recognized is determined and output.

[0035] Secondly, embodiments of this application provide a voice recognition device, the device comprising:

[0036] The recognition information acquisition unit is used to input the speech to be recognized into the trained speech recognition model and obtain the feature probability spectrum of the speech to be recognized.

[0037] The first search unit is used to decode and search the feature probability spectrum to obtain the first search result;

[0038] The second search unit is used to decode and search the feature probability spectrum based on the scene word graph model to obtain the second search result. The scene word graph model is a language model used to provide scene grammar rules.

[0039] The recognition result determination output unit is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result.

[0040] Thirdly, embodiments of this application provide an intelligent device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the speech recognition method as described in the first aspect above.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the speech recognition method as described in the first aspect above.

[0042] Fifthly, embodiments of this application provide a computer program product that, when run on a smart device, causes the smart device to execute the speech recognition method described in the first aspect above.

[0043] In this embodiment, the speech to be recognized is input into a trained speech recognition model to obtain a feature probability spectrum. The feature probability spectrum is then decoded and searched to obtain a first search result. A second search result is obtained by decoding and searching the feature probability spectrum based on a scene word graph model, where the scene word graph model is a language model that provides scene grammar rules. Finally, the recognition result of the speech to be recognized is determined and output based on the fusion result of the first and second search results. This application limits the scene through a word graph model, defining a legal range for speech recognition. This effectively eliminates interference from high-probability paths outside the scene on the search results, thereby improving the accuracy of speech recognition in real-time scenarios, enhancing speech recognition performance, and ultimately improving the user experience. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the implementation of a speech recognition method provided in an embodiment of this application;

[0046] Figure 2 This is a flowchart illustrating the specific implementation of step S104 of the speech recognition method provided in this application embodiment;

[0047] Figure 3 This is a flowchart illustrating a specific implementation of the speech recognition method for obtaining a first word graph model provided in this application embodiment;

[0048] Figure 4 This is a flowchart illustrating a specific implementation of the speech recognition method provided in this application, which determines and outputs the speech recognition result.

[0049] Figure 5 This is another specific implementation flowchart of the speech recognition method provided in the embodiments of this application for determining and outputting speech recognition results;

[0050] Figure 6 This is a structural block diagram of the speech recognition device provided in the embodiments of this application;

[0051] Figure 7 This is a schematic diagram of the smart device provided in the embodiments of this application. Detailed Implementation

[0052] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0053] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0054] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0055] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0056] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0058] Traditional speech recognition decoding schemes typically process data sentence by sentence and train deep learning models based on neural networks. The number of layers and contextual structure of the model determine the fixed receptive field of the data frame. Test data with a duration similar to the training samples yields the best matching results for the model. For long speech samples containing many words, an error of one or two words may not have a significant impact on semantic parsing. However, for short speech samples containing only a few words, a single recognition error can directly lead to incorrect semantic parsing. In real-time speech recognition scenarios such as intelligent customer service in complex and noisy environments, audio durations are often short, making speech recognition models prone to errors. Coupled with complex noise and human interference, recognition performance may further degrade.

[0059] To address the aforementioned issues, embodiments of this application provide a speech recognition method, apparatus, smart device, and storage medium, as detailed below.

[0060] It should be understood that the speech recognition method provided in this application is applicable to smart devices that require speech recognition of various types of data, specifically including smart devices such as mobile phones, tablets, wearable devices, laptops, and desktop computers. This application does not impose any limitations on the specific type of smart device.

[0061] The speech recognition method provided in this application will be described exemplarily below with reference to specific embodiments.

[0062] Figure 1 The implementation flow of the speech recognition method provided in this application embodiment is illustrated. The method flow includes steps S101 to S104. The specific implementation principle of each step is as follows:

[0063] Step S101: Input the speech to be recognized into the trained speech recognition model to obtain the feature probability spectrum of the speech to be recognized.

[0064] The aforementioned speech recognition model is used to convert audio into sentences. In this embodiment, the speech recognition model is trained based on labeled audio data. This speech recognition model can be a general-purpose model, learning the correspondence between audio and text through a large amount of audio and corresponding labels, and is applicable to different scenarios.

[0065] In the embodiments of this application, the above-mentioned speech recognition model can be a phoneme-level model based on a hybrid framework, or a model based on a word-level end-to-end framework.

[0066] The hybrid framework's speech recognition model consists of an acoustic model and a language model. The speech recognition model is built at the phoneme level (initials and finals in Chinese). The acoustic model converts audio to pronunciation, while the language model converts text to sentences. Decoding by the acoustic model requires the assistance of the language model. The language model assigns higher weights to more logical grammar, allowing pronunciation to be converted into more coherent sentences; that is, the language model guides the conversion of pronunciation into grammatically correct sentences. The output of the hybrid framework's speech recognition model is a phoneme sequence.

[0067] The end-to-end framework speech recognition model includes a neural network model built at the word level, capable of directly converting audio into spoken text. This neural network model learns both audio and grammatical information, enabling a single model to complete the entire recognition and conversion process. The output of the end-to-end framework speech recognition model is a sequence of words.

[0068] The aforementioned feature probability spectrum is a two-dimensional matrix representing the speech feature frames and their classifications. Audio is input into the speech recognition model frame by frame, and each frame yields a vector. The dimension of the vector is the number of output layer nodes (number of words in an end-to-end framework, number of phonemes in a hybrid framework), and the vector value is the probability value for each category, summing to 1. Assuming a frame length of 10ms, one second of speech contains 100 frames, and its output is a two-dimensional matrix (probability spectrum) of 100 * the number of categories.

[0069] Step S102: Decode and search the feature probability spectrum to obtain the first search result.

[0070] In this embodiment, the feature probability is subjected to a maximum probability search without model limitations based on beam search, that is, a decoding search of the maximum range is performed to obtain the first search result.

[0071] For example, the first search result is obtained by performing a full-graph search without syntax restrictions based on CTC.

[0072] Step S103: Decode and search the feature probability spectrum based on the scene word graph model to obtain a second search result. The scene word graph model is a language model used to provide scene grammar rules.

[0073] In this embodiment, the feature probabilities are searched using a scenario word graph model based on beam search to obtain a second search result.

[0074] The scene word graph model is obtained by training a statistical probability-based language model with scene samples and WFST. The scene word graph model can be a short word word graph model, a domain-specific word graph model, etc. The specific domain can be a limited professional field. In this embodiment, the scene word graph model is used to limit the scene, narrow the decoding search scope, and thus improve the efficiency of the decoding search.

[0075] Step S104: Based on the fusion result of the first search result and the second search result, determine and output the recognition result of the speech to be recognized.

[0076] The search results include the search path and its corresponding path score. The first search result includes a first search path and its corresponding path score, and the second search result includes a second search path and its corresponding path score. The search path includes a complete sentence composed of words or phonemes, and the path score represents the credibility of the sentence; the higher the path score, the higher the credibility that the sentence represents the true content of the speech to be recognized.

[0077] In this embodiment, the search path corresponding to the higher path score between the first search result and the second search result is determined as the target search path, and the complete sentence corresponding to the target search path is output to obtain the recognition result of the speech to be recognized.

[0078] As one possible implementation of this application Figure 2 A specific implementation flow of step S104 of the speech recognition method provided in this application embodiment is shown below:

[0079] A1: If the speech to be recognized has contextual knowledge, then calculate the degree of relevance between the search path in the first search result and the second search result and the contextual knowledge.

[0080] Contextual knowledge can be the content of the previous sentence during a dialogue or outbound call, or it can be information related to the voice to be recognized, such as order information. In this embodiment, contextual knowledge is defined according to the specific business scenario.

[0081] The first search result includes a first search path, and the second search result includes a second search path. The relevance of the first search path, the second search path, and the contextual knowledge are calculated respectively.

[0082] A2: Determine the first target path score based on the degree of relevance between the search path in the first search result and the contextual knowledge.

[0083] A3: Determine the second target path score based on the degree of relevance between the search path in the second search result and the contextual knowledge.

[0084] In this embodiment, a weight corresponding to a search path is determined based on the relevance of the search path to the contextual knowledge, and a target path score is determined based on the weight and the path score corresponding to the search path. Specifically, a first weight is determined based on the relevance of the first search path to the contextual knowledge, and a first target path score is determined based on the first weight and the path score corresponding to the first search path; a second weight is determined based on the relevance of the second search path to the contextual knowledge, and a second target path score is determined based on the second weight and the path score corresponding to the second search path.

[0085] For example, the path score corresponding to the first search path y1 is w1, and the path score corresponding to the second search path y2 is w2. After rewarding based on contextual relevance, the first target path score corresponding to the first search path is P(y1|x)*w1, and the second target path score is P(y2|x)*w2, where x represents contextual knowledge. The relevance between contextual knowledge and the search path can be determined by converting the text into embedding vectors and calculating the edit distance, or it can be calculated based on the BERT model.

[0086] A4: Determine the target search path, which is the search path corresponding to the higher path score between the first target path score and the second target path score.

[0087] For example, if the first target path has a higher score, the first search path is determined as the target search path; if the second target path has a higher score, the second search path is determined as the target search path.

[0088] A5: Based on the target search path, determine and output the recognition result of the speech to be recognized. Output the complete sentence corresponding to the target search path to obtain the recognition result of the speech to be recognized.

[0089] In this embodiment of the application, the path score is weighted by calculating the correlation between the contextual knowledge of the speech to be recognized and the search path, which makes the search path of the correct scenario more likely to be selected.

[0090] In one possible implementation, the scene word graph model includes a first word graph model, which is a language model trained on short word texts to provide grammatical rules for those short words. The second search result includes a first sub-search result.

[0091] In this embodiment, the feature probability spectrum is decoded and searched based on the first word graph model, and the search is limited to reaching the end of the word graph within a preset time (e.g., within 2 seconds) to obtain the first sub-search result. Based on the fusion result of the first search result and the first sub-search result, the recognition result of the speech to be recognized is determined and output.

[0092] Specifically, the search path corresponding to the higher path score in the first search result and the first sub-search result is determined as the target search path, and the complete sentence corresponding to the target search path is output to obtain the recognition result of the speech to be recognized.

[0093] In this embodiment, the first word graph model, which provides short word grammar rules, is used to limit the short word search for speech recognition. This can eliminate the interference of high-probability non-short word paths on the search results, thereby making the speech recognition more accurate.

[0094] As one possible implementation of this application, the first word graph model is trained based on the already trained speech recognition model. The first word graph model is obtained before decoding and searching the feature probability spectrum based on the first word graph model to obtain the first sub-search result. Figure 3 A specific implementation process for obtaining the first word graph model is shown below in detail:

[0095] B1: Input a specified number of preset short word test sets sequentially into the trained speech recognition model to obtain pseudo-labels. The pseudo-labels are the speech recognition results corresponding to the preset short word test sets.

[0096] In this embodiment, the process of obtaining pseudo-labels differs depending on the framework of the speech recognition model. For the hybrid framework speech recognition model, the phoneme-level acoustic model needs to be combined with the statistical probability-based language model for decoding to obtain the results. For the end-to-end framework speech recognition model, the results are obtained by performing a full graph search without syntax restrictions based on CTC.

[0097] B2: Filter the scene samples based on the preset audio duration and the pseudo-tags to obtain target short word samples. Filter the scene samples, and determine the scene samples whose audio duration does not exceed the preset audio duration and whose pseudo-tag character count does not exceed the preset character count as target short word samples.

[0098] One possible implementation involves training an initial language model based on statistical probability using labeled audio data text. This initial language model is then used to calculate the confusion level, and target short word samples that meet the requirements are filtered out by setting a confusion level threshold. The initial language model provides the probability of connections between words; for example, P(w3|w2,w1) represents the probability that w3 is preceded by w1 or w2. The probability of a sentence is the product of P.

[0099] B3: The language model is trained using the target short word samples to obtain the first word graph model.

[0100] For example, in one application scenario, a trained speech recognition model is used to decode short audio clips from a real-world scene to obtain recognition results (pseudo-labels). Samples that meet the requirements in terms of the number of characters, audio length, and grammatical fluency are selected as target short word samples. These target short word samples are then used to train a language model based on statistical probability or RNN, and a first word graph model is constructed based on WFST.

[0101] In this embodiment, pseudo-labels are obtained using the trained speech recognition model. By filtering out the pseudo-labels, the effectiveness of the training samples is improved, thereby increasing the efficiency and accuracy of the first word graph model training.

[0102] In one possible implementation, the scene word graph model further includes a second word graph model, which is a language model trained on domain-specific text to provide domain-specific grammatical rules. The second search result includes a second sub-search result.

[0103] In this embodiment, the feature probability spectrum is decoded and searched based on the second word graph model to obtain a second sub-search result. Based on the fusion result of the first search result, the first sub-search result, and the second sub-search result, the recognition result of the speech to be recognized is determined and output.

[0104] Specifically, the search path corresponding to the higher path score in the first search result and the first sub-search result is determined as the target search path, and the complete sentence corresponding to the target search path is output to obtain the recognition result of the speech to be recognized.

[0105] As one possible implementation of this application, the search results include the search path and its corresponding path score. Figure 4 The following illustrates the specific implementation process of the speech recognition method provided in this application, which determines and outputs the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result, and the second sub-search result.

[0106] C1: Determine the target search path and its path score, wherein the target search path is the search path corresponding to the highest path score among the first search result, the second search result, and the third search result.

[0107] C2: Based on the target search path, determine and output the recognition result of the speech to be recognized. That is,

[0108] The path scores in the first search result, the first sub-search result, and the second sub-search result are compared. The search path corresponding to the highest path score among the three is determined as the target search path. The complete sentence corresponding to the target search path is output, thereby obtaining the recognition result of the speech to be recognized.

[0109] In this embodiment, the second word graph model can be a word graph model for specific domains such as address words, names, and order information. This model is used to limit the search to specific scenarios, thereby further eliminating the interference of high-probability paths outside the scenario on the search results and improving the accuracy of speech recognition.

[0110] As one possible implementation of this application Figure 5The following illustrates the specific implementation process of the speech recognition method provided in this application, which determines and outputs the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result, and the second sub-search result.

[0111] D1: Obtain the audio duration of the speech to be recognized.

[0112] D2: If the audio duration is greater than a preset duration threshold, the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result and the second sub-search result.

[0113] D3: If the audio duration is less than or equal to the preset duration threshold, then the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result, the first sub-search result, and the second sub-search result.

[0114] In this embodiment, the search results for fusion are determined by combining the audio duration. For speech to be recognized that exceeds a preset duration threshold, the recognition result of the speech to be recognized is determined and output only by fusing the search results of the above-mentioned speech recognition model and the second word graph model, which can avoid wasting search time. For speech to be recognized that is within the preset duration threshold, the recognition result of the speech to be recognized is determined and output by fusing the search results of the above-mentioned speech recognition model, the first word graph model and the second word graph model.

[0115] In this embodiment, the speech to be recognized is input into a trained speech recognition model to obtain a feature probability spectrum. The feature probability spectrum is then decoded and searched to obtain a first search result. A second search result is obtained by decoding and searching the feature probability spectrum based on a scene word graph model, where the scene word graph model is a language model that provides scene grammar rules. Finally, the recognition result of the speech to be recognized is determined and output based on the fusion result of the first and second search results. This application limits the scene through a word graph model, thus defining a legal range for speech recognition. This effectively eliminates interference from high-probability paths outside the scene on the search results, improves the accuracy of speech recognition in real-time scenarios, enhances speech recognition performance, and ultimately improves the user experience.

[0116] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0117] Corresponding to the speech recognition method described in the above embodiments, Figure 6A structural block diagram of a speech recognition device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0118] Reference Figure 6 The speech recognition device includes: a recognition information acquisition unit 61, a first search unit 62, a second search unit 63, and a recognition result determination and output unit 64, wherein:

[0119] The recognition information acquisition unit 61 is used to input the speech to be recognized into the trained speech recognition model and obtain the feature probability spectrum of the speech to be recognized.

[0120] The first search unit 62 is used to decode and search the feature probability spectrum to obtain the first search result.

[0121] The second search unit 63 is used to decode and search the feature probability spectrum based on the scene word graph model to obtain the second search result. The scene word graph model is a language model used to provide scene grammar rules.

[0122] The recognition result determination output unit 64 is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result.

[0123] As one possible implementation of this application, the scene word graph model includes a first word graph model, and the second search unit 63 includes:

[0124] The first sub-search module is used to decode and search the feature probability spectrum based on the first word graph model to obtain the first sub-search result. The first word graph model is a language model trained on short word texts and used to provide short word grammar rules.

[0125] The above recognition results determine that output unit 64 is specifically used for:

[0126] Based on the fusion result of the first search result and the first sub-search result, the recognition result of the speech to be recognized is determined and output.

[0127] As one possible implementation of this application, the scene word graph model further includes a second word graph model, and the second search unit 63 further includes:

[0128] The second sub-search module is used to decode and search the feature probability spectrum based on the second word graph model to obtain the third search result. The second word graph model is a language model trained on text in a specific domain and used to provide grammatical rules for that specific domain.

[0129] The above recognition results determine that output unit 64 is specifically used for:

[0130] Based on the fusion result of the first search result, the first sub-search result, and the second sub-search result, the recognition result of the speech to be recognized is determined and output.

[0131] As one possible implementation of this application, the above-mentioned identification result determination output unit 64 includes:

[0132] The first target path determination module is used to determine the target search path, wherein the target search path is the search path corresponding to the highest path score among the first search result, the first sub-search result and the second sub-search result.

[0133] The first recognition result output module is used to determine and output the recognition result of the speech to be recognized based on the target search path.

[0134] As one possible implementation of this application, the above-mentioned identification result determination output unit 64 includes:

[0135] An audio duration acquisition module is used to acquire the audio duration of the speech to be recognized;

[0136] The second recognition result output module is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result and the second sub-search result if the audio duration is greater than a preset duration threshold.

[0137] The third recognition result output module is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result and the second sub-search result if the audio duration is less than or equal to the preset duration threshold.

[0138] As one possible implementation of this application, the above-mentioned language recognition device further includes:

[0139] The pseudo-label acquisition unit is used to sequentially input a specified number of preset short word test sets into the trained speech recognition model to acquire pseudo-labels;

[0140] The target short word sample acquisition unit is used to filter scene samples according to the preset audio duration and the pseudo-tags to acquire target short word samples;

[0141] The first word graph model training unit is used to train the language model using the target short word samples to obtain the first word graph model.

[0142] As one possible implementation of this application, the above-mentioned identification result determination output unit 64 further includes:

[0143] The relevance calculation module is used to calculate the relevance between the search path in the first search result and the contextual knowledge if the speech to be recognized has contextual knowledge.

[0144] The first path score determination module is used to determine the first target path score based on the degree of relevance between the search path in the first search result and the context knowledge.

[0145] The second path score determination module is used to determine the second target path score based on the degree of relevance between the search path in the second search result and the context knowledge.

[0146] The second target path determination module is used to determine the target search path, wherein the target search path is the search path corresponding to the higher path score between the first target path score and the second target path score.

[0147] The fourth recognition result output module is used to determine and output the recognition result of the speech to be recognized based on the target search path.

[0148] In this embodiment, the speech to be recognized is input into a trained speech recognition model to obtain a feature probability spectrum. The feature probability spectrum is then decoded and searched to obtain a first search result. A second search result is obtained by decoding and searching the feature probability spectrum based on a scene word graph model, where the scene word graph model is a language model that provides scene grammar rules. Finally, the recognition result of the speech to be recognized is determined and output based on the fusion result of the first and second search results. This application limits the scene through a word graph model, thus defining a legal range for speech recognition. This effectively eliminates interference from high-probability paths outside the scene on the search results, improves the accuracy of speech recognition in real-time scenarios, enhances speech recognition performance, and ultimately improves the user experience.

[0149] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0150] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements... Figures 1 to 5 The steps of any speech recognition method are represented.

[0151] This application embodiment also provides a smart device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements... Figures 1 to 5 The steps of any speech recognition method are represented.

[0152] This application also provides a computer program product that, when run on a server, causes the server to execute the implementation as described above. Figures 1 to 5 The steps of any speech recognition method are represented.

[0153] Figure 7 This is a schematic diagram of a smart device provided in an embodiment of this application. Figure 7 As shown, the smart device 7 in this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, it implements the steps in the various speech recognition method embodiments described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of units 61 to 64 are shown.

[0154] For example, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program 72 in the smart device 7.

[0155] The intelligent device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of smart device 7 and does not constitute a limitation on smart device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, smart device 7 may also include input / output devices, network access devices, buses, etc.

[0156] The processor 70 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0157] The memory 71 can be an internal storage unit of the smart device 7, such as a hard drive or memory of the smart device 7. The memory 71 can also be an external storage device of the smart device 7, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the smart device 7. Furthermore, the memory 71 can include both internal and external storage units of the smart device 7. The memory 71 is used to store the computer program and other programs and data required by the smart device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0158] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0161] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0162] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A speech recognition method, characterized in that, include: The speech to be recognized is input into a pre-trained speech recognition model to obtain the feature probability spectrum of the speech to be recognized; Decode and search the feature probability spectrum to obtain a first search result, which includes a first search path and its corresponding path score. The feature probability spectrum is decoded and searched based on beam search and scene word graph model to obtain a second search result. The scene word graph model is trained by combining a statistical probability-based language model with a weighted finite state machine. The scene word graph model is used to provide scene grammar rules, limit the scene, and narrow the decoding search range. The second search result includes the second search path and its corresponding path score. The second search result includes a first sub-search result and a second sub-search result. The scene word graph model includes a first word graph model and a second word graph model. The first word graph model is a language model trained on short word texts to provide short word grammar rules and limit the search for speech recognition to short words. The second word graph model is a language model trained on domain-specific texts to provide domain-specific grammar rules and limit the search to a specific scene. The first sub-search result is obtained by decoding the feature probability spectrum based on the first word graph model, and the second sub-search result is obtained by decoding the feature probability spectrum based on the second word graph model. Based on the fusion result of the first search result and the second search result, the recognition result of the speech to be recognized is determined and output. The fusion result is obtained by determining the search path corresponding to the higher path score between the first search result and the second search result as the target search path. The step of determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result includes: obtaining the audio duration of the speech to be recognized; if the audio duration is greater than a preset duration threshold, determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second sub-search result; if the audio duration is less than or equal to the preset duration threshold, determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result, and the second sub-search result.

2. The speech recognition method according to claim 1, characterized in that, The step of determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result includes: Based on the fusion result of the first search result and the first sub-search result, the recognition result of the speech to be recognized is determined and output.

3. The speech recognition method according to claim 1, characterized in that, Search results include the search path and its corresponding path score; The step of determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result, and the second sub-search result includes: Determine the target search path, which is the search path corresponding to the highest path score among the first search result, the first sub-search result, and the second sub-search result; Based on the target search path, the recognition result of the speech to be recognized is determined and output.

4. The speech recognition method according to claim 1, characterized in that, Before decoding and searching the feature probability spectrum based on the first word graph model to obtain the first sub-search result, the process includes: A specified number of preset short word test sets are sequentially input into the trained speech recognition model to obtain pseudo-labels; The scene samples are filtered based on the preset audio duration and the pseudo-tags to obtain target short word samples; The language model is trained using the target short word samples to obtain the first word graph model.

5. The speech recognition method according to claim 1, characterized in that, The step of determining and outputting the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result includes: If the speech to be recognized has contextual knowledge, then the relevance between the search path in the first search result and the second search result and the contextual knowledge is calculated respectively; The first target path score is determined based on the degree of relevance between the search path in the first search result and the contextual knowledge. The second target path score is determined based on the degree of relevance between the search path in the second search result and the contextual knowledge. Determine the target search path, which is the search path corresponding to the higher path score between the first target path score and the second target path score; Based on the target search path, the recognition result of the speech to be recognized is determined and output.

6. A voice recognition device, characterized in that, The device includes: The recognition information acquisition unit is used to input the speech to be recognized into the trained speech recognition model and obtain the feature probability spectrum of the speech to be recognized. The first search unit is used to decode the feature probability spectrum and obtain a first search result, which includes a first search path and its corresponding path score. The second search unit is used to decode the feature probability spectrum based on beam search and scene word graph model to obtain a second search result. The scene word graph model is obtained by training a language model based on statistical probability combined with a weighted finite state machine. The scene word graph model is used to provide scene grammar rules, limit the scene, and narrow the decoding search range. The second search result includes the second search path and its corresponding path score. The scene word graph model includes a first word graph model and a second word graph model. The first word graph model is a language model trained on short word texts to provide short word grammar rules and limit short word searches for speech recognition. The second word graph model is a language model trained on domain-specific texts to provide domain-specific grammar rules and limit searches to specific scenarios. The second search unit includes: a first sub-search module for decoding and searching the feature probability spectrum based on the first word graph model to obtain a first sub-search result; and a second sub-search module for decoding and searching the feature probability spectrum based on the second word graph model to obtain a second sub-search result. The recognition result determination output unit is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result and the second search result, wherein the fusion result is obtained by determining the search path corresponding to the higher path score between the first search result and the second search result as the target search path; the recognition result determination output unit includes: An audio duration acquisition module is used to acquire the audio duration of the speech to be recognized; The second recognition result output module is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result and the second sub-search result if the audio duration is greater than a preset duration threshold. The third recognition result output module is used to determine and output the recognition result of the speech to be recognized based on the fusion result of the first search result, the first sub-search result and the second sub-search result if the audio duration is less than or equal to the preset duration threshold.

7. A smart device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the speech recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech recognition method as described in any one of claims 1 to 5.

Images