Speech recognition method and device, intelligent equipment and storage medium

Abstract

The invention is suitable for the technical field of intelligent equipment, and provides a speech recognition method and device, intelligent equipment and a storage medium, and the method comprises the steps: inputting a to-be-recognized speech into a trained speech recognition model, and obtaining a feature probability spectrum of the to-be-recognized speech; performing decoding search on the characteristic probability spectrum to obtain a first search result; performing decoding search on the feature probability spectrum based on a scene word graph model to obtain a second search result, the scene word graph model being a language model used for providing scene grammar rules; and determining and outputting a recognition result of the to-be-recognized voice according to a fusion result of the first search result and the second search result. According to the method, the interference of a non-scene large-probability path on a search result can be effectively eliminated, the accuracy of voice recognition in a real-time voice recognition scene is improved, the performance of voice recognition is improved, and then the user experience is enhanced.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent devices, and in particular to a speech recognition method, apparatus, intelligent device and storage medium. Background Art

[0002] With the continuous deepening of deep learning research, speech recognition, as an important branch of artificial intelligence development, has made great progress in recent years. The introduction of end-to-end methods has made a qualitative leap in speech recognition performance and user experience. Unlike laboratory tests, speech recognition services will face more diverse and complex noise environments and equipment interference in actual applications. At the same time, its audience will be wider, covering people with different accents and pronunciation habits and chat scenarios in different fields, which makes speech recognition performance significantly lower than the test results in the laboratory.

[0003] Real-time speech recognition application scenarios require the service to immediately feedback the recognition results and respond. How to improve the accuracy of speech recognition in real-time speech recognition scenarios, improve the performance of speech recognition, and enhance the user experience is a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a speech recognition method, apparatus, intelligent device and storage medium, which can improve the accuracy of speech recognition in real-time speech recognition scenarios, enhance the performance of speech recognition, and enhance the user experience.

[0005] In a first aspect, an embodiment of the present application provides a speech recognition method, comprising:

[0006] Inputting the speech to be recognized into the trained speech recognition model to obtain the characteristic probability spectrum of the speech to be recognized;

[0007] Decoding and searching the characteristic probability spectrum to obtain a first search result;

[0008] Decoding and searching the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules;

[0009] According to the fusion result of the first search result and the second search result, the recognition result of the speech to be recognized is determined and output.

[0010] In a possible implementation manner of the first aspect, the scene word graph model includes a first word graph model, and the decoding search on the feature probability spectrum based on the scene word graph model to obtain a second search result includes:

[0011] Performing a decoding search on the feature probability spectrum based on the first word graph model to obtain a first sub-search result, wherein the first word graph model is a language model trained based on short word text and used to provide short word grammatical rules;

[0012] The determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result includes:

[0013] According to a fusion result of the first search result and the first sub-search result, a recognition result of the speech to be recognized is determined and output.

[0014] In a possible implementation manner of the first aspect, the scene word graph model further includes a second word graph model, and the decoding search on the feature probability spectrum based on the scene word graph model to obtain the second search result further includes:

[0015] Decoding and searching the feature probability spectrum based on the second word graph model to obtain a second sub-search result, wherein the second word graph model is a language model trained according to a specific domain text and used to provide grammatical rules for a specific domain;

[0016] The determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result includes:

[0017] According to a fusion result of the first search result, the first sub-search result and the second sub-search result, a recognition result of the speech to be recognized is determined and output.

[0018] In a possible implementation manner of the first aspect, the search result includes a search path and a corresponding path score; and determining and outputting a recognition result of the to-be-recognized speech according to a fusion result of the first search result, the first sub-search result, and the second sub-search result includes:

[0019] Determine a target search path, where the target search path is a search path corresponding to the highest path score among the first search result, the first sub-search result, and the second sub-search result;

[0020] According to the target search path, a recognition result of the speech to be recognized is determined and output.

[0021] In a possible implementation manner of the first aspect, determining and outputting a recognition result of the to-be-recognized speech according to a fusion result of the first search result, the first sub-search result, and the second sub-search result includes:

[0022] Obtaining the audio duration of the speech to be recognized;

[0023] If the duration of the audio is greater than a preset duration threshold, determining and outputting a recognition result of the to-be-recognized speech according to a fusion result of the first search result and the second sub-search result;

[0024] If the audio duration is less than or equal to the preset duration threshold, a recognition result of the speech to be recognized is determined and output according to a fusion result of the first search result, the first sub-search result, and the second sub-search result.

[0025] In a possible implementation manner of the first aspect, before decoding and searching the feature probability spectrum based on the first word graph model to obtain a first sub-search result, the method includes:

[0026] Inputting a specified number of preset short word test sets into the trained speech recognition model in sequence to obtain pseudo labels;

[0027] The scene samples are screened according to the preset audio duration and the pseudo-labels to obtain target short word samples;

[0028] The language model is trained using the target short word samples to obtain the first word graph model.

[0029] In a possible implementation manner of the first aspect, determining and outputting a recognition result of the speech to be recognized according to a fusion result of the first search result and the second search result includes:

[0030] If there is contextual knowledge for the speech to be recognized, respectively calculating the relevance between the search paths in the first search result and the second search result and the contextual knowledge;

[0031] Determining a first target path score according to the relevance between the search path in the first search result and the context knowledge;

[0032] Determining a second target path score according to the relevance between the search path in the second search result and the context knowledge;

[0033] Determine a target search path, where the target search path is a search path corresponding to a higher path score between the first target path score and the second target path score;

[0034] According to the target search path, a recognition result of the speech to be recognized is determined and output.

[0035] In a second aspect, an embodiment of the present application provides a speech recognition device, the device comprising:

[0036] A recognition information acquisition unit, used to input the speech to be recognized into the trained speech recognition model to obtain a feature probability spectrum of the speech to be recognized;

[0037] A first search unit, configured to decode and search the characteristic probability spectrum to obtain a first search result;

[0038] A second search unit is configured to perform a decoding search on the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules;

[0039] The recognition result determination output unit is used to determine and output the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result.

[0040] In a third aspect, an embodiment of the present application provides an intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the speech recognition method as described in the first aspect above when executing the computer program.

[0041] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the speech recognition method as described in the first aspect above is implemented.

[0042] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a smart device, the smart device executes the speech recognition method as described in the first aspect above.

[0043] In an embodiment of the present application, the speech to be recognized is input into a trained speech recognition model to obtain a characteristic probability spectrum of the speech to be recognized, and then the characteristic probability spectrum is decoded and searched to obtain a first search result, and the characteristic probability spectrum is decoded and searched based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules, and then the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result and the second search result. The present application defines the scene through the word graph model, and defines the legal scope for speech recognition, which can effectively exclude the interference of non-scene high-probability paths on search results, thereby improving the accuracy of speech recognition in real-time speech recognition scenarios, improving the performance of speech recognition, and thus enhancing user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] Figure 1 is a flow chart of an implementation of a speech recognition method provided in an embodiment of the present application;

[0046] Figure 2 is a specific implementation flow chart of step S104 of the speech recognition method provided in an embodiment of the present application;

[0047] Figure 3 This is a specific implementation flow chart of the speech recognition method provided in the embodiment of the present application for obtaining the first word graph model;

[0048] Figure 4 This is a specific implementation flow chart of determining and outputting speech recognition results in the speech recognition method provided in the embodiment of the present application;

[0049] Figure 5 is another specific implementation flow chart of determining and outputting a speech recognition result in the speech recognition method provided in an embodiment of the present application;

[0050] Figure 6 is a structural block diagram of a speech recognition device provided in an embodiment of the present application;

[0051] Figure 7 It is a schematic diagram of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0053] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0054] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0055] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0056] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0058] Traditional speech recognition decoding solutions generally process data in units of sentences and train deep learning models based on neural networks. The number of layers and context structure of the model determine that the data frame has a fixed receptive field. Test data with a similar duration to the training sample is most ideal for matching the model. For long speech containing many words, the error of one or two words may not have a big impact on semantic analysis. For short speech containing only a few words, a recognition error may directly lead to incorrect semantic analysis. In real-time speech recognition scenarios in complex noisy environments such as intelligent customer service, there are often short audio durations, and the speech recognition model is prone to recognition errors. Coupled with the interference of complex noise and human factors, the recognition performance may be further reduced.

[0059] In order to solve the above problems, the embodiments of the present application provide a speech recognition method, apparatus, intelligent device and storage medium, which are described below for details.

[0060] It should be understood that the speech recognition method provided in the embodiments of the present application is applicable to smart devices that need to perform speech recognition on various types of data, which may specifically include smart devices such as mobile phones, tablet computers, wearable devices, laptops, and desktop computers. The embodiments of the present application do not impose any restrictions on the specific types of smart devices.

[0061] The speech recognition method provided in this application is exemplarily described below in conjunction with specific embodiments.

[0062] Figure 1 The implementation process of the speech recognition method provided in the embodiment of the present application is shown, and the method flow includes steps S101 to S104. The specific implementation principle of each step is as follows:

[0063] Step S101: inputting the speech to be recognized into a trained speech recognition model to obtain a feature probability spectrum of the speech to be recognized.

[0064] The above-mentioned speech recognition model is used to convert audio into sentences. In this embodiment, the speech recognition model is trained based on the annotated audio data, and the speech recognition model can be a general model, which learns the correspondence between audio and text through a large amount of audio and corresponding labels, and is applicable to different scenarios.

[0065] In an embodiment of the present application, the above-mentioned speech recognition model can be a phoneme-level model based on a hybrid framework, or a model based on a word-level end-to-end framework.

[0066] The speech recognition model of the hybrid framework is composed of an acoustic model and a language model. The speech recognition model is built at the phoneme level (for Chinese, it is the initials and finals). The acoustic model realizes audio-to-pronunciation conversion, and the language model realizes text-to-sentence conversion. The acoustic model decoding requires the assistance of the language model. The language model provides a higher weight for more reasonable grammar, so that pronunciation can be converted into more reasonable sentence text, that is, the pronunciation is converted into grammatically smooth sentence text through the guidance of the language model. The output of the speech recognition model of the hybrid framework is a phoneme sequence.

[0067] The speech recognition model of the end-to-end framework includes a neural network model, which is built at the word level and can directly convert audio into sentence text. The neural network model learns both audio information and grammatical information, and can complete the overall recognition conversion through one model. The speech recognition model of the end-to-end framework outputs a word sequence.

[0068] The above feature probability spectrum is a two-dimensional matrix of speech feature frames and classifications. The audio is input into the speech recognition model frame by frame, and a vector is obtained for each frame. The dimension of the vector is the number of nodes in the output layer (the number of words in the end-to-end framework and the number of phonemes in the hybrid framework). The value of the vector is the probability value of each category, and the sum is 1. Under the premise of a frame length of 10ms, 1 second of speech contains 100 frames, and its output is a two-dimensional matrix (probability spectrum) of 100*the number of categories.

[0069] Step S102: Decode and search the characteristic probability spectrum to obtain a first search result.

[0070] In this embodiment, a maximum probability search without model restriction is performed on the feature probability based on beam search, that is, a decoding search with the largest range is performed to obtain a first search result.

[0071] Exemplarily, a full-image search without syntax restriction is performed based on CTC to obtain the first search result.

[0072] Step S103: decoding and searching the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules.

[0073] In this embodiment, a maximum probability search is performed on the feature probability using a scene word graph model based on a beam search to obtain a second search result.

[0074] The scene word graph model is a language model based on statistical probability, and the scene sample is trained in combination with WFST. The scene word graph model can be a short word word graph model, a specific field word graph model, etc. The specific field can be a limited professional field. In this embodiment, the scene word graph model is used to limit the scene, narrow the decoding search range, and thus improve the efficiency of the decoding search.

[0075] Step S104: Determine and output a recognition result of the speech to be recognized based on a fusion result of the first search result and the second search result.

[0076] The search results include a search path and a corresponding path score. The first search result includes a first search path and a corresponding path score, and the second search result includes a second search path and a corresponding path score. The search path includes a complete sentence composed of characters or phonemes, and the path score is used to indicate the credibility of the sentence. The higher the path score, the higher the credibility that the sentence is the true content of the speech to be recognized.

[0077] In this embodiment, the search path corresponding to the higher path score between the first search result and the second search result is determined as the target search path, and the complete sentence corresponding to the target search path is output, thereby obtaining the recognition result of the speech to be recognized.

[0078] As a possible implementation of this application, Figure 2 A specific implementation process of step S104 of the speech recognition method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0079] A1: If there is contextual knowledge for the speech to be recognized, the relevance between the search paths in the first search result and the second search result and the contextual knowledge is calculated respectively.

[0080] The context knowledge may be the content of the previous sentence during a conversation or an outbound call, or may be order information or other information that can be obtained and is related to the speech to be recognized. In this embodiment, the context knowledge is defined according to a specific business scenario.

[0081] The first search result includes a first search path, and the second search result includes a second search path. The relevance of the first search path, the second search path and the context knowledge is calculated respectively.

[0082] A2: Determine a first target path score according to the relevance between the search path in the first search result and the context knowledge.

[0083] A3: Determine a second target path score according to the relevance between the search path in the second search result and the context knowledge.

[0084] In this embodiment, the weight corresponding to the search path is determined according to the relevance between the search path and the contextual knowledge, and the target path score is determined based on the weight and the path score corresponding to the search path. Specifically, the first weight is determined according to the relevance between the first search path and the contextual knowledge, and the first target path score is determined according to the first weight and the path score corresponding to the first search path; the second weight is determined according to the relevance between the second search path and the contextual knowledge, and the second target path score is determined according to the second weight and the path score corresponding to the second search path.

[0085] For example, the path score corresponding to the first search path y1 is w1, and the path score corresponding to the second search path y2 is w2. After the context relevance reward, the first target path score corresponding to the first search path is P(y1|x)*w1, and the second target path score is P(y2|x)*w2, where x is the context knowledge. The relevance of the context knowledge to the search path can be determined by converting the text into an embedding vector to calculate the edit distance, or it can be calculated based on the BERT model.

[0086] A4: Determine a target search path, where the target search path is a search path corresponding to a higher path score between the first target path score and the second target path score.

[0087] Exemplarily, if the first target path score is higher, the first search path is determined to be the target search path, and if the second target path score is higher, the second search path is determined to be the target search path.

[0088] A5: Determine and output the recognition result of the speech to be recognized according to the target search path. Output the complete sentence corresponding to the target search path, so as to obtain the recognition result of the speech to be recognized.

[0089] In the embodiment of the present application, by calculating the degree of association between the contextual knowledge of the speech to be recognized and the search path and weighting the path score, the search path of the correct scene is more likely to be selected.

[0090] In a possible implementation, the scene word graph model includes a first word graph model, which is a language model trained based on short word text and used to provide short word grammatical rules. The second search result includes a first sub-search result.

[0091] In this embodiment, the feature probability spectrum is decoded and searched based on the first word graph model, the search is limited to reach the end of the word graph within a preset time length (for example, within 2S), a first sub-search result is obtained, and the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result and the first sub-search result.

[0092] Specifically, the search path corresponding to the higher path score in the first search result and the first sub-search result is determined as the target search path, and the complete sentence corresponding to the target search path is output, thereby obtaining the recognition result of the speech to be recognized.

[0093] In this embodiment, the first word graph model that provides short word grammatical rules is used to limit speech recognition to short word searches, which can eliminate the interference of non-short word high-probability paths on search results, thereby making speech recognition more accurate.

[0094] As a possible implementation manner of the present application, the first word graph model is obtained based on the trained speech recognition model, and before the feature probability spectrum is decoded and searched based on the first word graph model to obtain the first sub-search result, the first word graph model is obtained. Figure 3 A specific implementation process of obtaining the first word graph model is shown, which is described in detail as follows:

[0095] B1: Input a specified number of preset short word test sets into the trained speech recognition model in sequence to obtain pseudo labels, wherein the pseudo labels are speech recognition results corresponding to the preset short word test sets.

[0096] In this embodiment, the framework of the speech recognition model is different, and the process of obtaining pseudo labels is different. For the speech recognition model of the hybrid framework, the phoneme-level acoustic model needs to be decoded in conjunction with the language model based on statistical probability to obtain the result; for the speech recognition model of the end-to-end framework, a full-image search without grammatical restrictions is performed based on CTC to obtain the result.

[0097] B2: Filter the scene samples according to the preset audio duration and the pseudo-label to obtain the target short word samples. Filter the scene samples and determine the scene samples whose audio duration does not exceed the preset audio duration and whose pseudo-label word count does not exceed the preset word count as the target short word samples.

[0098] In one possible implementation, an initial language model based on statistical probability is obtained by training the annotated audio data text, and the confusion is calculated using the initial language model. The target short word samples that meet the requirements are screened by setting a confusion threshold. The initial language model provides the probability of connection between words. For example, P(w3|w2,w1) indicates the probability that w3 is preceded by w1w2, and the probability of the sentence is the product of P.

[0099] B3: Use the target short word samples to train the language model to obtain the first word graph model.

[0100] Exemplarily, in one application scenario, a short audio of an actual scene is decoded using a trained speech recognition model to obtain a recognition result (pseudo-label), from which samples that meet the requirements in terms of number of characters, audio length, and grammatical fluency are selected as target short word samples, and the target short word samples are used to train a language model based on statistical probability or RNN, and a first word graph model is constructed based on wfst.

[0101] In an embodiment of the present application, a trained speech recognition model is used to obtain pseudo labels, and the effectiveness of training samples is improved by screening the pseudo labels, thereby improving the efficiency and accuracy of the first word graph model training.

[0102] In a possible implementation, the scene word graph model further includes a second word graph model, where the second word graph model is a language model trained based on domain-specific text and used to provide domain-specific grammar rules. The second search result includes a second sub-search result.

[0103] In this embodiment, the feature probability spectrum is decoded and searched based on the second word graph model to obtain a second sub-search result, and the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result, the first sub-search result and the second sub-search result.

[0104] Specifically, the search path corresponding to the higher path score in the first search result and the first sub-search result is determined as the target search path, and the complete sentence corresponding to the target search path is output, thereby obtaining the recognition result of the speech to be recognized.

[0105] As a possible implementation of the present application, the search results include a search path and its corresponding path score. Figure 4 The specific implementation process of determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result, the first sub-search result and the second sub-search result in the speech recognition method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0106] C1: Determine a target search path and its path score, where the target search path is the search path corresponding to the highest path score among the first search result, the second search result, and the third search result.

[0107] C2: Determine and output the recognition result of the speech to be recognized according to the target search path. That is,

[0108] The path scores in the first search result, the first sub-search result and the second sub-search result are compared, the search path corresponding to the highest path score among the three is determined as the target search path, and the complete sentence corresponding to the target search path is output, thereby obtaining the recognition result of the speech to be recognized.

[0109] In this embodiment, the second word graph model can be a word graph model for specific fields such as address words, names, order information, etc., which is used to limit the search to specific scenarios, thereby further eliminating the interference of non-scenario high-probability paths on search results and improving the accuracy of speech recognition.

[0110] As a possible implementation of this application, Figure 5The specific implementation process of determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result, the first sub-search result and the second sub-search result in the speech recognition method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0111] D1: Obtain the audio duration of the speech to be recognized.

[0112] D2: If the duration of the audio is greater than a preset duration threshold, determining and outputting a recognition result of the speech to be recognized based on a fusion result of the first search result and the second sub-search result.

[0113] D3: If the audio duration is less than or equal to the preset duration threshold, determining and outputting a recognition result of the speech to be recognized according to a fusion result of the first search result, the first sub-search result, and the second sub-search result.

[0114] In this embodiment, the search results to be fused are determined in combination with the audio duration. For speech to be recognized that exceeds a preset duration threshold, fusion is performed only based on the search results of the above-mentioned speech recognition model and the second word graph model to determine and output the recognition result of the speech to be recognized, thereby avoiding wasting search time. For speech to be recognized whose duration is within the preset duration threshold, fusion is performed based on the search results of the above-mentioned speech recognition model, the first word graph model and the second word graph model to determine and output the recognition result of the speech to be recognized.

[0115] In an embodiment of the present application, the speech to be recognized is input into a trained speech recognition model to obtain a characteristic probability spectrum of the speech to be recognized, and then the characteristic probability spectrum is decoded and searched to obtain a first search result, and the characteristic probability spectrum is decoded and searched based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules, and then the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result and the second search result. The present application defines the scene through the word graph model, and defines the legal scope for speech recognition, which can effectively exclude the interference of non-scene high-probability paths on search results, improve the accuracy of speech recognition in real-time speech recognition scenarios, and improve the performance of speech recognition, thereby enhancing user experience.

[0116] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0117] Corresponding to the speech recognition method described in the above embodiment, Figure 6A structural block diagram of a speech recognition device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0118] Reference Figure 6 The speech recognition device comprises: a recognition information acquisition unit 61, a first search unit 62, a second search unit 63, and a recognition result determination output unit 64, wherein:

[0119] The recognition information acquisition unit 61 is used to input the speech to be recognized into the trained speech recognition model to obtain the characteristic probability spectrum of the speech to be recognized;

[0120] A first search unit 62 is used to decode and search the characteristic probability spectrum to obtain a first search result;

[0121] A second search unit 63 is used to perform a decoding search on the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules;

[0122] The recognition result determination output unit 64 is used to determine and output the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result.

[0123] As a possible implementation manner of the present application, the scene word graph model includes a first word graph model, and the second search unit 63 includes:

[0124] A first sub-search module, configured to perform a decoding search on the feature probability spectrum based on the first word graph model to obtain a first sub-search result, wherein the first word graph model is a language model trained based on short word text and used to provide short word grammatical rules;

[0125] The above-mentioned recognition result determination output unit 64 is specifically used for:

[0126] According to a fusion result of the first search result and the first sub-search result, a recognition result of the speech to be recognized is determined and output.

[0127] As a possible implementation manner of the present application, the scene word graph model further includes a second word graph model, and the second search unit 63 further includes:

[0128] A second sub-search module is used to decode and search the feature probability spectrum based on a second word graph model to obtain a third search result, wherein the second word graph model is a language model trained according to a specific domain text and used to provide grammatical rules for a specific domain;

[0129] The above-mentioned recognition result determination output unit 64 is specifically used for:

[0130] According to a fusion result of the first search result, the first sub-search result and the second sub-search result, a recognition result of the speech to be recognized is determined and output.

[0131] As a possible implementation of the present application, the recognition result determination output unit 64 includes:

[0132] A first target path determination module, configured to determine a target search path, wherein the target search path is a search path corresponding to the highest path score among the first search result, the first sub-search result, and the second sub-search result;

[0133] The first recognition result output module is used to determine and output the recognition result of the speech to be recognized according to the target search path.

[0134] As a possible implementation of the present application, the recognition result determination output unit 64 includes:

[0135] An audio duration acquisition module, used to acquire the audio duration of the speech to be recognized;

[0136] A second recognition result output module, configured to determine and output a recognition result of the speech to be recognized according to a fusion result of the first search result and the second sub-search result if the duration of the audio is greater than a preset duration threshold;

[0137] The third recognition result output module is used to determine and output the recognition result of the speech to be recognized according to the fusion result of the first search result, the first sub-search result and the second sub-search result if the audio duration is less than or equal to the preset duration threshold.

[0138] As a possible implementation of the present application, the above-mentioned language recognition device further includes:

[0139] A pseudo-label acquisition unit, used for sequentially inputting a specified number of preset short word test sets into the trained speech recognition model to acquire pseudo-labels;

[0140] A target short word sample acquisition unit, used to screen the scene samples according to the preset audio duration and the pseudo-label to acquire the target short word sample;

[0141] The first word graph model training unit is used to train the language model using the target short word sample to obtain the first word graph model.

[0142] As a possible implementation of the present application, the recognition result determination output unit 64 further includes:

[0143] A relevance calculation module, configured to calculate the relevance between the search paths in the first search result and the second search result and the contextual knowledge if the speech to be recognized has contextual knowledge;

[0144] A first path score determination module, configured to determine a first target path score according to the relevance between the search path in the first search result and the context knowledge;

[0145] A second path score determination module, configured to determine a second target path score according to the relevance between the search path in the second search result and the context knowledge;

[0146] A second target path determination module, configured to determine a target search path, wherein the target search path is a search path corresponding to a higher path score between the first target path score and the second target path score;

[0147] The fourth recognition result output module is used to determine and output the recognition result of the speech to be recognized according to the target search path.

[0148] In an embodiment of the present application, the speech to be recognized is input into a trained speech recognition model to obtain a characteristic probability spectrum of the speech to be recognized, and then the characteristic probability spectrum is decoded and searched to obtain a first search result, and the characteristic probability spectrum is decoded and searched based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules, and then the recognition result of the speech to be recognized is determined and output based on the fusion result of the first search result and the second search result. The present application defines the scene through the word graph model, and defines the legal scope for speech recognition, which can effectively exclude the interference of non-scene high-probability paths on search results, improve the accuracy of speech recognition in real-time speech recognition scenarios, and improve the performance of speech recognition, thereby enhancing user experience.

[0149] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0150] The present application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, Figures 1 to 5 The steps of any speech recognition method represented.

[0151] The embodiment of the present application also provides an intelligent device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, Figures 1 to 5 The steps of any speech recognition method represented.

[0152] The present application also provides a computer program product, which, when executed on a server, enables the server to execute the following Figures 1 to 5 The steps of any speech recognition method represented.

[0153] Figure 7 Schematic diagram of a smart device provided by an embodiment of the present application. Figure 7 As shown, the smart device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned various speech recognition method embodiments are implemented, for example Figure 1 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of the units 61 to 64 are shown.

[0154] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the smart device 7.

[0155] The smart device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the smart device 7 and does not constitute a limitation of the smart device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the smart device 7 may also include input and output devices, network access devices, buses, etc.

[0156] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0157] The memory 71 may be an internal storage unit of the smart device 7, such as a hard disk or memory of the smart device 7. The memory 71 may also be an external storage device of the smart device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smart device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the smart device 7. The memory 71 is used to store the computer program and other programs and data required by the smart device. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0158] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0159] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0161] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0162] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A speech recognition method, characterized in that: include: Inputting the speech to be recognized into the trained speech recognition model to obtain the characteristic probability spectrum of the speech to be recognized; Decoding and searching the characteristic probability spectrum to obtain a first search result; Decoding and searching the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules; According to the fusion result of the first search result and the second search result, the recognition result of the speech to be recognized is determined and output.

2. The speech recognition method according to claim 1, characterized in that: The scene word graph model includes a first word graph model, and the decoding search of the feature probability spectrum based on the scene word graph model to obtain a second search result includes: Performing a decoding search on the feature probability spectrum based on the first word graph model to obtain a first sub-search result, wherein the first word graph model is a language model trained based on short word text and used to provide short word grammatical rules; The determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result includes: According to a fusion result of the first search result and the first sub-search result, a recognition result of the speech to be recognized is determined and output.

3. The speech recognition method according to claim 2, characterized in that: The scene word graph model further includes a second word graph model, and the decoding search of the feature probability spectrum based on the scene word graph model to obtain a second search result further includes: Decoding and searching the feature probability spectrum based on the second word graph model to obtain a second sub-search result, wherein the second word graph model is a language model trained according to a specific domain text and used to provide grammatical rules for a specific domain; The determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result includes: According to a fusion result of the first search result, the first sub-search result and the second sub-search result, a recognition result of the speech to be recognized is determined and output.

4. The speech recognition method according to claim 3, characterized in that: The search results include search paths and their corresponding path scores; The step of determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result, the first sub-search result and the second sub-search result includes: Determine a target search path, where the target search path is a search path corresponding to the highest path score among the first search result, the first sub-search result, and the second sub-search result; According to the target search path, a recognition result of the speech to be recognized is determined and output.

5. The speech recognition method according to claim 3, characterized in that: The step of determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result, the first sub-search result and the second sub-search result includes: Obtaining the audio duration of the speech to be recognized; If the duration of the audio is greater than a preset duration threshold, determining and outputting a recognition result of the to-be-recognized speech according to a fusion result of the first search result and the second sub-search result; If the audio duration is less than or equal to the preset duration threshold, a recognition result of the speech to be recognized is determined and output according to a fusion result of the first search result, the first sub-search result, and the second sub-search result.

6. The speech recognition method according to claim 2, characterized in that: Before decoding and searching the feature probability spectrum based on the first word graph model to obtain a first sub-search result, the method includes: Inputting a specified number of preset short word test sets into the trained speech recognition model in sequence to obtain pseudo labels; The scene samples are screened according to the preset audio duration and the pseudo-labels to obtain target short word samples; The language model is trained using the target short word samples to obtain the first word graph model.

7. The speech recognition method according to claim 1, characterized in that: The determining and outputting the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result includes: If there is contextual knowledge for the speech to be recognized, respectively calculating the relevance between the search paths in the first search result and the second search result and the contextual knowledge; Determining a first target path score according to the relevance between the search path in the first search result and the context knowledge; Determining a second target path score according to the relevance between the search path in the second search result and the context knowledge; Determine a target search path, where the target search path is a search path corresponding to a higher path score between the first target path score and the second target path score; According to the target search path, a recognition result of the speech to be recognized is determined and output.

8. A speech recognition device, characterized in that: The device comprises: A recognition information acquisition unit, used to input the speech to be recognized into the trained speech recognition model to obtain a feature probability spectrum of the speech to be recognized; A first search unit, configured to decode and search the characteristic probability spectrum to obtain a first search result; A second search unit is configured to perform a decoding search on the feature probability spectrum based on a scene word graph model to obtain a second search result, wherein the scene word graph model is a language model for providing scene grammar rules; The recognition result determination output unit is used to determine and output the recognition result of the speech to be recognized according to the fusion result of the first search result and the second search result.

9. An intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the speech recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the speech recognition method according to any one of claims 1 to 7 is implemented.

Images