Speech recognition method, device, terminal device and computer readable storage medium

Abstract

The application is suitable for the terminal technical field, and particularly relates to a voice recognition method and device, a terminal device and a computer readable storage medium. In the method, after the terminal device obtains the voice to be recognized, the voice to be recognized can be recognized by using an end-to-end voice recognition model to obtain a recognition result. For each decoding of the voice recognition model, the terminal device can determine a first probability corresponding to a candidate decoding result according to an N-gram language model, so that the voice recognition model can determine the candidate decoding result obtained by the next decoding according to the first probability corresponding to the candidate decoding result. That is, in the application, when the voice recognition is performed by using the end-to-end voice recognition model, the first probability corresponding to the candidate decoding result can be determined according to the N-gram language model, so that the decoding result conforms to the grammatical structure, the decoding error caused by the inaccurate pronunciation such as accent is reduced, the accuracy of the voice recognition is improved, and the user experience is improved.

Description

Technical Field

[0001] This application belongs to the field of terminal technology, and in particular relates to a voice recognition method, device, terminal equipment and computer-readable storage medium. Background Technology

[0002] End-to-end speech recognition technology uses a neural network model to directly convert speech into text, such as attention-based neural network models. These models typically consist of an encoder and a decoder based on the attention mechanism. The encoder converts the speech feature sequence into a sequence of hidden state vectors. The decoder uses autoregression and the attention mechanism to focus attention on a subset of these hidden state vectors, outputting the decoding result step-by-step. During decoding, beam search is generally used to obtain the top few words / phrases / sentences with the highest probabilities as candidate decoding results. In other words, the neural network model searches the decoding space based on probability, which introduces uncertainty. In cases of inaccurate pronunciation, such as accents, it may produce grammatically incorrect decoding results, leading to lower speech recognition accuracy. Summary of the Invention

[0003] This application provides a speech recognition method, apparatus, terminal device, and computer-readable storage medium, which can solve the problem of low speech recognition accuracy.

[0004] In a first aspect, embodiments of this application provide a speech recognition method applied to a terminal device, the method including:

[0005] The terminal device acquires the voice to be recognized;

[0006] The terminal device uses a speech recognition model to perform speech recognition on the speech to be recognized, and obtains the recognition result.

[0007] The speech recognition model is an end-to-end speech recognition model. When using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding of the speech recognition model, the terminal device determines the first probability corresponding to the candidate decoding result according to the N-gram language model. The first probability corresponding to the candidate decoding result is used to determine the candidate decoding result obtained by the speech recognition model in the next decoding.

[0008] In the aforementioned speech recognition method, after the terminal device acquires the speech to be recognized, it can use an end-to-end speech recognition model to perform speech recognition on the speech to be recognized and obtain the recognition result. Specifically, when using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding step of the speech recognition model, the terminal device can determine the first probability corresponding to the candidate decoding result based on the N-gram language model. This allows the speech recognition model to determine the candidate decoding result for the next decoding step based on the first probability corresponding to the candidate decoding result. That is, in this embodiment, when performing speech recognition using the end-to-end speech recognition model, the first probability corresponding to the candidate decoding result can be determined based on the N-gram language model, ensuring that the decoding result conforms to the grammatical structure, reducing decoding errors caused by inaccurate pronunciation such as accents, improving the accuracy of speech recognition, and enhancing the user experience.

[0009] In one possible implementation, the terminal device determines the first probability corresponding to the candidate decoding result based on the N-gram language model, which may include:

[0010] The terminal device obtains a second probability corresponding to the candidate decoding result, and the second probability corresponding to the candidate decoding result is determined by the speech recognition model;

[0011] The terminal device determines the first probability corresponding to the candidate decoding result based on the N-gram language model and the second probability corresponding to the candidate decoding result.

[0012] For example, the terminal device determines the first probability corresponding to the candidate decoding result based on the N-gram language model and the second probability corresponding to the candidate decoding result, which may include:

[0013] The terminal device determines the first weight corresponding to the candidate decoding result based on the N-gram language model;

[0014] The terminal device determines the first probability corresponding to the candidate decoding result based on the second probability and the first weight corresponding to the candidate decoding result.

[0015] Optionally, the terminal device determines the first weight corresponding to the candidate decoding result based on the N-gram language model, which may include:

[0016] When the candidate decoding result exists in the N-gram language model, the terminal device determines the second weight corresponding to the candidate decoding result in the N-gram language model, and determines the first weight corresponding to the candidate decoding result based on the second weight;

[0017] When the candidate decoding result does not exist in the N-gram language model, the terminal device determines the first weight corresponding to the candidate decoding result as a preset value.

[0018] For example, the terminal device determines the first probability corresponding to the candidate decoding result based on the second probability and the first weight corresponding to the candidate decoding result, which may include:

[0019] The terminal device determines the scaling value corresponding to the first weight based on the candidate decoding result;

[0020] The terminal device determines the first probability corresponding to the candidate decoding result based on the first weight, the scaling value corresponding to the first weight, and the second probability corresponding to the candidate decoding result.

[0021] Optionally, the terminal device determines the scaling value corresponding to the first weight based on the candidate decoding result, which may include:

[0022] The terminal device determines the length of the candidate decoding result and, based on the length of the candidate decoding result, determines the scaling value corresponding to the first weight.

[0023] In one possible implementation, the method may further include:

[0024] The terminal device acquires the target text, and the target text and the text content corresponding to the speech to be recognized belong to the same domain.

[0025] The terminal device trains the N-gram language model using the target text to obtain the trained N-gram language model.

[0026] Secondly, embodiments of this application provide a voice recognition device applied to a terminal device, the device including:

[0027] The speech acquisition module is used to acquire the speech to be recognized;

[0028] The speech recognition module is used to perform speech recognition on the speech to be recognized using a speech recognition model to obtain the recognition result;

[0029] The speech recognition model is an end-to-end speech recognition model. When using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding of the speech recognition model, the speech recognition module is further used to determine the first probability corresponding to the candidate decoding result based on the N-gram language model. The first probability corresponding to the candidate decoding result is used to determine the candidate decoding result obtained by the speech recognition model in the next decoding.

[0030] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the speech recognition method as described in any one of the first aspects above.

[0031] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the speech recognition method as described in any one of the first aspects above.

[0032] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the speech recognition method described in any one of the first aspects.

[0033] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the structure of a neural network model based on Attention;

[0036] Figure 2 This application provides a flowchart illustrating a speech recognition method.

[0037] Figure 3 This is a schematic diagram of a weighted finite state machine provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the structure of a speech recognition device provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0040] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0041] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0042] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0043] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0044] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0045] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0046] End-to-end speech recognition technology typically uses a neural network model to directly convert speech features into text, reducing development complexity and leading to its increasing adoption. For example, attention-based neural network models.

[0047] Please see Figure 1 , Figure 1 A schematic diagram of an Attention-based neural network model is shown.

[0048] like Figure 1 As shown, a neural network model based on attention generally includes an encoder 101 and a decoder 102 based on the attention mechanism.

[0049] The encoder 101 converts the speech feature sequence into a hidden state vector sequence. The decoder 102 uses an autoregressive approach and an attention mechanism to focus attention on a portion of the hidden state vector sequence, outputting the decoding result in a single step. For speech recognition, the input to the encoder 101 can be Mel-spectrum features extracted from the audio. In the decoder 102, the first step is to input special symbols. <sos>That is, the start of sentence. Then, the first character is decoded and output. Next, the first character is input, the second character is decoded, and so on until the output is complete. <eos>"end of sentence" indicates that decoding is completed. During the decoding process of the decoder 102, generally BeamSearch is used to obtain the top several words / phrases / sentences with the highest probabilities as candidate decoding results. There may be multiple words / phrases / sentences with the same pronunciation among the candidate decoding results. The decoder 102 depends on the acoustic features and language context features generated by the encoder 101. When the pronunciation is inaccurate due to an accent or the like, it may be biased towards the pronunciation and result in the selection of incorrect words / phrases / sentences. For example, decoding "初次见面请多多关照” as "抽次见面请多多关照”. That is, the neural network model searches in the decoding space based on probability and has uncertainty. For situations where the pronunciation is inaccurate due to an accent or the like, it may give decoding results that do not conform to the grammar structure, resulting in a low accuracy of speech recognition.

[0050] To solve the above problems, the embodiments of the present application provide a speech recognition method, apparatus, terminal device, and computer-readable storage medium. In this method, after the terminal device obtains the speech to be recognized, it can use an end-to-end speech recognition model to perform speech recognition on the speech to be recognized and obtain a recognition result. Among them, when using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding of the speech recognition model, the terminal device can determine the first probability corresponding to the candidate decoding result according to the N-gram language model, so that the speech recognition model can determine the candidate decoding result obtained by the next decoding according to the first probability corresponding to the candidate decoding result. That is, in the embodiments of the present application, when performing speech recognition through an end-to-end speech recognition model, the first probability corresponding to the candidate decoding result can be determined according to the N-gram language model, so that the decoding result conforms to the grammar structure, reduces decoding errors caused by inaccurate pronunciations such as accents, improves the accuracy of speech recognition, enhances the user experience, and has strong usability and practicality.

[0051] The speech recognition method provided by the embodiments of the present application can be applied to terminal devices such as robots, mobile phones, tablet computers, smart TVs, wearable devices, vehicle-mounted devices, smart speakers, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0052] Please refer to Figure 2 , Figure 2 which shows a schematic flowchart of the speech recognition method provided by the embodiments of the present application. This method can be applied to a terminal device. As Figure 2 As shown, the method may include:

[0053] S201. The terminal device acquires the voice to be recognized;

[0054] S202. The terminal device uses a speech recognition model to perform speech recognition on the speech to be recognized and obtains the recognition result.

[0055] Among them, the speech recognition model is an end-to-end speech recognition model. When using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding of the speech recognition model, the terminal device determines the first probability corresponding to the candidate decoding result according to the N-gram language model. The first probability corresponding to the candidate decoding result is used to determine the candidate decoding result obtained by the speech recognition model in the next decoding.

[0056] Optionally, the speech recognition model may include an encoder and an attention-based decoder. The encoder can be used to convert the speech feature sequence into a sequence of hidden state vectors. The decoder can use autoregression and the attention mechanism to focus attention on a portion of the hidden state vector sequence, outputting candidate decoding results step by step. In each decoding step (also known as each decoding iteration), the decoder can use BeamSearch to select the top B characters, words, or sentences with the highest probabilities as candidate decoding results.

[0057] It should be understood that the value of B can be determined according to the actual scenario, and the embodiments of this application do not impose any restrictions on it. For example, the value of B can be any value such as 1, 2, 3, or 4. For example, when the value of B is 3, in each decoding step, the decoder can select the top 3 with the highest probability as candidate decoding results.

[0058] It should be noted that the speech recognition model can be an end-to-end speech recognition model trained using a large amount of audio and its corresponding annotations. This application does not impose any restrictions on the specific training process of the speech recognition model or the entity performing the training; these can be determined according to the actual situation.

[0059] Optionally, the N-gram language model can be trained using a large amount of target text. It should be understood that the target text can belong to the same domain as the text content corresponding to the speech to be recognized. Training the N-gram language model with target text from the same domain ensures that the N-gram language model conforms to the grammatical requirements of that domain. Therefore, when correcting the candidate decoding results of the speech recognition model using the N-gram language model, the final decoding result obtained by the speech recognition model can conform to the grammatical requirements of that domain, ensuring the accuracy of speech recognition and improving the user experience.

[0060] It should be understood that the execution entity for N-gram language model training can be a terminal device or other devices, and the embodiments of this application do not impose any restrictions on this. Hereinafter, an example will be given with the execution entity for N-gram language model training being a terminal device. That is, the terminal device can obtain a target text and use the target text to train the N-gram language model to obtain a trained N-gram language model.

[0061] Among them, the value of N in the N-gram language model can be determined according to the actual situation, and the embodiments of this application do not impose any restrictions on this. For example, N can be 1, that is, the N-gram language model can be a unigram language model. For example, N can be 2, that is, the N-gram language model can be a bigram language model. For example, N can be 3, that is, the N-gram language model can be a trigram language model, and so on.

[0062] Exemplarily, the N-gram language model can be in the form of a weighted finite state machine to improve the search efficiency during the decoding process. It should be understood that the embodiments of this application do not impose any restrictions on the specific process of converting the N-gram language model into a weighted finite state machine, which can be determined according to the actual situation.

[0063] Please refer to Figure 3 , Figure 3 which shows a schematic diagram of a weighted finite state machine provided by the embodiments of this application.

[0064] As Figure 3 shown, the weighted finite state machine can be represented by a directed graph, and the state transition arcs in the directed graph include output results and the weights corresponding to the output results.

[0065] For example, Figure 3 in "[初 / a1]", it can represent that the output result is "初" and the weight corresponding to "初" is "a1", "[抽 / a2]" can represent that the output result is "抽" and the weight corresponding to "抽" is "a2", and so on. Therefore, according to Figure 3 the weighted finite state machine shown, the weighted finite state machine can include "初次", "抽水", "楚辞", "初次见", "抽水机", "抽水泵", and "楚辞赋", etc. The weight corresponding to "初次" can be "a1*b1", the weight corresponding to "抽水" can be "a2*b2", the weight corresponding to "楚辞" can be "a3*b3", the weight corresponding to "初次见" can be "a1*b1*c1", the weight corresponding to "抽水机" can be "a2*b2*c2", the weight corresponding to "抽水泵" can be "a2*b2*c3", the weight corresponding to "楚辞赋" can be "a3*b3*c4", and so on.

[0066] Next, the process by which the terminal device determines the first probability corresponding to the candidate decoding result according to the N-gram language model in each decoding will be described in detail.

[0067] It should be understood that in each decoding iteration, the decoder acquires the top B candidate decoding results with the highest probability (hereinafter referred to as the second probability). That is, in each decoding iteration, when the decoder acquires B candidate decoding results, it can simultaneously acquire the second probability corresponding to each candidate decoding result. Therefore, in each decoding iteration, the terminal device can determine the first probability corresponding to each candidate decoding result based on the second probability and the N-gram language model. This allows it to correct the candidate decoding results obtained by the decoder according to the N-gram language model that conforms to the grammatical structure of the corresponding domain, correcting decoding errors caused by inaccurate pronunciation such as accents, and improving speech recognition accuracy.

[0068] In one possible implementation, for each candidate decoding result in each decoding step, the terminal device can determine the first weight corresponding to the candidate decoding result based on the N-gram language model. Subsequently, the terminal device can determine the first probability corresponding to the candidate decoding result based on the second probability corresponding to the candidate decoding result and the first weight.

[0069] Optionally, for each candidate decoding result in each decoding process, the terminal device can search for that candidate decoding result in the N-gram language model (i.e., a weighted finite state machine). When the candidate decoding result is found in the weighted finite state machine (i.e., the candidate decoding result exists in the N-gram language model), the terminal device can determine the weight (hereinafter referred to as the second weight) corresponding to the candidate decoding result in the weighted finite state machine, and determine the first weight corresponding to the candidate decoding result based on the second weight. For example, the second weight corresponding to the candidate decoding result can be determined as the first weight corresponding to the candidate decoding result.

[0070] When the candidate decoding result is not found in the weighted finite state machine (i.e., the candidate decoding result does not exist in the N-gram language model), it indicates that the candidate decoding result does not conform to the grammatical structure of the corresponding domain. In this case, the terminal device can set the first weight corresponding to the candidate decoding result to a preset value. The preset value can be determined according to the actual situation. For example, the preset value can be set to 0 according to the actual situation.

[0071] Subsequently, the terminal device can determine the first probability corresponding to the candidate decoding result based on the second probability and the first weight corresponding to the candidate decoding result.

[0072] In one example, the terminal device can directly add the second probability corresponding to the candidate decoding result and the first weight to obtain the first probability corresponding to the candidate decoding result. That is, for the candidate decoding results existing in the weighted finite state machine, that is, the candidate decoding results that conform to the syntactic structure of the corresponding field, the terminal device can increase the probability corresponding to the candidate decoding result according to the first weight corresponding to the candidate decoding result, so as to increase the probability of the candidate decoding result being selected in subsequent decoding, correct the decoding errors caused by inaccurate pronunciation such as accents, and make the recognition result finally obtained by the speech recognition model conform to the syntactic structure, thereby improving the accuracy of speech recognition.

[0073] For example, in a certain decoding, assume that the candidate decoding results include "chou ci", "chu ci", "chu ci", and "chou ci". The second probability corresponding to "chou ci" is 0.5, the second probability corresponding to "chu ci" is 0.3, the second probability corresponding to "chu ci" is 0.15, and the second probability corresponding to "chou ci" is 0.05.

[0074] Assume that, as Figure 3 shown, when the terminal device finds "chu ci" and "chu ci" in the weighted finite state machine, and the second weight of "chu ci" in the weighted finite state machine is 0.5, and the second weight of "chu ci" in the weighted finite state machine is 0.5, the terminal device can determine that the first weight corresponding to "chou ci" is 0, the first weight corresponding to "chu ci" is 0.5, the first weight corresponding to "chu ci" is 0.5, and the first weight corresponding to "chou ci" is 0. Therefore, the terminal device can determine that the first probability corresponding to "chou ci" is 0.5 + 0 = 0.5, the first probability corresponding to "chu ci" is 0.3 + 0.5 = 0.8, the first probability corresponding to "chu ci" is 0.15 + 0.5 = 0.65, and the first probability corresponding to "chou ci" is 0.05 + 0 = 0.05.

[0075] In another example, the terminal device can determine the scaling value of the first weight corresponding to the candidate decoding result according to the candidate decoding result, and can determine the first probability corresponding to the candidate decoding result according to the first weight, the scaling value of the first weight, and the second probability corresponding to the candidate decoding result.

[0076] Optionally, the first probability corresponding to the candidate decoding result = the second probability corresponding to the candidate decoding result + the first weight * the scaling value of the first weight. Among them, the scaling value of the first weight can be any value such as 0, 1, 2, or 3. The embodiments of the present application do not limit this and can be determined according to the actual situation.

[0077] For example, given the B candidate decoding results obtained from the first decoding, the correctness of the grammatical structure cannot be determined due to the lack of prior information, such as the initial, extracted, and clear characters obtained from the first decoding. In this case, the terminal device can set the scaling value of the first weight corresponding to these B candidate decoding results to 0, that is, the first probability corresponding to these B candidate decoding results is still the second probability corresponding to each candidate decoding result.

[0078] For example, in the second decoding, suppose the candidate decoding results include "chicken", "initial", "Chu Ci", and "crack". The second probability corresponding to "chicken" is 0.5, the second probability corresponding to "initial" is 0.3, the second probability corresponding to "Chu Ci" is 0.15, and the second probability corresponding to "crack" is 0.05. When the terminal device finds "initial" and "Chu Ci" in the weighted finite state machine, and the second weight of "initial" in the weighted finite state machine is 0.5, and the second weight of "Chu Ci" in the weighted finite state machine is 0.5, the terminal device can determine that the first weight corresponding to "chicken" is 0, the first weight corresponding to "initial" is 0.5, the first weight corresponding to "Chu Ci" is 0.5, and the first weight corresponding to "crack" is 0.

[0079] Suppose that the terminal device determines the scaling value of the first weight to be 1 based on the candidate decoding result obtained from the second decoding. Therefore, the terminal device can determine the first probability corresponding to the second extraction as 0.5 + 0 * 1 = 0.5, the first probability corresponding to the initial extraction as 0.3 + 0.5 * 1 = 0.8, the first probability corresponding to the Chu Ci as 0.15 + 0.5 * 1 = 0.65, and the first probability corresponding to the thorn extraction as 0.05 + 0 * 1 = 0.05.

[0080] For example, in the fourth decoding, suppose the candidate decoding results include the next meeting, the first meeting, the meeting in the Chu Ci (Songs of Chu) section, and the next meeting after the first meeting. The second probability corresponding to the next meeting is 0.15, the second probability corresponding to the first meeting is 0.5, the second probability corresponding to the meeting in the Chu Ci section is 0.15, and the second probability corresponding to the next meeting after the first meeting is 0.2. When the terminal device finds the first meeting in the weighted finite state machine, and the second weight of the first meeting in the weighted finite state machine is 0.6, the terminal device can determine that the first weight corresponding to the next meeting is 0, the first weight corresponding to the first meeting is 0.6, the first weight corresponding to the meeting in the Chu Ci section is 0, and the first weight corresponding to the next meeting after the first meeting is 0.

[0081] Suppose that the terminal device determines the scaling value of the first weight to be 2 based on the candidate decoding result obtained from the fourth decoding. Therefore, the terminal device can determine the first probability corresponding to the next meeting as 0.15 + 0 * 2 = 0.15, the first probability corresponding to the first meeting as 0.5 + 0.6 * 2 = 1.7, the first probability corresponding to the meeting in the Chu Ci as 0.15 + 0 * 2 = 0.15, and the first probability corresponding to the next meeting as 0.15 + 0 * 2 = 0.15.

[0082] For example, the scaling value of the first weight corresponding to the candidate decoding result can be determined based on the length of the candidate decoding result. Specifically, the longer the candidate decoding result, the larger the scaling value of the first weight can be; the shorter the candidate decoding result, the smaller the scaling value of the first weight can be. For instance, for candidate decoding result A consisting of 4 characters and candidate decoding result B consisting of 2 characters, the scaling value of the first weight corresponding to candidate decoding result A is greater than the scaling value of the first weight corresponding to candidate decoding result B.

[0083] Optionally, the terminal device may have a mapping between the length of the candidate decoding result and the scaling value. Therefore, the terminal device can obtain the length of the candidate decoding result and determine the scaling value of the first weight corresponding to the candidate decoding result based on the length of the candidate decoding result and the mapping.

[0084] Optionally, the terminal device may have a correspondence between the length range of the candidate decoding result and the scaling value. Therefore, the terminal device can obtain the length of the candidate decoding result and determine the length range within which the candidate decoding result falls. Subsequently, the terminal device can determine the scaling value of the first weight corresponding to the candidate decoding result based on the length range within which the candidate decoding result falls and the correspondence.

[0085] In this embodiment, after acquiring the speech to be recognized, the terminal device can use an end-to-end speech recognition model to perform speech recognition and obtain a recognition result. Specifically, when using the speech recognition model to perform speech recognition, for each decoding step, the terminal device can determine the first probability corresponding to the candidate decoding result based on an N-gram language model. This allows the speech recognition model to determine the candidate decoding result for the next decoding step based on the first probability corresponding to the candidate decoding result. In other words, in this embodiment, when performing speech recognition on the speech to be recognized using an end-to-end speech recognition model, the first probability corresponding to the candidate decoding result can be determined based on an N-gram language model, ensuring that the final decoding result conforms to the grammatical structure, reducing decoding errors caused by inaccurate pronunciation such as accents, improving the accuracy of speech recognition, and enhancing the user experience.

[0086] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0087] Corresponding to the speech recognition method described in the above embodiments, Figure 4 A structural block diagram of a speech recognition device provided in an embodiment of this application is shown. This device can be applied to a terminal device. For ease of explanation, Figure 4 Only the parts relevant to the embodiments of this application are shown.

[0088] like Figure 4 As shown, the device may include:

[0089] The voice acquisition module 401 is used to acquire the voice to be recognized;

[0090] The speech recognition module 402 is used to perform speech recognition on the speech to be recognized using a speech recognition model to obtain a recognition result;

[0091] The speech recognition model is an end-to-end speech recognition model. When the speech recognition model is used to recognize the speech to be recognized, for each decoding of the speech recognition model, the speech recognition module 402 is further used to determine the first probability corresponding to the candidate decoding result according to the N-gram language model. The first probability corresponding to the candidate decoding result is used to determine the candidate decoding result obtained by the speech recognition model in the next decoding.

[0092] In one possible implementation, the speech recognition module 402 is specifically used to obtain a second probability corresponding to the candidate decoding result, wherein the second probability corresponding to the candidate decoding result is determined by the speech recognition model; and to determine a first probability corresponding to the candidate decoding result based on the N-gram language model and the second probability corresponding to the candidate decoding result.

[0093] For example, the speech recognition module 402 is further configured to determine the first weight corresponding to the candidate decoding result based on the N-gram language model; and to determine the first probability corresponding to the candidate decoding result based on the second probability corresponding to the candidate decoding result and the first weight.

[0094] Optionally, the speech recognition module 402 is further configured to, when the candidate decoding result exists in the N-gram language model, determine the second weight corresponding to the candidate decoding result in the N-gram language model, and determine the first weight corresponding to the candidate decoding result based on the second weight; when the candidate decoding result does not exist in the N-gram language model, determine the first weight corresponding to the candidate decoding result as a preset value.

[0095] For example, the speech recognition module 402 is further configured to determine the scaling value of the first weight pair based on the candidate decoding result; and to determine the first probability corresponding to the candidate decoding result based on the first weight, the scaling value corresponding to the first weight, and the second probability corresponding to the candidate decoding result.

[0096] Optionally, the speech recognition module 402 is further configured to determine the length of the candidate decoding result and, based on the length of the candidate decoding result, determine the scaling value corresponding to the first weight.

[0097] In one possible implementation, the device may further include:

[0098] The text acquisition module is used to acquire target text, wherein the target text and the text content corresponding to the speech to be recognized belong to the same domain.

[0099] The language model training module is used to train the N-gram language model using the target text to obtain the trained N-gram language model.

[0100] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0102] Figure 5 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 5 As shown, the terminal device 5 in this embodiment includes: at least one processor 50 ( Figure 5 (Only one is shown in the diagram), memory 51, and computer program 52 stored in said memory 51 and executable on said at least one processor 50, wherein said processor 50 executes said computer program 52 to implement the steps in any of the above-described speech recognition method embodiments.

[0103] The terminal device 5 can be a robot, mobile phone, tablet computer, smart TV, wearable device, in-vehicle device, smart speaker, AR / VR device, laptop computer, or other terminal device. This terminal device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of terminal device 5 and does not constitute a limitation on terminal device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0104] The processor 50 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0105] In some embodiments, the memory 51 may be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. In other embodiments, the memory 51 may be an external storage device of the terminal device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 5. Furthermore, the memory 51 may include both internal and external storage units of the terminal device 5. The memory 51 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0106] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described speech recognition method embodiments.

[0107] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various speech recognition method embodiments.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable storage media cannot be electrical carrier signals or telecommunication signals.

[0109] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0110] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.< / eos> < / sos>

Claims

1. A speech recognition method, applied to a terminal device, characterized in that, The method includes: The terminal device acquires the voice to be recognized; The terminal device uses a speech recognition model to perform speech recognition on the speech to be recognized, and obtains the recognition result. Wherein, the speech recognition model is an end-to-end speech recognition model. When using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding step of the speech recognition model, the terminal device determines a first probability corresponding to the candidate decoding result based on the N-gram language model. The first probability corresponding to the candidate decoding result is used to determine the candidate decoding result obtained by the speech recognition model in the next decoding step. The terminal device determines the first probability corresponding to the candidate decoding result based on the N-gram language model, including: The terminal device obtains a second probability corresponding to the candidate decoding result, and the second probability corresponding to the candidate decoding result is determined by the speech recognition model; The terminal device determines the first weight corresponding to the candidate decoding result based on the N-gram language model; The terminal device determines the length of the candidate decoding result and, based on the length of the candidate decoding result, determines the scaling value corresponding to the first weight; The terminal device determines the first probability corresponding to the candidate decoding result based on the first weight, the scaling value corresponding to the first weight, and the second probability corresponding to the candidate decoding result.

2. The method according to claim 1, characterized in that, The terminal device determines the first weight corresponding to the candidate decoding result based on the N-gram language model, including: When the candidate decoding result exists in the N-gram language model, the terminal device determines the second weight corresponding to the candidate decoding result in the N-gram language model, and determines the first weight corresponding to the candidate decoding result based on the second weight; When the candidate decoding result does not exist in the N-gram language model, the terminal device determines the first weight corresponding to the candidate decoding result as a preset value.

3. The method according to any one of claims 1 to 2, characterized in that, The method further includes: The terminal device acquires the target text, and the target text and the text content corresponding to the speech to be recognized belong to the same domain. The terminal device trains the N-gram language model using the target text to obtain the trained N-gram language model.

4. A voice recognition device, applied to a terminal device, characterized in that, The device includes: The speech acquisition module is used to acquire the speech to be recognized; The speech recognition module is used to perform speech recognition on the speech to be recognized using a speech recognition model to obtain the recognition result; Wherein, the speech recognition model is an end-to-end speech recognition model. When using the speech recognition model to perform speech recognition on the speech to be recognized, for each decoding of the speech recognition model, the speech recognition module is further used to determine the first probability corresponding to the candidate decoding result according to the N-gram language model. The first probability corresponding to the candidate decoding result is used to determine the candidate decoding result obtained by the speech recognition model in the next decoding. The speech recognition module is further configured to obtain a second probability corresponding to the candidate decoding result, wherein the second probability corresponding to the candidate decoding result is determined by the speech recognition model; determine a first weight corresponding to the candidate decoding result according to the N-gram language model; determine the length of the candidate decoding result, and determine a scaling value corresponding to the first weight according to the length of the candidate decoding result; and determine the first probability corresponding to the candidate decoding result according to the first weight, the scaling value corresponding to the first weight, and the second probability corresponding to the candidate decoding result.

5. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the speech recognition method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech recognition method as described in any one of claims 1 to 3.

Images