Speech recognition method and device, equipment and storage medium

Abstract

The invention discloses a voice recognition method and device, equipment and a storage medium. The method comprises the steps of obtaining a plurality of initial recognition texts of a to-be-recognized voice; determining a sequence difference between a first pronunciation sequence of each initial recognition text and a second pronunciation sequence of each target hot word; for each first pronunciation sequence, selecting a second pronunciation sequence of which the sequence difference is not greater than a difference threshold value from each second pronunciation sequence according to a sequence of a plurality of difference threshold values from small to large, and taking the selected second pronunciation sequence as a candidate pronunciation sequence matched with the first pronunciation sequence; and based on each candidate pronunciation sequence, determining a target recognition text of the to-be-recognized speech. In this way, the accuracy of speech recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech recognition method, apparatus, device and storage medium. Background Art

[0002] The application of speech recognition technology is becoming more and more widespread. From smart phones, smart homes to self-driving cars, etc., they all rely on the support of speech recognition technology. The interaction method based on speech recognition has become a widely used service application solution. Many application scenarios such as making calls, sending messages, checking weather information, navigation positioning, searching for music and videos can all be based on speech recognition technology to assist users in completing specific behavioral intentions. Therefore, it is of great significance to improve the recognition effect of speech recognition technology in various specific scenarios. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a speech recognition method, apparatus, device and storage medium, which can improve the accuracy of speech recognition.

[0004] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a speech recognition method, the method comprising: obtaining a number of initial recognition texts of the speech to be recognized; determining the sequence differences between the first pronunciation sequences of each of the initial recognition texts and the second pronunciation sequences of each target hot word; for each of the first pronunciation sequences, in order of multiple difference thresholds from small to large, selecting the second pronunciation sequence whose sequence difference is not greater than the difference threshold in each of the second pronunciation sequences as a candidate pronunciation sequence matching the first pronunciation sequence; based on each of the candidate pronunciation sequences, determining the target recognition text of the speech to be recognized.

[0005] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide a speech recognition device, including: an acquisition module, a difference determination module, a selection module and a text determination module; the acquisition module is used to acquire a number of initial recognition texts of the speech to be recognized; the difference determination module is used to determine the sequence differences between the first pronunciation sequence of each of the initial recognition texts and the second pronunciation sequence of each target hot word; the selection module is used to select, for each of the first pronunciation sequences, in the order of multiple difference thresholds from small to large, the second pronunciation sequence whose sequence difference is not greater than the difference threshold in each of the second pronunciation sequences as a candidate pronunciation sequence matching the first pronunciation sequence; the text determination module is used to determine the target recognition text of the speech to be recognized based on each of the candidate pronunciation sequences.

[0006] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, comprising a memory and a processor coupled to each other, the memory storing program instructions; the processor is used to execute the program instructions stored in the memory to implement the above method.

[0007] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium for storing program instructions, which can be executed to implement the above method.

[0008] The above scheme is to select the second pronunciation sequence whose sequence difference with each first pronunciation sequence satisfies the difference threshold as the candidate pronunciation sequence matching each first pronunciation sequence in the order of multiple difference thresholds from small to large, and then determine the target recognition text of the speech to be recognized based on each candidate pronunciation sequence. Compared with the method of selecting the candidate pronunciation sequence according to a preset threshold, the method of setting multiple preset thresholds and selecting the candidate pronunciation sequence in the order of multiple difference thresholds from small to large in the present application can consider the differences between the pronunciation sequences more carefully, and avoid the situation where the pronunciation sequence that actually matches the first pronunciation sequence is mistakenly excluded due to improper threshold setting, or the pronunciation sequence that actually does not match the first pronunciation sequence is mistakenly included. Therefore, the above method of the present application can more accurately determine the candidate pronunciation sequence matching the first pronunciation sequence, and thus make the target recognition text determined based on each candidate pronunciation sequence more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of an embodiment of a speech recognition method provided by the present application;

[0010] Figure 2 yes Figure 1 The flowchart of step S13 is shown as an embodiment;

[0011] Figure 3 yes Figure 1 The flowchart of step S14 is shown as an embodiment;

[0012] Figure 4 A schematic diagram of a framework of an embodiment of a speech recognition device provided by the present application;

[0013] Figure 5 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0014] Figure 6 It is a schematic diagram of the framework of the computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0016] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0017] It should be noted that the speech recognition method provided in the present application can obtain the final target recognition text by combining the sequence differences between the initial recognition texts of the input speech to be recognized and the pronunciation sequence of the target hot words, as well as the set multiple difference thresholds for speech recognition. It can be used in any scenario where speech recognition is required, including but not limited to making phone calls, sending messages, querying weather information, navigation positioning, or searching for music videos.

[0018] Among them, hot words are words that the user inputs to express his / her desired expression, or are words or phrases that are predicted to be most likely to be said or used by the user based on the user's historical behavior, screen display content or specific application scenarios.

[0019] Each initial recognition text is obtained by recognizing the speech to be recognized using a speech recognition model. The speech recognition model can be, but is not limited to, an end-to-end recognition model, or a traditional speech recognition model.

[0020] Specifically, see Figure 1 , Figure 1 is a flow chart of an embodiment of the speech recognition method provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not take Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0021] S11: Obtaining a number of initial recognition texts of the speech to be recognized.

[0022] The speech to be recognized is the input speech to be subjected to text recognition, which may be speech composed of Chinese or speech composed of other languages ​​(such as English).

[0023] Each initial recognition text is obtained by recognizing the speech to be recognized using the speech recognition model. Specifically, each initial recognition text can be all texts obtained by recognizing the speech to be recognized by the speech recognition model, or can be the best texts among all the recognized texts, and the best texts are the most acoustically and linguistically consistent texts obtained by combining the acoustic information and language information of the speech to be recognized input by the user.

[0024] In one embodiment, after obtaining the speech to be recognized input by the user, the speech to be recognized can be preprocessed, for example, by using signal processing techniques such as windowing and framing, Fourier transform, and Mel cepstrum to extract audio features in the speech to be recognized, and then using a speech recognition model to process the audio features to obtain a number of candidate recognition results for each speech frame in the speech to be recognized, wherein each candidate recognition result includes a candidate recognition text of the corresponding speech frame and a corresponding ED score (a comprehensive score output by combining acoustic information and speech information); then, the candidate recognition results of each speech frame are combined to obtain a number of initial recognition texts.

[0025] In a specific embodiment, the main structure of the speech recognition model includes an encoder (Encoder), a decoder (Decoder) and a search network (BeamSearch), wherein the encoder is equivalent to the acoustic model in the traditional model, the decoder is similar to the language model, and the BeamSearch is similar to the path search process. The speech recognition model will pre-set the number of beams, wherein the number of beams determines the number of candidate texts for each speech frame output each time. If the beam is 2, taking the text corresponding to the speech to be recognized as "I am *" as an example, after the speech to be recognized is input into the speech recognition model, each beam will first output the candidate recognition result of the first frame of speech and then select [{I (edScore: 10), lying (edScore: 9)}, {Oh (edScore: 8), Five (edScore: 7)}], and then BeamSearch will judge the edScore scores corresponding to the output candidate texts and select the one with the highest score. The two highest candidate recognition results [I (edScore: 10), bedroom (edScore: 9)] are sent to the decoder to predict the output of the next frame of speech. The next frame outputs [{I (edScore: 10), four (edScore: 9)}, {room (edScore: 11), Si (edScore: 8)}]. After the BeamSearch score (the score of the previous corresponding beam will be accumulated) is sorted and output [I (edScore: 10+10), room (edScore: 9+11)], the result is sent to the decoder to predict the output of the next frame. The two best candidate recognition results of each time will be saved during the BeamSearch process. Finally, after the last frame judgment, the two initial recognition results with the highest scores are determined according to the score sorting [{I am * (edScore: 10+10+12), bedroom × (edScore: 11+9+11)}].

[0026] S12: Determine the sequence differences between the first pronunciation sequence of each initial recognition text and the second pronunciation sequence of each target hot word.

[0027] In one embodiment, each initial recognition text is subjected to pronunciation alignment (FA, Forced Alignment) to obtain a first pronunciation sequence corresponding to each initial recognition text. For example, each initial recognition text is sent to a pronunciation dictionary (or pronunciation model) to obtain candidate pronunciations corresponding to each word or character in the initial recognition text, and then these candidate pronunciations are combined to form the first pronunciation sequence of each initial recognition text.

[0028] In a specific embodiment, determining the first pronunciation sequence of each initial recognition text includes the following steps:

[0029] First, obtain the acoustic information of the speech to be recognized and multiple candidate pronunciation sequences; wherein the acoustic information includes the predicted probabilities of the speech frames in the speech to be recognized being recognized as several candidate pronunciations respectively, and any candidate pronunciation sequence is obtained by combining the candidate pronunciations of each speech frame.

[0030] Second, based on the predicted probabilities of each speech frame in the speech to be recognized being respectively identified as a number of candidate pronunciations, a candidate pronunciation sequence with the highest acoustic probability is selected from a plurality of candidate pronunciation sequences as the first pronunciation sequence.

[0031] In summary, this embodiment determines the acoustic probability corresponding to each candidate pronunciation sequence (determined by the predicted probability of each candidate pronunciation in the candidate pronunciation sequence) after determining the predicted probability of each speech frame in the speech to be recognized as several candidate pronunciations and combining the candidate pronunciations of each speech frame to obtain the corresponding multiple candidate pronunciation sequences, and then selects the candidate pronunciation sequence with the highest acoustic probability as the first pronunciation sequence from the multiple candidate pronunciation sequences.

[0032] In one embodiment, while determining the first pronunciation sequence of each initial recognition text, the acoustic score corresponding to each first pronunciation sequence is recorded so that the target recognition text can be determined later based on the acoustic score. The acoustic score corresponding to each first pronunciation sequence is obtained by performing a preset operation on the corresponding acoustic probability.

[0033] In this embodiment, the target hot word is a word obtained in response to the user's input operation, or a word predicted by analyzing the user's historical behavior data. The target hot word can be obtained before, during, or after obtaining the speech to be recognized. The input or acquisition timing of the target hot word can be selected according to actual needs, and is not specifically limited here.

[0034] In one embodiment, the target hot word only includes the target hot word itself.

[0035] In another embodiment, the target hot word includes a command word and the associated hot words of the command word. After obtaining each initial recognition text, the command word in the recognition text is determined, and then the associated hot words associated with the command word are found from a preset hot word library, and the command word and the associated hot words of the command word are used as the target hot words together.

[0036] In this embodiment, after the target hot word is acquired, the target hot word needs to be processed to obtain a second pronunciation sequence corresponding to the target hot word.

[0037] In this embodiment, the sequence difference between the first pronunciation sequence of the initial recognition text and the second pronunciation sequence of the target hot word is used to characterize the pronunciation similarity between the initial recognition text and the target hot word. It can be understood that the smaller the sequence difference, the higher the corresponding pronunciation similarity.

[0038] In one embodiment, determining the sequence difference between the first pronunciation sequence of each initial recognition text and the second pronunciation sequence of each target hot word includes the following steps:

[0039] First, based on a preset confusion matrix, the pronunciation distance between each phoneme in the first pronunciation sequence and the second pronunciation sequence is calculated; wherein the preset confusion matrix includes the degree of confusion between different phonemes.

[0040] Second, the pronunciation distances between the phonemes in the first pronunciation sequence and the second pronunciation sequence are integrated to obtain sequence differences.

[0041] In this embodiment, the preset confusion matrix includes the degree of confusion between different phonemes. The preset confusion matrix is ​​obtained based on the recognition results of all phonemes in Chinese Pinyin and / or based on the recognition results of all phonemes in other languages. It is a quantitative description of the degree of confusion between each phoneme, that is, a description of acoustic similarity from the perspective of speech recognition.

[0042] In a specific embodiment, the specific preset confusion matrix is ​​the weight information corresponding to each phoneme predicted by using the acoustic model, and the calculated vector gap between different phonemes can be used as the pronunciation distance.

[0043] The higher the degree of confusion between two phonemes, the closer the pronunciation distance between the two phonemes. In this embodiment, after determining the pronunciation distance between each phoneme in the first pronunciation sequence and the second pronunciation sequence, the pronunciation distance between each phoneme can be integrated to obtain a comprehensive pronunciation distance between the two pronunciation sequences, wherein the comprehensive pronunciation distance is used to characterize the sequence difference between the two pronunciation sequences.

[0044] S13: For each first pronunciation sequence, in the order of multiple difference thresholds from small to large, select a second pronunciation sequence whose sequence difference is not greater than the difference threshold from each second pronunciation sequence as a candidate pronunciation sequence matching the first pronunciation sequence.

[0045] In this embodiment, multiple difference thresholds are set, and when selecting a candidate pronunciation sequence that matches the first pronunciation sequence from the second pronunciation sequence, the selection is performed in order of the multiple difference thresholds from small to large, that is, when selecting a matching candidate pronunciation sequence, the second pronunciation sequence that has a relatively small difference with the first pronunciation sequence is preferentially selected as the candidate pronunciation sequence that matches the first pronunciation sequence, that is, the second pronunciation sequence that is most similar in pronunciation to the first pronunciation sequence is preferentially selected as the candidate pronunciation sequence that matches the first pronunciation sequence.

[0046] In one embodiment, see Figure 2 , Figure 2 yes Figure 1 The flowchart of an embodiment of step S13 is shown. In this embodiment, step S13 further includes:

[0047] S21: selecting a difference threshold as the current threshold in order from small to large among the multiple difference thresholds.

[0048] S22: Detect whether there is a second pronunciation sequence whose sequence difference is not greater than the current threshold in each second pronunciation sequence.

[0049] S23: In response to detecting a second pronunciation sequence whose sequence difference is not greater than the current threshold, directly selecting the second pronunciation sequence whose sequence difference is not greater than the current threshold as a candidate pronunciation sequence matching the first pronunciation sequence.

[0050] S24: In response to not detecting a second pronunciation sequence whose sequence difference is not greater than the current threshold, return to executing the step of selecting the difference threshold as the current threshold in order of multiple difference thresholds from small to large, until a second pronunciation sequence whose sequence difference is not greater than the current threshold is detected, or all difference thresholds have been selected.

[0051] In summary, in this embodiment, for each first pronunciation sequence, first, a difference threshold is selected as the current threshold in order of multiple difference thresholds from small to large, and then each second pronunciation sequence is detected whether there is a second pronunciation sequence whose sequence difference with the first pronunciation sequence is not greater than the current threshold. If it is detected that there is a second pronunciation sequence whose sequence difference with the first pronunciation sequence is not greater than the current threshold, the second pronunciation sequence whose sequence difference with the first pronunciation sequence is not greater than the current threshold is directly selected as a candidate pronunciation sequence matching the first pronunciation sequence; if the second pronunciation sequence whose sequence difference with the first pronunciation sequence is not greater than the current threshold is not detected, the process returns to step S21 until a second pronunciation sequence whose sequence difference with the first pronunciation sequence is not greater than the current threshold is detected, or all difference thresholds have been selected.

[0052] The number of difference thresholds and each specific difference threshold can be preset according to actual needs. For example, two difference thresholds are set, wherein the minimum difference threshold is set to 0 and the other difference threshold is a% (a>0), so that the second pronunciation sequence that is completely matched with each first pronunciation sequence (sequence difference is not greater than 0) is preferentially used as a candidate pronunciation sequence in the above manner. If no completely matching candidate pronunciation sequence is found, another difference threshold is used to detect whether there is a second pronunciation sequence with a sequence difference not greater than another difference threshold a%.

[0053] That is to say, the method provided in this embodiment can find a second pronunciation sequence that is not completely similar to the first pronunciation sequence but has a relatively high similarity as a candidate pronunciation sequence matching the first pronunciation sequence when no second pronunciation sequence that completely matches the first pronunciation sequence is found.

[0054] Exemplarily, the initial recognition text of the speech to be recognized is ["Navigate to Jing'an Primary School", "Navigate to Jin'an Primary School"], and through the processing of step S13, it is found that "Navigate to Jing'an Primary School (d ao h ang quj ing an x ​​iao x ue)" and the target hot word "Navigate to Jing'an Primary School (d ao h ang quj ing an x ​​iao x ue)" are completely matched. At this time, the second pronunciation sequence corresponding to the target hot word "Navigate to Jing'an Primary School (d ao h ang quj ing an x ​​iao x ue)" is used as a candidate pronunciation sequence that matches the first pronunciation sequence corresponding to "Navigate to Jing'an Primary School (d ao h ang quj ing an x ​​iao x ue).

[0055] For another example, the initial recognition text of the speech to be recognized is [“Navigation to Shenghai Primary School”, “Navigation to Shenghuai Primary School”], where, for “Navigation to Shenghai Primary School (d ao hang qu sh en h ai x iao x ue)”, through the processing of step S13, it is found that there is no second pronunciation sequence that completely matches its first pronunciation sequence, but there is a second pronunciation sequence that is very similar to its first pronunciation sequence, specifically the second pronunciation sequence corresponding to “Navigation to Shenhai Primary School (d ao hang qu sh eng h ai x iao xue)”. Therefore, at this time, the second pronunciation sequence corresponding to “Navigation to Shenhai Primary School (d ao hang qu sh eng h ai xiao x ue)” can be used as a candidate pronunciation sequence matching the first pronunciation sequence.

[0056] S14: Determine the target recognition text of the speech to be recognized based on each candidate pronunciation sequence.

[0057] In this embodiment, the number of target recognition texts of the speech to be recognized is finally determined to be one. In order to make the target recognition text finally determined more accurate, the candidate pronunciation sequences found can be screened again to select the desired target pronunciation sequence from the candidate pronunciation sequences, and then the target recognition text is determined based on the target pronunciation sequence. For specific implementation methods, please refer to Figure 3 Description of the illustrated embodiment.

[0058] The above scheme is to select the second pronunciation sequence whose sequence difference with each first pronunciation sequence satisfies the difference threshold as the candidate pronunciation sequence matching each first pronunciation sequence in the order of multiple difference thresholds from small to large, and then determine the target recognition text of the speech to be recognized based on each candidate pronunciation sequence. Compared with the method of selecting the candidate pronunciation sequence according to a preset threshold, the method of setting multiple preset thresholds and selecting the candidate pronunciation sequence in the order of multiple difference thresholds from small to large in the present application can consider the differences between the pronunciation sequences more carefully, and avoid the situation where the pronunciation sequence that actually matches the first pronunciation sequence is mistakenly excluded or the pronunciation sequence that actually matches the first pronunciation sequence is mistakenly included due to improper threshold setting. Therefore, the above method of the present application can more accurately determine the candidate pronunciation sequence matching the first pronunciation sequence, and thus make the target recognition text determined based on each candidate pronunciation sequence more accurate.

[0059] See also Figure 3 , Figure 3 yes Figure 1 The flowchart of an embodiment of step S14 is shown. In this embodiment, step S14 further includes:

[0060] S31: Select a target pronunciation sequence from each candidate pronunciation sequence.

[0061] Preferably, the similarity between the acoustic information and the pronunciation sequence (which can be represented by the degree of matching or the degree of difference) can be comprehensively considered to score each candidate pronunciation sequence, and finally the candidate pronunciation sequence with the highest comprehensive score can be selected as the target pronunciation sequence. For example, the candidate pronunciation sequence with the highest score can be selected as the target pronunciation sequence.

[0062] Specifically, in one embodiment, for each candidate pronunciation sequence, a second acoustic score of the first pronunciation sequence matching the candidate pronunciation sequence can be obtained first, and a penalty score associated with the sequence difference between the candidate pronunciation sequence and the first pronunciation sequence matching it can be obtained. Then, based on the second acoustic score and the penalty score, a second comprehensive score of the candidate pronunciation sequence can be obtained. Finally, based on the second comprehensive score of each candidate pronunciation sequence, the corresponding candidate pronunciation sequence can be selected as the target pronunciation sequence.

[0063] In order to reduce the influence of candidate pronunciation sequences with low similarity to the initial recognition text on speech recognition, the penalty score can be set to be positively correlated with the sequence difference, that is, the greater the sequence difference, the higher the corresponding penalty score.

[0064] Of course, in other embodiments, the target pronunciation sequence may be selected by only using the second acoustic score of the first pronunciation sequence matching the candidate pronunciation sequence, or the matching degree between the candidate pronunciation sequence and the first pronunciation sequence matching it.

[0065] S32: Determine the target recognition text based on the target pronunciation sequence.

[0066] In one embodiment, the first pronunciation sequence of the initial recognition text includes the pronunciation sequence corresponding to the keyword in the initial recognition text, and the target hotword is the hotword itself. In this embodiment, the initial recognition text corresponding to the first pronunciation sequence that matches the target pronunciation sequence can be used as the reference recognition text; the keyword in the reference recognition text is adjusted to the target hotword corresponding to the target pronunciation sequence, and the adjusted reference recognition text is used as the target recognition text.

[0067] Exemplarily, the target hotword corresponding to the target pronunciation sequence is "Shenhai Primary School", and the reference recognition text is "Navigate to Shenghai Primary School", where the keyword is "Shenhai Primary School". Therefore, the keyword "Shenhai Primary School" in "Navigate to Shenhai Primary School" can be adjusted to "Shenhai Primary School", and the final target recognition text obtained is "Navigate to Shenhai Primary School".

[0068] In another embodiment, the target hotword includes a command word and an associated hotword of the command word. In this embodiment, the command word and the associated hotword of the command word corresponding to the target pronunciation sequence can be directly used as the target recognition text, that is, the target hotword is directly used as the target recognition text. For example, the target hotword corresponding to the target pronunciation sequence is "Navigate to Shenhai Primary School", where the command word is "Navigate to", and the associated hotword is "Shenhai Primary School", and the final target recognition text is "Navigate to Shenhai Primary School".

[0069] Exemplarily, suppose the initial recognition text includes ["Navigate to Shenhai Primary School" AcScore, "Navigate to Shenghuai Primary School"]. Among them, after processing in step S13, it is found that the penalty score associated with the sequence difference between an initial recognition text "Navigate to Shenhai Primary School (dao hang qu shen hai xiao xue)" and the target hotword "Navigate to Shenhai Primary School (dao hang qu sheng hai xiao xue)" is 5 points, and the corresponding second acoustic score is 1777 penalty points: total score: 1772. The penalty score associated with the sequence difference between another initial recognition text "Navigate to Shenghuai Primary School (dao hang qu sheng huai xiao xue)" and the target hotword "Navigate to Shenhuai Primary School (dao hang qu shen huai xiao xue)" is 5 points, and the corresponding second acoustic score is 1677, total score: 1672. Finally, after comparison, it is determined that the second pronunciation sequence corresponding to the target hotword is the target pronunciation sequence, and the determined target recognition text is "Navigate to Shenhai Primary School".

[0070] It should be noted that Figure 3The illustrated embodiment is executed on the basis of determining in step S13 that there is a candidate pronunciation sequence (second pronunciation sequence) whose sequence difference with the first pronunciation sequence is not greater than the difference threshold. In some embodiments, if S13 determines that there is no second pronunciation sequence matching any first pronunciation sequence, the target recognition text of the speech to be recognized can be determined based on the first comprehensive score of each initial recognition text.

[0071] Specifically, when it is determined that there is no second pronunciation sequence matching any first pronunciation sequence, the first comprehensive score of each initial recognition text is obtained, and then the initial recognition text with the highest first comprehensive score is used as the target recognition text. The first comprehensive score of the initial recognition text is obtained by combining the first acoustic score and the language score of the initial recognition text.

[0072] The first acoustic score can be directly predicted by using the speech recognition model, and the language score is obtained by querying the language model.

[0073] See also Figure 4 , Figure 4 A schematic diagram of the framework of an embodiment of a speech recognition device provided in the present application. In this embodiment, the speech recognition device 40 includes an acquisition module 41, a difference determination module 42, a selection module 43 and a text determination module 44. The acquisition module 41 is used to acquire a number of initial recognition texts of the speech to be recognized; the difference determination module 42 is used to determine the sequence differences between the first pronunciation sequence of each initial recognition text and the second pronunciation sequence of each target hot word; the selection module 43 is used to select, for each first pronunciation sequence, a second pronunciation sequence whose sequence difference is not greater than the difference threshold in each second pronunciation sequence in the order of multiple difference thresholds from small to large, as a candidate pronunciation sequence matching the first pronunciation sequence; the text determination module 44 is used to determine the target recognition text of the speech to be recognized based on each candidate pronunciation sequence.

[0074] In some embodiments, the selection module 43 selects a second pronunciation sequence whose sequence difference is not greater than the difference threshold in each second pronunciation sequence in the order of multiple difference thresholds from small to large, as a candidate pronunciation sequence matching the first pronunciation sequence, including: selecting the difference threshold in turn as the current threshold in the order of small to large; detecting whether there is a second pronunciation sequence whose sequence difference is not greater than the current threshold in each second pronunciation sequence; in response to detecting a second pronunciation sequence whose sequence difference is not greater than the current threshold, directly selecting a second pronunciation sequence whose sequence difference is not greater than the current threshold as a candidate pronunciation sequence matching the first pronunciation sequence; in response to not detecting a second pronunciation sequence whose sequence difference is not greater than the current threshold, returning to the step of selecting the difference threshold as the current threshold in the order of multiple difference thresholds from small to large, until a second pronunciation sequence whose sequence difference is not greater than the current threshold is detected, or all difference thresholds have been selected.

[0075] In some embodiments, the text determination module 44 is also used to obtain a first comprehensive score for each initial recognition text in response to the absence of a second pronunciation sequence that matches any first pronunciation sequence; wherein the first comprehensive score is obtained by integrating the first acoustic score and the language score of the initial recognition text; and the initial recognition text with the highest first comprehensive score is used as the target recognition text.

[0076] In some embodiments, the text determination module 44 determines the target recognition text of the speech to be recognized based on each candidate pronunciation sequence, including: selecting a target pronunciation sequence from each candidate pronunciation sequence; and determining the target recognition text based on the target pronunciation sequence.

[0077] In some embodiments, a target pronunciation sequence is selected from each candidate pronunciation sequence, including: for each candidate pronunciation sequence, obtaining a second acoustic score of a first pronunciation sequence that matches the candidate pronunciation sequence, and obtaining a penalty score associated with the sequence difference between the candidate pronunciation sequence and the first pronunciation sequence that matches it; wherein the penalty score is positively correlated with the sequence difference; obtaining a second comprehensive score of the candidate pronunciation sequence based on the second acoustic score and the penalty score; and selecting the candidate pronunciation sequence as the target pronunciation sequence based on the second comprehensive score of each candidate pronunciation sequence.

[0078] In some embodiments, the first pronunciation sequence of the initial recognition text includes a pronunciation sequence corresponding to a keyword in the initial recognition text; based on the target pronunciation sequence, the target recognition text is determined, including: using the initial recognition text corresponding to the first pronunciation sequence matching the target pronunciation sequence as a reference recognition text; adjusting the keywords in the reference recognition text to target hot words corresponding to the target pronunciation sequence, and using the adjusted reference recognition text as the target recognition text; or, the target hot words include command words and associated hot words of the command words; based on the target pronunciation sequence, the target recognition text is determined, including: using the command words corresponding to the target pronunciation sequence and associated hot words of the command words as the target recognition text.

[0079] In some embodiments, before the difference determination module 42 determines the sequence differences between the first pronunciation sequence of each initial recognition text and the second pronunciation sequence of each target hot word, it also includes: obtaining acoustic information of the speech to be recognized and multiple candidate pronunciation sequences; wherein the acoustic information includes the predicted probability that the speech frames in the speech to be recognized are respectively recognized as several candidate pronunciations, and any candidate pronunciation sequence is obtained by combining the candidate pronunciations of each speech frame; based on the predicted probability that each speech frame in the speech to be recognized is respectively recognized as several candidate pronunciations, from the multiple candidate pronunciation sequences, select the candidate pronunciation sequence with the highest acoustic probability as the first pronunciation sequence.

[0080] In some embodiments, the difference determination module 42 determines the sequence differences between the first pronunciation sequence of each initial recognition text and the second pronunciation sequence of each target hot word, including: based on a preset confusion matrix, calculating the pronunciation distance between each phoneme in the first pronunciation sequence and the second pronunciation sequence; wherein the preset confusion matrix includes the degree of confusion between different phonemes; and obtaining the sequence difference by integrating the pronunciation distance between each phoneme in the first pronunciation sequence and the second pronunciation sequence.

[0081] In some embodiments, the target hot word is obtained during or after receiving the user's voice to be recognized; and / or, the target hot word is obtained in response to the user's input operation, or is obtained by analyzing the user's historical behavior data; and / or, each initial recognition text is obtained by recognizing the voice to be recognized using a speech recognition model.

[0082] See also Figure 5 , Figure 5 1 is a schematic diagram of a framework of an electronic device according to an embodiment of the present application. In this embodiment, the electronic device 50 includes a memory 51 and a processor 52 coupled to each other.

[0083] The memory 51 stores program instructions, and the processor 52 is used to execute the program instructions stored in the memory 51 to implement the steps of any of the above method implementations. In a specific implementation scenario, the electronic device 50 may include, but is not limited to: a microcomputer, a server, and in addition, the electronic device 50 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0084] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps of any of the above-mentioned embodiments. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.

[0085] See also Figure 6 , Figure 6 It is a schematic diagram of the framework of the computer-readable storage medium provided by the present application. The computer-readable storage medium 60 of the embodiment of the present application stores a program instruction 61, and when the program instruction 61 is executed, the method provided by any embodiment of the above method and any non-conflicting combination is implemented. Among them, the program instruction 61 can form a program file and be stored in the above-mentioned computer-readable storage medium 60 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 60 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0086] The above scheme is to select the second pronunciation sequence whose sequence difference with each first pronunciation sequence satisfies the difference threshold as the candidate pronunciation sequence matching each first pronunciation sequence according to the order of multiple difference thresholds from small to large, and then determine the target recognition text of the speech to be recognized based on each candidate pronunciation sequence. Compared with the method of selecting the candidate pronunciation sequence according to a preset threshold, the present application sets multiple preset thresholds and selects the candidate pronunciation sequence according to the order of multiple difference thresholds from small to large, which can consider the differences between the pronunciation sequences more carefully, and avoid the situation where the pronunciation sequence that actually matches the first pronunciation sequence is mistakenly excluded or the pronunciation sequence that actually matches the first pronunciation sequence is mistakenly included due to improper threshold setting. Therefore, the above method of the present application can more accurately determine the candidate pronunciation sequence matching the first pronunciation sequence, and thus make the target recognition text determined based on each candidate pronunciation sequence more accurate.

[0087] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0088] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0093] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A speech recognition method, characterized in that: The method comprises: Obtaining a number of initial recognition texts of the speech to be recognized; Determine the sequence difference between the first pronunciation sequence of each of the initial recognition texts and the second pronunciation sequence of each of the target hot words; For each of the first pronunciation sequences, in the order of multiple difference thresholds from small to large, select a second pronunciation sequence whose sequence difference is not greater than the difference threshold from each of the second pronunciation sequences as a candidate pronunciation sequence matching the first pronunciation sequence; Based on each of the candidate pronunciation sequences, a target recognition text of the speech to be recognized is determined.

2. The method according to claim 1, characterized in that The step of selecting, in order of the plurality of difference thresholds from small to large, a second pronunciation sequence whose sequence difference is not greater than the difference threshold from each of the second pronunciation sequences as a candidate pronunciation sequence matching the first pronunciation sequence comprises: Selecting the difference threshold as the current threshold in order from small to large; Detecting whether there is a second pronunciation sequence in each of the second pronunciation sequences whose sequence difference is not greater than the current threshold; In response to detecting a second pronunciation sequence whose sequence difference is not greater than the current threshold, directly selecting the second pronunciation sequence whose sequence difference is not greater than the current threshold as a candidate pronunciation sequence matching the first pronunciation sequence; In response to not detecting a second pronunciation sequence whose sequence difference is not greater than the current threshold, returning to the step of selecting the difference thresholds as the current threshold in order from small to large according to the multiple difference thresholds, until a second pronunciation sequence whose sequence difference is not greater than the current threshold is detected, or all the difference thresholds have been selected.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to the absence of a second pronunciation sequence matching any of the first pronunciation sequences, obtaining a first comprehensive score for each of the initial recognition texts; wherein the first comprehensive score is obtained by combining a first acoustic score and a language score for the initial recognition text; The initial recognized text with the highest first comprehensive score is used as the target recognized text.

4. The method according to claim 1, characterized in that: The step of determining the target recognition text of the speech to be recognized based on each of the candidate pronunciation sequences comprises: Selecting a target pronunciation sequence from each of the candidate pronunciation sequences; Based on the target pronunciation sequence, the target recognition text is determined.

5. The method according to claim 4, characterized in that The step of selecting a target pronunciation sequence from each of the candidate pronunciation sequences comprises: For each of the candidate pronunciation sequences, obtaining a second acoustic score of a first pronunciation sequence matching the candidate pronunciation sequence, and obtaining a penalty score associated with a sequence difference between the candidate pronunciation sequence and the first pronunciation sequence matching the candidate pronunciation sequence; wherein the penalty score is positively correlated with the sequence difference; Obtaining a second comprehensive score for the candidate pronunciation sequence based on the second acoustic score and the penalty score; Based on the second comprehensive scores of the candidate pronunciation sequences, the candidate pronunciation sequences are selected as the target pronunciation sequences.

6. The method according to claim 4, characterized in that The first pronunciation sequence of the initial recognition text includes a pronunciation sequence corresponding to a keyword in the initial recognition text; and determining the target recognition text based on the target pronunciation sequence includes: Using the initial recognition text corresponding to the first pronunciation sequence matching the target pronunciation sequence as the reference recognition text; Adjusting the keywords in the reference recognition text to target hot words corresponding to the target pronunciation sequence, and using the adjusted reference recognition text as the target recognition text; Alternatively, the target hot words include command words and hot words associated with the command words; and determining the target recognition text based on the target pronunciation sequence includes: The command word corresponding to the target pronunciation sequence and the associated hot words of the command word are used as the target recognition text.

7. The method according to claim 1, characterized in that Before determining the sequence differences between the first pronunciation sequences of the respective initial recognition texts and the second pronunciation sequences of the respective target hot words, the method further comprises: Acquiring acoustic information of the speech to be recognized and a plurality of candidate pronunciation sequences; wherein the acoustic information includes the predicted probabilities of the speech frames in the speech to be recognized being respectively recognized as a plurality of candidate pronunciations, and any of the candidate pronunciation sequences is obtained by combining the candidate pronunciations of the respective speech frames; Based on the predicted probabilities that each speech frame in the to-be-recognized speech is respectively recognized as a plurality of candidate pronunciations, a candidate pronunciation sequence with the highest acoustic probability is selected from the plurality of candidate pronunciation sequences as the first pronunciation sequence.

8. The method according to claim 1, characterized in that: The determining of the sequence differences between the first pronunciation sequence of each of the initial recognition texts and the second pronunciation sequence of each of the target hot words comprises: Based on a preset confusion matrix, calculating the pronunciation distance between each phoneme in the first pronunciation sequence and the second pronunciation sequence; wherein the preset confusion matrix includes the degree of confusion between different phonemes; The sequence difference is obtained by synthesizing the pronunciation distances between the phonemes in the first pronunciation sequence and the second pronunciation sequence.

9. The method according to claim 1, characterized in that: The target hot word is obtained during or after receiving the user's speech to be recognized; And / or, the target hot word is obtained in response to the user's input operation, or obtained by analyzing the user's historical behavior data; And / or, each of the initial recognition texts is obtained by recognizing the speech to be recognized using a speech recognition model.

10. A speech recognition device, characterized in that: The device comprises: An acquisition module, used to acquire a number of initial recognition texts of the speech to be recognized; A difference determination module, used for determining the sequence difference between the first pronunciation sequence of each of the initial recognition texts and the second pronunciation sequence of each of the target hot words; A selection module is used for selecting, for each of the first pronunciation sequences, a second pronunciation sequence whose sequence difference is not greater than the difference threshold in the order of a plurality of difference thresholds from small to large from each of the second pronunciation sequences as a candidate pronunciation sequence matching the first pronunciation sequence; The text determination module is used to determine the target recognition text of the speech to be recognized based on each of the candidate pronunciation sequences.

11. An electronic device, characterized in that: comprising a memory and a processor coupled to each other, The memory stores program instructions; The processor is used to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions that can be run by a processor, and the program instructions can be executed by the processor to implement the method according to any one of claims 1 to 9.

Images