Speech Processing Methods and Systems

Abstract

This application belongs to the field of speech processing technology and provides a speech processing method, apparatus, electronic device, and program product. The speech processing method acquires probability information of multiple output units corresponding to an input speech frame, each output unit representing the confidence level of the input speech frame being classified into a corresponding character; determines a first unit with the highest probability value and a second unit with the second highest probability value among the multiple output units based on the probability information; if the first unit indicates that the input speech frame is classified into a text character, then determines a set of characters that match the text character; according to a preset smoothing coefficient and the probability difference between the first unit and the second unit, allocates a predetermined proportion of the probability value of the first unit to the output units corresponding to each text character in the text set to obtain smoothed probability information; performs speech processing on the input speech frame based on the smoothed probability information to obtain a speech processing result, wherein the speech processing includes at least one of decoding processing and context bias enhancement processing.

Description

Technical Field

[0001] This application belongs to the field of speech processing technology, and more specifically, relates to a speech processing method, apparatus, electronic device and program product. Background Technology

[0002] In the field of speech recognition technology, character modeling acoustic models (AM models) trained based on the Connectionist Temporal Classification (CTC) loss function are widely used in smart TVs, in-vehicle voice systems, and intelligent customer service scenarios because they do not require pinyin annotation and have low training costs.

[0003] However, this type of model exhibits a significant spike effect, meaning that it assigns extremely high probabilities to the most confident characters, while the probabilities of other characters (especially those with similar pronunciations) approach zero. This spike effect leads to the pruning of key near-phonetic characters in some words during the beam search stage, resulting in lower accuracy in recognizing new words and proper nouns. Summary of the Invention

[0004] The purpose of this application is to provide a speech processing method, apparatus, electronic device, and program product, which aims to solve the technical problem that the existing context bias enhancement processing technology based on shallow fusion technology fails due to the AM model spike effect, resulting in low speech recognition accuracy.

[0005] To achieve the above objectives, according to the first aspect of this application, a speech processing method is provided, the method comprising: Obtain probability information of multiple output units corresponding to the input speech frame, wherein each output unit is used to characterize the confidence level of the input speech frame being classified into the corresponding character, and the character includes text characters used to identify text and special characters used to identify the silence state; Based on the probability information, determine the first unit with the highest probability value and the second unit with the second highest probability value among the multiple output units; If the first unit indicates that the input speech frame is classified as the text character, then the set of characters that match the text character is determined; Based on the preset smoothing coefficient and the probability difference between the first unit and the second unit, a predetermined proportion of the probability value of the first unit is allocated to the output unit corresponding to each character in the text set to obtain smoothed probability information. Speech processing is performed on the input speech frame based on the smoothed probability information to obtain the speech processing result. The speech processing includes at least one of decoding processing and context bias enhancement processing.

[0006] Furthermore, the speech processing method provided in this application embodiment can effectively mitigate the spike effect problem in speech recognition technology, improve speech recognition accuracy and stability, and enhance scenario applicability and user experience.

[0007] According to a second aspect of this application, a speech recognition system is provided, comprising: The acquisition unit is configured to acquire probability information of multiple output units corresponding to the input speech frame, wherein each output unit is used to characterize the confidence level of the input speech frame being classified into a corresponding character, and the character includes text characters used to identify text and special characters used to identify a silence state; A probability smoothing unit is configured to determine, based on the probability information, a first unit with the highest probability value and a second unit with the second highest probability value among a plurality of output units; if the first unit indicates that the input speech frame is classified as a text character, then a set of text characters matching the text character is determined; based on a preset smoothing coefficient and the probability difference between the first unit and the second unit, a predetermined proportion of the probability value of the first unit is allocated to the output units corresponding to each text character in the text set to obtain smoothed probability information; The speech processing unit is configured to perform speech processing on the input speech frame based on the smoothed probability information to obtain a speech processing result, wherein the speech processing includes at least one of decoding processing and context bias enhancement processing.

[0008] According to a third aspect of this application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device causes the electronic device to perform the method as described in any one of the claims.

[0009] According to a fourth aspect of this application, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the method as described in any one of the claims.

[0010] According to a fifth aspect of this application, a computer program product is provided that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects above.

[0011] It is understandable that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of a speech processing method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating an optional speech processing method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating an optional speech processing method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating an optional speech processing method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating an optional speech processing method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a voice processing system provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0016] It should also be understood that, in the description of this application, unless otherwise stated, the " / " used in the specification and appended claims indicates that the related objects are in an "or" relationship. For example, A / B can mean A or B. The "and / or" in this application is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0017] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, but are only used for distinguishing descriptions, and the terms "first" and "second" do not necessarily imply that they are different, nor should they be construed as indicating or implying relative importance.

[0018] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0019] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0020] This disclosure relates to the field of speech recognition (ASR) technology, specifically applicable to context-biased (also known as hot word enhancement) scenarios. Addressing the spike effect in acoustic models (AM models) trained using character modeling and Connectionist Temporal Classification (CTC) loss functions, which leads to the failure of shallow fusion hot word enhancement techniques and low accuracy in speech recognition of new words and hot words, this disclosure provides a speech processing method. By smoothing the probability of recognition results output by the AM model, it increases the success rate of shallow fusion techniques, ultimately reducing the word error rate (WER) and improving the speech recognition effect after hot word enhancement. This method is suitable for smart TVs, in-vehicle voice systems, and intelligent customer service human-computer interaction scenarios.

[0021] Currently, in the field of speech recognition, mainstream speech recognition systems consist of both acoustic and language models. Among them, character-based AM models, due to their training not requiring text pinyin sequences and their superior modeling granularity, have gradually replaced traditional phoneme-based AM models and are widely used in streaming recognition scenarios such as smart TVs and in-vehicle voice systems. These AM models are trained using the CTC loss function, and their output exhibits a significant spike effect, meaning the model assigns a probability of over 99.9% to the most confident classification result, while the probability values ​​of all other output units (including characters with similar pronunciations) are extremely low, almost approaching 0. Furthermore, context-biased (hot word enhancement) techniques generally employ shallow fusion schemes, which cannot retain corresponding candidate recognition paths during beamsearch decoding. Some output units with excessively low probability values ​​are prematurely pruned, resulting in hot words not being recognized.

[0022] In real-world applications, such as smart TVs, new movies and TV shows are released every week. For example, if a new drama, "Yu Yao," is released, before it becomes widely known, the language model cannot cover the new word. If a user pronounces "Yu Yao," the AM model trained on CTC will assign the vast majority of probabilities to "Yao," which sounds similar to "Yao," while the probability of "Yao" will approach 0. Even with shallow fusion techniques in WFST (Weighted Finite-State)... Adding a hot word path for "Yu Yao" to the decoding graph of a Transducer (Weighted Finite State Transducer) can also lead to the inability to correctly identify the hot word because the candidate recognition path for "Yao" is truncated by beam search. Similarly, in a car voice scenario, when a user issues the command "Navigate to Lishuiqiao," the AM model may assign the probability of "Li" to the similarly pronounced "Ni," resulting in an extremely low probability of "Li" and thus failing to recognize the complete hot word. In another scenario, when a user mentions the hot word "Lihao Service," the low probability of "Li" can also cause hot word recognition failure, severely impacting the user experience.

[0023] To address the aforementioned technical problems, this disclosure provides an example of a speech processing method, please refer to... Figure 1 As shown, Figure 1 A schematic flowchart of a speech processing method provided in this disclosure is shown. This is by way of example and not limitation; the method can be applied to or run in an electronic device. The method includes: S101, Obtain probability information of multiple output units corresponding to the input speech frame.

[0024] S102, based on probability information, determine the first unit with the highest probability value and the second unit with the second highest probability value among multiple output units.

[0025] S103, if the first unit indicates that the input speech frame is classified as a text character, then determine the set of characters that match the text character.

[0026] S104. Based on the preset smoothing coefficient and the probability difference between the first unit and the second unit, the probability value of the first unit is allocated to the output unit corresponding to each character in the character set according to a predetermined proportion, so as to obtain the smoothed probability information.

[0027] S105, Perform speech processing on the input speech frame based on the smoothed probability information to obtain the speech processing result.

[0028] Speech processing includes at least one of decoding processing and context bias enhancement processing.

[0029] In some embodiments, first, probability information of multiple output units corresponding to an input speech frame is obtained. The probability information can be output after forward calculation of the input speech frame by an AM model based on word modeling. The AM model is trained using a CTC loss function.

[0030] In some embodiments, each output unit is used to represent the confidence that the input speech frame belongs to the corresponding character. The characters include literal characters for identifying words and special characters for identifying a silent state.

[0031] In some embodiments, the probability information of multiple output units includes a probability value corresponding to each output unit, which is the confidence that the input speech frame belongs to the corresponding literal character or special character. The higher the probability value, the higher the confidence level of the AM model for the classification result. For example, when the user pronounces "立", in the probability information of multiple output units output by the AM model, the probability value of "你" reaches 0.999, while the probability value of "立" is only 0.0001, and the probability values of the remaining literal characters similar in pronunciation to "你" or "立" also tend to 0, forming a typical spike effect.

[0032] Subsequently, according to the above probability information, the probability values of all output units are sorted to determine the first unit with the highest probability value and the second unit with the second highest probability value. It should be understood that the first unit is the recognition result with the highest current confidence level of the AM model, and the second unit is the recognition result with the second highest confidence level. The probability difference between the first unit and the second unit is used to reflect the significant degree of the spike effect, that is, the larger the probability difference, the more obvious the spike effect, and the higher the probability of speech recognition failure for hot words and the like.

[0033] As an example but not a limitation, for example, in the above recognition of the pronunciation of "立", the first unit is "你" (probability 0.999), the second unit is "我" (probability 0.0005), and the difference between the two is 0.9985. The spike effect is extremely significant, and the probability of "立" approaches 0 and cannot enter the candidate recognition path.

[0034] If it is determined that the first unit indicates that the input speech frame belongs to a literal character rather than a special character that represents a silent state, then a set of words matching the literal character is further determined. The set of words can be obtained by querying a preset replacement table (such as a homophone and near-homophone replacement table) based on the literal character. Taking the literal character as a Chinese character as an example, the preset replacement table can be constructed by traversing a preset Chinese character dictionary to obtain all literal characters as the original literal characters in the set of words, and then based on the pinyin of each original literal character and the preset similar pronunciation rules, searching for literal characters with the same or similar pronunciation as the matching literal characters.

[0035] In some embodiments, the preset similar pronunciation rules include approximate pronunciation rules corresponding to different tones, differences between flat and rolled tongues, differences between front and back nasal sounds, and easily confused initial consonants (such as n and l, z and zh, etc.), which conform to the characteristics of differences and easy confusion in users' pronunciations in actual voice interactions.

[0036] As an example rather than a limitation, for example, when the first unit corresponds to the Chinese character "你 (nǐ)", through the replacement table, Chinese characters with similar pronunciations to "你 (nǐ)" such as "倪 (ní), 拟 (nǐ), 逆 (nì), 尼 (ní), 呢 (ne), 您 (nín), 里 (lǐ), 立 (lì), 例 (lì)" can be queried to form a set of matching characters.

[0037] Then, according to the preset smoothing coefficient and the probability difference between the first unit and the second unit, a predetermined proportion of the probability value of the first unit is allocated to the output units corresponding to each Chinese character in the above set of matching characters to obtain the smoothed probability information.

[0038] Among them, the value range of the preset smoothing coefficient can be set to be greater than 0 and less than 0.5. Furthermore, it can be ensured that after adjusting the probability information, the probability value of the first unit is still the highest probability value among all output units, avoiding the problem of incorrect speech recognition results caused by the adjustment of probability information, and effectively narrowing the gap between the probability value of the first unit and the probability values of each Chinese character in the set of matching characters to weaken the peak effect.

[0039] Specifically, the probability of a predetermined proportion can be calculated based on the product of the preset smoothing coefficient and the probability difference, that is, the probability share corresponding to the product is extracted from the probability value of the first unit, and then this probability share is evenly distributed to the output units corresponding to each Chinese character in the set of matching characters, so that Chinese characters with pronunciations similar to or the same as that of the first unit and with initial probability values tending to 0 can obtain reasonably allocated probabilities.

[0040] As an example rather than a limitation, in combination with the above pronunciation recognition scenario of "立 (lì)", for example, if the preset smoothing coefficient is selected as 0.2, the probability difference between the first unit "你 (nǐ)" and the second unit "我 (wǒ)" is 0.9985, the extracted probability share is 0.2×0.9985≈0.1997, and this part of the probability is evenly distributed to the output units corresponding to 9 matching Chinese characters such as "倪 (ní), 拟 (nǐ), 立 (lì)", and each output unit can obtain an allocated probability of approximately 0.0222. After adjustment, the probability value of "你 (nǐ)" is 0.999 - 0.1997 = 0.7993 (still the highest), and the probability value of "立 (lì)" is increased from 0.0001 to 0.0223 and no longer tends to 0, which is conducive to context bias enhancement processing.

[0041] Finally, at least one of decoding processing and context bias enhancement processing is performed on the input speech frame according to the smoothed probability information to obtain a speech processing result, so as to adapt to the decoding process of the speech recognition system. During the decoding process, the beam search algorithm can be used, but is not limited to, to perform path search on the smoothed probability information. Since the text characters in the matching text set (such as "立", "遥", etc.) have effective probabilities, the key text characters of target words such as hot words in the speech frame can be effectively recognized, solving the technical problem that the key text characters of target words such as hot words are cropped in the related art; furthermore, decoding processing is performed on the input speech frame according to the smoothed probability information, and context bias enhancement processing is performed using the shallow fusion technology, and finally a speech processing result with better recognition accuracy is obtained.

[0042] Furthermore, the speech processing method provided by the embodiments of the present disclosure can effectively weaken the spike effect problem in speech recognition technology, improve speech recognition accuracy and speech recognition stability, and enhance scene applicability and user experience.

[0043] In some embodiments, as Figure 2 shown, S104, according to the preset smoothing coefficient and the probability difference between the first unit and the second unit, allocate the probability values of a predetermined proportion of the first unit to the output units corresponding to each text character in the text set to obtain smoothed probability information, including: S201, determine the predetermined proportion according to the product of the preset smoothing coefficient and the probability difference.

[0044] S202, allocate the probability values of a predetermined proportion extracted from the probability values of the first unit to the output units corresponding to each text character in the text set to obtain smoothed probability information.

[0045] Among them, the value range of the preset smoothing coefficient is greater than 0 and less than 0.5.

[0046] In some embodiments, after determining the text set corresponding to the first unit, based on the preset smoothing coefficient and the probability difference between the first unit and the second unit, calculate the probability share to be extracted from the first unit, that is, the predetermined proportion, which can also be understood as the probability allocation ratio.

[0047] In some embodiments, the value range of the preset smoothing coefficient is limited to the interval greater than 0 and less than 0.5 to standardize the probability adjustment range and achieve a balance between weakening the spike effect and ensuring speech recognition stability. The value range of this preset smoothing coefficient is a key constraint set according to the spike effect characteristics of the acoustic model and the requirements of the shallow fusion technology, which not only ensures that enough probabilities are extracted and allocated to similar-sounding characters, effectively narrowing the probability gap between the text characters of the output unit with the highest probability value and the near-sounding text characters, but also avoids the near-sounding text characters being cropped by the beam search.

[0048] By limiting the range of preset smoothing coefficients, the output unit with the highest probability value is ensured to maintain the highest probability value, preventing the speech recognition result from shifting due to excessive probability transfer. This further improves the rationality and stability of probability smoothing processing, thereby indirectly improving the stability and accuracy of speech recognition.

[0049] After calculating the predetermined proportion, a corresponding proportion of probability values ​​is extracted from the original probability values ​​of the first unit, and this portion of the probability is evenly distributed to the output units corresponding to each character in the character set, resulting in smoothed probability information. It should be understood that after the above-mentioned operations of extracting and redistributing probabilities, the probability distribution state in the original probability information, where the probability values ​​were highly concentrated in the first unit, is optimized. Characters with the same or similar pronunciation to the characters in the first unit obtain effective probabilities, meaning that the probability value of these characters no longer approaches zero.

[0050] This processing method reduces the probability gap between the characters in the first unit and similar characters, satisfying the requirements of shallow fusion technology for the output probability distribution of the acoustic model. This ensures that candidate recognition paths corresponding to key characters in target words, such as hot words, are preserved during the beam search stage without premature pruning, while maintaining the classification rationality of the acoustic model itself. In practical scenarios such as smart TVs and in-vehicle voice systems, this method can stably improve the speech recognition success rate, effectively reducing the overall word error rate by approximately 10%. In scenarios where the original acoustic model output alone cannot recognize the target word, it can achieve correct recognition of the target word, fully balancing recognition stability and target word enhancement effects.

[0051] In some embodiments, determining the set of characters that match the text characters includes: Based on a preset replacement table, determine the set of characters that match the text characters.

[0052] The preset replacement table is used to record all text characters, and text characters that have the same or similar pronunciation as each text character.

[0053] In some embodiments, the preset replacement table is generated based on the following method: Based on all the text characters obtained by traversing the preset dictionary, determine each original text character in the preset replacement table; Based on the phonetic symbols of each original character and the preset similar pronunciation rules, matching characters with the same or similar pronunciation as each original character are determined. The preset similar pronunciation rules include at least one of the following: different tones, differences between retroflex and alveolar consonants, differences between front and back nasal consonants, and easily confused initial consonants. Based on each original text character and the corresponding matching text characters associated with each original text character, a preset replacement table is obtained.

[0054] Taking Chinese characters as an example, in order to ensure that the probabilistic smoothing of speech frames can fit the real speech confusion scenario and adapt to Chinese pronunciation habits, a preset replacement table can be used to determine the set of characters that match the corresponding characters in the first unit. In some embodiments, the preset replacement table is a pronunciation association data table that is pre-built and fixed in the speech recognition system, which is used to record all the characters in the Chinese dictionary, as well as characters with the same or similar pronunciation as each character.

[0055] In some embodiments, Chinese characters are still used as an example, and the preset replacement table can be, but is not limited to, the following Table 1: Table 1

[0056] In some embodiments, the preset replacement table can be automatically constructed and generated through a standardized process: for example, firstly, a publicly available and universally used preset Chinese character dictionary is traversed to obtain all the characters in the dictionary, and then each character is used as the original character in the preset replacement table to ensure that the preset replacement table covers all characters that may appear during the speech recognition process, and avoids missing target words such as scenario-specific proper nouns, business terms and custom words during the speech recognition process.

[0057] Taking Chinese characters as an example, after determining the original characters, based on the standard pinyin phonetic symbols of each original character, and combined with preset similar pronunciation rules that are manually constructed and verified, for example, based on, but not limited to, the similar pronunciation table / rule table of similar sounds shown in Table 2 below, we can find and determine matching characters that have the same or similar pronunciation as the original characters. These similar pronunciation rules conform to the actual pronunciation confusion rules of Chinese. For example, they include, but are not limited to, differences in different tones, differences in retroflex and alveolar consonants, differences in front and back nasal consonants, and at least one similar pronunciation rule among easily confused initials such as n / l, z / zh, c / ch. They also cover issues such as user accent deviation, pronunciation deviation, and near-sound confusion in different scenarios such as in-vehicle systems, smart TVs, and smart customer service.

[0058] Table 2

[0059] After determining all matching characters corresponding to each original character, a one-to-one correspondence is established between the original characters and the matching characters to form a preset replacement table, as shown in Table 1. Each record in the preset replacement table contains an original character and a set of matching characters with the same or similar pronunciation. During speech processing, the speech processing system can quickly locate the original character entry of the corresponding character in the preset replacement table by looking up the table, extract all matching characters under that entry, and obtain a character set for probability allocation. The characters in this character set are all homophones and / or near-homophones that are easily confused with the original characters in actual speech.

[0060] By recording homophones and near-homophones for each character in a preset replacement table, and combining this with preset similar pronunciation rules (covering different tones, retroflex consonants, front and back nasal sounds, easily confused initials, etc.) based on Chinese phonetic symbols, the preset replacement table can accurately match near-homophone confusion caused by pronunciation deviations and accent differences in real voice interactions. It avoids missing key near-homophones, eliminates interference from irrelevant characters, and ensures that probability allocation only applies to characters with the same or similar pronunciation as the character with the highest probability, thus improving the targeting and effectiveness of probability smoothing.

[0061] Furthermore, the preset replacement table generated in this way possesses high coverage, high accuracy, and high scene adaptability. It ensures that homophones and near-homophones are matched without omission, allowing key characters to be included in the text set for probability allocation. It also avoids irrelevant characters interfering with the probability smoothing effect, guaranteeing the rationality of the acoustic model's output probability. In practical applications, this preset replacement table can effectively solve speech recognition failures caused by pronunciation deviations and accent differences, providing a reliable text matching foundation for probability smoothing processing. This further improves the success rate of context-biased enhancement processing based on shallow fusion technology and reduces the word error rate in speech recognition.

[0062] In some embodiments, obtaining probability information of multiple output units corresponding to the input speech frame includes: The input speech frame is fed into the text-based acoustic model, which is trained using a connection-temporal classification loss function.

[0063] It receives probability information from multiple output units of the acoustic model in response to the input speech frame.

[0064] The number of output units is equal to the total number of all characters in the preset dictionary plus one. The additional output unit corresponds to a special character, and the remaining output units correspond to each character in the preset dictionary.

[0065] In this embodiment, the above probability information can be obtained based on a hybrid model speech recognition architecture, but is not limited to. It can be adapted to an acoustic model (AM model) trained using text modeling and connection-time classification (CTC) loss function, and can be adapted and compatible with streaming speech recognition scenarios such as smart TVs, in-vehicle voice, and intelligent customer service.

[0066] As an example, and not a limitation, the input speech frames, segmented into fixed durations, can first be fed into a pre-trained acoustic model based on text modeling (AM model). This AM model is a deep neural network structure that uses a connection-time classification loss function for training. Compared to traditional acoustic models based on initials and finals, this AM model does not require labeling the training data with pinyin sequences, offering greater modeling granularity and a simpler training process. It is a mainstream acoustic model structure that is gradually becoming more prevalent in the field of Chinese speech recognition.

[0067] In some embodiments, after performing forward computation on the input speech frame, the AM model outputs probability information for multiple output units corresponding to that speech frame. The number of output units is the total number of all characters in a preset dictionary plus one; the additional output unit corresponds to a special character, and the remaining output units correspond to various characters in the preset dictionary.

[0068] It should be noted that the special character used to identify the silence state can also be a blank character, which is used to indicate that the input speech frame is a silent segment that does not contain a valid text character. All other output units correspond one-to-one with each text character in the preset dictionary. The probability value of each output unit directly represents the confidence level of classifying the input speech frame into the corresponding text character or blank character.

[0069] Since the AM model is trained using the CTC loss function, the original output probability of the AM model exhibits a typical spike effect, that is, the probability value of the output unit corresponding to a single character accounts for more than 99.9% of the probability distribution, while the probability values ​​of other characters and special characters are extremely low and close to zero.

[0070] The probability information obtained in this embodiment is directly used as the original input for probability smoothing. This reduces the spike effect, improves the confidence of near-phonetic characters, and ensures that the acoustic model output format is fully compatible with the WFST decoding process and the context bias enhancement processing process based on shallow fusion technology. No additional development or modification is required for the relevant speech recognition system architecture. In other words, by integrating the probability smoothing processing method provided in this embodiment, the speech recognition effect can be improved.

[0071] In some embodiments, such as Figure 3 As shown, speech processing is performed on the input speech frame based on the smoothed probability information to obtain the speech processing results, including: S301 decodes the smoothed probability information to obtain the speech decoding information.

[0072] The speech decoding information includes at least the candidate recognition path corresponding to each output unit of the text set. The candidate recognition path is used to characterize the text sequence link containing the text characters in the text set, which is constructed based on the smoothed probability information.

[0073] S302, based on the preset target word set, adds the target word recognition path corresponding to the target word to the speech decoding information to obtain the corrected speech decoding information.

[0074] Among them, the target word recognition path is used to represent the text sequence link formed by sequentially associating the various text characters contained in the target word.

[0075] S303: After adjusting the priority of the target word recognition path in the corrected speech decoding information, the optimally ranked recognition path is selected as the speech processing result.

[0076] Among them, adjusting the priority of the target word recognition path is used to make the target word recognition path rank better among all candidate recognition paths.

[0077] In this embodiment, the process of performing speech processing based on the smoothed probability information can combine WFST decoding of related hybrid model speech recognition with context bias enhancement processing logic based on shallow fusion technology, making full use of the homophones and near-homophones retained after probability smoothing to achieve effective and accurate recognition of target words.

[0078] In some embodiments, target words refer to the set of words for which recognition priority and accuracy need to be improved in specific voice interaction scenarios, including but not limited to: user-defined words, business-specific words, proper nouns, movie and TV show titles, place names, personal names, brand names, and high-frequency new words. These words can be added to the decoding process through contextual bias enhancement or shallow fusion to improve the problems of missed or false recognition caused by acoustic model recognition bias.

[0079] In some embodiments, the speech processing system first decodes the smoothed probability information obtained in the above embodiments. For example, a beam search algorithm can be used to search for paths in the WFST decoding graph. Since the smoothing process weakens the spike effect of the acoustic model, the output units corresponding to each homophone or near-homophone in the text set all have effective probabilities. Therefore, during the decoding process, the candidate recognition paths corresponding to these homophones and near-homophones will not be prematurely pruned due to low probability values, and can be completely preserved in the speech decoding information, avoiding the situation where the target word cannot be matched due to the absence of key near-homophones.

[0080] In some embodiments, since the speech decoding information includes at least the candidate recognition path corresponding to each output unit of the text set, the candidate recognition path is used to characterize the text sequence link containing the text characters in the text set, which is constructed based on the smoothed probability information.

[0081] After obtaining the speech decoding information, which includes candidate recognition paths for each output unit corresponding to the text set, the speech processing system, based on contextual bias requirements (e.g., target word enhancement), adds a dedicated path corresponding to the target word, i.e., a target word recognition path, to the basic decoding path of the speech decoding information, thereby obtaining the corrected speech decoding information. For example, this target word recognition path can be embedded into the original WFST decoding graph using shallow fusion technology, and together with the aforementioned candidate recognition paths, constitutes a complete set of candidate recognition paths (i.e., all candidate recognition paths).

[0082] Subsequently, the speech processing system prioritizes the target word recognition paths in the corrected decoded information. This can be achieved by increasing the score weight of the target word recognition path or decreasing its path cost, thus improving its ranking within the candidate recognition path set. Consequently, the speech processing system can preferentially select the target word recognition path with higher priority and better overall score as the output, thereby obtaining the final speech processing result.

[0083] Through the above processing flow, the candidate recognition path retained after smoothing the probability information and the target word recognition path work together effectively. This not only ensures the accuracy of speech recognition, but also enables stable output of correct recognition results in target word scenarios such as new words, movie titles, and proper nouns. This significantly improves the effectiveness of shallow fusion technology, reduces the speech recognition word error rate by about 10%, and still achieves correct recognition in target word scenarios that rely on near-phonetic characters.

[0084] In some embodiments, such as Figure 4 As shown, in some embodiments, after determining the first unit with the highest probability value and the second unit with the second highest probability value among multiple output units based on probability information, the method further includes: S401, if the first unit indicates that the input speech frame is classified as a special character, then the probability information is used as the smoothed probability information.

[0085] In this embodiment, in order to optimize the speech recognition process, avoid invalid probability adjustments and ensure system operating efficiency, after determining the first unit with the highest probability value and the second unit with the second highest probability value, the speech processing system can first judge the state corresponding to the first unit.

[0086] If it is determined through judgment that the first unit with the highest probability value indicates that the input speech frame belongs to a special character (i.e., a blank character, a silence frame), that is, the acoustic model determines that the current input speech frame is most likely a silence segment without valid semantics, then no smoothing operations such as extraction and assignment of the original probability information need to be performed. Instead, the original probability information output by the acoustic model can be directly used as the smoothed probability information and continue to enter subsequent speech processing links such as decoding and enhancement processing.

[0087] This processing method fully conforms to the actual distribution characteristics of speech. Silence frames do not participate in the composition of the text sequence, and there is no need for near-sound text association related to the target word. Performing probability smoothing on silence frames not only has no practical significance but also increases the system calculation overhead. By directly retaining the original probability information, the processing flow is simplified, the inference speed of speech recognition is improved, the context bias and target word enhancement effects are not affected, and the stability of silence frame recognition is ensured, making the probability smoothing processing have reasonable processing logic for both valid text frames and silence frames, and better conforming to the engineering implementation requirements of streaming speech recognition.

[0088] This embodiment combines the Figure 5 network structure of the acoustic model shown and the probability smoothing effect to detail the specific implementation process of the present disclosure scheme in single-frame speech processing. <null>

[0089] As Figure 5 shown, the input layer receives a segment of speech signal from the user (the waveform is not limited to Figure 5 shown), and the speech signal can be segmented into single-frame speech according to a preset fixed duration, that is, the input speech frame, and the input speech frame is sent to the hidden layer (Layer1~LayerN) for feature extraction and modeling processing, and finally the probability information of multiple output units corresponding to this frame of speech is output by the output layer.

[0090] Still as Figure 5 shown, after the frame of speech is calculated by the acoustic model based on CTC-trained character modeling, the original output probability information presents a typical spike effect. For example: the probability value corresponding to the text character "你" is 0.999, which is the first unit with the highest probability value; the probability values of pronunciation-similar text characters such as "您", "李", "里", "逆", "立" are all 0.0001, which are sub-optimal units with extremely low probability values; the probability values of the remaining text characters and special characters approach 。Therefore, if directly entering the decoding process, pronunciation-similar text characters such as "立" will be prematurely clipped in the beam search stage due to their too low probability and cannot provide an effective candidate recognition path for the context bias enhancement processing based on the shallow fusion technology.

[0091] The present disclosure scheme adjusts the above original probability information through a probability smoothing unit: First, determine that the first unit with the highest probability value is "你" (corresponding text), and the second unit with the second highest probability value is "您" (probability 0.0001). Calculate the probability difference between the two as Δ = 0.999 - 0.0001 = 0.9989.

[0092] Secondly, based on a preset replacement table (such as a homophone and near-homophone replacement table), query the set of matching texts corresponding to "你". This set of texts may but is not limited to include 9 text characters with similar pronunciations to "你", such as "您, 李, 里, 逆, 立", etc.

[0093] After that, if the preset smoothing coefficient α = 0.2 (satisfying 0 < α < 0.5) is selected, calculate the probability share to be extracted as α×Δ = 0.2×0.9989 ≈ 0.1998, and evenly distribute this share to the output units corresponding to the 9 text characters. The assigned probability of each output unit is approximately 0.0222.

[0094] After that, the probability value of the first unit "你" is updated to 0.999 - 0.1998 = 0.7992 (still the highest), and the probability values of matching text characters such as "您", "李", "里", "逆", "立" are increased to approximately 0.0222 - 0.0223, and the probability values of the remaining text characters and special characters remain unchanged, obtaining the smoothed probability information.

[0095] Finally, perform speech processing on the input speech frame based on this smoothed probability information: For example, in the beam search decoding stage, text characters with similar pronunciations to "你" such as "立" are retained due to their effective probabilities, and their corresponding candidate recognition paths are not prematurely pruned; subsequently, the recognition paths of target words such as "立" are added to the WFST decoding graph through the shallow fusion technology, and the priority of this recognition path among all candidate recognition paths is enhanced. Finally, the candidate recognition path containing the target word can be preferentially output, achieving enhanced speech recognition.

[0096] Furthermore, based on the above specific implementation manners, through probability smoothing processing, the spike effect existing in the acoustic model can be effectively weakened, the probability gap between the text character with the highest probability and text characters with similar pronunciations can be narrowed, providing a premise for the context bias enhancement processing based on the shallow fusion technology, significantly improving the speech recognition success rate, and ensuring the stability of the speech recognition result.

[0097] Based on the demonstrations of the above various embodiments, the embodiments of the present disclosure have the following obvious advantages compared with the related technologies: First, it effectively weakens the spike effect of the character modeling AM model trained based on CTC, narrows the probability gap between the first unit and text characters with similar pronunciations, ensures that the key text characters of target words with similar pronunciations can be retained in the candidate recognition paths, and provides a processing premise for the context bias enhancement processing based on the shallow fusion technology; Secondly, it significantly improves the accuracy of speech recognition in context-biased (target word enhancement) scenarios. According to actual tests, compared with not using this solution, the overall word error rate (WER) can be reduced by about 10%. In speech recognition scenarios such as new film and television drama titles and proper nouns, only by using this solution can correct recognition be achieved. This solves the technical defects of related shallow fusion technologies that rely on the original output of the AM model and have limited applicability. Third, it does not affect the accuracy of speech recognition. Due to the limited range of the preset smoothing coefficient, the probability value of the first unit remains the highest after adjustment, ensuring the stability of the speech recognition results and adapting to various human-computer interaction scenarios such as smart TVs, in-vehicle voice systems, and intelligent customer service. Fourth, it has strong scene adaptability. The specific value of the preset smoothing coefficient, the range of matching text characters in the preset replacement table, and the preset similar pronunciation rules can all be flexibly adjusted according to different application scenarios so as to adapt to the target word enhancement needs of different fields. Fifth, there is no need to modify or change the model architecture of the AM model itself. Only a probability smoothing unit is added to the output layer of the AM model to realize the speech processing method provided in this embodiment. It is not only low-cost and highly compatible, but can also be quickly integrated into relevant speech recognition systems without large-scale modification of related hardware or software architecture.

[0098] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0099] Corresponding to the speech processing method in the above embodiments, Figure 6 This is a schematic diagram of the structure of a voice processing system provided in an embodiment of this disclosure. The system can be implemented as part or all of a computer device by software, hardware, or a combination of both. This computer device can be... Figure 7 The electronic device shown.

[0100] Reference Figure 6 The voice processing system includes: The acquisition unit 601 is configured to acquire probability information of multiple output units corresponding to the input speech frame, wherein each output unit is used to characterize the confidence level of the input speech frame being classified into the corresponding character, and the character includes text characters used to identify text and special characters used to identify the silence state.

[0101] The probability smoothing unit 602 is configured to determine the first unit with the highest probability value and the second unit with the second highest probability value among multiple output units based on probability information; if the first unit indicates that the input speech frame is classified as a text character, then determine the set of text characters that match the text characters; according to the preset smoothing coefficient and the probability difference between the first unit and the second unit, allocate a predetermined proportion of the probability value of the first unit to the output units corresponding to each text character in the text set to obtain smoothed probability information; The speech processing unit 603 is configured to perform speech processing on the input speech frame based on smoothed probability information to obtain a speech processing result. The speech processing includes at least one of decoding processing and context bias enhancement processing.

[0102] This embodiment provides a speech recognition system that can be applied to streaming recognition and target word enhancement scenarios such as smart TVs, in-vehicle voice, and intelligent customer service. By introducing a probability smoothing mechanism at the output of the acoustic model, the system alleviates the spike effect of the word modeling acoustic model (AM model) trained by connection-time classification (CTC), improves the effectiveness probability of shallow fusion context bias technology, and ultimately reduces the word error rate (WER) of speech recognition.

[0103] In some embodiments, the probability smoothing unit is specifically configured to perform: A predetermined ratio is determined by multiplying the preset smoothing coefficient by the probability difference, wherein the preset smoothing coefficient ranges from greater than 0 to less than 0.5. A predetermined proportion of probability values ​​will be extracted from the probability values ​​of the first unit and distributed to the output units corresponding to each character in the character set to obtain smoothed probability information.

[0104] In some embodiments, the probability smoothing unit is specifically configured to perform: Based on a preset replacement table, a set of characters that match the text characters is determined. The preset replacement table is used to record all text characters and text characters that have the same or similar pronunciation as each text character. The preset replacement table is generated based on the following method: Based on all the text characters obtained by traversing the preset dictionary, determine each original text character in the preset replacement table; Based on the phonetic symbols of each original character and the preset similar pronunciation rules, matching characters with the same or similar pronunciation as each original character are determined. The preset similar pronunciation rules include at least one of the following: different tones, differences between retroflex and alveolar consonants, differences between front and back nasal consonants, and easily confused initial consonants. Based on each original text character and the corresponding matching text characters associated with each original text character, a preset replacement table is obtained.

[0105] In some embodiments, the speech processing unit is further configured to perform: The smoothed probability information is decoded to obtain speech decoding information, which includes at least the candidate recognition path corresponding to each output unit of the text set. The candidate recognition path is used to characterize the text sequence link containing the text characters in the text set, which is constructed based on the smoothed probability information. Based on a preset set of target words, a target word recognition path corresponding to the target word is added to the speech decoding information to obtain the corrected speech decoding information. The target word recognition path is used to represent the text sequence link formed by the sequential association of each character contained in the target word. After adjusting the priority of the target word recognition path in the corrected speech decoding information, the optimal recognition path is selected as the speech processing result. The adjustment of the priority of the target word recognition path is used to make the target word recognition path rank better among all candidate recognition paths.

[0106] It is understood that the speech processing system embodiments and any implementation methods correspond to the speech processing method embodiments and any implementation methods, respectively. The technical effects corresponding to the speech processing system embodiments and any implementation methods can be found in the aforementioned speech processing method embodiments and any implementation methods, and will not be repeated here.

[0107] It should be noted that the voice processing system provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0108] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0109] It should be noted that the information interaction and execution process between the above systems / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0110] This application also provides an electronic device, which includes one or more processors and a memory; The memory is coupled to one or more processors. The memory is used to store computer program code, which includes computer instructions. One or more processors invoke the computer instructions to cause the electronic device to perform the aforementioned speech processing method.

[0111] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 700 can be a mobile phone, smart screen, tablet computer, wearable electronic device, in-vehicle electronic device, augmented reality (AR) device, virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), projector, or a communication device such as a server, storage device, or base station, or a smart car, etc. This application embodiment does not impose any limitations on the specific type of electronic device.

[0112] The memory 701 can be used to store computer software programs 702 and modules. The processor 703 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 701. The memory 701 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, telephone directory, etc.). In addition, the memory 701 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0113] The processor 703 may include one or more processors such as a central processing unit (CPU), an application processor (AP), and a baseband processor. The processor can serve as the nerve center and command center of the wireless router. The processor 703 can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The memory 701 can be used to store executable program code, including instructions. The processor 703 executes various functional applications and data processing of the network device by running the instructions stored in the memory. The memory 701 may include a program storage area and a data storage area, such as storing data for audio signals to be played. For example, the memory may be Double Data Rate Synchronous Dynamic Random Access Memory (DDR) or Flash memory.

[0114] This application also provides a computer-readable storage medium storing computer instructions; when the computer-readable storage medium is used on an electronic device, it causes the electronic device to execute the aforementioned voice processing method.

[0115] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or can include one or more data storage devices such as servers or data centers that can be integrated with media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media, or semiconductor media (e.g., solid-state disks (SSDs)).

[0116] This application also provides a computer program product containing computer instructions, which, when run on an electronic device, enables the electronic device to execute the aforementioned voice processing method.

[0117] The computer storage medium and computer program product provided in the embodiments of this application are used to execute the methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects corresponding to the methods provided above, and will not be repeated here.

[0118] In the above embodiments, implementation can also be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc., and the storage medium can also include combinations of the above types of memory.

[0119] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments claimed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A speech processing method, characterized in that, include: Obtain probability information of multiple output units corresponding to the input speech frame, wherein each output unit is used to characterize the confidence level of the input speech frame being classified into the corresponding character, and the character includes text characters used to identify text and special characters used to identify the silence state; Based on the probability information, determine the first unit with the highest probability value and the second unit with the second highest probability value among the multiple output units; If the first unit indicates that the input speech frame is classified as the text character, then the set of characters that match the text character is determined; Based on the preset smoothing coefficient and the probability difference between the first unit and the second unit, a predetermined proportion of the probability value of the first unit is allocated to the output unit corresponding to each character in the text set to obtain smoothed probability information. Speech processing is performed on the input speech frame based on the smoothed probability information to obtain the speech processing result. The speech processing includes at least one of decoding processing and context bias enhancement processing.

2. The method according to claim 1, characterized in that, The step of allocating a predetermined proportion of the probability value of the first unit to the output unit corresponding to each character in the text set according to a preset smoothing coefficient and the probability difference between the first unit and the second unit, to obtain smoothed probability information, includes: The predetermined ratio is determined by multiplying the preset smoothing coefficient by the probability difference, wherein the preset smoothing coefficient is greater than 0 and less than 0.

5. A predetermined proportion of probability values ​​are extracted from the probability values ​​of the first unit and distributed to the output units corresponding to each character in the character set to obtain the smoothed probability information.

3. The method according to claim 1, characterized in that, The determination of the set of characters that match the text characters includes: Based on a preset replacement table, a set of characters that match the given characters is determined, wherein the preset replacement table is used to record all characters and characters that have the same or similar pronunciation as each given character; The preset replacement table is generated based on the following method: Based on all the text characters obtained by traversing the preset dictionary, determine each original text character in the preset replacement table; Based on the phonetic symbols of each of the original characters and the preset similar pronunciation rules, matching characters with the same or similar pronunciation as each of the original characters are determined. The preset similar pronunciation rules include at least one of the following: different tones, differences between retroflex and alveolar consonants, differences between front and back nasal consonants, and easily confused initial consonants. The preset replacement table is obtained based on each of the original text characters and the corresponding matching text characters associated with each of the original text characters.

4. The method according to claim 1, characterized in that, The step of obtaining probability information of multiple output units corresponding to the input speech frame includes: The input speech frame is input into a text-based acoustic model, wherein the acoustic model is trained using a connection-temporal classification loss function; Receive probability information of multiple output units output by the acoustic model for the input speech frame; The number of output units is equal to the total number of all characters in the preset dictionary plus one. Each additional output unit corresponds to a special character, and the remaining output units correspond to each character in the preset dictionary.

5. The method according to any one of claims 1 to 4, characterized in that, The step of performing speech processing on the input speech frame based on the smoothed probability information to obtain the speech processing result includes: The smoothed probability information is decoded to obtain speech decoding information, wherein the speech decoding information includes at least the candidate recognition path corresponding to each output unit of the text set, and the candidate recognition path is used to characterize the text sequence link containing the text characters in the text set, which is constructed based on the smoothed probability information. Based on a preset set of target words, a target word recognition path corresponding to the target word is added to the speech decoding information to obtain the corrected speech decoding information. The target word recognition path is used to characterize the text sequence link formed by sequentially associating the text characters contained in the target word. After adjusting the priority of the target word recognition path in the corrected speech decoding information, the optimally ranked recognition path is selected as the speech processing result. The adjustment of the priority of the target word recognition path is used to make the target word recognition path rank better among all candidate recognition paths.

6. The method according to any one of claims 1 to 4, characterized in that, After determining the first unit with the highest probability value and the second unit with the second highest probability value among the plurality of output units based on the probability information, the method further includes: If the first unit indicates that the input speech frame is classified as the special character, then the probability information is used as the smoothed probability information.

7. A speech recognition system, characterized in that, include: The acquisition unit is configured to acquire probability information of multiple output units corresponding to the input speech frame, wherein each output unit is used to characterize the confidence level of the input speech frame being classified into a corresponding character, and the character includes text characters used to identify text and special characters used to identify a silence state; A probability smoothing unit is configured to determine, based on the probability information, a first unit with the highest probability value and a second unit with the second highest probability value among a plurality of output units; if the first unit indicates that the input speech frame is classified as a text character, then a set of text characters matching the text character is determined; based on a preset smoothing coefficient and the probability difference between the first unit and the second unit, a predetermined proportion of the probability value of the first unit is allocated to the output units corresponding to each text character in the text set to obtain smoothed probability information; The speech processing unit is configured to perform speech processing on the input speech frame based on the smoothed probability information to obtain a speech processing result, wherein the speech processing includes at least one of decoding processing and context bias enhancement processing.

8. The system according to claim 7, characterized in that, The probability smoothing unit is specifically configured to perform: The predetermined ratio is determined by multiplying the preset smoothing coefficient by the probability difference, wherein the preset smoothing coefficient is greater than 0 and less than 0.

5. A predetermined proportion of probability values ​​are extracted from the probability values ​​of the first unit and distributed to the output units corresponding to each character in the character set to obtain the smoothed probability information.

9. The system according to claim 7, characterized in that, The probability smoothing unit is further configured to perform: Based on a preset replacement table, a set of characters that match the given characters is determined, wherein the preset replacement table is used to record all characters and characters that have the same or similar pronunciation as each given character; The preset replacement table is generated based on the following method: Based on all the text characters obtained by traversing the preset dictionary, determine each original text character in the preset replacement table; Based on the phonetic symbols of each of the original characters and the preset similar pronunciation rules, matching characters with the same or similar pronunciation as each of the original characters are determined. The preset similar pronunciation rules include at least one of the following: different tones, differences between retroflex and alveolar consonants, differences between front and back nasal consonants, and easily confused initial consonants. The preset replacement table is obtained based on each of the original text characters and the corresponding matching text characters associated with each of the original text characters.

10. The system according to any one of claims 7 to 9, characterized in that, The speech processing unit is further configured to perform: The smoothed probability information is decoded to obtain speech decoding information, wherein the speech decoding information includes at least the candidate recognition path corresponding to each output unit of the text set, and the candidate recognition path is used to characterize the text sequence link containing the text characters in the text set, which is constructed based on the smoothed probability information. Based on a preset set of target words, a target word recognition path corresponding to the target word is added to the speech decoding information to obtain the corrected speech decoding information. The target word recognition path is used to characterize the text sequence link formed by sequentially associating the text characters contained in the target word. After adjusting the priority of the target word recognition path in the corrected speech decoding information, the optimally ranked recognition path is selected as the speech processing result. The adjustment of the priority of the target word recognition path is used to make the target word recognition path rank better among all candidate recognition paths.

Images