Speech recognition method and electronic equipment

Abstract

The invention provides a voice recognition method and electronic equipment, and the method comprises the steps: carrying out the recognition of a to-be-recognized voice segment in to-be-recognized voice information through a voice recognition model, and obtaining a first recognition result, and the to-be-recognized voice information comprises a plurality of voice segments; using a text processing model to determine a second recognition result according to the first recognition result, both the first recognition result and the second recognition result representing text features of the to-be-recognized speech segment, and the first recognition result and the second recognition result being different; using a text processing model to determine target text information of the to-be-recognized voice segment according to the first recognition result and the second recognition result; and determining the text information of the to-be-recognized voice information according to the target text information of each voice segment in the to-be-recognized voice information.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically, to a speech recognition method and electronic device. Background Art

[0002] End-to-End Automatic Speech Recognition (E2E ASR) is a deep learning model that directly maps speech input to text output, eliminating the complex multi-stage processing required by traditional speech recognition systems. ASR has lower recognition accuracy when dealing with complex contexts or domain-specific content. Summary of the Invention

[0003] The present disclosure provides a speech recognition method and an electronic device.

[0004] According to one aspect of the present disclosure, a speech recognition method is provided, including: using a speech recognition model to recognize a speech segment to be recognized in speech information to obtain a first recognition result, where the speech information to be recognized includes multiple speech segments; using a text processing model to determine a second recognition result based on the first recognition result, where the first recognition result and the second recognition result both represent text features of the speech segment to be recognized, and the first recognition result and the second recognition result are different; using the text processing model to determine target text information of the speech segment to be recognized based on the first recognition result and the second recognition result; and determining text information of the speech information to be recognized based on the target text information of each speech segment in the speech information to be recognized.

[0005] According to an embodiment of the present disclosure, determining a second recognition result based on the first recognition result includes: determining the second recognition result based on the first recognition result and sliding window information; wherein the sliding window information includes target text information of other voice segments located before the voice segment to be recognized, and the length of the sliding window represents the amount of target text information in the sliding window information.

[0006] According to an embodiment of the present disclosure, a second recognition result is determined based on the first recognition result and the sliding window information, including: determining an initial second recognition result based on the first recognition result and the sliding window information; the initial second recognition result includes initial text information and an initial confidence level; in response to the initial confidence level being less than a preset confidence level, increasing the length of the sliding window to obtain updated sliding window information; and determining the second recognition result based on the first recognition result and the updated sliding window information.

[0007] According to an embodiment of the present disclosure, the first recognition result includes first text information and a first confidence level of the first text information, and the second recognition result includes second text information and a second confidence level of the second text information; based on the first recognition result and the second recognition result, determining the target text information of the voice segment to be recognized includes: based on the first confidence level and the second confidence level, determining the target text information of the voice segment to be recognized from the first text information and the second text information.

[0008] According to an embodiment of the present disclosure, a text processing model is used to determine a target knowledge base based on target text information of other voice segments that precede the voice segment to be recognized; based on the target knowledge base, a first knowledge base matching degree and a second knowledge base matching degree are determined according to the first recognition result and the second recognition result, respectively; according to the first confidence level and the second confidence level, the target text information of the voice segment to be recognized is determined from the first text information and the first text information, including: determining the target text information of the voice segment to be recognized from the first text information and the second text information according to the first confidence level, the second confidence level, the first knowledge base matching degree and the second knowledge base matching degree.

[0009] According to an embodiment of the present disclosure, a text processing model is used to determine the language features of target text information; and an acoustic model of a speech recognition model is adjusted according to the language features.

[0010] According to an embodiment of the present disclosure, the text processing model includes a search algorithm, which determines the target text information of the voice segment to be recognized based on the first recognition result and the second recognition result. The method also includes: adjusting the parameters of the search algorithm based on the audio features of the voice information to be recognized.

[0011] According to an embodiment of the present disclosure, the parameters of the search algorithm are adjusted according to the audio features of the speech information to be recognized, including: adjusting the search width or confidence threshold of the search algorithm according to the audio features; the search width represents the number of candidate paths retained by the search algorithm during the decoding process.

[0012] According to an embodiment of the present disclosure, a text processing model is used to update target text information of other speech segments before the speech segment to be recognized based on target text information of the speech segment to be recognized.

[0013] Another aspect of the present disclosure provides an electronic device, comprising: a memory for storing voice information to be recognized; a processor, communicatively connected to the memory, for using a voice recognition model to recognize voice segments to be recognized in the voice information to be recognized, and obtaining a first recognition result, wherein the voice information to be recognized includes multiple voice segments; using a text processing model to determine a second recognition result based on the first recognition result, wherein the first recognition result and the second recognition result both represent text features of the voice segment to be recognized, and the first recognition result and the second recognition result are different; using the text processing model to determine target text information of the voice segment to be recognized based on the first recognition result and the second recognition result; and determining text information of the voice information to be recognized based on the target text information of each voice segment in the voice information to be recognized.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0016] Figure 1 is a flow chart of a speech recognition method according to an embodiment of the present disclosure;

[0017] Figure 2 is a schematic diagram of a sliding window according to an embodiment of the present disclosure;

[0018] Figure 3 is a schematic diagram of a sliding window according to another embodiment of the present disclosure;

[0019] Figure 4 is a schematic diagram of a process for recognizing speech information to be recognized according to an embodiment of the present disclosure; and

[0020] Figure 5 is a schematic block diagram of an example electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application of the data involved (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0023] Figure 1 is a flow chart of a speech recognition method according to an embodiment of the present disclosure, such as Figure 1 As shown, the speech recognition method of this embodiment includes operations S110-S140.

[0024] In operation S110, a speech segment to be recognized in the speech information to be recognized is recognized using a speech recognition model to obtain a first recognition result. The speech information to be recognized includes a plurality of speech segments.

[0025] In the disclosed embodiments, the speech recognition model can be an end-to-end speech recognition (ASR) model. ASR uses an algorithmic model to understand the content of speech information and output corresponding text results. The speech information to be recognized refers to raw speech input containing complete semantics. It can be a continuous recording (such as a meeting transcript) or a collection of discrete speech segments (such as multiple voice commands from a smart device).

[0026] The speech information to be recognized includes multiple speech segments, and the speech segments can be obtained by segmenting the speech information to be recognized through time windows.

[0027] For example, continuous speech information to be recognized is divided into speech segments of equal length according to a fixed duration (such as 2 seconds), without considering semantic integrity.

[0028] The speech segments can be obtained by segmenting the speech information to be recognized through semantic boundary detection.

[0029] For example, the voice information to be recognized can be segmented by detecting silence intervals, setting the minimum voice segment length to 300ms, setting continuous silence greater than 500ms as a semantic boundary, and segmenting the voice information to be recognized into multiple voice segments based on the semantic boundary.

[0030] For example, voice activity detection (VAD) can be used to locate the start and end points of a speech segment, and the speech information to be recognized can be segmented into multiple speech segments according to semantic units.

[0031] In the disclosed embodiment, a speech recognition model is used to recognize a speech segment in speech information to be recognized, thereby obtaining a first recognition result. The speech recognition model may convert the speech segment into text information, and the first recognition result may represent text features of the speech segment. For example, the first recognition result may include the text information "sunny day".

[0032] In operation S120 , a second recognition result is determined based on the first recognition result using a text processing model. The first recognition result and the second recognition result both represent text features of the speech segment to be recognized, and the first recognition result and the second recognition result are different.

[0033] In the disclosed embodiment, the first recognition result and the second recognition result both represent text features of the speech segment to be recognized, and the first recognition result and the second recognition result are different. The first recognition result and the second recognition result both may include text information, and the text processing model may generate text information of the second recognition result based on the text information of the first recognition result.

[0034] The text processing model can be a Large Language Model (LLM). For example, it can be a Transformer-based LLM (such as GPT-4 or PaLM). The text processing model can use a bidirectional attention mechanism to semantically reconstruct the first recognition result to obtain a second recognition result.

[0035] In the disclosed embodiments, at least one second recognition result can be determined based on a first recognition result. For example, if a speech recognition model recognizes a speech segment and obtains a first recognition result of "meeting," a text processing model can generate multiple second recognition results based on "meeting," such as "recollection," "understanding," and "regret."

[0036] In operation S130 , target text information of the to-be-recognized speech segment is determined based on the first recognition result and the second recognition result using the text processing model.

[0037] In an embodiment of the present disclosure, the text processing model can determine the target text information based on the first and second recognition results using a search algorithm. The search algorithm can be a greedy search algorithm, a Viterbi algorithm, a beam search algorithm, or the like. The search algorithm can be embedded within the text processing model, sharing memory with the text processing model's inference process to reduce data handling overhead. The search algorithm can also be located within an external search model, with the text processing model invoking the search algorithm within the search model via an interface.

[0038] The text processing model can re-understand and infer the first recognition result and the second recognition result through a search algorithm, and finally select the result with the highest probability as the target text information of the voice segment to be recognized.

[0039] For example, the first recognition result is "Qingtian", and the second recognition result includes "Sunny Day" and "Qingtian". After re-understanding and reasoning, the text processing model selects "Sunny Day" which best fits the context as the target text information.

[0040] In operation S140 , text information of the speech information to be recognized is determined according to target text information of each speech segment in the speech information to be recognized.

[0041] In an embodiment of the present disclosure, after determining the target text information of the speech segment to be recognized, speech recognition can be performed on other speech segments in the semantic information to be recognized to obtain the target text information corresponding to each speech segment, and the text information of the speech information to be recognized can be determined based on multiple target text information.

[0042] In an embodiment of the present disclosure, the voice information to be recognized includes multiple voice segments, and the multiple voice segments can be arranged in chronological order. During the voice recognition process, the multiple voice segments in the voice information to be recognized are recognized in chronological order. For the voice segment before the voice segment to be recognized, its corresponding target text information has been determined. After determining the target text information of the voice segment to be recognized, voice recognition can be performed on the voice segment after the voice segment to be recognized based on the same recognition method to obtain the corresponding multiple target text information. After obtaining the multiple target text information corresponding to the multiple voice segments of the voice information to be recognized, the multiple target text information can be combined according to the chronological order to obtain the text information of the voice information to be recognized.

[0043] Through the disclosed embodiments, a speech recognition model is combined with a text processing model, and the text processing model is used to generate a second recognition result based on the first recognition result output by the speech recognition model. The text processing model can, with its powerful contextual analysis capabilities, deeply re-understand and reason about the input result, thereby improving the accuracy of the obtained second recognition result. The text processing model is used to determine the target text information of the speech segment to be recognized based on the first and second recognition results. The text processing model determines the target text information based on the optimal recognition result, thereby improving the accuracy of the text information of the speech information to be recognized.

[0044] In some embodiments of the present disclosure, determining a second recognition result based on the first recognition result includes: determining the second recognition result based on the first recognition result and sliding window information; wherein the sliding window information includes target text information of other voice segments located before the voice segment to be recognized, and the length of the sliding window represents the amount of target text information in the sliding window information.

[0045] Figure 2 is a schematic diagram of a sliding window according to an embodiment of the present disclosure.

[0046] like Figure 2 As shown, the length of the sliding window represents the amount of target text information in the sliding window information. Figure 2The sliding window in includes 3 target text information. The sliding window information corresponding to the sliding window includes the target text information of 3 other voice segments before the voice segment to be recognized. Figure 2 The first recognition result in is the first recognition result obtained by using the speech recognition model to recognize the speech segment to be recognized.

[0047] In the embodiment of the present disclosure, the text processing model can be used to determine the second recognition result based on the first recognition result and the sliding window information. Figure 2 In the embodiment, two second recognition results can be determined based on the first recognition result and the sliding window information.

[0048] In an embodiment of the present disclosure, the sliding window information includes target text information of other voice segments located before the voice segment to be recognized, and the sliding window information includes the contextual information of the voice segment to be recognized. Using the text processing model, based on the contextual information, the text processing model can further determine the second recognition result in combination with the context of the upper-level information.

[0049] For example, the first recognition result is "full force requirement", and the sliding window information contains "patent application". By using the text processing model to recognize the above context, the second recognition result can be determined to be "claim" based on the first recognition result and the sliding window information.

[0050] Through the embodiments of the present disclosure, the sliding window information can represent the context information preceding the speech segment to be recognized. Using the text processing model, the accuracy of the second recognition result can be improved based on the first recognition result and the sliding window information.

[0051] In some embodiments of the present disclosure, a second recognition result is determined based on the first recognition result and the sliding window information, including: determining an initial second recognition result based on the first recognition result and the sliding window information; the initial second recognition result includes initial text information and an initial confidence level; in response to the initial confidence level being less than a preset confidence level, increasing the length of the sliding window to obtain updated sliding window information; and determining the second recognition result based on the first recognition result and the updated sliding window information.

[0052] In the embodiment of the present disclosure, the initial second recognition result includes initial text information and initial confidence. When the initial confidence is less than the preset confidence, it means that the reliability of the second recognition structure is low. At this time, the length of the sliding window can be increased (that is, the amount of the above information can be increased) to improve the accuracy of the second recognition result.

[0053] For example, Figure 2As shown, the sliding window information includes the target text information of three other voice segments. The first recognition result is "Dance Rhyme", and the sliding window information includes "Look", "Sky", "in". Using the text processing model, based on the first recognition result and the sliding window information, it is determined that the initial text information in the second recognition result is "Cloudless", and the initial confidence level is 0.52. The preset confidence level can be 0.6.

[0054] Figure 3 It is a schematic diagram of a sliding window according to another embodiment of the present disclosure.

[0055] As Figure 3 shown, since the initial confidence level is lower than the preset confidence level, the length of the sliding window can be increased so that the sliding window contains the target text information of five other voice segments. At this time, the updated sliding window information includes "Weather Forecast", "Rainy", "Look", "Sky", "in". Using the text processing model, based on the first recognition result and the updated sliding window information, it can be determined that the text information of the second recognition result is "Dark Clouds", and the corresponding confidence level is 0.78. At this time, it can be determined that "Dark Clouds" and the confidence level 0.78 are the second recognition results.

[0056] In the embodiments of the present disclosure, the confidence level is a quantitative evaluation of the credibility of the prediction result of the model. The initial confidence level corresponds to the quantitative evaluation of the credibility of the initial text information in the initial second recognition result. The value range of the initial confidence level is between 0 and 1. When the initial confidence level is low, it means that the credibility of the obtained initial second recognition result is low. At this time, the length of the sliding window can be increased to increase the number of upstream information considered by the text processing model. By introducing more upstream information, the accuracy of the second recognition result determined by the text processing model can be increased.

[0057] In the embodiments of the present disclosure, after determining the text information of the current voice segment to be recognized, when recognizing the next voice segment to be recognized, the length of the sliding window can be adjusted back to the initial length. For example, the length of the sliding window as Figure 3 shown can be adjusted back to the length of the sliding window as Figure 2 shown, thereby saving the computing power and processing time of the subsequent text processing model.

[0058] Through the embodiments of the present disclosure, the length of the sliding window is dynamically adjusted according to the confidence level of the second recognition result. When the confidence level corresponding to the second recognition result is low, the length of the sliding window is increased, so that the text processing model can consider more upstream information when determining the second recognition result, and the accuracy and reliability of the obtained second recognition result can be increased.

[0059] In some embodiments of the present disclosure, the method further includes: utilizing a text processing model to update target text information of other speech segments preceding the speech segment to be recognized based on target text information of the speech segment to be recognized.

[0060] In an embodiment of the present disclosure, after using a text processing model to determine the target text information of the voice segment to be recognized based on the first recognition result and the second recognition result, the text processing model can be used to update the target text information of other voice segments located before the voice segment to be recognized based on the target text information of the voice segment to be recognized.

[0061] In the disclosed embodiments, the length of the context cache window can be dynamically adjusted based on the scenario. For example, for a doctor-patient conversation, the target text corresponding to the voice message before and after 10 seconds can be retained. For a legal statement conversation, due to the long context dependency of the legal statement, the target text corresponding to the voice message before and after 30 seconds can be retained.

[0062] In the disclosed embodiments, a dynamic context cache window (e.g., the target text information of the last five voice segments) can be maintained. When a new voice segment is recognized and the corresponding target text information is obtained, the text processing model can be used to re-evaluate the old segments in the context cache window to determine whether the target text information needs to be modified.

[0063] For example, the target text information of the voice segment to be recognized is "college entrance examination". The text processing model is used to re-evaluate the old segments in the context cache window. The old segments in the context cache window include "school", "on the playground", "held", and "party member meeting". At this time, the text processing model determines that "mobilization meeting" is more in line with the context than "party member meeting", so the above "party member meeting" is updated to "mobilization meeting".

[0064] Through the embodiments of the present disclosure, the target text information of the preceding text can be corrected in a timely manner according to the target text information of the speech segment to be recognized, thereby avoiding the occurrence of context errors in subsequent speech segments due to errors in the preceding text.

[0065] In some embodiments of the present disclosure, the first recognition result includes first text information and a first confidence level of the first text information, and the second recognition result includes second text information and a second confidence level of the second text information; determining the target text information of the voice segment to be recognized based on the first recognition result and the second recognition result includes: determining the target text information of the voice segment to be recognized from the first text information and the second text information based on the first confidence level and the second confidence level.

[0066] In the disclosed embodiment, the first recognition result includes first text information and a first confidence level for the first text information, where the first confidence level is used to characterize the credibility of the first text information output by the speech recognition model. The second recognition result includes second text information and a second confidence level for the second text information, where the second confidence level is used to characterize the credibility of the second text information output by the text processing model.

[0067] In an embodiment of the present disclosure, using a text processing model to determine a second recognition result based on a first recognition result includes: using a text processing model to determine the second recognition result based on first text information.

[0068] For example, the speech recognition model determines that the first text message "kaihui" has a first confidence level of 0.35. Based on the first text message "kaihui", the text processing model can determine multiple second recognition results, including "kaihui", with a corresponding confidence level of 0.75, and "kaihui", with a corresponding confidence level of 0.1.

[0069] In the embodiment of the present disclosure, the target text information of the speech segment to be recognized is determined from the first text information and the second text information according to the first confidence level and the second confidence level. The text information corresponding to the highest confidence level can be selected as the target text information of the speech segment to be recognized.

[0070] For example, the first recognition result includes "kaihui" (opening wisdom) with a confidence of 0.35. The second recognition result includes "kaihui" (opening meeting) with a confidence of 0.75 and "kaihui" (opening exchange) with a corresponding confidence of 0.1. The "kaihui" with the highest confidence can be selected as the target text information of the speech segment to be recognized.

[0071] In the disclosed embodiment, different weights can be set for the first confidence level and the second confidence level. Because the text processing model has powerful contextual analysis capabilities and contains a large number of domain-related terms, a higher weight can be set for the second confidence level of the second recognition result than for the first confidence level of the first result. After the first confidence level and the second confidence level are multiplied by their corresponding weights, the text information corresponding to the highest confidence level is selected as the target text information of the speech segment to be recognized.

[0072] For example, the first weight corresponding to the first confidence level is 1, and the second weight corresponding to the second confidence level is 1.5. The first recognition result includes "kaihui" with a confidence level of 0.6. The second recognition result includes "kaihui" with a confidence level of 0.5 and "kaihui" with a corresponding confidence level of 0.1. After multiplying the first confidence level by the first weight and the second confidence level by the second weight, the highest confidence level is 0.75 corresponding to "kaihui". Therefore, "kaihui" is selected as the target text information of the speech segment to be recognized.

[0073] In the embodiment of the present disclosure, the second recognition result can be determined based on the first recognition result and the sliding window information. That is, the second confidence level in the second recognition result can be determined by the text processing model based on the first recognition result and the sliding window information (the above information).

[0074] In the embodiment of the present disclosure, the beam search algorithm in the text processing model can be used to determine the target text information of the speech segment to be recognized from the first text information and the second text information according to the first confidence level and the second confidence level.

[0075] Through the embodiments of the present disclosure, target text information is determined from text information according to the confidence level, which can improve the accuracy of the determined target text information.

[0076] In some embodiments of the present disclosure, it also includes: using a text processing model to determine a target knowledge base based on target text information of other voice segments located before the voice segment to be recognized; based on the target knowledge base, determining the first knowledge base matching degree and the second knowledge base matching degree according to the first recognition result and the second recognition result respectively; determining the target text information of the voice segment to be recognized from the first text information and the first text information according to the first confidence level and the second confidence level, including: determining the target text information of the voice segment to be recognized from the first text information and the second text information according to the first confidence level, the second confidence level, the first knowledge base matching degree and the second knowledge base matching degree.

[0077] In the disclosed embodiments, the text processing model includes multiple knowledge bases. These knowledge bases contain a large number of terms, phrases, and contextual information related to specific fields, which can help the text processing model improve recognition accuracy when processing professional terms or specific scenarios. For example, the text processing model may include a knowledge base related to the medical field, a knowledge base related to the mathematics field, and a knowledge base related to the home appliance field.

[0078] In the disclosed embodiment, a text processing model is used to determine a target knowledge base based on target text information (previous context) of other speech segments preceding the speech segment to be recognized. The target knowledge base is the knowledge base corresponding to the speech information to be recognized, determined based on the previous context. Based on the target knowledge base, the text processing model is used to determine the matching degree of the first knowledge base corresponding to the first recognition result, and the matching degree of the second knowledge base corresponding to the second recognition result, based on the second recognition result.

[0079] In an embodiment of the present disclosure, a text processing model is utilized to determine target text information for a speech segment to be recognized from first text information and second text information based on a first confidence level, a second confidence level, a first knowledge base match level, and a second knowledge base match level. A first comprehensive score for the first text information can be calculated based on the first confidence level and the first knowledge base match level, and a second comprehensive score for the second text information can be calculated based on the second confidence level and the second knowledge base match level. The text information corresponding to the higher of the first and second comprehensive scores is selected as the target text information for the speech segment to be recognized. The above steps can be implemented using a beam search algorithm within the text processing model.

[0080] For example, the specific calculation process can be seen in Table 1.

[0081] Table 1

[0082]

[0083] In Table 1, "patient needs PIC treatment" is the first text information, corresponding to a first confidence of 0.88, a first knowledge base matching degree of 0.15, a weight of the first confidence of 0.6, and a weight of the knowledge base matching degree of 0.1. Therefore, the first comprehensive score can be calculated to be 0.543.

[0084] "The patient needs PCI treatment" is the second text information, corresponding to the second confidence level of 0.95, the first knowledge base matching level of 0.99, the weight of the second confidence level of 0.7, and the weight of the knowledge base matching level of 0.1. The first comprehensive score can be calculated as 0.764.

[0085] "The patient needs PCI surgery" is the second text information, corresponding to the second confidence level of 0.93, the first knowledge base matching level of 0.98, the weight of the second confidence level of 0.7, and the weight of the knowledge base matching level of 0.1. The first comprehensive score can be calculated as 0.749.

[0086] Through the embodiments of the present disclosure, by dynamically combining multi-source confidence and knowledge bases, the text processing model can verify and identify specific terms, ensuring the accuracy and consistency of the identified target text information within a specific field.

[0087] In some embodiments of the present disclosure, the text processing model includes a search algorithm, which determines the target text information of the voice segment to be recognized based on the first recognition result and the second recognition result. The method also includes: adjusting the parameters of the search algorithm based on the audio characteristics of the voice information to be recognized.

[0088] In the embodiment of the present disclosure, the search algorithm may include a greedy search algorithm, a Viterbi algorithm, a Monte Carlo tree search algorithm, a genetic algorithm, a beam search algorithm, and the like.

[0089] In the embodiment of the present disclosure, the audio features may include the number of speakers of the voice information to be recognized, the background noise level, the speaking speed and pause features, etc.

[0090] For example, for the number of speakers, voiceprint recognition can be used to determine whether the speech is from a single speaker or a group conversation. For background noise level, the signal-to-noise ratio (SNR) can be calculated to determine whether the environment is quiet (e.g., meeting recordings) or noisy (e.g., outdoor conversations). For speech rate and pauses, the number of words per minute and pause frequency can be counted to distinguish between formal speeches (slow, regular pauses) and impromptu conversations (fast, irregular pauses).

[0091] In the disclosed embodiment, the parameters of the search algorithm can be adjusted according to the audio features. For example, for speech audio, the term library weight can be adjusted to give priority to matching professional vocabulary related to the speech.

[0092] In the disclosed embodiments, voiceprint features of the speech information to be recognized are obtained. Based on the voiceprint features, it is determined whether the speech information to be recognized is single-speaker audio or multi-speaker audio. If the speech information to be recognized is multi-speaker audio, a speaker separation model is used to segment the speech information to be recognized into multiple independent speech messages based on the voiceprint features, each of which is single-speaker audio.

[0093] In the disclosed embodiment, the steps of the speech recognition method of this embodiment can be performed separately for each individual audio recording. During the separate execution of the speech recognition method steps for each individual audio recording, when determining the target text information for the speech segment to be recognized for each individual audio recording, the target text information for the speech segment to be recognized can be determined based on the contextual information preceding the speech segment to be recognized in the individual audio recording.

[0094] For example, if the speech information to be recognized is the audio of a conversation between subjects A and B, a speaker separation model (such as PyAnnote or SpeechBrain) can be used to segment the speech information to be recognized into independent segments A and B, and timestamp them. Independent segment A is the audio of subject A, and independent segment B is the audio of subject B. The steps of the speech recognition method in this embodiment can be performed separately for independent segments A and B.

[0095] Through the embodiment of the present disclosure, when only the voice information of a single object needs to be recognized, the embodiment can effectively reduce the system workload and improve the efficiency of the voice recognition process.

[0096] In the embodiment of the present disclosure, if the confidence level of the speech recognition result of a single object is low, the speech information of other objects can be combined for recognition. The target text information of the speech segment to be recognized can be determined by combining the speech information of other objects.

[0097] For example, in the case where the speech information to be recognized is the dialogue audio of object A and object B, the speaker separation model can be used to split the speech information to be recognized into independent segments of A and B, and time stamps are marked. Perform the steps of the speech recognition method in this embodiment on the independent segment A. When the confidence level of the recognition result corresponding to the speech segment to be recognized in the independent segment A is low, the recognition result corresponding to the speech segment to be recognized in the independent segment A can be optimized by combining the target text information corresponding to other speech segments in the independent segment B. For example, there may be multiple rounds of dialogue between A and B, and corrections can be made based on the current round of dialogue, or corrections can be made to the text of the current round, or corrections can be made to the relevant dialogue in the rounds before the current round. For example, if there are matching keywords in the historical round of dialogue and the current round, it can be considered as relevant dialogue.

[0098] For example, B's answer triggers a correction to A's question.

[0099] A asks, "Is there radiation pain?" The ASR misrecognizes it as "Is there sour pain?"

[0100] B answers, "The pain radiates to the left arm."

[0101] The LLM detects that B's answer "radiates to the left arm" is related to pain radiation, traces back to A's question, finds that "sour pain" is homophonic with "radiation pain", automatically corrects A's text to "Is there radiation pain?", and updates the text of A in the context cache.

[0102] Through the embodiments of the present disclosure, correcting the speech recognition result according to the dialogue of different objects can further improve the accuracy of the speech recognition result.

[0103] In the embodiments of the present disclosure, the parameters of the search algorithm can also be adjusted according to user feedback. User feedback can include explicit feedback and implicit feedback.

[0104] For example, explicit feedback can include that the user manually corrects the recognition error (such as changing "myocardial infarction" to "myocardial infarction"), and the user marks a specific segment as important content (such as medical keywords).

[0105] For example, implicit feedback can include that the user repeats playing a certain audio segment (hinting that the recognition may be inaccurate), skips or deletes certain recognition results (indicating that there are recognition errors).

[0106] Through the embodiments of the present disclosure, adjusting the parameters of the search algorithm according to audio features can improve the reliability of the text processing model, and further improve the accuracy of the determined target text information.

[0107] In some embodiments of the present disclosure, the parameters of the search algorithm are adjusted according to the audio features of the speech information to be recognized, including: adjusting the search width or confidence threshold of the search algorithm according to the audio features; the search width represents the number of candidate paths retained by the search algorithm during the decoding process.

[0108] In the disclosed embodiments, the search width represents the number of candidate paths retained by the search algorithm during the decoding process. The search algorithm determines the target text information of the speech segment to be recognized based on the first and second recognition results. The number of second recognition results can be multiple, and the candidate path represents the number of recognition results. During the decoding process, the search algorithm continuously screens and prunes the multiple candidate paths to determine a final candidate path as the target text information.

[0109] In the disclosed embodiment, a larger search width can retain more candidate paths and reduce the probability of missing the optimal path. A smaller search width can reduce the amount of calculation and improve real-time performance. The search width of the search algorithm can be adjusted according to the audio features of the speech information to be recognized. For example, when the audio features represent that the speech information to be recognized is a long sentence, since the long sentence has strong context dependence, the search width can be increased at this time to avoid premature pruning and resulting in term loss. For example, when the audio features represent that the speech information to be recognized is a short sentence, since the short sentence has high real-time requirements, the search width can be reduced at this time, and it can quickly converge to the local optimum.

[0110] In the disclosed embodiments, the search algorithm can be a beam search algorithm, and the beam width of the beam search algorithm can be adjusted based on audio characteristics. The beam width represents the number of candidate paths retained in each search step. A larger beam width explores more potential paths, reducing the probability of missing the global optimal solution. Increasing the beam width also results in a linear increase in memory usage and computation time. A smaller beam width results in a faster search, but is more likely to get stuck in a local optimum.

[0111] For example, the type of speech information to be recognized can be determined based on audio features. For speech audio, the beam width of the beam search algorithm can be increased (Beam Width = 10). This allows easy-to-overlook correct paths to be retained during the middle and later stages of decoding, ensuring the coherence of long sentences. For conversational audio, the beam width can be shortened (Beam Width = 5) to quickly eliminate low-probability paths, thereby improving real-time performance.

[0112] The confidence threshold represents the minimum probability score for retaining a candidate path. Using the confidence threshold, low-confidence paths can be eliminated to improve output quality.

[0113] For example, when transient noise is detected, the confidence threshold can be temporarily lowered by 20% to prevent the accidental cutting of valid paths. For professional fields (such as medical care), the confidence threshold can be increased by 15% to strictly filter out ambiguous paths.

[0114] In the disclosed embodiments, for a beam search algorithm, the beam width controls the breadth of the search, determining the system's ability to explore the possibility space. The confidence threshold acts as a filter for path quality, influencing the reliability of the results. Dynamically adjusting the beam width or confidence threshold of the beam search algorithm based on audio features can improve the accuracy of the target text information determined.

[0115] In some embodiments of the present disclosure, the method includes: determining language features of target text information using a text processing model; and adjusting an acoustic model of a speech recognition model according to the language features.

[0116] In the embodiments of the present disclosure, language features are abstract features in text or speech that reflect language attributes, structure, and usage habits. Language features include lexical features, grammatical features, semantic features, pragmatic features, etc.

[0117] In the embodiments of this disclosure, the acoustic model is the core component of the speech recognition system, responsible for mapping audio signals into text information. Its core function is to model the probabilistic relationship between acoustic features and text information. For example, different languages correspond to different acoustic models.

[0118] For example, if the text processing model determines that the target text information's language characteristics correspond to Cantonese, the speech recognition model's acoustic model can be adjusted to a Cantonese acoustic model. If the text processing model determines that the target text information's language characteristics correspond to a mixture of Chinese and English, the Chinese general acoustic model and the English technical acoustic model can be loaded simultaneously, with a weight ratio of 4:6 between the Chinese general acoustic model and the English technical acoustic model.

[0119] Through the embodiments of the present disclosure, the speech recognition model can be accurately matched to language, dialect, and domain requirements, improving resource utilization, avoiding the loading of irrelevant models, and reducing memory usage. It can also improve the accuracy of key term recognition and enhance the robustness of speech recognition.

[0120] Figure 4 3 is a schematic diagram of the process of recognizing speech information to be recognized according to an embodiment of the present disclosure.

[0121] In an embodiment of the present disclosure, the speech information to be recognized is input into a speech recognition model, and the speech recognition model determines a first recognition result including first text information and a first confidence level based on the speech segment to be recognized in the speech information to be recognized. The first text information is input into a text processing model, and the text processing model outputs a second recognition result including second text information and a second confidence level based on the above information and the first text information. The text processing model determines the first knowledge base matching degree corresponding to the first text information, and also determines the second knowledge base matching degree corresponding to the second text information. Using a search algorithm, the target text information of the speech segment to be recognized is determined from the first text information and the second text information based on the first confidence level, the second confidence level, the first knowledge base matching degree, and the second knowledge base matching degree.

[0122] In the embodiment of the present disclosure, a speech recognition model (ASR) is used to perform preliminary recognition on a speech segment to be recognized in the speech information to be recognized, and generate first text information and a first confidence level.

[0123] The text processing model (large language model) is used to understand the language and tone characteristics of the recognition results and dynamically control the parameters of the speech recognition model. For example, the large language model analyzes the language characteristics (such as language, dialect, and tone) in the recognition results, and selects and switches the most suitable acoustic model in real time to optimize recognition accuracy. For example, the large language model adaptively adjusts the ASR decoding strategy based on information such as the tone characteristics, emotional expression, and speech rate changes in the recognition results, ensuring optimal recognition results in different scenarios.

[0124] Large language models can correct recognition results through long context windows. For example, by leveraging their long context windows, they can review and analyze previous contextual information, identify and correct errors in recognition results, and ensure the coherence and semantic consistency of recognition results.

[0125] In some embodiments of the present disclosure, the present invention includes: a memory for storing voice information to be recognized; a processor, communicatively connected to the memory, for using a voice recognition model to recognize voice segments to be recognized in the voice information to be recognized, and obtaining a first recognition result, wherein the voice information to be recognized includes multiple voice segments; using a text processing model to determine a second recognition result based on the first recognition result, wherein the first recognition result and the second recognition result both represent text features of the voice segment to be recognized, and the first recognition result and the second recognition result are different; using a text processing model to determine target text information of the voice segment to be recognized based on the first recognition result and the second recognition result; and determining text information of the voice information to be recognized based on the target text information of each voice segment in the voice information to be recognized.

[0126] In an embodiment of the present disclosure, the processor executes the speech recognition method described above based on the speech information to be recognized stored in the memory.

[0127] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0128] Figure 5 is a schematic block diagram of an example electronic device 500 for implementing embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptops, notebooks, tablets, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0129] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. Computing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0130] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0131] The computing unit 501 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the speech recognition method. For example, in some embodiments, the speech recognition method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the speech recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the speech recognition method by any other suitable means (e.g., via firmware).

[0132] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: an electronic device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0137] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a host product within the cloud computing service ecosystem, addressing the management difficulties and limited scalability of traditional physical hosts and VPS ("Virtual Private Server") services. The server may also be a server in a distributed system or a server integrated with blockchain.

[0138] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0139] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A speech recognition method, comprising: Recognizing a speech segment to be recognized in the speech information to be recognized using a speech recognition model to obtain a first recognition result, wherein the speech information to be recognized includes a plurality of speech segments; Determining a second recognition result based on the first recognition result using a text processing model, wherein the first recognition result and the second recognition result both represent text features of the speech segment to be recognized, and the first recognition result and the second recognition result are different; Determining target text information of the to-be-recognized speech segment using the text processing model according to the first recognition result and the second recognition result; The text information of the voice information to be recognized is determined according to the target text information of each voice segment in the voice information to be recognized.

2. The method according to claim 1, wherein determining the second recognition result based on the first recognition result comprises: Determining the second recognition result according to the first recognition result and the sliding window information; The sliding window information includes target text information of other speech segments before the speech segment to be recognized, and the length of the sliding window represents the amount of target text information in the sliding window information.

3. The method according to claim 2, wherein determining the second recognition result based on the first recognition result and the sliding window information comprises: Determining an initial second recognition result based on the first recognition result and the sliding window information; The initial second recognition result includes initial text information and initial confidence; In response to the initial confidence being less than a preset confidence, increasing the length of the sliding window to obtain updated sliding window information; The second recognition result is determined according to the first recognition result and the updated sliding window information.

4. The method according to claim 1, wherein the first recognition result includes first text information and a first confidence level of the first text information, and the second recognition result includes second text information and a second confidence level of the second text information; The determining, based on the first recognition result and the second recognition result, target text information of the speech segment to be recognized includes: According to the first confidence level and the second confidence level, target text information of the speech segment to be recognized is determined from the first text information and the second text information.

5. The method according to claim 4, further comprising: Determine a target knowledge base based on target text information of other speech segments preceding the speech segment to be recognized using the text processing model; Based on the target knowledge base, determining a first knowledge base matching degree and a second knowledge base matching degree according to the first recognition result and the second recognition result respectively; The determining, based on the first confidence level and the second confidence level, target text information of the to-be-recognized speech segment from the first text information and the second text information includes: Target text information of the speech segment to be recognized is determined from the first text information and the second text information according to the first confidence level, the second confidence level, the first knowledge base matching level, and the second knowledge base matching level.

6. The method according to claim 1, further comprising: Determining the language characteristics of the target text information using the text processing model; An acoustic model of the speech recognition model is adjusted according to the language features.

7. The method according to claim 1, wherein the text processing model includes a search algorithm, wherein the search algorithm determines target text information of the speech segment to be recognized based on the first recognition result and the second recognition result, and the method further includes: The parameters of the search algorithm are adjusted according to the audio features of the speech information to be recognized.

8. The method according to claim 7, wherein adjusting the parameters of the search algorithm according to the audio features of the speech information to be recognized comprises: adjusting a search width or a confidence threshold of the search algorithm according to the audio feature; The search width represents the number of candidate paths retained by the search algorithm during the decoding process.

9. The method according to claim 1, further comprising: The target text information of other speech segments before the speech segment to be recognized is updated according to the target text information of the speech segment to be recognized by utilizing the text processing model.

10. An electronic device comprising: A memory, used for storing voice information to be recognized; A processor is communicatively connected to the memory and is used to use a speech recognition model to recognize the speech segments to be recognized in the speech information to be recognized to obtain a first recognition result, where the speech information to be recognized includes multiple speech segments; use a text processing model to determine a second recognition result based on the first recognition result, where the first recognition result and the second recognition result both represent text features of the speech segments to be recognized, and the first recognition result and the second recognition result are different; use the text processing model to determine the target text information of the speech segments to be recognized based on the first recognition result and the second recognition result; and determine the text information of the speech information to be recognized based on the target text information of each speech segment in the speech information to be recognized.

Images