Speech transcription method, apparatus, device, medium, and program product

Abstract

The application discloses a speech transcription method, device, equipment, medium and program product, and belongs to the voice processing field. The method comprises the following steps: decoding audio data collected in a conversation based on an automatic speech recognition algorithm and a pre-set hot word set to obtain a corresponding target word sequence; forming a to-be-corrected segment by combining a to-be-corrected word in the target word sequence and the context of the to-be-corrected word, searching the to-be-corrected segment in a hot word context segment database to obtain a matched hot word context segment, and the to-be-corrected word comprises a word with a confidence lower than a pre-set confidence threshold and a word hitting a homonym ambiguity word table; performing correction processing on the to-be-corrected segment based on the similarity between the to-be-corrected segment and the matched hot word context segment to obtain a corresponding corrected segment; and outputting a transcription text according to the corrected segment and other words in the target word sequence except the to-be-corrected word. According to the embodiment of the application, the information restoration accuracy of speech transcription can be improved.

Description

Technical Field

[0001] This application belongs to the field of speech processing, and in particular relates to a speech transcription method, apparatus, device, medium and program product. Background Technology

[0002] Speech-to-text technology, also known as Automatic Speech Recognition (ASR), is a technique that converts speech into text. It has wide applications in scenarios such as meeting minutes, smart offices, remote collaboration, and medical records. However, in practical use, ASR systems, trained on large-scale general corpora, naturally tend to favor high-frequency general vocabulary. They lack accurate recognition capabilities for domain-specific terms, names, project codes, and keywords from users' personalized expressions. This can lead to the transcription of these terms into text with similar pronunciations but different meanings, resulting in inaccuracies in the transcribed text and affecting the accuracy of information restoration. Summary of the Invention

[0003] This application provides a speech transcription method, apparatus, device, medium, and program product that can improve the accuracy of information restoration in speech transcription.

[0004] In a first aspect, embodiments of this application provide a speech transcription method, comprising: decoding audio data collected in a conversation based on an automatic speech recognition algorithm and a pre-set set of hot words to obtain a corresponding target word sequence; forming a segment to be transcribed from the word to be transcribed in the target word sequence and its context; searching a hot word context segment database based on the segment to be transcribed to obtain a matching hot word context segment; the word to be transcribed includes words with a confidence level lower than a pre-set confidence threshold and words that hit a homonymous word list; the hot word context segment includes segments formed by hot words and their contexts; correcting the segment to be transcribed based on the similarity between the segment to be transcribed and the matching hot word context segment to obtain a corresponding corrected segment; and outputting the transcribed text based on the corrected segment and other words in the target word sequence besides the word to be transcribed.

[0005] Secondly, embodiments of this application provide a speech transcription device, comprising: a target word sequence determination module, used to decode audio data collected in a conversation based on an automatic speech recognition algorithm and a pre-set set of hot words to obtain a corresponding target word sequence; a semantic matching module, used to form a segment to be corrected by combining the word to be corrected in the target word sequence with the context of the word to be corrected, and to search a hot word context segment database based on the segment to be corrected to obtain a matching hot word context segment, wherein the word to be corrected includes words with a confidence level lower than a pre-set confidence threshold and words that hit a homonymous word list, and the hot word context segment includes a segment formed by the hot word and the context of the hot word; a correction module, used to correct the segment to be corrected based on the similarity between the segment to be corrected and the matching hot word context segment to obtain a corresponding corrected segment; and a text output module, used to output transcribed text based on the corrected segment and other words in the target word sequence except for the word to be corrected.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the speech transcription method of the first aspect.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the speech transcription method of the first aspect.

[0008] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the speech transcription method of the first aspect.

[0009] This application provides a speech-to-text method, apparatus, device, medium, and program product. Based on an automatic speech recognition algorithm and a hot word set, it decodes audio data collected in a conversation to obtain the target word sequence with the highest probability corresponding to the audio data. Words with confidence levels below a preset confidence threshold are combined with their context to form a segment to be corrected. The segment to be corrected is then searched in a hot word context segment database to obtain a matching hot word context segment. The hot word context segments represented by the data in the hot word context segment database incorporate semantic information of the hot words. The retrieval of the segment to be corrected in the hot word context segment database is a semantic and contextual retrieval, and it can correct the segment to be corrected based on the similarity between the segment to be corrected and the matching hot word context segment to obtain a corrected segment. The aforementioned semantic and contextual hot word context segment database retrieval and semantic correction of the segment to be corrected can compensate for the weak semantic understanding ability of the automatic speech recognition algorithm, resulting in accurate transcribed text and thus improving the accuracy of information restoration in speech-to-text transcription. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a speech transcription method provided in an embodiment of this application; Figure 2 A flowchart illustrating an example of a speech-to-text process provided in an embodiment of this application; Figure 3 A schematic diagram illustrating an example of the architecture of the speech transcription method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a speech transcription device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the purpose, technical solution, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of the details in these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application. It should be noted that the acquisition, storage, use, and processing of information and data in the embodiments of this application are all authorized by users or relevant organizations and comply with the relevant provisions of national laws and regulations. In the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or necessarily used such a solution.

[0013] Speech-to-text (STR) technology, also known as Automatic Speech Reproduction (ASR), is a technique that converts speech into text. It is widely used in scenarios such as meeting minutes, smart offices, remote collaboration, and medical records. However, in practical use, ASR systems, trained on large-scale general corpora, naturally tend to favor high-frequency general vocabulary. They lack accurate recognition of buzzwords such as domain-specific terms, names, project codes, and keywords from users' personalized expressions. This can lead to errors in the transcribed text, affecting the accuracy of information restoration. To improve the accuracy of hot word recognition, a hot word library can be pre-built. The accuracy largely depends on the comprehensiveness of this library, but since users need to create it themselves, its comprehensiveness is difficult to guarantee. In some cases, some words originally converted by the ASR system can be added to the hot word library as hot words. However, this depends on the accuracy of the original ASR conversion. If the original ASR conversion yields incorrect words, adding them to the hot word library will not only fail to improve the accuracy but may also have a negative impact. In some related technologies, after the full text is generated by the ASR system, it can be input into a large language model for correction, thereby improving the accuracy of speech-to-text transcription. However, speech-to-text technology is used in scenarios with high real-time requirements, and this approach would prolong the user's waiting time for the transcribed text, making it difficult to meet the real-time requirements of speech-to-text applications.

[0014] This application provides a speech-to-text method, apparatus, device, medium, and program product that can perform real-time speech-to-text transcription at the short sentence level. It can form segments from words and their contexts, and retrieve these segments from a hot word context segment database. The hot word context segments not only contain the hot words but also their context, allowing them to reflect the contextual meaning of the hot words. The segments can be corrected in real-time based on their similarity to the retrieved hot word context segments, introducing rapid semantic correction for hot word recognition. This avoids misidentification of homophones with different meanings, ensuring the accuracy of hot word recognition and thus improving the accuracy of the transcribed text. The rapid correction of hot word recognition and short sentence-level speech-to-text transcription also significantly shortens the time users wait for the transcribed text, meeting the real-time requirements of speech-to-text application scenarios and reducing generalization and migration costs. This application can also distinguish between general hot words in a conversation and user-specific hot words, enabling user-level hot word maintenance. This avoids problems arising from unclear referencing between different users under similar semantics and also prevents user-specific hot words from interfering with general hot words in a conversation. This application can also use words with high frequency of syllables that are not in the hot word set as potential hot words. Through the deep semantic reasoning capabilities of a large language model, the most suitable hot words in the current semantic environment are obtained and added to the hot word set. These hot words, along with their context, form a hot word context fragment and are added to the hot word context fragment database. This allows for the identification of hot words that the ASR system cannot generate and that are missing from the hot word set and hot word context fragment database, achieving zero-sample updates of hot words and dynamically supplementing the hot word set and hot word context fragment database. In application scenarios such as meeting recording, intelligent office, remote collaboration, and medical records, this can improve the processing efficiency of business processes. While ensuring data security and not changing users' original communication habits, it enables real-time capture and intelligent management of fragmented tasks, improving the continuity and execution of business workflows.

[0015] The speech transcription method, apparatus, equipment, medium, and program products provided in this application will be described separately below.

[0016] This application provides a speech-to-text method applicable to dialogue scenarios that require speech to be transcribed into text. These scenarios may include, but are not limited to, meeting minutes, smart office environments, remote collaboration, and medical records. The speech-to-text method can be executed by a speech-to-text device, electronic device, etc., and is not limited thereto. Figure 1 A flowchart of a speech transcription method provided in an embodiment of this application is shown below. Figure 1 As shown, the speech transcription method may include steps S101 to S104.

[0017] In step S101, the audio data collected in the conversation is decoded based on the automatic speech recognition algorithm and the pre-set hot word set to obtain the corresponding target word sequence.

[0018] The audio data is acquired in real time. For example, in a conversational scenario, audio data corresponding to each user's speech can be acquired in real time. The audio data corresponding to each speech sentence is then decoded to obtain the corresponding target word sequence. The target word sequence can be a short sentence-level word sequence. In some examples, the audio data may include one or more audio streams, with different users' speech corresponding to different audio streams. Each audio stream can be associated with a user identifier (user ID), which identifies the user. This association can be achieved through voiceprint recognition of the audio stream data or the source device of the audio stream data. Each audio stream can also be associated with a session identifier (Session ID), which identifies the session. The user identifier distinguishes audio stream data from different users, and the session identifier distinguishes audio stream data from different sessions.

[0019] The hotword set includes hot words, which may include pre-set hot words and new hot words continuously acquired during the use of the speech-to-text method. Hot words in the hotword set may include, but are not limited to, one or more of the following: domain-specific terms, personal names, project codes, and keywords from users' personalized expression habits. In some examples, the hotword set includes a general hotword set and a user-specific hotword set; the general hotword set includes common hot words in the conversation scenario and is not differentiated by user; the user-specific hotword set includes user-specific hot words, with different users having different user-specific hotword sets. Distinguishing between general and user-specific hot words avoids interference from user-level hot words in the overall conversation scenario; distinguishing between user-specific hot words avoids interference between hot words from different users, further improving the accuracy of hotword recognition.

[0020] Automatic speech recognition (ASR) algorithms convert speech data into text. Specifically, they decode audio data and use a weighted finite-state transducer (WFST) to search a graph to obtain the most probable word sequence from multiple word sequences. In this embodiment, the ASR algorithm uses audio data and a set of hot words to obtain the most probable word sequence, i.e., the target word sequence. The target word sequence is the most probable word sequence obtained by the ASR algorithm based on the audio data and the set of hot words. Hot words in the set can influence the recognition of words in the audio data. If the speech in the audio data includes hot words, those hot words can be identified, avoiding misidentification as other words.

[0021] In step S102, the word to be corrected in the target word sequence and its context are combined to form a segment to be corrected. The segment to be corrected is then searched in the hot word context segment database to obtain a matching hot word context segment.

[0022] The target word sequence contains at least one word. To detect homophone confusion and rare word combinations, confidence scores and homonym lists can be used to identify words in the target word sequence that require correction. Words to be corrected include those in the target word sequence with confidence scores below a preset confidence threshold and those that match the homonym list. The confidence score of each word in the target word sequence can be obtained; the confidence score characterizes the credibility of the word. Words with confidence scores below the preset confidence threshold are identified as words to be corrected. The preset confidence threshold is the threshold for determining whether a word needs correction and can be set according to the scenario, requirements, experience, etc.; a confidence score higher than or equal to the preset confidence threshold indicates that the word's credibility is high enough and does not need correction, and can be directly used as a word in the transcribed text; a confidence score lower than the preset confidence threshold indicates that the word's credibility is not high enough and needs correction. A homonym table can also be pre-set, which records the syllables of words that are homophones but ambiguous. If a word in the target sequence table matches the homonym table, further correction is needed to increase the accuracy of word recognition.

[0023] After obtaining the word to be corrected, the word and its context can be combined to form a fragment to be corrected. Specifically, a short context window centered on the word to be corrected can be extracted to obtain the fragment to be corrected. The fragment to be corrected can include the first N1 words of the word to be corrected, the word to be corrected, and the last N2 words of the word to be corrected, where N1 and N2 are positive integers. For example, the fragment to be corrected can be represented as Fragment(w i ) = (w max(1,i- N1) ,…,w i ,…,w min(W*,i+ N2)), where W* is the number of words in the target word sequence. A marker can be added to the segment to be corrected so that the segment can be identified based on the marker.

[0024] The data in the hot word context fragment database can represent hot word context fragments, which can correspond to hot words in the hot word set. In some examples, the hot word context fragments represented by the data in the hot word context fragment database may include pre-set hot word context fragments, and may also include hot word context fragments corresponding to new hot words continuously obtained during the use of the speech-to-text method. Hot word context fragments include fragments formed by hot words and their contexts. A hot word context fragment may include the first N3 words of the hot word, the hot word itself, and the last N4 words of the hot word, where N3 and N4 are positive integers. N3 and N1 may be the same or different, and N4 and N2 may be the same or different. In some examples, the hot word context fragment database may include hot word context fragments. In other examples, the hot word context fragment database may include vectors of hot word context fragments, as well as vectors and indexes of hot word context fragments. The corresponding hot word context fragment can be obtained from the vector of the hot word context fragment through the index. The hot word context fragments can be pre-encoded using a lightweight embedding model to obtain vectors of hot word context fragments and store them, avoiding the overhead of repeated encoding in the speech transcription process and further improving the speech transcription efficiency.

[0025] By searching the hot word context fragment database for the fragment to be corrected, at least one matching hot word context fragment can be retrieved. In some examples, the hot word context fragment database includes vectors of hot word context fragments. The fragment to be corrected can be converted into a vector, and the vector of the fragment to be corrected can be used to search the hot word context fragment database to obtain matching hot word context fragments.

[0026] In step S103, the segment to be corrected is corrected based on the similarity between the segment to be corrected and the matching hot word context segment to obtain the corresponding corrected segment.

[0027] The similarity between the segment to be corrected and the matched hot word context segment can be calculated. This similarity can be represented by parameters such as Euclidean distance and cosine similarity. Different similarities correspond to different correction methods. Based on the similarity between the segment to be corrected and the matched hot word context segment, the corresponding correction method can be determined, and the segment to be corrected can be processed using this method to obtain the corrected segment. Correction methods may include, but are not limited to, replacing the segment to be corrected with a matched hot word context segment, using a large language model to obtain the corrected segment based on the segment to be corrected and the matched hot word context segment, or identifying the segment to be corrected as the corrected segment, etc., to achieve semantic correction of the segment to be corrected. Through the correction processing of the segment to be corrected, the accuracy of speech-to-text transcription can be further improved. The segment to be corrected is at the short sentence level, for example, a window length of 5 to 10 characters, rather than the full text of the speech-to-text transcription. This allows for maintaining semantic accuracy while keeping the correction delay within a range imperceptible to the user.

[0028] In step S104, the transcribed text is output based on the corrected fragment and the other words in the target word sequence, excluding the word to be corrected.

[0029] It can assemble the corrected segment and the corresponding words of the word to be corrected, along with other words in the target word sequence except the word to be corrected, according to the word order in the target word sequence, to obtain the transcribed text and output it. Since it can perform real-time speech transcription for each sentence in a conversation to obtain the transcribed text of the corresponding sentence, the delay for each sentence transcription is very short, only 1 to 2 seconds, or even shorter, which can effectively ensure the timeliness of speech transcription.

[0030] In this embodiment, based on an automatic speech recognition algorithm and a hot word set, audio data collected during a conversation is decoded to obtain the target word sequence with the highest probability corresponding to the audio data. Words with confidence levels below a preset confidence threshold are combined with their context to form a segment to be corrected. The segment to be corrected is then searched in a hot word context segment database to obtain a matching hot word context segment. The hot word context segment represented by the data in the hot word context segment database incorporates the semantic information of the hot words. The search between the segment to be corrected and the hot word context segment database is a semantic and contextual search, and the segment to be corrected can be corrected based on the similarity between the segment to be corrected and the matching hot word context segment to obtain a corrected segment. The above-mentioned semantic and contextual hot word context segment database search and semantic correction of the segment to be corrected can compensate for the weak semantic understanding ability of the automatic speech recognition algorithm, obtain accurate transcribed text, and thus improve the accuracy of information restoration in speech transcription. Moreover, the real-time correction of hot word recognition and the short sentence-level speech-to-text transcription significantly shorten the time users wait for the transcribed text, improve the timeliness of speech-to-text transcription, and meet the real-time requirements of speech-to-text transcription application scenarios.

[0031] In some embodiments, the audio data includes one or more audio streams, each associated with a user identifier. The hot word set may include a general hot word set and a user-specific hot word set, as detailed in the above embodiments, and will not be repeated here. The audio stream data can be input into a decoder, and a weighted finite state converter search graph can be used to determine the target word sequence with the highest sequence score from multiple word sequences. The target word sequence is the word sequence with the highest sequence score among multiple word sequences in the weighted finite state converter search graph. The weighted finite state converter search graph can perform acoustic path matching of the audio stream data based on the original dictionary. In this embodiment, the sequence score of each word sequence obtained from the weighted finite state converter search graph may include the sum of multiple score parameters, at least some of which are related to a first weight and a second weight. When a word in the word sequence hits the general hot word set, the first weight increases at least some of the score parameters. When a word in the word sequence hits the user-specific hot word set corresponding to the user identifier associated with the audio stream data, the second weight increases at least some of the score parameters. The sequence score of a word sequence represents the probability that the word sequence is the correct word sequence corresponding to the speech in the audio stream data. The sequence score is positively correlated with the probability that the word sequence is the correct word sequence corresponding to the speech in the audio stream data; the higher the sequence score, the greater the probability that the word sequence is the correct word sequence corresponding to the speech in the audio stream data. Score parameters are parameters that affect the sequence score. In some examples, multiple score parameters may include, but are not limited to, acoustic model score, language model score, and bias term. The acoustic model score is the score given by the acoustic model to the word sequence in the automatic speech recognition algorithm, representing the degree of matching of the acoustic state of the audio stream data at the pronunciation level. The language model score is the score given by the language model to the word sequence in the automatic speech recognition algorithm, representing the reasonableness of the audio stream data appearing in natural language. The bias term represents the influence of hot words in the hot word set on the word sequence. The bias term is related to the first weight and the second weight; for example, the bias term of a word sequence can be the bias sub-item of each word in the word sequence. When the words in the word sequence hit a general set of hot words, the first weight takes effect, which improves the language model score and bias term, thereby improving the sequence score. When the words in the word sequence hit a user-specific set of hot words corresponding to the user identifier associated with the audio stream data, the second weight takes effect, which improves the bias term, thereby improving the sequence score. For example, the objective function of the target word sequence can be shown in equations (1) and (2) below: (1) (2) in, For the target word sequence; Score the acoustic model; Score the language model; For bias terms; For the biased sub-items of words in the word sequence; These are the weights for the language model; It is the first weight; For words in a word sequence; Score the sequence; A collection of general hot words; As the second weight; For a user-specific hot word set. When a word in a word sequence hits a general hot word set and / or a user-specific hot word set, the bias sub-item can be superimposed on the transition arc of the corresponding word in the word sequence in a weighted finite state converter search graph, thereby increasing the decoding priority of that word.

[0032] The confidence level in the above embodiments may include, but is not limited to, the confidence level based on the posterior probability of the weighted finite state converter search graph. For example, the confidence level may be as shown in equation (3): (3) in, The sum of probabilities of a word sequence containing the i-th word in the target word sequence in the search graph of a weighted finite state converter; The sum of probabilities of all word sequences in the search graph by the weighted finite state converter; represents the confidence score of the i-th word in the target word sequence.

[0033] In some embodiments, the hot word context fragment database includes vectors of hot word context fragments. The fragment to be corrected can be encoded as a query vector. Using the query vector, an Approximate Nearest Neighbor (ANN) search is performed in the hot word context fragment database to obtain the top K hot word context fragments with the highest to lowest similarity, where K is a positive integer. Approximate nearest neighbor search can reduce search complexity by building an index, thereby achieving a more timely response. K can be set according to the scenario, requirements, experience, etc., and is not limited here. For example, K can be 1 or a positive integer greater than 1. The similarity between the query vector and the vector of the hot word context fragment can represent the similarity between the fragment to be corrected and the hot word context fragment. Specifically, similarity refers to semantic similarity. For example, the fragment to be corrected T... text With the context of hot words T hot The similarity can be represented as Sim(T) text T hot ).

[0034] In some examples, a fuzzy matching range for similarity can be pre-set. This fuzzy matching range represents a similarity threshold indicating semantic ambiguity between the matched hot word context fragment and the fragment to be corrected, requiring further processing. The fuzzy matching range can be less than a first similarity threshold and greater than a second similarity threshold, where the second similarity threshold is less than the first. That is, the first similarity threshold is the upper limit of the fuzzy matching range, and the second similarity threshold is the lower limit. The first and second similarity thresholds can be set according to the scenario, requirements, experience, etc., and are not limited here. For example, the first similarity threshold could be 0.92, and the second similarity threshold 0.75, but this is not a limitation. In the similarity analysis of K matching hot word context fragments and the fragment to be corrected, if the largest similarity is less than the first similarity threshold but greater than or equal to the second similarity threshold, it indicates that the matched hot word context fragment has high reference value. A prompt word can be generated based on the fragment to be corrected and the K matching hot word context fragments. This prompt word is then input into a large language model, which, based on semantic coherence, selects the most suitable hot word from the K matching hot word context fragments based on the fragment to be corrected. This hot word replaces the word to be corrected in the fragment to be corrected, resulting in the corrected fragment. Alternatively, the large language model, based on semantic coherence, generates a new word corresponding to the word to be corrected based on the fragment to be corrected and the K matching hot word context fragments. This new word replaces the word to be corrected in the fragment to be corrected, resulting in the corrected fragment. The large language model can be a lightweight model, but is not limited to this.

[0035] In some examples, if the maximum similarity score is greater than or equal to the first similarity threshold, the word to be corrected is replaced with a hot word from the context of the hottest word with the highest similarity score, resulting in a corrected segment. If the maximum similarity score among the K matching contexts of hot words and the segment to be corrected is greater than or equal to the first similarity threshold, it indicates a clear match and semantic consistency between the segment to be corrected and the context of the hottest word with the highest similarity score. Therefore, the word to be corrected in the segment to be corrected is considered to be consistent with the hot word in the context of the hottest word with the highest similarity score, and can be replaced with the hot word in the context of the hottest word with the highest similarity score, thus obtaining the corrected segment.

[0036] In some examples, if the maximum similarity score is less than the second similarity threshold, the segment to be corrected is identified as the corrected segment. If the maximum similarity score among the K matching hot word context segments and the segment to be corrected is less than the second similarity threshold, it indicates that the matching hot word context segment is of low reference value to the segment to be corrected and is not interfered with. The original output of the segment to be corrected is preserved, and the segment to be corrected is used as the corrected segment.

[0037] In some examples, if the hot word context fragment database becomes unavailable or the large language model times out, the word to be corrected is matched against the static hot word table to ensure that the basic speech-to-text function is not interrupted. An error log is also logged and stored. The static hot word table contains preset hot words. If the word to be corrected matches a hot word in the static hot word table, the matched hot word replaces the word to be corrected.

[0038] In some embodiments, during speech transcription, corrected segments whose similarity to the matched hot word context segment is less than a first similarity threshold, and / or segments from the historical transcribed text of the conversation, can be stored in a buffer as cumulative data for zero-sample hot word generation. This accumulated data is then processed to achieve zero-sample hot word generation, enriching the hot word set and hot word context segment database. Segments can be read from the buffer periodically in an asynchronous manner. Syllable conversion is performed on the segments in the buffer, and the frequency of homophones is counted. Homophones with frequencies higher than a preset hot word frequency threshold are identified as suspected hot words. The segments in the buffer include corrected segments whose similarity to the matched hot word context segment is less than the first similarity threshold, and / or segments from the historical transcribed text of the conversation. The segments containing suspected hot words are clustered, and standard hot words corresponding to the suspected hot words are obtained through cluster inference using a large language model. The standard hot words are added to the hot word set, and the standard hot words and their contexts are combined to form hot word context segments and added to the hot word context segment database.

[0039] The preset hot word frequency threshold is a limit on the frequency of occurrence of suspected hot words. If the frequency of a homophone is higher than the preset hot word frequency threshold, it indicates that the homophone is a suspected hot word and requires further processing. The segment containing each suspected hot word can be obtained, and the segment can include the suspected hot word and its context. The segments containing suspected hot words can be clustered to obtain clusters. Each cluster contains segments with similar semantics and context. Based on the clusters, prompt words can be constructed. These prompt words are input into a large language model, which infers and generates the most suitable hot word (standard hot word) for the corresponding context of each cluster. The standard hot words are added to the hot word set to automatically supplement the set and improve its comprehensiveness. The standard hot words and their contexts form hot word context segments. These context segments can be converted into vectors and added to the hot word context segment database to automatically supplement the database. Through the above processing, hot words that traditional automatic speech recognition algorithms have not generated and that are missing from the hot word set and hot word context fragment database can be generated, thus achieving zero-sample update and maintenance of hot words.

[0040] In some examples, suspected hot words include general suspected hot words and user-specific suspected hot words, and the hot word set includes a general hot word set and a user-specific hot word set. Homophones in segments associated with the same user identifier within the buffer that have a frequency higher than a preset hot word frequency can be identified as user-specific suspected hot words. If a suspected hot word associated with the same user identifier and having a frequency higher than a preset hot word frequency exists in the buffer, it indicates that a certain suspected hot word frequently appears in the speech of the same user, and this suspected hot word should be identified as a user-specific suspected hot word for the user indicated by that user identifier. Other suspected hot words with frequencies higher than the hot word frequency but not affected by the user identifier can be identified as general suspected hot words. Segments containing user-specific suspected hot words can be clustered, and standard hot words corresponding to user-specific suspected hot words can be obtained through cluster inference using a large language model. These standard hot words can be added to the user-specific hot word set corresponding to the user identifier, thus updating the user-specific hot word set for each user. Similarly, the segments containing common suspected hot words can be clustered, and the standard hot words corresponding to the common suspected hot words can be obtained by reasoning from the clusters obtained by the large language model. The standard hot words corresponding to the common suspected hot words can be added to the common hot word set to realize the update of the common hot word set.

[0041] In some embodiments, hot words in the hot word set can be monitored, and the hot word set can be updated based on the usage effect of the hot words, thereby forming a feedback loop for the maintenance of the hot word set. The monitoring includes the hit frequency and secondary correction status of hot words in the hot word set; if the hit frequency is greater than a preset frequency threshold and the secondary correction status indicates that secondary correction has not been triggered, the confidence parameter of the hot word is increased, and the confidence parameter includes confidence level and / or weight; if the secondary correction status indicates that secondary correction has been triggered, the confidence parameter of the hot word is decreased; if the confidence parameter is lower than the elimination threshold, the hot word is deleted from the hot word set or a warning label is set for the hot word.

[0042] The hit rate of hot words can include the case where words in the target word sequence hit the hot word set, and the case where the similarity between the segment to be corrected and the context segment of the hot word is greater than or equal to the first similarity threshold. The preset frequency threshold can be a frequency judgment threshold for improving the credibility parameter, and can be set according to the scenario, needs, experience, etc., and is not limited here. The secondary correction situation is the case where the hot word is corrected by the large language model or manually corrected by the user, which triggers the secondary correction. If the hit frequency of the hot words in the hot word set is greater than the preset frequency threshold and the secondary correction situation indicates that the secondary correction has been triggered, it means that the hit frequency of the hot word is high and the credibility is high, which can improve the credibility parameter of the hot word, indicating that the credibility of the hot word is improved. The credibility parameter of the hot word can represent the credibility of the hot word. For the specific content of the confidence in the credibility parameter, please refer to the relevant explanation of the confidence in the above embodiment, and it will not be repeated here. The weight in the credibility parameter can include the first weight and the second weight in the above embodiment, and for the specific content, please refer to the relevant explanation in the above embodiment, and it will not be repeated here. The secondary correction status indicates that the hot word needs correction, meaning its credibility is relatively low. The credibility parameter of the hot word can be lowered to represent this decrease in credibility. The credibility parameter is positively correlated with credibility. The elimination threshold is the minimum value for a hot word to be eliminated from the hot word set. If the credibility parameter is lower than the elimination threshold, it indicates that the credibility of the hot word is too low, and the hot word can be eliminated from the hot word set. This can be done by deleting the hot word from the set or by setting a low-credibility warning label for standard hot words. The warning label can then prompt the user, allowing the user to decide whether to remove the hot word from the set. It should be noted that when removing a hot word from the set, the relevant contextual data of the corresponding hot word can also be deleted from the hot word contextual data database.

[0043] By updating the reliability parameters of hot words in the hot word set in real time through the usage effect of hot words, the determination of target word sequences and the retrieval of hot word context fragment database can be directly affected. This realizes the feedback loop and life cycle management of hot words in speech transcription, ensuring that the hot word set and hot word context fragment database can be continuously self-optimized, and further improving the accuracy of information restoration in speech transcription.

[0044] To facilitate understanding, the following example illustrates the specific process of the speech-to-text method. Figure 2 A flowchart illustrating an example of the speech-to-text process provided in an embodiment of this application is shown below. Figure 2 As shown, the speech-to-text process may include steps a1 to a11.

[0045] In step a1, audio data is accessed and bound to the user identifier.

[0046] Audio data may include one or more audio streams, and each audio stream may be associated with the user identifier of the user who spoke on that audio stream.

[0047] In step a2, ASR decoding and differential weighting of the hot word set are performed to obtain the target word sequence.

[0048] In step a3, each word in the target word sequence is assessed for confidence level and homonym list, resulting in words to be corrected and words not to be corrected. Specifically, it is determined whether the confidence level of a word in the target word sequence is lower than a preset confidence threshold, and whether a word in the target word sequence matches the homonym list. If a word's confidence level is lower than the preset confidence threshold and / or the word matches the homonym list, then the word is identified as a word to be corrected. If a word's confidence level is higher than or equal to the preset confidence threshold and the word does not match the homonym list, then the word is identified as a word not to be corrected.

[0049] In step a4, the corresponding segment to be corrected is retrieved from the hot word context segment database. The retrieval yields hot word context segments that match the segment to be corrected.

[0050] In step a5, the similarity between the segment to be corrected and the matched hot word context segments is determined. If there are more than two matching hot word context segments, the largest similarity among the corresponding hot word context segments is used for comparison. If the similarity is greater than or equal to the first similarity threshold, step a6 is executed; if the similarity is less than the first similarity threshold but greater than or equal to the second similarity threshold, step a7 is executed; if there is a service anomaly, step a8 is executed, where service anomalies include but are not limited to ASR service anomalies, large language model anomalies, etc.; if the similarity is less than the second similarity threshold, step a9 is executed.

[0051] In step a6, the word to be corrected is replaced with a hot word from the matched hot word context fragment.

[0052] In step a7, a corrected fragment is generated based on the prompt word using a large language model. The corrected fragment includes the word to be corrected, i.e., the corrected word. The prompt word is generated based on the word to be corrected and the context fragment of the matched hot words. If the corrected word in the generated corrected fragment is a hot word generated by the large language model that is not in the hot word set, then step a10 is executed.

[0053] In step a8, a reduction service is performed based on the word to be corrected and the static hot word list.

[0054] In step a9, the segment to be corrected is retained. That is, the words to be corrected in the segment are retained.

[0055] In step a10, zero-sample generation of hot words, updating of the hot word set, and updating of the hot word context fragment database are performed.

[0056] In step a11, the usage effectiveness of hot words in the hot word set is monitored to achieve a closed-loop feedback mechanism and lifecycle management for hot words. The feedback on the usage effectiveness of hot words can be applied to step a2.

[0057] The specific details of steps a1 to a11 above can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0058] This application's embodiments employ an end-to-cloud collaborative architecture, combining a user-end and cloud platform, and a dual-stream parallel design encompassing a real-time speech-to-text stream and a dynamic hot word evolution stream. The real-time speech-to-text stream enables real-time audio data acquisition, acoustic decoding, hot word context-based matching and retrieval, and real-time correction, ensuring a low-latency speech-to-text experience for the user. The dynamic hot word evolution stream, using asynchronous or real-time methods in the background, performs syllable-level clustering analysis on historical transcribed text, classifies user-specific hot words, and generates zero-sample hot words, continuously feeding back and optimizing the hot word set and hot word context fragment database.

[0059] The architecture of the speech-to-text method in this application embodiment can be implemented through five core functional units. Figure 3 A schematic diagram illustrating an example of the architecture of the speech transcription method provided in the embodiments of this application, as shown below. Figure 3 As shown, the architecture of this speech-to-text method may include an audio acquisition and user identification unit 201, an ASR real-time decoding and initial screening unit 202, a hot word context real-time matching and correction unit 203, a hot word zero-sample generation and update unit 204, and a hot word knowledge base unit 205.

[0060] The audio acquisition and user identification unit 201 can serve as the entry point for the speech-to-text method architecture. It is used to acquire audio stream data of the session in real time and bind a unique user identifier and session identifier to each audio stream data. The audio stream data associated with the user identifier and session identifier is transmitted to the downstream ASR real-time decoding and screening unit 202. This provides the prerequisite for distinguishing between general hot words and user-specific hot words, ensuring that hot words can be accurately attributed to the specific user who spoke, and avoiding interference from hot words across users.

[0061] The ASR real-time decoding and initial screening unit 202 can decode audio stream data using ASR technology, and obtain the target word sequence based on the acoustic model, speech model, and whether the word matches the hot word set. It calculates the confidence of each word in the target word sequence and determines the words and segments to be corrected. The segments to be corrected and their correction tags are output to the hot word context real-time matching and correction unit 203 as seed data for real-time correction and subsequent zero-sample generation.

[0062] The hot word context real-time matching and correction unit 203 uses the hot word context fragment database to perform real-time hot word context matching and semantic correction of the fragment to be corrected, solving the problems of weak semantic understanding of the traditional weighted finite state converter search graph method and high latency of large language models in correcting the full text of speech transcription.

[0063] The hot word zero-sample generation and update unit 204 can automatically generate new hot words that are not recognized by automatic speech recognition and are missing in the hot word set by mining the segments to be corrected that are generated during speech transcription and the segments in the historical transcribed text of the conversation, which have a similarity of less than the first similarity threshold with the matched hot word context segments. This enables dynamic supplementation of the hot word set and the hot word context segment database.

[0064] The hot word knowledge base unit 205 serves as a continuous storage center, supporting the data read / write needs of the aforementioned audio acquisition and user identification unit 201, ASR real-time decoding and initial screening unit 202, hot word context real-time matching and correction unit 203, and hot word zero-sample generation and update unit 204. The hot word knowledge base unit 205 may include a weighted original-text dictionary table and hot word set for a weighted finite-state converter search graph used for automatic speech recognition, as well as a hot word context fragment database for semantic matching. The hot word set includes a general hot word set and a user-specific hot word set. The hot word knowledge base unit 205 can also record information such as the generation source, effective time, and confidence level of hot words. The generation source may include, but is not limited to, manual addition and generation by large language models. The hot word knowledge base unit 205 also supports the rapid deletion of abnormal hot words and version backtracking of hot words to ensure the robustness of the speech-to-text system.

[0065] To facilitate understanding of the application effect of the speech transcription method in this application, the following explanation uses three scenarios as examples: project review meeting minutes, medical outpatient records, and cross-department meeting minutes.

[0066] Scenario 1: Recording of R&D Project Review Meeting During R&D project review meetings, the internal project code name "Tian Shu 3.0", the internal R&D algorithm name "DeepFlow-X", and the personal usage habits of the participating engineers frequently appeared. Traditional ASR technology easily mistranslated "Tian Shu 3.0" as "Tian Shu 3.0". Different participating engineers had different names for the same module. The method of manually maintaining a static hot word table caused the hot word update to lag behind the iteration speed of the project's hot words.

[0067] Using the speech-to-text method provided in this application, at the start of a meeting, each participant's user identifier can be bound through voiceprint recognition. When engineer A first mentions "DeepFlow-X," since the word is not included in the ASR (Automatic Speech Recognition System), its confidence level is lower than a preset confidence threshold. Therefore, this word is identified as a word to be corrected, resulting in a segment to be corrected that includes this word. A hot word context segment database is used for retrieval to obtain matching hot word context segments. The similarity between the segment to be corrected and the matching hot word context segment is less than a first similarity threshold but greater than or equal to a second similarity threshold, thus hitting the fuzzy matching range. A lightweight large language model can combine the semantics of the context such as "model deployment" and "inference acceleration" in the segment to be corrected to obtain "DeepFlow-X" in real time, which is then displayed in the subtitles. Within 5 minutes after the meeting ends, the segment to be corrected can be extracted from the buffer. Syllable clustering confirms that the word "DeepFlow-X" only appeared frequently during engineer A's speaking time, determining that "DeepFlow-X" is a user-specific hot word. "DeepFlow-X" is then automatically written into the user-specific hot word set corresponding to user A. When Engineer A mentioned "DeepFlow-X" again later, its second weight could be directly increased during the ASR decoding stage, thus identifying "DeepFlow-X" without further correction through the large language model. When Project Manager B mentioned "TianShu 3.0," it could be identified as a common hot word shared by multiple users, and its weight was uniformly increased across all participants' decoding paths to avoid being mistakenly converted into a common term.

[0068] Therefore, the speech transcription method in this application embodiment can accurately distinguish between user-level hot words and meeting-level hot words. The zero-sample generation mechanism enables new terms, i.e. new hot words, to be correctly identified as soon as they appear, without the need for manual intervention in the hot word set, which significantly improves the transcription accuracy and minutes generation efficiency of R&D project review meetings.

[0069] Scenario 2: Medical outpatient record scenario In medical outpatient record scenarios, doctors speak quickly during consultations, use a lot of medical terminology, and there are significant differences in terminology between different departments. Traditional static hot word lists are difficult to cover all sub-fields, and large language models can cause delays in electronic medical record entry after transcribing the entire consultation into full text, which affects consultation efficiency.

[0070] Using the speech-to-text method provided in this application, when an endocrinologist mentions "metformin extended-release tablets," the confidence level of the word is lower than a preset confidence threshold. This word is then identified as a word to be corrected, resulting in a segment to be corrected that includes the word. A hot word context segment database is used for retrieval to obtain matching hot word context segments. These matching hot word context segments contain the context of "diabetes" and "oral hypoglycemic agents," and have a similarity of 0.88 with the segment to be corrected. This similarity is less than the first similarity threshold but greater than or equal to the second similarity threshold, thus hitting the fuzzy matching range. By combining a lightweight large language model with the context of the patient's complaint of "high blood sugar" in the segment to be corrected, the word to be corrected can be changed to "metformin extended-release tablets." The entire process has a latency of less than 200ms, without affecting the doctor's consultation rhythm. When Doctor A mentions the department-specific examination "OGTT test," the system uses syllable clustering to identify that the word appears frequently only under Doctor A's account. It then automatically generates and marks this as a user-specific hot word, adding it to Doctor A's user-specific hot word set to avoid interfering with the transcription results of doctors in other departments. If the system encounters the rare disease name "Fabry disease," which ASR has not recognized, the speech-to-text method in this embodiment can use context such as "limb pain" and "proteinuria" from historical consultation records to generate "Fabry disease" through a heavyweight language model and add it to the hot word set and hot word context fragment database. Subsequent descriptions of the same symptoms can then be matched in real time.

[0071] Therefore, it can be seen that the real-time context correction at the short sentence level in the speech transcription method of this application can ensure the low latency requirement of medical outpatient record scenarios. The differentiation of user-specific hot words of different users also avoids cross-departmental terminology interference. The zero-sample generation mechanism enables the system to quickly adapt to professional terms in subdivided fields, improving the accuracy and timeliness of electronic medical record entry.

[0072] Scenario 3: Cross-department meeting minutes In cross-department meetings, the product, operations, and technology departments use different terminology systems. For example, the same concept might be called "user acquisition campaign" in operations, "user growth interface" in technology, and "customer acquisition funnel" in product. Traditional ASR technology cannot distinguish between different expressions of the same concept, leading to semantic confusion in meeting minutes. Furthermore, newly added business terms require manual addition to a static hot word list after the meeting.

[0073] Using the speech-to-text method provided in this application, when operations personnel mention "user acquisition campaign," "user acquisition campaign" can be identified as a hot word in the entire text and weighted in the decoding path of all participants. When technical personnel subsequently mention "user growth interface," a search of the hot word context fragment database reveals that "user growth interface" and "user acquisition campaign" have highly similar contextual semantics, but belong to specific expressions of different users. Therefore, "user growth interface" and "user acquisition campaign" are marked as user-specific hot words of technical personnel and operations personnel, respectively, avoiding mutual interference between hot words of different users. When the product manager proposed the new business concept "private domain retention pool" on short notice, the ASR (Application Service Registry) did not include this term. The confidence level of this term was lower than the preset confidence threshold, so it was identified as a term to be corrected. A fragment containing this term was obtained, and a hot word context fragment database was used for retrieval to find matching hot word context fragments. The similarity between the fragment to be corrected and the matching hot word context fragment was less than the first similarity threshold but greater than or equal to the second similarity threshold, thus hitting the fuzzy matching range. A lightweight large language model, combined with contexts such as "community operation" and "user retention," corrected it to "private domain retention pool" in real time. After the meeting, "private domain retention pool" can be categorized as a general hot word, and can be directly identified when mentioned by all participants in subsequent meetings. In subsequent processes, through a feedback verification mechanism, if "private domain retention pool" is found to be frequently hit in subsequent meetings without correction records, its weight can be automatically increased; if "private domain retention pool" is mistakenly transcribed as "private domain retention pool," the confidence level is reduced and manual review is triggered to ensure the accuracy of the hot word set and hot word context fragment database.

[0074] Therefore, it can be seen that the differentiation of user-specific hot word sets for different users can solve the ambiguity problem of cross-departmental terminology. The zero-sample generation and real-time correction closed loop enables temporary business terms to take effect immediately. The feedback verification mechanism can ensure the long-term accuracy of hot word sets and hot word context fragment databases, which can significantly improve the communication efficiency and minutes quality of cross-departmental collaborative meetings.

[0075] This application provides a speech transcription device, which corresponds to the speech transcription method in the above embodiments. Figure 4 This is a schematic diagram of the structure of a speech transcription device provided in an embodiment of this application, as shown below. Figure 4 As shown, the speech transcription device 300 may include a target word sequence determination module 301, a semantic matching module 302, a correction module 303, and a text output module 304.

[0076] The target word sequence determination module 301 can be used to decode the audio data collected in the conversation based on the automatic speech recognition algorithm and the pre-set hot word set to obtain the corresponding target word sequence.

[0077] The semantic matching module 302 can be used to form a segment to be corrected by combining the word to be corrected in the target word sequence with the context of the word to be corrected. Based on the segment to be corrected, a search is performed in the hot word context segment database to obtain the matching hot word context segment. The word to be corrected includes words with confidence scores lower than the preset confidence threshold and words that hit the homonymous word list. The hot word context segment includes the segment formed by the hot word and the context of the hot word.

[0078] The correction module 303 can be used to correct the segment to be corrected based on the similarity between the segment to be corrected and the matching hot word context segment, so as to obtain the corresponding corrected segment.

[0079] The text output module 304 can be used to output transcribed text based on the corrected fragment and other words in the target word sequence, excluding the word to be corrected.

[0080] In some embodiments, the audio data includes one or more audio streams, each audio stream being associated with a user identifier, and the hot word set includes a general hot word set and a user-specific hot word set. The target word sequence determination module 301 can be used to: input the audio stream data into a decoder, and use a weighted finite state converter search graph to determine the target word sequence with the highest sequence score from multiple word sequences; wherein the sequence score includes the sum of multiple score parameters, at least some of which are related to a first weight and a second weight; if a word in the word sequence matches the general hot word set, the first weight increases at least some of the score parameters; if a word in the word sequence matches the user-specific hot word set corresponding to the user identifier associated with the audio stream data, the second weight increases at least some of the score parameters.

[0081] In some embodiments, the hot word context fragment database includes vectors of hot word context fragments. The semantic matching module 302 can be used to: encode the fragment to be corrected into a query vector, and use the query vector to perform an approximate nearest neighbor search in the hot word context fragment database to obtain the top K hot word context fragments with the highest to lowest similarity as the matched hot word context fragments, where K is a positive integer.

[0082] In some embodiments, the correction module 303 can be used to: generate a prompt word based on the segment to be corrected and the matched hot word context segment when the maximum value in the similarity is less than a first similarity threshold and greater than or equal to a second similarity threshold, wherein the second similarity threshold is less than the first similarity threshold; and output the corrected segment based on the prompt word through a large language model.

[0083] In some embodiments, the correction module 303 can be used to: replace the word to be corrected with the hot word in the context of the hot word with the highest similarity when the maximum value in the similarity is greater than or equal to the first similarity threshold, and obtain the corrected segment; and determine the segment to be corrected as the corrected segment when the maximum value in the similarity is less than the second similarity threshold, wherein the second similarity threshold is less than the first similarity threshold.

[0084] In some embodiments, the speech transcription device 300 may further include a zero-sample generation module. The zero-sample generation module can be used to: convert segments in a buffer into syllables, count the frequency of homophones, identify homophones with frequencies higher than a preset hot word frequency threshold as suspected hot words, the segments in the buffer including segments to be corrected whose similarity to the matched hot word context segment is less than a first similarity threshold, and / or segments from the historical transcribed text in the conversation; cluster the segments containing suspected hot words, and obtain standard hot words corresponding to the suspected hot words through a large language model based on the clusters obtained from the clustering; add the standard hot words to the hot word set, and form hot word context segments from the standard hot words and their contexts and add them to the hot word context segment database.

[0085] In some embodiments, suspected hot words include general suspected hot words and user-specific suspected hot words, and the hot word set includes a general hot word set and a user-specific hot word set. The zero-sample generation module can be used to: identify homophones in segments associated with the same user identifier in the buffer that have a frequency higher than a preset hot word frequency as user-specific suspected hot words. The zero-sample generation module can also be used to: add the standard hot words corresponding to user-specific suspected hot words to the user-specific hot word set corresponding to the user identifier.

[0086] In some embodiments, the speech-to-text device 300 may further include a monitoring feedback module. The monitoring feedback module can be used to: monitor the hit frequency and secondary correction status of hot words in the hot word set; increase the confidence parameter of the hot word when the hit frequency is greater than a preset frequency threshold and the secondary correction status indicates that secondary correction has not been triggered, the confidence parameter including confidence level and / or weight; decrease the confidence parameter of the hot word when the secondary correction status indicates that secondary correction has been triggered; and delete the hot word from the hot word set or set a prompt label for the hot word when the confidence parameter is lower than an elimination threshold.

[0087] It should be noted that the speech transcription device 300 is a device corresponding to the speech transcription method described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0088] This application also provides an electronic device. Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, as shown below. Figure 5As shown, the electronic device 400 includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

[0089] In some examples, the processor 402 described above may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that may be configured to implement the embodiments of this application.

[0090] Memory 401 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the speech transcription method according to embodiments of this application.

[0091] The processor 402 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 401, so as to implement the speech transcription method in the above embodiments.

[0092] In some examples, electronic device 400 may also include a communication interface 403 and a bus 404. For example, Figure 5 As shown, the memory 401, processor 402, and communication interface 403 are connected through bus 404 and complete communication with each other.

[0093] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application. Input devices and / or output devices can also be connected through the communication interface 403.

[0094] Bus 404 includes hardware, software, or both, that couples components of electronic device 400 together. For example, and not as a limitation, bus 404 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0095] This application also provides a computer-readable storage medium storing computer program instructions. When these computer program instructions are executed by a processor, they can implement the speech-to-text method described in the above embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The aforementioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., and is not limited thereto.

[0096] This application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the speech transcription method in the above embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0097] It should be clarified that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. For the device embodiments, equipment embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description section of the method embodiments. This application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application. Furthermore, for the sake of brevity, detailed descriptions of known methods and techniques are omitted here.

[0098] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0099] Those skilled in the art will understand that the above embodiments are exemplary and not restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, specification, and claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other means or steps; the quantifier "a" does not exclude a plurality; the terms "first" and "second" are used to identify names and not to indicate any particular order. No reference numerals in the claims should be construed as limiting the scope of protection. The functionality of multiple parts appearing in the claims can be implemented by a single hardware or software module. The appearance of certain technical features in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A speech-to-text method, characterized in that, include: Based on an automatic speech recognition algorithm and a pre-set set of hot words, the audio data collected in the conversation is decoded to obtain the corresponding target word sequence; The words to be corrected in the target word sequence are combined with the context of the words to be corrected to form a fragment to be corrected. The fragment to be corrected is then searched in the hot word context fragment database to obtain a matching hot word context fragment. The words to be corrected include words with confidence scores lower than a preset confidence threshold and words that hit the homonym list. The hot word context fragment includes a fragment formed by the hot word and the context of the hot word. Based on the similarity between the segment to be corrected and the matching hot word context segment, the segment to be corrected is corrected to obtain the corresponding corrected segment; Based on the corrected fragment and the other words in the target word sequence except for the word to be corrected, output the transcribed text.

2. The method according to claim 1, characterized in that, The audio data includes one or more audio streams, each audio stream is associated with a user identifier, and the hot word set includes a general hot word set and a user-specific hot word set. The method, based on an automatic speech recognition algorithm and a pre-defined set of hot words, decodes the audio data collected during the conversation to obtain the corresponding target word sequence, including: The audio stream data is input into the decoder, and the target word sequence with the highest sequence score is determined from multiple word sequences using a weighted finite state converter search graph; The sequence score includes the sum of multiple scoring parameters, at least some of which are related to the first weight and the second weight. When the words in the word sequence hit the general hot word set, the first weight increases at least some of the scoring parameters. When the words in the word sequence hit the user-specific hot word set corresponding to the user identifier associated with the audio stream data, the second weight increases at least some of the scoring parameters.

3. The method according to claim 1, characterized in that, The hot word context fragment database includes vectors of hot word context fragments; The step of retrieving matching hot word context fragments from the hot word context fragment database based on the fragment to be corrected includes: The segment to be corrected is encoded into a query vector. The query vector is then used to perform an approximate nearest neighbor search in the hot word context segment database to obtain the top K hot word context segments with the highest to lowest similarity as the matching hot word context segments, where K is a positive integer.

4. The method according to claim 1, characterized in that, The step of correcting the segment to be corrected based on the similarity between the segment to be corrected and the matched hot word context segment to obtain the corresponding corrected segment includes: If the maximum value in the similarity is less than the first similarity threshold but greater than or equal to the second similarity threshold, a prompt word is generated based on the segment to be corrected and the matched hot word context segment, wherein the second similarity threshold is less than the first similarity threshold; The corrected fragment is output based on the prompt words using a large language model.

5. The method according to claim 1, characterized in that, The step of correcting the segment to be corrected based on the similarity between the segment to be corrected and the matched hot word context segment to obtain the corresponding corrected segment includes: If the maximum value in the similarity is greater than or equal to the first similarity threshold, the word to be corrected is replaced with the hot word in the context of the hot word with the highest similarity to obtain the corrected segment; If the maximum value of the similarity is less than the second similarity threshold, the segment to be corrected is determined as the corrected segment, where the second similarity threshold is less than the first similarity threshold.

6. The method according to claim 1, characterized in that, Also includes: The segments in the buffer are converted into syllables, and the frequency of homophones is counted. Homophones with a frequency higher than a preset hot word frequency threshold are identified as suspected hot words. The segments in the buffer include the segments to be corrected that have a similarity to the context segments of the matched hot words that is less than a first similarity threshold, and / or segments in the historical transcribed text of the session. Cluster the segments containing the suspected hot words, and use a large language model to infer the standard hot words corresponding to the suspected hot words based on the clusters obtained from the clustering. The standard hot words are added to the hot word set, and the standard hot words and their contexts are combined to form hot word context fragments and added to the hot word context fragment database.

7. The method according to claim 6, characterized in that, The suspected hot words include general suspected hot words and user-specific suspected hot words, and the hot word set includes a general hot word set and a user-specific hot word set; The step of identifying homophones with a frequency higher than the preset hot word frequency as suspected hot words includes: The homophones in the segments associated with the same user identifier in the buffer that have a frequency higher than the preset hot word frequency are identified as the user's specific suspected hot words. Adding the standard hot words to the hot word set includes: Add the standard hot words corresponding to the user-specific suspected hot words to the user-specific hot word set corresponding to the user identifier.

8. The method according to claim 1, characterized in that, Also includes: Monitor the hit frequency and secondary correction status of hot words in the hot word set; If the hit frequency is greater than a preset frequency threshold and the secondary correction condition does not trigger secondary correction, the credibility parameter of the hot words is increased. The credibility parameter includes confidence and / or weight. In the case where the secondary correction is triggered by the secondary correction scenario, the credibility parameter of the hot words is reduced. If the credibility parameter is lower than the elimination threshold, the hot word will be removed from the hot word set or a prompt label will be set for the hot word.

9. A speech-to-text device, characterized in that, include: The target word sequence determination module is used to decode the audio data collected in the conversation based on the automatic speech recognition algorithm and the pre-set hot word set to obtain the corresponding target word sequence; The semantic matching module is used to form a segment to be corrected by combining the word to be corrected in the target word sequence with the context of the word to be corrected, and to search the hot word context segment database based on the segment to be corrected to obtain the matching hot word context segment. The word to be corrected includes words with confidence scores lower than a preset confidence threshold and words that hit the homonym list. The hot word context segment includes a segment formed by the hot word and the context of the hot word. The correction module is used to correct the segment to be corrected based on the similarity between the segment to be corrected and the matching hot word context segment, so as to obtain the corresponding corrected segment; The text output module is used to output transcribed text based on the corrected fragment and other words in the target word sequence except for the word to be corrected.

10. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the speech transcription method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the speech transcription method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the speech transcription method according to any one of claims 1 to 8.

Images