Speech recognition method, device and equipment and readable storage medium

Abstract

The invention discloses a voice recognition method, device and equipment and a readable storage medium, and the method comprises the steps: responding to a voice interaction demand, and determining each voice channel participating in voice interaction; recognizing a real-time voice stream input to the voice channel, and generating a real-time translation text of the real-time voice stream; based on the structural features of the real-time translation text and / or the sentence segmentation features of the real-time voice stream, judging whether the current character triggers a global sentence segmentation event or not; if yes, generating a clause of the voice channel based on the current character, and determining a start timestamp of the clause; and generating an interaction context of each voice channel based on the clauses and the corresponding start timestamps. Visibly, through real-time processing of voice streams input by different voice channels and real-time generation of clauses, timeliness and accuracy of interaction context generation are improved, an AI host can conveniently perceive, understand and respond to interaction content in a complex multi-user voice scene in real time, and man-machine interaction experience is greatly improved.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and more specifically, to a speech recognition method, apparatus, device, and readable storage medium. Background Technology

[0002] With the popularization of artificial intelligence technology, AI hosts are widely deployed in the voice chat room field. Their core role is to intervene in a timely manner and proactively provide new topics when there is a lull in the chat room or other interactive stagnation, thereby effectively improving the user experience.

[0003] Currently, the common processing flow in voice chat rooms is to recognize the entire audio segment, output text segments, sort the text segments to form an interactive context, and analyze the interactive context to determine whether to trigger an AI host. Specifically, the interactive context formation process is as follows: after the user explicitly issues a command to end voice recording, the entire audio segment is recognized, the recognized text content is output as independent segments, and the different text contents are sorted according to the output time to form a multi-user interactive context.

[0004] However, in actual voice chat room interactions, dynamic interactive phenomena such as interruptions, overlapping voices, and buzzer-beating are common. The aforementioned interaction context formation process based on independent recognition of entire voice segments can only output independent text paragraphs corresponding to single voice segments. It cannot reflect real interactive states such as interruptions, overlapping voices, and buzzer-beating in the generated interaction context, resulting in a low degree of consistency and reliability between the final multi-user interaction context and the actual interaction process. Consequently, the accuracy of AI hosts in recognizing user intent based on this unreliable interaction context decreases, making it difficult to accurately match and meet users' real-time interaction needs. Summary of the Invention

[0005] In view of this, this application provides a speech recognition method, apparatus, device, and readable storage medium to address the shortcomings of low accuracy in multi-user interaction context in the prior art.

[0006] To achieve the above objectives, the following solution is proposed:

[0007] A speech recognition method, comprising:

[0008] In response to the need for voice interaction, identify the various voice channels involved in the voice interaction;

[0009] For each voice channel, the real-time voice stream input to the voice channel is identified, and a real-time translated text of the real-time voice stream is generated, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, it is determined whether the current character triggers a global sentence segmentation event; if so, a clause of the voice channel is generated based on the current character, and the start timestamp of the clause is determined;

[0010] Based on the clauses and their corresponding start timestamps, the interaction context for each voice channel is generated.

[0011] Optionally, determining whether the current character triggers a global sentence segmentation event based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream includes:

[0012] Identify each target character in the real-time translated text corresponding to the current time period, and identify the speech stream segment in the real-time speech stream corresponding to the current time period;

[0013] Based on the structural features of each target character and / or the punctuation features of the speech stream segment, determine whether any current character in each target character triggers a global punctuation event.

[0014] Optionally, based on the structural features of each target character, determine whether any current character in each target character triggers a global sentence segmentation event, including:

[0015] Based on the real-time translated text, each target character is analyzed to detect whether it contains punctuation features and / or sentence break keywords;

[0016] If the sentence segmentation keyword is included, the current character that triggers the global sentence segmentation event is selected from each target character based on the sentence segmentation keyword; if the punctuation feature is included but the sentence segmentation keyword is not included, the target character matched by the punctuation feature is used as the current character that triggers the global sentence segmentation event.

[0017] Optionally, based on the sentence segmentation features of the speech stream segment, it is determined whether any current character in each target character triggers a global sentence segmentation event, including:

[0018] Based on the speech stream segment, calculate the pause duration between two adjacent target characters;

[0019] When the pause duration exceeds a preset duration threshold, the target character with the earlier timestamp among the two target characters is taken as the current character that triggers the global sentence segmentation event.

[0020] Optionally, generating a clause for the voice channel based on the current character and determining the start timestamp of the clause includes:

[0021] Determine the starting character that matches the current character;

[0022] The clause is formed by extracting the text between the starting character and the current character from the real-time translated text.

[0023] The timestamp corresponding to the starting character is used as the start timestamp of the clause.

[0024] Optionally, generating the interaction context for each voice channel based on the clause and its corresponding start timestamp includes:

[0025] Determine the speaker identifier corresponding to the clause;

[0026] An interaction context is generated based on the start timestamp and speaker identifier corresponding to the clause.

[0027] Optionally, the step of recognizing the real-time speech stream input to the speech channel and generating the real-time translated text of the real-time speech stream includes:

[0028] The Automatic Speech Recognition (ASR) stream matched to the speech channel is used to recognize the real-time speech stream input to the speech channel and generate real-time translated text of the real-time speech stream.

[0029] A voice recognition device, comprising:

[0030] The determination module is used to determine the various voice channels involved in the voice interaction in response to voice interaction needs.

[0031] The recognition module is used to recognize the real-time speech stream input to each speech channel, generate real-time translated text of the real-time speech stream, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream, determine whether the current character triggers a global sentence segmentation event; if so, generate a clause for the speech channel based on the current character, and determine the start timestamp of the clause;

[0032] The generation module is used to generate the interaction context of each voice channel based on the clause and its corresponding start timestamp.

[0033] A voice recognition device, including a memory and a processor;

[0034] The memory is used to store programs;

[0035] The processor is used to execute the program to implement the various steps of the above-described speech recognition method.

[0036] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the various steps of the above-described speech recognition method.

[0037] As can be seen from the above technical solution, the method provided by this application can, in response to the needs of voice interaction, determine each voice channel involved in the voice interaction; based on this, this application can determine the participating entities of the voice interaction and the corresponding voice transmission carrier, and the subsequent parallel processing of voice streams by channel provides a physical basis for differentiation, avoiding the mixing of voice signals from different users, and solving the problem of multi-user voice confusion from the source. For each voice channel, this application identifies the real-time voice stream input to the voice channel and generates real-time translated text of the real-time voice stream, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, it determines whether the current character triggers a global sentence segmentation event; if so, it generates a clause for the voice channel based on the current character and determines the start timestamp of the clause; thus, this application can overcome the limitation of the prior art that requires the completion of the entire voice recording before voice recognition by performing real-time recognition of the voice stream input in each voice channel, realize the synchronous progress of voice input and text translation, reduce the latency of voice recognition, reduce the latency of user interaction, improve the real-time perception capability of the AI ​​host, and improve the intervention efficiency of the AI ​​host; at the same time, this application adopts a joint adjudication mechanism of the structural features of the real-time translated text and the sentence segmentation features of the real-time voice stream to improve the triggering of global sentence segmentation events. The reliability and effectiveness of this method are ensured by reconstructing the true timeline and sequence of dialogue at the clause level, making the logical relationship of speech clear. Each character has a corresponding timestamp, solving the problem that traditional text cannot correlate with the timing of speech. Each speech channel is processed separately, allowing for parallel clause recognition across different channels. The global sentence segmentation event triggering mechanism can respond in real-time to changes in the speech and text streams, adapting to dynamic interactive behaviors such as interruptions and quick responses in multi-user scenarios, avoiding missing real-time interaction details while waiting for the entire speech segment to end. Furthermore, the timing reliability of each clause in each interactive context is guaranteed. This application generates interactive contexts for each speech channel based on the clauses and their corresponding start timestamps. By sorting the clauses using their start timestamps as the sorting criterion, the generated interactive contexts strictly follow the actual time sequence of the speech interaction, truly reconstructing dynamic interactive phenomena such as interruptions, overlaps, and quick responses, solving the problem of context distortion caused by disordered timing recognition in traditional solutions. As can be seen, this application improves the timeliness and accuracy of interactive context generation by real-time processing of input speech streams from different speech channels and real-time generation of clauses, making it easier for AI hosts to perceive, understand and respond to interactive content in complex multi-user speech scenarios, and greatly enhancing the human-computer interaction experience. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 This is a flowchart of a speech recognition method disclosed in an embodiment of this application;

[0040] Figure 2 This is a structural block diagram of a speech recognition device disclosed in an embodiment of this application;

[0041] Figure 3 This is a hardware structure block diagram of a speech recognition device disclosed in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] This application provides a speech recognition method that can be applied to various speech recognition systems or voice chat management systems, as well as to various computer terminals or smart terminals. The executing entity can be the processor or server of the computer terminal or smart terminal.

[0044] Next, combine Figure 1 The speech recognition method of this application is described in detail, including the following steps:

[0045] Step S1: In response to the voice interaction requirements, determine the various voice channels involved in the voice interaction.

[0046] Specifically, it can respond to user-initiated voice interaction requests and determine the number of users participating in the voice interaction and the corresponding voice channel for each user based on the voice interaction request.

[0047] For example, in a multi-user voice chat room, when a user initiates a voice conversation, the system can identify each user currently participating in the chat and assign an independent voice channel to each user.

[0048] Voice channels can be virtual channels defined at the software level to ensure that the voice signals of different users can be accurately distinguished and processed separately, avoiding mutual interference and confusion.

[0049] Step S2: For each voice channel, identify the real-time voice stream input to the voice channel and generate a real-time translated text of the real-time voice stream, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, determine whether the current character triggers a global sentence segmentation event; if so, generate a clause for the voice channel based on the current character and determine the start timestamp of the clause.

[0050] Specifically, it can receive real-time audio streams input to the audio channel in real time, and recognize the received real-time audio streams to generate real-time translated text.

[0051] Each character in the real-time transcribed text has a corresponding timestamp, which represents the point in time when the corresponding character appeared in the speech stream.

[0052] It can combine the structural features of real-time translated text with the sentence segmentation features of real-time speech stream to determine whether the current character triggers a global sentence segmentation event.

[0053] If so, it can be determined that the current character triggers a global sentence segmentation event. Based on the current character, a clause for the speech channel can be generated, and the start timestamp of the clause can be determined.

[0054] It can interrupt the speech recognition process of other speech channels and can also collect incomplete clause fragments recognized in other speech channels.

[0055] If not, real-time voice stream recognition can continue.

[0056] Step S3: Based on the clause and its corresponding start timestamp, generate the interaction context for each voice channel.

[0057] Specifically, the clauses of each voice channel can be sorted according to their corresponding start timestamps to form an interactive context.

[0058] Specifically, when an interaction context already exists, once a global sentence segmentation event is triggered by a voice channel, the generated clause can be used as the latest interaction content to update the interaction context.

[0059] It can also determine the start timestamp of incomplete clause fragments identified in other speech channels;

[0060] The clauses and clause fragments are sorted according to their start timestamps, and the sorting results are used as the latest interaction content to update the interaction context.

[0061] As can be seen from the above technical solution, the method provided by this application can, in response to the needs of voice interaction, determine each voice channel involved in the voice interaction; based on this, this application can determine the participating entities of the voice interaction and the corresponding voice transmission carrier, and the subsequent parallel processing of voice streams by channel provides a physical basis for differentiation, avoiding the mixing of voice signals from different users, and solving the problem of multi-user voice confusion from the source. For each voice channel, this application identifies the real-time voice stream input to the voice channel and generates real-time translated text of the real-time voice stream, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, it determines whether the current character triggers a global sentence segmentation event; if so, it generates a clause for the voice channel based on the current character and determines the start timestamp of the clause; thus, this application can overcome the limitation of the prior art that requires the completion of the entire voice recording before voice recognition by performing real-time recognition of the voice stream input in each voice channel, realize the synchronous progress of voice input and text translation, reduce the latency of voice recognition, reduce the latency of user interaction, improve the real-time perception capability of the AI ​​host, and improve the intervention efficiency of the AI ​​host; at the same time, this application adopts a joint adjudication mechanism of the structural features of the real-time translated text and the sentence segmentation features of the real-time voice stream to improve the triggering of global sentence segmentation events. The reliability and effectiveness of this method are ensured by reconstructing the true timeline and sequence of dialogue at the clause level, making the logical relationship of speech clear. Each character has a corresponding timestamp, solving the problem that traditional text cannot correlate with the timing of speech. Each speech channel is processed separately, allowing for parallel clause recognition across different channels. The global sentence segmentation event triggering mechanism can respond in real-time to changes in the speech and text streams, adapting to dynamic interactive behaviors such as interruptions and quick responses in multi-user scenarios, avoiding missing real-time interaction details while waiting for the entire speech segment to end. Furthermore, the timing reliability of each clause in each interactive context is guaranteed. This application generates interactive contexts for each speech channel based on the clauses and their corresponding start timestamps. By sorting the clauses using their start timestamps as the sorting criterion, the generated interactive contexts strictly follow the actual time sequence of the speech interaction, truly reconstructing dynamic interactive phenomena such as interruptions, overlaps, and quick responses, solving the problem of context distortion caused by disordered timing recognition in traditional solutions. As can be seen, this application improves the timeliness and accuracy of interactive context generation by real-time processing of input speech streams from different speech channels and real-time generation of clauses, making it easier for AI hosts to perceive, understand and respond to interactive content in complex multi-user speech scenarios, and greatly enhancing the human-computer interaction experience.

[0062] The following section will provide a detailed introduction to this application through specific examples.

[0063] Identify the real-time voice stream input to the first voice channel, and generate the real-time translated text as "Let's go out and play together on Saturday? Lend me". Identify the real-time voice stream input to the second voice channel, and generate the real-time translated text as "Okay".

[0064] Among them, the timestamp corresponding to the character "Saturday" is t0, the timestamp corresponding to the character "?" is t1, the timestamp corresponding to the character "Okay" is t2, and the timestamp corresponding to the character "Lend" is t3.

[0065] After applying this application, by identifying the structural features of the real-time translated text, it is determined that the character "?" is the current character triggering the global sentence-breaking event, then the clause "Let's go out and play together on Saturday?" of the first voice channel is formed, and the start timestamp of this clause is t0.

[0066] Since the voice input has ended after the voice stream "Okay" is input to the second voice channel, therefore, by identifying the sentence-breaking features of the voice stream, it is determined that the character "Okay" is the current character triggering the global sentence-breaking event, then the clause "Okay" of the second voice channel is formed, and its corresponding timestamp is t2.

[0067] Since t2 is after the timestamp t0 of the clause "Let's go out and play together on Saturday?" of the first voice channel, the interaction context formed is:

[0068] User of the first voice channel: Let's go out and play together on Saturday?

[0069] User of the second voice channel: Okay.

[0070] Continuously identify the real-time voice stream input to the first voice channel, and generate the real-time translated text as "Let's go out and play together on Saturday? Lend me 10,000 yuan".

[0071] Among them, the timestamp corresponding to the character "yuan" is t4.

[0072] The chronological order of each timestamp is: t0, t1, t2, t3, and t4.

[0073] Similarly, since the voice input has ended after the character "yuan" is input to the first voice channel, therefore, by identifying the sentence-breaking features of the voice stream, it is determined that the character "yuan" is another current character triggering the global sentence-breaking event, then the clause "Lend me 10,000 yuan" of the first voice channel is formed, and its corresponding timestamp is t3.

[0074] Combined with the timestamp t3 of "yuan" in the first voice channel, the interaction context can be updated as:

[0075] User of the first voice channel: Let's go out and play together on Saturday?

[0076] User of the second voice channel: Okay.

[0077] First voice channel user: Lend me 10,000 yuan.

[0078] Therefore, this application demonstrates that it can recreate the actual interaction in a voice interaction scenario and improve the reliability of the interaction context. Through fine-grained sorting, the clauses of the second voice channel can be accurately placed before a certain clause of the first voice channel, perfectly recreating the instantaneous response and answering relationship between the second user in the second voice channel and the first user in the first voice channel on the first question. This enables the AI ​​host to perform accurate intent recognition based on a logically complete and timely context.

[0079] In some embodiments of this application, the process of identifying the real-time speech stream input to each speech channel and generating real-time translated text of the real-time speech stream in step S2, wherein the real-time translated text contains a timestamp corresponding to each character, is described in detail below:

[0080] S20. Using the Automatic Speech Recognition Stream (ASR) matched to the speech channel, the real-time speech stream input to the speech channel is recognized, and the real-time translated text of the real-time speech stream is generated.

[0081] Specifically, each voice channel can correspond to an independent Automatic Speech Recognition Stream (ASR). The ASR can be matched with the voice channel via a WebSocket connection to ensure that the real-time voice stream can be transmitted to the corresponding ASR for recognition.

[0082] When a real-time audio stream is input to the audio channel, the ASR matched by the audio channel is used to recognize the input audio stream character by character, and the recognition result is output in text form. Each character is marked with a corresponding timestamp, generating a real-time translated text containing the timestamp of each character.

[0083] As can be seen from the above technical solution, this embodiment provides an optional method for recognizing the real-time speech stream input to the speech channel using the Automatic Speech Recognition (ASR) stream matched to the speech channel, and generating real-time translated text of the real-time speech stream. Through this method, the ASR matched to each speech channel can be utilized to ensure that the speech recognition process of each speech channel does not interfere with each other, improving the accuracy and efficiency of recognition, and enhancing the reliability of the real-time translated text.

[0084] In some embodiments of this application, the process of determining whether the current character triggers a global sentence segmentation event based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream in step S2 is described in detail, and the steps are as follows:

[0085] S20. Determine each target character corresponding to the current time period in the real-time translated text, and determine the speech stream segment corresponding to the current time period in the real-time speech stream.

[0086] Specifically, the current time period can be the time period N seconds before the current moment.

[0087] N can be set according to actual needs.

[0088] Each character within the current time period can be extracted from the real-time translated text as a target character;

[0089] It can extract the audio stream segment corresponding to the current time period from the real-time audio stream.

[0090] S21. Based on the structural features of each target character and / or the punctuation features of the speech stream segment, determine whether any current character in each target character triggers a global punctuation event.

[0091] Specifically, VAD can be used to determine whether any current character in each target character triggers a global sentence segmentation event, based on the structural features of each target character and / or the sentence segmentation features of the speech stream segment.

[0092] Structural features include, but are not limited to, syntactic structural features and character composition features.

[0093] For example, it can determine whether the target characters constitute a complete sentence component, or whether punctuation marks are present, which may trigger sentence segmentation.

[0094] Sentence segmentation features may include tone features, silence features, and pause features.

[0095] Tone features may include interrogative tone, affirmative tone, etc.

[0096] The silence feature can be the end of a speech stream by silencing.

[0097] Pause features can include short pauses in speech.

[0098] It can combine information from both structural features and sentence segmentation features to determine whether any current character in each target character satisfies the condition for triggering a global sentence segmentation event.

[0099] If it exists, then the current character is determined to trigger a global sentence segmentation event;

[0100] If it does not exist, it is determined that no character in the current time period triggers a global sentence segmentation event, and similar judgments are continuously made for subsequent audio streams.

[0101] As can be seen from the above technical solution, this embodiment provides an optional method for determining whether the current character triggers a global sentence segmentation event based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream. By determining the current time period, redundancy in sentence segmentation recognition content can be reduced, improving the accuracy and efficiency of sentence segmentation recognition and avoiding the waste of computing resources and recognition delays caused by recognizing the entire speech stream. Furthermore, combining structural features and sentence segmentation features for comprehensive judgment allows for a more comprehensive consideration of various factors in voice interaction, making the triggering of global sentence segmentation events more reliable and effective.

[0102] In some embodiments of this application, the process of determining whether any current character in each target character triggers a global sentence segmentation event based on the structural features of each target character in step S21 is described in detail, and the steps are as follows:

[0103] S210. Based on the real-time translated text, analyze each target character and detect whether each target character contains punctuation features and / or sentence break keywords.

[0104] Specifically, punctuation features can include punctuation marks and potential punctuation locations.

[0105] Key words for sentence segmentation can include transition words, causal conjunctions, and sentence initiation words.

[0106] Transition words can include "but" and "however".

[0107] Causal conjunctions can include "because" and "therefore".

[0108] The starting word of a statement can be "currently", "subsequently", or "at xx time", etc.

[0109] S211. If the sentence segmentation keyword is included, the current character that triggers the global sentence segmentation event is selected from each target character based on the sentence segmentation keyword; if the punctuation feature is included but the sentence segmentation keyword is not included, the target character matched by the punctuation feature is used as the current character that triggers the global sentence segmentation event.

[0110] Specifically, when each target character includes sentence-breaking keywords and punctuation features, or includes sentence-breaking keywords but not punctuation features, the target character preceding the sentence-breaking keyword can be used as the current character that triggers the global sentence-breaking event.

[0111] When a punctuation feature is included but a sentence-breaking keyword is not included, if the punctuation feature is a punctuation mark, the punctuation mark can be used as the current character that triggers the global sentence-breaking event; if the punctuation feature is a potential punctuation position, the target character preceding the potential punctuation position can be used as the current character that triggers the global sentence-breaking event.

[0112] As can be seen from the above technical solution, this embodiment provides an optional method for determining whether any current character in each target character triggers a global sentence segmentation event based on the structural features of each target character. Through this method, punctuation features and sentence segmentation keywords in real-time translated text can be used to locate the current character that triggers the global sentence segmentation event, improving the accuracy of sentence segmentation recognition. Punctuation features and sentence segmentation keywords, as important components of text structure, can intuitively reflect the completeness of sentences and semantic transitions, providing a reliable basis for triggering global sentence segmentation events. Simultaneously, by combining the dual judgment mechanism of punctuation features and sentence segmentation keywords, errors that may arise from single feature judgment can be avoided, further improving the reliability of sentence segmentation recognition.

[0113] In some embodiments of this application, the process of determining whether any current character in each target character triggers a global sentence segmentation event based on the sentence segmentation features of the speech stream segment in step S21 is described in detail, and the steps are as follows:

[0114] S210. Based on the speech stream segment, calculate the pause duration between two adjacent target characters.

[0115] Specifically, speech stream segments can be analyzed, and the pause duration between two adjacent target characters can be calculated based on their timestamps.

[0116] S211. When the pause duration exceeds a preset duration threshold, the target character with the earliest timestamp among the two target characters is taken as the current character that triggers the global sentence segmentation event.

[0117] Specifically, the duration threshold can be the average pause value when different users express adjacent speech clauses.

[0118] The pause duration can be compared with a preset duration threshold;

[0119] If the pause duration exceeds the preset duration threshold, the target character with the earliest timestamp among the two corresponding target characters can be used as the current character to trigger the global sentence segmentation event.

[0120] If the pause duration does not exceed the preset duration threshold, the global sentence segmentation event will not be triggered.

[0121] As can be seen from the above technical solution, this embodiment provides an optional method for determining whether any current character in each target character triggers a global sentence segmentation event based on the structural features of each target character. Using this method, sentence segmentation detection can be completed by analyzing the pause duration between adjacent characters in the speech stream. Since pauses in the speech stream often correspond to semantic segmentation points in the text, by calculating the pause duration and comparing it with a preset threshold, the current character triggering the global sentence segmentation event can be identified relatively accurately.

[0122] In some embodiments of this application, the process of generating a clause of the voice channel based on the current character and determining the start timestamp of the clause in step S2 is described in detail, and the steps are as follows:

[0123] S20. Determine the starting character that the current character matches.

[0124] Specifically, each real-time audio stream is started by the user issuing a voice recording command and ends by the user issuing a voice recording end command. Therefore, each real-time transcribed text is also started by the user issuing a voice recording command and ends by the user issuing a voice recording end command.

[0125] During real-time audio stream recording, speech recognition is performed simultaneously to generate real-time translated text, thereby generating interactive context in real time.

[0126] It can detect whether the previous character exists in the real-time translated text;

[0127] If so, then the character following the previous current character will be used as the starting character of the current character.

[0128] If not, the first character of the real-time translated text will be used as the starting character of the current character.

[0129] S21. Extract the text between the starting character and the current character from the real-time translated text to form the clause.

[0130] Specifically, text content within a range can be extracted from the real-time translated text, starting from the first character and ending at the current character. The extracted text content is a clause.

[0131] S22. Use the timestamp corresponding to the starting character as the start timestamp of the clause.

[0132] Specifically, the timestamp corresponding to the starting character can be used as the start timestamp of the clause.

[0133] As can be seen from the above technical solution, this embodiment provides an optional method for generating clauses of the voice channel based on the current character and determining the start timestamp of the clauses. This method allows for better decomposition of clauses in a real-time voice stream, improving the reliability of the interactive context of this application.

[0134] In some embodiments of this application, the process of generating the interaction context of each voice channel based on the clause and its corresponding start timestamp is described in detail, and the steps are as follows:

[0135] S30. Determine the speaker identifier corresponding to the clause.

[0136] Specifically, the speaker identifier corresponding to each voice channel can be determined.

[0137] Assign each clause of the same voice channel to the speaker identifier of that voice channel.

[0138] S31. Generate an interaction context based on the start timestamp and speaker identifier corresponding to the clause.

[0139] Specifically, clauses and speaker identifiers can be written into the historical interaction context.

[0140] As can be seen from the above technical solution, this embodiment provides an optional method for generating the interaction context of each voice channel based on the clauses and their corresponding start timestamps. This method allows each clause in the interaction context to be associated with a speaker, making the interaction scenario clearer.

[0141] Next, we will combine Figure 2 The speech recognition device provided in this application is described in detail. The speech recognition device described below can be compared with the speech recognition method described above.

[0142] See Figure 2 It can be observed that a voice recognition device may include:

[0143] The determination module 10 is used to determine the various voice channels involved in the voice interaction in response to the voice interaction requirements.

[0144] The recognition module 20 is used to recognize the real-time speech stream input to each speech channel, generate real-time translated text of the real-time speech stream, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream, determine whether the current character triggers a global sentence segmentation event; if so, generate a clause of the speech channel based on the current character, and determine the start timestamp of the clause;

[0145] The generation module 30 is used to generate the interaction context of each voice channel based on the clause and its corresponding start timestamp.

[0146] Furthermore, the identification module 20 may include:

[0147] The speech stream segment determination unit is used to determine each target character corresponding to the current time period in the real-time translated text, and to determine the speech stream segment corresponding to the current time period in the real-time speech stream;

[0148] The target character determination unit is used to determine whether any current character in each target character triggers a global sentence segmentation event based on the structural features of each target character and / or the sentence segmentation features of the speech stream segment.

[0149] Furthermore, the target character determination unit may include:

[0150] The target character analysis component is used to analyze each target character in conjunction with the real-time translated text, and detect whether each target character contains punctuation features and / or sentence break keywords; if it contains sentence break keywords, it selects the current character that triggers the global sentence break event from each target character based on the sentence break keywords; if it contains punctuation features but does not contain sentence break keywords, it uses the target character matched by the punctuation features as the current character that triggers the global sentence break event.

[0151] Furthermore, the target character determination unit may also include:

[0152] A pause duration calculation component is used to calculate the pause duration between two adjacent target characters based on the speech stream segment;

[0153] The pause duration comparison component is used to select the target character with the earlier timestamp among the two corresponding target characters as the current character that triggers the global sentence segmentation event when the pause duration exceeds a preset duration threshold.

[0154] Furthermore, the identification module 20 may also include:

[0155] A starting character determination unit is used to determine the starting character that the current character matches;

[0156] The clause extraction unit is used to extract the text between the starting character and the current character from the real-time translated text to form the clause;

[0157] The start timestamp determination unit is used to use the timestamp corresponding to the start character as the start timestamp of the clause.

[0158] Furthermore, the generation module 30 may include:

[0159] The first generation unit is used to determine the speaker identifier corresponding to the clause;

[0160] The second generation unit is used to generate an interaction context based on the start timestamp and speaker identifier corresponding to the clause.

[0161] Furthermore, the identification module 20 may also include:

[0162] The real-time translated text generation unit is used to recognize the real-time speech stream input to the speech channel using the automatic speech recognition stream (ASR) matched to the speech channel, and generate the real-time translated text of the real-time speech stream.

[0163] The speech recognition device provided in this application embodiment can be applied to speech recognition equipment, such as PC terminals, cloud platforms, servers, and server clusters. Optionally, Figure 3 The hardware structure block diagram of the speech recognition device is shown below. Figure 3 The hardware structure of a voice recognition device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0164] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0165] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0166] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0167] The memory stores a program, which the processor can call. The program is used for:

[0168] In response to the need for voice interaction, identify the various voice channels involved in the voice interaction;

[0169] For each voice channel, the real-time voice stream input to the voice channel is identified, and a real-time translated text of the real-time voice stream is generated, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, it is determined whether the current character triggers a global sentence segmentation event; if so, a clause of the voice channel is generated based on the current character, and the start timestamp of the clause is determined;

[0170] Based on the clauses and their corresponding start timestamps, the interaction context for each voice channel is generated.

[0171] Optionally, the refined and extended functions of the program can be referred to the above description.

[0172] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0173] In response to the need for voice interaction, identify the various voice channels involved in the voice interaction;

[0174] For each voice channel, the real-time voice stream input to the voice channel is identified, and a real-time translated text of the real-time voice stream is generated, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, it is determined whether the current character triggers a global sentence segmentation event; if so, a clause of the voice channel is generated based on the current character, and the start timestamp of the clause is determined;

[0175] Based on the clauses and their corresponding start timestamps, the interaction context for each voice channel is generated.

[0176] Optionally, the refined and extended functions of the program can be referred to the above description.

[0177] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0178] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0179] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. The various embodiments of this application can be combined with each other. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech recognition method, characterized in that, include: In response to the need for voice interaction, identify the various voice channels involved in the voice interaction; For each voice channel, the real-time voice stream input to the voice channel is identified, and a real-time translated text of the real-time voice stream is generated, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time voice stream, it is determined whether the current character triggers a global sentence segmentation event; if so, a clause of the voice channel is generated based on the current character, and the start timestamp of the clause is determined; Based on the clauses and their corresponding start timestamps, the interaction context for each voice channel is generated.

2. The speech recognition method according to claim 1, characterized in that, The determination of whether the current character triggers a global sentence segmentation event based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream includes: Identify each target character in the real-time translated text corresponding to the current time period, and identify the speech stream segment in the real-time speech stream corresponding to the current time period; Based on the structural features of each target character and / or the punctuation features of the speech stream segment, determine whether any current character in each target character triggers a global punctuation event.

3. The speech recognition method according to claim 2, characterized in that, Based on the structural features of each target character, determine whether any current character in each target character triggers a global sentence segmentation event, including: Based on the real-time translated text, each target character is analyzed to detect whether it contains punctuation features and / or sentence break keywords; If the sentence segmentation keyword is included, the current character that triggers the global sentence segmentation event is selected from each target character based on the sentence segmentation keyword; if the punctuation feature is included but the sentence segmentation keyword is not included, the target character matched by the punctuation feature is used as the current character that triggers the global sentence segmentation event.

4. The speech recognition method according to claim 2, characterized in that, Based on the sentence segmentation features of the speech stream segment, determine whether any current character in each target character triggers a global sentence segmentation event, including: Based on the speech stream segment, calculate the pause duration between two adjacent target characters; When the pause duration exceeds a preset duration threshold, the target character with the earlier timestamp among the two target characters is taken as the current character that triggers the global sentence segmentation event.

5. The speech recognition method according to claim 1, characterized in that, The step of generating a clause for the voice channel based on the current character and determining the start timestamp of the clause includes: Determine the starting character that matches the current character; The clause is formed by extracting the text between the starting character and the current character from the real-time translated text. The timestamp corresponding to the starting character is used as the start timestamp of the clause.

6. The speech recognition method according to claim 1, characterized in that, The step of generating the interaction context for each voice channel based on the clause and its corresponding start timestamp includes: Determine the speaker identifier corresponding to the clause; An interaction context is generated based on the start timestamp and speaker identifier corresponding to the clause.

7. The speech recognition method according to any one of claims 1-6, characterized in that, The process of recognizing the real-time speech stream input to the speech channel and generating real-time translated text of the real-time speech stream includes: The Automatic Speech Recognition (ASR) stream matched to the speech channel is used to recognize the real-time speech stream input to the speech channel and generate real-time translated text of the real-time speech stream.

8. A voice recognition device, characterized in that, include: The determination module is used to determine the various voice channels involved in the voice interaction in response to voice interaction needs. The recognition module is used to recognize the real-time speech stream input to each speech channel, generate real-time translated text of the real-time speech stream, wherein the real-time translated text contains a timestamp corresponding to each character; based on the structural features of the real-time translated text and / or the sentence segmentation features of the real-time speech stream, determine whether the current character triggers a global sentence segmentation event; if so, generate a clause for the speech channel based on the current character, and determine the start timestamp of the clause; The generation module is used to generate the interaction context of each voice channel based on the clause and its corresponding start timestamp.

9. A voice recognition device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the speech recognition method as described in any one of claims 1-7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the speech recognition method as described in any one of claims 1-7.

Images