Asynchronous alignment of audio frames with streaming recognized text and accurate dynamic truncation method

Abstract

The application relates to the technical field of voice processing, artificial intelligence and streaming media communication interaction, and provides an asynchronous alignment and accurate dynamic truncation method for audio frames and streaming recognition text. The method comprises the following steps: dividing an original audio stream collected by a microphone into two paths, one of which is continuously transmitted to a cloud voice recognition service, and the other of which is unidirectionally written into a local independent binary audio buffer; after receiving the recognition text and the sentence breaking time axis position issued by the cloud, the conversion relationship between time and audio byte length is determined according to the fixed physical characteristics of audio sampling, the time difference obtained by combining the sentence breaking time axis position and the local global time cursor is obtained, and the target truncation byte length is obtained; according to the target truncation byte length, the audio data corresponding to the bytes is extracted from the head of the audio buffer; after each truncation is completed, the actual time length corresponding to the audio byte length actually extracted from the audio buffer is reversely calculated, and the time cursor calibration is completed.

Description

Technical Field

[0001] This application relates to the fields of speech processing, artificial intelligence and streaming media communication interaction technology, and in particular to a method for asynchronous alignment and precise dynamic truncation of audio frames and streaming recognized text. Background Technology

[0002] In real-time speech interaction scenarios such as real-time speech transcription, simultaneous interpretation, and real-time speech translation, streaming speech recognition technology has become a core supporting technology. Current mainstream streaming speech recognition systems generally adopt an edge-cloud distributed architecture, where the edge audio acquisition device continuously acquires real-time audio streams and transmits them to the cloud-based recognition server. The cloud-based recognition server receives the audio and performs recognition calculations simultaneously, returning the recognized text and sentence segmentation control signals to the edge device.

[0003] Due to network transmission latency between the edge and the cloud, and the computation time of the cloud-based recognition algorithm, there is inevitably a fixed asynchronous time difference between the cloud's return of the sentence segmentation signal and the corresponding recognized text to the edge and the real-time audio stream acquired by the edge. This asynchronous time difference is an inherent physical conflict of the edge-cloud distributed architecture and cannot be completely eliminated through network optimization or computing power improvement. This physical conflict directly results in the following: when the edge receives the sentence segmentation signal of a certain sentence from the cloud, the local audio buffer on the edge has already been written with the beginning part of the audio data of the next sentence spoken by the user.

[0004] To address the aforementioned issues, current mainstream audio truncation solutions in the industry include cache-clearing truncation. This involves directly clearing the entire local audio cache when the endpoint receives the cloud-based sentence segmentation signal. While this solution is simple to implement, it also clears the beginning audio data of the next sentence already written in the cache. In practical applications, this results in severe word swallowing and omission, especially in scenarios where the speaker speaks quickly and there are no obvious pauses between sentences. The audio data loss problem is particularly prominent, directly leading to a significant decrease in the accuracy of downstream speech transcription and machine translation, failing to meet the usage requirements of scenarios such as high-reliability simultaneous interpretation. Summary of the Invention

[0005] This application provides an asynchronous alignment and precise dynamic truncation method for audio frames and streaming text recognition, aiming to solve the problem that existing technologies cannot meet the usage requirements of scenarios such as high-reliability simultaneous interpretation.

[0006] In a first aspect, embodiments of this application provide a method for asynchronous alignment and precise dynamic truncation of audio frames and streaming text, the method comprising: The raw audio stream captured by the microphone is split into two paths: one path is continuously transmitted to the cloud speech recognition service, and the other path is unidirectionally appended to a local independent binary audio buffer. After receiving the recognized text and the timeline position of the sentence segment from the cloud, the conversion relationship between time and audio byte length is determined based on the fixed physical characteristics of audio sampling. The target truncated byte length is obtained by combining the time difference calculated by the sentence segment timeline position and the local global time cursor. Based on the target truncation byte length, the corresponding byte of audio data is extracted from the head of the audio buffer, and the remaining audio data in the audio buffer that exceeds the extraction length is completely preserved and moved to the head of the audio buffer to enter the next processing cycle. After each truncation is completed, the actual duration is calculated in reverse based on the actual length of audio bytes extracted from the audio buffer, and the actual duration is accumulated to the local global time cursor to complete the time cursor calibration.

[0007] In some embodiments, dividing the raw audio stream captured by the microphone into two streams includes: synchronously splitting the raw audio stream continuously captured by the microphone in real time into two audio streams with completely identical content; wherein the audio frame to be sent to the cloud speech recognition service is sent to the cloud speech recognition service through a streaming connection, and the same audio frame is synchronously appended to the local audio buffer. Before the truncation operation is triggered, the audio buffer is appended in the order of audio reception; after the truncation operation is triggered, only the extracted prefix audio data is removed, and the unextracted suffix audio data is completely retained.

[0008] In some embodiments, after receiving the recognized text and the timeline position of the sentence segment from the cloud, determining the conversion relationship between time and audio byte length based on the fixed physical characteristics of audio sampling includes: after receiving the complete recognized text and the corresponding timeline position of the sentence segment from the cloud, not using the local system clock as the time calculation reference, but using the fixed sampling rate, sampling bit depth and number of channels preset when collecting audio as the basic parameters; determining the fixed number of audio bytes corresponding to a single millisecond duration based on the basic parameters, so as to establish a fixed conversion relationship between time length and audio byte length.

[0009] In some embodiments, the step of obtaining the target truncated byte length by combining the time difference calculated from the sentence segmentation timeline position and the local global time cursor includes: subtracting the local global time cursor from the sentence segmentation timeline position issued by the cloud to obtain the audio time difference to be truncated; selectively adding configurable tail note compensation duration based on the sentence segmentation characteristics of the target cloud speech recognition service based on the audio time difference to obtain the final truncated duration; and converting the final truncated duration into the corresponding target truncated byte length according to a fixed conversion relationship.

[0010] In some embodiments, the step of extracting the corresponding byte of audio data from the head of the audio buffer according to the target truncated byte length includes: before performing the audio extraction operation, obtaining the total number of bytes currently stored in the audio buffer; comparing the target truncated byte length with the current total number of bytes in the audio buffer, and taking the smaller of the two values ​​as the actual extracted byte length; extracting the corresponding byte of audio data from the starting position of the head of the audio buffer according to the determined actual extracted byte length, and encapsulating the extracted audio data into an independent audio slice that matches the recognized text.

[0011] In some embodiments, the step of completely preserving and moving the remaining audio data in the audio buffer that exceeds the extraction length to the beginning of the audio buffer for the next processing cycle includes: after completing the audio data extraction operation of the current cycle, not performing a complete clearing operation of the audio buffer; moving the remaining audio data in the audio buffer that exceeds the actual extraction byte length completely and without modification to the beginning of the audio buffer; after the movement is completed, the audio buffer only retains the aforementioned remaining audio data, waiting for the next round of truncation extraction operation.

[0012] In some embodiments, the step of reversely calculating the corresponding actual duration based on the actual length of audio bytes extracted from the audio buffer after each truncation includes: after each truncation operation, obtaining the actual length of audio bytes extracted from the audio buffer; and converting the actual extracted audio byte length into the corresponding actual truncation duration according to a predetermined fixed conversion relationship between time and audio byte length.

[0013] In some embodiments, the actual duration is calculated in reverse based on the actual length of the audio bytes extracted from the audio buffer after each truncation, including: using a fixed conversion relationship to implement the reverse conversion process, which is completely consistent with the fixed conversion relationship used when determining the target truncation byte length; wherein, the reverse conversion process uses only the actual extracted audio byte length as the sole calculation basis.

[0014] In some embodiments, the method further includes: uniquely binding and storing the encapsulated independent audio slices with the corresponding cloud-delivered recognition text, generating audio-text matching data units with time anchors, and outputting the audio-text matching data units to downstream voice playback, voice re-examination, or machine translation processes.

[0015] In some embodiments, obtaining the end time position of the recognized sentence corresponding to the sentence segmentation timeline position includes: when the cloud speech recognition service returns the sentence end time field, the value of the sentence end time field is used as the end time position; when the cloud speech recognition service returns the sentence start offset and the sentence duration, the value of the sentence start offset and the value of the sentence duration are added together to obtain the end time position; when the cloud speech recognition service returns the recognition result end offset, the value of the recognition result end offset is used as the end time position.

[0016] This application divides the original audio stream into two parallel processing paths. One path ensures the continuity and real-time performance of audio uploading to the cloud without interfering with the underlying audio cloud transmission. The other path provides an independent cache data source for precise local truncation, avoiding the operational conflicts between cloud transmission and local truncation in single-path audio processing, and laying a stable architectural foundation for subsequent precise truncation operations.

[0017] Based on the fixed physical characteristics of audio sampling, a conversion relationship between time length and audio byte length is established, completely eliminating the dependence on unstable local system clocks and end-to-cloud synchronization clocks. This fundamentally eliminates the cutting reference error caused by clock drift, providing a stable and definite mathematical benchmark for precise byte-level truncation.

[0018] By extracting the target audio data from the head of the buffer while retaining the remaining audio data and moving it to the head of the buffer, the truncation operation avoids accidentally damaging the audio data at the beginning of subsequent sentences, achieving a lossless truncation effect. Even in extreme scenarios with fast speech and no pauses between sentences, all audio data can be completely retained, significantly improving the processing accuracy of downstream speech transcription, machine translation, and other processes.

[0019] By using the actual extracted audio byte length to inversely calculate the actual truncation time and calibrate the local global time cursor through a closed-loop mechanism, the time cursor is hard-bound to the actual audio byte operation. Each truncation operation can complete precise error calibration, effectively avoiding the generation of cumulative errors. Even if the device runs continuously for tens of hours, the audio-text alignment accuracy can still be maintained at the millisecond level, ensuring the reliability of the device's long-term stable operation.

[0020] The entire solution is based on the underlying audio byte stream for processing, decoupling it from dependence on specific cloud vendors, private communication protocols, and dedicated hardware. It only requires the cloud recognition service to provide the timeline position of sentence breaks to complete the adaptation, which greatly reduces the deployment and adaptation costs of cross-platform and cross-cloud architectures and has certain value for large-scale application.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic flowchart illustrating the steps of an asynchronous alignment and precise dynamic truncation method for audio frames and streaming text recognition provided in an embodiment of this application; Figure 2 This is a software processing flowchart of an embodiment of the present application for an asynchronous alignment and precise dynamic truncation method for audio frames and streaming text recognition. Figure 3 This is a comparative schematic diagram of a traditional word-swallowing error solution provided in an embodiment of this application and the non-destructive truncation process of the present invention; Figure 4 This is a schematic diagram illustrating the principle of vernier calibration provided in one embodiment of this application; Figure 5 This is a schematic block diagram of an asynchronous alignment and precise dynamic truncation system for audio frames and streaming text recognition provided in one embodiment of this application; Figure 6 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0027] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0028] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0029] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] In real-time speech interaction scenarios such as real-time speech transcription, simultaneous interpretation, and real-time speech translation, streaming speech recognition technology has become a core supporting technology. Current mainstream streaming speech recognition systems generally adopt an edge-cloud distributed architecture, where the edge audio acquisition device continuously acquires real-time audio streams and transmits them to the cloud-based recognition server. The cloud-based recognition server receives the audio and performs recognition calculations simultaneously, returning the recognized text and sentence segmentation control signals to the edge device.

[0031] Due to network transmission latency between the edge and the cloud, and the computation time of the cloud-based recognition algorithm, there is inevitably a fixed asynchronous time difference between the cloud's return of the sentence segmentation signal and the corresponding recognized text to the edge and the real-time audio stream acquired by the edge. This asynchronous time difference is an inherent physical conflict of the edge-cloud distributed architecture and cannot be completely eliminated through network optimization or computing power improvement. This physical conflict directly results in the following: when the edge receives the sentence segmentation signal of a certain sentence from the cloud, the local audio buffer on the edge has already been written with the beginning part of the audio data of the next sentence spoken by the user.

[0032] To address the aforementioned issues, current mainstream audio truncation solutions in the industry can be broadly categorized into two types. The first type is cache-clearing truncation, where the endpoint clears all local audio caches upon receiving the cloud-based sentence segmentation signal. While simple to implement, this approach also removes the beginning audio data of the next sentence already written to the cache. In practical applications, this results in severe word loss and omissions, especially in scenarios where the speaker speaks quickly with no clear pauses between sentences. The audio data loss problem is particularly pronounced, directly leading to a significant decrease in the accuracy of downstream speech-to-text and machine translation, failing to meet the requirements of high-reliability simultaneous interpretation and other similar scenarios.

[0033] The second type of solution is a coarse segmentation scheme based on system time. This involves using the local system clock as a reference on the device side, combined with the timestamp of the cloud-based sentence segmentation signal, to roughly segment the local audio cache along the time dimension. This scheme suffers from several insurmountable technical flaws: First, the local system clock and the cloud clock naturally experience clock drift, compounded by timestamp deviations caused by network jitter, resulting in inherent errors in the segmentation reference itself, making high-precision audio-text alignment impossible. Second, this scheme does not establish a deterministic mapping relationship between the physical characteristics of audio sampling and the time dimension, limiting segmentation accuracy to the millisecond level and failing to achieve precise byte-level truncation. Third, the time reference of this scheme continuously relies on an unreliable system clock, introducing new errors with each segmentation operation. Prolonged continuous operation leads to severe cumulative time axis offset errors, resulting in complete audio-text inaccuracy and sentence misalignment after several hours of continuous operation. Fourth, the segmentation logic of this type of scheme still focuses on the complete segmentation of single sentences, neglecting the retention of subsequent audio data already in the cache under asynchronous latency, thus failing to fundamentally prevent truncation operations from damaging subsequent audio data.

[0034] In addition, some industry optimization solutions attempt to alleviate the problems caused by asynchronous latency by optimizing cloud-based silence detection algorithms and shortening cloud-based recognition processing time. However, such solutions can only reduce the asynchronous time difference on the cloud side and cannot fundamentally eliminate the inherent asynchronous latency of the edge-cloud architecture. Furthermore, over-optimizing silence detection algorithms can lead to excessive sentence segmentation and loss of sentence ending sounds, which in turn introduces new recognition errors. Other solutions rely on the proprietary communication protocols and edge-cloud synchronization clock mechanisms of specific cloud vendors to achieve alignment. Such solutions have extremely poor universality, cannot be adapted across platforms and cloud vendors, have extremely high deployment and maintenance costs, and cannot be applied at scale.

[0035] To solve the above problem, please refer to Figure 1 This application provides a method for asynchronous alignment and precise dynamic truncation of audio frames and streaming text recognition, applied to computer devices. The computer devices can be deployed on a single server or server cluster. They can also be deployed on handheld terminals, laptops, wearable devices, or robots, etc. It should be noted that all information involved in the method provided in this application is extracted with the authorization of the relevant users and in accordance with relevant regulations, and will not infringe on user privacy.

[0036] This embodiment provides an asynchronous alignment and precise dynamic truncation method for audio frames and streaming recognized text, applicable to streaming speech recognition scenarios with an edge-cloud distributed architecture, including but not limited to devices and systems such as real-time speech transcription, simultaneous interpretation, real-time speech translation, and real-time voice interaction terminals. The execution subject of this method is an intelligent processing device on the edge side with audio acquisition, data processing, and network communication capabilities, including but not limited to simultaneous interpretation hosts, intelligent recording devices, intelligent terminals with voice interaction functions, and embedded speech processing devices. This method is implemented entirely on the server side, and while not interfering with the computational and data transmission logic of the cloud-based speech recognition service, it converts the sentence segmentation timeline positions of the cloud-based speech recognition service.

[0037] This method establishes a time-byte deterministic mapping relationship based on the physical characteristics of audio sampling through a dual-track parallel audio stream processing architecture. It employs a lossless truncation mechanism to address audio misinterpretation caused by asynchronous latency, and a closed-loop time cursor calibration mechanism to prevent accumulated errors. The provided method for asynchronous alignment and precise dynamic truncation of audio frames and streaming text includes steps S101 to S103. Details are as follows: Step S101. Divide the raw audio stream captured by the microphone into two paths. One path is continuously transmitted to the cloud speech recognition service, while the other path is unidirectionally appended to a local independent binary audio buffer.

[0038] Specifically, this step is the basic audio data processing stage of the entire method. The core is to achieve complete decoupling between real-time cloud recognition and transmission and local audio buffer truncation by synchronously splitting and processing the audio stream in parallel, thereby avoiding operational conflicts between the two processing links and ensuring the integrity and continuity of the audio data.

[0039] The device continuously acquires human voice signals from the environment in real time via a microphone, converting the acquired voice signals into a raw audio stream in a fixed format. This raw audio stream is a binary audio data stream in pulse code modulation format, with its sampling rate, sampling bit depth, and number of channels all preset and fixed parameters that remain constant throughout the entire process. The device's audio processing module synchronously splits this raw audio stream, generating two identical and synchronously synchronized sub-audio streams. These two sub-audio streams enter two independent processing links, operating in parallel without interference. The first sub-audio stream enters the cloud transmission link and is transmitted uninterruptedly to the cloud-based speech recognition service via a long-term, unbiased connection. During this transmission, the link does not perform any buffering, truncation, modification, or deletion operations on the audio stream; it only performs real-time forwarding of the raw audio stream, ensuring that the cloud-based speech recognition service can continuously and completely receive the real-time audio stream acquired by the device, thus ensuring that the real-time performance and accuracy of cloud-based recognition are unaffected. The second sub-audio stream enters the local buffer processing link and is appended unidirectionally to a pre-allocated independent binary audio buffer on the endpoint, following the chronological order of audio capture. This audio buffer is a contiguous, fixed-address binary data storage area in the endpoint's local memory, used solely to store the audio data of the second sub-audio stream. During the write process, only a unidirectional append operation is performed; that is, newly captured audio data is appended only to the end of the buffer. No active deletion, clearing, or data modification operations are performed, ensuring that the audio data stored in the buffer is completely consistent with the audio stream captured by the microphone in timing and is complete and without any missing content. This provides a complete and continuous data source for subsequent precise truncation operations.

[0040] Step S102. After receiving the recognized text and the time axis position of the sentence segmentation sent from the cloud, determine the conversion relationship between time and audio byte length based on the fixed physical characteristics of audio sampling, and obtain the target truncated byte length by combining the time difference calculated by the time axis position of the sentence segmentation and the local global time cursor.

[0041] Specifically, this step is the core of the accuracy of the entire method. The core is to get rid of the dependence on the unstable local system clock, establish a deterministic mapping relationship between time and audio byte length based on the fixed physical characteristics of audio sampling, and calculate the accurate target truncation byte length by combining the sentence break time axis position returned by the cloud, so as to provide accurate numerical basis for subsequent truncation operations.

[0042] During step S101, the client continuously monitors data packets sent by the cloud-based speech recognition service. Upon receiving the complete recognized text from the cloud, along with the corresponding sentence break timeline position, this step is triggered. The sentence break timeline position is the absolute timestamp of the end time of the audio segment corresponding to the recognized text, as identified by the cloud-based speech recognition service. The time reference of this timestamp is consistent with the time reference of the audio stream received by the cloud-based speech recognition service. After receiving the data, the client does not use the local system clock as the time calculation reference. Instead, it uses the fixed sampling rate, sampling bit depth, and number of channels pre-set in step S101 for acquiring the original audio stream as the sole basic calculation parameters. These parameters are determined before audio acquisition begins and remain constant throughout the entire method execution, without dynamic fluctuations. Therefore, deterministic calculation results can be obtained based on these parameters. Based on these basic parameters, the client calculates the fixed number of audio bytes corresponding to a single millisecond duration, thereby establishing a fixed conversion relationship between time length and audio byte length. This fixed conversion relationship is determined solely by the inherent physical parameters of the audio sampling and is unaffected by network status, system clock, or device operating status. Therefore, it possesses absolute determinism and stability, enabling error-free bidirectional conversion between time length and audio byte length. The client acquires a locally maintained global time cursor, which records the latest time node corresponding to the currently extracted truncated audio data. Its initial value is the absolute time value at the start of audio acquisition, and it is calibrated and updated after each truncation operation. The client subtracts the local global time cursor from the sentence segmentation timeline position sent from the cloud to obtain the audio time difference to be truncated. This time difference corresponds to the time length of the audio data matching the recognized text to be extracted from the audio buffer. Based on the obtained audio time difference and the pre-established fixed conversion relationship between time length and audio byte length, the client converts the audio time difference into the corresponding byte length value, thus obtaining the target truncated byte length for this truncation operation.

[0043] Step S103. Based on the target truncation byte length, extract the corresponding byte of audio data from the head of the audio buffer, and completely retain the remaining audio data in the audio buffer that exceeds the extraction length and move it to the head of the audio buffer to enter the next processing cycle; after each truncation, calculate the corresponding actual duration in reverse based on the actual length of audio bytes extracted from the audio buffer, and accumulate the actual duration to the local global time cursor to complete the time cursor calibration.

[0044] Specifically, this step is the core execution link of the entire method. The core is to solve the problem of the next audio sentence being mistakenly affected by asynchronous latency through the lossless truncation mechanism of left cutting and right retention. At the same time, through the reverse calibration mechanism based on the actual number of bytes extracted, the closed-loop calibration of the global time cursor is realized, fundamentally eliminating the generation of accumulated errors.

[0045] This step is divided into two sub-steps: audio lossless truncation and time vernier closed-loop calibration. The two sub-steps are executed sequentially, with each truncation operation corresponding to one calibration operation, forming a complete closed-loop processing flow.

[0046] The first sub-step is the lossless audio truncation operation. Specifically, the endpoint extracts the corresponding number of audio bytes from the beginning of the local binary audio buffer based on the target truncation byte length calculated in step S102. After extraction, instead of clearing the entire audio buffer, the remaining audio data exceeding the actual extracted byte length is moved intact to the beginning of the audio buffer, making it the new starting data. After the move, only the remaining audio data is retained in the audio buffer. Newly acquired audio data is appended to the end of the buffer, and this remaining audio data naturally enters the next round of truncation extraction processing. Through this operation, even if the endpoint receives a sentence segmentation signal from the cloud and the buffer has already contained the beginning of the next sentence's audio data, this portion of audio data will be completely preserved in the buffer and will not be accidentally damaged or cleared by the truncation operation. This fundamentally solves the problem of word swallowing and omission caused by asynchronous latency in traditional solutions.

[0047] The second sub-step is the time cursor closed-loop calibration operation. Specifically, after completing the audio data extraction operation for the current round, the endpoint obtains the actual length of audio bytes extracted from the audio buffer. This value represents the actual number of bytes of audio data processed in this truncation operation, eliminating any discrepancy between theoretical calculations and actual operation. Based on the fixed conversion relationship between time length and audio byte length established in step S102, the endpoint reverse-converts the actual extracted audio byte length into the corresponding actual truncation duration. This actual truncation duration is the true time length corresponding to the extracted audio data. The endpoint then adds the actual truncation duration obtained from the reverse conversion to the local global time cursor, completing the global time cursor calibration update after this truncation operation. The calibrated global time cursor perfectly matches the time node corresponding to the new starting position of the audio buffer after this truncation operation, achieving a hard binding between the time cursor and the actual audio byte operation. Through the aforementioned closed-loop calibration mechanism, each truncation operation calibrates the time cursor based on the actual number of audio bytes processed, without introducing any errors caused by theoretical calculations. This avoids the accumulation of errors during long-term continuous operation and ensures the long-term accuracy of audio-text alignment.

[0048] In some embodiments, dividing the raw audio stream captured by the microphone into two streams includes: synchronously splitting the raw audio stream continuously captured by the microphone in real time into two audio streams with completely identical content; wherein the audio frame to be sent to the cloud speech recognition service is sent to the cloud speech recognition service through a streaming connection, and the same audio frame is synchronously appended to the local audio buffer. Before the truncation operation is triggered, the audio buffer is appended in the order of audio reception; after the truncation operation is triggered, only the extracted prefix audio data is removed, and the unextracted suffix audio data is completely retained.

[0049] This embodiment is a further detailed implementation of step S101. In this embodiment, the terminal continuously acquires human voice signals from the environment in real time through a microphone. After analog-to-digital conversion, a pulse code modulation (PCM) original audio stream with a fixed format is generated. In this embodiment, the sampling rate, sampling bit depth, and number of channels of the original audio stream are preset in a configuration file before audio acquisition begins. After being set, the settings remain constant throughout the entire audio acquisition and processing process and are not dynamically modified. The terminal audio processing module sets a synchronous splitting node in the output link of the original audio stream to perform synchronous copying and splitting processing on the original audio stream, generating two sub-audio streams with completely identical content, completely synchronized timing, and identical format parameters. During the splitting process, no modification, delay, or truncation is performed on the original audio stream to ensure that there are no content differences or timing deviations between the two sub-audio streams and the original audio stream. After the splitting is completed, the first sub-audio stream is directly sent to the terminal network communication module. The network communication module transmits the sub-audio stream to the cloud speech recognition service in real time without any difference through a pre-established uninterrupted long-term communication link with the cloud speech recognition service. During the pass-through process, the network communication module only performs real-time forwarding of audio data and does not perform any caching, splicing, truncation, modification, or deletion operations on the sub-audio stream. Even in the event of brief network jitter, it only performs retransmission according to network transmission rules, without changing the original content and timing of the audio stream, ensuring that the cloud-based speech recognition service can receive a real-time audio stream that is completely consistent with the microphone acquisition. After splitting, the second sub-audio stream is directly sent to the local cache management module on the edge. The cache management module unidirectionally appends the sub-audio stream to a pre-allocated independent binary audio buffer in the local memory on the edge, according to the acquisition time sequence.

[0050] In this embodiment, the audio buffer is a pre-allocated, fixed-size, contiguous memory space. It employs a circular queue storage management mechanism, supporting only append-to-end and read-from-head operations, and does not support write, modify, or delete operations at random positions. During audio data writing, the buffer management module only appends newly received audio data to the end of the buffer, without performing any active deletion, clearing, or data modification operations. This ensures that the audio data stored in the buffer is completely consistent in timing with the original audio stream captured by the microphone, and that the content is complete and without gaps, providing a continuous and complete data source for subsequent precise truncation operations.

[0051] In some embodiments, after receiving the recognized text and the timeline position of the sentence segment from the cloud, determining the conversion relationship between time and audio byte length based on the fixed physical characteristics of audio sampling includes: after receiving the complete recognized text and the corresponding timeline position of the sentence segment from the cloud, not using the local system clock as the time calculation reference, but using the fixed sampling rate, sampling bit depth and number of channels preset when collecting audio as the basic parameters; determining the fixed number of audio bytes corresponding to a single millisecond duration based on the basic parameters, so as to establish a fixed conversion relationship between time length and audio byte length.

[0052] This embodiment is a further refined implementation of step S102, which involves determining the conversion relationship between time and audio byte length based on the fixed physical characteristics of audio sampling after receiving the recognized text and sentence segmentation timeline position from the cloud. During step S101, the client continuously monitors the communication messages sent by the cloud-based speech recognition service and parses them. When the complete recognized text content and the sentence segmentation timeline position uniquely bound to the recognized text are obtained, the processing flow of this embodiment is triggered. In this embodiment, the sentence segmentation timeline position is the absolute time value generated by the cloud-based speech recognition service based on the time reference of the received audio stream, corresponding to the end time of the sentence to which the recognized text belongs. This timestamp corresponds one-to-one with the recognized text, uniquely identifying the end time node of the sentence. After obtaining the above data, the client does not use the local system clock time value as the time calculation reference to avoid the impact of clock drift, jumps, or errors between the local system clock and the cloud clock on the calculation results. The device directly retrieves the three basic parameters—fixed sampling rate, sampling bit depth, and number of channels—pre-set in step S101 for acquiring the original audio stream. These three parameters are inherent physical parameters of audio sampling, configured before audio acquisition begins, and remain constant throughout the entire method execution process without any dynamic fluctuations. Therefore, the conversion relationship calculated based on these parameters has absolute determinism.

[0053] Based on the three fundamental parameters mentioned above, the endpoint calculates the fixed number of audio bytes corresponding to a single millisecond duration. The specific calculation logic is as follows: the number of audio sampling points within a single millisecond duration is calculated based on the sampling rate; the number of bytes corresponding to a single sampling point is calculated based on the sampling bit depth; and finally, the total number of fixed audio bytes corresponding to a single millisecond duration is calculated by combining this with the number of channels. For example, when the original audio stream uses a 16kHz sampling rate, a 16-bit sampling bit depth, and a mono configuration, the fixed number of audio bytes corresponding to a single millisecond duration can be calculated, thus establishing a fixed conversion relationship. Based on this total number of fixed audio bytes, the endpoint establishes a fixed conversion relationship between time length and audio byte length. This conversion relationship is a bidirectional, unique, deterministic relationship; that is, any time length can be uniquely converted to a corresponding byte length, and any byte length can be uniquely converted to a corresponding time length. In this embodiment, the fixed conversion relationship is calculated and stored before the audio acquisition starts and remains constant throughout the audio processing. It is only recalculated and updated when the audio acquisition sampling rate, sampling bit depth, and number of channels are changed, ensuring the stability and consistency of the conversion relationship and providing a unified and unbiased calculation benchmark for subsequent truncation length calculation and time vernier calibration.

[0054] In some embodiments, the step of obtaining the target truncated byte length by combining the time difference calculated from the sentence segmentation timeline position and the local global time cursor includes: subtracting the local global time cursor from the sentence segmentation timeline position issued by the cloud to obtain the audio time difference to be truncated; selectively adding configurable tail note compensation duration based on the sentence segmentation characteristics of the target cloud speech recognition service based on the audio time difference to obtain the final truncated duration; and converting the final truncated duration into the corresponding target truncated byte length according to a fixed conversion relationship.

[0055] This embodiment is a further refined implementation of step S102, which involves calculating the target truncated byte length by combining the time difference obtained from the sentence segmentation timeline position and the local global time cursor. After confirming the fixed conversion relationship on the client side, the locally maintained global time cursor is retrieved. This global time cursor records the absolute time value of the latest time node corresponding to the currently truncated and extracted audio data. Its initial value is the absolute time value of the moment when the microphone starts audio acquisition. After each complete truncation operation, it is updated through a closed-loop calibration mechanism to ensure that the global time cursor is always consistent with the time node corresponding to the beginning position of the audio buffer. On the client side, the sentence segmentation timeline position sent from the cloud is used as the endpoint value, and the local global time cursor is used as the starting value. The difference between the endpoint value and the starting value is calculated. This difference is the audio time difference that needs to be truncated this time. This time difference corresponds to the original time length of the audio data that needs to be extracted from the audio buffer and matches the currently recognized text. To prevent excessive silence detection and sentence segmentation in the cloud-based speech recognition service, which could lead to the loss of the final consonant of the recognized text being excluded from the sentence segmentation timeline and thus causing contamination of the next sentence, the client-side adds a pre-set fixed duration for final consonant compensation to the calculated audio time difference, resulting in the final truncation duration. In this embodiment, the final consonant compensation duration can be set to 300 milliseconds, but can also be adjusted according to the needs of the actual application scenario to ensure complete coverage of the final consonant of the sentence and avoid loss of the final consonant.

[0056] Based on the fixed conversion relationship between the established time length and the audio byte length, the terminal side converts the final truncation time obtained above into the corresponding byte length value, which is the target truncation byte length for this truncation operation. This embodiment solves the problem of sentence tail sound loss caused by insufficient accuracy of cloud-based silence detection by increasing the tail sound compensation time, ensuring that the extracted audio data can completely cover the entire sentence speech content corresponding to the recognized text, and improving the completeness and accuracy of audio-text matching.

[0057] In some embodiments, the step of extracting the corresponding byte of audio data from the head of the audio buffer according to the target truncated byte length includes: before performing the audio extraction operation, obtaining the total number of bytes currently stored in the audio buffer; comparing the target truncated byte length with the current total number of bytes in the audio buffer, and taking the smaller of the two values ​​as the actual extracted byte length; extracting the corresponding byte of audio data from the starting position of the head of the audio buffer according to the determined actual extracted byte length, and encapsulating the extracted audio data into an independent audio slice that matches the recognized text.

[0058] This embodiment is a further refined implementation of step S103, which involves extracting the corresponding bytes of audio data from the head of the audio buffer based on the target truncated byte length. After obtaining the target truncated byte length calculated in step S102 on the client side, before performing the audio data extraction operation, the total number of bytes corresponding to the audio data already stored but not yet extracted in the current binary audio buffer is obtained through the local cache management module. This value represents the maximum byte length of the audio data that can be extracted from the current buffer. To prevent extreme network jitter and excessive asynchronous latency from causing the target truncated byte length to exceed the current total number of bytes in the buffer, thereby triggering memory out-of-bounds errors and data read errors, a dynamic boundary protection check operation is performed on the client side. The target truncated byte length is compared with the current total number of bytes in the buffer, and the smaller of the two values ​​is taken as the actual extracted byte length for this truncation operation. Through this check operation, it is ensured that the extracted byte length will not exceed the actual audio data byte length stored in the buffer, fundamentally avoiding the problem of memory out-of-bounds reads and ensuring the stability of the method under extreme network environments.

[0059] After determining the actual length of bytes to be extracted, the client-side continuously reads a number of audio data points corresponding to the actual length of bytes to be extracted, starting from the beginning of the binary audio buffer, thus completing the audio data extraction operation. After extraction, the client-side encapsulates the extracted audio data into independent audio slice files. These audio slice files have the same format as the original audio stream and are uniquely bound to the currently distributed recognition text from the cloud. These slices serve as the original audio data for matching the recognition text and are used in subsequent steps. This embodiment solves the memory overflow problem in extreme network environments through a dynamic boundary protection verification mechanism, ensuring the stability of the method. Simultaneously, the encapsulation of audio slices achieves a one-to-one correspondence between audio data and recognition text, providing accurate audio-text matching data for downstream applications.

[0060] In some embodiments, the step of completely preserving and moving the remaining audio data in the audio buffer that exceeds the extraction length to the beginning of the audio buffer for the next processing cycle includes: after completing the audio data extraction operation of the current cycle, not performing a complete clearing operation of the audio buffer; moving the remaining audio data in the audio buffer that exceeds the actual extraction byte length completely and without modification to the beginning of the audio buffer; after the movement is completed, the audio buffer only retains the aforementioned remaining audio data, waiting for the next round of truncation extraction operation.

[0061] This embodiment is a further refined implementation of step S103, which involves completely preserving and moving the remaining audio data exceeding the extraction length in the audio buffer to the beginning of the audio buffer before entering the next processing cycle. After the audio data extraction operation is completed on the client side, the entire audio buffer is not cleared or reset to avoid accidental deletion of subsequent audio data stored in the buffer. The client side uses the cache management module to obtain the remaining audio data in the audio buffer that exceeds the actual extracted byte length. This part of the audio data consists of all audio data stored in the buffer after the audio data extracted in this operation that has not yet been extracted, including the beginning part of the next sentence spoken by the user that entered the buffer early due to asynchronous latency between the client and cloud. The client side uses low-level memory operations to move the remaining audio data completely and without modification to the beginning position of the audio buffer, making the first byte of the remaining audio data the new beginning byte of the audio buffer.

[0062] During the movement, no modification, deletion, splicing, or format conversion is performed on the remaining audio data, ensuring that the content and timing of the remaining audio data are completely consistent with the original audio stream. After the remaining audio data is moved, the cache management module on the device side resets the write pointer of the audio buffer to the end of the remaining audio data. Subsequent newly acquired audio data continues to be appended to the end of the buffer, achieving seamless splicing with the remaining audio data. The remaining audio data retained in the buffer naturally enters the next round of truncation and extraction processing cycle. When the cloud-based sentence segmentation signal is received again, it participates in the truncation and extraction operation as the starting part of the audio data for the next sentence. This embodiment avoids accidental damage to the audio data at the beginning of subsequent sentences by losslessly retaining the remaining audio data and moving the head, fundamentally solving the problem of swallowing or missing words caused by asynchronous latency between the device and the cloud in traditional solutions. Even in scenarios where the speaker speaks very fast and there are no obvious pauses between sentences, all audio data can be completely retained, ensuring the accuracy of speech recognition and subsequent processing.

[0063] In some embodiments, the step of reversely calculating the corresponding actual duration based on the actual length of audio bytes extracted from the audio buffer after each truncation includes: after each truncation operation, obtaining the actual length of audio bytes extracted from the audio buffer; and converting the actual extracted audio byte length into the corresponding actual truncation duration according to a predetermined fixed conversion relationship between time and audio byte length.

[0064] This embodiment is a further refined implementation of step S103, in which the actual duration is calculated in reverse based on the actual length of audio bytes extracted from the audio buffer after each truncation. Specifically, after the terminal completes the audio data extraction operation for the current round, the processing flow of this embodiment is immediately triggered. First, the actual length of audio bytes extracted from the audio buffer is obtained. This value is the actual number of bytes of audio data actually read from the buffer in this operation, which is determined after boundary protection verification in embodiment 4. It is the only true measurement value of the audio data actually processed in this truncation operation, and there is no deviation between the theoretical calculation value and the actual operation value.

[0065] The endpoint retrieves a pre-established fixed conversion relationship between time length and audio byte length. This conversion relationship is completely consistent with the one used when calculating the target truncated byte length, ensuring a unified benchmark for both forward and reverse calculations, with no benchmark deviation. The endpoint uses the actual extracted audio byte length as the sole calculation basis. Based on the aforementioned fixed conversion relationship, the actual extracted audio byte length is converted in reverse to the corresponding actual truncated duration. This actual truncated duration is the true time length corresponding to the extracted audio data, completely consistent with the actual playback duration of the extracted audio data. This embodiment obtains the true time length corresponding to the truncation operation through reverse calculation based on the actual extracted byte length, providing unbiased, true data for subsequent global time cursor calibration and avoiding the error accumulation problem caused by calibration based on theoretical calculations.

[0066] In some embodiments, the actual duration is calculated in reverse based on the actual length of the audio bytes extracted from the audio buffer after each truncation, including: using a fixed conversion relationship to implement the reverse conversion process, which is completely consistent with the fixed conversion relationship used when determining the target truncation byte length; wherein, the reverse conversion process uses only the actual extracted audio byte length as the sole calculation basis.

[0067] In this embodiment, the specific implementation of the reverse calculation of the corresponding actual duration based on the actual audio byte length extracted from the audio buffer after each truncation is as follows: When the reverse calculation process is performed on the terminal side, the fixed conversion relationship used is completely consistent with the fixed conversion relationship used in step S102 when calculating the target truncation byte length. This includes the calculation base parameters and calculation logic of the conversion relationship being completely identical. Moreover, this fixed conversion relationship remains constant throughout the forward calculation and reverse calculation process after the audio acquisition is started, without any changes, ensuring that the bidirectional conversion between time length and byte length is based on a completely unified base, fundamentally avoiding system errors caused by inconsistent conversion bases.

[0068] In this embodiment, the reverse conversion process uses only the actual length of the audio bytes extracted from the audio buffer as the sole calculation basis. It does not incorporate any other parameters, such as the timeline position of the segment sent from the cloud, the local system clock time value, or the theoretically calculated target truncation byte length. This ensures that the actual truncation duration obtained through reverse calculation completely corresponds to the true duration of the audio data actually processed, unaffected by external factors such as cloud timestamp deviation, network latency, or system clock errors. After the reverse calculation is completed on the device side, the obtained actual truncation duration is uniquely bound and stored with the extracted audio data as the true duration parameter corresponding to the audio slice. Simultaneously, this actual truncation duration is input into the global time cursor calibration stage to update the time cursor. This embodiment, by limiting the conversion benchmark and calculation basis of the reverse calculation, further ensures the accuracy and uniqueness of the reverse calculation results, eliminates errors caused by conversion benchmark deviation and external interference parameters, and provides a reliable guarantee for the accurate calibration of the global time cursor.

[0069] In some embodiments, the method further includes: uniquely binding and storing the encapsulated independent audio slices with the corresponding cloud-delivered recognition text, generating audio-text matching data units with time anchors, and outputting the audio-text matching data units to downstream voice playback, voice re-examination, or machine translation processes.

[0070] This embodiment is an extended application implementation based on the above core steps. In this embodiment, the method also includes steps for binding audio-text matching data and downstream application processing. After the audio slice encapsulation operation is completed on the client side, the encapsulated independent audio slice is uniquely bound and stored with the corresponding cloud-delivered recognition text. During the binding process, the client side generates a unique identifier for the audio slice and the recognition text, and associates and stores parameters such as the sentence break timeline position, global time cursor value, and actual truncation duration corresponding to this truncation operation with the identifier, generating an audio-text matching data unit with time anchors.

[0071] In this embodiment, each audio-text matching data unit contains a uniquely corresponding recognized text, original audio slice, time anchor parameters, and duration parameters, achieving a precise one-to-one correspondence between text content and audio data, eliminating temporal misalignment and content deviation. After generating the audio-text matching data unit on the client side, it is output to downstream stages such as voice playback, voice verification, machine translation, and voice content archiving, according to the needs of the actual application scenario. In the voice playback scenario, the recognized text and corresponding audio can be synchronously played back based on the audio-text matching data unit; in the machine translation scenario, the audio slice and recognized text in the audio-text matching data unit can be synchronously input into the translation model to improve translation accuracy; in the voice verification scenario, the accuracy of the recognized text can be manually or automatically verified based on the audio-text matching data unit. This embodiment, through the generation and output of audio-text matching data units, provides precisely aligned audio-text data for various downstream voice processing applications, expanding the application scenarios of this method and improving the performance and availability of the entire real-time voice processing system.

[0072] In some embodiments, the method further includes: after each preset number of truncation operations, performing a consistency check between the cumulative byte length corresponding to the local global time cursor and the total byte length cumulatively written to the audio buffer; if a deviation occurs in the check, correcting the global time cursor based on the actual cumulative byte length written to the audio buffer to eliminate abnormal deviations.

[0073] This embodiment is a supplementary implementation of the deviation prevention method based on the core steps described above. In this embodiment, the method further includes a periodic consistency verification and deviation correction step. Verification triggering conditions are preset on the device side. In this embodiment, the verification triggering condition is a preset number of truncation operations. The preset number is a fixed value preset through a configuration file and can be adjusted according to the device's operating scenario and runtime requirements. Whenever the preset number of truncation operations is reached on the device side, the consistency verification process of this embodiment is automatically triggered.

[0074] After triggering the verification process, the client first obtains the current value of the local global time cursor. Based on the established fixed conversion relationship, it converts the cumulative duration corresponding to the global time cursor into the corresponding cumulative total length of extracted bytes. This value is the total length of audio data extracted from the buffer, calculated based on the time cursor. Simultaneously, the client, through the cache management module, obtains the total length of bytes written to the audio buffer since audio acquisition began, as well as the length of bytes of unextracted audio data remaining in the buffer. By subtracting the remaining unextracted bytes from the total length of bytes written, the actual cumulative total length of extracted bytes in the buffer is calculated. This value represents the true cumulative number of bytes extracted from the buffer, with no calculation bias.

[0075] The endpoint compares the total length of extracted bytes calculated based on the time cursor with the actual total length of extracted bytes in the buffer. If the two values ​​are completely identical, the verification passes and no correction is performed. If there is a discrepancy, the verification fails and a deviation correction process is triggered. In the deviation correction process, the endpoint uses the actual total length of extracted bytes in the buffer as the sole reference. Based on a fixed conversion relationship, the actual total length of extracted bytes is converted in reverse to the corresponding actual cumulative duration. Using this actual cumulative duration as the reference, the local global time cursor is corrected and updated to ensure that the corrected global time cursor perfectly matches the duration of the actually extracted audio data, eliminating abnormal deviations.

[0076] This embodiment further eliminates abnormal deviations that may occur during long-term continuous operation of the equipment by periodically verifying consistency and correcting deviations. This ensures that the method can maintain millisecond-level audio-text alignment accuracy even in long-term operation scenarios, and further improves the reliability of long-term stable operation of the equipment.

[0077] In some embodiments, obtaining the end time position of the recognized sentence corresponding to the sentence segmentation timeline position includes: when the cloud speech recognition service returns the sentence end time field, the value of the sentence end time field is used as the end time position; when the cloud speech recognition service returns the sentence start offset and the sentence duration, the value of the sentence start offset and the value of the sentence duration are added together to obtain the end time position; when the cloud speech recognition service returns the recognition result end offset, the value of the recognition result end offset is used as the end time position.

[0078] This embodiment addresses the industry issue of inconsistent time parameter formats returned by different cloud-based speech recognition services by providing a multi-format adaptation method for determining the end time position corresponding to the sentence segmentation timeline position. This further expands the method's cross-vendor and cross-platform adaptability. During step S102, after receiving the communication message from the cloud-based speech recognition service, the client first performs full-field parsing of the message, extracting the time-related fields uniquely bound to the currently recognized text. Based on the extracted field types, the corresponding method is automatically matched to determine the end time position corresponding to the sentence segmentation timeline position. The final end time position value is the sentence segmentation timeline position value used in step S102 to calculate the audio time difference. In this embodiment, the end time position is determined using any of the following methods. The client automatically triggers the corresponding execution logic based on the actual parsed field types, without requiring manual pre-configuration of adaptation rules: The first determination method: After the terminal side completes the parsing of the communication message and confirms that the cloud-based speech recognition service has returned the sentence end time field, the value stored in the sentence end time field is directly extracted and used as the value of the end time position, thus obtaining the sentence break timeline position corresponding to this truncation operation. This method is suitable for cloud-based speech recognition services that directly return the absolute timestamp of the sentence end. The terminal side can directly extract and use it without additional calculation processing, resulting in optimal execution efficiency and compatibility with the standard return format of most mainstream commercial cloud-based speech recognition services.

[0079] The second determination method: After the terminal side completes the parsing of the communication message, if it confirms that the cloud-based speech recognition service did not return the sentence end time field, but instead returned the sentence start offset and sentence duration fields, first extract the values ​​stored in the sentence start offset field and the sentence duration field respectively. Then, add the sentence start offset value and the sentence duration value together. The result is the value of the end time position, which is the sentence break timeline position corresponding to this truncation operation. This method is suitable for lightweight cloud-based speech recognition services that only return the sentence start time reference and sentence duration. The accurate sentence end time can be obtained through fixed calculation logic, and adaptation can be completed without modifying the cloud service.

[0080] The third determination method: After the terminal side completes the parsing of the communication message, if it confirms that the cloud-based speech recognition service has not returned the sentence end time field, nor the sentence start offset and sentence duration fields, but has returned the recognition result end offset field, then the value stored in the recognition result end offset field is directly extracted. This extracted value is used directly as the value of the end time position, thus obtaining the sentence break timeline position corresponding to this truncation operation. This method is suitable for cloud-based speech recognition service architectures that use audio stream byte offsets to identify the sentence end position. It can be directly adapted to private cloud services that return sentence break information based on audio stream time sequence offsets, further expanding the adaptability of the method.

[0081] In this embodiment, the three determination methods share the same message parsing trigger logic. The end only needs to automatically match the corresponding execution method according to the actual field type obtained by parsing. There is no need to develop customized code for specific cloud vendors' services in advance. It can adapt to the return format of most time parameters of the current mainstream cloud speech recognition services, further reducing the adaptation and transformation cost under different cloud architectures. This fully echoes the core beneficial effect of the invention's strong versatility and cross-platform capability.

[0082] In some embodiments, such as Figure 2 As shown, this application constructs an end-to-end closed-loop processing flow, specifically including the following steps: Audio Acquisition and Buffering: Continuous real-time voice signals are acquired through a microphone, converted into PCM binary audio streams, and written into a continuous buffer queue for storage, forming a continuous audio data stream that can be read randomly. ASR recognition result reception: Receive the real-time recognition result returned by the cloud ASR engine through an asynchronous communication interface. The recognition result carries the absolute end timestamp of the current recognition statement. Timestamp extraction: Extract the absolute end timestamp corresponding to the current recognized sentence from the recognition results returned by the cloud ASR. The absolute end timestamp is the absolute time position of the end of the current sentence in the continuous audio stream. Byte count calculation: Substitute the extracted absolute end timestamp into the timestamp to byte count formula to calculate the number of audio bytes N that should be cut off; Audio truncation execution: Input the PCM audio stream in the continuous buffer queue into the audio splitter, and perform audio truncation operation based on the calculated number of bytes N; Left-cut audio output: The audio splitter performs a left-cut operation, extracts the first N bytes of audio data from the audio stream, encapsulates this part of the audio data into an audio segment that absolutely matches the currently recognized statement, and completes the output and archiving of the audio of the current statement; Right-left audio buffer: The audio splitter performs a right-left operation, which retains the remaining audio data after N bytes in the audio stream in the PCM binary persistent buffer queue without loss by shifting the pointer N bytes forward. This data serves as the beginning audio data of the next recognition statement, waiting for the next truncation operation.

[0083] Furthermore, in this embodiment, after each left-cut output and right-buffered operation is completed, a cursor calibration step is performed to selectively compensate for the tail tone of the audio stream reading cursor (the tail tone compensation duration is a configurable parameter, and its value can be from 0 to 300 milliseconds; in one embodiment, for ASR services with severe sentence breaks, the tail tone compensation duration is set to 300 milliseconds; in another embodiment, the tail tone compensation duration is set to 0 milliseconds), to eliminate the accumulated error in the continuous truncation process.

[0084] In some embodiments, such as Figure 3 As shown, in traditional audio truncating schemes, for real-time audio streams continuously captured by a microphone, when the truncating result returned by the ASR is obtained, the actual point of truncating is directly used as the termination position of the audio segmentation, and the audio data between the actual point of the end of the sentence and the actual point of truncating is directly discarded. For example... Figure 3As shown on the left, the discarded audio data corresponds to a word loss of 6400 bytes, which directly causes the speech content corresponding to the audio segment to be unrecognizable, resulting in word loss and incomplete recognition. Moreover, this error will continue to accumulate with the continuous recognition process, seriously affecting the accuracy and consistency of real-time speech recognition.

[0085] This embodiment abandons the traditional approach of directly discarding the audio data after the truncation point and adopts a lossless truncation mechanism of left-cut output and right-buffered buffer. For the continuously buffered PCM audio stream data, the cutting point is determined according to the sentence segmentation result returned by ASR. The cutting point corresponds to the absolute byte position of the end of the currently recognized sentence in the audio stream. Perform a left cut operation: Using the cut point as the termination boundary, extract N bytes of audio data before the cut point from the audio stream, encapsulate this part of the audio data into an audio segment that absolutely matches the currently recognized sentence, and complete the output of the current sentence audio; Perform the right-leave operation: retain all remaining audio data after the cut point losslessly in the audio buffer queue, use this remaining audio data as the beginning data of the next recognition sentence, and wait for the next sentence truncation operation to process it.

[0086] The beneficial effects of this embodiment are as follows: by using the lossless truncation mechanism of "left cut and right retention", the problem of audio data loss during sentence truncation in traditional schemes is eliminated, fundamentally solving the word swallowing error in real-time ASR scenarios; at the same time, the audio data after the cut point is completely preserved and continued into the next recognition process, ensuring the integrity and coherence of the audio stream in the continuous speech recognition process, avoiding the accumulation of sentence segmentation errors, and improving the accuracy of continuous long speech recognition.

[0087] In some embodiments, such as Figure 4 As shown, for real-time acquired PCM audio streams, a precise mapping relationship between audio duration and audio byte count is established. Based on the absolute end timestamp returned by ASR, the number of bytes N to be cut is accurately calculated, thereby achieving unbiased positioning of the cutting point. First, fixed encoding parameters for the audio stream are preset. In this embodiment, the PCM audio stream adopts a general encoding format of 16kHz sampling rate, 16-bit depth, and mono. Based on these encoding parameters, the byte rate of the audio stream is calculated to be 32 Bytes / ms. The specific calculation logic is as follows: 16kHz sampling rate corresponds to 16,000 sampling points per second, 16-bit depth corresponds to each sampling point occupying 2 bytes, and mono has no additional channel overhead. Therefore, the number of bytes corresponding to the audio per second is 16,000 × 2 = 32,000 Bytes, which is equivalent to 32 Bytes / ms in milliseconds.

[0088] Based on the above byte rate, the formula for calculating the number of bytes converted from a timestamp is established: △T = N_bytes / 32 (Bytes / ms); Where △T is the difference between the absolute end timestamp of the current recognized statement returned by ASR and the timestamp of the last cutting point, in ms; N_bytes is the number of audio bytes to be cut off, i.e., N bytes as described in Example 10.

[0089] Furthermore, to eliminate the time accumulation error generated during continuous truncation, this embodiment adds a fixed offset calibration mechanism to the audio stream reading cursor. Specifically, after each audio truncation operation, the audio stream reading cursor is adjusted by adding an offset calibration amount based on the preset tail note compensation duration, from its original cursor position. Figure 2 As shown; the 200ms calibration value corresponds to 6400 bytes of audio data, which matches the fixed word swallowing loss in the traditional scheme.

[0090] By establishing a precise time-byte mapping formula, byte-level accurate positioning of audio segmentation points based on the timestamps returned by ASR is achieved, ensuring the matching of the segmented audio fragments with the recognized sentences and avoiding truncation errors caused by time and byte conversion deviations. Through a fixed-offset cursor calibration mechanism, the fixed word swallowing loss existing in traditional schemes is offset, while eliminating the time accumulation error caused by multiple consecutive truncation operations. This ensures the continuous alignment between the audio stream reading position and the timestamp of the recognized sentences during long speech continuous recognition, further improving the accuracy of sentence segmentation and recognition accuracy.

[0091] This embodiment constructs a closed-loop processing solution for the entire process, from audio acquisition, caching, recognition result reception, segmentation point calculation, lossless truncation to audio output and caching continuation. It fully realizes lossless sentence segmentation of real-time audio streams, fundamentally solving the word swallowing problem in traditional real-time ASR scenarios. At the same time, the entire process adopts an asynchronous processing mechanism, adapting to the asynchronous return characteristics of cloud-based ASR, ensuring low latency and high continuity of real-time speech recognition. It can be directly applied to various real-time speech recognition scenarios such as real-time speech transcription, intelligent voice interaction, and real-time meeting recording.

[0092] Please see Figure 5 As shown, Figure 5This is a schematic diagram of the structure of the asynchronous alignment and precise dynamic truncation system 200 for audio frames and streaming text recognition provided in this application embodiment. The asynchronous alignment and precise dynamic truncation system 200 for audio frames and streaming text recognition is used to execute the steps of the asynchronous alignment and precise dynamic truncation methods for audio frames and streaming text recognition shown in the above embodiments. The asynchronous alignment and precise dynamic truncation system 200 for audio frames and streaming text recognition can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0093] like Figure 5 As shown, the asynchronous alignment and precise dynamic truncation system 200 for audio frames and streaming recognized text includes: The audio splitting unit 201 is used to split the raw audio stream captured by the microphone into two paths: one path is continuously transmitted to the cloud speech recognition service, and the other path is unidirectionally appended to the local independent binary audio buffer. The target acquisition unit 202 is used to receive the recognition text and the time axis position of the sentence segmentation sent from the cloud, determine the conversion relationship between time and audio byte length according to the fixed physical characteristics of audio sampling, and obtain the target truncated byte length by combining the time difference calculated by the time axis position of the sentence segmentation and the local global time cursor. The data acquisition unit 203 is used to extract the corresponding byte of audio data from the head of the audio buffer according to the target truncation byte length, and to completely retain the remaining audio data in the audio buffer that exceeds the extraction length and move it to the head of the audio buffer to enter the next processing cycle; after each truncation is completed, the actual duration is calculated in reverse according to the actual length of audio bytes extracted from the audio buffer, and the actual duration is accumulated to the local global time cursor to complete the time cursor calibration.

[0094] In some embodiments, dividing the raw audio stream captured by the microphone into two streams includes: synchronously splitting the raw audio stream continuously captured by the microphone in real time into two audio streams with completely identical content; wherein the audio frame to be sent to the cloud speech recognition service is sent to the cloud speech recognition service through a streaming connection, and the same audio frame is synchronously appended to the local audio buffer. Before the truncation operation is triggered, the audio buffer is appended in the order of audio reception; after the truncation operation is triggered, only the extracted prefix audio data is removed, and the unextracted suffix audio data is completely retained.

[0095] In some embodiments, after receiving the recognized text and the timeline position of the sentence segment from the cloud, determining the conversion relationship between time and audio byte length based on the fixed physical characteristics of audio sampling includes: after receiving the complete recognized text and the corresponding timeline position of the sentence segment from the cloud, not using the local system clock as the time calculation reference, but using the fixed sampling rate, sampling bit depth and number of channels preset when collecting audio as the basic parameters; determining the fixed number of audio bytes corresponding to a single millisecond duration based on the basic parameters, so as to establish a fixed conversion relationship between time length and audio byte length.

[0096] In some embodiments, the step of obtaining the target truncated byte length by combining the time difference calculated from the sentence segmentation timeline position and the local global time cursor includes: subtracting the local global time cursor from the sentence segmentation timeline position issued by the cloud to obtain the audio time difference to be truncated; selectively adding configurable tail note compensation duration based on the sentence segmentation characteristics of the target cloud speech recognition service based on the audio time difference to obtain the final truncated duration; and converting the final truncated duration into the corresponding target truncated byte length according to a fixed conversion relationship.

[0097] In some embodiments, the step of extracting the corresponding byte of audio data from the head of the audio buffer according to the target truncated byte length includes: before performing the audio extraction operation, obtaining the total number of bytes currently stored in the audio buffer; comparing the target truncated byte length with the current total number of bytes in the audio buffer, and taking the smaller of the two values ​​as the actual extracted byte length; extracting the corresponding byte of audio data from the starting position of the head of the audio buffer according to the determined actual extracted byte length, and encapsulating the extracted audio data into an independent audio slice that matches the recognized text.

[0098] In some embodiments, the step of completely preserving and moving the remaining audio data in the audio buffer that exceeds the extraction length to the beginning of the audio buffer for the next processing cycle includes: after completing the audio data extraction operation of the current cycle, not performing a complete clearing operation of the audio buffer; moving the remaining audio data in the audio buffer that exceeds the actual extraction byte length completely and without modification to the beginning of the audio buffer; after the movement is completed, the audio buffer only retains the aforementioned remaining audio data, waiting for the next round of truncation extraction operation.

[0099] In some embodiments, the step of reversely calculating the corresponding actual duration based on the actual length of audio bytes extracted from the audio buffer after each truncation includes: after each truncation operation, obtaining the actual length of audio bytes extracted from the audio buffer; and converting the actual extracted audio byte length into the corresponding actual truncation duration according to a predetermined fixed conversion relationship between time and audio byte length.

[0100] In some embodiments, the actual duration is calculated in reverse based on the actual length of the audio bytes extracted from the audio buffer after each truncation, including: using a fixed conversion relationship to implement the reverse conversion process, which is completely consistent with the fixed conversion relationship used when determining the target truncation byte length; wherein, the reverse conversion process uses only the actual extracted audio byte length as the sole calculation basis.

[0101] In some embodiments, the method further includes: uniquely binding and storing the encapsulated independent audio slices with the corresponding cloud-delivered recognition text, generating audio-text matching data units with time anchors, and outputting the audio-text matching data units to downstream voice playback, voice re-examination, or machine translation processes.

[0102] In some embodiments, obtaining the end time position of the recognized sentence corresponding to the sentence segmentation timeline position includes: when the cloud speech recognition service returns the sentence end time field, the value of the sentence end time field is used as the end time position; when the cloud speech recognition service returns the sentence start offset and the sentence duration, the value of the sentence start offset and the value of the sentence duration are added together to obtain the end time position; when the cloud speech recognition service returns the recognition result end offset, the value of the recognition result end offset is used as the end time position.

[0103] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described asynchronous alignment and precise dynamic truncation system for audio frames and streaming text and its modules can be found in the corresponding contents of the various embodiments of the above-described asynchronous alignment and precise dynamic truncation method for audio frames and streaming text, and will not be repeated here.

[0104] The aforementioned asynchronous alignment and precise dynamic truncation method for audio frames and streaming text recognition can be implemented as a computer program, which can, for example... Figure 5 It runs on the device shown.

[0105] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0106] The storage medium may store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform an asynchronous alignment and precise dynamic truncation method for any audio frame and streaming recognized text.

[0107] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0108] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When executed by a processor, the computer program enables the processor to perform any method of asynchronous alignment and precise dynamic truncation of audio frames and streaming recognized text.

[0109] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0111] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: The raw audio stream captured by the microphone is split into two paths: one path is continuously transmitted to the cloud speech recognition service, and the other path is unidirectionally appended to a local independent binary audio buffer. After receiving the recognized text and the timeline position of the sentence segment from the cloud, the conversion relationship between time and audio byte length is determined based on the fixed physical characteristics of audio sampling. The target truncated byte length is obtained by combining the time difference calculated by the sentence segment timeline position and the local global time cursor. Based on the target truncation byte length, the corresponding byte of audio data is extracted from the head of the audio buffer, and the remaining audio data in the audio buffer that exceeds the extraction length is completely preserved and moved to the head of the audio buffer to enter the next processing cycle. After each truncation is completed, the actual duration is calculated in reverse based on the actual length of audio bytes extracted from the audio buffer, and the actual duration is accumulated to the local global time cursor to complete the time cursor calibration.

[0112] In some embodiments, dividing the raw audio stream captured by the microphone into two streams includes: synchronously splitting the raw audio stream continuously captured by the microphone in real time into two audio streams with completely identical content; wherein the audio frame to be sent to the cloud speech recognition service is sent to the cloud speech recognition service through a streaming connection, and the same audio frame is synchronously appended to the local audio buffer. Before the truncation operation is triggered, the audio buffer is appended in the order of audio reception; after the truncation operation is triggered, only the extracted prefix audio data is removed, and the unextracted suffix audio data is completely retained.

[0113] In some embodiments, after receiving the recognized text and the timeline position of the sentence segment from the cloud, determining the conversion relationship between time and audio byte length based on the fixed physical characteristics of audio sampling includes: after receiving the complete recognized text and the corresponding timeline position of the sentence segment from the cloud, not using the local system clock as the time calculation reference, but using the fixed sampling rate, sampling bit depth and number of channels preset when collecting audio as the basic parameters; determining the fixed number of audio bytes corresponding to a single millisecond duration based on the basic parameters, so as to establish a fixed conversion relationship between time length and audio byte length.

[0114] In some embodiments, the step of obtaining the target truncated byte length by combining the time difference calculated from the sentence segmentation timeline position and the local global time cursor includes: subtracting the local global time cursor from the sentence segmentation timeline position issued by the cloud to obtain the audio time difference to be truncated; selectively adding configurable tail note compensation duration based on the sentence segmentation characteristics of the target cloud speech recognition service based on the audio time difference to obtain the final truncated duration; and converting the final truncated duration into the corresponding target truncated byte length according to a fixed conversion relationship.

[0115] In some embodiments, the step of extracting the corresponding byte of audio data from the head of the audio buffer according to the target truncated byte length includes: before performing the audio extraction operation, obtaining the total number of bytes currently stored in the audio buffer; comparing the target truncated byte length with the current total number of bytes in the audio buffer, and taking the smaller of the two values ​​as the actual extracted byte length; extracting the corresponding byte of audio data from the starting position of the head of the audio buffer according to the determined actual extracted byte length, and encapsulating the extracted audio data into an independent audio slice that matches the recognized text.

[0116] In some embodiments, the step of completely preserving and moving the remaining audio data in the audio buffer that exceeds the extraction length to the beginning of the audio buffer for the next processing cycle includes: after completing the audio data extraction operation of the current cycle, not performing a complete clearing operation of the audio buffer; moving the remaining audio data in the audio buffer that exceeds the actual extraction byte length completely and without modification to the beginning of the audio buffer; after the movement is completed, the audio buffer only retains the aforementioned remaining audio data, waiting for the next round of truncation extraction operation.

[0117] In some embodiments, the step of reversely calculating the corresponding actual duration based on the actual length of audio bytes extracted from the audio buffer after each truncation includes: after each truncation operation, obtaining the actual length of audio bytes extracted from the audio buffer; and converting the actual extracted audio byte length into the corresponding actual truncation duration according to a predetermined fixed conversion relationship between time and audio byte length.

[0118] In some embodiments, the actual duration is calculated in reverse based on the actual length of the audio bytes extracted from the audio buffer after each truncation, including: using a fixed conversion relationship to implement the reverse conversion process, which is completely consistent with the fixed conversion relationship used when determining the target truncation byte length; wherein, the reverse conversion process uses only the actual extracted audio byte length as the sole calculation basis.

[0119] In some embodiments, the method further includes: uniquely binding and storing the encapsulated independent audio slices with the corresponding cloud-delivered recognition text, generating audio-text matching data units with time anchors, and outputting the audio-text matching data units to downstream voice playback, voice re-examination, or machine translation processes.

[0120] In some embodiments, obtaining the end time position of the recognized sentence corresponding to the sentence segmentation timeline position includes: when the cloud speech recognition service returns the sentence end time field, the value of the sentence end time field is used as the end time position; when the cloud speech recognition service returns the sentence start offset and the sentence duration, the value of the sentence start offset and the value of the sentence duration are added together to obtain the end time position; when the cloud speech recognition service returns the recognition result end offset, the value of the recognition result end offset is used as the end time position.

[0121] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the asynchronous alignment and precise dynamic truncation method for audio frames and streaming recognized text as provided in any embodiment of this application.

[0122] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for asynchronous alignment and precise dynamic truncation of audio frames and streaming text recognition, characterized in that, include: The raw audio stream captured by the microphone is split into two paths: one path is continuously transmitted to the cloud speech recognition service, and the other path is unidirectionally appended to a local independent binary audio buffer. After receiving the recognized text and the timeline position of the sentence segment from the cloud, the conversion relationship between time and audio byte length is determined based on the fixed physical characteristics of audio sampling. The target truncated byte length is obtained by combining the time difference calculated by the sentence segment timeline position and the local global time cursor. Based on the target truncation byte length, the corresponding byte of audio data is extracted from the head of the audio buffer, and the remaining audio data in the audio buffer that exceeds the extraction length is completely preserved and moved to the head of the audio buffer to enter the next processing cycle. After each truncation is completed, the actual duration is calculated in reverse based on the actual length of audio bytes extracted from the audio buffer, and the actual duration is accumulated to the local global time cursor to complete the time cursor calibration.

2. The method according to claim 1, characterized in that, The process of splitting the raw audio stream captured by the microphone into two paths includes: The raw audio stream continuously acquired by the microphone in real time is synchronously split into two audio streams with completely identical content. The audio frame to be sent to the cloud speech recognition service is sent to the cloud speech recognition service through a streaming connection, and the same audio frame is synchronously appended to the local audio buffer. Before the truncation operation is triggered, the audio buffer is appended in the order of audio reception. After the truncation operation is triggered, only the extracted prefix audio data is removed, and the unextracted suffix audio data is completely retained.

3. The method according to claim 1, characterized in that, After receiving the recognized text and sentence segmentation timeline position from the cloud, the conversion relationship between time and audio byte length is determined based on the fixed physical characteristics of audio sampling, including: After receiving the complete recognized text and the corresponding sentence segmentation timeline position from the cloud, the local system clock is not used as the time calculation reference. Instead, the fixed sampling rate, sampling bit depth and number of channels preset during audio acquisition are used as the basic parameters. The fixed number of audio bytes corresponding to a single millisecond duration is determined based on the basic parameters, so as to establish a fixed conversion relationship between the duration and the audio byte length.

4. The method according to claim 3, characterized in that, The target truncated byte length is obtained by combining the time difference calculated from the sentence segmentation timeline position and the local global time cursor, including: Subtract the local global time cursor from the timeline position of the segment sent from the cloud to obtain the audio time difference that needs to be extracted this time; Based on the audio time difference, and according to the sentence segmentation characteristics of the target cloud speech recognition service, a configurable tail sound compensation duration is selectively added to obtain the final truncation duration; Based on a fixed conversion relationship, the final truncation duration is converted into the corresponding target truncation byte length.

5. The method according to claim 1, characterized in that, The step of extracting the corresponding byte of audio data from the head of the audio buffer according to the target truncated byte length includes: Before performing the audio extraction operation, obtain the total number of bytes currently stored in the audio buffer; The target truncated byte length is compared with the current total number of bytes in the audio buffer, and the smaller of the two values ​​is taken as the actual extracted byte length. According to the determined actual extraction byte length, the corresponding bytes of audio data are extracted from the beginning position of the audio buffer, and the extracted audio data is encapsulated into an independent audio slice that matches the recognized text.

6. The method according to claim 5, characterized in that, The step of completely preserving and moving any remaining audio data exceeding the extraction length from the audio buffer to the beginning of the audio buffer for the next processing cycle includes: After completing the audio data extraction operation for the current round, the audio buffer is not completely cleared; any remaining audio data in the audio buffer that exceeds the actual extracted byte length is moved completely and without modification to the beginning of the audio buffer. After the movement is complete, the audio buffer retains only the remaining audio data mentioned above, waiting for the next round of truncation and extraction operations.

7. The method according to claim 1, characterized in that, After each truncation, the actual duration is calculated in reverse based on the actual length of the audio bytes extracted from the audio buffer, including: After each truncation operation is completed, obtain the length of the audio bytes actually extracted from the audio buffer in this operation; Based on a predetermined fixed conversion relationship between time and audio byte length, the actual extracted audio byte length is converted in reverse to the corresponding actual extraction duration.

8. The method according to claim 7, characterized in that, After each truncation, the actual duration is calculated in reverse based on the actual length of the audio bytes extracted from the audio buffer, including: The reverse conversion process is implemented using a fixed conversion relationship that is completely consistent with the fixed conversion relationship used when determining the target truncated byte length; wherein, the reverse conversion process uses only the actual extracted audio byte length as the sole calculation basis.

9. The method according to claim 1, characterized in that, The method further includes: The packaged independent audio slices are uniquely bound and stored with the corresponding cloud-delivered recognition text to generate audio-text matching data units with time anchors. These audio-text matching data units are then output to downstream voice playback, voice re-examination, or machine translation processes.

10. The method according to claim 1, characterized in that, Obtaining the end time position of the identified sentence corresponding to the timeline position of the sentence segmentation includes: When the cloud-based speech recognition service returns the statement end time field, the value of the statement end time field will be used as the end time position. When the cloud-based speech recognition service returns the statement start offset and statement duration, the value of the statement start offset and the value of the statement duration are added together to obtain the end time position; when the cloud-based speech recognition service returns the recognition result end offset, the value of the recognition result end offset is used as the end time position.

Images