Method, device, equipment, medium and product for concurrent processing of voice collection and optimization

Abstract

The application discloses a speech collection and optimization concurrent processing method, device, equipment, medium and product, and relates to the technical field of artificial intelligence, which comprises the following steps: in the continuous speech collection process, a first speech recognition session is maintained to collect speech and output recognized text; when an optimization trigger condition is detected, the first speech recognition session is terminated, and the accumulated recognized text is obtained as snapshot text; within a time period less than a preset delay threshold, a second speech recognition session is started to continue collecting subsequent speech and output new recognized text; the snapshot text is sent to a language processing service for optimization processing, and the returned optimized text is received; after receiving the optimized text, the optimized text and the new recognized text are merged into the current text in the working buffer. The application enables the user to continue dictating without waiting for the optimization to be completed, reduces the end-to-end response delay, automatically merges the optimized text and the new recognized text, and realizes seamless connection and multi-round natural closed loop of speech.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and product for concurrent processing of voice acquisition and optimization. Background Technology

[0002] On computing devices, users often obtain natural language text through voice capture, and then use language processing services to polish, correct errors, or change the style of the text.

[0003] The closest existing technology is a voice input system that combines continuous dictation with post-processing full-text editing. This type of system continuously receives audio streams and outputs incremental recognition results within a single voice acquisition session. After the session ends or the user manually stops the process, the complete recognized text is submitted to the language processing service, returning a one-time optimized result. Audio acquisition and voice recognition / decoding are completed within the same pipeline; the language processing service call is triggered only after the voice recognition session terminates or the user explicitly stops the process, and the two are sequential in the control flow.

[0004] The shortcomings of this solution are: it cannot start parallel language processing service streaming optimization on the accumulated recognized text while keeping audio acquisition uninterrupted; and it cannot continue to receive user voice in a new speech recognition session while optimization is in progress, and logically merge the new content and optimization results into a unified working buffer. Summary of the Invention

[0005] This invention provides a method, apparatus, device, medium, and product for concurrent processing of voice acquisition and optimization, in order to solve the technical problems in the prior art where voice acquisition and language processing service optimization are executed sequentially, making it impossible to continue acquiring voice during optimization, and where optimized text and newly recognized text cannot be automatically merged within the same conversation logic.

[0006] According to one aspect of the present invention, a method for concurrent processing of voice acquisition and optimization is provided, the method comprising: During continuous voice acquisition, the first speech recognition session is maintained to receive user voice and output recognized text; When an optimization trigger condition is detected, the first speech recognition session is terminated, and the accumulated recognition text up to the time the optimization trigger condition is detected is obtained as snapshot text. Initiate a second speech recognition session within a time frame less than a preset delay threshold to continue collecting subsequent user speech and output new recognized text; While the second speech recognition session outputs the new recognized text, the snapshot text is sent to the language processing service for optimization, and the optimized text returned by the language processing service is received. Upon receiving the optimized text, the optimized text is merged with the newly recognized text output by the second speech recognition session to form the current text in the working buffer.

[0007] According to another aspect of the present invention, an apparatus for concurrent processing and optimization of voice acquisition is provided, the apparatus comprising: The text recognition output module is used to maintain the first speech recognition session during continuous speech acquisition to receive user speech and output recognized text. The snapshot text acquisition module is used to terminate the first speech recognition session when an optimization trigger condition is detected, and to acquire the accumulated recognition text up to the time the optimization trigger condition is detected as snapshot text. The new recognized text output module is used to start a second speech recognition session within a time period less than a preset delay threshold, so as to continue to collect the user's subsequent speech and output new recognized text. The optimized text receiving module is used to send the snapshot text to the language processing service while the second speech recognition session outputs the newly recognized text, so that the language processing service can optimize the snapshot text and receive the optimized text returned by the language processing service. The text merging module is used to merge the optimized text with the newly recognized text output by the second speech recognition session into the current text in the working buffer after receiving the optimized text.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method described in any one of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the method described in any one of the present invention.

[0010] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method of any one of the present invention.

[0011] The beneficial effects of this invention are as follows: Firstly, by orderly terminating the current speech recognition session and generating snapshot text, and starting a new speech recognition session within a very short delay, the speech acquisition process is almost imperceptibly uninterrupted. Users can continue speaking during the optimization process, avoiding the experience gaps that occur in traditional serial processes where users must wait for optimization to complete before they can continue speaking. At the same time, the snapshot is sent to the language processing service for streaming optimization. The optimization processing and the new speech acquisition are executed in parallel in time, so users do not need to wait for both to complete sequentially, reducing end-to-end response latency. After optimization, the optimization results and the accumulated text of the new session are merged into a unified working buffer according to rules, ensuring seamless connection between the recognized text before optimization and the newly acquired text during optimization within the same logical session, avoiding text duplication, omissions, or situations where users need to manually merge them.

[0012] Secondly, the aforementioned concurrent session restart and optimization process can be repeated an arbitrary number of times within a single user session. The working buffer after each merge automatically becomes the starting point for the next round of optimization. Users can trigger optimization multiple times and continuously add new content during a continuous speech, forming a natural multi-round closed loop of speaking, optimizing, speaking again, and optimizing again. There is no need to manually manage multiple sessions or segmented operations, which improves the interaction coherence and expression efficiency in long-term voice input scenarios.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a method for concurrent voice acquisition and optimization provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a current text update method provided in Embodiment 2 of the present invention; Figure 3 This is a flowchart of a text rewriting method provided in Embodiment 3 of the present invention; Figure 4 This is a flowchart of a method for command-based voice acquisition and optimized concurrent processing provided in Embodiment 4 of the present invention; Figure 5 This is a schematic diagram of the structure of a voice acquisition and optimization concurrent processing device provided in Embodiment 5 of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device that implements the method for concurrent processing and optimization of voice acquisition according to an embodiment of the present invention. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] Example 1 Figure 1 This is a flowchart illustrating a method for concurrent voice acquisition and optimization processing according to Embodiment 1 of the present invention. This embodiment is applicable to situations where a user needs to trigger language processing service optimization multiple times during continuous voice acquisition, and continues speaking uninterruptedly during the optimization period. This method can be executed by a device for concurrent voice acquisition and optimization processing, which can be implemented in hardware and / or software, such as through a client application. Figure 1 As shown, the method includes: S101. During continuous voice acquisition, maintain the first voice recognition session to receive user voice and output recognized text.

[0019] Continuous voice acquisition refers to the client continuously capturing user voice data throughout the user-device interaction process via an ongoing underlying audio acquisition pipeline. This process is uninterrupted by the client switching speech recognition sessions or invoking language processing services. Continuous voice acquisition ensures that users can continue speaking new content while optimization is being implemented, forming the foundation for a "speak-as-you-go" experience. In practice, the client typically uses the operating system's audio interface (such as a microphone input stream) to continuously read voice data until the user actively stops or the process is interrupted by a system event.

[0020] The first speech recognition session refers to a logical session instance maintained by the automatic speech recognition engine during continuous speech acquisition by the client. This session is used to recognize the current segment of continuous speech and output the corresponding recognized text. This session has an independent recognition context and can output incremental recognition results in real time.

[0021] User voice refers to the sound signal emitted by the user, collected by the client through audio input devices such as microphones. Its content includes natural language statements that the user wishes to be recognized and processed. During continuous voice acquisition, the client converts the user's voice into an audio stream in real time for speech recognition session processing. User voice can contain text content to be optimized, or it can contain spoken commands or requests to modify existing text.

[0022] Recognized text refers to the character sequence in text form obtained by the client through automatic speech recognition technology. Recognized text retains the word order and vocabulary of the user's original speech, and may contain colloquial expressions, incomplete grammar, or repetitive content.

[0023] In one implementation, during continuous voice acquisition, the client initiates a continuously running audio acquisition task through the operating system's audio interface. The client pushes the acquired user voice to the automatic speech recognition engine in real-time segments and designates a newly created recognition instance as the first speech recognition session. The automatic speech recognition engine decodes each segment of voice, and the client receives the recognized text incrementally, appending each stable segment to the current recognition result until it receives the end marker of the current segment or the user's pause times out.

[0024] For example, a user says "The weather is really nice today." The client continuously collects the user's voice, and the first speech recognition session outputs "Today," "Weather," and "Nice" in sequence. The client then concatenates these outputs to obtain the recognized text "The weather is really nice today."

[0025] In another implementation, the client activates a voice activity detection module during continuous voice acquisition to continuously monitor the start and end of the user's speech. When the client detects that the user has started speaking, it dynamically creates a first speech recognition session and sends subsequently acquired user speech into this session. During the user's speech, the automatic speech recognition engine continuously outputs incremental recognized text, which the client accumulates in real time. When the client detects that the user continues speaking after a short pause, the same session continues until the overall session is detected to have ended.

[0026] For example, a user first says "I want to book a flight," pauses for one second, and then continues saying "to Shanghai." The client's voice activity detection determines that it is the same round of speaking, the first speech recognition session continues to output, and the client eventually obtains the complete recognized text "I want to book a flight to Shanghai."

[0027] S102. When an optimization trigger condition is detected, terminate the first speech recognition session and obtain the accumulated recognition text up to the time the optimization trigger condition is detected as snapshot text.

[0028] The optimization trigger condition refers to the precondition or rule that the client determines should initiate the language processing service to optimize the currently accumulated recognized text. This condition can be an operation actively triggered by the user, or a state automatically detected by the system according to preset rules. In this embodiment, the optimization trigger condition is a signal to start the concurrent optimization process. When the client detects this condition, it terminates the first speech recognition session and saves the snapshot text.

[0029] "Up to the detection of the optimization trigger condition" refers to the time interval from the start of the first speech recognition session to the moment the client detects and confirms that the optimization trigger condition is met. This statement emphasizes that the scope of the snapshot text includes only the recognized text that has been stably output before the detection of the optimization trigger condition, excluding recognition results generated after the trigger. This time boundary ensures that the snapshot text does not overlap with subsequent new recognized text in terms of time period, thus supporting seamless integration of the merging logic.

[0030] Snapshot text refers to a complete segment of recognized text that has been accumulated and stably output up to the moment the client detects the optimization trigger condition, and is read from the first speech recognition session. This text represents all the content the user has spoken before the optimization trigger and is about to be sent to the language processing service as material to be optimized. The content of the snapshot text is fixed at the moment of triggering, and subsequent new recognized text generated by the user's continued speaking will not affect the snapshot text.

[0031] In one implementation, during continuous voice acquisition, the client provides an optimization trigger button (e.g., "Optimize Now") on the user interface. When the user clicks this button, the client immediately recognizes that an optimization trigger condition has been detected. Subsequently, the client sends a termination command to the currently maintained first speech recognition session and reads all accumulated recognition text up to this point from the session's output buffer, copying this text to a separate temporary storage area as snapshot text.

[0032] In another implementation, the client analyzes the recognized text output from the first speech recognition session in real time during continuous speech acquisition. When a preset optimization trigger keyword (such as "optimize," "polish," or "help me revise") appears in the recognized text, the client determines that the optimization trigger condition has been detected. The client then terminates the first speech recognition session and uses the recognized text accumulated up to the appearance of the keyword as snapshot text; the keyword itself is not included in the snapshot content.

[0033] For example, a user dictates, "Help me polish this sentence: The weather is nice today." The client detects the "help me polish" request and triggers a snapshot of the text, "The weather is nice today."

[0034] In another implementation, during continuous voice acquisition, the client simultaneously listens for system events from the operating system or notifications from other applications (e.g., clipboard changes, timer expiration, etc.). When a predetermined event is detected (e.g., a timer reaches a set time, or new text content appears on the clipboard), the client determines that an optimization trigger condition has been detected. The client immediately terminates the first voice recognition session and uses the complete recognized text accumulated up to the time the event occurs as a snapshot text.

[0035] For example, the client is set to automatically optimize every 30 seconds. When the timer reaches 30 seconds, the user is saying "The meeting minutes are as follows, point one...", the client triggers optimization and retrieves the accumulated snapshot text "The meeting minutes are as follows, point one...".

[0036] S103. Start the second speech recognition session within a time period less than the preset delay threshold to continue collecting the user's subsequent speech and output new recognized text.

[0037] The preset delay threshold refers to the maximum time interval allowed for the client to start a second speech recognition session after terminating the first. This threshold is a configurable duration parameter used to ensure that the session switching process is imperceptible to the user. In actual implementation, the client optimizes the startup process, pre-allocates recognition engine resources, or adopts lightweight session reconstruction technology to keep the switching delay within the preset threshold, thereby ensuring the continuity of speech acquisition and avoiding interruptions or perceptible pauses in user narration. Optionally, the preset delay threshold is less than 100 milliseconds.

[0038] The second speech recognition session refers to an instance of automatic speech recognition established within a preset delay threshold after the client detects the optimization trigger condition and terminates the first speech recognition session. This session shares the same underlying audio acquisition pipeline as the first speech recognition session but has its own independent recognition context and buffer. The second speech recognition session begins processing from the audio frame after the termination of the first speech recognition session, and is used to identify new content uttered by the user during the optimization process, which is then merged with the snapshot text of the first speech recognition session in subsequent steps.

[0039] Subsequent speech refers to the speech content uttered by the user after the current optimization trigger condition is detected but before the optimization process is complete. This part of the speech is generated after the first speech recognition session ends and during and after the start of the second speech recognition session. Its content consists of newly added text material during the optimization process and is not covered by the snapshot text. The acquisition of subsequent speech benefits from the continuous operation of the underlying audio pipeline and can be naturally continued without any additional user operation.

[0040] Newly recognized text refers to the character sequence in text form obtained by the client after decoding subsequent speech through a second speech recognition session. Unlike snapshot text, this text represents the user's new dictation after the optimization trigger, and is therefore later in time than the snapshot text.

[0041] In one implementation, before continuous voice acquisition begins, the client pre-creates two instances of an automatic speech recognition engine, labeled as the first recognition engine and the second recognition engine, respectively. When the first speech recognition session is running normally, the client simultaneously sends audio data to the first recognition engine, while the second recognition engine remains idle and does not output recognized text. Upon detecting an optimization trigger condition and terminating the first speech recognition session, the client immediately switches the target receiver of the audio data, redirecting subsequent audio frames from the first recognition engine to the second recognition engine, and simultaneously sends a start command to the second recognition engine. Since the model and resources of the second recognition engine are pre-loaded, the switching operation only requires modifying the data routing pointer, and the time consumed is less than a preset delay threshold. The second recognition engine then begins decoding the subsequent audio and outputs new recognized text.

[0042] For example, the client pre-creates two engine instances. After the user says "The weather is nice today," the optimization is triggered, the first speech recognition session terminates, and the client switches the audio stream to the second engine as the second speech recognition session within a time of less than a preset delay threshold (e.g., 20 milliseconds). When the user then says "What about tomorrow?", the second speech recognition session outputs the newly recognized text "What about tomorrow?"

[0043] In another implementation, the client maintains only one instance of the automatic speech recognition engine during continuous voice acquisition. When an optimization trigger condition is detected and the first speech recognition session is terminated, the client does not destroy the engine instance. Instead, it sends a session reset command to the engine, clearing the current session's context and buffer, but retaining the engine's model parameters and network connection. Immediately after sending the reset command, the client assigns a new session identifier to the new session and sends subsequent audio frames to the same engine. Since the engine instance itself is not destroyed and rebuilt, there is no need to reload the model, and the time spent on session reset and creation is less than a preset delay threshold. The engine then outputs the recognized text of the new session, i.e., the new recognized text.

[0044] For example, the client uses a single engine. After the user utters "Write an article about climate," an optimization is triggered. The client resets the engine and creates a second speech recognition session for subsequent audio within a time frame less than a preset delay threshold. The user continues by saying "Add data on extreme weather," and the engine outputs the newly recognized text "Add data on extreme weather."

[0045] S104. While outputting new recognized text in the second speech recognition session, send the snapshot text to the language processing service so that the language processing service can optimize the snapshot text and receive the optimized text returned by the language processing service.

[0046] The language processing service, in this context, refers to a computational service capable of semantic-level understanding, generation, or transformation of text. In this solution, it receives text data sent by the client and optimizes the text based on an internal model (such as a large language model). This service can be deployed on a cloud server or run locally. The client interacts with the language processing service via communication protocols (such as Hypertext Transfer Protocol, WebSockets, etc.), sending a snapshot of the text, after which the language processing service returns the corresponding optimized text. The language processing service is the core unit for implementing optimization functions such as text polishing, expansion, summarization, and style transfer.

[0047] Optimization processing refers to the process by which language processing services perform semantic analysis and content reconstruction on received snapshot text. Specifically, based on preset optimization rules or model capabilities, the language processing service corrects grammatical errors, inappropriate word choice, and unclear logic in the snapshot text, or performs style adjustments, sentence compression, and content expansion according to the user's implicit intent. The goal of optimization processing is to generate a result that better conforms to written expression or user expectations than the original snapshot text—that is, optimized text.

[0048] Optimized text refers to the text content returned to the client by the language processing service after optimization. This text is a new version generated based on the snapshot text, which may have corrected grammatical errors, eliminated colloquial expressions, adjusted sentence structure, standardized terminology, or reorganized the content while maintaining the original meaning.

[0049] In one implementation, while the second speech recognition session outputs new recognized text, the client encapsulates the snapshot text into a request body and initiates a synchronization request to the language processing service via Hypertext Transfer Protocol (HTTP). The client declares in the request header that it accepts a regular text response. Upon receiving the request, the language processing service performs optimization processing on the snapshot text (e.g., polishing, error correction, or style adjustment). Once the entire optimization result is generated, it returns the complete optimized text to the client in the response body. The client remains in a waiting state until it receives the complete response, but during this period, since the second speech recognition session has started, the user can still continue to dictate subsequent speech and output new recognized text; therefore, the overall experience remains concurrent.

[0050] For example, the client sends the snapshot text "The weather is nice today" to the language processing service, which then returns the optimized text "The weather is sunny today" after optimization. The client receives the complete result all at once.

[0051] In another implementation, the client pre-establishes a long-lived web socket connection with the language processing service. When optimization is needed, the client sends a snapshot of the text as a message to the language processing service through this connection. During processing, the language processing service generates fragments of optimized text and pushes these fragments to the client multiple times through the same connection. Each time the client receives a fragment, it appends it to its internal optimized text buffer until it receives an end marker. Then, the client uses the completed content in the buffer as the optimized text. Because the fragment push is done incrementally, the client can display partial results during the generation process, but ultimately must wait for all fragments to be received to obtain the complete optimized text.

[0052] For example, the client sends a snapshot text "Write a thank you letter" via a web socket. The language processing service then pushes the fragments "Dear," "User," and "Thank you" in sequence. The client then concatenates these fragments to obtain the complete and optimized text "Dear customer, thank you."

[0053] S105. Upon receiving the optimized text, merge the optimized text with the newly recognized text output by the second speech recognition session into the current text in the working buffer.

[0054] The working buffer, in this context, refers to a temporary data storage area maintained by the client in memory to store the complete text content in the current state. In this embodiment, the working buffer may initially be empty or contain the initial text. During execution, the working buffer successively carries the intermediate states of snapshot text, optimized text, and newly recognized text. When the client receives the optimized text returned by the language processing service, it does not directly replace the original content. Instead, it merges the optimized text with the newly recognized text output by the second speech recognition session, and writes the merged result into the working buffer to form the current text. The design of the working buffer allows the client to independently process the output results from different speech recognition sessions, avoiding content overwriting or corruption.

[0055] The current text refers to the complete text content that is finally formed in the working buffer after a complete "acquisition-optimization-merge" cycle, used for display or as the starting point for subsequent processing. The current text is obtained by merging the optimized text and the newly recognized text output from the second speech recognition session according to rules (e.g., sequential concatenation, line breaks, etc.). This text represents the overall result of all the content the user has spoken up to the end of the current round, after optimization, plus the newly added spoken content during the optimization period. In a single process, the current text is the result of this round of processing; if optimization is triggered again later, this current text will participate in the next round of the cycle as a new text to be optimized.

[0056] In one implementation, after receiving the optimized text returned by the language processing service, the client immediately reads all newly recognized text output up to this point from the output buffer of the second speech recognition session. The client concatenates the optimized text and the newly recognized text according to a preset merging rule (e.g., separating by newline characters, or appending the newly recognized text directly to the optimized text). The resulting concatenated string is the current text. The client writes this current text to the working buffer and triggers a UI refresh to display it.

[0057] For example, the optimized text is "Today's weather is sunny," and the new recognized text output by the second speech recognition session is "What about tomorrow?" The client concatenates these texts sequentially to obtain the current text "Today's weather is sunny\nWhat about tomorrow?", and writes it to the working buffer.

[0058] In another implementation, after receiving the optimized text, the client does not immediately perform the merging. Instead, it stores the optimized text in a temporary area of ​​the working buffer. The client continues to listen to the second speech recognition session, allowing a slight delay in the output of newly recognized text after optimization. When the client confirms that there is no more new text output in the second speech recognition session (e.g., through a silence timeout or session end event), the client retrieves the optimized text from the working buffer and merges all accumulated newly recognized text according to rules. For example, it intelligently inserts semantic elements such as line breaks and punctuation that may be present in the recognized text, rather than simply appending them. After merging, the final result is written back to the working buffer as the current text.

[0059] For example, the optimized text is "Please reply: Received", and the newly recognized text is "Then add thank you". The client interprets "add thank you" according to preset rules as appending it to the end of the optimized text, resulting in "Please reply: Received thank you", which is used as the current text.

[0060] The beneficial effects of the embodiments of the present invention are as follows: Firstly, by orderly terminating the current speech recognition session and generating snapshot text, and starting a new speech recognition session within a very short delay, the speech acquisition process is almost imperceptibly uninterrupted. Users can continue speaking during the optimization process, avoiding the experience gaps that occur in traditional serial processes where users must wait for optimization to complete before they can continue speaking. At the same time, the snapshot is sent to the language processing service for streaming optimization. The optimization processing and the new speech acquisition are executed in parallel in time, so users do not need to wait for both to complete sequentially, reducing end-to-end response latency. After optimization, the optimization results and the accumulated text of the new session are merged into a unified working buffer according to rules, ensuring seamless connection between the recognized text before optimization and the newly acquired text during optimization within the same logical session, avoiding text duplication, omissions, or situations where users need to manually merge them.

[0061] Secondly, the aforementioned concurrent session restart and optimization process can be repeated an arbitrary number of times within a single user session. The working buffer after each merge automatically becomes the starting point for the next round of optimization. Users can trigger optimization multiple times and continuously add new content during a continuous speech, forming a natural multi-round closed loop of speaking, optimizing, speaking again, and optimizing again. There is no need to manually manage multiple sessions or segmented operations, which improves the interaction coherence and expression efficiency in long-term voice input scenarios.

[0062] The applicant found that the existing technology for optimization triggering relies on manual operation. The optimization triggering depends entirely on the user manually clicking buttons or performing gestures, which interrupts the natural rhythm of narration and makes it difficult for users to accurately grasp the best timing for triggering, thus affecting the smoothness of use.

[0063] Optionally, methods for detecting and optimizing trigger conditions include: A1. Monitor the valid recognition events output by the first speech recognition session.

[0064] Among them, a valid recognition event refers to the event generated each time a non-empty, semantically meaningful text fragment (such as a word, a phrase, or a complete sentence) is output during the first speech recognition session. This event does not contain empty strings, punctuation marks alone, or meaningless filler characters issued by the recognition engine. A valid recognition event indicates that the user is currently speaking continuously and that the speech recognition system has successfully output meaningful text, serving as an important basis for determining whether the user is in an active speaking state.

[0065] A2. Reset the silent timer whenever a new valid recognition event is generated.

[0066] The silent timer is a software timer maintained internally by the client to measure the time elapsed during the first speech recognition session without any new valid recognition events. This timer is reset to zero each time a new valid recognition event is generated, and continues to accumulate during periods without new events. The purpose of the silent timer is to measure the length of natural pauses in the user's speech so that optimization is automatically triggered when a pause exceeds a certain threshold.

[0067] Resetting the silence timer refers to the operation where the client resets the current count value of the silence timer to zero when it detects a new valid recognition event in the first speech recognition session. This operation ensures that optimization is only triggered when the interval between consecutive valid recognition events reaches the silence duration threshold, avoiding accidental triggering when the user resumes speaking after a short pause.

[0068] A3. During the accumulation of the silent timer, if the silent timer reaches the preset silent duration threshold without generating a new valid recognition event, the optimization trigger condition is deemed met, and optimization is automatically triggered.

[0069] The silent timer accumulation process refers to the continuous increase of the silent timer value during the period after a reset (or from the start of the timer) when no new valid identified events occur. The accumulation process is incremental (e.g., incrementing by one every millisecond), and the client detects whether the timer has reached the preset silent duration threshold through polling or interrupt mechanisms.

[0070] The silence duration threshold refers to a pre-configured time parameter (e.g., 10 seconds) on the client side, used to determine whether the user has paused speaking for a sufficient period to meet the conditions for automatic optimization. This threshold can be adjusted according to the actual application scenario; for example, it can be set shorter in quiet environments and longer in noisy environments. When the accumulated value of the silence timer reaches this threshold, the client determines that the user has completed a full utterance and can initiate optimization.

[0071] Automatically triggered optimization refers to a process where the client proactively initiates a concurrent optimization process (i.e., terminating the first speech recognition session, obtaining a snapshot, and starting the second speech recognition session) based on the result of a silence timer reaching a silence duration threshold, without relying on manual user actions (such as clicking a button). Automatically triggered optimization aims to reduce the burden of user interaction and align the optimization timing with the natural rhythm of speech.

[0072] Automatic triggering of the underlying audio pipeline without interrupting the voice acquisition session. The underlying audio pipeline refers to the entire software processing chain from the client acquiring raw audio data from the hardware microphone to transmitting it to the speech recognition engine, including modules such as audio driver, buffer management, sampling rate conversion, and silence detection. Automatic triggering without interrupting the underlying audio pipeline means that audio data acquisition and buffering continue throughout the entire optimization triggering process, without stopping or resetting due to session switching or optimization initiation, thus ensuring the continuity of the user's speech.

[0073] In one implementation, the client maintains a silent timer for measuring idle time during the first speech recognition session, with an initial value of zero. Whenever the first speech recognition session outputs a new valid recognition event, the client resets the silent timer to zero. Simultaneously, the client starts or restarts a timer loop that increments the silent timer at fixed time intervals (e.g., every 100 milliseconds).

[0074] During the accumulation of the silence timer, the client continuously checks two conditions: whether a new valid recognition event has occurred during this period; and whether the current silence timer has reached the preset silence duration threshold. If the accumulated duration reaches the silence duration threshold without any new valid recognition events occurring, the client determines that the optimization trigger condition is met and automatically triggers optimization—that is, terminating the first speech recognition session, obtaining the snapshot text, and starting the second speech recognition session, etc.

[0075] Throughout the entire detection and triggering process described above, the client keeps the underlying audio pipeline running continuously, continuing to collect audio data from the microphone and send it to the speech recognition engine to ensure that the user's subsequent speech is not lost.

[0076] For example, a user says "The weather is nice today" and then pauses. The first speech recognition session outputs the recognized text "The weather is nice today," generating a valid recognition event, and the client resets the silence timer. The user then remains silent, and the silence timer continues to accumulate. When the accumulated time reaches a preset silence duration threshold, the client determines that the optimization trigger condition has been met, automatically triggers optimization, obtains the snapshot text "The weather is nice today," and starts the second speech recognition session within a preset delay threshold. The user then continues speaking "What about tomorrow?", and the second speech recognition session outputs the newly recognized text "What about tomorrow?"

[0077] The advantages of this optional embodiment are as follows: The client can automatically determine that the optimization trigger condition is met when the user's natural pause meets the silence duration threshold, and automatically trigger the optimization without requiring the user to manually click a button or issue a voice command. This method aligns the optimization timing with the natural rhythm of the user's speech, reducing interactive interference. Simultaneously, because the underlying audio pipeline maintains continuous acquisition throughout the process, the user's speech is not interrupted by automatic triggering, resulting in a smooth user experience. The silence timer's reset mechanism avoids accidental triggering during brief periods of user thought, improving the accuracy and reliability of automated judgment.

[0078] Optionally, the silence duration threshold can be adaptively adjusted based on the user's voice pattern after multiple optimization triggers.

[0079] In one implementation, the system maintains a record of the user's historical optimization trigger behaviors, including parameters such as whether the user continues speaking shortly after each automatic trigger, the distribution of pause intervals between adjacent voice inputs, and the user's speech rate characteristics. When the system detects that the user immediately continues speaking after multiple consecutive optimization triggers, it indicates that the current silence duration threshold may be lower than the user's natural pause rhythm, and the system automatically extends this threshold (e.g., from 10 seconds to 12 to 15 seconds). Conversely, if the system detects that the user frequently stops speaking during silence but the trigger is delayed, the threshold is appropriately shortened. This adaptive adjustment gradually converges to a steady-state value that matches the user's individual speaking habits during continuous use.

[0080] The advantages of this optional embodiment are as follows: Compared to using a fixed silence duration threshold, this implementation dynamically aligns the triggering timing with the individual user's speaking rhythm, reducing false triggers (the user is still thinking but the trigger interrupts them too early) and missed triggers (the user has stopped but the trigger does not respond for a long time) caused by improper threshold settings. At the same time, users do not need to manually calibrate the threshold parameters. The system continuously learns and adapts to the user's behavior patterns in the background, lowering the barrier to entry and improving the smoothness of interaction in automatically optimized triggering scenarios.

[0081] Optional, also includes: During the accumulation of the silent timer, a visual progress indicator is provided to the user; when a new valid recognized event is generated before the silent timer reaches the silent duration threshold, the silent timer is reset and the automatic trigger optimization is canceled.

[0082] In this context, a visual progress indicator refers to the graphical representation of the countdown to automatic optimization being triggered, displayed on the client's interface. This indicator can be a circular progress bar, a linear fill bar, a numerical countdown, or other dynamic visual elements. The purpose of the visual progress indicator is to allow users to clearly perceive how much time is left before automatic optimization is triggered, enabling them to anticipate the upcoming optimization and intervene by speaking when necessary.

[0083] The visual progress indicator displays the remaining time until automatic optimization is triggered. Remaining time refers to the length of time required from the current moment until the silent timer reaches its silent duration threshold. The client displays the changes in remaining time in real time through the visual progress indicator, such as the length of the progress bar or the decrease of the countdown numbers. The remaining time suggests to the user how much time they have left to think or pause. If the user does not wish to trigger optimization immediately, they can reset the timer by continuing to speak.

[0084] Cancel automatic trigger optimization means that when the client detects a new valid identified event before the silent timer reaches its silent duration threshold, it will not perform the automatic trigger optimization action, but will reset the silent timer to zero. Cancel automatic trigger optimization means the client abandons the current timeout determination, returns to monitoring mode, and waits for the next possible silent timeout. This mechanism avoids accidental triggering after a short pause by the user, ensuring that automatic optimization does not interrupt the user's natural speech.

[0085] In one implementation, the client simultaneously initiates a front-end update task while the silent timer is accumulating. The client calculates the remaining time based on the difference between the current accumulated value of the silent timer and the silent duration threshold. For example, if the silent duration threshold is 10 seconds and the current accumulated value is 3 seconds, then the remaining time is 7 seconds. The client converts this remaining time into a visual progress indicator, such as a progress bar from 0% to 100%, where the percentage completed = current accumulated value / silent duration threshold * 100%. The client refreshes the visual progress indicator at fixed intervals, allowing the user to see the progress bar gradually fill or the countdown numbers gradually decrease.

[0086] While refreshing the progress indicator, the client continues to monitor the first speech recognition session for any new valid recognition events. If the client detects a new valid recognition event before the silence timer reaches its silence duration threshold, it executes the following: resets the silence timer to zero, clears the current progress indicator (or sets it to zero), and cancels automatic trigger optimization, i.e., abandons the upcoming automatic trigger. The client then returns to its initial monitoring state, waiting for the next valid recognition event before restarting the silence timer.

[0087] The advantages of this optional embodiment are as follows: Firstly, the visual progress indicator makes the system's automatic behavior transparent and predictable, allowing users to decide whether to adjust their speaking pace based on the remaining time, thus enhancing the active participation of human-computer interaction.

[0088] Secondly, by allowing users to continue speaking before the timeout, automatic optimization is canceled, avoiding erroneous optimizations caused by involuntary pauses such as brief thinking or document browsing, thus improving the system's intelligence and accuracy.

[0089] Thirdly, the mechanism does not require additional manual operation or learning costs. Users can intervene in the timing simply by speaking naturally, maintaining the natural continuity of voice input and conforming to the intuition of human-computer interaction.

[0090] Optionally, a second speech recognition session may be initiated within a duration less than a preset delay threshold, including: Within a preset cross window before the first speech recognition session terminates, the last audio frame of the first speech recognition session is simultaneously sent to the second speech recognition session for preheating processing.

[0091] In one implementation, when the system detects the optimized trigger condition, before issuing an orderly stop command to the first speech recognition session, a session handle for the second speech recognition session is pre-established. Within a preset cross-window (e.g., 50 to 100 milliseconds), the last audio frames still being processed by the first speech recognition session are simultaneously diverted to the second speech recognition session. After receiving these audio frames during the warm-up phase, the second speech recognition session can complete the initialization inference of the acoustic and language models in advance. As a result, when it officially takes over the audio acquisition pipeline, the first character recognition latency is significantly reduced, and the user-perceived speech recognition continuity is further improved.

[0092] The advantages of this optional embodiment are as follows: Compared to session switching methods that rely solely on rapid cold starts, this implementation uses double-buffered cross-warm-up to provide a "warm-up" condition for the second speech recognition session by utilizing the audio frame at the end of the first speech recognition session, effectively compressing the first-word delay window during session switching. At the same time, since the preheating stage reuses the already collected audio data, the initial recognition result of the second speech recognition session is more naturally connected to the context of the user's speech, reducing transient recognition gaps caused by session switching and further enhancing the user's subjective perception of uninterrupted speech acquisition during concurrent optimization.

[0093] The applicant found that the existing technology lacks a cyclical multi-round optimization. During continuous voice acquisition, it is difficult for users to trigger optimization multiple times in a single session and seamlessly add new spoken content after each optimization, thus failing to form a natural multi-round interactive closed loop of "speaking - optimizing - speaking again - optimizing again".

[0094] Example 2 Figure 2 This is a flowchart of a current text update method provided in Embodiment 2 of the present invention. It further optimizes and expands the execution method following the step of "merging the optimized text with the newly recognized text output by the second speech recognition session into the current text in the working buffer" in the above technical solution, and can be combined with the various optional implementation methods described above. For example... Figure 2 As shown, the method includes: S201. Use the current text in the current working buffer as the new text to be optimized.

[0095] The text to be optimized refers to the current text in the current working buffer, obtained after the previous "collection-optimization-merging" cycle. This text serves as the raw material for the next round of optimization and will soon be sent to the language processing service for further processing. Essentially, the text to be optimized is the complete content from the previous working buffer, containing both the previously optimized results and the newly recognized text from the previous optimization. Its function is similar to snapshot text, but it originates from the working buffer rather than the first speech recognition session.

[0096] S202. In response to the detection of the optimization trigger condition again, terminate the second speech recognition session and obtain the text to be optimized as the new snapshot text.

[0097] The "re-detection of optimization trigger conditions" refers to the process by which the client, after completing a round of optimization and merging into the current text, re-determines that the optimization trigger conditions are met through detection methods (such as silent timeout, user-initiated operation, etc.). This condition is semantically consistent with the "optimization trigger conditions," but it is triggered after the current text has been formed, and is used to start a new round of concurrent optimization loop.

[0098] The new snapshot text refers to the recognized text accumulated up to the moment of the next trigger, obtained from the currently running speech recognition session (i.e., the second speech recognition session, or more generally, the speech recognition session currently being acquired) when the client determines that the optimization trigger conditions are met again. Similar to the snapshot text, this text is the material that will be sent to the language processing service for optimization. The name "new" is intended to distinguish it from the snapshot text in the previous cycle, describing the content to be optimized in the new round of optimization.

[0099] S203. Start a new speech recognition session within the preset delay threshold to continue collecting subsequent user speech and output new recognized text.

[0100] Here, a new speech recognition session refers to an instance of automatic speech recognition that is initiated by the client within a preset delay threshold after the previous speech recognition session (e.g., the second speech recognition session) is terminated. This session shares the underlying audio pipeline with the previous session and is used to continue collecting and recognizing subsequent user speech during optimized concurrency, and output the corresponding new recognized text.

[0101] Newly recognized text refers to the text content output after a new speech recognition session recognizes the subsequent speech uttered by the user during the current optimization round. It consists of newly added uttered content during the concurrent optimization process and will be merged with the returned optimized text upon completion of the new optimization. Because the number of iterations differs, the text content is generated in a new round, hence the name "newly recognized text."

[0102] S204. Send the new snapshot text to the language processing service and receive the new optimized text returned by the language processing service.

[0103] The newly optimized text refers to the optimized result text returned to the client by the language processing service after performing optimization processing on the new snapshot text. This text is the version after the language processing service has polished, corrected, or styled the new snapshot text. The name "new" indicates that it is a response to a new round of optimization requests, distinguishing it from the optimized text in the previous cycle.

[0104] S205. Merge the newly optimized text with the newly recognized text output by the new speech recognition session again into the current text in the working buffer.

[0105] In one implementation, after completing the initial optimization and merging the optimized text with the newly identified text into the current text in the working buffer, the client saves this current text as the new text to be optimized in the working buffer. The client continues to run, maintaining monitoring of the optimization trigger conditions. When the client detects the optimization trigger condition again, it executes a new round of concurrent optimization: First, the client terminates the currently running second speech recognition session (or the speech recognition session started in the previous loop if multiple loops have been performed) and retrieves the saved text to be optimized from the working buffer as the new snapshot text. Next, the client starts a new speech recognition session within a preset delay threshold to continue capturing subsequent dictated speech from the user during the optimization period and outputs new recognized text. This new session is completely independent of the previous session, sharing only the underlying audio pipeline, thus ensuring the continuity of audio acquisition.

[0106] The client then sends the new snapshot text to the language processing service and receives the newly optimized text returned by the service. The language processing service optimizes the new snapshot text, performing processes such as polishing, error correction, and style adjustment. Finally, the client merges the newly optimized text with the newly recognized text output from the new speech recognition session and writes it to the working buffer to form the updated current text.

[0107] For example, after the first round of merging, the current text in the working buffer is "Today's weather is sunny. What about tomorrow?". The user pauses and triggers optimization again. The client sends "Today's weather is sunny. What about tomorrow?" as the new snapshot text to the language processing service, and simultaneously starts a new speech recognition session. During this time, the user continues to say "What about the day after tomorrow?", and the new speech recognition session outputs the newly recognized text "What about the day after tomorrow?". The language processing service returns the newly optimized text "Today's weather is sunny. What about tomorrow?". The client merges the new optimized text with the new recognized text to obtain "Today's weather is sunny. What about tomorrow? What about the day after tomorrow?", and writes it to the working buffer. If the user continues to pause and triggers optimization again, the above process can be repeated, achieving an infinite number of rounds of optimization.

[0108] The beneficial effects of the embodiments of the present invention are as follows: Through the above implementation, the client can repeatedly execute the "acquisition-optimization-merging" process within a single continuous voice conversation, using the complete results of the previous round as the optimization material for the next round. This achieves cyclical multi-round optimization, naturally forming an interactive closed loop of "speak-optimize-speak again-optimize again." Since each round's new snapshot text contains all the optimized content from the first round to the previous round, the language processing service can perform coherent optimization based on global history, effectively avoiding issues such as inconsistent terminology and style. Simultaneously, users do not need to manually copy historical text or restart the conversation, significantly improving the interaction efficiency and user experience in scenarios involving long text creation and multiple revisions.

[0109] The applicant found that existing voice input technology cannot drive editing. When the selected text already exists in the target application, voice input can only insert new content or directly replace the selection area. It cannot intelligently rewrite according to the semantics of the selected text, which limits the application scope of voice interaction.

[0110] Example 3 Figure 3 This is a flowchart of a text rewriting method provided in Embodiment 3 of the present invention. The execution method of the above technical solution is further optimized and expanded, and can be combined with the various optional implementation methods described above. For example... Figure 3 As shown, the method includes: S301. In response to detecting that the user has selected text in the target application, obtain the content of the selected text.

[0111] The target application refers to a third-party application associated with or embedded in the client, such as a text editor, instant messaging window, browser input box, document processing software, etc. Users can perform text editing operations within this target application. This invention achieves the voice-to-text function through interaction with the target application.

[0112] Selecting text refers to the user actively selecting a section of text within the target application using methods such as mouse dragging, keyboard shortcuts, or long-pressing on a touchscreen. This operation results in an active text selection area within the target application. The client detects this operation by listening for system accessibility events or clipboard changes, using these as triggers to enter overwrite mode.

[0113] S302. Use the collected subsequent verbal content from the user as a rewriting instruction for the selected text content.

[0114] Subsequent user-generated speech refers to the newly spoken text captured by the user's microphone after the client detects the selected text. This speech is no longer the ordinary text to be optimized, but is interpreted as specific modification requests for the selected text.

[0115] Rewriting instructions are natural language descriptions or structured commands obtained after semantic parsing of the user's subsequent spoken content. These instructions tell the language processing service how to modify the selected text content. Rewriting instructions can include specific directions such as style adjustment, length control, structural changes, and terminology standardization, for example, "change the passive sentence to an active sentence" or "delete the second example." The client does not need to parse the specific meaning of the instructions; it only needs to send the user's subsequent spoken content, either verbatim or after simple editing, as rewriting instructions to the language processing service, which is responsible for understanding and executing them.

[0116] S303. The selected text content and the rewrite instruction are sent to the language processing service.

[0117] The selected text content refers to the text actually selected by the user through the text selection operation. This text content is usually highlighted in the target application and can be read by the client through the system interface. In this embodiment, the selected text content, as the original material to be rewritten, will be sent to the language processing service and modified according to the user's rewriting instructions.

[0118] S304. Receive the rewritten text returned by the language processing service and replace the selected text content with the rewritten text.

[0119] In this context, rewritten text refers to the modified text returned by the language processing service after optimizing the selected text content and rewriting instructions. The rewritten text is semantically related to the original selected text content, but may have changed its expression, structure, or length. After receiving the rewritten text, the client replaces the original selected text content in the target application with it, thereby enabling voice-driven editing operations.

[0120] In one implementation, the client uses accessibility interfaces provided by the operating system or clipboard monitoring mechanisms between applications to detect in real time whether the user has selected text in the target application. When the client detects this operation, it immediately calls the relevant interface to read the selected text content and temporarily stores it in memory. Simultaneously, the client activates a voice capture module, using the captured subsequent dictated content from the user as a rewriting instruction for the selected text content. The client does not parse the specific meaning of the dictated content; it only preserves it in its original text form.

[0121] Subsequently, the client encapsulates the selected text content and the rewrite instruction together as request parameters and sends them to the language processing service via Hypertext Transfer Protocol or WebSockets. Upon receiving the request, the language processing service modifies the selected text content according to the semantics of the rewrite instruction (e.g., polishing, rewriting, style conversion), and returns the modified result as the rewritten text to the client. After receiving the rewritten text, the client writes the rewritten text back to the target application through the same accessibility interface or clipboard replacement operation, replacing the original selected text content.

[0122] For example, suppose a user selects a text in a text editor: "The project is progressing smoothly and is expected to be completed next month." The client then detects this selection and retrieves the text. The user then utters: "To make it more formal, change 'smoothly' to 'steadily' and 'expected' to 'anticipate'." The client sends this uttered text as a rewriting instruction, along with the original text, to the language processing service. The server returns the rewritten text: "The project is progressing steadily and is expected to be completed next month." The client then replaces the original selection in the editor with this rewritten text.

[0123] The beneficial effects of the embodiments of the present invention are as follows: This implementation allows users to edit existing text in a target application simply by speaking natural language, without having to manually modify the text. This solves the problem that traditional voice input can only insert new content and cannot intelligently rewrite selected text. By sending the selected text and spoken commands to the language processing service, the server understands the user's intent and generates a rewritten result that meets the requirements, reducing the interaction cost for users editing text. This is particularly suitable for writing or translation scenarios that require repeated revisions and style adjustments. Furthermore, because the rewriting process seamlessly integrates with the user's subsequent spoken commands, it maintains the continuity and naturalness of the voice interaction.

[0124] Optionally, after receiving the rewritten text returned by the language processing service, the following may also be included: The received rewritten text is used as the current rewritten object; new spoken content from the user is collected and used as a new rewritten instruction; the current rewritten object and the new rewritten instruction are sent to the language processing service; the new rewritten text returned by the language processing service is received and the current rewritten object is replaced with the new rewritten text.

[0125] The current rewrite object refers to the text entity that needs further modification during multiple rounds of iterative rewrite by the client. Initially, the current rewrite object is the rewritten text returned by the language processing service. In each subsequent iteration, the client uses the new rewritten text received in the previous round as the current rewrite object for the new round of rewrite processing. The current rewrite object represents the latest rewrite result up to the current round, and users can continue to propose new modification requests based on this result.

[0126] New spoken content refers to a new spoken text input by the user via voice after the client has completed the previous round of rewriting and updated the current rewritten object. This content typically contains further modification intentions for the current rewritten object and serves as the original instruction material that triggers the new round of rewriting. New spoken content is directly collected by the client without undergoing semantic parsing or conversion.

[0127] The new rewriting instruction refers to the client directly submitting new spoken content as a modification request in natural language form, instructing the language processing service how to modify the current rewriting object. Similar to rewriting instructions, the new instruction does not require structured parsing by the client; it simply needs to be sent to the language processing service along with the current rewriting object, either verbatim or with slight modifications. The server is responsible for understanding the semantics of the instruction and performing the corresponding text modifications.

[0128] The new rewritten text refers to the modified text returned by the language processing service after optimizing the current rewritten object and new rewritten instructions. This text is the result of further iterations of the current rewritten object, and its content better reflects the user's latest modification intentions than the previous rewritten text. After receiving the new rewritten text, the client replaces the current rewritten object and can use it for the next iteration.

[0129] In one implementation, after completing the first rewrite, the client stores the received rewritten text as the current rewrite object in memory. The client continues to listen for the user's voice input. When the user utters a new modification request, the client captures the voice recognition result as the new uttered content and directly uses it as the new rewrite instruction.

[0130] Subsequently, the client encapsulates the current rewrite object along with the new rewrite instruction and sends it to the language processing service via the communication interface. The language processing service performs semantic analysis and text modification on the current rewrite object based on the new rewrite instruction, generating new rewrite text and returning it to the client. Upon receiving the new rewrite text, the client first replaces the current rewrite object with the new rewrite text using the target application's text replacement interface, and simultaneously updates the current rewrite object in memory with the new rewrite text, preparing for the next iteration. This process can be repeated any number of times as needed by the user, forming a multi-round iterative rewrite loop.

[0131] For example, in the first round, the user selects the original text "The project is progressing smoothly," and verbally says "Change to a passive sentence." The client receives the rewritten text "The project is progressing smoothly" and uses it as the current rewriting object. The user then verbally says "Add the completion time 'next month'," and the client uses this as a new rewriting instruction, sending it along with the current rewriting object to the language processing service. The server returns the new rewritten text "Next month, the project will be progressing smoothly." The client replaces the original text and updates the current rewriting object to "Next month, the project will be progressing smoothly." If the user then verbally says "Delete the word 'smoothly'," the client will repeat the above process, obtaining "Next month, the project will be progressing."

[0132] The advantages of this optional embodiment are as follows: The client allows users to continuously verbally request new revisions based on existing rewrite results, enabling multi-round iterative rewriting. Users do not need to reselect text or re-describe all revision intentions; they only need to incrementally propose adjustments to gradually approach the final desired text expression. This mechanism reduces the cognitive burden and operational steps in revising long sentences or complex paragraphs, making it particularly suitable for writing, translation, or proofreading scenarios that require repeated review and gradual refinement. Furthermore, since the client does not need to store historical instruction sequences, processing only the current object and current instruction each time results in low resource consumption, fast response speed, and a smooth interactive experience.

[0133] Example 4 Figure 4 This is a flowchart illustrating a method for command-based voice acquisition and optimized concurrent processing according to Embodiment 4 of the present invention. It further optimizes and expands the execution method of the above technical solution and can be combined with the various optional implementation methods described above. For example... Figure 4 As shown, the method includes: S401. During continuous voice acquisition, in response to the user pressing the command key, terminate the current voice recognition session and acquire the accumulated recognized text as candidate text.

[0134] S402. Start an independent command speech recognition session to collect user-spoked meta-instructions; wherein, the meta-instructions are used to indicate the optimization method for candidate text.

[0135] S403, in response to the user releasing the command key, terminate the command voice recognition session and obtain the collected meta-instructions.

[0136] S404. Send the candidate text and the collected meta-instructions together to the language processing service for optimization.

[0137] The operation of pressing the command key refers to the user pressing and holding a predefined function key on a physical or virtual input device. This operation is typically used to temporarily switch to command mode and initiate a separate command voice recognition session. In this embodiment, pressing the command key triggers a signal to switch from normal content acquisition mode to command voice acquisition mode, which is different from the normal triggering method of clicking or short pressing.

[0138] The accumulated recognized text refers to all the recognized text that has been recognized and output by the current speech recognition session before the user presses the command key. This text is the target content for the user to issue optimized instructions in command mode.

[0139] A command speech recognition session refers to a separate, automated speech recognition session instance initiated by the client after the user presses a command key. This session is logically isolated from the regular content acquisition session, and its output recognized text is treated as a meta-instruction, not the main text to be optimized. A command speech recognition session typically terminates after the user releases the command key, and its lifecycle is strictly limited to the duration of the key press.

[0140] Meta-instructions are natural language descriptions uttered by the user while pressing a command key, instructing how to optimize candidate text. Meta-instructions can include requirements such as style adjustments, terminology standardization, and length control, for example, "unify technical terms with English abbreviations" or "shorten to 300 words." Unlike regular optimization instructions, meta-instructions target existing candidate text, rather than newly uttered content. The client sends the meta-instructions as is to the language processing service, which then parses and applies them to the optimization of the candidate text.

[0141] Releasing the command key refers to the user releasing the command key they previously pressed. This action triggers the termination of the command speech recognition session; the client stops receiving meta-instructions and acquires the complete meta-instruction text recognized during this period. Releasing the command key marks the end of the command acquisition phase, after which the client sends the candidate text along with the meta-instructions to the language processing service for optimization.

[0142] In one implementation, during continuous voice acquisition, the client continuously monitors the status of preset command keys on the physical keyboard or touchscreen. When a user presses a command key, the client immediately terminates the currently running voice recognition session and reads all accumulated recognized text from the session's output buffer, storing it as candidate text in a temporary buffer. Simultaneously, the client starts an independent command voice recognition session that only receives the user's spoken voice during the command key press and outputs the recognized text in real time. The client accumulates the recognized text output by the command voice recognition session into the command buffer.

[0143] The client continues to listen for command key releases. Upon detecting a command key release, the client immediately terminates the command speech recognition session and reads the complete meta-instruction text from the instruction buffer. The client then encapsulates the candidate text and meta-instruction together into a request and sends it to the language processing service via the communication interface. The language processing service optimizes the candidate text based on the semantics of the meta-instruction and returns the optimized text. Upon receiving the optimized text, the client can either use it as the current text in its working buffer or display it to the user.

[0144] For example, a user utters "The main goal of this project is to improve the user experience," presses and holds the command key, and then utters "Change 'main goal' to 'core task' and 'improve' to 'optimize'." During the press, the client initiates a command speech recognition session, capturing the meta-instruction "Change 'main goal' to 'core task' and 'improve' to 'optimize'." After the user releases the command key, the client sends the candidate text ("The main goal of this project is to improve the user experience") along with the meta-instruction to the language processing service. The server returns the optimized text "The core task of this project is to optimize the user experience." The client either replaces the original text with this optimized text or stores it in the working buffer.

[0145] The beneficial effects of the embodiments of the present invention are as follows: Users can press the command key at any time during continuous voice acquisition without stopping their speech to directly modify the accumulated recognized text, achieving a natural interactive loop of "speak-press-speak-release-optimize". This mechanism simplifies the complex operation that traditionally required manual editing or re-entering voice into a single press and speech, significantly improving modification efficiency. Simultaneously, the command voice recognition session is independent of the content acquisition session, avoiding the mixing of meta-instructions and the main text, enabling the language processing service to accurately distinguish between "materials to be optimized" and "optimization rules," thus improving instruction compliance. Furthermore, the press-and-release interaction method is intuitive and has a low learning curve, making it suitable for writing or programming scenarios that require frequent text adjustments.

[0146] Optionally, after initiating a separate command speech recognition session, it also includes: If no valid meta-instructions are collected through the command speech recognition session, a default optimization rule is generated. The candidate text and the default optimization rule are then sent to the language processing service, which optimizes the candidate text according to the default optimization rule.

[0147] Among them, "no valid meta-instruction collected" means that during the operation of the command speech recognition session (i.e., the time period from when the user presses the command key to when the user releases the command key), the client failed to obtain natural language instruction text that meets the preset validity criteria from the output of the session.

[0148] Default optimization rules refer to a set of preset standard text conversion strategies that are automatically activated when the client fails to capture valid meta-instructions during a command speech recognition session (e.g., the user did not speak while pressing the command key, or the spoken content was not recognized as a valid instruction). Default optimization rules do not rely on the user's specific spoken instructions; instead, they serve as a fallback mechanism to ensure that the optimization process can still be executed and produce acceptable results.

[0149] In one implementation, after the user releases the command key and terminates the command speech recognition session, the client checks whether a valid meta-instruction exists in the instruction buffer. If the instruction buffer is empty or its content is invalid, the client determines that no valid meta-instruction was collected. At this time, the client reads a set of preset default optimization rules from local storage or server-side pre-configured settings, such as "Please perform grammatical correction on the candidate text, remove colloquial filler words, standardize punctuation, and maintain the original meaning." The client encapsulates this default optimization rule, in a form equivalent to the user's meta-instruction, along with the candidate text, into a request and sends it to the language processing service. Upon receiving the request, the language processing service performs optimization processing on the candidate text according to the default optimization rules and returns the optimized text.

[0150] The advantages of this optional embodiment are as follows: Even when the user doesn't provide a valid meta-instruction in command mode, the client can automatically generate default optimization rules and complete the optimization process, avoiding empty requests or processing interruptions caused by user errors, brief microphone malfunctions, or recognition failures. Default optimization rules provide consistent basic optimization capabilities, enabling the system to provide valuable text improvements even without explicit user instructions, enhancing the stability of the interaction and the consistency of the user experience. Simultaneously, this mechanism reduces the cognitive burden on users—they don't always need to remember or verbally recite complex instructions, and occasional accidental triggering or forgetting to verbally recite instructions won't cause optimization tasks to fail. For batch processing or long-term use scenarios, default rules ensure at least one basic optimization after each round of voice acquisition, enhancing the system's usability and robustness.

[0151] Optional, also includes: 1) Maintain the current logical state during continuous voice acquisition.

[0152] In this context, logical state refers to the current operational stage or working mode of the client during the continuous voice acquisition and optimization process. Logical state reflects the client's primary behavior at a given moment. By maintaining and migrating logical state, the client can manage various functions such as concurrent sessions, silent triggering, command interaction, and rewriting modes in an orderly manner, ensuring predictable and scalable behavior.

[0153] The logical states include idle state, content acquisition state, command acquisition state, acquisition state during optimization, rewrite mode state, and confirmation state.

[0154] The idle state refers to the initial state where the client has not yet started voice acquisition or has actively stopped acquisition. In this state, the client does not conduct any voice recognition sessions, nor does it initiate language processing service calls.

[0155] Content acquisition state refers to the state in which the client is in normal voice content acquisition, maintaining a regular speech recognition session to receive the user's dictated text. In this state, the client continuously outputs the recognized text and accumulates it in the working buffer.

[0156] Command acquisition state refers to the temporary state that the client enters after the user presses the command key, specifically for acquiring meta-instructions. In this state, the client runs an independent command speech recognition session, where the user's spoken content is interpreted as optimized instructions for candidate text, rather than the main text.

[0157] The "Optimization Period Acquisition State" refers to the concurrent state where, after triggering optimization, the client sends a snapshot to the language processing service for optimization, while the second speech recognition session continues to acquire subsequent user speech. In this state, the client is simultaneously performing optimization processing and real-time speech acquisition. The "Optimization Period Acquisition State" awaits the language processing service's completion event; upon completion, it transitions back to the content acquisition state and merges the results.

[0158] Rewrite mode is a special state that the client automatically enters after detecting that the user has selected text in the target application. In this state, subsequent verbal commands from the user are interpreted as rewrite instructions for the selected text, rather than ordinary text to be optimized. Rewrite mode allows users to edit existing text using natural language commands, and allows for multiple iterations.

[0159] The confirmation status refers to the intermediate state in which the client waits for the user's confirmation of whether to accept the current optimization or rewrite result. The confirmation status typically appears after optimization or rewriting is complete, at which point the user can choose to accept and replace the original text, or cancel and revert to the original content.

[0160] 2) In response to at least one of the following events, the current logical state is transitioned according to a predetermined transition rule: In response to a voice acquisition start event, the system transitions from idle to content acquisition state; in response to a voice acquisition stop event, it transitions from content acquisition state to idle state; in response to an optimization trigger event, it transitions from content acquisition state to acquisition state during optimization; in response to a command acquisition start event, it transitions from content acquisition state to command acquisition state; in response to a command acquisition end event, it transitions from command acquisition state to acquisition state during optimization; in response to a language processing service completion event, it transitions from acquisition state during optimization to content acquisition state; in response to a text selection detection event, it transitions from idle or content acquisition state to rewrite mode state; in response to a confirmation operation event, it transitions from content acquisition state or acquisition state during optimization to confirmation state; in response to a confirmation completion event, it transitions from confirmation state to content acquisition state or idle state.

[0161] The voice acquisition start event refers to the event that occurs when the client starts acquiring audio from the microphone and creating the first voice recognition session by means of the user clicking the start button, voice wake-up, or automatic system triggering.

[0162] The voice capture stop event refers to the event that occurs when the user actively stops voice capture or when the client closes audio capture due to timeout, error, or other reasons.

[0163] Optimization trigger events refer to events generated when the client detects that the optimization trigger conditions are met.

[0164] The command acquisition start event refers to the event generated when the client detects that the user has pressed a command key.

[0165] The command acquisition end event refers to the event generated when the client detects that the user has released the command key.

[0166] The Language Processing Service Completion Event refers to the event that occurs after the client receives the optimized (or rewritten) text returned by the Language Processing Service and completes the processing of the result.

[0167] The text selection detection event refers to the event that occurs when the client detects that a user has selected a piece of text in the target application through the operating system's accessibility interface or clipboard listening.

[0168] The confirmation operation event refers to the event that occurs when a user confirms and accepts the intermediate result after rewriting the mode or optimizing the system.

[0169] The confirmation completion event refers to the event that occurs after the user explicitly indicates acceptance of the current result in the confirmation state (such as clicking "OK").

[0170] The advantages of this optional embodiment are as follows: This makes the client's behavior predictable and the switching conflict-free under multi-mode collaboration. At the same time, the closed loop of confirming the state ensures the security of the result and the fault tolerance of the interaction, and provides a unified framework foundation for future state expansion.

[0171] Optionally, embodiments of the present invention can be implemented using general-purpose computing devices, including but not limited to: tablet devices or 2-in-1 devices with touchscreens, personal computers in a traditional keyboard and mouse environment, wearable devices (such as smartwatches and smart glasses), and voice remote control devices. As long as the device has continuous audio acquisition capability (i.e., can continuously receive user voice streams from a microphone), provides an automatic speech recognition engine interface, and can connect to (locally or via a network) language processing services, embodiments of the present invention can be implemented.

[0172] This invention does not rely on a specific input method framework or a specific mobile terminal form factor. In actual deployment, the integration of general-purpose computing devices with target applications (such as text editors, instant messaging software, document processors, etc.) can be achieved through at least one of the following methods: system-level auxiliary services, accessibility channels, clipboard data exchange, accessibility application programming interfaces, or software development kits embedded in the target application. Regardless of the integration method used, the core definition of this invention remains unchanged—that is, concurrently performing speech recognition session switching and language processing service optimization during continuous speech acquisition.

[0173] With the above deployment configuration, users can obtain a continuous speaking and real-time text optimization experience with low perceptible interruption on any computing device that supports continuous audio acquisition and language processing services, without being limited by specific application environments or input method frameworks.

[0174] Optionally, embodiments of the present invention also provide several alternative solutions: Alternative Solution 1: Optimize the scope and granularity.

[0175] In the main approach, the snapshot text is typically the entire segment of recognized text accumulated before the optimization trigger time. Alternatively, the client can limit the optimization scope to the most recent N sentences of recognized text, the recognized text corresponding to the most recent M seconds of audio, or only optimize segments with high confidence levels, leaving low-confidence segments for the next round of optimization or for manual verification. This alternative approach reduces the computational cost and response latency of a single language processing service, but may weaken the overall text coherence and polishing effect, requiring coordinated design with the interface's merging rules. In practical applications, the choice can be flexibly made based on the user's different requirements for real-time performance and accuracy.

[0176] Alternative Solution 2: Different implementation forms of language processing services.

[0177] In the main solution, language processing services are primarily implemented using a streaming, large language model approach. Alternatively, non-streaming Hypertext Transfer Protocol (HTTP) requests, locally deployed small language models, or a hybrid approach based on rules and templates can be employed (e.g., dictionary replacement for proper nouns and invoking the language processing service for long sentences). This alternative reduces reliance on the network and mitigates data privacy risks, but may result in decreased expression quality and semantic generalization capabilities. Developers can choose the appropriate service model based on device computing power and application scenarios.

[0178] Alternative Option 3: Second speech recognition session initiation strategy.

[0179] In the main scheme, the second speech recognition session starts immediately after the first speech recognition session terminates. As an alternative, a double-buffering mechanism can be used: within a very short crossfade window, the last few audio frames of the first speech recognition session are simultaneously sent to the second speech recognition session for warm-up, reducing first-character recognition latency; or the session identifier in the user interface is switched only after the second speech recognition session receives the first non-empty recognized text, reducing user-visible session switching jitter. This alternative can further smooth out the first-character latency, but its implementation has a higher coupling with the automatic speech recognition engine vendor interface and needs to be adjusted according to the specific API characteristics.

[0180] Alternative Solution 4: Adaptive Strategy for Silence Duration Threshold.

[0181] In the main solution, automatic silence triggering uses a fixed, configurable duration threshold (e.g., 10 seconds). As an alternative, a tiered threshold strategy can be adopted: initially using a shorter silence duration threshold (e.g., 5 seconds), if the user continues speaking at a high speed immediately after optimization multiple times, the subsequent threshold is automatically extended to 12-15 seconds; alternatively, the silence duration threshold can be dynamically adjusted based on the ambient noise level or the confidence level of the automatic speech recognition results. This alternative solution reduces false triggers and missed triggers, but increases the complexity of parameter calibration and tuning, making it suitable for advanced users or professional scenarios.

[0182] Alternative Option 5: Different triggering methods for rewriting mode.

[0183] In the main solution, rewrite mode is automatically activated by detecting a text selection in the target application. Alternatively, rewrite mode can be explicitly activated via a dedicated button, gesture, or voice hotword (e.g., the user says "rewrite" followed by the command), without relying on text selection detection. This alternative avoids accidentally activating rewrite mode when the user only wants to replace the selected content (not rewrite), but adds an extra step. Both triggering methods can coexist according to user preference, allowing the user to choose.

[0184] Alternative Option Six: Dual Role of Oral Content (Combination of Instructions and Text).

[0185] In the main solution, user-uttered content in rewrite mode is interpreted entirely as rewrite instructions. Alternatively, user-uttered content can include both rewrite instructions and supplementary text: the system uses language processing services or preset rules to determine which parts of the uttered content are rewrite instructions and which parts are new text to be appended to the end of the selected area. This alternative provides a more flexible interaction method, but increases the complexity and error probability of identifying the boundary between instructions and text, typically requiring stronger semantic understanding capabilities integrated on the client or server side.

[0186] Example 5 Figure 5This is a schematic diagram of a device for concurrent processing of voice acquisition and optimization provided in Embodiment 5 of the present invention. It is applicable to situations where a user needs to trigger language processing service optimization multiple times during continuous voice acquisition, and continues speaking uninterruptedly during the optimization period. Figure 5 As shown, the device includes: The text recognition output module 51 is used to maintain the first speech recognition session during continuous speech acquisition to receive user speech and output recognized text. The snapshot text acquisition module 52 is used to terminate the first speech recognition session when an optimization trigger condition is detected, and to acquire the recognition text accumulated up to the time the optimization trigger condition is detected as snapshot text. The new recognized text output module 53 is used to start a second speech recognition session within a time period less than a preset delay threshold, so as to continue to collect the user's subsequent speech and output new recognized text. The optimized text receiving module 54 is used to send the snapshot text to the language processing service while the second speech recognition session outputs the new recognized text, so that the language processing service can optimize the snapshot text and receive the optimized text returned by the language processing service. The text merging module 55 is used to merge the optimized text with the newly recognized text output by the second speech recognition session into the current text in the working buffer after receiving the optimized text.

[0187] Optionally, the device further includes a current text update module, specifically used for: Use the current text in the current working buffer as the new text to be optimized; In response to the detection of the optimization trigger condition again, the second speech recognition session is terminated, and the text to be optimized is obtained as a new snapshot text; A new speech recognition session is initiated within the preset delay threshold to continue collecting subsequent user speech and output new recognized text; The new snapshot text is sent to the language processing service, and the new optimized text returned by the language processing service is received. The newly optimized text is then merged again with the newly recognized text output by the new speech recognition session into the current text in the working buffer.

[0188] Optionally, the methods for detecting the optimized triggering conditions include: Listen for valid recognition events output by the first speech recognition session; Reset the silent timer whenever a new valid recognition event occurs; During the accumulation of the silent timer, if the silent timer reaches the preset silent duration threshold without generating a new valid identification event, the optimization triggering condition is deemed met, and optimization is automatically triggered. Among them, the underlying audio pipeline that automatically triggers without interrupting the voice acquisition session.

[0189] Optionally, the device further includes a progress visualization module, specifically used for: During the silent timer's accumulation process, a visual progress indicator is provided to the user; wherein, the visual progress indicator is used to display the remaining time until automatic triggering of optimization; If a new valid identification event is generated before the silence timer reaches the silence duration threshold, the silence timer is reset and automatic triggering optimization is canceled.

[0190] Optionally, the device further includes a text rewriting module, specifically used for: In response to detecting that a user has selected text in the target application, the content of the selected text is obtained; The collected subsequent verbal content from the user will be used as a rewriting instruction for the selected text content; The selected text content and the rewrite instruction are sent together to the language processing service; Receive the rewritten text returned by the language processing service, and replace the selected text content with the rewritten text.

[0191] Optionally, the device further includes a text rewriting iteration module, specifically used for: The received rewritten text is used as the current rewriting object; Continue to collect new verbal content from the user and use this new verbal content as a new rewrite instruction; The current object to be rewritten and the new rewriting instruction are sent together to the language processing service; Receive the new rewritten text returned by the language processing service, and replace the current rewritten object with the new rewritten text.

[0192] Optionally, the device further includes a command-based voice processing module, specifically used for: During continuous voice acquisition, in response to the user pressing the command key, the current voice recognition session is terminated, and the accumulated recognized text is acquired as candidate text. Initiate a separate command speech recognition session to collect user-spoked meta-instructions; wherein the meta-instructions are used to indicate the optimization method for the candidate text; In response to the user releasing the command key, the command voice recognition session is terminated, and the collected meta-instructions are acquired. The candidate text and the collected meta-instructions are sent together to the language processing service for optimization.

[0193] Optionally, the device further includes an optimization redundancy module, specifically used for: If no valid meta-instructions are collected through the command speech recognition session, a default optimization rule is generated; The candidate text and the default optimization rule are sent together to the language processing service, which then optimizes the candidate text according to the default optimization rule.

[0194] Optionally, the device further includes a logic state management module, specifically used for: During continuous voice acquisition, the current logical state is maintained; wherein, the logical state includes idle state, content acquisition state, command acquisition state, acquisition state during optimization, rewrite mode state, and confirmation state; In response to at least one of the following events, the current logical state is transitioned according to a predetermined transition rule: In response to a voice acquisition start event, the system transitions from the idle state to the content acquisition state. In response to a voice acquisition stop event, the system transitions from the content acquisition state to the idle state. In response to the optimization trigger event, the process transitions from the content acquisition state to the acquisition state during the optimization period; In response to the command acquisition start event, the system transitions from the content acquisition state to the command acquisition state. In response to a command acquisition end event, the process transitions from the command acquisition state to the acquisition state during optimization. In response to the language processing service completion event, the process transitions from the acquisition state during optimization to the content acquisition state. In response to a text selection detection event, the system transitions from the idle state or the content acquisition state to the rewrite mode state. In response to a confirmation operation event, the process transitions from the content acquisition state or the acquisition state during optimization to the confirmation state. In response to a confirmation completion event, the system transitions from the confirmation state to the content acquisition state or the idle state.

[0195] Optionally, the preset delay threshold is less than 100 milliseconds.

[0196] Optionally, the newly recognized text output module 53 is specifically used for: Within a preset cross window before the first speech recognition session terminates, the last audio frame of the first speech recognition session is simultaneously sent to the second speech recognition session for preheating processing.

[0197] Optionally, the silence duration threshold can be adaptively adjusted based on the user's voice pattern after multiple optimization triggers.

[0198] The speech acquisition and optimization concurrent processing apparatus provided in this embodiment of the invention can execute the speech acquisition and optimization concurrent processing method provided in this embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0199] According to embodiments of this disclosure, embodiments of the present invention also provide an electronic device, a readable storage medium, and a computer program product.

[0200] Figure 6 A schematic diagram of an electronic device 60 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0201] like Figure 6 As shown, the electronic device 60 includes at least one processor 61 and a memory, such as a read-only memory (ROM) 62 and a random access memory (RAM) 63, communicatively connected to the at least one processor 61. The memory stores computer programs executable by the at least one processor. The processor 61 can perform various appropriate actions and processes based on the computer program stored in the ROM 62 or loaded from storage unit 68 into the RAM 63. The RAM 63 can also store various programs and data required for the operation of the electronic device 60. The processor 61, ROM 62, and RAM 63 are interconnected via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0202] Multiple components in electronic device 60 are connected to I / O interface 65, including: input unit 66, such as keyboard, mouse, etc.; output unit 67, such as various types of monitors, speakers, etc.; storage unit 68, such as disk, optical disk, etc.; and communication unit 69, such as network card, modem, wireless transceiver, etc. Communication unit 69 allows electronic device 60 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0203] Processor 61 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 61 performs the various methods and processes described above, such as methods for concurrent processing of voice acquisition and optimization.

[0204] In some embodiments, the method for concurrent voice acquisition and optimization processing can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 60 via ROM 62 and / or communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the method for concurrent voice acquisition and optimization processing described above can be performed. Alternatively, in other embodiments, processor 61 can be configured to perform the method for concurrent voice acquisition and optimization processing by any other suitable means (e.g., by means of firmware).

[0205] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0206] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0207] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0208] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0209] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0210] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system to address the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0211] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0212] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for concurrent processing and optimization of voice acquisition, characterized in that, The method includes: During continuous voice acquisition, the first speech recognition session is maintained to receive user voice and output recognized text; When an optimization trigger condition is detected, the first speech recognition session is terminated, and the accumulated recognition text up to the time the optimization trigger condition is detected is obtained as snapshot text. Initiate a second speech recognition session within a time frame less than a preset delay threshold to continue collecting subsequent user speech and output new recognized text; While the second speech recognition session outputs the new recognized text, the snapshot text is sent to the language processing service for optimization, and the optimized text returned by the language processing service is received. Upon receiving the optimized text, the optimized text is merged with the newly recognized text output by the second speech recognition session to form the current text in the working buffer.

2. The method according to claim 1, further comprising, after merging the optimized text with the newly recognized text output by the second speech recognition session into the current text in the working buffer: Use the current text in the current working buffer as the new text to be optimized; In response to the detection of the optimization trigger condition again, the second speech recognition session is terminated, and the text to be optimized is obtained as a new snapshot text; A new speech recognition session is initiated within the preset delay threshold to continue collecting subsequent user speech and output new recognized text; The new snapshot text is sent to the language processing service, and the new optimized text returned by the language processing service is received. The newly optimized text is then merged again with the newly recognized text output by the new speech recognition session into the current text in the working buffer.

3. The method according to claim 1, characterized in that, The methods for detecting the optimized triggering conditions include: Listen for valid recognition events output by the first speech recognition session; Reset the silent timer whenever a new valid recognition event occurs; During the accumulation of the silent timer, if the silent timer reaches the preset silent duration threshold without generating a new valid identification event, the optimization triggering condition is deemed met, and optimization is automatically triggered. Among them, the underlying audio pipeline that automatically triggers without interrupting the voice acquisition session.

4. The method according to claim 3, characterized in that, Also includes: During the silent timer's accumulation process, a visual progress indicator is provided to the user; wherein, the visual progress indicator is used to display the remaining time until automatic triggering of optimization; If a new valid identification event is generated before the silence timer reaches the silence duration threshold, the silence timer is reset and automatic triggering optimization is canceled.

5. The method according to claim 1, characterized in that, Also includes: In response to detecting that a user has selected text in the target application, the content of the selected text is obtained; The collected subsequent verbal content from the user will be used as a rewriting instruction for the selected text content; The selected text content and the rewrite instruction are sent together to the language processing service; Receive the rewritten text returned by the language processing service, and replace the selected text content with the rewritten text.

6. The method according to claim 5, further comprising, after receiving the rewritten text returned by the language processing service: The received rewritten text is used as the current rewriting object; Continue to collect new verbal content from the user and use this new verbal content as a new rewrite instruction; The current object to be rewritten and the new rewriting instruction are sent together to the language processing service; Receive the new rewritten text returned by the language processing service, and replace the current rewritten object with the new rewritten text.

7. The method according to claim 1, characterized in that, Also includes: During continuous voice acquisition, in response to the user pressing the command key, the current voice recognition session is terminated, and the accumulated recognized text is acquired as candidate text. Initiate a separate command speech recognition session to collect user-spoked meta-instructions; wherein the meta-instructions are used to indicate the optimization method for the candidate text; In response to the user releasing the command key, the command voice recognition session is terminated, and the collected meta-instructions are acquired. The candidate text and the collected meta-instructions are sent together to the language processing service for optimization.

8. The method according to claim 7, further comprising, after initiating an independent command speech recognition session: If no valid meta-instructions are collected through the command speech recognition session, a default optimization rule is generated; The candidate text and the default optimization rule are sent together to the language processing service, which then optimizes the candidate text according to the default optimization rule.

9. The method according to claim 1, characterized in that, Also includes: During continuous voice acquisition, the current logical state is maintained; wherein, the logical state includes idle state, content acquisition state, command acquisition state, acquisition state during optimization, rewrite mode state, and confirmation state; In response to at least one of the following events, the current logical state is transitioned according to a predetermined transition rule: In response to a voice acquisition start event, the system transitions from the idle state to the content acquisition state. In response to a voice acquisition stop event, the system transitions from the content acquisition state to the idle state. In response to the optimization trigger event, the process transitions from the content acquisition state to the acquisition state during the optimization period; In response to the command acquisition start event, the system transitions from the content acquisition state to the command acquisition state. In response to a command acquisition end event, the process transitions from the command acquisition state to the acquisition state during optimization. In response to the language processing service completion event, the process transitions from the acquisition state during optimization to the content acquisition state. In response to a text selection detection event, the system transitions from the idle state or the content acquisition state to the rewrite mode state. In response to a confirmation operation event, the process transitions from the content acquisition state or the acquisition state during optimization to the confirmation state. In response to a confirmation completion event, the system transitions from the confirmation state to the content acquisition state or the idle state.

10. The method according to claim 1, characterized in that, The preset delay threshold is less than 100 milliseconds.

11. The method according to claim 1, characterized in that, The step of initiating a second speech recognition session within a duration less than a preset delay threshold includes: Within a preset cross window before the first speech recognition session terminates, the last audio frame of the first speech recognition session is simultaneously sent to the second speech recognition session for preheating processing.

12. The method according to claim 3, characterized in that, The silence duration threshold is adaptively adjusted based on the user's voice mode after multiple optimization triggers.

13. A device for concurrent processing and optimization of voice acquisition, characterized in that, The device includes: The text recognition output module is used to maintain the first speech recognition session during continuous speech acquisition to receive user speech and output recognized text. The snapshot text acquisition module is used to terminate the first speech recognition session when an optimization trigger condition is detected, and to acquire the accumulated recognition text up to the time the optimization trigger condition is detected as snapshot text. The new recognized text output module is used to start a second speech recognition session within a time period less than a preset delay threshold, so as to continue to collect the user's subsequent speech and output new recognized text. The optimized text receiving module is used to send the snapshot text to the language processing service while the second speech recognition session outputs the newly recognized text, so that the language processing service can optimize the snapshot text and receive the optimized text returned by the language processing service. The text merging module is used to merge the optimized text with the newly recognized text output by the second speech recognition session into the current text in the working buffer after receiving the optimized text.

14. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method of any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-12.

Images