A seamless fallback method, system, device, and medium for streaming speech translation

Abstract

The application discloses a seamless degradation method for streaming speech translation, comprising the following steps: constructing an audio buffer queue independent of a speech recognition engine and a translation engine on a server, and appending all received audio streams to the audio buffer queue; constructing a speech recognition degradation link comprising at least two speech recognition engines, and a translation degradation link comprising at least two translation engines; if the current speech recognition engine fails, starting the next priority speech recognition engine, and migrating the session core metadata of the audio buffer queue to the new engine through a state transition algorithm, so that the new engine continues to recognize from the queue breakpoint; if the current translation engine fails, retaining the original text and context information of the current translation task, starting the next priority translation engine, and injecting the original text and context information into the new engine for silent retry. Seamless switching is realized when the underlying service fails.

Description

Technical Field

[0001] This invention relates to the field of speech recognition and translation technology, and in particular to a seamless degradation method, system, device and medium for streaming speech translation. Background Technology

[0002] In real-time speech translation and simultaneous interpretation scenarios, streaming speech recognition (ASR) technology is commonly used to meet the low latency requirements. The terminal device and the management server, as well as the management server and the cloud service ASR provider, transmit audio streams and processing results through persistent WebSocket connections.

[0003] However, in real-world production environments, failures typically occur during the connection establishment phase between the terminal and the server, and between the server and the cloud engine. These failures include recoverable issues such as TCP connection establishment failures, TLS handshake timeouts, API authentication errors, server-side concurrency throttling, and temporary node unavailability. Once the connection is successfully established, the underlying network links and cloud services will remain stable throughout the entire session lifecycle, with fewer abnormal interruptions.

[0004] like Figure 1 As shown, in traditional disaster recovery solutions, when the underlying speech recognition or translation service (LLM) fails, the management server typically passively disconnects the persistent connection with the terminal and throws an exception. This approach has the following problems: 1. The server will only attempt to connect to a single preset primary speech recognition engine and a single primary translation engine. If the primary engine fails to connect due to any of the above recoverable failures, the server will directly return a "connection error" to the terminal. The user must manually re-initiate the connection request and try multiple times to succeed, which seriously affects the user experience.

[0005] 2. The terminal begins collecting and sending audio data simultaneously with initiating the connection request. In traditional solutions, if the server fails to connect to the cloud engine, the received audio frames will be discarded directly, resulting in a "swallowing of the first character" phenomenon and incomplete recognition results.

[0006] 3. Even if some systems support manual switching of alternative engines, they cannot inherit the received audio data and session status. The new engine must start processing again, resulting in duplicate recognition or data omission. Summary of the Invention

[0007] In order to overcome the shortcomings of existing technologies such as data loss and fragmented user experience.

[0008] In a first aspect, the present invention provides a seamless degradation method for streaming speech translation, comprising the following steps: An audio buffer queue, independent of specific speech recognition and translation engines, is built on the server side, and audio frames sent by the terminal are written into the audio buffer queue. Construct a speech recognition degradation link that includes at least two speech recognition engines, and a translation degradation link that includes at least two translation engines; After the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process, the next priority speech recognition engine is started. The session core metadata of the audio buffer queue is migrated to the new engine through a state transition algorithm, so that the new engine can continue recognition from the queue breakpoint. If the current translation engine fails, the original text and context information of the current translation task are retained, the next priority translation engine is started, and the original text and context information are injected into the new engine for silent retry.

[0009] Optionally, the server can intercept abnormal signals and maintain a persistent communication connection with the terminal after the current speech recognition engine fails.

[0010] Optionally, the session core metadata includes at least one of the following: audio buffer queue object reference, queue access handle, task unique identifier, stop event, sentence break event, language parameters, audio encoding, decoding status, historical context, cumulative audio duration, billing status, performance statistics status, and timestamp status.

[0011] Optionally, the context information includes historical dialogue context and user-specific translation dictionary.

[0012] Optionally, if the speech recognition engine or translation engine fails, a circuit breaker operation is performed, and during the circuit breaker period, new business requests are scheduled to other available engines.

[0013] Optionally, the circuit breaker operation is performed based on a distributed caching system, setting an unavailable flag with a valid lifetime for the failed speech recognition engine or translation engine.

[0014] Optionally, a speech recognition circuit breaker is configured in the speech recognition degradation link, and a translation circuit breaker is configured in the translation degradation link. The speech recognition circuit breaker is used to perform the circuit breaking operation within the speech recognition engine, and the translation circuit breaker is used to perform the circuit breaking operation within the translation engine.

[0015] Secondly, the present invention provides a seamless degradation system for streaming speech translation, comprising: The session management module is used to build an audio buffer queue on the server side that is independent of the speech recognition engine and the translation engine, and to append all received audio streams to the audio buffer queue. The link configuration module is used to build a speech recognition degradation link containing at least two speech recognition engines and a translation degradation link containing at least two translation engines, and to determine the order of candidate engines based on preset priority, language support status and circuit breaker status. The degradation processing module is used so that after the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process, the next priority speech recognition engine is started. The session core metadata of the audio buffer queue is migrated to the new engine through a state transition algorithm, so that the new engine can continue recognition from the queue breakpoint. If the current translation engine fails, the original text and context information of the current translation task are retained, the next priority translation engine is started, and the original text and context information are injected into the new engine for silent retry.

[0016] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0017] Fourthly, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the method described in the first aspect.

[0018] The beneficial effects of this invention are as follows: The independent audio buffer queue fundamentally solves the problem of "audio data loss directly caused by underlying engine failure" in traditional solutions. All audio data has a unified caching layer on the server side. Even if all underlying engines fail simultaneously, the audio data will be safely stored in the queue, preventing "word swallowing" (data loss). Simultaneously, the decoupled design makes the switching of underlying engines completely transparent to the audio reception process, requiring no awareness of any underlying changes at the terminal. The dual-downgrade link provides independent redundancy guarantees for the two core processing stages of speech recognition and translation, avoiding the risk of a single service provider failure causing the entire service to crash. The multi-vendor heterogeneous link design further reduces the impact of regional failures or overall service provider outages, significantly improving the system's disaster recovery capabilities. The application of the state transition algorithm enables "hot-swappable" switching of the speech recognition engine. The new engine can seamlessly resume processing from the point of interruption without restarting recognition, ensuring the continuity and integrity of the recognition results and avoiding duplicate or interrupted recognition. When the translation engine malfunctions, the user is completely unaware of it. The system automatically retryes and switches in the background, and the complete preservation of context ensures the semantic coherence of the translation results, preventing translation errors or ambiguities caused by loss of context. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Figure 1 This is a flowchart of disaster recovery processing in traditional streaming speech translation; Figure 2 These are flowcharts of the degradation processing during the speech recognition stage in some embodiments; Figure 3 These are flowcharts of the downgrade processing during the translation stage in some embodiments; Figure 4 Here is a flowchart of seamless state transition for speech recognition based on audio buffer queue pointer sharing in some embodiments; Figure 5 These are topologies of dual-link high-availability circuit breaker architectures in some embodiments; Figure 6 This is a flowchart of a seamless degradation method for streaming speech translation in some embodiments. Detailed Implementation

[0021] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.

[0022] This invention provides a seamless degradation method for streaming speech translation, used for disaster recovery processing in streaming speech translation, such as... Figures 2 to 6 As shown, it includes the following steps: S1. Construct an audio buffer queue on the server side that is independent of the specific speech recognition engine and translation engine, and write the audio frames sent by the terminal into the audio buffer queue. S2. Construct a speech recognition degradation link containing at least two speech recognition engines and a translation degradation link containing at least two translation engines, and the server determines the order of candidate engines based on preset priority, language support status and circuit breaker status. S3. After the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process, the next priority speech recognition engine is started. The session core metadata of the audio buffer queue is migrated to the new engine through a state transition algorithm, so that the new engine can continue recognition from the queue breakpoint. If the current translation engine fails, the original text and context information of the current translation task are retained, the next priority translation engine is started, and the original text and context information are injected into the new engine for silent retry.

[0023] Specifically, the basic processing flow of streaming speech translation involves establishing a persistent long-lived connection between the server and the terminal device based on the WebSocket protocol. PCM format audio streams captured by the terminal's microphone are continuously sent to the server through this connection at fixed frame lengths (e.g., 20ms / frame). The server first sends the audio stream to a speech recognition engine for real-time transcription, generating text, which is then sent to a translation engine for target language translation. Finally, the recognized text and translation results are returned to the terminal for real-time display through the same long-lived connection.

[0024] A generic session container (BaseASRHandler) is deployed on the server side. Within this container, an audio buffer queue is built that is completely independent of all speech recognition and translation engines, decoupling the audio reception process from the underlying speech recognition engine's processing. All audio frames received from the terminal, regardless of whether the underlying engine is currently functioning correctly, are unconditionally appended to this audio buffer queue in chronological order of reception. This audio buffer queue serves as the sole entry point for audio data, completely decoupling the "audio data reception" action from the "speech recognition processing" action both physically and logically. The audio stream sent by the terminal is no longer directly passed through to any specific ASR engine.

[0025] Two independent multi-level degradation links are pre-configured on the server side. For example... Figure 2 As shown, the speech recognition degradation link includes at least two speech recognition engine nodes from different cloud service providers, sorted by priority from high to low. The speech recognition degradation link includes primary ASR engine 1, ASR engine 2, and ASR engine 3; as shown... Figure 3 As shown, the translation degradation chain contains at least two large language translation model nodes from different service providers, also sorted by priority. The translation degradation chain includes primary LLM translation 1, LLM translation 2, LLM translation 3, and LLM translation 4. By default, the system uses the primary engine with the highest priority in each chain to process business. The preset basic priority (configured by the administrator based on factors such as service quality and price) indicates that the engine supporting the current conversation's source and target languages ​​is prioritized, and the circuit breaker status indicates that faulty engines within the circuit breaker's validity period are excluded.

[0026] After the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process, such as connection failure, handshake timeout, or authentication error in the primary speech recognition engine, the system immediately stops attempting to connect with the primary engine and starts the next-priority backup speech recognition engine in the speech recognition degradation chain. Simultaneously, a preset state transition algorithm is triggered to completely migrate all session core metadata corresponding to the audio buffer queue to the newly started backup engine. After obtaining this metadata, the backup engine directly locates the end of the data already written in the audio buffer queue (i.e., the queue breakpoint) and continues reading and processing audio frames from that position without reprocessing any data already written to the queue.

[0027] When the primary translation engine fails during connection establishment, the system does not return any error information to the terminal, nor does it interrupt the audio reception process. The system fully preserves the currently generated original text to be translated, as well as all contextual information related to the translation task. Then, it starts the next priority alternative translation engine in the translation degradation chain, injects the preserved original text and contextual information into the alternative engine, and the alternative engine completes the translation processing and returns the result.

[0028] By employing a decoupled design with independent audio buffer queues, a "caching first, processing later" architecture is achieved. Even if the underlying engine fails multiple times consecutively during the connection establishment phase, all audio data sent by the terminal will be securely stored in the queue, completely resolving the issue of audio data loss due to connection failures in traditional solutions. When the primary engine connection fails, the system automatically switches to the backup engine in the background and resumes processing from the breakpoint, all completely transparent to the end user. Users only need to initiate a connection request once, without manual retries or receiving any error messages, eliminating any sense of disjointed user experience. The state transition algorithm achieves complete inheritance of session state, allowing the new engine to seamlessly resume processing from the queue breakpoint, avoiding duplicate recognition or recognition interruptions, ensuring the continuity and integrity of recognition results. The two degradation links for speech recognition and translation are independent of each other; a connection failure in one link will not affect the normal operation of the other, significantly improving the success rate of connection establishment and the overall reliability of the system.

[0029] In some embodiments, abnormal signals are intercepted and a persistent communication connection with the terminal is maintained after the current speech recognition engine fails.

[0030] Specifically, when the currently operating speech recognition engine malfunctions, the system's built-in exception interceptor immediately captures all exception signals thrown by the engine (including connection loss exceptions, timeout exceptions, rate limiting exceptions, authentication exceptions, etc.). The exception interceptor prevents these exception signals from being passed to the upper-layer application, thus preventing the upper-layer application from executing the default logic of "disconnecting the terminal and throwing an error." Simultaneously, the system continuously maintains the established WebSocket persistent communication connection between the server and the terminal, continuing to receive audio frames sent by the terminal and write them to the audio buffer queue. Throughout this process, the terminal will not receive any connection loss notification. Through exception interception and connection maintenance, the terminal connection remains valid throughout the entire failover process. Users do not need to perform any additional operations; the conversation continues naturally, completely eliminating any sense of discontinuity in the user experience and meeting the stringent business continuity requirements of high-time-sensitivity scenarios such as simultaneous interpretation.

[0031] In some embodiments, the session core metadata includes at least one of the following: audio buffer queue object reference, queue access handle, task unique identifier, stop event, sentence break event, language parameters, audio encoding, decoding status, historical context, cumulative audio duration, billing status, performance statistics status, and timestamp status.

[0032] Specifically, a queue access handle or audio buffer queue object reference is used to point to the audio buffer queue corresponding to the current session, enabling alternative speech recognition engines to reuse the same audio buffer queue and continue consuming unprocessed audio data. By passing queue object references or access handles instead of copying the entire audio data, data copying, memory usage, and processing latency during switching are reduced. The unique task identifier ensures that in a distributed cluster environment, the new engine can accurately find the corresponding session task, avoiding data confusion between different sessions and guaranteeing task isolation in multi-concurrency scenarios. The queue access handle points to a unique identifier for the audio buffer queue in the server's memory, enabling the new engine to directly access and read audio data from the queue. Alternatively, an audio buffer queue object reference can be used as the unique identifier for the audio buffer queue. The stop event indicates whether the current session has received a stop recording command from the terminal. The sentence segmentation event records the position where the primary engine last completed sentence segmentation processing before the connection failed. Language parameters include configuration information such as source language, target language, dialect recognition switch, and accent adaptation mode. The audio encoding or decoding status records the encoding format, sampling rate, bit depth, and decoding progress of the current audio stream. The historical context records the text content that has been recognized in the current session. The cumulative audio duration records the total audio duration received from the start of the session until the connection failed. The billing status records the billing duration and cost information generated in the current session. The performance statistics status records the processing latency, throughput, and other performance indicators of the primary engine. The timestamp status records the timestamp corresponding to the last processed audio frame.

[0033] In some embodiments, the context information includes historical dialogue context and user-specific translation dictionaries.

[0034] Specifically, the historical dialogue context is the system's automatic recording of the most recent N rounds (N is a configurable parameter, such as 10 rounds) of speech recognition text and their corresponding translation results in the current conversation, forming the historical dialogue context. This context enables alternative translation engines to acquire a complete dialogue context, understand the semantics of the current sentence within the entire conversation, and avoid translation errors such as ambiguous references and semantic breaks caused by missing context, ensuring the coherence and logic of the translation results. The user-specific translation dictionary consists of custom translation rules pre-uploaded by the user or configured in the system, containing fixed translation methods for specific words such as professional terms, names of people, places, and organizations. The user-specific translation dictionary ensures that alternative translation engines follow the user's personalized translation habits, using a consistent translation method for professional terms and specific words, avoiding the confusion caused by inconsistent translation styles between different engines. This is particularly suitable for scenarios with extremely high requirements for terminology accuracy, such as professional conferences and technical exchanges.

[0035] In some embodiments, a circuit breaker operation is performed after the speech recognition engine or translation engine fails, and new service requests are scheduled to other available engines during the circuit breaker period.

[0036] Specifically, when a speech recognition or translation engine fails, the system immediately marks the engine as "unavailable" and sets a fixed circuit breaker period. During this period, the system's scheduler will reschedule all newly generated business requests to other engines of the same type that are "available." Only after the circuit breaker period ends will the system retry sending a small number of test requests to the engine to verify whether it has recovered. If it has recovered, it will be reinstated into the pool of available engines; otherwise, the circuit breaker period will be extended. The circuit breaker prevents the system from continuously sending a large number of requests to the faulty engine during a failure, thus avoiding unnecessary waste of computing power and network bandwidth. It also prevents the faulty engine from being overwhelmed by a large number of retries, thus prolonging its recovery time. In distributed systems, if a single node failure is not isolated in time, it can lead to request backlog and spread to other nodes, ultimately causing a cascading collapse of the entire system. The circuit breaker mechanism effectively prevents the spread and propagation of failures by quickly isolating faulty nodes and limiting the impact to a single node. During the isolation of the faulty engine, all new business requests can be scheduled to healthy nodes for normal processing, ensuring the overall throughput and response speed of the system and preventing the entire service from becoming unavailable due to the failure of a single node.

[0037] In some embodiments, the circuit breaker operation is performed based on a distributed caching system, setting an unavailable flag with a valid lifetime for the failed speech recognition engine or translation engine.

[0038] Specifically, the circuit breaker operation is implemented based on the Redis distributed caching system, and the process is as follows: When the speech recognition engine or translation engine fails, the system generates an "unavailable flag" in the Redis cache with a unique identifier for that engine as the key, and sets a configurable time-to-live (TTL) for this key. Within the TTL, whenever a new business request needs to be scheduled, the scheduler first checks whether the corresponding engine's "unavailable flag" exists in the Redis cache. If it exists, the engine is determined to be unavailable, and the request is not scheduled to that engine; if it does not exist, the engine is determined to be available, and it can be scheduled normally. When the TTL expires, Redis automatically deletes the "unavailable flag," and the engine returns to an available state. Leveraging the high consistency of the Redis distributed cache, the circuit breaker state is ensured to be synchronized across all nodes in the distributed cluster. All scheduling nodes will see the same engine state, avoiding inconsistencies in the judgment of the same engine state by different nodes, which could lead to some nodes still sending requests to the failed engine. The configurable TTL allows the system to flexibly adjust the circuit breaker duration based on the failure characteristics and recovery speed of different engines. For engines with fast recovery speeds, shorter circuit breaker intervals can be set; for engines with slow recovery speeds, longer circuit breaker intervals can be set, improving system adaptability. Redis's automatic key expiration deletion mechanism enables automatic recovery detection of failed engines. After the circuit breaker period ends, the system will automatically try to use the engine again without manual intervention, reducing system maintenance costs.

[0039] In some embodiments, a speech recognition circuit breaker is configured in the speech recognition degradation link, and a translation circuit breaker is configured in the translation degradation link. The speech recognition circuit breaker is used to perform a circuit breaker operation within the speech recognition engine, and the translation circuit breaker is used to perform a circuit breaker operation within the translation engine.

[0040] Specifically, the system deploys two completely independent circuit breaker components: a speech recognition circuit breaker and a translation circuit breaker. The speech recognition circuit breaker is associated only with the speech recognition degradation link, and is dedicated to monitoring the operational status of all speech recognition engines, detecting faults, and executing circuit breaker operations for the speech recognition engines. The translation circuit breaker is associated only with the translation degradation link, and is dedicated to monitoring the operational status of all translation engines, detecting faults, and executing circuit breaker operations for the translation engines. The two circuit breakers have independent state storage spaces, independent fault detection logic, and independent scheduling rules. Their operation does not interfere with each other; a state change in one circuit breaker will not affect the normal operation of the other.

[0041] The speech recognition fuse and the translation fuse operate independently, achieving complete isolation between the speech recognition and translation links. When an engine in the speech recognition link fails, only the speech recognition fuse will be triggered, without affecting the normal operation of any engine in the translation link; conversely, when an engine in the translation link fails, only the translation fuse will be triggered, without affecting the normal operation of the speech recognition link.

[0042] This invention provides a seamless degradation system for streaming speech translation, comprising: The session management module is used to build an audio buffer queue on the server side that is independent of the specific speech recognition engine and translation engine, and to write the audio frames sent by the terminal into the audio buffer queue. The link configuration module is used to build a speech recognition degradation link containing at least two speech recognition engines and a translation degradation link containing at least two translation engines. The server determines the order of candidate engines based on preset priority, language support status and circuit breaker status. The degradation processing module is used so that after the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process, the next priority speech recognition engine is started. The session core metadata of the audio buffer queue is migrated to the new engine through a state transition algorithm, so that the new engine can continue recognition from the queue breakpoint. If the current translation engine fails, the original text and context information of the current translation task are retained, the next priority translation engine is started, and the original text and context information are injected into the new engine for silent retry.

[0043] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a seamless degradation method for speech translation.

[0044] This invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The steps of implementing a seamless degradation method for streaming speech translation when a processor executes a program stored in memory.

[0045] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A seamless degradation method for streaming speech translation, characterized in that, Includes the following steps: An audio buffer queue, independent of specific speech recognition and translation engines, is built in a general session container on the server side, and audio frames sent by the terminal are written into the audio buffer queue. Construct a speech recognition degradation link that includes at least two speech recognition engines, and a translation degradation link that includes at least two translation engines. The server determines the order of candidate engines based on preset priority, language support status, and circuit breaker status. After the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process with the speech recognition engine, the next priority speech recognition engine is started. The session core metadata of the audio buffer queue is migrated to the new engine through a state transition algorithm, so that the new engine can continue to recognize from the queue breakpoint. If the current translation engine fails, the original text and context information of the current translation task are retained, the next priority translation engine is started, the original text and context information are injected into the new engine for silent retry. After the current speech recognition engine fails, the server captures the abnormal signal thrown by the engine through an exception interceptor and prevents the abnormal signal from being transmitted to the upper layer application. The server intercepts the abnormal signal and maintains a persistent communication connection with the terminal. During the engine switching process, the server continues to receive audio frames sent by the terminal and write them to the audio buffer queue. A speech recognition circuit breaker is configured in the speech recognition degradation link, and a translation circuit breaker is configured in the translation degradation link. The speech recognition circuit breaker is used to perform the circuit breaking operation in the speech recognition engine, and the translation circuit breaker is used to perform the circuit breaking operation in the translation engine.

2. The seamless degradation method for streaming speech translation according to claim 1, characterized in that, The session core metadata includes at least one of the following: audio buffer queue object reference, queue access handle, task unique identifier, stop event, sentence break event, language parameters, audio encoding, decoding status, historical context, cumulative audio duration, billing status, performance statistics status, and timestamp status.

3. The seamless degradation method for streaming speech translation according to claim 1, characterized in that, The contextual information includes historical dialogue context and user-specific translation dictionary.

4. The seamless degradation method for streaming speech translation according to claim 1, characterized in that, If the speech recognition engine or translation engine fails, a circuit breaker operation will be executed. During the circuit breaker period, new business requests will be scheduled to other available engines.

5. The seamless degradation method for streaming speech translation according to claim 4, characterized in that, The circuit breaker operation is performed based on a distributed caching system, which sets an unavailable flag with a valid lifetime for the failed speech recognition engine or translation engine.

6. A seamless degradation system for streaming speech translation, characterized in that, include: The session management module is used to build an audio buffer queue that is independent of the specific speech recognition engine and translation engine in the general session container on the server side, and write the audio frames sent by the terminal into the audio buffer queue. The link configuration module is used to build a speech recognition degradation link containing at least two speech recognition engines and a translation degradation link containing at least two translation engines. The server determines the order of candidate engines based on preset priority, language support status and circuit breaker status. The downgrade processing module is used so that after the terminal establishes communication with the server, the server immediately establishes a connection with the speech recognition engine. If a failure occurs during the connection establishment process with the speech recognition engine, the next priority speech recognition engine is started. The session core metadata of the audio buffer queue is migrated to the new engine through the state transition algorithm, so that the new engine can continue to recognize from the queue breakpoint. If the current translation engine fails, the original text and context information of the current translation task are retained, the next priority translation engine is started, the original text and context information are injected into the new engine for silent retry. After the current speech recognition engine fails, the server captures the abnormal signal thrown by the engine through an exception interceptor and prevents the abnormal signal from being transmitted to the upper layer application. The server intercepts the abnormal signal and maintains a persistent communication connection with the terminal. During the engine switching process, the server continues to receive audio frames sent by the terminal and write them to the audio buffer queue. A speech recognition circuit breaker is configured in the speech recognition degradation link, and a translation circuit breaker is configured in the translation degradation link. The speech recognition circuit breaker is used to perform the circuit breaking operation in the speech recognition engine, and the translation circuit breaker is used to perform the circuit breaking operation in the translation engine.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-5.

Images