Systems and methods for enabling low-latency, high-reliability ai voice interactions

Abstract

This invention provides a system and method for achieving low-latency, high-reliability interaction between humans and artificial intelligence (AI). As shown in some embodiments, the AI ​​client first receives audio signals from humans and performs speech detection on the audio signals to identify the speech data. Simultaneously, the encoded frame length can be dynamically adjusted according to network bandwidth. Subsequently, the audio data is encoded according to the adjusted encoded frame length and transmitted to the AI ​​server. Compared to traditional communication systems, the encoded audio data is cached in a larger buffer on the AI ​​server side. Once all encoded audio data has been received sequentially, the received encoded audio is quickly decoded. If packet loss occurs, retransmission packets are checked at preset time intervals. If a retransmission packet is received within the maximum waiting time, fast decoding is immediately initiated until all lost packets and subsequent data packets are decoded; if no packet is received after a timeout, packet loss compensation (PLC) is activated. Playback strategies can also be adjusted as needed. The decoded audio data can then be provided to a large AI model for processing.

Description

[0001] Cross-references to related applications

[0002] This invention claims priority to U.S. Patent Application No. 19 / 176,308, filed on April 11, 2025. Technical Field

[0003] This invention generally relates to the field of audio processing, and more specifically, to methods, computer programs, and systems for low-latency, high-reliability voice interaction between humans and artificial intelligence (AI) systems. Specifically, this invention relates to a system and method for adjusting transmission strategies between a human client system and an AI server system to improve the latency performance and reliability of the communication system. Background Technology

[0004] In voice interaction systems, users typically don't converse with another person, but rather with a computer system running one or more AI algorithms that parse and analyze human speech. For example, a smart speaker system allows users to issue voice commands, which are interpreted by an AI system on a remote server, enabling the computer system to perform the corresponding actions. Especially when network connectivity between the human client device and the AI ​​server is poor, latency and reliability become crucial factors in ensuring accurate and efficient communication between the AI ​​system and humans.

[0005] Existing systems typically use communication protocols similar to those used in human-to-human interactions for communication between humans and AI systems. For example, the transmission rate is kept constant, and data is transmitted in segments at fixed time intervals. While this approach is ideal for human-to-human voice communication, it may not be optimal for human-computer voice interaction.

[0006] The above examples primarily illustrate smart speaker devices (such as Amazon's Alexa devices), but the following disclosures also apply to any type of human-computer voice interaction device, such as Internet of Things (IoT) devices, video and audio conferencing programs, and other voice assistants.

[0007] Given that optimal communication between humans and AI systems is crucial for ensuring the efficiency and effectiveness of AI systems, and also for enabling AI systems to provide clear responses to humans, there is an urgent need for an improved method to achieve low-latency, high-reliability transmission between human clients and AI server systems. Summary of the Invention

[0008] This system and method relate to the field of audio processing, and more specifically, to low-latency, high-reliability methods for human-AI interaction. Such systems and methods can process received speech signals more accurately and provide clearer speech responses to human users.

[0009] As illustrated in some embodiments, this invention provides methods and systems for low-latency, high-reliability human-artificial intelligence (AI) interaction. In some cases, the AI ​​client first receives audio signals from a human user. Because the AI ​​server can handle more bursts of retransmitted data packets, the retransmission rate and retransmission window can be higher than those of traditional communication protocols. The AI ​​client can also perform speech detection on the audio signal, sending only audio data containing speech content, or setting different transmission priorities for different audio data packets. The encoded frame length can also be dynamically adjusted according to network bandwidth to balance end-to-end transmission latency with payload ratio.

[0010] As shown in some embodiments, the encoded frame length increases as bandwidth decreases. In specific cases, the encoded frame length can vary between approximately 10 milliseconds and approximately 100 milliseconds.

[0011] The AI ​​server receives encoded audio data sent by the AI ​​client. The system only begins decoding after the audio data is received completely and in sequence to ensure audio integrity. If data is not received completely, the system pauses decoding, buffers the received audio, and enters a maximum wait time (max_wait_timeout). After the wait time expires, the system buffers data packets available for decoding in the current packet cache. If data packets arrive in sequence during the wait period, the decoding process continues until all received or buffered audio has been decoded, significantly reducing end-to-end latency. The decoded audio data can then be sent to a large AI model for further processing.

[0012] After processing human audio data, the AI ​​model may generate a response. This response data is received by the AI ​​server. The AI ​​server can encode the response data in 10-millisecond frames and package them into 100-millisecond groups. The encoding frame rate can be gradually adjusted from the initial startup rate (e.g., 10 milliseconds) to a stable rate (e.g., over 60 milliseconds). The encoded response data can then be transmitted from the AI ​​server to the AI ​​client. If an interruption is caused by the AI ​​server's Voice Activity Detection (VAD), the AI ​​server will clear the uplink unencoded buffer and stop transmission.

[0013] On the AI ​​client, because the AI ​​server can generate response data faster, the response data received from the AI ​​server can be cached, thus better handling network quality fluctuations. The playback strategy has also been optimized accordingly. In traditional communication systems, the receiving end typically needs to maintain a jitter buffer of tens or even hundreds of milliseconds to offset network jitter. However, for the AI ​​client, due to the faster data reception speed, decoding and playback can be initiated directly within 10 milliseconds without waiting for the jitter buffer time, significantly reducing end-to-end latency. The cached response data will be decoded and played at a constant rhythm suitable for human hearing. When an interruption occurs, the system will generate a playback timestamp and send it back to the AI ​​server, allowing the AI ​​server to know the specific time of the interruption and maintain accurate dialogue context.

[0014] It should be noted that the functions described above in this invention can be implemented individually or in combination. These and other functions of this invention will be described in more detail below with reference to the accompanying drawings. Attached Figure Description

[0015] To more clearly illustrate the present invention, some embodiments of the present invention will be described below with reference to the accompanying drawings:

[0016] Figure 1 is a schematic diagram of a system for realizing human-to-large AI model interaction via network according to an embodiment of the present invention;

[0017] Figure 2 This is an example block diagram of a client device drawn according to an embodiment of the present invention;

[0018] Figure 3 This is an example block diagram of an AI server device drawn according to an embodiment of the present invention;

[0019] Figure 4A and 5B This is an example process flowchart of human client device operation drawn according to an embodiment of the present invention;

[0020] Figure 5A and Figure 4B This is an example process flowchart of the operation of an AI server device drawn according to an embodiment of the present invention;

[0021] Figures 6A and 6B are schematic diagrams of a computer system capable of distinguishing different speakers according to an embodiment of the present invention. Detailed Implementation

[0022] This invention will be described in detail with reference to several embodiments shown in the accompanying drawings. In the following description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that some or all of these specific details may be omitted when implementing these embodiments. To avoid unnecessarily obscuring the invention, well-known technical steps and / or structures are not described in detail. The functions and advantages of the various embodiments can be more clearly understood through the following discussion in conjunction with the accompanying drawings.

[0023] The accompanying drawings will help to understand the various functions and advantages of the exemplary embodiments of the present invention. Those skilled in the art should understand that the embodiments of the present invention provided herein are merely illustrative and not restrictive. Unless otherwise expressly stated, all functions disclosed in this specification can be replaced by other functions that achieve the same or similar purpose. Therefore, other embodiments not exhaustively listed, or modifications thereof, should be considered included within the scope of the present invention as defined herein and its equivalents. It should be particularly noted that the imperative and / or sequential terms used herein (such as "will," "will not," "should," "should not," "must," "must not," "firstly," "initially," "then," "subsequently," "before," "after," "finally," and "ultimately," etc.) are not intended to limit the scope of the present invention, as the embodiments listed herein are merely exemplary.

[0024] This invention relates to a system and method for low-latency, high-reliability human-computer interaction. These systems are particularly useful when users interact with AI client devices connected to an AI server via the internet. In traditional real-time voice communication, audio acquisition, transmission, decoding, and playback are typically controlled by a fixed clock. This fixed clock usually has a period of 10 milliseconds, meaning 10 milliseconds of audio data are transmitted every 10 milliseconds. To achieve low latency, almost no buffering is used in the entire transmission link. At the receiving end, only a small jitter buffer is configured to smooth latency fluctuations caused during network transmission. The playback link is also driven by a hardware clock to ensure that audio data is played at a constant rate. These traditional systems typically employ short audio acquisition intervals, short encoded frame lengths, and small send and receive buffers to ensure low-latency transmission. They also employ constant-rate decoding and playback mechanisms to ensure the audio is perceived as continuous and smooth by the human ear. While these traditional techniques are well-suited for human-to-human voice communication, they are not optimized for real-time human-computer voice interaction. The system and method described in this invention can maintain strong anti-interference capabilities under poor network conditions, and achieve high-performance real-time human-computer voice interaction while ensuring high-quality audio.

[0025] Furthermore, it should be noted that terms such as “approximately,” “roughly,” or “about” are used in this disclosure and the appended claims. These terms specifically indicate a range of plus or minus 20% of the listed values. For example, a timer with a length of approximately 10 milliseconds may actually vary between 8 and 12 milliseconds.

[0026] As shown in the figure Figure 1 An example system is shown, comprising a human user 110 generating a voice audio signal 115, which is received by an AI client device 120. The AI ​​client device 120 can be a smartphone, smart speaker, computer, or Internet of Things (IoT) device (e.g., a smart doorbell). The AI ​​client establishes a bidirectional communication connection via network 130 with an AI server 140 located at an edge node or network operations center. The AI ​​server collaborates with a large AI model 150 to process the human-generated audio stream and generate a response. This response can be performing an action (e.g., turning on a light or adjusting the temperature of a smart thermostat), generating a voice reply, or a combination of both.

[0027] In some embodiments, the AI ​​client 120 has a microphone or microphone array for collecting audio 115 from humans and converting it into a digital format for further processing and transmission. The AI ​​client also has local data storage capabilities for data caching and other operations. Simultaneously, the AI ​​client 120 is equipped with a certain level of computing resources for preprocessing and encoding the digitized audio signal. The microphone collects speech signals and converts them into electronic audio signals, which the AI ​​client can further process. These audio processing functions include at least Acoustic Echo Cancellation (AEC), Adaptive Noise Suppression (ANS), and Automatic Gain Control (AGC). The audio collected at the front end can be provided to the AEC module for echo cancellation processing along with a far-end reference signal (if any), where the far-end reference signal is the signal previously played through the device's speakers. The AEC module subtracts the delayed far-end reference signal from the near-end audio signal to eliminate the echo. The adjusted signal is then provided to the ANS module for analyzing ambient noise. The ANS module further adjusts the signal to remove ambient noise. The further adjusted signal is then provided to AGC, a closed-loop feedback regulation circuit used to adjust the relative amplification of the signal to ensure consistent volume. This produces a clear and stable audio signal, ideal for subsequent analysis by AI servers, such as speech recognition systems and command word recognition algorithms, or for encoding and transmission to other systems.

[0028] In many embodiments, network 130 may consist of the Internet. In other embodiments, network 130 may also consist of a wide area network (WAN), a local area network (LAN), a cellular network, or any combination thereof.

[0029] AI server 140 may include a backend system consisting of one or more computing devices located in a data center or other network hub. AI server 140 includes a processing platform that communicates with a large AI model 150. Large AI model 150 can be any AI model suitable for receiving human speech and parsing instructions from speech signals. In some cases, AI model 150 may include a large language model (LLM). Unlike human speakers, AI model 150 can understand and process speech signals played at a different speed than human speech. Furthermore, unlike human listeners, AI model 150 has a lower fault tolerance and requires high-quality speech input to achieve optimal performance. Therefore, interruptions or poor network quality in network 130 can significantly affect the performance of AI model 150 in human-computer interaction. This system addresses these issues by leveraging the fact that AI understands speech much faster than humans, thereby eliminating the limitation that speech transmission must be at a constant rate. Furthermore, speech data can be transmitted in complete segments, allowing for larger retransmission windows and larger buffers, thus minimizing the impact of network packet loss and jitter.

[0030] Referring to Figure 2, a logical block diagram of the AI ​​client 120 is shown. This diagram illustrates two logical paths for the AI ​​client 120: one from human user 110 to AI server 140, and a return path from AI server 140 to human user 110. In the human-to-AI transmission direction, the system first receives input signal 210. This signal is converted from an acoustic signal to a digital signal and undergoes front-end preprocessing, including the aforementioned acoustic echo cancellation (AEC), adaptive noise suppression (ANS), and automatic gain control (AGC) (not shown in the figure). The resulting clean and gain-controlled audio signal is then buffered in extended input buffer 220. The buffered audio is then analyzed by speech activity detection (VAD) system 240 to identify signal segments containing speech activity and segments without speech activity. In human-to-human voice interaction, all audio information is meaningful, and pauses between speech can also convey information as contextual cues; therefore, the entire audio needs to be transmitted. However, in a human-to-AI voice interaction system, it is not necessary to transmit non-speech segments. By utilizing VAD information, audio segments containing speech can be separated, and only these speech segments can be transmitted, thereby effectively reducing the overall transmission bandwidth.

[0031] The cached audio data and voice activity information are transmitted to the transmission manager 230, which controls the transmission of data to the AI ​​server. Because the AI ​​server possesses powerful computing capabilities, it can quickly process more "historical" audio data upon reception. The transmission manager 230 can increase the number of retransmissions compared to human-to-human communication, thereby enhancing the system's resilience against weak networks. As mentioned above, the retransmission buffer increases proportionally to store earlier audio data. The retransmission window is extended to improve transmission reliability. Simultaneously, by utilizing voice activity information, only audio segments containing speech are transmitted, reducing bandwidth consumption. "Silence" packets are not sent to reduce bandwidth requirements and lower the computational load on the AI ​​server. Finally, the dynamic encoded frame length adjuster 255 can dynamically adjust the audio encoded frame length (ptime) as needed. In bandwidth-constrained situations, ptime can be appropriately increased to reduce packet header load.

[0032] Encoder 250 receives non-silent data packets from transmission manager 230 and ptime information from dynamic encoding frame length adjuster 255, encodes the data packets, and then generates encoded output 215. Furthermore, the AI ​​client can retransmit necessary signal segments multiple times as needed or according to network quality.

[0033] When the signal is sent to AI server 140, human speech can be processed and provided to a large AI model, as detailed below. At this point, the AI ​​server generates encoded input 225 and returns it to AI client 120. Encoded input 225 is then decoded by decoder 260. Because the AI ​​model can generate large amounts of content in milliseconds, encoded input 225 may contain audio signals longer than normal human speech. Since humans need to receive content gradually, this information cannot be "fast-forwarded," but it can be stored in a larger-capacity receive buffer 270 to prevent network packet loss. The decoded audio can dynamically begin playback after accumulating a certain amount of buffer space. This buffer size can be adjusted based on information loss or reception rate: a smaller buffer can be set to ensure timely playback when network conditions are poor and packet loss is high; a larger buffer can be used to ensure smooth playback when the network is stable and packet loss is low. This dynamic playback initiation based on variable buffer length can be controlled by playback regulator 280, and playback 245 is performed through one or more speakers of AI client 120. The timestamp feedback module 290 can generate feedback information 235 and send the current playback timestamp back to the AI ​​server 140, so that the server can know the playback progress of the AI ​​client, thus making it easier to maintain the consistency of the dialogue context when an interruption occurs.

[0034] See Figure 3. Figure 3 The AI ​​server 140 is shown in more detail. The AI ​​server first receives encoded output 215 from the AI ​​client 120, including retransmission data as needed based on network conditions. Encoded output 215 is stored in buffer 310. Since the AI ​​client 120 may retransmit more "historical" data, the AI ​​server 140 needs a larger receive buffer to accommodate this data. The intelligent decoder 320 adjusts according to a decoding strategy to ensure decoding only occurs when complete and continuous audio data is detected, thus guaranteeing audio integrity. In a traditional system, the decoder would play the audio immediately after decoding in the received sequence to maintain a constant timescale, but this would result in audio defects if data is missing. For human listeners, even with minor audio defects, constant playback is superior to audio with unstable or excessively fast / slow playback speeds because humans are tolerant of minor audio defects, which have little impact on the listening experience. In contrast, AI models can process audio very quickly (far faster than humans), so complete and high-quality audio segments are more suitable for AI model processing, while audio with a high error rate is detrimental to model performance. In some embodiments, the smart decoder 320 sets a waiting time to allow for the reception of complete, continuous data. If all data has not been received by the time the waiting time reaches its maximum value, the system can force decoding of existing data to ensure that the AI ​​model's response time remains timely. During the waiting period, if consecutive data packets arrive, they can be inserted into the packet set to continue decoding.

[0035] The decoded data is passed to the adjusted playback manager 330 to generate AI model input 305. Adjusting the playback manager 330 supports fast playback. In the initial stage, or before the audio arrives, the manager can maintain a fixed timer (approximately 10 milliseconds by default) to attempt decoding. Once decodeable audio is detected, playback enters continuous decoding mode, thus quickly completing the audio burst decoding. The decoded Pulse Code Modulation (PCM) data is the AI ​​model input 305, provided to the AI ​​model for further processing.

[0036] After processing by the AI ​​model, model output 315 is generated, containing complete audio content. AI model output 315 can be passed to the fast encoder 360 and the interrupt manager 340. Due to the large amount of data generated by the large AI model, the AI ​​server needs to send the generated data to the AI ​​client 120 at a faster rate, for example, encoding and sending approximately 100 milliseconds of data every 10 milliseconds. The encoding frame length adjuster 350 can dynamically increase ptime to reduce the overhead of data packet headers, thereby improving transmission efficiency. In dynamic ptime encoding, approximately 10 milliseconds of ptime can be used during the startup phase, while increasing ptime to more than approximately 60 milliseconds during the stable phase to reduce overall overhead while maintaining transmission quality. The interrupt manager 340 receives timestamp feedback from the AI ​​client 120 to indicate interruption events on the AI ​​server 140. If an interruption occurs during the conversation, the interrupt manager 340 will clear the AI ​​server's uplink unencoded buffer or encoding task queue to ensure system state consistency.

[0037] Referring to Figure 4A, the communication process between the human / machine interface and the AI ​​model is presented from the user / human's perspective: the user / human generates an audio signal by speaking, the AI ​​client device receives the audio signal (at step 410) and converts the acoustic audio into a PCM data signal. Compared to traditional communication systems, this data signal is buffered (at step 420) in a larger cache memory. The system can perform a series of audio preprocessing techniques to ensure the audio signal is clean; these processes include noise and echo cancellation, as well as gain control (not shown in the figure).

[0038] The cleaned audio signal undergoes speech detection on the AI ​​client device (step 430). Audio segments containing only speech content are transmitted from the AI ​​client to the AI ​​server to reduce overall bandwidth requirements. Compared to traditional human voice interaction, the retransmission window is larger (step 440) to enhance data reliability in unstable network environments. Furthermore, the encoded frame length is dynamically adjusted based on bandwidth conditions (step 450). When bandwidth is limited, ptime can be appropriately increased to a longer time period (e.g., approximately 60 milliseconds or longer) to reduce packet header overhead.

[0039] The audio signal is encoded for transmission (step 460) and sent to the AI ​​server (step 470). Because the AI ​​system can process the received data quickly, it can retransmit the data multiple times within a longer retransmission window. See the flowchart below. Figure 4B The AI ​​server receives the encoded signal (step 415). The AI ​​server is equipped with a larger capacity receive buffer (step 425) to fully cache the interactive content for the AI ​​model to process. The decoding strategy can also be adjusted (step 435) to ensure that decoding can wait for a preset maximum waiting time before the arrival of complete, continuous audio data. Decoding only begins when complete audio data has been received or after the maximum waiting time has been reached. If complete audio data is not received within the maximum waiting time, decoding will still start, enabling packet loss compensation (PLC) to compensate for lost packets and insert them sequentially into the decoded data stream. The signal is thus decoded (step 445), and the playback strategy of the AI ​​model is adjusted (step 455). The adjusted playback strategy supports fast playback, at speeds far exceeding human comprehension, but easily handled by the AI ​​model. Playback maintains a fixed timer until the audio arrives. In some embodiments, the timer is approximately 10 milliseconds. Once decodeable data is available, the system enters continuous decoding mode, achieving fast and complete burst audio decoding. The decoded data is then provided to a large AI model for processing (step 465).

[0040] AI models process human voice interactions, determine the appropriate action, and output a return audio signal for human reception. Figure 5AThe process begins with the AI ​​server receiving large AI model inputs (step 510). Since the AI ​​model's processing speed is much faster than human speech, this data is typically provided as a single, all-time stream, or within a very short period. The audio encoded frame length can be dynamically increased during stable transmission phases (step 520) to reduce packet header overhead and thus improve transmission efficiency. In some cases, bandwidth conditions affect the encoded frame length; lower bandwidth allows for a longer frame length. During encoding, the system may receive an interruption request from the AI ​​client (step 530). If an interruption occurs, the task queue or unencoded buffer is cleared (step 540), terminating the processing flow. If no interruption is received, rapid encoding and transmission are performed (step 550). In some embodiments, the system encodes and transmits approximately 100 milliseconds of audio playback data every 10 milliseconds.

[0041] See the process for further details. Figure 5B The AI ​​client receives the sent audio data (step 515). Since the AI ​​server transmits a large audio segment to the client, a larger buffer is needed (step 525) to store all relevant data. The received AI audio data is decoded (step 535), and the playback strategy is adjusted (step 545). In some embodiments, buffer accumulation may depend on the packet loss rate. In some embodiments, the initiation of decoding may depend on the initial received bitrate. Under normal circumstances, the receiving rate is much higher than the playback rate, which reduces the waiting time for decoding and playback. If the receiving rate is lower than expected, the playback waiting time will increase accordingly to ensure timely playback under normal network conditions and to prevent buffering when network conditions are poor. Subsequently, the audio is played to the user (step 555), and a feedback timestamp is generated and sent back to the AI ​​server (step 565) to ensure that the server knows the playback progress when an interruption occurs. Understanding the specific time of the interruption during playback helps generate contextual information related to the interruption. The process ends after playback is complete.

[0042] The aforementioned methods and systems for low-latency, high-reliability human-machine-to-AI interaction maintain audio integrity during audio transmission and reception. Audio sent to humans also exhibits lower latency and better sound quality. Compared to traditional fixed-time communication systems, this system performs better under network jitter, packet loss, and even network disconnection conditions. Interrupt handling capabilities are also improved, and response latency is significantly reduced. Response latency, interruption latency, and response quality are important indicators for measuring the quality of human-machine-to-AI system dialogue. Therefore, by implementing the systems and methods disclosed herein, the user's dialogue experience can be significantly improved.

[0043] The above describes the systems and methods for improving human-machine communication. The following section will focus on discussing devices that can perform the above functions in real time. Figure 6A and6B All depict a computer system, identified as computer system 600, which is suitable for implementing various embodiments of the present invention, wherein... Figure 6A This presents one physical form of the system. It should be noted that the physical form of the computer system 600 is highly scalable; it can be a micro-device such as a printed circuit board or integrated circuit, or it can be expanded into a large system such as a supercomputer. A typical configuration of this system includes: a monitor 602, a display 604, a chassis 606, a blade server containing multiple storage drives 608, a keyboard 610, and a mouse 612. Medium 614 is a computer-readable medium used for data exchange with the computer system 600. Figure 6B The modular architecture of computer system 600 is illustrated. The main subsystems connected via system bus 620 include: processor 622 (also known as Central Processing Unit / CPU), adapted to storage devices (including memory 624). Memory 624 consists of Random Access Memory (RAM) and Read-Only Memory (ROM), where ROM is responsible for unidirectional data and instruction transfer to the CPU, while RAM supports bidirectional data transfer. Both types of memory can take the form of various computer-readable media described below. Fixed storage medium 626 works in conjunction with processor 622 via bidirectional connection, not only expanding data storage capacity but also taking the form of various computer-readable media. This medium is typically used as secondary storage (such as a hard disk) with a speed lower than main storage for storing programs and data. It should be noted that the information stored in fixed storage medium 626 can be merged into virtual memory and incorporated into memory 624 as needed using a standard conversion method. Removable medium 614 can take any of the computer-readable media forms described below.

[0044] Processor 622 connects to various input / output devices, including display 604, keyboard 610, mouse 612, and speaker 630. Broadly speaking, input / output devices can encompass any of the following types: video display devices, trackballs, mice, keyboards, microphones, touch-sensitive displays, sensor card readers, magnetic stripe / paper tape readers, tablet computers, styluses, voice / handwriting recognition devices, biometric readers, motion sensors, EEG readers, or other computer devices. Processor 622 can also connect to other computers or telecommunications networks via network interface 640. Through this network interface, processor 622 can both receive and output information to the network when executing the aforementioned audio anomaly detection method. Furthermore, embodiments of the present invention can run independently on processor 622 or collaborate with a remote CPU via a network such as the Internet to achieve distributed execution of computational tasks.

[0045] Software is typically stored in non-volatile memory and / or drive units. For large programs, their complete code may not all be loaded into memory for execution. However, it should be clarified that during software execution, the system will move it to a suitable computer-readable location for processing if necessary; for ease of explanation, this document will consistently refer to this location as "memory." Even if the software is loaded into memory, the processor will still use hardware registers to store relevant variable values ​​and improve execution efficiency through local caching (ideally). In this technical solution, when describing a software program as "implemented on a computer-readable medium," it should be understood that the program can be stored in any known or applicable location (non-volatile storage devices or hardware registers, etc.). When at least one parameter value related to the program is stored in a processor-readable register, the processor is considered "configured to execute the program."

[0046] During operation, computer system 600 is controlled by operating system software, such as a disk operating system, that includes a file management system. An example of operating system software with an associated file management system is the Windows® family of operating systems and its associated file management system from Microsoft Corporation in Redmond, Washington. Another example is the Linux operating system and its associated file management system. The file management system is typically stored in non-volatile memory and / or drive units, allowing the processor to perform various operations required by the operating system, including inputting and outputting data, storing data in memory, and storing files on non-volatile memory and / or drive units, etc.

[0047] Certain portions of this specification may be presented in the form of algorithms and symbolic representations of data bit manipulations in computer memory. These algorithmic descriptions and representations are the most effective way for those skilled in the art of data processing to communicate the essence of their work to others skilled in the art. As defined herein with common sense, algorithms are generally considered to be a self-consistent sequence of operations that leads to a desired result. These operations require physical manipulation of physical quantities. These physical quantities typically exist in the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and subjected to other operations. Sometimes, primarily for general purposes, these signals are referred to as bits, values, elements, symbols, characters, items, numbers, etc.

[0048] The algorithms and representations described herein are essentially independent of any particular computer or device. The program methods described herein can be implemented using various general-purpose systems, or more convenient dedicated devices can be designed to run specific embodiments. The required architectures for various systems will be illustrated in the following description. Furthermore, this technical solution does not limit the specific programming language; various embodiments can be implemented using multiple programming languages.

[0049] In alternative embodiments, the machine can operate as a standalone device or connect to other machines (via a network, etc.). In network deployments, the machine can act as a server or client device in a client-server network, or operate as a node device in a peer-to-peer (or distributed) network environment.

[0050] The aforementioned machines can be server computers, client computers, personal computers (PCs), tablets, laptops, set-top boxes (STBs), personal digital assistants (PDAs), cellular phones, iPhones, Blackberry devices, smart glasses, smart headphones, virtual reality devices, standalone processors, collaborative distributed processor systems, telephones, network devices (routers / switches / bridges), and any machine capable of executing a set of instructions (sequential or non-sequential) to perform a specific operation.

[0051] While machine-readable media or machine-readable storage media are shown as a single medium in exemplary embodiments, "machine-readable media" and "machine-readable storage media" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions. The term should also include any medium capable of storing, encoding, or carrying a set of instructions that can be executed by a machine and cause the machine to perform one or more methods of the currently disclosed technologies and inventions.

[0052] Generally, routines implementing embodiments of this disclosure may be implemented as part of an operating system or as a specific application, component, program, object, module, or sequence of instructions (referred to as a "computer program"). These computer programs typically contain one or more sets of instructions time-sharing across different storage devices of a computer (or distributed across computers), which, when read and executed by one or more processing units in the computer (or across computers), cause the computer to perform operations to implement the functions disclosed in this invention.

[0053] Furthermore, although the embodiments are described in the context of fully functional computers and systems, those skilled in the art will understand that the embodiments can be distributed as program products in various forms, and the disclosure of this invention applies equally to any specific machine or type of computer-readable medium used for actual implementation and distribution.

[0054] Although the present invention has been described through several embodiments, various changes, modifications, permutations, combinations, and equivalent alternatives still exist, all of which fall within the scope of the present invention. While subheadings have been added for ease of explanation, these headings are illustrative only and are not intended to limit the scope of the invention. Furthermore, it should be noted that there are many different implementations of the methods and apparatus of the present invention; therefore, the appended claims should be interpreted as including all changes, modifications, permutations, combinations, and equivalent alternatives that conform to the spirit and scope of the present invention.

Claims

1. A computerized method for low-latency, high-reliability human-AI interaction, characterized in that, The method includes: Receive audio signals from human users; The audio signal is subjected to speech detection to identify the speech audio signal; Increase the retransmission buffer to have enough storage space to retain at least 2 seconds of audio recording; The length of the encoded frame is dynamically adjusted based on network bandwidth. The audio signal is encoded according to the encoded frame length; and The encoded audio signal is transmitted repeatedly, at least 10 times.

2. The method according to claim 1, wherein the encoded frame length increases when the network bandwidth decreases.

3. The method of claim 2, wherein the encoded frame length varies between 10 milliseconds and 100 milliseconds.

4. The method according to claim 1, wherein the transmission is transmitted to the AI ​​server via a network.

5. The method of claim 4 further includes receiving at least 100 milliseconds of audio data from the AI ​​server every 10 milliseconds.

6. The method according to claim 5, further comprising: The audio data received from the AI ​​server is dynamically buffered based on network conditions, so that the buffer length is inversely proportional to the network quality. The audio data in the decoding buffer; Play the decoded buffered audio data at constant time intervals suitable for human hearing; Generate playback timestamp; as well as The timestamp is sent as feedback to the AI ​​server.

7. A computerized method for low-latency, high-reliability human-AI interaction, characterized in that, The method includes: Receive encoded audio data from AI clients at the AI ​​server; Decoding begins only after the encoded audio data has been received completely and in order, to ensure audio integrity. During the decoding process, the AI ​​server pauses decoding, caches the received audio, and enters the maximum waiting time max_wait_timeout; Once the maximum waiting time is reached, the AI ​​server searches the current packet buffer for the next packet to be decoded. When data packets arrive in sequence, the decoding process continues until all received or buffered audio data has been decoded. The decoded audio data is then fed into a large AI model for processing.

8. The method of claim 7, wherein the maximum waiting time is set to 2000 milliseconds.

9. The method according to claim 5, further comprising: Receive response data from large AI models; Every X milliseconds, response data for Y milliseconds is encoded and sent to the AI ​​client, where X is less than Y; Increase the encoding frame rate from the starting rate to the stable rate, where the starting rate is lower than the stable rate.

10. The method of claim 9, wherein X has a maximum value of 10 milliseconds and Y has a maximum value of 100 milliseconds.

11. The method according to claim 9, wherein the startup rate is 10 milliseconds and the stable rate is greater than 60 milliseconds.

12. The method according to claim 9, further comprising: Receive an interrupt signal from the AI ​​client and clear the uplink uncoded buffer or clear the coding task queue of the AI ​​server.

13. A computerized method for low-latency, high-reliability human-AI interaction, characterized in that, The method includes: Receive encoded audio data from AI clients at the AI ​​server; Buffer the received encoded audio data; Once all encoded audio data has been received in sequence, or the maximum waiting time has been reached, the system immediately performs rapid decoding on the received audio data. In the event of packet loss due to a weak network, the AI ​​server will wait for retransmission of the packet and check every X milliseconds to see if the lost packet has arrived. If a retransmission packet is received before the maximum waiting time is reached, fast decoding is started immediately until all lost packets and their subsequent data packets are decoded, and then the system waits for the next packet to arrive. If the retransmission packet is not received after the timeout, the packet loss compensation PLC will be activated for compensation processing. Adjust the playback strategy: before audio data is detected, maintain a fixed timer to attempt to decode the received encoded audio data; once audio data is detected, enter continuous decoding state until all received data is decoded. The decoded audio data is then fed into a large AI model for processing. Receive response data from large AI models; Every X milliseconds, response data for Y milliseconds is encoded and sent to the AI ​​client, where X is less than Y; Increase the encoded frame rate from the starting rate to the stable rate, where the starting rate is lower than the stable rate; and The encoded response data is transmitted to the AI ​​client.

14. The method of claim 13, wherein the fixed timer is set to 10 milliseconds.

15. The method of claim 13, further comprising: The encoded response data received from the AI ​​server is dynamically buffered based on network conditions, so that the buffer length is inversely proportional to the network quality. Decode the buffered response data; Play the decoded buffered response data at constant time intervals suitable for human hearing; Generate playback timestamp; as well as The timestamp is sent as feedback to the AI ​​server.

16. The method of claim 13, further comprising: Receives audio signals from humans; Buffer audio signals; Perform speech detection on audio signals and identify audio data; The length of the encoded frame is dynamically adjusted based on network bandwidth. The audio data is encoded according to the length of the encoded frame; as well as Transmit encoded audio data.

Images