An edge computing-based camera router gateway low-delay voice interaction method

Abstract

The application provides a kind of low-delay voice interaction method based on edge computing camera router gateway in the field of smart home and Internet of Things communication technology, comprising the following steps: continuously collecting audio by camera and detecting wake-up word locally, triggering the router to establish a priority transmission channel, and forwarding the message to the gateway for reception. Subsequently, the camera cuts the voice command into fragmented data packets and sends them to the gateway via the priority channel; the gateway receives and recognizes the streaming audio in parallel, and starts semantic analysis when the recognition result meets the confidence condition, achieving parallel processing. Finally, the control command is generated and the task is executed, and the reply audio is pushed to the designated device via the priority channel for playback. In the entire process, voice collection, transmission, recognition, analysis and broadcast are completed in a closed loop within the local area network, independent of the public network, ensuring low latency and data security. The application has the advantages of greatly improving the real-time performance, reliability and user experience of smart home voice interaction.

Description

Technical Field

[0001] This invention relates to the field of smart home and Internet of Things communication technology, and in particular to a low-latency voice interaction method based on edge computing camera router gateway. Background Technology

[0002] With the deep integration of the Internet of Things (IoT) and artificial intelligence (AI), smart home systems are evolving from single-product intelligence to whole-house intelligence. Voice interaction, due to its naturalness and convenience, has become the core human-computer interaction method. A typical smart home local area network (LAN) deploys various functional devices such as cameras, routers, and gateways, each undertaking functions such as security monitoring, network connectivity and data forwarding, protocol conversion, and device management. Although these devices physically coexist in the same subnet, their hardware computing power exhibits significant hierarchical differences: cameras often use embedded low-power SoCs with limited computing power; routers employ medium-power network processors; while gateways or home hubs may be equipped with AI acceleration chips, possessing relatively strong computing capabilities. While this heterogeneous computing architecture has the potential for resource complementarity, it also presents real technical obstacles to collaborative computing.

[0003] Currently, the mainstream implementation solutions for voice interaction in smart homes can be categorized into two types.

[0004] The first category is the single-device independent interaction solution. A single device independently completes the entire processing chain from voice acquisition, wake-up, recognition to semantic understanding. Some high-end products integrate an NPU through a SoC to run lightweight models. However, limited by terminal power consumption and size, local computing power is far inferior to that of the cloud, making it difficult to support large-scale deep learning models. This results in significant performance deficiencies in scenarios such as complex semantic understanding, multi-turn dialogue, and far-field noise reduction. Furthermore, the lack of information sharing and collaborative decision-making between devices can easily lead to abnormal responses such as "one call, a hundred responses" and mutual interference when multiple devices coexist in the same area, severely impacting the user experience.

[0005] The second type is the pure cloud-based interaction solution. The terminal is only responsible for audio acquisition, encoding, and uploading. The cloud server performs computationally intensive tasks such as Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), and Text-to-Speech (TTS), and returns the results to the terminal for execution. This solution leverages the elastic computing power of the cloud to achieve high recognition accuracy, but its fundamental flaw lies in its strong dependence on public network transmission: voice data must undergo multi-hop routing, protocol encapsulation, and cloud queuing, resulting in end-to-end latency generally exceeding 500ms, and potentially reaching several seconds in network congestion or cross-regional situations. This far exceeds the threshold for perceived smoothness in human-computer interaction (research indicates that within 200ms is considered smooth, and above 300ms indicates significant lag), leading to a poor interactive experience. Furthermore, continuous uplink transmission of audio streams over the public network also poses significant privacy and security risks.

[0006] Based on the above situation, the existing solutions have at least the following technical defects: (i) Excessive interaction latency. Cloud-based solutions are affected by public network round-trip latency, cloud queuing, and encoding / decoding time, making it difficult to compress end-to-end latency to an acceptable range. Even with optimization methods such as WebSocket long connections, the bottleneck of public network physical latency cannot be eliminated. On the other hand, single-device solutions have limited computing power and slow local inference speed, making it difficult to meet low latency requirements.

[0007] (ii) Lack of priority guarantee and adaptive adjustment mechanism for voice data transmission. Voice data competes for resources with other service data on the same Wi-Fi link. In high-traffic scenarios, voice packets are prone to queuing and packet loss retransmission due to the lack of priority guarantee, resulting in drastic latency fluctuations. Existing Quality of Service (QoS) mechanisms are mostly statically prioritized, unable to dynamically adjust transmission strategies based on real-time confidence of voice recognition or network transmission status, nor can they proactively adjust the audio encoding bitrate to adapt to current network conditions when recognition quality deteriorates, thus affecting the stability and recognition accuracy of end-to-end interaction.

[0008] (iii) Lack of multi-device collaboration mechanism. In a home environment, there are often multiple devices with voice acquisition capabilities, but the existing solutions lack a unified arbitration and collaboration protocol. They cannot dynamically select the optimal response node based on information such as device location, audio signal-to-noise ratio, and current load, resulting in frequent occurrences of multiple devices waking up simultaneously, repeated responses, or false wake-ups.

[0009] Therefore, how to provide a low-latency voice interaction method based on edge computing camera router gateway to improve the real-time performance, reliability, and user experience of smart home voice interaction has become an urgent technical problem to be solved. Summary of the Invention

[0010] The technical problem to be solved by this invention is to provide a low-latency voice interaction method based on edge computing camera router gateway, so as to improve the real-time performance, reliability and user experience of voice interaction in smart homes.

[0011] This invention provides a low-latency voice interaction method based on edge computing camera router gateway, comprising the following steps: Step S1: The camera continuously collects ambient audio and the local wake-up engine detects a preset wake-up word. When the wake-up word is detected, the camera generates a wake-up trigger message and sends it to the router. Step S2: After receiving the wake-up trigger message, the router identifies the service type of the voice interaction service, establishes a priority transmission channel for the voice interaction service, and forwards the wake-up trigger message to the gateway, triggering the gateway to enter the voice reception preparation state. Step S3: After the wake-up word, the camera cuts the received voice command audio data into continuous segmented data packets according to a preset segmentation rule. Each segmented data packet contains an audio content segment of the voice command audio data and is sent to the gateway via the router's priority transmission channel. Step S4: During the process of receiving the fragmented data packets, the gateway performs real-time speech recognition processing on the streaming audio data and performs semantic parsing based on the recognition results to generate control commands; wherein, the gateway starts the semantic parsing in advance when the recognition results meet the preset confidence conditions, so as to realize streaming parallel processing of receiving, recognizing and parsing at the same time. Step S5: The gateway executes the task operation according to the control command, and converts the execution result or the reply content obtained by the query into a reply audio stream, which is then pushed to the designated playback device for playback via the priority transmission channel of the router; In this method, voice acquisition, transmission, recognition, semantic parsing, and response playback are all completed in a closed loop within a local area network, without relying on public network transmission.

[0012] Furthermore, in step S2, establishing a priority transmission channel for the voice interaction service specifically includes: the router parsing the service type identifier in the received data packet, the received data packet including at least the wake-up trigger message, allocating a higher transmission priority than regular service data to the data stream belonging to the voice interaction service, and enabling the wireless access point scheduling mechanism to allocate dedicated transmission resources to the camera; The step of triggering the gateway to enter the voice reception preparation state includes: the gateway loading the voice recognition model and semantic parsing model into the cache, determining the fragmentation transmission parameters according to its current computing load, feeding back the fragmentation transmission parameters to the camera via the router, and the camera performing the operation of cutting the voice command audio data into continuous fragmented data packets in step S3 according to the fragmentation transmission parameters.

[0013] Furthermore, in step S3, the fragmented data packets carry timing identification information. The gateway performs timing alignment and out-of-order rearrangement of each fragmented data packet according to the timing identification information, and detects packet loss according to the timing identification information, triggering a retransmission mechanism for the detected packet loss. During the transmission of the segmented data packets, the camera performs endpoint detection on the audio data of the voice command. When it detects that the user's voice pause exceeds a preset duration, it automatically determines that the voice command has ended and sends a transmission end identifier to the gateway.

[0014] Furthermore, in step S4, the preset confidence conditions include a first confidence threshold and a second confidence threshold, wherein the second confidence threshold is higher than the first confidence threshold; when the confidence of the recognition result reaches the first confidence threshold, the gateway initiates semantic parsing of the currently recognized text and generates a set of candidate control instructions; when the confidence of a candidate instruction in the set of candidate control instructions reaches the second confidence threshold, the gateway determines the candidate instruction as the final control instruction.

[0015] Furthermore, during the period when the confidence level of the identification result reaches the first confidence level threshold but has not yet reached the second confidence level threshold, the gateway performs parallel pre-execution operations according to each candidate instruction in the candidate control instruction set, generates corresponding pre-execution results and caches them; when the final control instruction is determined, if the final control instruction matches a certain pre-execution result, the pre-execution result is directly called to execute the task operation in step S5.

[0016] Furthermore, between step S4 and step S5, the method further includes a transmission feedback adjustment step: The gateway generates a feedback adjustment signal based on the current confidence level of speech recognition and network transmission status and sends it to the router; the router dynamically adjusts the transmission priority level of the priority transmission channel and sends a bitrate adjustment instruction to the camera based on the feedback adjustment signal.

[0017] Furthermore, the router dynamically adjusts the transmission priority level of the priority transmission channel based on the feedback adjustment signal, specifically including: when the feedback adjustment signal indicates that network congestion is worsening, the router increases the priority level of the priority transmission channel; when the feedback adjustment signal indicates that the identification confidence level is consistently low, the router sends a bitrate increase instruction to the camera to increase the audio encoding bitrate.

[0018] Furthermore, prior to step S1, the method further includes a multi-device arbitration step: When multiple cameras detect the wake word simultaneously, each camera reports the audio feature information of its collected audio to the gateway; the gateway selects a main response terminal from the multiple cameras based on the audio feature information, and sends a suppression response command to the unselected cameras, and the main response terminal performs the subsequent steps; The audio feature information includes audio signal-to-noise ratio and sound source direction angle; the gateway prioritizes the one with the highest audio signal-to-noise ratio or the one with the smallest deviation between the sound source direction angle and the preset reference direction.

[0019] Furthermore, in step S4, before performing real-time speech recognition processing on the streaming audio data, echo cancellation processing is also included: The gateway obtains a reference signal of the response audio stream sent to the designated playback device within a preset time window, compares the reference signal with the audio signal collected by the camera, and filters out the echo component related to the reference signal from the collected audio signal. The gateway dynamically adjusts the echo cancellation processing parameters based on the network latency information forwarded by the router, wherein the network latency information reflects the loopback latency of the response audio stream from the gateway to the designated playback device and then collected by the camera.

[0020] Furthermore, in step S5, after the designated playback device receives the response audio stream, it buffers the response audio stream. When the amount of buffered data reaches the pre-playback threshold, playback begins. During playback, for any detected missing data packets, a local retransmission request is initiated to the router, and the router responds to the retransmission request from the local retransmission cache. The gateway performs task operations according to the control instructions, including: If the control command is a device control type, the gateway sends the control signal to the target IoT device via the router; If the control command is an information query type, the gateway will first query the local cache data. If the local cache is not found, it will send a query request to the cloud through an encrypted channel. The advantages of this invention are: 1. By constructing a completely closed-loop end-to-end processing architecture within the local area network, the inherent latency bottlenecks caused by public network transmission and cloud queuing are eliminated from the physical path. Furthermore, a streaming parallel processing mechanism is introduced, enabling the gateway to receive and recognize audio segmented data packets as they arrive one by one. Simultaneously, a dual-confidence threshold strategy is used to pre-initiate semantic parsing and candidate instruction pre-execution, highly overlapping recognition computation and task execution on the timeline, thereby significantly reducing end-to-end response time. At the transmission layer, the router dynamically establishes priority transmission channels for voice services and adaptively adjusts transmission priority and audio encoding bitrate in real time based on the recognition confidence and network congestion status fed back by the gateway. The timing alignment of data packets, local retransmission of lost packets, and buffer preloading mechanisms at the playback end not only ensure stable and reliable transmission of voice streams in complex home Wi-Fi environments, but also filter out interference from the device's own audio playback through echo cancellation, ensuring the purity of the collected user commands and thus improving recognition accuracy. At the same time, for home scenarios with multiple devices, a multi-device arbitration mechanism is introduced to intelligently select a unique primary response terminal and suppress other devices based on the audio signal-to-noise ratio and sound source direction angle, fundamentally avoiding "one call, many responses" and mutual interference. Thus, in complex and dynamic home network environments, it provides a complete technical solution for smart home voice interaction that combines low latency, high reliability, strong anti-interference, and a good human-computer experience.

[0021] 2. The voice interaction method completes the closed loop of voice acquisition, transmission, recognition, semantic parsing, and response playback within the local area network, without relying on public network transmission. This architecture design makes full use of the physical proximity of cameras, routers, and gateways within the home LAN, offloading the computationally intensive voice recognition and semantic parsing tasks to gateway devices with strong computing power. This avoids the bottleneck of physical latency on the public network and the latency fluctuations caused by network congestion, and also avoids the privacy leakage risks caused by continuous audio stream uplink transmission to the public network. It provides a fundamental architectural guarantee for compressing end-to-end latency to within the threshold of smooth human-computer interaction perception.

[0022] 3. On the router side, voice interaction services are given a higher transmission priority than regular service data, and a wireless access point scheduling mechanism is enabled to allocate dedicated transmission resources for the camera's voice data. At the same time, the gateway does not passively receive data, but determines the segmentation transmission parameters based on its current computing load when entering the voice reception preparation state and feeds them back to the camera, so that the camera can adaptively segment audio data according to the gateway's computing power. This two-way adaptation mechanism not only ensures a low-latency path for voice data at the network transmission level, but also controls the inflow rate of audio data at the source, effectively avoiding processing delays caused by gateway computing overload.

[0023] 4. A streaming parallel processing architecture that receives, recognizes, and parses simultaneously, combined with a hierarchical processing mechanism based on a first confidence threshold and a second confidence threshold: when the recognition result reaches a lower first confidence threshold, semantic parsing is initiated in advance and a set of candidate control commands is generated; when a candidate command reaches a higher second confidence threshold, the final command is determined, and during the confidence transition period, the candidate commands are pre-executed in parallel and the results are cached; this mechanism makes the voice interaction processing pipeline highly parallel, and the pre-execution of candidate commands makes the task response after the final command is determined almost instantaneous, significantly compressing the end-to-end latency from the end of the voice command to the execution of the task operation.

[0024] 5. Construct a feedback control loop from the gateway through the router to the camera. The gateway generates feedback adjustment signals based on the current voice recognition confidence level and network transmission status. The router then dynamically increases the priority level of the priority transmission channel to cope with increased network congestion, or sends a bitrate increase instruction to the camera to improve the audio encoding bitrate, thereby improving the problem of persistently low recognition confidence. This closed-loop feedback mechanism enables the system to adaptively adjust the allocation of transmission resources and audio encoding strategies under changing network conditions and recognition quality, effectively suppressing interactive delay jitter and recognition accuracy decline caused by network fluctuations or insufficient audio quality.

[0025] 6. A multi-device arbitration step is set up. When multiple cameras detect the wake word at the same time, each camera reports audio feature information such as audio signal-to-noise ratio and sound source direction angle to the gateway. The gateway selects a main response terminal from multiple candidate devices based on the priority condition of the highest signal-to-noise ratio or the smallest deviation of the sound source direction angle from the preset reference direction, and sends a suppression response command to the unselected cameras. This arbitration mechanism enables the system to intelligently and accurately locate the most suitable sound acquisition device in multi-device coexistence scenarios, fundamentally eliminating the conflict of multiple devices responding at the same time, and significantly improving the voice interaction experience in multi-device home environments.

[0026] 7. By carrying timing identifier information in fragmented data packets, the gateway can perform timing alignment, out-of-order rearrangement, and trigger packet loss retransmission for each fragment. At the same time, the camera automatically determines the end of the voice command through endpoint detection, ensuring the integrity and timing correctness of the audio data during transmission. The gateway obtains the reference signal of the reply audio stream and compares it with the acquired audio signal, filtering out echo components related to the reference signal. The echo cancellation processing parameters are dynamically adjusted based on network latency information, effectively suppressing the interference of playback device echo on speech recognition. The combination of these two methods ensures that even in complex acoustic environments containing playback device echo, the system can still obtain high-quality voice input, guaranteeing that the recognition accuracy is not affected.

[0027] 8. An adaptive transmission and encoding adjustment capability has been constructed. On the one hand, fragmented data packets carry timing identifiers, enabling the gateway to perform out-of-order reordering and accurately trigger packet loss retransmission. At the same time, the camera automatically determines the end of the voice through endpoint detection, avoiding tail truncation or redundant transmission. On the other hand, the gateway generates feedback signals in real time based on recognition confidence and network congestion status. The router dynamically upgrades the priority channel level or instructs the camera to increase the audio encoding bitrate accordingly. This two-way closed-loop feedback mechanism enables the system to prioritize voice transmission when the network deteriorates and actively improve the audio source quality when recognition is difficult, greatly enhancing the stability of the system in complex environments such as Wi-Fi interference.

[0028] 9. For home or office scenarios with multiple devices coexisting, an arbitration mechanism is proposed for simultaneous wake-up of multiple cameras. The gateway intelligently selects the device closest to the sound source or with the clearest sound pickup as the main response terminal based on the audio signal-to-noise ratio and sound source direction angle, and suppresses other devices. This advantage effectively avoids duplicate recognition and resource waste caused by "one call, a hundred responses". At the same time, echo cancellation processing based on network latency dynamic adjustment parameters is introduced. The reference signal sent to the playback device is used to filter out the echo component in the acquired audio, which solves the acoustic interference caused by the close coupling between the microphone and the speaker in the local closed loop, ensuring the purity of the input audio and thus improving the recognition accuracy.

[0029] 10. At the level of response broadcasting and task execution, a buffer pre-playback mechanism and local retransmission cache are proposed at the receiving end. The playback device only starts playing when the buffer amount reaches the threshold, effectively eliminating stuttering caused by network jitter. Missing data packets can be directly retransmitted from the router without retransmission at the source. When executing task operations, control commands are classified and processed: device control commands are directly sent to IoT devices, while information query commands are prioritized for querying the local cache. This advantage cleverly balances the efficiency of real-time control with the timeliness of information query. Access to the cloud is only made through an encrypted channel when the local cache is not hit, which reduces unnecessary public network access and ensures that query commands can obtain the latest data, reflecting the comprehensiveness of the system design. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] Figure 1 This is a flowchart of a low-latency voice interaction method based on edge computing camera router gateway according to the present invention. Detailed Implementation

[0032] The technical solution in this application embodiment has the following general idea: It fully utilizes the heterogeneous computing power and physical proximity of cameras, routers, and gateways within a home LAN to construct a completely closed-loop end-to-end low-latency voice interaction pipeline within the LAN. The camera continuously collects audio and detects wake words locally to trigger the process. The router then dynamically establishes a priority transmission channel for the voice service and adaptively adjusts the transmission priority and audio encoding bitrate based on gateway feedback. The camera segments the voice command following the wake word into time-sequenced data packets and sends them through the priority channel. During the reception of these segments, the gateway performs parallel processing of receiving, recognizing, and parsing the streaming audio, and utilizes dual confidence levels. A threshold mechanism initiates semantic parsing and candidate command pre-execution in advance to compress end-to-end response time. Simultaneously, a multi-device arbitration mechanism selects a unique primary response terminal based on audio signal-to-noise ratio and sound source direction angle to resolve multi-device conflicts. The echo cancellation parameters are dynamically adjusted using the response audio reference signal combined with network latency to filter out playback interference. Finally, control commands are executed in categories (device control is directly issued, and information queries are prioritized to hit the local cache), and the response audio is pushed to the designated device via a buffer pre-playback mechanism. The entire process does not rely on public network transmission, thus providing a comprehensive technical solution for voice interaction in complex and dynamic home network environments that combines ultra-low latency, high reliability, strong anti-interference, and privacy and security protection. In this preferred embodiment, the system architecture includes three core devices: a camera, a router, and a gateway, all deployed within the same home local area network (LAN). The camera is an intelligent camera device equipped with an embedded microphone array and a local wake-up engine, responsible for continuous ambient audio acquisition, local wake-up word detection, and segmented transmission of voice command audio data. The router is a network device supporting Quality of Service (QoS) mechanisms and wireless access point scheduling, responsible for establishing priority transmission channels for voice interaction services and dynamically adjusting transmission priority and sending bitrate adjustment instructions to the camera based on gateway feedback during interaction. The gateway is a home hub device equipped with an AI acceleration chip or a high-performance processor, with a built-in Automatic Speech Recognition (ASR) model, Natural Language Understanding (NLU) engine, and Text-to-Speech (TTS) module, responsible for streaming speech recognition, semantic parsing, control command generation, task execution, and response audio synthesis and push. The three types of devices are connected via Wi-Fi or wired Ethernet, and voice acquisition, transmission, recognition, semantic parsing, and response playback are all completed within a closed loop within the LAN, without relying on public network transmission.

[0033] Please refer to Figure 1 As shown, a preferred embodiment of the low-latency voice interaction method based on edge computing camera router gateway of the present invention includes the following steps: Step S1: The camera continuously collects ambient audio and the local wake-up engine detects a preset wake-up word. When the wake-up word is detected, the camera generates a wake-up trigger message and sends it to the router. Step S2: After receiving the wake-up trigger message, the router identifies the service type of the voice interaction service, establishes a priority transmission channel for the voice interaction service, and forwards the wake-up trigger message to the gateway, triggering the gateway to enter the voice reception preparation state. Step S3: After the wake-up word, the camera cuts the received voice command audio data into continuous segmented data packets according to a preset segmentation rule. Each segmented data packet contains an audio content segment of the voice command audio data and is sent to the gateway via the router's priority transmission channel. Step S4: During the process of receiving the fragmented data packets, the gateway performs real-time speech recognition processing on the streaming audio data and performs semantic parsing based on the recognition results to generate control commands; wherein, the gateway starts the semantic parsing in advance when the recognition results meet the preset confidence conditions, so as to realize streaming parallel processing of receiving, recognizing and parsing at the same time. Preferably, the gateway incorporates a lightweight natural language understanding engine, which is implemented based on an edge-optimized intent classification model and slot-filling model. The specific semantic parsing process is as follows: the text obtained in real-time is input into the engine, which outputs structured control commands. These control commands follow a preset JSON format and contain at least... Intent field (intent, such as control_device or query_info) Device field (devices, such as light and curtain) and move or parameter field (action or params, such as on, temperature=26). When the recognized text is "turn on the living room light", the parsing engine outputs the intent as control_device, the device as light, the location as living_room, and the action as on, thereby generating standardized control instructions.

[0034] Step S5: The gateway executes the task operation according to the control command, and converts the execution result or the reply content obtained by the query into a reply audio stream, which is then pushed to the designated playback device for playback via the priority transmission channel of the router; In this embodiment, the designated playback device is a smart speaker, smart TV, or smart home control screen with audio playback function deployed within the same local area network, connected to the router via Wi-Fi or a wired network. The pre-playback threshold is set as follows: the target buffering time of the audio decoding buffer built into the playback device is set to 300ms. That is, when the buffered data reaches an audio frame of 300ms (calculated at a 16kHz sampling rate, 16-bit depth, mono PCM format, corresponding to approximately 9600 bytes of original audio data, which is approximately 1200-2400 bytes after compression encoding, depending on the encoding format), playback begins. This threshold aims to minimize the first frame playback delay while ensuring playback continuity. During playback, the playback device monitors the sequence number of the received fragmented data packets in real time. When a discontinuous sequence number is detected, a local retransmission request is immediately initiated to the router, with a retransmission timeout of 50ms. If the retransmission is successful within 50ms, the data packet is inserted into the corresponding position in the decoding buffer. If the retransmission data packet is not received after the timeout, the playback device initiates packet loss concealment processing, using interpolation compensation with audio data from the preceding and following frames to ensure uninterrupted playback.

[0035] In this method, voice acquisition, transmission, recognition, semantic parsing, and response playback are all completed in a closed loop within a local area network, without relying on public network transmission.

[0036] In step S2, establishing a priority transmission channel for the voice interaction service specifically includes: the router parsing the service type identifier in the received data packet, the received data packet including at least the wake-up trigger message, allocating a higher transmission priority to the data stream belonging to the voice interaction service than to the regular service data, and enabling the wireless access point scheduling mechanism to allocate dedicated transmission resources to the camera; The step of triggering the gateway to enter the voice reception preparation state includes: the gateway loading the voice recognition model and semantic parsing model into the cache, determining the fragmentation transmission parameters according to its current computing load, feeding back the fragmentation transmission parameters to the camera via the router, and the camera performing the operation of cutting the voice command audio data into continuous fragmented data packets in step S3 according to the fragmentation transmission parameters.

[0037] Specifically, the fragmentation transmission parameters include fragmentation duration and audio encoding sampling bit depth. The gateway calculates a recommended fragmentation duration based on the current CPU utilization and remaining memory space. For example, when the gateway's computational load is below 60%, a fragmentation duration of 200ms is recommended to reduce encapsulation overhead; when the load is above 80%, a fragmentation duration of 500ms is recommended to reduce interrupt processing frequency. The fragmentation transmission parameters are fed back to the camera via the router using a custom extended wireless access point control frame or application layer UDP signaling. The camera dynamically configures the audio sampling output buffer size of the analog-to-digital converter based on these parameters.

[0038] As a specific implementation of this embodiment, the gateway determines the segmentation transmission parameters based on its current computing load using the following logic: The gateway monitors its CPU utilization and remaining memory space in real time. When the CPU utilization is below 60% and the remaining memory space is greater than 512MB, it is determined to be in a low-load state. At this time, the segmentation duration is set to 200ms and the audio encoding sampling bit depth is set to 16bit to reduce encapsulation overhead and improve resolution granularity. When the CPU utilization is between 60% and 80% or the remaining memory space is between 256MB and 512MB, it is determined to be in a medium-load state. The segmentation duration is set to 300ms and the sampling bit depth is set to 16bit. When the CPU utilization is above 80% or the remaining memory space is below 256MB, it is determined to be in a high-load state. The segmentation duration is set to 500ms and the sampling bit depth is set to 8bit to reduce the gateway's interrupt processing frequency and memory usage. The segmented transmission parameters are fed back to the camera via a router in a custom extended wireless access point control frame. The camera dynamically configures the audio sampling output buffer size of the analog-to-digital converter based on these parameters and segments and packages the audio data according to the segment duration.

[0039] In step S3, the fragmented data packets carry timing identification information. The gateway performs timing alignment and out-of-order rearrangement of each fragmented data packet according to the timing identification information, and detects packet loss according to the timing identification information, triggering a retransmission mechanism for the detected packet loss. During the transmission of the segmented data packets, the camera performs endpoint detection on the audio data of the voice command. When it detects that the user's voice pause exceeds a preset duration, it automatically determines that the voice command has ended and sends a transmission end identifier to the gateway.

[0040] The timing identification information includes an audio frame sequence number and a raw timestamp. The audio frame sequence number is a 32-bit unsigned integer starting from 0 and incrementing sequentially, used to uniquely identify the order in which audio segments are captured. The raw timestamp is the local system clock value when the camera captured the first byte of the audio data, in milliseconds. After receiving the segmented data packets, the gateway extracts the sequence number and timestamp. If a non-contiguous sequence number is detected, packet loss is determined, and the gateway immediately requests the router to retransmit data packets within the specified sequence number range.

[0041] In step S4, the preset confidence conditions include a first confidence threshold and a second confidence threshold, wherein the second confidence threshold is higher than the first confidence threshold; when the confidence of the recognition result reaches the first confidence threshold, the gateway initiates semantic parsing of the currently recognized text and generates a set of candidate control instructions; when the confidence of a candidate instruction in the set of candidate control instructions reaches the second confidence threshold, the gateway determines the candidate instruction as the final control instruction.

[0042] During the period when the confidence level of the identification result reaches the first confidence level threshold but has not yet reached the second confidence level threshold, the gateway performs parallel pre-execution operations according to each candidate instruction in the candidate control instruction set, generates corresponding pre-execution results and caches them; when the final control instruction is determined, if the final control instruction matches a certain pre-execution result, the pre-execution result is directly called to execute the task operation in step S5.

[0043] In this embodiment, the confidence scores of the candidate control commands output by the semantic parsing engine are softmax normalized probability values, ranging from 0 to 1, and the sum of the confidence scores of all candidate commands is 1. For example, when a user says "turn on the living room light," the semantic parsing engine may output three candidate commands: Command A (intent: control_device, device: light, location: living_room, action: on) has a confidence score of 0.72, Command B (intent: control_device, device: light, location: living_room, action: dim, parameter: brightness 50%) has a confidence score of 0.18, and Command C (intent: control_device, device: fan, location: living_room, action: on) has a confidence score of 0.10. A first confidence threshold is set to 0.45, and a second confidence threshold is set to 0.80. Since the confidence score of instruction A is 0.72, reaching the first confidence threshold of 0.45, the gateway immediately initiates semantic parsing and generates a set of candidate control instructions. Simultaneously, it performs parallel pre-execution operations on this instruction—for example, sending a pre-on signal to the living room light and caching the pre-execution result. Because the confidence score of instruction A is 0.72, which has not yet reached the second confidence threshold of 0.80, the gateway continues to receive subsequent audio segments and accumulate recognition information. Once all subsequent segments have been received, the confidence score of instruction A is updated to 0.85, reaching the second confidence threshold. The gateway then identifies it as the final control instruction and directly calls the cached pre-execution result to complete the light-on operation. If the same recognized text yields multiple mutually exclusive candidate instructions (such as "turn on the light" and "turn off the light"), the gateway only retains the candidate instruction with the highest confidence score for the threshold determination process, discarding the remaining mutually exclusive candidate instructions to avoid instruction conflicts.

[0044] Between step S4 and step S5, the method further includes a transmission feedback adjustment step: The gateway generates a feedback adjustment signal based on the current confidence level of speech recognition and network transmission status and sends it to the router; the router dynamically adjusts the transmission priority level of the priority transmission channel and sends a bitrate adjustment instruction to the camera based on the feedback adjustment signal.

[0045] The router dynamically adjusts the transmission priority level of the priority transmission channel based on the feedback adjustment signal, specifically including: when the feedback adjustment signal indicates that network congestion is worsening, the router increases the priority level of the priority transmission channel; when the feedback adjustment signal indicates that the identification confidence level is consistently low, the router sends a bitrate increase instruction to the camera to increase the audio encoding bitrate.

[0046] In this embodiment, the specific judgment logic for the gateway to generate feedback adjustment signals is as follows: After the gateway receives each audio segment data packet, it calculates the average confidence score C_avg of the current cumulative recognition result and the network transmission delay jitter value J of the most recent N segments (N is set to 5) (defined as the standard deviation of the time interval between adjacent segments). When C_avg is below the first confidence threshold of 0.45 and J is greater than 20ms, it is determined to be "concurrently with increased network congestion and low recognition quality". The feedback adjustment signal carries a priority upgrade request (suggesting a priority level upgrade of 2 levels) and a bitrate upgrade indication (upgrading from 8-bit sampling depth to 16-bit sampling depth). When C_avg is below the first confidence threshold of 0.45 but J is not greater than 20ms, it is determined to be "low recognition quality but good network condition". The feedback adjustment signal only carries a bitrate upgrade indication, and the priority level remains unchanged. When C_avg is not lower than the first confidence threshold of 0.45 but J is greater than 20ms, it is determined to be "normal recognition quality but network congestion". The feedback adjustment signal only carries a priority upgrade request (suggesting a priority level upgrade of 1 level), and the coding bitrate is not adjusted. When C_avg is not lower than the first confidence threshold and J is not greater than 20ms, it is determined to be "normal status". The feedback adjustment signal does not carry any adjustment indication and maintains the current transmission parameters. After receiving the feedback adjustment signal, the router performs the corresponding priority level adjustment and bit rate adjustment operations according to the instructions in the signal.

[0047] Prior to step S1, the method further includes a multi-device arbitration step: When multiple cameras detect the wake word simultaneously, each camera reports the audio feature information of its collected audio to the gateway; the gateway selects a main response terminal from the multiple cameras based on the audio feature information, and sends a suppression response command to the unselected cameras, and the main response terminal performs the subsequent steps; The audio feature information includes audio signal-to-noise ratio and sound source direction angle; the gateway prioritizes the one with the highest audio signal-to-noise ratio or the one with the smallest deviation between the sound source direction angle and the preset reference direction.

[0048] To clearly define the arbitration criteria, the preset reference direction is set as the user's commonly used interaction direction pre-calibrated by the gateway or home hub. The system provides an initial configuration phase, allowing users to set the living room sofa direction as the reference direction (e.g., 15 degrees east of due north) via a mobile application. When multiple cameras simultaneously report sound source direction angles (e.g., camera A reports an angle deviation of 5 degrees, camera B reports an angle deviation of 20 degrees) and signal-to-noise ratios (e.g., A is 30dB, B is 35dB), the gateway employs a weighted decision algorithm: prioritizing the device with the highest signal-to-noise ratio as the primary response terminal. If the deviation of the device with the highest signal-to-noise ratio from the reference direction exceeds a preset deviation threshold (e.g., 25 degrees), then the device with the second highest signal-to-noise ratio but the smallest direction deviation is selected as the primary response terminal.

[0049] In this embodiment, the gateway uses a weighted decision algorithm to select the primary response terminal from multiple cameras. The specific calculation method is as follows: For each camera i that reports audio feature information, the gateway calculates its comprehensive score S_i = α × SNR_i − β × |θ_i − θ_ref|, where SNR_i is the audio signal-to-noise ratio (SNR) reported by the camera (in dB), θ_i is the sound source direction angle reported by the camera, θ_ref is a preset reference direction angle, and α and β are weighting coefficients. In this embodiment, α is set to 1.0 and β is set to 0.2 (in minutes / degree), so that a 1 dB SNR advantage is equivalent to a 5° direction angle deviation advantage. The preset deviation threshold is set to 25 degrees. The gateway selects the camera with the highest comprehensive score S_i as the primary response terminal. For example, if camera A reports an SNR of 30dB and a directional deviation of 5°, its overall score S_A = 1.0 × 30 − 0.2 × 5 = 29.0; and camera B reports an SNR of 35dB and a directional deviation of 20°, its overall score S_B = 1.0 × 35 − 0.2 × 20 = 31.0, the gateway will select camera B, which has the higher overall score, as the primary response terminal, rather than simply choosing the one with the highest SNR. If the deviation of the device with the highest SNR from the reference direction exceeds a preset deviation threshold of 25°, the next candidate device will be selected based on its overall score ranking, avoiding the selection of a device with a high SNR but whose sound source location is significantly deviated from the user's commonly used interaction location.

[0050] In step S4, before the streaming audio data undergoes real-time speech recognition processing, echo cancellation processing is also included: The gateway obtains a reference signal of the response audio stream sent to the designated playback device within a preset time window, compares the reference signal with the audio signal collected by the camera, and filters out the echo component related to the reference signal from the collected audio signal. The gateway dynamically adjusts the echo cancellation processing parameters based on the network latency information forwarded by the router, wherein the network latency information reflects the loopback latency of the response audio stream from the gateway to the designated playback device and then collected by the camera.

[0051] The network latency information is estimated by the router through the exchange of Sender Report (SR) and Receiver Report (RR) messages using the Real-time Transmission Control Protocol (RTCP). Specifically, when the gateway sends a response audio stream, it sends an RTCP SR message to the playback device via the router. The playback device replies with an RTCP RR message containing the link round-trip time (RTT), and the router forwards this RTT value to the gateway in real time. Based on this dynamically changing RTT value, the gateway's echo cancellation module synchronously adjusts the delay estimation window length of the adaptive filter, thereby accurately aligning the echo components in the reference signal and the acquired signal, ensuring that the echo suppression effect is not degraded by network fluctuations.

[0052] In step S5, after the designated playback device receives the response audio stream, it buffers the response audio stream. When the amount of buffered data reaches the pre-playback threshold, it starts playing. During the playback process, it sends a local retransmission request to the router for any detected missing data packets. The router responds to the retransmission request from the local retransmission cache. The router has a built-in circular buffer as a local retransmission cache, which uses a first-in, first-out (FIFO) strategy. When the router receives a response audio data packet from the gateway to the playback device, it simultaneously copies and stores the data packet in its local cache, setting the cache aging time to 500 milliseconds. If the playback device detects a missing sequence number during playback and initiates a retransmission request, the router first looks up the corresponding data packet in its local retransmission cache and replies directly. If the data packet in the cache has expired and been discarded, then the router initiates a retransmission request to the gateway. This avoids long-distance data source retransmissions and significantly reduces packet loss compensation latency.

[0053] The gateway performs task operations according to the control instructions, including: If the control command is a device control type, the gateway sends the control signal to the target IoT device via the router; If the control command is an information query type, the gateway will first query the local cache data. If the local cache is not found, it will send a query request to the cloud through an encrypted channel. In summary, the advantages of this invention are as follows: 1. By constructing a completely closed-loop end-to-end processing architecture within the local area network, the inherent latency bottlenecks caused by public network transmission and cloud queuing are eliminated from the physical path. Furthermore, a streaming parallel processing mechanism is introduced, enabling the gateway to receive and recognize audio segmented data packets as they arrive one by one. Simultaneously, a dual-confidence threshold strategy is used to pre-initiate semantic parsing and candidate instruction pre-execution, highly overlapping recognition computation and task execution on the timeline, thereby significantly reducing end-to-end response time. At the transmission layer, the router dynamically establishes priority transmission channels for voice services and adaptively adjusts transmission priority and audio encoding bitrate in real time based on the recognition confidence and network congestion status fed back by the gateway. The timing alignment of data packets, local retransmission of lost packets, and buffer preloading mechanisms at the playback end not only ensure stable and reliable transmission of voice streams in complex home Wi-Fi environments, but also filter out interference from the device's own audio playback through echo cancellation, ensuring the purity of the collected user commands and thus improving recognition accuracy. At the same time, for home scenarios with multiple devices, a multi-device arbitration mechanism is introduced to intelligently select a unique primary response terminal and suppress other devices based on the audio signal-to-noise ratio and sound source direction angle, fundamentally avoiding "one call, many responses" and mutual interference. Thus, in complex and dynamic home network environments, it provides a complete technical solution for smart home voice interaction that combines low latency, high reliability, strong anti-interference, and a good human-computer experience.

[0054] 2. The voice interaction method completes the closed loop of voice acquisition, transmission, recognition, semantic parsing, and response playback within the local area network, without relying on public network transmission. This architecture design makes full use of the physical proximity of cameras, routers, and gateways within the home LAN, offloading the computationally intensive voice recognition and semantic parsing tasks to gateway devices with strong computing power. This avoids the bottleneck of physical latency on the public network and the latency fluctuations caused by network congestion, and also avoids the privacy leakage risks caused by continuous audio stream uplink transmission to the public network. It provides a fundamental architectural guarantee for compressing end-to-end latency to within the threshold of smooth human-computer interaction perception.

[0055] 3. On the router side, voice interaction services are given a higher transmission priority than regular service data, and a wireless access point scheduling mechanism is enabled to allocate dedicated transmission resources for the camera's voice data. At the same time, the gateway does not passively receive data, but determines the segmentation transmission parameters based on its current computing load when entering the voice reception preparation state and feeds them back to the camera, so that the camera can adaptively segment audio data according to the gateway's computing power. This two-way adaptation mechanism not only ensures a low-latency path for voice data at the network transmission level, but also controls the inflow rate of audio data at the source, effectively avoiding processing delays caused by gateway computing overload.

[0056] 4. A streaming parallel processing architecture that receives, recognizes, and parses simultaneously, combined with a hierarchical processing mechanism based on a first confidence threshold and a second confidence threshold: when the recognition result reaches a lower first confidence threshold, semantic parsing is initiated in advance and a set of candidate control commands is generated; when a candidate command reaches a higher second confidence threshold, the final command is determined, and during the confidence transition period, the candidate commands are pre-executed in parallel and the results are cached; this mechanism makes the voice interaction processing pipeline highly parallel, and the pre-execution of candidate commands makes the task response after the final command is determined almost instantaneous, significantly compressing the end-to-end latency from the end of the voice command to the execution of the task operation.

[0057] 5. Construct a feedback control loop from the gateway through the router to the camera. The gateway generates feedback adjustment signals based on the current voice recognition confidence level and network transmission status. The router then dynamically increases the priority level of the priority transmission channel to cope with increased network congestion, or sends a bitrate increase instruction to the camera to improve the audio encoding bitrate, thereby improving the problem of persistently low recognition confidence. This closed-loop feedback mechanism enables the system to adaptively adjust the allocation of transmission resources and audio encoding strategies under changing network conditions and recognition quality, effectively suppressing interactive delay jitter and recognition accuracy decline caused by network fluctuations or insufficient audio quality.

[0058] 6. A multi-device arbitration step is set up. When multiple cameras detect the wake word at the same time, each camera reports audio feature information such as audio signal-to-noise ratio and sound source direction angle to the gateway. The gateway selects a main response terminal from multiple candidate devices based on the priority condition of the highest signal-to-noise ratio or the smallest deviation of the sound source direction angle from the preset reference direction, and sends a suppression response command to the unselected cameras. This arbitration mechanism enables the system to intelligently and accurately locate the most suitable sound acquisition device in multi-device coexistence scenarios, fundamentally eliminating the conflict of multiple devices responding at the same time, and significantly improving the voice interaction experience in multi-device home environments.

[0059] 7. By carrying timing identifier information in fragmented data packets, the gateway can perform timing alignment, out-of-order rearrangement, and trigger packet loss retransmission for each fragment. At the same time, the camera automatically determines the end of the voice command through endpoint detection, ensuring the integrity and timing correctness of the audio data during transmission. The gateway obtains the reference signal of the reply audio stream and compares it with the acquired audio signal, filtering out echo components related to the reference signal. The echo cancellation processing parameters are dynamically adjusted based on network latency information, effectively suppressing the interference of playback device echo on speech recognition. The combination of these two methods ensures that even in complex acoustic environments containing playback device echo, the system can still obtain high-quality voice input, guaranteeing that the recognition accuracy is not affected.

[0060] 8. An adaptive transmission and encoding adjustment capability has been constructed. On the one hand, fragmented data packets carry timing identifiers, enabling the gateway to perform out-of-order reordering and accurately trigger packet loss retransmission. At the same time, the camera automatically determines the end of the voice through endpoint detection, avoiding tail truncation or redundant transmission. On the other hand, the gateway generates feedback signals in real time based on recognition confidence and network congestion status. The router dynamically upgrades the priority channel level or instructs the camera to increase the audio encoding bitrate accordingly. This two-way closed-loop feedback mechanism enables the system to prioritize voice transmission when the network deteriorates and actively improve the audio source quality when recognition is difficult, greatly enhancing the stability of the system in complex environments such as Wi-Fi interference.

[0061] 9. For home or office scenarios with multiple devices coexisting, an arbitration mechanism is proposed for simultaneous wake-up of multiple cameras. The gateway intelligently selects the device closest to the sound source or with the clearest sound pickup as the main response terminal based on the audio signal-to-noise ratio and sound source direction angle, and suppresses other devices. This advantage effectively avoids duplicate recognition and resource waste caused by "one call, a hundred responses". At the same time, echo cancellation processing based on network latency dynamic adjustment parameters is introduced. The reference signal sent to the playback device is used to filter out the echo component in the acquired audio, which solves the acoustic interference caused by the close coupling between the microphone and the speaker in the local closed loop, ensuring the purity of the input audio and thus improving the recognition accuracy.

[0062] 10. At the level of response broadcasting and task execution, a buffer pre-playback mechanism and local retransmission cache are proposed at the receiving end. The playback device only starts playing when the buffer amount reaches the threshold, effectively eliminating stuttering caused by network jitter. Missing data packets can be directly retransmitted from the router without retransmission at the source. When executing task operations, control commands are classified and processed: device control commands are directly sent to IoT devices, while information query commands are prioritized for querying the local cache. This advantage cleverly balances the efficiency of real-time control with the timeliness of information query. Access to the cloud is only made through an encrypted channel when the local cache is not hit, which reduces unnecessary public network access and ensures that query commands can obtain the latest data, reflecting the comprehensiveness of the system design.

[0063] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A low-latency voice interaction method based on edge computing camera router gateway, characterized in that, Includes the following steps: Step S1: The camera continuously collects ambient audio and the local wake-up engine detects a preset wake-up word. When the wake-up word is detected, the camera generates a wake-up trigger message and sends it to the router. Step S2: After receiving the wake-up trigger message, the router identifies the service type of the voice interaction service, establishes a priority transmission channel for the voice interaction service, and forwards the wake-up trigger message to the gateway, triggering the gateway to enter the voice reception preparation state. Step S3: After the wake-up word, the camera cuts the received voice command audio data into continuous segmented data packets according to a preset segmentation rule. Each segmented data packet contains an audio content segment of the voice command audio data and is sent to the gateway via the router's priority transmission channel. Step S4: During the process of receiving the fragmented data packets, the gateway performs real-time speech recognition processing on the streaming audio data and performs semantic parsing based on the recognition results to generate control commands; wherein, the gateway starts the semantic parsing in advance when the recognition results meet the preset confidence conditions, so as to realize streaming parallel processing of receiving, recognizing and parsing at the same time. Step S5: The gateway executes the task operation according to the control command, and converts the execution result or the reply content obtained by the query into a reply audio stream, which is then pushed to the designated playback device for playback via the priority transmission channel of the router; In this method, voice acquisition, transmission, recognition, semantic parsing, and response playback are all completed in a closed loop within a local area network, without relying on public network transmission.

2. The low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, In step S2, establishing a priority transmission channel for the voice interaction service specifically includes: the router parsing the service type identifier in the received data packet, the received data packet including at least the wake-up trigger message, allocating a higher transmission priority to the data stream belonging to the voice interaction service than to the regular service data, and enabling the wireless access point scheduling mechanism to allocate dedicated transmission resources to the camera; The step of triggering the gateway to enter the voice reception preparation state includes: the gateway loading the voice recognition model and semantic parsing model into the cache, determining the fragmentation transmission parameters according to its current computing load, feeding back the fragmentation transmission parameters to the camera via the router, and the camera performing the operation of cutting the voice command audio data into continuous fragmented data packets in step S3 according to the fragmentation transmission parameters.

3. The low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, In step S3, the fragmented data packets carry timing identification information. The gateway performs timing alignment and out-of-order rearrangement of each fragmented data packet according to the timing identification information, and detects packet loss according to the timing identification information, triggering a retransmission mechanism for the detected packet loss. During the transmission of the segmented data packets, the camera performs endpoint detection on the audio data of the voice command. When it detects that the user's voice pause exceeds a preset duration, it automatically determines that the voice command has ended and sends a transmission end identifier to the gateway.

4. The low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, In step S4, the preset confidence conditions include a first confidence threshold and a second confidence threshold, wherein the second confidence threshold is higher than the first confidence threshold; when the confidence of the recognition result reaches the first confidence threshold, the gateway initiates semantic parsing of the currently recognized text and generates a set of candidate control instructions; when the confidence of a candidate instruction in the set of candidate control instructions reaches the second confidence threshold, the gateway determines the candidate instruction as the final control instruction.

5. A low-latency voice interaction method based on edge computing camera router gateway as described in claim 4, characterized in that, During the period when the confidence level of the identification result reaches the first confidence level threshold but has not yet reached the second confidence level threshold, the gateway performs parallel pre-execution operations according to each candidate instruction in the candidate control instruction set, generates corresponding pre-execution results, and caches them; Once the final control command is determined, if the final control command matches a certain pre-execution result, then the pre-execution result is directly invoked to execute the task operation in step S5.

6. The low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, Between step S4 and step S5, the method further includes a transmission feedback adjustment step: The gateway generates a feedback adjustment signal based on the current confidence level of speech recognition and network transmission status and sends it to the router; the router dynamically adjusts the transmission priority level of the priority transmission channel and sends a bitrate adjustment instruction to the camera based on the feedback adjustment signal.

7. A low-latency voice interaction method based on edge computing camera router gateway as described in claim 6, characterized in that, The router dynamically adjusts the transmission priority level of the priority transmission channel based on the feedback adjustment signal, specifically including: when the feedback adjustment signal indicates that network congestion is worsening, the router increases the priority level of the priority transmission channel; when the feedback adjustment signal indicates that the identification confidence level is consistently low, the router sends a bitrate increase instruction to the camera to increase the audio encoding bitrate.

8. A low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, Prior to step S1, the method further includes a multi-device arbitration step: When multiple cameras detect the wake word simultaneously, each camera reports the audio feature information of the audio it has collected to the gateway. The gateway selects a primary response terminal from the plurality of cameras based on the audio feature information, and sends a suppression response command to the unselected cameras, with the primary response terminal then executing the subsequent steps. The audio feature information includes audio signal-to-noise ratio and sound source direction angle; the gateway prioritizes the one with the highest audio signal-to-noise ratio or the one with the smallest deviation between the sound source direction angle and the preset reference direction.

9. A low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, In step S4, before the streaming audio data undergoes real-time speech recognition processing, echo cancellation processing is also included: The gateway obtains a reference signal of the response audio stream sent to the designated playback device within a preset time window, compares the reference signal with the audio signal collected by the camera, and filters out the echo component related to the reference signal from the collected audio signal. The gateway dynamically adjusts the echo cancellation processing parameters based on the network latency information forwarded by the router, wherein the network latency information reflects the loopback latency of the response audio stream from the gateway to the designated playback device and then collected by the camera.

10. A low-latency voice interaction method based on edge computing camera router gateway as described in claim 1, characterized in that, In step S5, after the designated playback device receives the response audio stream, it buffers the response audio stream. When the amount of buffered data reaches the pre-playback threshold, it starts playing. During the playback process, it sends a local retransmission request to the router for any detected missing data packets. The router responds to the retransmission request from the local retransmission cache. The gateway performs task operations according to the control instructions, including: If the control command is a device control type, the gateway sends the control signal to the target IoT device via the router; If the control command is an information query type, the gateway will first query the local cache data. If the local cache is not found, it will send a query request to the cloud through an encrypted channel.

Images