Multi-terminal robot real-time communication system and method based on ros2 architecture

Abstract

This invention discloses a multi-terminal robot real-time communication system and method based on the ROS2 architecture, relating to the fields of artificial intelligence and robot communication technology. The system includes interconnected user terminal layers, a unified communication gateway layer, a robot device layer, and a backend management system layer. The user terminal layer issues control commands; the unified communication gateway layer responds to the protocol type of heterogeneous terminals, dynamically loads matching communication protocol plugins, and routes the commands to the robot device layer; the robot device layer receives and executes the commands and reports its status; the backend management system layer sends update plugins to the unified communication gateway layer or the robot device layer. This invention achieves seamless access to heterogeneous protocols across multiple terminals through dynamic protocol plugin loading in the unified communication gateway layer, and eliminates the risk of duplicate command execution due to network anomalies through unique identifier caching and idempotent deduplication mechanisms during reconnection, thus realizing the secure, continuous, and dynamic expansion of AI robot services.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and robot communication technology, and specifically to a multi-terminal robot real-time communication system and method based on the ROS2 architecture. Background Technology

[0002] With the rapid development of AI robot projects, robot application scenarios are becoming increasingly complex, often requiring concurrent control and status monitoring across multiple heterogeneous terminals such as web platforms, mobile apps, PC clients, and mini-programs. Traditional robot communication systems typically employ a single communication protocol, making it difficult to flexibly adapt to the diverse access needs of multiple terminals.

[0003] Furthermore, robots often face network fluctuations or brief disconnections in real-world operating environments (such as outdoors or in factories). Under the existing communication architecture, reconnection after a disconnection often leads to the repeated issuance of cached commands, causing the robot to perform repetitive actions (such as repeated grasping and repetitive movement), posing serious safety risks. At the same time, adding support for new protocols or updating the robot's underlying algorithms typically requires a system shutdown and restart, making seamless dynamic expansion of business functions impossible. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a multi-terminal robot real-time communication system and method based on the ROS2 architecture, aiming to achieve dynamic compatibility of multi-terminal heterogeneous protocols, instruction deduplication under disconnection and reconnection, and uninterrupted hot loading of system functions.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a multi-terminal robot real-time communication system based on the ROS2 architecture, comprising a user terminal layer (101), a unified communication gateway layer (102), a robot device layer (103), and a back-end management system layer (104) that are interconnected.

[0007] The user terminal layer (101) includes multiple heterogeneous terminals, such as Web clients, mobile apps, PC clients and mini programs. The heterogeneous terminals establish a communication link with the unified communication gateway layer (102) and send data packets containing control instructions.

[0008] The unified communication gateway layer (102) includes a protocol plugin manager (1021). The protocol plugin manager (1021) responds to the communication protocol type of the heterogeneous terminal, dynamically calls and loads the matching communication protocol plugin, and converts and routes the control command to the robot device layer (103).

[0009] The robot device layer (103) receives and executes the control command, and reports the execution result and sensor status data to the unified communication gateway layer (102);

[0010] The background management system layer (104) includes a plugin repository (1041). The background management system layer (104) establishes a push link with the unified communication gateway layer (102) or the robot device layer (103) and sends out updated plugins from the plugin repository (1041).

[0011] Furthermore, the protocol plugin manager (1021) loads protocol plugins including MQTT, HTTPS, WebSocketPP, and WeChat JS-SDK.

[0012] Furthermore, the unified communication gateway layer (102) also includes: a protocol conversion module (1022) for performing message format reassembly between different protocol layers; an authentication service (1024) for extracting access tokens from the user terminal layer (101) and performing timeliness verification; and a caching mechanism (1026) for storing unsuccessfully sent instructions into a waiting queue during network fluctuations.

[0013] Furthermore, the robot device layer (103) includes: a robot entity (1031) equipped with at least one actuator and sensor among a camera, radar, robotic arm, chassis and IMU; and a ROS2 module (1032) that establishes distributed communication between internal robot nodes through a publish / subscribe mechanism.

[0014] Furthermore, the backend management system layer (104) calls the reflection mechanism of the protocol plugin from the plugin repository (1041) through the operation and maintenance console (1043) and pushes it to the unified communication gateway layer (102) to perform memory-level hot loading.

[0015] A second aspect of the present invention provides a real-time communication method for multi-terminal robots based on the aforementioned system, comprising:

[0016] When the robot device layer (103) reports its status, the unified communication gateway layer (102) calculates the differential data between the current status and the status snapshot and pushes the differential data to the user terminal layer (101).

[0017] When the user terminal layer (101) issues an instruction, the unified communication gateway layer (102) extracts the verification information through the authentication service (1024) to perform identity verification, and forwards the instruction after performing idempotency verification through the message assurance mechanism (1023).

[0018] Furthermore, the process of the user terminal layer (101) issuing instructions also includes:

[0019] If a network disconnection is detected, the unified communication gateway layer (102) generates a unique identifier for the current instruction and stores it in the cache queue;

[0020] After the network is restored and the session is restored, the unified communication gateway layer (102) performs deduplication based on the unique identifier and forwards the instructions in the cache queue in timestamp order.

[0021] Furthermore, the unified communication gateway layer (102) is configured with a priority arbitration mechanism; when a high-priority control instruction is received, the currently executing low-priority task is suspended, and the execution context of the low-priority task is stored in persistent storage (such as Redis or a local file). After the high-priority instruction is completed, the low-priority task is restored according to the execution context.

[0022] Furthermore, the unified communication gateway layer (102) adopts a dual-channel communication mechanism with each layer, including: a control channel, which uses WebSocket or MQTT long connection to transmit control commands, heartbeat and status data; and a media channel, which transmits audio and video streams based on WebRTC, and the media channel and the control channel are physically separated in network transmission.

[0023] The beneficial effects of this invention are as follows:

[0024] 1. Multi-terminal heterogeneous compatibility and dynamic expansion: Through a unified communication gateway layer and protocol plugin manager, the system can dynamically identify and load protocol plugins for different terminals. Combined with the hot-loading mechanism of the backend management system, communication capabilities and AI robot business functions can be expanded without downtime.

[0025] 2. High-security command deduplication prevention: A comprehensive network disconnection detection and fast reconnection mechanism is designed. When the network is restored, strict idempotency verification and deduplication are performed through a unique identifier and cache window, which completely eliminates the danger of command duplication caused by disconnection and reconnection.

[0026] 3. Low latency and high concurrency adaptation: Introduction of state differential data push and DDS

[0027] The QoS parameter adaptive adjustment mechanism significantly reduces the network bandwidth pressure during high-concurrency multi-device synchronization, ensuring the real-time performance of the core control flow. Attached Figure Description

[0028] Figure 1 This is a flowchart of the disconnection detection and rapid reconnection process of the present invention.

[0029] Figure 2 This is a timing diagram for hot loading of the protocol / function of the present invention.

[0030] Figure 3This is the system architecture diagram of the present invention.

[0031] Figure 4 This is a diagram showing the action relationships of the present invention. Detailed Implementation

[0032] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0033] Example 1: System Infrastructure and Hierarchical Division

[0034] Combination Figure 3 This embodiment provides a multi-terminal robot real-time communication system based on the ROS2 architecture.

[0035] The user terminal layer (101) includes a web client (1011), a mobile app (1012), a PC client (1013), and a WeChat official account mini-program (1014).

[0036] The unified communication gateway layer (102) includes a protocol plugin manager (1021), a protocol conversion module (1022), a message assurance mechanism (1023), an authentication service (1024), a device session (1025), a caching mechanism (1026), permission management (1027), a log system (1028), and traffic statistics (1029). Among them, the protocol plugin manager (1021) integrates support for protocols including MQTT, HTTPS, WebSocketPP, and WeChat JS-SDK.

[0037] The robot equipment layer (103) includes a robot body (1031), a ROS2 module (1032), a communication module (1033), and a hardware control module (1034). The robot body (1031) is equipped with actuators such as a camera, radar, microphone, robotic arm, chassis, IMU, and gimbal.

[0038] The backend management system layer (104) includes a plugin repository (1041), a user and device database (1042), and an operation and maintenance console (1043).

[0039] Example 2: Actions and control relationships at different levels within the system

[0040] Combination Figure 4 The workflow and information exchange between different levels are as follows:

[0041] The user terminal layer (101) sends a login request to the unified communication gateway layer (102), obtains the device list, and issues control commands. After receiving the commands, the unified communication gateway layer (102) forwards the control message to the robot device layer (103) according to the permission verification. After execution, the robot device layer (103) reports the execution result and real-time status to the unified communication gateway layer (102). The unified communication gateway layer (102) then broadcasts the status update, realizing multi-terminal synchronous display of the user terminal layer (101). In addition, the background management system layer (104) issues and dynamically loads protocol plugins to the unified communication gateway layer (102). At the same time, the background management system layer (104) issues new function plugins to the robot device layer (103), which completes the signature verification and loads and executes them.

[0042] Example 3: Differential Data Push Mechanism

[0043] In order to reduce the pressure of high bandwidth concurrency during communication, when the robot device layer (103) continuously reports the status, the unified communication gateway layer (102) does not send the full data every time, but calculates the difference data between the current status and the system status snapshot. The difference data is encoded in the JSON Patch standard format and includes operation type, JSON path and new value fields. The difference data is only pushed to the user terminal layer (101) for the terminal to restore the data.

[0044] Example 4: Hot-loading mechanism for protocols and functions

[0045] Combination Figure 2 This embodiment details the interactive process of breakpoint-free hot loading.

[0046] The unified communication gateway layer (102) sends a plugin upload request to the backend management system layer (104) (S201) and performs plugin file transfer (S202); the backend management system layer (104) initiates signature verification and loading execution to the robot device layer (103) (S203), as well as integrity verification (S204); the unified communication gateway layer (102) initiates a plugin information query to the backend management system layer (104) (S205), and the backend management system layer (104) returns metadata (S206); subsequently, the unified communication gateway... Layer (102) sends a plugin download request (S207), and the background management system layer (104) downloads the file to the robot device layer (103) (S208); the unified communication gateway layer (102) internally performs memory-level hot loading (S209), dynamic invocation and loading of protocol plugins (S210), and sends an activation notification to the robot device layer (103) (S211); the robot device layer (103) performs robot loading (S212), and returns a ready confirmation to the unified communication gateway layer (102) after completion (S213). Subsequently, the unified communication gateway layer (102) updates the capability registry and broadcasts it to the user terminal layer (101) so that the UI interface of the heterogeneous terminal refreshes and displays the new action plugin in real time.

[0047] Example 5: DDS QoS Adaptive Adjustment

[0048] During system operation, the DDS bridging module in the unified communication gateway layer (102) monitors network link quality in real time. When the packet loss rate exceeds 5% for five consecutive sampling periods or the detected RTT exceeds 80% of the time limit set in Deadline QoS, the DDS Reliability parameter is dynamically adjusted from BEST_EFFORT to RELIABLE, and the History QoS cache depth is dynamically increased. Experimental data show that, using the differential data push and DDS QoS adjustment mechanism of this invention, network bandwidth usage is reduced by about 40% under complex outdoor conditions, and the average control command response delay is shortened by 45ms, effectively improving the real-time performance of the system.

[0049] Example 6: Method for Disconnection Detection and Deduplication / Reconnection (combined with...) Figure 1 )

[0050] Combination Figure 1 When the system encounters a network anomaly, the following complete procedure will be executed:

[0051] 1) Network disconnection detection and caching (S301-S3022):

[0052] The system triggers a network disconnection detection (S301), specifically performs a heartbeat packet timeout detection (S3011), records the disconnection timestamp (S3012), and starts a reconnection timer (S3013). During this period, instructions generated by the terminal enter the offline instruction cache (S302). The unified communication gateway layer (102) generates a unique identifier for the instruction and stores it in the LRU cache queue (S3021), and returns a cache confirmation (S3022).

[0053] 2) Network recovery and identity re-verification (S303-S3041):

[0054] After triggering network recovery detection (S303), the process of rebuilding the DDS bridge connection, verifying the heartbeat response record and recovery timestamp is performed (S3031). Subsequently, identity re-authentication and session recovery are performed (S304), specifically performing session ID verification, JWT Token validity verification and restoration of subscription relationship (S3041).

[0055] 3) Command deduplication and resumption of execution (S305-S3061): including the following steps: S305 read command, S3051 timestamp filtering, S306 idempotency verification and S3061 command interception window management.

[0056] Steps S305-S3051: The unified communication gateway layer (102) reads the offline cache instructions (S305), performs preliminary filtering based on the timestamp, removes expired instructions that have expired, and reports preliminary execution feedback (S3051).

[0057] Steps S306-S3061: Proceed to idempotency verification and deduplication (S306). The unique identifier is generated using a combination of timestamp (millisecond level) + client UUID + incrementing sequence number. The unified communication gateway layer (102) searches the cache window based on this identifier. If it is determined that the instruction was executed before the disconnection, it is directly intercepted and discarded; if it was not executed, it is sent to the machine entity (1031) for execution in sequence (S3061).

[0058] Example 7: Concurrent Arbitration and Interruption Mechanism

[0059] For multi-terminal concurrent control, the unified communication gateway layer (102) is configured with a priority arbitration mechanism based on "permission priority + timestamp sorting". The priority can be divided into multiple levels, such as emergency level (can interrupt all tasks but cannot be interrupted itself), high priority, medium priority and low priority.

[0060] When a high-priority instruction arrives, the unified communication gateway layer (102) determines the interrupt type of the currently executing task:

[0061] 1) If the task is resumable (such as patrolling or path following), the system serializes the execution context of the current task (including instruction ID, parameters, execution progress, timestamp, etc.) and stores it in persistent storage (such as Redis or a local file) and suspends it. At the same time, it sends a "pause" instruction to the robot device layer (103). After the high-priority instruction is completed, the system automatically checks the state stack, resumes the execution of the suspended task, and continues from the saved interrupt point. If the suspended task is not resumed after the preset timeout period, it is automatically marked as expired and discarded.

[0062] 2) If it is an unrecoverable task (such as charging, calibration and other safety-critical tasks), it will terminate directly after being interrupted, and the status will be marked as canceled. Recovery is not supported.

[0063] Example 8: Dual-channel communication mechanism

[0064] The system also includes a dual-channel communication mechanism, specifically: a control channel, which uses WebSocket or MQTT long connection to transmit control commands, heartbeats and status, with latency controlled within 100ms; and a media channel, which transmits audio and video streams based on WebRTC. The media channel and the control channel are physically separated to avoid network bandwidth contention.

Claims

1. A multi-terminal robot real-time communication system based on ROS2 architecture, characterized in that, It includes a user terminal layer, a unified communication gateway layer, a robot device layer, and a back-end management system layer that are interconnected; The user terminal layer includes multiple heterogeneous terminals, which establish communication links with the unified communication gateway layer and send data packets containing control commands. The unified communication gateway layer includes a protocol plugin manager. In response to the communication protocol type of the heterogeneous terminal, the protocol plugin manager dynamically calls and loads the matching communication protocol plugin, and converts and routes the control commands to the robot device layer. The robot device layer receives and executes the control commands, and reports the execution results and sensor status data to the unified communication gateway layer. The backend management system layer includes a plugin repository. The backend management system layer establishes a push link with the unified communication gateway layer or the robot device layer and distributes updated plugins from the plugin repository.

2. The system according to claim 1, characterized in that, The protocol plugin manager loads protocol plugins including MQTT, HTTPS, WebSocket PP, and WeChat JS-SDK.

3. The system according to claim 1, characterized in that, The unified communication gateway layer also includes: The protocol conversion module performs message format reassembly between different protocol levels; The authentication service extracts and performs time-validation verification on the access token at the user terminal layer. A caching mechanism stores unsuccessfully sent commands in a waiting queue during network fluctuations.

4. The system according to claim 1, characterized in that, The robot equipment layer includes: The machine entity is equipped with at least one actuator and sensor selected from cameras, radar, robotic arms, chassis, and IMU; The ROS2 module establishes distributed communication between internal robot nodes through a publish / subscribe mechanism.

5. The system according to claim 1, characterized in that, The backend management system layer invokes the reflection mechanism of the protocol plugin from the plugin repository through the operation and maintenance console, and pushes it to the unified communication gateway layer for memory-level hot loading.

6. A real-time communication method for multi-terminal robots based on the system described in any one of claims 1-5, characterized in that, include: When the robot device layer reports its status, the unified communication gateway layer calculates the difference between the current status and the status snapshot, encodes the difference data according to the JSON Patch format, and pushes the difference data to the user terminal layer. When the user terminal layer issues a command, the unified communication gateway layer extracts verification information through the authentication service to verify the identity, and then forwards the command after performing idempotency verification through the message assurance mechanism of the unified communication gateway layer.

7. The method according to claim 6, characterized in that, The process of issuing instructions from the user terminal layer also includes: If a network disconnection is detected, the unified communication gateway layer generates a unique identifier for the current instruction and stores it in a cache queue. The unique identifier is generated by a combination of timestamp, client UUID and incrementing sequence number. After the network is restored and the session is restored, the unified communication gateway layer performs deduplication based on the unique identifier and forwards the instructions in the cache queue in timestamp order.

8. The method according to claim 6, characterized in that, The unified communication gateway layer executes a priority arbitration mechanism; when a high-priority control instruction is received, the currently executing low-priority task is suspended, and the execution context of the low-priority task is stored in persistent storage. After the high-priority instruction is completed, the low-priority task is resumed according to the execution context.

9. The method according to claim 6, characterized in that, A DDS QoS adaptive adjustment mechanism is configured between the robot device layer and the unified communication gateway layer. If the packet loss rate exceeds a preset threshold within a set sampling period, the reliability QoS policy is upgraded to reliable transmission, and the cache depth of History QoS is dynamically increased. If the round-trip delay exceeds a preset proportion within a set period, the data release frequency is reduced.

10. The system according to claim 1 or the method according to claim 6, characterized in that, The unified communication gateway layer employs a dual-channel communication mechanism with each layer, including: The control channel uses WebSocket or MQTT long-lived connections to transmit control commands, heartbeats, and status data. The media channel transmits audio and video streams based on WebRTC, and the media channel and the control channel are physically separated in network transmission.

Images