A robotic teleoperation communication system and a robot

Abstract

The embodiment of the present application provides a robot teleoperation communication system and a robot. The system comprises: a robot end comprising a control module of a robot body, a first WebRTC client module, a signaling service module and a network penetration service module; the first WebRTC client module is in signal connection with the control module; a control end for controlling the robot body and / or the control module, comprising a second WebRTC client module; the signaling service module is used for realizing data interaction between the WebRTC client modules; the network penetration service module is used for providing a candidate address to the second WebRTC client module; and the second WebRTC client module is used for establishing a data transmission channel between the first WebRTC client module. Since the signaling service module and the network penetration service module are integrated in the robot end, and an intermediate server does not need to be additionally deployed, the flexibility and reliability of the system can be improved.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to robot teleoperation communication systems and robots. Background Technology

[0002] With the rapid development of robotics technology, robot teleoperation systems have been widely used in industrial automation, medical robotics, and autonomous driving. The core of teleoperation lies in achieving efficient communication between the operator (e.g., the control unit) and the operated end (e.g., the robot itself). This communication typically requires support for real-time data transmission and low-latency response. However, achieving effective communication connections often faces numerous challenges. For example, how to build a flexible and reliable robot teleoperation communication system has become a pressing technical issue that needs to be addressed in this industry.

[0003] Based on this, embodiments of this application provide a robot teleoperation communication system and a robot to improve related technologies. Utility Model Content

[0004] The purpose of this application is to provide a robot teleoperation communication system and a robot to improve the flexibility and reliability of the robot teleoperation communication system.

[0005] The objective of this application embodiment is achieved using the following technical solutions: In a first aspect, embodiments of this application provide a robot teleoperation communication system, comprising: a robot end, the robot end including a robot body control module, a first WebRTC client module, a signaling service module, and a network traversal service module; the first WebRTC client module is signal-connected to the control module; and a control end, the control end being used to control the robot body and / or the control module, the control end including a second WebRTC client module; wherein, the signaling service module is used to realize data interaction between the first WebRTC client module and the second WebRTC client module, the network traversal service module is used to provide candidate addresses to the second WebRTC client module, and the second WebRTC client module is used to establish a data transmission channel with the first WebRTC client module.

[0006] In some embodiments, the network traversal service module includes a STUN service submodule and / or a TURN service submodule, and the candidate address includes a first candidate address and / or a second candidate address; wherein, the STUN service submodule is used to provide the first candidate address to the second WebRTC client module; and the TURN service submodule is used to provide the second candidate address to the second WebRTC client module.

[0007] In some embodiments, the robot end further includes a network interface module, which is signal-connected to the control module; the network interface module has a fixed IP address.

[0008] In some embodiments, the network interface module is used to allocate temporary IP addresses to other terminal devices in a shared mode; the other terminal devices include the control terminal.

[0009] In some embodiments, the second WebRTC client module is further configured to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network.

[0010] Secondly, embodiments of this application provide a robot, including: a robot body, a control module of the robot body, a first WebRTC client module, a signaling service module, and a network traversal service module; the first WebRTC client module is signal-connected to the control module; wherein, the signaling service module is used to realize data interaction between the first WebRTC client module and a second WebRTC client module, the second WebRTC client module is used to control the robot body and / or the control module; the network traversal service module is used to provide candidate addresses to the second WebRTC client module, and the second WebRTC client module is also used to establish a data transmission channel with the first WebRTC client module.

[0011] In some embodiments, the network traversal service module includes a STUN service submodule and / or a TURN service submodule, and the candidate address includes a first candidate address and / or a second candidate address; wherein, the STUN service submodule is used to provide the first candidate address to the second WebRTC client module; and the TURN service submodule is used to provide the second candidate address to the second WebRTC client module.

[0012] In some embodiments, the robot further includes a network interface module, which is signal-connected to the control module; the network interface module has a fixed IP address.

[0013] In some embodiments, the network interface module is used to assign temporary IP addresses to other terminal devices in a shared mode.

[0014] In some embodiments, the second WebRTC client module is further configured to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network.

[0015] This application provides a robot teleoperation communication system and a robot. The robot teleoperation communication system includes: a robot end, which includes a robot body control module, a first WebRTC client module, a signaling service module, and a network penetration service module; the first WebRTC client module is signal-connected to the control module; and a control end, which is used to control the robot body and / or the control module, and includes a second WebRTC client module; wherein the signaling service module is used to realize data interaction between the first WebRTC client module and the second WebRTC client module, the network penetration service module is used to provide candidate addresses to the second WebRTC client module, and the second WebRTC client module is used to establish a data transmission channel with the first WebRTC client module. Because this application integrates the signaling service module and the network penetration service module into the robot end without the need for additional intermediate servers, it can improve data transmission efficiency, enhance system reliability, and this integrated design also allows the control end to perform teleoperation more flexibly. In other words, it can effectively improve the flexibility and reliability of the system. Attached Figure Description

[0016] The embodiments of this application are further described below with reference to the accompanying drawings and specific implementation details.

[0017] Figure 1 This is a schematic diagram of a robot teleoperation communication system provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram illustrating the construction process of a data transmission channel provided in an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of a robot end and a control end provided in an embodiment of this application.

[0020] Figure 4 This is a schematic diagram of a network interface module provided in an embodiment of this application.

[0021] Figure 5 This is a schematic diagram of a robot module provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0023] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0024] Related robots or robot teleoperation communication systems typically rely on WebRTC client modules to achieve P2P connections. WebRTC (Web Real-Time Communication) is a widely used real-time audio and video communication technology that enables WebRTC client modules to establish direct connections via P2P (Peer-to-Peer) methods. However, this process usually requires an additional central server (such as an intermediate server carrying signaling and STUN / TURN services), which leads to several technical drawbacks. First, the aforementioned additional servers or modules increase hardware costs and operational complexity, limiting the flexibility and deployability of the related robots or systems. Second, because WebRTC client modules need to manually modify relevant network configurations and specify the address of the server within the local area network when establishing P2P connections, this reduces the ease of use and portability of the related robots or systems. Third, in mobile, field, or temporary deployment scenarios, it may be difficult to conveniently provide the aforementioned additional intermediate servers, potentially limiting the application scope of the related robots or systems. Furthermore, the connection establishment process described above is highly dependent on additional intermediate servers, which threatens the reliability of the relevant robots or systems. For example, if a server fails, the entire connection may be interrupted.

[0025] Therefore, this embodiment integrates the signaling service module and the network penetration service module into the robot without requiring an additional intermediate server. This improves data transmission efficiency, enhances system security and reliability, and allows the robot to perform remote operations more flexibly. In other words, it effectively improves the system's flexibility and reliability.

[0026] Figure 1 This is a schematic diagram of a robot teleoperation communication system provided in an embodiment of this application.

[0027] To address the shortcomings in flexibility and reliability of related systems, this application proposes a robot teleoperation communication system. Please see [link / reference]. Figure 1 The system includes a robot terminal 100 and a control terminal 200. The robot terminal 100 includes a robot body control module 102, a first WebRTC client module 103, a signaling service module 104, and a network traversal service module 105. The first WebRTC client module 103 is signal-connected to the control module 102. The control terminal 200 is used to control the robot body and / or the control module 102, and includes a second WebRTC client module 201. The signaling service module 104 is used to implement data interaction between the first WebRTC client module 103 and the second WebRTC client module 201. The network traversal service module 105 is used to provide candidate addresses to the second WebRTC client module 201. The second WebRTC client module 201 is used to establish a data transmission channel with the first WebRTC client module 103.

[0028] It should be noted that the embodiments of this application do not involve any software and / or algorithm-level improvements to the control module, the first WebRTC client module, the signaling service module, the network traversal service module (hereinafter referred to as the STUN service submodule and / or TURN service submodule), and the second WebRTC client module.

[0029] In some embodiments, the robot body may be a physical entity of a robot. The robot body can be used to perform related tasks, such as moving or manipulating objects. The robot body can be managed and controlled by a control module.

[0030] In some embodiments, the control module of the robot body can be considered as the brain of the robot body. The control module can be used to process information from the control terminal. The control module can also be used to send relevant instructions to the robot body based on the above information.

[0031] In some embodiments, the first WebRTC client module can refer to a device or module used to implement the aforementioned WebRTC connection. The first WebRTC client module can be mounted on the robot body. The first WebRTC client module may include at least one hardware device or module selected from embedded computers, single-board computers, intelligent controllers, or other hardware devices or modules supporting WebRTC communication. The first WebRTC client module can be signal-connected to the control module to enable efficient data transmission between the two. The first WebRTC client module can be used to receive control commands from the control terminal. The first WebRTC client module can also be used to transmit the robot body's status information back to the control terminal.

[0032] In some embodiments, a signal connection can refer to a connection method used for transmitting signals between different devices or systems. This connection enables data communication, state synchronization, or command transmission. In specific implementations, a "signal connection" can be implemented using network protocols such as the WebSocket security protocol. The WebSocket security protocol allows for a continuous connection between the client (such as the aforementioned first WebRTC client module) and the server (such as the aforementioned signaling service module and network traversal service module), enabling real-time bidirectional data streaming and thus efficiently transmitting signaling information.

[0033] In some embodiments, the signaling service module can be signal-connected to both the first and second WebRTC client modules. The signaling service module can support data exchange between the first and second WebRTC client modules and complete signaling negotiation during the establishment of a data transmission channel. Specifically, when the second WebRTC client module needs to establish the aforementioned data transmission channel with the first WebRTC client module, the second WebRTC client module can initiate a request through the signaling service module and exchange Session Description Information (SDP). The SDP can describe the format of the media session, such as audio / video codecs, transport protocols, and media formats. The SDP helps both communicating parties understand how to negotiate and transmit data.

[0034] Understandably, the main function of the signaling service module is to enable effective data interaction between the first WebRTC client module and the second WebRTC client module. For example, the signaling service module can establish the aforementioned data transmission channel by sending Session Description Information (SDP) to the two clients (such as the first WebRTC client module and the second WebRTC client module).

[0035] In some embodiments, the network traversal service module can be signal-connected to a second WebRTC client module. The network traversal service module can provide network-layer support, such as providing candidate addresses for the second WebRTC client module. This is crucial for network obstacles such as Network Address Translation (NAT) environments and firewalls, enabling the relevant modules to successfully establish point-to-point connections. The candidate address can refer to an available network address. Candidate addresses can reflect the reachability of the relevant device or module within the network. Candidate addresses help ensure that two devices or modules can successfully establish a connection. Through these candidate addresses, devices on the Internet can identify each other, thereby enabling data transmission and overcoming the limitations of NAT and firewalls, enhancing the reliability and flexibility of the connection. In related robot or robot teleoperation communication systems, Session Description Information (SDP) and candidate addresses are important components for achieving stable and efficient connections.

[0036] It is understandable that candidate addresses refer to IP addresses used to determine and select the optimal data transmission channel when establishing the aforementioned data transmission channel. Candidate addresses can be obtained through network probing to ensure efficient and stable connections during communication. The purpose of candidate addresses is to allow WebRTC client modules (such as the first WebRTC client module and the second WebRTC client module) to attempt connection establishment based on these addresses when establishing a P2P connection.

[0037] Understandably, the main function of the network traversal service module is to provide candidate addresses to the second WebRTC client module, enabling connection traversal in different network environments to mitigate network restrictions caused by Network Address Translation (NAT) and firewalls. These candidate addresses help the second WebRTC client module locate the first WebRTC client module and establish a stable data transmission channel.

[0038] In some embodiments, the network traversal service module may be signal-connected to the signaling service module.

[0039] In some embodiments, the network traversal service module may also be signal-connected to the first WebRTC client module.

[0040] In some embodiments, the control terminal can refer to a device used for remotely controlling the robot body and / or control module. The main function of the control terminal is to communicate with the robot via a relevant network, thereby sending relevant data to the robot body and / or control module. The control terminal can also be used to receive status feedback from the robot body and / or control module. A second WebRTC client module is typically configured on the control terminal. The second WebRTC client module may include at least one of the following hardware devices or modules: an embedded computer, a single-board computer, an intelligent controller (such as a smartphone), or other hardware devices or modules that support WebRTC communication. Users can communicate in real-time with the first WebRTC client module through the second WebRTC client module to control the robot body and / or control module. For example, the second WebRTC client module can be a computer or mobile device with a display screen and input devices. Users can select tasks, such as moving, rotating, or performing specific operations, through a graphical interface on the second WebRTC client module. The second WebRTC client module can send relevant instructions to the first WebRTC client module on the robot in the WebRTC protocol format. The WebRTC protocol format is a standardized information structure used to define data transmission and connection establishment in WebRTC communication. For example, the WebRTC protocol format may include session description information, transmission configuration, and candidate addresses. Then, the control module receives the relevant instructions and drives the robot to perform the corresponding actions. This not only enables two-way communication between the control unit and the robot but also enhances the user's interaction experience with the robot, making remote control operation more intuitive and efficient.

[0041] In some embodiments, the robot teleoperation communication system can interact according to the following general process. For example, the control terminal first issues relevant control commands through a second WebRTC client module. Then, the control commands are transmitted to a first WebRTC client module via a signaling service module. The first WebRTC client module can then parse the control commands and cause the control module to perform corresponding operations. Simultaneously, the second WebRTC client module can obtain the required candidate addresses through a network penetration service module to ensure a stable network connection. The status information and feedback information of the robot body after performing relevant operations can be sent to the first WebRTC client module to form a complete closed loop. Through the above interaction, the system can achieve flexible robot teleoperation, making the operation process efficient, intuitive, and stable.

[0042] Figure 2 This is a schematic diagram illustrating the construction process of a data transmission channel provided in an embodiment of this application.

[0043] Please see Figure 2 , Figure 2 The "WebRTC Client" on the left could refer to the first WebRTC client module. Figure 2 The "WebRTC Client" on the right could refer to the second WebRTC client module. "Signaling Service" could refer to the signaling service module. "STUN / TURN" could refer to either the STUN service submodule or the TURN service submodule.

[0044] Please see Figure 2 "Signaling exchange" refers to the exchange of Session Description Information (SDP) between the first WebRTC client module and / or the second WebRTC client through the signaling service module. The Session Description Information can contain information about each WebRTC client, such as supported codecs, network information, and media formats. The purpose of exchanging Session Description Information is to ensure that the necessary parameters for establishing a connection between the first and second WebRTC client modules are consistent.

[0045] Please see Figure 2 "Acquiring candidate addresses" refers to the process by which a WebRTC client (such as a second WebRTC client) can acquire candidate addresses (such as ICE candidate addresses) through the STUN or TURN service submodules after the exchange of session description information. ICE candidate addresses are a series of potential network addresses collected during the Interactive Connection Establishment (ICE) process to establish peer-to-peer (P2P) connections between devices (such as WebRTC clients). ICE candidate addresses can include the device's (such as the WebRTC client's) local IP address, a public address obtained through the STUN service submodule, and a relay address obtained through the TURN service submodule. ICE candidate addresses help devices (such as WebRTC clients) find optimal communication paths in complex network environments. The process of acquiring candidate addresses can be performed with prior knowledge of SDP (Software as a Protocol Provider) to find available network paths in Network Address Translation (NAT) or firewall environments.

[0046] Please see Figure 2 "P2P" can refer to a direct connection established between the first WebRTC client module and the second WebRTC client module through a P2P (Peer-to-Peer) method.

[0047] Please see Figure 2Before establishing a P2P connection, two WebRTC clients (such as the first WebRTC client module and the second WebRTC client) can exchange Session Description Information (SDP) using the signaling service module located on the robot's end, and obtain candidate addresses using the STUN or TURN service submodule located on the robot's end, in order to effectively penetrate the Network Address Translation (NAT) environment. Afterwards, the two WebRTC clients can establish a P2P connection.

[0048] In some embodiments, the robot end may also include a robot body. For example, the robot body may be a remotely controlled robot with a camera, sensors, and a mobility device. The robot body can effectively receive control commands from the first WebRTC client module, while simultaneously transmitting the real-time video and sensor data it collects back to the control terminal. This design allows the robot to adjust its behavior in a timely manner through real-time data feedback while performing tasks, improving the flexibility and accuracy of its control.

[0049] As can be seen, this embodiment integrates the signaling service module and the network penetration service module on the robot end, and pairs them with the first WebRTC client module. This allows the first WebRTC client module and the second WebRTC client module to establish a direct connection, thereby improving network restrictions such as firewalls and Network Address Translation (NAT) to facilitate smooth data interaction and reduce reliance on additional intermediate servers. Because this embodiment integrates the signaling service module and the network penetration service module on the robot end, relevant data can be transmitted directly between the robot end and the control end via a P2P connection without going through an additional intermediate server. This improves data transmission efficiency, enhances system security and reliability, and this integrated design also allows the robot to perform remote operation more flexibly in various environments. In other words, it effectively improves the system's flexibility and reliability; furthermore, it mitigates the cost and maintenance issues associated with additional hardware (such as additional intermediate servers).

[0050] To address the issue of the difficulty in effectively establishing the aforementioned data transmission channels in complex network environments, in some embodiments, the network traversal service module may include a STUN service submodule and / or a TURN service submodule, and the candidate address may include a first candidate address and / or a second candidate address; wherein, the STUN service submodule can be used to provide the first candidate address to the second WebRTC client module; and the TURN service submodule can be used to provide the second candidate address to the second WebRTC client module.

[0051] In some embodiments, the primary function of the STUN (Session Traversal Utilities for NAT) service submodule is to provide a first candidate address to the second WebRTC client module. The first candidate address can be an IP address directly accessible on the local network, enabling the second WebRTC client to establish a fast connection without going through any additional intermediate servers. The TURN (Traversal Using Relays around NAT) service submodule can be used to provide a second candidate address to the second WebRTC client module. For example, the first candidate address provided by the STUN service submodule can be derived after NAT mapping. The first candidate address can be an address that allows devices on the network to find and connect to each other. For instance, when a direct P2P connection cannot be established, the TURN service submodule can relay data streams to enable communication between the control end and the robot end, thereby improving connection reliability. For example, when the control unit and the robot are located on different networks and direct connection is limited, the STUN service submodule can provide a first-choice address (such as a LAN address) to enable direct connection between the control unit and the robot. If the control unit and the robot cannot connect directly, the TURN service submodule can ensure smooth data flow through relay. This not only enhances the system's connection stability but also expands its applicability in different network environments.

[0052] Understandably, when the first WebRTC client module interacts with the second WebRTC client module, they can first use the signaling service module to exchange information such as candidate addresses. During this exchange, the first candidate address is usually an address that can be directly connected, while the second candidate address is a backup address used for relaying data when a direct connection is not feasible. By utilizing candidate addresses, various network environments and conditions can be flexibly addressed, thereby ensuring the establishment of data transmission channels as quickly and reliably as possible. This mechanism significantly improves the connection success rate in complex or restricted network environments.

[0053] As can be seen, this embodiment of the application introduces a STUN service submodule and / or a TURN service submodule to provide candidate addresses for the second WebRTC client (e.g., the STUN service submodule provides a first candidate address, and the TURN service submodule provides a second candidate address), in order to ensure that a data transmission channel can be established smoothly as much as possible. Since the STUN and TURN service submodules can work together, they can resolve network traversal issues in real time, ensuring that the data stream has multiple available paths, thus improving the stability and flexibility of the connection. In other words, it can improve the success rate and reliability of WebRTC connections, enabling effective connections even under poor network conditions.

[0054] To address the issue of the control terminal in the relevant system having difficulty quickly accessing the robot terminal, in some embodiments, the robot terminal may further include a network interface module, which can be signal-connected to the control module; the network interface module may have a fixed IP address.

[0055] In some embodiments, the network interface module can refer to a hardware or software component on the robot end used to achieve network connectivity. The network interface module can manage communication between the robot end and external devices (such as a control unit). The network interface module can be connected to the control module. The control module can process relevant instructions from the control unit and manage the robot to perform specific tasks. The network interface module enables network communication between the control module and the control unit. Through this connection, when the control unit sends relevant instructions via a fixed IP address, the network interface module can transmit the relevant instructions to the control module, which then executes the corresponding operation based on the instructions. Furthermore, the network interface module can also send feedback information from the control module to the control unit via the same network path, thereby achieving bidirectional communication. This interconnection ensures efficient operation of the robot end, allows real-time response to the control unit's needs, and improves the smoothness and reliability of teleoperation.

[0056] In some embodiments, both the control module and the first WebRTC client module may be signal-connected to the network interface module.

[0057] Understandably, the network interface module has a fixed IP address. This means that the network interface module is assigned an unchanging address within the network, allowing any device attempting to communicate with the robot to access that address. For example, assuming the robot's fixed IP address is set to 192.168.1.100, the control terminal only needs to input this fixed IP address to directly establish a connection with the robot's signaling service module and network traversal service module to exchange Session Description Information (SDP) and obtain candidate addresses (such as ICE candidate addresses), without requiring additional configuration of the second WebRTC client module. This design not only improves user convenience but also ensures the reliability and speed of data transmission, thereby achieving a smooth teleoperation experience.

[0058] As can be seen, this embodiment of the application enables direct access from the control end to the robot by setting a fixed IP address on the robot side and combining it with a network interface module. The control end can easily connect using only a fixed IP address without manually modifying the network configuration. Since a fixed IP address reduces reliance on dynamic IP addresses or additional configurations, it ensures connection consistency and convenience, thus improving the connection failure problem caused by network changes. In other words, it significantly simplifies the data transmission channel establishment process, allowing the control end to establish a P2P connection with the robot side more quickly and conveniently.

[0059] To address the issue of the robot in related systems struggling to provide convenient connection methods for multiple control terminals, in some embodiments, the network interface module can be used to allocate temporary IP addresses to other terminal devices in a shared mode; these other terminal devices may include the control terminal.

[0060] In some embodiments, the aforementioned sharing mode can refer to a state where the network interface module allows other terminal devices to connect through the local network and obtain temporary IP addresses. This configuration enables the robot to act as a "DHCP (Dynamic Host Configuration Protocol) server," automatically assigning temporary IP addresses to other terminal devices (such as the control unit) connected to its network, enabling them to communicate. DHCP (Dynamic Host Configuration Protocol) is a network management protocol used to automatically assign IP addresses and other related network configurations, such as subnet masks, gateways, and DNS servers, to devices connected to the network. When a device (such as a computer or mobile phone) connects to the network, it sends a request to the "DHCP server," which then assigns a temporary IP address to the device based on a preset address pool, ensuring efficient network management and effective resource utilization. This approach simplifies the network configuration process, reduces the frequency of manual IP address configuration, improves system flexibility and scalability, and allows connected devices to quickly come online and communicate.

[0061] Understandably, the purpose of the aforementioned sharing mode is to simplify the network configuration process on the control end, eliminating the need for manual IP address setting or further network adjustments, thus enhancing the system's flexibility and ease of use. For example, when the control end connects to the local area network generated by the robot, it will automatically obtain a temporary IP address, allowing it to directly exchange signaling and transmit data with the robot, thereby achieving a smooth remote operation experience. In this way, the network interface module not only improves the system's accessibility but also ensures that the control end can quickly integrate into the network environment.

[0062] In some embodiments, other terminal devices may also include at least one device or module selected from designated sensors, cameras, mobile terminals, computers, and other robots. Designated sensors may be used to collect environmental data, such as temperature sensors, humidity sensors, etc. Cameras may be used for monitoring and image capture. Mobile terminals, such as smartphones or tablets, may act as remote control terminals to control the robot. Multiple robots can collaborate via P2P connections.

[0063] As can be seen, the network interface module in this embodiment can dynamically allocate temporary IP addresses to other terminal devices (such as the control terminal) in shared mode, enabling these devices to communicate conveniently within the same local area network without requiring manual configuration of each device's network settings. Since the network interface module can automatically provide IP address allocation in shared mode, reducing manual intervention, it ensures smooth connection and effective communication of related devices (such as other terminal devices) within the network. In other words, it allows multiple related devices (such as other terminal devices) to easily connect and exchange data with the robot, thereby improving the system's flexibility and scalability.

[0064] To address the issue of the second WebRTC client in the relevant system having difficulty effectively establishing a stable data transmission channel with the robot, in some embodiments, the second WebRTC client module can also be used to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network.

[0065] Understandably, the feature that "the second WebRTC client module is also used to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network" allows the second WebRTC client module to connect to the first WebRTC client module more flexibly and conveniently without relying on a wired connection, thereby improving the system's adaptability and mobility. For example, in a robot operation scenario, the operator can connect the second WebRTC client module (such as a smartphone or tablet) to the robot's network interface module via Wi-Fi. Since the network interface module provides a fixed IP address, the user only needs to select and connect to the local area network corresponding to the robot on the second WebRTC client module to quickly and easily communicate with the robot. This fixed IP setting allows the control end to quickly identify and connect to the robot's signaling service module and network penetration service module, thereby successfully establishing a data transmission channel. Once the connection is successful, the second WebRTC client can establish a data transmission channel with the robot through this local area network, thereby enabling real-time control and data transmission. This wireless access method makes the connection between related devices more flexible, allowing users to easily control the robot from different locations without complicated settings, thus enhancing the convenience and smoothness of operation.

[0066] As can be seen, the second WebRTC client module in this embodiment can wirelessly access the local area network (LAN) corresponding to the network interface module and establish a data transmission channel using the LAN, enabling users to achieve flexible control and data interaction without relying on a wired connection. This is because wireless access to the LAN corresponding to the network interface module, using a fixed IP address, ensures that the second WebRTC client can accurately locate the robot (e.g., the second WebRTC client module, as a wireless access terminal, can quickly find and connect to the signaling service module of the robot using the network configuration within the LAN). Therefore, a stable data transmission channel can be established between the second WebRTC client module and the first WebRTC client module. In other words, it ensures smooth and stable data transmission while improving the convenience of the system, allowing users to use the robot in a wider range of environments.

[0067] As can be seen, the embodiments of this application can improve some limitations of related robot teleoperation communication systems in robot teleoperation scenarios, and provide an efficient, stable, and convenient communication solution that can be used for robot operation in complex environments. It not only has significant practical application value but also demonstrates broad market prospects, greatly improving the operational efficiency and flexibility of robots in various application scenarios.

[0068] Figure 3 This is a schematic diagram of a robot end and a control end provided in an embodiment of this application.

[0069] Please see Figure 3 In this application, "humanoid robot" refers to a robot terminal in one embodiment of the invention, and "VR" (such as a virtual reality device) refers to a control terminal in one embodiment of the invention. "Signaling / STUN" in the "humanoid robot" box refers to the signaling service module and the STUN service sub-module. "WebRTC Client" in the "humanoid robot" box refers to the first WebRTC client module. "WebRTC Client" in the "VR" box refers to the second WebRTC client module. "Signaling / wss" in the "humanoid robot" box refers to signaling exchange using the WebSocket security protocol (wss). The WebSocket security protocol (wss) allows a continuous connection to be maintained between the client (such as the aforementioned first WebRTC client module) and the server (such as the aforementioned signaling service module and network traversal service module), enabling real-time bidirectional data flow and thus efficiently transmitting signaling information.

[0070] Please see Figure 3"Control data / SCTP" refers to the control data transmitted between the first and second WebRTC client modules via SCTP (Stream Control Transmission Protocol). SCTP is a reliable transport layer protocol that supports the concurrent transmission of multiple data streams, thereby enabling efficient transmission of control commands and feedback information. In this way, the "humanoid robot" can receive control commands from "VR" and simultaneously return control status and related data to "VR." This two-way communication not only improves data transmission efficiency but also enhances the real-time performance and reliability of operations, ensuring a smoother control experience for users.

[0071] Please see Figure 3 "Audio / Video / RTP" refers to the audio and video data transmitted between the first and second WebRTC client modules using RTP (Real-time Transport Protocol). RTP is a protocol designed specifically for real-time data transmission and is widely used in audio and video communication to ensure the real-time nature and stability of data. This RTP protocol can transmit audio and video streams from a "humanoid robot" to a "VR," allowing users to receive real-time audio and video feedback from the "humanoid robot."

[0072] Figure 4 This is a schematic diagram of a network interface module provided in an embodiment of this application.

[0073] Please see Figure 4In this application, "robot" refers to a robot terminal in one embodiment of the invention, "shared network port" refers to a network interface module in a shared mode, and "VR" (such as a virtual reality device) refers to a control terminal in one embodiment of the invention. The "robot" can first be connected to the "router" via a "wired direct connection." The "WAN" within the "router" box refers to the WAN port. The router can connect to the service provider's network through the WAN port to provide internet access. A router is a network device used to connect different networks and forward data to enable data transmission between networks. WAN is an abbreviation for Wide Area Network. A WAN refers to a network covering a wide geographical area, typically connecting multiple local area networks. "VR wireless connection to router Wifi" means connecting the "VR" (such as a virtual reality device) signal to the router via a wireless network (WiFi). "VR wireless access to onboard services via robot fixed IP" means that the "VR" (such as a virtual reality device) can directly access the services provided by relevant modules on the robot terminal using the wireless network connection and the robot's fixed IP address. This approach reduces complex network configurations and improves the efficiency and convenience of interaction between "VR" (such as virtual reality devices) and "robots".

[0074] As can be seen, the embodiments of this application can reduce the dependence on additional intermediate servers and simplify the configuration of related client modules, thereby improving the flexibility and reliability of the system. The core of these embodiments lies in directly deploying the signaling service module and network traversal service module on the robot end, enabling it to perform the functions of both the WebRTC client module and the signaling service module and network traversal service module. In other words, the robot end can act as a WebRTC client to participate in the establishment of point-to-point data channels, and simultaneously assume the role of a signaling and STUN / TURN server. Furthermore, by appropriately configuring the robot end's network interface module, the control end can directly access the related services provided by the robot end through a fixed IP address without requiring additional modifications to the relevant programs. This design not only simplifies the connection process but also significantly improves the availability of the system.

[0075] As can be seen, the embodiments of this application can achieve direct point-to-point connection between the robot end and the control end without the need to deploy an additional intermediate server in the local area network or external network. The client (such as the first WebRTC client module) and the server (such as the signaling service module and the network penetration service module) can work together on the same device (such as the robot end) to complete the self-consistent direct connection establishment process.

[0076] The above describes a robot teleoperation communication system. This application also provides a robot. Figure 5 This is a schematic diagram of a robot module provided in an embodiment of this application.

[0077] Please see Figure 5 The robot includes: a robot body 101, a control module 102 for the robot body 101, a first WebRTC client module 103, a signaling service module 104, and a network traversal service module 105. The first WebRTC client module 103 is signal-connected to the control module 102. The signaling service module 104 is used to enable data interaction between the first WebRTC client module 103 and the second WebRTC client module 201. The second WebRTC client module 201 is used to control the robot body 101 and / or the control module 102. The network traversal service module 105 is used to provide candidate addresses to the second WebRTC client module 201. The second WebRTC client module 201 is also used to establish a data transmission channel with the first WebRTC client module 103.

[0078] In some embodiments, the second WebRTC client module can be used to control the robot body and / or the control module.

[0079] In some embodiments, the network traversal service module may include a STUN service submodule and / or a TURN service submodule, and the candidate address may include a first candidate address and a second candidate address. The STUN service submodule can be used to provide the first candidate address to the second WebRTC client module. The TURN service submodule can be used to provide the second candidate address to the second WebRTC client module.

[0080] In some embodiments, the robot may further include a network interface module, which may be signal-connected to the control module; the network interface module may have a fixed IP address.

[0081] In some embodiments, the network interface module can be used to assign temporary IP addresses to other terminal devices in a shared mode.

[0082] In some embodiments, the second WebRTC client module can also be used to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network.

[0083] The specific functions and effects achieved by the robot in this embodiment can be explained by referring to other embodiments of this application, and will not be repeated here.

[0084] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0085] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0086] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0087] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0088] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0089] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0090] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0092] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0093] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot teleoperation communication system, characterized in that, include: The robot terminal includes a robot body control module, a first WebRTC client module, a signaling service module, and a network penetration service module; the first WebRTC client module is signal-connected to the control module. The control terminal is used to manipulate the robot body and / or the control module, and the control terminal includes a second WebRTC client module; The signaling service module is used to realize data interaction between the first WebRTC client module and the second WebRTC client module, the network traversal service module is used to provide candidate addresses to the second WebRTC client module, and the second WebRTC client module is used to establish a data transmission channel with the first WebRTC client module.

2. The system according to claim 1, characterized in that, The network traversal service module includes a STUN service submodule and / or a TURN service submodule, and the candidate address includes a first candidate address and / or a second candidate address; wherein, the STUN service submodule is used to provide the first candidate address to the second WebRTC client module; and the TURN service submodule is used to provide the second candidate address to the second WebRTC client module.

3. The system according to claim 1, characterized in that, The robot terminal also includes a network interface module, which is signal-connected to the control module; the network interface module has a fixed IP address.

4. The system according to claim 3, characterized in that, The network interface module is used to allocate temporary IP addresses to other terminal devices in shared mode; the other terminal devices include the control terminal.

5. The system according to claim 3, characterized in that, The second WebRTC client module is also used to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network.

6. A robot, characterized in that, include: The robot body, the robot body's control module, a first WebRTC client module, a signaling service module, and a network penetration service module; the first WebRTC client module is signal-connected to the control module. The signaling service module is used to realize data interaction between the first WebRTC client module and the second WebRTC client module. The second WebRTC client module is used to control the robot body and / or the control module. The network traversal service module is used to provide candidate addresses to the second WebRTC client module. The second WebRTC client module is also used to establish a data transmission channel with the first WebRTC client module.

7. The robot according to claim 6, characterized in that, The network traversal service module includes a STUN service submodule and / or a TURN service submodule, and the candidate address includes a first candidate address and / or a second candidate address; wherein, the STUN service submodule is used to provide the first candidate address to the second WebRTC client module; and the TURN service submodule is used to provide the second candidate address to the second WebRTC client module.

8. The robot according to claim 6, characterized in that, The robot also includes a network interface module, which is signal-connected to the control module; the network interface module has a fixed IP address.

9. The robot according to claim 8, characterized in that, The network interface module is used to allocate temporary IP addresses to other terminal devices in shared mode.

10. The robot according to claim 8, characterized in that, The second WebRTC client module is also used to wirelessly access the local area network corresponding to the network interface module and establish the data transmission channel using the local area network.

Images