Robot system, communication method thereof, communication system, storage medium and product

Abstract

The invention provides a robot system, a communication method thereof, a communication system, a storage medium and a product. The method comprises the following steps: in response to a received interaction request sent by a service calling module, analyzing the interaction request through a service providing module, and determining an interaction instruction contained in the interaction request; under the condition that the interaction instruction is a control instruction, forwarding the control instruction to an action execution module through a service providing module; the message format of the control instruction is converted through an action execution module, an action instruction is obtained, and the message format of the action instruction meets the first message format in the first robot system; the action instruction is sent to a first execution mechanism of the first robot system through the action execution module, and the action instruction is used for controlling the first execution mechanism to move. The technical problem that the universality of the communication method of the robot system in the related technology is poor is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical fields of intelligent robots and communication, in particular to a robot system, a communication method thereof, a communication system, a storage medium and a product. BACKGROUND

[0002] Currently, a robot system can be controlled by an artificial intelligence model. However, in the process of interaction between the robot system and the artificial intelligence model, or in the process of building a new robot system to interact with the artificial intelligence, since different data formats are usually used for interaction by the robot system, in order to realize the interaction requirements of different robot systems, a developer needs to perform a large amount of debugging, or even modify the entire link. Therefore, the current communication method of the robot system has poor versatility.

[0003] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0004] Embodiments of the present application provide a robot system, a communication method thereof, a communication system, a storage medium and a product, to at least solve the technical problem of poor versatility of the communication method of the robot system in the related art.

[0005] According to an aspect of embodiments of the present application, a communication method of a robot system is provided, applied to a first robot system, a service providing module and an action executing module are deployed on the first robot system, the method comprising: in response to receiving an interaction request sent by a service calling module, analyzing the interaction request by the service providing module to determine an interaction instruction contained in the interaction request, wherein the service calling module is deployed in a control system; in the case that the interaction instruction is a control instruction, forwarding the control instruction to the action executing module by the service providing module; converting a message format of the control instruction by the action executing module to obtain an action instruction, wherein the message format of the action instruction satisfies a first message format inside the first robot system; sending the action instruction to a first executing mechanism of the first robot system by the action executing module, wherein the action instruction is used to control the first executing mechanism to move.

[0006] According to another aspect of the embodiments of this application, a communication system for a robot system is also provided, comprising: a service invocation module for sending an interaction request containing an interaction instruction; a service providing module, communicatively connected to the service invocation module, for parsing the interaction request to obtain the interaction instruction, and forwarding the interaction instruction if the interaction instruction is a control instruction; and an action execution module, communicatively connected to the service providing module, for converting the message format of the control instruction to obtain an action instruction, and sending the action instruction to a first actuator of the first robot system, wherein the message format of the action instruction satisfies a first message format, and the action instruction is used to control the movement of the first actuator of the first robot system.

[0007] According to another aspect of the embodiments of this application, a robot system is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, which includes a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute the methods in various embodiments of this application.

[0009] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0010] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program. When executed by a processor, the computer program implements the methods of various embodiments of this application.

[0011] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0012] In this embodiment, in response to receiving an interaction request from the service call module, the service provider module parses the interaction request to determine the interaction instructions contained within it. If the interaction instruction is a control instruction, the service provider module forwards the control instruction to the action execution module. The action execution module converts the message format of the control instruction to obtain the action instruction. The action execution module then sends the action instruction to the first actuator of the first robot system, thereby controlling the movement of the first actuator. In this embodiment, the first robot system can parse the interaction requests from the control system via the service call module, achieving standardization, consistency, and universality in external interactions. This ensures that even with model adjustments, the interaction requests sent by the control system can be accurately parsed. Furthermore, the first robot system can convert the message format of the control instructions via the action execution module, achieving standardization, consistency, and universality in interaction with the first robot system itself. This ensures that even if the first robot system itself, such as the actuator, changes, action instructions that can be accurately understood and executed by the actuator can be generated. This improves the universality of the robot system's communication method, thus solving the technical problem of poor universality in robot system communication methods in related technologies.

[0013] The above general description and the following detailed description are for illustrative and explanatory purposes only and do not constitute a limitation thereof. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0015] Figure 1 This is a structural block diagram of a robot system according to an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of the RCP architecture according to an embodiment of this application;

[0017] Figure 3 This is a flowchart of a communication method for a robot system according to an embodiment of this application;

[0018] Figure 4 This is a schematic diagram of a centralized collaboration mode according to an embodiment of this application;

[0019] Figure 5 This is a schematic diagram of a distributed collaboration mode according to an embodiment of this application;

[0020] Figure 6 This is a schematic diagram of the identity-aware and capability-aware architecture according to an embodiment of this application;

[0021] Figure 7 This is a schematic diagram of the communication system of a robot system according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some, not all, of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in other orders. Other orders herein refer to orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0025] RCP: Robotics Context Protocol.

[0026] MCP: Model Context Protocol.

[0027] VLA: Vision Language Action, is an artificial intelligence model that unifies visual perception, natural language understanding, and action control into a single framework.

[0028] LCM: Lightweight Communications and Marshalling, a lightweight data communication middleware.

[0029] ROS: Robot Operating System, is an open-source framework for robot software development.

[0030] According to an embodiment of this application, a communication method for a robot system is provided. The steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than that shown here.

[0031] The methods and embodiments provided in this application can be executed in a robot system. A robot system is a complex integration of multiple disciplines, including mechanics, electronics, control, computer science, and artificial intelligence, designed to achieve automated operation, environmental perception, information processing, and task execution. Figure 1 A structural block diagram of a robot system is shown. Figure 1 As shown, the robot system 10 includes at least: an execution structure 102, a sensor system 104, a control module 106, an energy system 108, and a communication system 110. The execution structure 102 is the physical component of the robot system, including the frame, joints, limbs, arms, etc., which determines the robot's shape, mobility, and operating range. The sensor system 104 contains various types of sensors, such as vision sensors (e.g., cameras), hearing sensors (e.g., microphones), tactile sensors, torque sensors, and positioning sensors, which can perceive environmental information and the robot's own state, providing data support for decision-making and control. The control module 106 can translate control commands into physical actions, such as motors, hydraulic systems, and servo systems. The energy system 108 includes batteries, power management systems, and charging systems, providing the necessary energy support for the robot system. The communication system 110 allows the robot system to exchange data with external systems.

[0032] In this embodiment, the communication method of the robot system mainly describes the method corresponding to the communication system 110. That is, other structural components besides the communication system can use commonly used components. The structure of the robot system can be adjusted according to actual needs, and may include not only the above-described structure, but also more or fewer structures than the above-described structure. Moreover, the above-described structure can also be modified according to actual needs.

[0033] In one alternative embodiment, Figure 1 The robot system shown can be any robot system in a multi-robot collaborative scenario. Multiple robot systems can communicate with each other to complete a task together, or if one robot system fails, other robot systems can take over and complete the subsequent task.

[0034] Currently, robot systems are deployed primarily using two approaches: First, deploying powerful VLA (Visual Analog Array) models in the cloud, outputting corresponding commands or action sequences, and transmitting them to the robot system via network for execution. In some models, deployments are separated between the cloud and edge sides depending on the system's speed. Second, deploying optimized models directly on the edge side, which can directly drive the robot system to perform corresponding tasks. This approach is mainly used in scenarios with high real-time requirements or privacy compliance. However, current model interaction with robot systems, or the construction of new robot systems, presents the following engineering challenges: for example, different robot systems typically use different data formats for interaction; the time developers spend debugging on the system significantly reduces the time available for developing new features; and when the model is adjusted, the entire pipeline may need to be modified.

[0035] In response, this application proposes an RCP architecture, which aims to solve the above problems in terms of the synergy between edge-cloud deployment and inference, the compatibility of different robot systems, the uniformity of robot system external capabilities, and the standardization of interaction between robot systems. This will make the entire robot system simple and clear, and related functions and modules reusable, making it easier for developers to build and deploy physical intelligence more quickly.

[0036] like Figure 2 As shown, the RCP architecture comprises four key modules: RCP Client (robot service invocation module), RCP Server (robot service provider module), Action Server (robot motion execution module), and Sensor Server (robot sensor information provider module). The RCP Client can interact with the RCP Server through various modalities. The RCP Server generally refers to the brain of the robot system, capable of human-robot interaction, task planning, and skill combination. The Action Server can communicate with RobotMotion (control module), which directly interacts with the mechanical structure of the robot system, thereby controlling the actuators and receiving their motion states, thus abstracting and standardizing the control and state of the actuators. The Sensor Server can communicate with SensorNodes within the robot system, integrating sensor information into interfaces and data for the model to use.

[0037] Most intelligent robot systems require not only powerful execution capabilities but also abundant perceptual information for reasoning and decision-making. Currently, VLA largely relies on 2D / 3D image information and ontological observation data. However, with the gradual development of world models and other technologies, video and audio may also become inference inputs. The primary function of a Sensor Server is to uniformly configure and transmit the sensors that generate this perceptual information.

[0038] Taking 2D image sensors as an example, in model inference scenarios, sensor configuration mainly focuses on common settings such as resolution, frame rate, exposure mode, and image format. The Sensor Server interacts with relevant sensor nodes through middleware within the on-device based on these common configurations. Regarding data transmission, image information can be compressed into byte arrays, similar to the robotic arm control / status data format mentioned above (only several orders of magnitude larger), and the overall protocol format does not require modification. In this way, the Sensor Server standardizes the configuration and data transmission of 2D image sensors; other types of sensors are handled similarly.

[0039] Within a robotic system, the roles extend beyond Action Server, Sensor Server, RobotMotion, and Sensor Node. In different scenarios, other modules may exist, such as edge-side perception modules. Furthermore, developers can easily and freely add custom modules and embed specific processing logic. While RCP is primarily designed for a holistic end-to-end or layered end-to-end architecture, its open-source nature and compatibility allow for easy modification and integration by users.

[0040] The RCP architecture also includes a Communication Layer module. The capabilities provided by the Action Server and Sensor Server, including control, status feedback, configuration, and sensor information transmission, all converge to two transmission protocols in the Communication Layer: JSON-RPC and Binary Serialize. Specifically, binary serialize can be used for data transmission of status feedback and sensor information, while JSON-RPC can be used for control and configuration capabilities.

[0041] The primary purpose of the Action Server and Sensor Server is to standardize hardware-related functions. These interfaces are relatively low-level and do not align with actual user interaction needs. The RCP Server, on the other hand, acts as the interface for calling the standard hardware interfaces and providing actual intelligent services. The RCP Server has four underlying modules: ActionClient (robot action invocation module), Sensor Client (robot sensor information invocation module), ModelService (model service module), and R2R Node (robot-to-robot interaction service module). The Action Server and Sensor Server interact with their respective Servers through the Communication Layer, exposing common functional interfaces and providing data transmission services to the Skill Layer. The Model Service provides corresponding model inference services and can be deployed on the edge or cloud, but the external interface remains consistent. The R2R Node provides inter-robot interaction capabilities. The underlying modules remain independent. Model developers only need to encapsulate the Model Service and specify data format requirements; the intermediate modules require minimal modification, only the edge SensorNode and RobotMotion need adaptation. The middle layer is the Skill Layer, which constructs Skill Units based on service requirements, functional interfaces, data, and model inference interfaces, and can build long-range tasks based on this. The upper layer is the Agent, which can combine LLM or other models with the interfaces provided by the Skill Layer to build complex functions for various interactions. In addition, the Skill Layer can also directly expose relevant service interfaces, which can be called by RCPClient.

[0042] With the above design, the modules are decoupled, and when facing different robots and different models, the other modules can remain almost unchanged, which makes the entire development process faster and more robust, and developers no longer need to do some repetitive work.

[0043] Under the aforementioned operating environment, this application provides the following: Figure 2 The method illustrates a communication method for a robot system. This method is applied to a first robot system, on which a service providing module and an action execution module are deployed.

[0044] Optionally, the first robot system mentioned above can be a robot system in a single robot scenario or any robot system in a multi-robot collaborative scenario, working collaboratively with other robot systems in the same trusted shared domain.

[0045] To achieve universality in communication between robot systems and between robot systems and models, a service-providing module and an action execution module can be deployed in the first robot system. The service-providing module can be a module that interacts with the first robot system from the outside, providing an interface for actual tasks. For example, the service-providing module could be... Figure 2 The RCP Server is shown. The action execution module can be a module within the first robot system that interacts with the robot system's execution structure. For example, the action execution module can be like... Figure 2 The Action Server is shown. By deploying the service provision module, standardization of external interactions can be achieved, while by deploying the action execution module, standardization of hardware functions can be achieved.

[0046] Figure 3 This is a flowchart of a communication method for a robot system according to an embodiment of this application. Figure 3 The method may include the following steps:

[0047] Step S302: In response to receiving the interaction request sent by the service call module, the service provider module parses the interaction request to determine the interaction instructions contained in the interaction request. The service call module is deployed in the control system.

[0048] The service invocation module mentioned above can be a module used for interaction and task initiation. For example, the service invocation module can be like... Figure 2 The RCP Client is shown. To enable communication between the first robot system and the control system (such as a model), the service call module can be deployed in the control system.

[0049] The aforementioned interactive instructions can be commands sent by the service invocation module to the first robot system. By sending these interactive instructions, the service invocation module can control the first robot system and obtain its status and sensor information. To send the interactive instructions to the first robot system, the service invocation module can encapsulate the instructions according to a communication protocol, obtain an interaction request, and then transmit the interaction request to the service provider module of the first robot system.

[0050] In an optional embodiment, when the control system needs to control the first robot system, the control system can use a service call module to encapsulate the interaction instructions using a pre-agreed communication protocol to obtain the corresponding interaction request, and then transmit the interaction request to the service provider module of the first robot system. After receiving the interaction request, the service provider module can use the aforementioned communication protocol to decapsulate the interaction request to obtain the interaction instructions carried in the interaction request.

[0051] Step S304: If the interaction command is a control command, the control command is forwarded to the action execution module through the service providing module.

[0052] The aforementioned control instructions can be instructions for controlling the operation of the first robot, specifically including actions that can be performed by different actuators in the first robot system.

[0053] In one optional embodiment, after determining the interaction instruction, the service providing module can identify the interaction instruction and determine its specific type. If the interaction instruction is a control instruction, it means that the control system needs to control the first robot. Therefore, the service providing module can forward the control instruction to the action execution module, which will process it and send it to the actuator.

[0054] Step S306: The message format of the control command is converted by the action execution module to obtain the action command, wherein the message format of the action command satisfies the first message format inside the first robot system.

[0055] The aforementioned first message format can be the message format corresponding to the environment of the first robot system itself. For example, the first message format can be the message format corresponding to the actuator. For another example, for something like... Figure 2 In the RCP architecture shown, the action execution module does not communicate directly with the execution mechanism. The first message format can be the common message format of the middleware LCM and ROS.

[0056] In an optional embodiment, considering that the message format used by the control system may be different from the first message format inside the first robot system, in order to improve the versatility of the first robot system, the control instructions can be converted into message formats by the action execution module to obtain action instructions that meet the first message format.

[0057] Step S308: The motion execution module sends the motion command to the first actuator of the first robot system, wherein the motion command is used to control the movement of the first actuator.

[0058] The aforementioned first actuator can be the actuator that the action command needs to control. For example, if the action command is to control the movement of a robotic arm, the first actuator can be the robotic arm. Or, if the action command is to control the movement of a leg, the first actuator can be the leg. Or, if the action command is to control the rotation of the upper body, the first actuator can be the upper body.

[0059] In one optional embodiment, if the message format of the first actuator is a first message format, indicating that the first actuator can understand and accurately execute the action instruction, the action execution module can directly send the action instruction to the first actuator to control its movement. In another optional embodiment, the action execution module can use widely used middleware to convert the action instruction satisfying the first message format into an instruction that the first actuator can understand and accurately execute, and then forward the converted instruction to the first actuator to control its movement.

[0060] In this embodiment, in response to receiving an interaction request from the service call module, the service provider module parses the interaction request to determine the interaction instructions contained within it. If the interaction instruction is a control instruction, the service provider module forwards the control instruction to the action execution module. The action execution module converts the message format of the control instruction to obtain the action instruction. The action execution module then sends the action instruction to the first actuator of the first robot system, thereby controlling the movement of the first actuator. Through this scheme, the first robot system can parse interaction requests from the control system via the service call module, achieving standardization, consistency, and universality in external interactions. This ensures that even with model adjustments, the interaction requests sent by the control system can be accurately parsed. Furthermore, the first robot system can convert the message format of the control instruction via the action execution module, achieving standardization, consistency, and universality in interaction with the first robot system itself. This ensures that even if the first robot system itself, such as the actuator, changes, action instructions that can be accurately understood and executed by the actuator can be generated. This improves the universality of the robot system's communication method and solves the technical problem of poor universality in robot system communication methods in related technologies.

[0061] In the above embodiments of this application, parsing the interaction request to determine the interaction instructions contained in the interaction request includes: parsing the interaction request using a preset communication protocol to obtain the interaction instructions, wherein the preset communication protocol is used to transmit information between the service providing module and the service calling module.

[0062] The aforementioned preset communication protocol can be a communication protocol agreed upon between the control system and the first robot system. In order to achieve communication universality, the preset communication protocol can be a general protocol, such as the RCP protocol.

[0063] In one optional embodiment, when the interaction command is transmitted between the service providing module and the service invoking module, the service invoking module encapsulates the interaction command using a preset communication protocol to obtain the interaction request. Therefore, the service providing module can decapsulate the interaction request using the same preset communication protocol to obtain the aforementioned interaction command.

[0064] In the above embodiments of this application, forwarding control instructions to the action execution module through the service providing module includes: determining the first interface protocol corresponding to the action execution module through the skill layer module in the service providing module; and forwarding the control instructions to the action execution module through the action invocation module in the service providing module using the first interface protocol.

[0065] The aforementioned first interface protocol can be an interface protocol for data transmission between the service provider module and the action invocation module. For example, the first interface protocol could be JSON-RPC. JSON-RPC is a relatively simple protocol that not only possesses characteristics such as statelessness, wide usage, ease of parsing, and readability, but also cross-language and cross-platform compatibility, and a clear request-response pattern. This allows clients / servers to use it in common languages ​​or frameworks, and it is also easy to extend, making it suitable for interaction with intelligent applications. The main content transmitted in JSON-RPC includes device control, configuration, and corresponding responses.

[0066] The action invocation module described above is the module within the service provider module that interacts with the action execution module. For example, the action invocation module could be like this: Figure 2 The Action Client shown.

[0067] Since the action execution module exposes a common functional interface to the skill layer module and provides data transmission services, in an optional embodiment, the skill layer module can determine the first interface protocol used by the action execution module, and then use the first interface protocol to encapsulate the control instructions. The encapsulated instructions can then be sent to the action execution module through the action invocation module, so that the action execution module can use the first interface protocol to decapsulate and obtain the control instructions.

[0068] In the above embodiments of this application, the control instructions include: a first action sequence, wherein information in different rows of the first action sequence corresponds to different actuators of the first robot system, and information in different columns of the first action sequence corresponds to different states; converting the message format of the control instructions to obtain action instructions includes: parsing the first action sequence using a first interface protocol to determine the first actuator in the first robot system and the target state corresponding to the first actuator, wherein the first interface protocol is used to transmit messages between the service providing module and the action execution module; and converting the target state using a first message format to obtain the action instructions corresponding to the first actuator.

[0069] Currently, robot systems come in various forms, and it will take time to reach a unified form. These robot systems can achieve specific functions or certain general capabilities in some areas. Therefore, the executable actions and states of robot systems can be abstracted to form a unified interface protocol. For example, actions and states can be abstracted into a two-dimensional matrix, where each row represents multiple states of an actuator, and different states are stored in elements of different columns, so that the number of rows in the two-dimensional matrix is ​​consistent with the number of actuators. Furthermore, since the control of robot systems is often continuous, control commands can be action chunks, requiring the addition of a time dimension to the two-dimensional matrix.

[0070] The aforementioned target state can be the state of the first actuator after it has run according to the action command; that is, the state that the control system needs to control the first actuator to achieve.

[0071] In one optional embodiment, the action execution module can use a first interface protocol to parse the control instructions, i.e., the first action sequence, transmitted by the service provider module, thereby identifying the first actuator that the control system needs to control, and the target state that the first actuator needs to achieve. Then, the target state can be format-converted using a first message format to obtain the action instruction corresponding to the first actuator; that is, the message format of the action instruction is the first message format.

[0072] In the above embodiments of this application, when the actuator is a limb joint of the first robot system, the information in the first row corresponding to the limb joint in the first action sequence includes at least: pose, angle, and action; when the actuator is a torso joint of the first robot system, the information in the second row corresponding to the limb joint in the first action sequence includes at least: distance traveled, speed, and direction.

[0073] Currently, the control of robotic arms in robot systems can be broadly divided into two categories: "end-effector pose and gripper / dexterous hand movement" or "joint angle and gripper / dexterous hand movement". Regardless of the type, each line in the first action sequence represents the pose, angle, and hand movement information of a robotic arm, thus ensuring compatibility with robotic arms of different degrees of freedom and different end-effector forms.

[0074] For whole-body robotic systems, the control of the legs and torso is similar to that of the arms. Only the matrix needs to be expanded to correspond with the degrees of freedom of the limbs, and the entire data structure can remain consistent. For systems where the upper and lower body are controlled independently, such as those using high-frequency actuation of leg joints via reinforcement learning models deployed on the edge, or wheeled chassis driven by PID (Proportional-Integral-Derivative Controller) models, the movement distance, speed, and direction are exposed to the RCP Server, without needing to provide the underlying control interfaces of the joints or drive wheels to the upper layer.

[0075] In the above embodiments of this application, sending an action instruction to the first actuator of the first robot system through the action execution module includes: sending the action instruction to the first actuator when the message format of the first actuator is a first message format; and sending the action instruction to a first middleware when the message format of the first actuator is not the first message format, and forwarding the action instruction to the first actuator through the first middleware, wherein the message format of the first middleware is the first message format, and the first middleware is used to perform format conversion on the action instruction.

[0076] The aforementioned first middleware can be the widely used middleware LCM and ROS, enabling communication between the motion execution module and the first actuator. Therefore, hardware vendors can refer to the data formats of LCM and ROS to build corresponding SDKs (Software Development Kits), allowing specific devices to be integrated into the entire intelligent robot system.

[0077] In one optional embodiment, the actuator of the robot system can be provided by the hardware supplier. The message format corresponding to the actuator may be different. Therefore, if the message format of the first actuator is a first message format, the action command can be directly sent to the first actuator. However, if the message format of the first actuator is not a first message format, the action command can be forwarded to the first actuator through a first middleware. In this process, the first middleware can convert the format of the action command, changing the message format of the action command from the first message format to the message format of the first actuator.

[0078] Preferably, the action execution module and the first execution structure do not interact directly, but instead forward information through the first middleware.

[0079] Through the above embodiments, the action execution module can parse the received action sequence into the message format corresponding to the ontology environment and send it to the first execution mechanism, thereby achieving the function of controlling the joints and other execution mechanisms. At the same time, it can receive the motion state of each joint and other execution mechanism and integrate it into a general message type for interaction with the model, thereby abstracting and standardizing the control and state of the ontology joints and other execution structures.

[0080] In the above embodiments of this application, the first robot system is further equipped with an information providing module. The method further includes: when the interaction instruction is an information reporting instruction, forwarding the information reporting instruction to the action execution module and the information providing module through the service providing module; collecting the status information of different actuators of the first robot system based on the information reporting instruction through the action execution module to obtain a second action sequence, and sending the second action sequence to the service providing module; collecting sensor information of different sensors of the first robot system based on the information reporting instruction through the information providing module, and sending the sensor information to the service providing module; and sending the second action sequence and sensor information to the service calling module through the service providing module.

[0081] The aforementioned information providing module can be an internal module of the first robot system that interacts with the robot system's sensor system. For example, the information providing module can be such as... Figure 2 The Sensor Server shown can also achieve hardware standardization by deploying an information provision module.

[0082] In one optional embodiment, after determining the interaction instruction, the service providing module can identify the interaction instruction and determine its specific type. If the interaction instruction is an information reporting instruction, it means that the control system needs to obtain the current state of the first robot system and the environmental information collected by the sensor system. Therefore, the service providing module can forward the information reporting instruction to the action execution module and the information providing module.

[0083] For the action execution module, the message format of the information reporting command is converted to obtain a status command. This status command is then sent to different execution mechanisms, which provide status information in return. This process collects the status information from different execution mechanisms, generates a second action sequence, and returns the second action sequence to the service provider module. The processing details of both information reporting commands and interactive commands are detailed and will not be elaborated upon here.

[0084] For the information providing module, the message format of the information reporting instruction can be converted to obtain the reporting instruction, and the reporting instruction can be sent to different sensors. The different sensors will then provide feedback on their status information, thereby collecting sensor information from different sensors and returning the sensor information from different sensors to the service providing module.

[0085] The service provider module can aggregate the second action sequence and sensor information, generate a response result, and return it to the service invocation module. Optionally, the service provider module can encapsulate the response result using a preset communication protocol, obtain an information reporting response, and send it to the service invocation module.

[0086] In the above embodiments of this application, forwarding the information reporting instruction to the action execution module and the information providing module includes: determining the first interface protocol corresponding to the action execution module and the second interface protocol corresponding to the information providing module through the skill layer module in the service providing module; forwarding the information reporting instruction to the action execution module based on the first interface protocol through the action invocation module in the service providing module; and sending the information reporting instruction to the information providing module based on the second interface protocol through the information invocation module in the service providing module.

[0087] The aforementioned second interface protocol can be an interface protocol for data transmission between the service provider module and the information provider module. For example, the second interface protocol can be binary serialization. Compared to JSON, binary serialization has advantages including smaller data size after serialization for the same transmitted content, and better efficiency in data packaging and parsing, especially in scenarios involving large amounts of sensor data transmission. However, its disadvantage is that both parties must process messages in a unified, predetermined format. Since we are targeting a wide range of terminal devices with varying performance and need to transmit relevant data frequently in different communication environments to ensure smooth inference, we choose binary serialization to transmit sensor data. Furthermore, the communication protocol should be designed to be as universal as possible from the initial design stage to avoid frequent modifications.

[0088] The aforementioned information retrieval module is the module within the service provider module that interacts with the information provider module. For example, the information retrieval module could be as follows: Figure 2 The Sensor Client shown.

[0089] Since the action execution module and the information providing module expose common functional interfaces to the skill layer module and provide data transmission services, in an optional embodiment, the skill layer module can determine the first interface protocol used by the action execution module, and then use the first interface protocol to encapsulate the information reporting instructions, and send the encapsulated instructions to the action execution module through the action calling module, so that the action execution module can collect the status information of different actuators based on the information reporting instructions.

[0090] The second interface protocol used by the information providing module can be determined through the skill layer module. Then, the information reporting command can be encapsulated using the second interface protocol, and the encapsulated command can be sent to the information providing module through the information calling module, so that the information providing module can collect sensor information from different sensors based on the information reporting command.

[0091] In the above embodiments of this application, collecting sensor information from different sensors of the first robot system includes: parsing the information reporting instruction based on the second interface protocol to determine the target sensor in the first robot system, wherein the second interface protocol is used to transmit messages between the service providing module and the information providing module; converting the information reporting instruction according to the second message format to obtain the perception instruction corresponding to the target sensor; sending the perception instruction to the target sensor and receiving the sensor information returned by the target sensor.

[0092] The second interface protocol described above is the same as the second interface protocol in the previous embodiment, and will not be repeated here.

[0093] In one optional embodiment, the information providing module can use a second interface protocol to parse the information reporting instructions transmitted by the service providing module, thereby identifying the target sensor for which the control system needs to report information. Then, the information reporting instructions can be format-converted using a second message format to obtain the sensing instructions corresponding to the target sensor; that is, the message format of the sensing instructions is the second message format. Finally, the sensing instructions are sent to the target sensor, which then returns the collected sensor information to the information providing module.

[0094] In the above embodiments of this application, sending a sensing instruction to a target sensor and receiving sensor information returned by the target sensor includes: when the message format of the target sensor is a second message format, sending the sensing instruction to the target sensor and receiving the sensor information returned by the target sensor; when the message format of the target sensor is not the second message format, sending the sensing instruction to a second middleware, forwarding the sensing instruction to the target sensor through the second middleware, and forwarding the sensor information returned by the target sensor to the service providing module through the second middleware, wherein the message format of the second middleware is the second message format, and the second middleware is used to perform format conversion on the sensing instruction and the sensor information.

[0095] The second message format mentioned above can also be the message format corresponding to the environment of the first robot system itself. For example, the second message format can be the message format corresponding to the sensor. For another example, for something like... Figure 2 In the RCP architecture shown, the service provider module does not communicate directly with the sensor, and the second message format can be a common message format of the middleware LCM and ROS.

[0096] The second middleware mentioned above can be the widely used middleware LCM and ROS, enabling communication between the information providing module and the target sensor. Therefore, hardware vendors can refer to the data formats of LCM and ROS to build corresponding SDKs (Software Development Kits), allowing specific devices to be integrated into the entire intelligent robot system.

[0097] In one alternative embodiment, the sensors of the robot system can be provided by the hardware vendor. The message formats of these sensors may differ. Therefore, if the target sensor's message format is the second message format, the perception command can be directly sent to the target sensor. However, if the target sensor's message format is not the second message format, the perception command can be forwarded to the target sensor through a second middleware. During this process, the second middleware can convert the perception command's message format from the second message format to the target sensor's message format.

[0098] Preferably, the information providing module and the target sensor do not interact directly, but instead forward information through a second middleware.

[0099] In the above embodiments of this application, sending sensor information to the service invocation module includes: sending sensor information to the service invocation module through the model service module in the service provision module.

[0100] The aforementioned model service module can be a module that provides model inference services, and can be deployed locally on the First Robot or in the cloud. For example, the model service module can be as follows: Figure 2 The Model Service shown.

[0101] In one optional embodiment, the sensor information can be sent to the model through a connection between the model service module and the service call module. The model can then identify and fuse the sensor information to guide the subsequent operation of the first robot system.

[0102] In the above embodiments of this application, the method further includes: responding to receiving a task collaboration request sent by the second robot system, parsing the task collaboration request through the interaction service module in the service providing module to determine the task collaboration mode; determining a task execution instruction based on the task collaboration mode through the interaction service module, and forwarding the task execution instruction to the action execution module; converting the message format of the task execution instruction through the action execution module to obtain a task action instruction, wherein the message format of the task action instruction satisfies a first message format; and sending the task action instruction to the second actuator inside the first robot system through the action execution module, wherein the task action instruction is used to control the movement of the second actuator.

[0103] The aforementioned task collaboration request can be a request in a multi-robot collaboration scenario where a first robot system needs to collaborate to execute a task. The first robot system can be a robot system that needs to take over and complete subsequent tasks after other robot systems fail, or it can be a robot system that works directly with other robot systems.

[0104] The aforementioned task collaboration mode can be a specific mode requiring the participation of the first robot system. For example, task collaboration modes are mainly divided into two types: centralized collaboration mode and distributed collaboration mode. Among them, such as... Figure 4 As shown, the centralized collaboration model coordinates the overall work through a powerful master robot system (e.g., one with strong computing power). This master robot then drives all robot systems (master and slave robots) to complete tasks by calling interfaces in a shared domain through established trusted links. All perceptual information is processed uniformly by the master robot, which places high demands on its computing power and model capabilities. Figure 5As shown, the implementation path of the distributed collaboration mode is as follows: the task initiator infers the corresponding target scenario through a model and distributes and aligns the target task within a trusted shared domain (Task Target Broadcast & World Align). All parties establish their own behavioral trajectories based on the task objectives and verify them within the trusted shared domain. After verification, each party fine-tunes its actions based on the actual interaction state, ensuring consistency with the initial trajectory as much as possible. When significant deviations occur, the task initiator decides whether the task should continue and re-adjusts and aligns the distributed behavioral trajectories.

[0105] The above verification process can be divided into the following steps: load allocation rationality, each robot's load requirements need to match its load and endurance; execution trajectory spatiotemporal uniformity, based on time and space dimensions, the entire execution process is consistent for each robot; path conflict detection, the trajectories of robots will not intersect in a certain time dimension.

[0106] The aforementioned interactive service module is the module within the service provision module that interacts with other robot systems. For example, the interactive service module could be as follows: Figure 2 The R2R Node shown.

[0107] The aforementioned second actuator can be the actuator that the task collaboration request needs to control. For example, if the task collaboration request is to control the movement of a robotic arm, the second actuator can be the robotic arm. Or, if the task collaboration request is to control the movement of a leg, the second actuator can be the leg. Or, if the task collaboration request is to control the rotation of the upper body, the second actuator can be the upper body.

[0108] In an optional embodiment, before the control system needs to control the first robot system and send control commands, or when the control system needs to obtain the current state and sensor information of the first robot system, the second robot system can transmit a task collaboration request to the first robot system through the interaction service module in the service provision module. Upon receiving the interaction request, the interaction service module in the service provision module of the first robot system can parse the task collaboration request and determine the task collaboration mode. For example, the task collaboration request may carry identification information corresponding to the task collaboration mode, thereby determining the specific task collaboration mode based on this identification information.

[0109] The source of task execution instructions for the first robot system differs depending on the task collaboration mode. For example, in a centralized collaboration mode, since the task execution instructions are issued by the main robot system, it can be determined that the instructions are carried in the task collaboration request. Therefore, the task execution instructions can be obtained by unloading the task collaboration request. In a distributed collaboration mode, since the task execution instructions need to be generated by the robot system, they can be generated based on the task objective.

[0110] Furthermore, the forwarding process of task execution instructions is similar to that of control instructions, the only difference being that the executing mechanism to be controlled may be different, which will not be elaborated here.

[0111] In the above embodiments of this application, determining the task execution instruction based on the task collaboration mode includes: parsing the task collaboration request to obtain the task execution instruction when the task collaboration mode is a centralized collaboration mode; and parsing the task collaboration request to obtain the task target when the task collaboration mode is a distributed collaboration mode, and generating the task execution instruction based on the task target.

[0112] like Figure 4 As shown, in the centralized collaboration mode, the main robot system drives all robots to complete the task, and the sub-tasks that the robot system needs to execute are determined by the main robot system. Therefore, the task execution instructions are sent from the main robot system to the first robot system.

[0113] like Figure 5 As shown, in the distributed collaboration mode, the task initiator only determines the goal of task collaboration, and the robot system needs to determine its own behavior trajectory based on the task goal. Therefore, the task execution instructions are generated by the first robot system based on the task goal.

[0114] In the above embodiments of this application, parsing the task collaboration request and determining the task collaboration mode includes: in response to receiving the task collaboration request sent by the second robot system, authenticating the second robot system through the interaction service module and obtaining a verification result, wherein the verification result is used to characterize whether the identity of the second robot system is trustworthy; if the verification result characterizes the identity of the second robot system as trustworthy, parsing the task collaboration request through the interaction service module and determining the task collaboration mode.

[0115] Robots are intelligent agents in the physical world. Interactions between them require not only the exchange of information such as identity, location, and state in the physical world, but also communication and interoperability between intelligent agent applications. This document mainly discusses the following three points.

[0116] In the human world, identity perception can be achieved through various existing methods. However, in the physical world of fully autonomous robotic systems, their deployment scenarios are far more diverse than those of driverless cars. While a robotic system's identity perception will be closer to that of a human, its unique perceptual capabilities allow it to perceive identity through multiple external sensors. It extracts relevant identity identifiers, communication methods, and other information from these sensors and establishes a two-way identity verification mechanism. In an untrusted environment, identity perception becomes even more complex.

[0117] In one optional embodiment, before executing the task collaboration request, the first robot system first needs to authenticate the second robot system to determine whether the identity of the second robot system is trustworthy. If it is trustworthy, the first robot system can continue to execute the task execution request. For details of the specific execution process, please refer to the embodiments described above, which will not be repeated here.

[0118] In the above embodiments of this application, the authentication of the second robot system and the obtaining of the authentication result include: parsing the task collaboration request through the interaction service module to obtain the first identity information of the second robot system; collecting the second identity information of the second robot system through the information providing module; and matching the first identity information and the second identity information through the interaction service module to obtain the authentication result.

[0119] In one optional embodiment, to achieve authentication, the second robot system can carry its own identity information in the task collaboration request, facilitating authentication by the first robot system. Therefore, the interaction service module can parse the task collaboration request to obtain the first identity information carried within it. Simultaneously, the first robot system can use external sensors to perceive the identity of the second robot system, thereby obtaining its second identity information. Finally, by matching the first and second identity information, if the two pieces of information match successfully, the second robot system's identity can be determined to be trustworthy; otherwise, its identity can be determined to be untrustworthy.

[0120] In the above embodiments of this application, the method further includes: if the verification result indicates that the identity of the second robot system is trustworthy, sending the capability information of the first robot system to the second robot system through the interaction service module.

[0121] The aforementioned capability information can refer to the physical operational capabilities and intelligent capabilities of the first robot system, which are recorded in the Robot Card and AI Card within the robot. Furthermore, the first robot system possesses an event-driven, global update mechanism.

[0122] like Figure 6As shown, due to the complexity of collaborative scenarios, a trusted shared domain can be constructed based on different scenarios and tasks. Within the domain, each robot can share its own identity information, physical and intelligent interfaces, capabilities, and trusted communication interfaces to facilitate task collaboration. When the Card information changes, the domain information is updated synchronously, and other robots in the domain are notified simultaneously.

[0123] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse.

[0124] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions. This is because, according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0126] According to an embodiment of this application, a communication system for a robot system implementing the above-described communication method for a robot system is also provided, such as... Figure 7 As shown, the system 700 includes:

[0127] Service invocation module 702 is used to send an interactive request containing interactive instructions;

[0128] The service provider module 704 communicates with the service invocation module and is used to parse the interaction request, obtain the interaction instruction, and forward the interaction instruction if the interaction instruction is a control instruction.

[0129] The action execution module 706 is communicatively connected to the service provider module. It is used to convert the message format of the control command to obtain the action command and send the action command to the first actuator of the first robot system. The message format of the action command satisfies the first message format, and the action command is used to control the movement of the first actuator of the first robot system.

[0130] In the above embodiments of this application, the service providing module includes: a skill layer module, used to determine the first interface protocol corresponding to the action execution module; and an action invocation module, which is communicatively connected to the skill layer module and the action execution module, used to forward control instructions to the action execution module using the first interface protocol.

[0131] In the above embodiments of this application, the system further includes: a first middleware, which is communicatively connected to the action execution module and the first execution mechanism, and is used to convert the action instruction into a format when the message format of the first execution mechanism is not the first message format, to obtain the converted action instruction, and to forward the converted action instruction to the first execution mechanism, wherein the message format of the first middleware is the first message format.

[0132] In the above embodiments of this application, the system further includes: an information providing module, which is communicatively connected to the service providing module, for collecting sensor information of different sensors of the first robot system based on the information reporting instruction sent by the service providing module, and sending the sensor information to the service providing module.

[0133] In the above embodiments of this application, the service providing module includes: a skill layer module, used to determine the second interface protocol corresponding to the information providing module; and an information invocation module, connected to the skill layer module, used to send information reporting instructions to the information providing module based on the second interface protocol.

[0134] In the above embodiments of this application, the system further includes: a second middleware, which is communicatively connected to the information providing module and the target sensor, and is used to convert the format of the sensing instruction sent by the information providing module to obtain the converted sensing instruction when the message format of the target sensor is not the second message format, send the converted sensing instruction to the target sensor, and forward the sensor information returned by the target sensor to the service providing module, wherein the message format of the second middleware is the second message format.

[0135] In the above embodiments of this application, the service providing module includes: a model service module, used to send sensor information to the service calling module.

[0136] In the above embodiments of this application, the service providing module includes: an interaction service module, communicatively connected to the action execution module, used to respond to a task collaboration request sent by the second robot system, parse the task collaboration request, determine the task collaboration mode, determine the task execution instruction based on the task collaboration mode, and forward the task execution instruction to the action execution module; the action execution module is also used to convert the message format of the task execution instruction to obtain the task action instruction, and send the task action instruction to the second actuator inside the first robot system, wherein the message format of the task action instruction satisfies the first message format, and the task action instruction is used to control the movement of the second actuator.

[0137] The preferred embodiments involved in the above embodiments of this application are the same as the solutions, application scenarios and implementation processes provided in the above embodiments, and will not be repeated here.

[0138] Embodiments of this application may provide a robot system, which may be any robot system in a multi-robot collaborative scenario.

[0139] In this embodiment, the robot system described above can execute program code in the robot system's communication method.

[0140] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. This program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0141] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store program code executed by the method provided in the above embodiments.

[0142] Optionally, in this embodiment, the storage medium may be located in any one of the electronic devices in the group of electronic devices in the computer network, or in any one of the mobile terminals in the group of mobile terminals.

[0143] Optionally, in this embodiment, the computer-readable storage medium includes a stored executable program, wherein the methods of various embodiments of the present application are implemented when the executable program is executed by a processor.

[0144] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0145] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium can be used to store a computer program. When executed by a processor, the computer program implements the methods provided in the embodiments described above.

[0146] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.

[0147] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] The technical content disclosed in the several embodiments provided in this application can be implemented in other ways. The device embodiments described above are illustrative; for example, the division of units is a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined, integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling, direct coupling, or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, units, or modules, and may be electrical or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0150] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0151] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0152] The above description represents the preferred embodiments of this application. For those skilled in the art, various improvements and modifications can be made without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A communication method for a robot system, characterized in that, Applied to a first robot system, on which a service provision module and an action execution module are deployed, the method includes: In response to receiving an interaction request sent by the service invocation module, the service providing module parses the interaction request to determine the interaction instructions contained in the interaction request, wherein the service invocation module is deployed in the control system; When the interaction command is a control command, the service providing module forwards the control command to the action execution module; The action execution module converts the message format of the control command to obtain the action command, wherein the message format of the action command satisfies the first message format inside the first robot system. The motion execution module sends the motion command to the first actuator of the first robot system, wherein the motion command is used to control the movement of the first actuator.

2. The method according to claim 1, characterized in that, The step of parsing the interaction request to determine the interaction instructions contained in the interaction request includes: The interaction request is parsed using a preset communication protocol to obtain the interaction instruction, wherein the preset communication protocol is used to transmit information between the service providing module and the service calling module.

3. The method according to claim 1, characterized in that, The step of forwarding the control command to the action execution module through the service providing module includes: The first interface protocol corresponding to the action execution module is determined by the skill layer module in the service provision module; The control command is forwarded to the action execution module through the action invocation module in the service provision module using the first interface protocol.

4. The method according to claim 1, characterized in that, The control instructions include: a first action sequence, where information in different rows of the first action sequence corresponds to different actuators of the first robot system, and information in different columns of the first action sequence corresponds to different states; the conversion of the message format of the control instructions to obtain action instructions includes: The first action sequence is parsed using a first interface protocol to determine the first actuator in the first robot system and the target state corresponding to the first actuator. The first interface protocol is used to transmit messages between the service providing module and the action execution module. The target state is formatted using the first message format to obtain the action instruction corresponding to the first actuator.

5. The method according to claim 4, characterized in that, When the actuator is a limb joint of the first robot system, the information in the first row corresponding to the limb joint in the first action sequence includes at least: pose, angle, and action; when the actuator is a torso joint of the first robot system, the information in the second row corresponding to the limb joint in the first action sequence includes at least: distance traveled, speed, and direction.

6. The method according to claim 1, characterized in that, The step of sending the action command to the first actuator of the first robot system through the action execution module includes: When the message format of the first actuator is the first message format, the action instruction is sent to the first actuator; If the message format of the first actuator is not the first message format, the action instruction is sent to the first middleware, and the action instruction is forwarded to the first actuator through the first middleware. The message format of the first middleware is the first message format, and the first middleware is used to convert the format of the action instruction.

7. The method according to any one of claims 1 to 6, characterized in that, The first robot system is also equipped with an information providing module, and the method further includes: When the interaction instruction is an information reporting instruction, the information reporting instruction is forwarded to the action execution module and the information providing module through the service providing module; The action execution module collects the status information of different actuators of the first robot system based on the information reporting instruction to obtain a second action sequence, and sends the second action sequence to the service providing module. The information providing module collects sensor information from different sensors of the first robot system based on the information reporting instruction, and sends the sensor information to the service providing module. The service providing module sends the second action sequence and the sensor information to the service calling module.

8. The method according to claim 7, characterized in that, Sending the sensor information to the service invocation module includes: The sensor information is sent to the service invocation module through the model service module in the service provision module.

9. The method according to any one of claims 1 to 6, characterized in that, The method further includes: In response to receiving a task collaboration request from the second robot system, the task collaboration request is parsed through the interaction service module in the service provision module to determine the task collaboration mode; The interactive service module determines the task execution instruction based on the task collaboration mode and forwards the task execution instruction to the action execution module. The action execution module converts the message format of the task execution instruction to obtain a task action instruction, wherein the message format of the task action instruction satisfies the first message format. The task action instruction is sent to the second actuator inside the first robot system through the action execution module, wherein the task action instruction is used to control the movement of the second actuator.

10. The method according to claim 9, characterized in that, The step of determining the task execution instruction based on the task collaboration mode includes: When the task collaboration mode is a centralized collaboration mode, the task collaboration request is parsed to obtain the task execution instruction; When the task collaboration mode is a distributed collaboration mode, the task collaboration request is parsed to obtain the task objective, and the task execution instruction is generated based on the task objective.

11. The method according to claim 9, characterized in that, The step of parsing the task collaboration request and determining the task collaboration mode includes: In response to receiving a task collaboration request from the second robot system, the second robot system is authenticated through the interaction service module to obtain a verification result, wherein the verification result is used to characterize whether the identity of the second robot system is trustworthy; If the verification result indicates that the identity of the second robot system is trustworthy, the task collaboration request is parsed by the interaction service module to determine the task collaboration mode; Wherein, if the verification result indicates that the identity of the second robot system is trustworthy, the capability information of the first robot system is sent to the second robot system through the interaction service module.

12. The method according to claim 11, characterized in that, The authentication of the second robot system to obtain the authentication result includes: The task collaboration request is parsed by the interactive service module to obtain the first identity information of the second robot system; The second identity information of the second robot system is collected through the information providing module; The verification result is obtained by matching the first identity information and the second identity information through the interactive service module.

13. A communication system for a robot system, characterized in that, include: The service invocation module is used to send interactive requests containing interactive instructions; The service providing module is communicatively connected to the service invocation module and is used to parse the interaction request to obtain the interaction instruction. If the interaction instruction is a control instruction, the module forwards the interaction instruction. An action execution module, which is communicatively connected to the service providing module, is used to convert the message format of the control command to obtain an action command, and send the action command to the first actuator of the first robot system. The message format of the action command satisfies a first message format, and the action command is used to control the movement of the first actuator of the first robot system.

14. The system according to claim 13, characterized in that, Also includes: The information providing module is communicatively connected to the service providing module and is used to collect sensor information from different sensors of the first robot system based on the information reporting instructions sent by the service providing module, and send the sensor information to the service providing module.

15. The system according to claim 13 or 14, characterized in that, The service provision module includes: The interactive service module, which is communicatively connected to the action execution module, is used to respond to a task collaboration request sent by the second robot system, parse the task collaboration request, determine the task collaboration mode, determine the task execution instruction based on the task collaboration mode, and forward the task execution instruction to the action execution module. The action execution module is further configured to convert the message format of the task execution instruction to obtain a task action instruction, and send the task action instruction to the second actuator inside the first robot system, wherein the message format of the task action instruction satisfies the first message format, and the task action instruction is used to control the movement of the second actuator.

16. A robot system, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 12.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 12.

18. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images