Robot control system and robot

Abstract

The invention discloses a robot control system and a robot, the system comprises an upper computer and a lower computer, the lower computer is used for obtaining data from various sensors, decoding and operating the data, uploading message data to be uploaded to the upper computer, obtaining instructions from the upper computer, and executing control operation according to the instructions. A native system application, a message transfer service and a robot operating system (ROS) application are arranged in the upper computer, the native system application can restore and integrate existing native system applications of a team, and various mature algorithms and tool resources of the ROS can be fully utilized through the ROS application. The lower computer and the native system application in the upper computer carry out message communication through a first communication protocol, the message transfer service is responsible for communication adaptation between the native system application and the ROS application, and communication interaction between the native system application and the ROS application is achieved through conversion of message formats.

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and more specifically, to a robot control system and a robot. Background Art

[0002] As a powerful tool for quickly building robots, the Robot Operating System (ROS) is widely used in the development of the robotics industry. The Robot Operating System includes the first-generation ROS and the second-generation ROS2. Taking ROS2 as an example, it has a powerful ecosystem and provides a large number of software packages and tool sets based on the general OS. These componentized toolkits are free and open source, with a large number of developers, which can shorten the product development cycle and improve stability.

[0003] There are two main types of current robot software development solutions:

[0004] One is that all the upper computer software in the robot is based on ROS (taking ROS2 as an example) to make full use of the rich robot algorithms and application tools of ROS2.

[0005] The second is that all software on the host computer is based on a pure native system (taking Linux as an example) to achieve the best compatibility.

[0006] For the first solution, all software is based on ROS2, which means that the software development team needs to make major adaptations to the applications previously developed based on Linux before they can be applied to the ROS2-based system platform. Moreover, when other better system platforms replace ROS2 in the future, all adaptation work needs to be done again, which is not conducive to the reuse and inheritance of the team's software.

[0007] For the second option, all software is based on Linux. Although it has good compatibility and scalability, it is unable to utilize the rich algorithm resources and mature software tools of ROS2. Many basic development tasks need to be implemented from scratch, which prolongs the project cycle. Summary of the invention

[0008] In view of the above problems, this application is proposed to provide a robot control system and robot, which can not only utilize various mature algorithms and tool resources of the robot operating system, but also have good inheritance and reusability, and make full use of the software achievements developed by the team. The specific plan is as follows:

[0009] In a first aspect, a robot control system is provided, comprising: a lower computer and an upper computer;

[0010] The lower computer is used to obtain data from various sensors and decode and calculate the data to obtain message data to be uploaded, upload the message data to be uploaded to the upper computer according to a set format, and obtain instructions from the upper computer and perform control operations according to the instructions;

[0011] The host computer is provided with a native system application, a message transfer service and a robot operating system ROS application, and the native system is different from the robot operating system;

[0012] The lower computer and the native system application in the upper computer perform message communication through a first communication protocol. The message transfer service is responsible for the communication adaptation between the native system application and the ROS application, and realizes the communication interaction between the native system application and the ROS application through the conversion of the message format.

[0013] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the message relay service performs message communication with the native system application through a second communication protocol, and the message relay service performs message communication with the ROS application through a third communication protocol;

[0014] The second communication protocol adopts an inter-process communication IPC communication protocol;

[0015] The third communication protocol adopts the communication mechanism of ROS.

[0016] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the first communication protocol includes at least one of the following:

[0017] Serial peripheral interface SPI communication protocol and universal asynchronous receiver and transmitter UART communication protocol;

[0018] The second communication protocol includes a local socket communication protocol;

[0019] The third communication protocol includes at least one of the following:

[0020] Topic, service, and action.

[0021] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the setting format is a binary code format; and the process in which the native system application sends the message to the message forwarding service includes:

[0022] The native system application converts the message in binary code format into a Json string, and sends the Json string to the message relay service.

[0023] In one possible design, in another implementation of the first aspect of the embodiment of the present application, a sending queue, a receiving queue, a sending thread and a receiving thread are provided in the host computer. When the native system application needs to send an instruction message to the lower computer, the instruction message is placed in the sending queue; the sending thread reads the sending queue at a set frequency, and if the sending queue is empty, all 0 data is sent; if the sending queue is not empty, the instruction message is read from the sending queue and sent.

[0024] In a possible design, in another implementation of the first aspect of the embodiment of the present application, a message processor is further provided in the upper computer, and after receiving a message other than all 0s uploaded by the lower computer, the message processor puts the message into the receiving queue and wakes up the receiving thread;

[0025] The receiving thread cyclically reads the receiving queue. If the receiving queue is empty, the receiving thread is blocked until it is awakened next time. If the receiving queue is not empty, the message is read from the receiving queue and reported to the native system application.

[0026] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the SPI communication protocol includes the following fields:

[0027] Message header, version number, action number, parameters and check digit, wherein the message header field contains fixed content, the version number field contains the version information of the message, the action number field contains the unique number of the message instruction issued by the host computer, the parameter field contains the uploaded parameter value, and the check digit contains the check value of the entire message.

[0028] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the SPI communication protocol further includes the following fields:

[0029] Action type and / or action hold flag, wherein the action type field includes the command type number corresponding to the message instruction sent by the host computer, different command type numbers correspond to different motion mechanisms and sensors on the robot, and the action hold flag is used to indicate whether the motor maintains the current position or unloads the force after the action is completed.

[0030] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0031] A cloud server that provides background services for the robot operation, wherein the cloud server communicates with the native system application.

[0032] In a second aspect, a robot is provided, comprising the robot control system described in any one of the first aspects.

[0033] By means of the above technical scheme, the robot control system provided by the present application includes an upper computer and a lower computer, the lower computer is used to obtain data from various sensors and decode and calculate the data, obtain the message data to be uploaded, upload the message data to be uploaded to the upper computer according to the set format, and obtain instructions from the upper computer, and perform control operations according to the instructions. The upper computer is provided with native system applications (examples such as Linux applications, Android applications, etc.), message transfer services and robot operating system ROS applications (ROS applications in this application generally refer to applications developed based on robot operating systems, examples such as applications developed through the first generation of ROS, applications developed through ROS2, etc.), here, the native system application is different from the robot operating system, the native system application can restore and integrate the existing native system applications of the team, and through the ROS application, various mature algorithms and tool resources of ROS can be fully utilized. The lower computer and the native system application in the upper computer communicate messages through the first communication protocol, the message transfer service is responsible for the communication adaptation between the native system application and the ROS application, and the communication interaction between the native system application and the ROS application is realized through the conversion of the message format. Obviously, the control system of this application can not only make full use of the team's existing applications developed based on the native system, but also use various mature algorithms and tool resources of ROS to develop ROS applications. Native system applications and ROS applications can be deployed simultaneously in the host computer, and communication interaction between the two applications can be achieved through message forwarding services to ensure the normal operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0035] Figure 1 A schematic diagram of the structure of a robot control system provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of the structure of another robot control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] The present application provides a robot control system, combined with Figure 1 As shown, the robot control system includes: a lower computer and a host computer.

[0039] The lower computer is used to obtain data from various sensors and decode and calculate the data to obtain the message data to be uploaded, upload the message data to be uploaded to the upper computer according to the set format, and obtain instructions from the upper computer and perform control operations according to the instructions.

[0040] like Figure 1 As shown, the sensor can be of various types, including but not limited to: wireless charging module, motor / motor, six-axis accelerometer, magnetic encoder, infrared sensor, TOF, button, etc.

[0041] The lower computer is responsible for executing the instructions issued by the upper computer and controlling each module, such as the power supply, motor, cylinder, etc. In addition, it reads data in real time through various sensors and pre-processes the collected data, such as filtering, amplification, conversion, etc., to ensure the accuracy and reliability of the data. In addition, it calculates and analyzes the data according to the preset algorithm to extract useful information. The message data to be uploaded is sent to the upper computer in a set format so that the upper computer can monitor and manage the entire system.

[0042] The host computer is provided with a native system application, and the native system is different from the robot operating system, and it can be various types of systems such as Linux system, Android system, etc. Figure 1 The Linux application is used as an example to illustrate the native system application.

[0043] Native system applications are the core of the entire software system and are responsible for interacting with users and completing data exchange with cloud servers, lower computers, and ROS applications. Native system applications are developed based on a pure native system environment and are independent of ROS. Native system applications can inherit and reuse native system applications developed by the design team in the past, or they can be applications developed later based on the native system.

[0044] In addition, the host computer also includes the robot operating system ROS application. It should be noted here that in this embodiment, the ROS application generally refers to an application developed based on the robot operating system, which can be an application developed based on the first generation ROS or an application developed based on the second generation ROS2. Figure 1 The ROS2 application is used as an example to illustrate.

[0045] ROS applications are applications or services based on the ROS compilation environment, including but not limited to: SLAM algorithms, navigation algorithms, human recognition algorithms, face recognition algorithms, gesture recognition algorithms, target following algorithms, etc. These algorithms have mature and stable resources in ROS and can be directly used in robots.

[0046] In this embodiment, the ROS application in the host computer can be an application developed using mature algorithms and tool resources provided by ROS, thereby shortening the system design cycle.

[0047] Furthermore, in order to ensure the communication interaction between native system applications and ROS applications, the host computer also includes a message transfer service. The message transfer service is responsible for the communication adaptation between native system applications and ROS applications, and realizes the communication interaction between native system applications and ROS applications through the conversion of message formats.

[0048] The lower computer and the native system application in the upper computer communicate with each other through the first communication protocol to upload data in the lower computer. The message data uploaded by the lower computer can be called by the native system application, or can be further forwarded to the ROS application through the message transfer service for calling by the ROS application.

[0049] The robot control system provided by the embodiment of the present application includes a host computer and a lower computer. The host computer is provided with native system applications, message transfer services and robot operating system ROS applications. Here, the native system applications are different from the robot operating system. The native system applications can restore and integrate the existing native system applications of the team, and through the ROS application, the various mature algorithms and tool resources of ROS can be fully utilized. The lower computer and the native system applications in the host computer communicate messages through the first communication protocol. The message transfer service is responsible for the communication adaptation between the native system application and the ROS application, and the communication interaction between the native system application and the ROS application is realized through the conversion of the message format. Obviously, the control system of the present application can make full use of the existing applications developed by the team based on the native system, and can also use the various mature algorithms and tool resources of ROS to develop ROS applications. The native system application and the ROS application can be deployed simultaneously in the host computer, and the communication interaction between the two applications is realized through the message transfer service to ensure the normal operation of the system.

[0050] Combination Figure 2 As shown, the robot control system of the present application may further include a cloud server, which provides background services for the robot operation. The cloud server communicates with the native system application and provides functions such as user information storage and management, voice recognition, and interaction with the mobile client.

[0051] In some embodiments of the present application, the communication process between the modules involved in the robot control system is further described.

[0052] As described above, the lower computer communicates with the native system application in the upper computer through the first communication protocol.

[0053] The first communication protocol can adopt various types of communication protocols. Figure 1 Two optional first communication protocols are exemplified, including a serial peripheral interface SPI communication protocol and a universal asynchronous receiver-transmitter UART communication protocol.

[0054] Taking the SPI communication protocol as an example, the SPI interface hardware occupies up to 4 pins, which saves more space in PCB layout. In addition, the SPI interface has the characteristics of high speed, full duplex, synchronization, and no defined speed limit. The general implementation can usually reach more than 10Mbps, which can meet the robot's needs for real-time and high-frequency acquisition of sensor data in scenarios such as navigation and SLAM mapping.

[0055] The SPI communication protocol belongs to a single master device communication protocol. Each communication is initiated by the master device, which sends fixed-length data to the slave device and reads fixed-length data from the slave device. Based on this, a specific format of the SPI communication protocol is designed in the embodiment of the present application, as shown in Table 1 below:

[0056] Table 1

[0057]

[0058] Among them, the fields: reserved, action type (including action main type and action subtype), and action hold flag are optional fields.

[0059] Next, the meaning of each field in Table 1 is introduced respectively:

[0060] Message header: The beginning of each message, containing fixed content. The message header in the example in Table 1 contains a fixed 0X550XAA, but it can also be set to other content.

[0061] Version number: Contains the version information of the message. If the message format is upgraded in the future, this field is used to be compatible with different versions of messages.

[0062] Action number: contains the unique number of the message instruction sent by the host computer. Each message instruction (action instruction) sent by the host computer has a unique number. When the lower computer reports the execution completion message after executing the corresponding message instruction, it needs to carry this number. The host computer uses the number to distinguish which message instruction (action) has been completed.

[0063] Action type: includes the command type number corresponding to the message instruction sent by the host computer. Different command type numbers correspond to different motion mechanisms and sensors on the robot. The robot has multiple degrees of freedom (motion mechanisms) and sensors. For example, the head, arm, wheel, TOF, six-axis sensor, etc. Different motion mechanisms and sensors are referred to by different command type numbers. Furthermore, the action type field can be divided into the main action type and the sub-action type. The main type and sub-type are used to distinguish different motion mechanisms and sensors. The specific division method can be set by the user.

[0064] Action hold flag: used to mark whether the motor maintains the current position or unloads after the action is completed. For example, for a structure such as a robot arm, if the motor unloads the force after the lifting action is completed, the arm will naturally droop.

[0065] In the example in Table 1, when the field value is 0X01, it means maintaining the current position; when the field value is 0X00, it means unloading.

[0066] Parameters 1-9: include uploaded parameter values, such as the time and speed to complete the action, or data parameters uploaded by the sensor.

[0067] Check bit: includes the check value of the entire message. The check bit value can be calculated using CRC16 encoding to prevent abnormal levels during transmission caused by electromagnetic interference.

[0068] When the upper computer and the lower computer communicate, the upper computer acts as the host, responsible for providing the clock and initiating the communication. According to the communication protocol shown in Table 1, 29 bytes of data are exchanged in each communication.

[0069] This embodiment provides a communication mechanism that can ensure the isolation between the data transmission part and the upper layer application. Specifically:

[0070] When implementing duplex communication, the host computer uses two message queues, one of which is a sending queue (which can be a non-blocking sending queue) and the other is a receiving queue (which can be a blocking receiving queue). Also, two threads are used, one of which is a sending thread and the other is a receiving thread.

[0071] When the native system application needs to send a command message to the lower computer, the command message is placed in the sending queue. The sending thread reads the sending queue at a set frequency (such as 1000Hz). If the sending queue is empty, all 0 data is sent. If the sending queue is not empty, the command message is read from the sending queue and sent.

[0072] The sending thread reads a fixed length (in bytes) each time it reads a command message from the sending queue. Figure 1Taking the communication protocol of the example as an example, the fixed length of the message is 29 bytes).

[0073] Furthermore, the upper computer may also include a message processor, which may directly discard the message containing all zeros uploaded by the lower computer. If a message containing non-all zeros is received, the message is placed in a receiving queue and the receiving thread is awakened.

[0074] The receiving thread reads the receiving queue in a loop. If the receiving queue is empty, the receiving thread blocks and hands over the CPU controller until it is woken up next time. If the receiving queue is not empty, the message is read from the receiving queue and reported to the native system application.

[0075] By adopting the communication mechanism provided in this embodiment, the isolation between the data transmission part and the upper layer application can be ensured, the jamming of one party will not cause the jamming of the other party, and the communication delay is ensured to be within a controllable range. The theoretical communication delay is the delay of SPI data transmission + the delay caused by polling. If the delay exceeds the acceptable range, the delay of SPI data transmission can be reduced by increasing the communication clock frequency, and the delay caused by polling can be reduced by increasing the polling frequency.

[0076] In some possible implementations, the communication between the native system application and the message relay service may be implemented through a second communication protocol.

[0077] The second communication protocol may be an inter-process communication (IPC) communication protocol, which includes various types, such as a local socket communication protocol.

[0078] The Local Socket communication protocol is an IPC communication mechanism derived from the Socket, which is used to solve the problem of communication between different processes on the same host. Compared with the Socket used for network communication, it does not need to go through the network protocol stack, does not require packaging, unpacking, calculation and verification, and has higher execution efficiency.

[0079] Local Socket has a Linux standard implementation and a simple interface, so it is particularly suitable for cross-process communication between Linux applications and message transfer services.

[0080] Generally, the communication between the host computer and the slave computer uses binary code to improve transmission efficiency, while the two parties of Local Socket communication are two processes on the same system, and the communication efficiency is very high. Therefore, in this embodiment, the message format between the native system application and the message transfer service uses a Json string with more flexible processing and higher readability. On this basis, the process of the native system application sending a message to the message transfer service includes:

[0081] The native system application converts the binary code format message into a Json string and sends the Json string to the message transfer service.

[0082] Local Socket is based on bytecode streaming. In order to separate messages from the data stream, you can add a flag to the head and tail of each message. The flag can be a character contained in a non-JSON string.

[0083] In the usage scenario of this application, special symbols cannot appear in the Json string, so in this embodiment, special symbols can be used as the head and tail marks. For example, "@" is used as the message header mark and "&" is used as the message tail mark. The message example of a communication is as follows:

[0084] @{"cmd_type":"foot_move","data":{"linear":{"x":1,"y":2,"z":3},"angular":{"x":4,"y":5,"z":6},"target_type":1,"target":3}}&.

[0085] In some possible implementations, the communication mechanism of two message queues and two threads introduced in the above embodiment may also be used in the message transfer service to achieve isolation between message sending and receiving and upper-layer logic.

[0086] In some possible implementations, the communication between the message relay service and the ROS application may be implemented through a third communication protocol.

[0087] The third communication protocol may adopt the communication mechanism of ROS, for example, the communication mechanism of ROS2, including but not limited to:

[0088] Topic, service, and action.

[0089] The above communication methods all have the characteristics of low latency and low resource usage. You can flexibly select one of them for communication according to different usage scenarios.

[0090] Using the robot control system introduced in the previous embodiment of this application, after real machine testing, the delay of the entire link data transmission (SPI communication + Local Socket communication + ROS2 communication) from the lower computer to the ROS2 application is between 4.5ms and 5ms, which can meet the user's usage needs.

[0091] In the embodiment of the present application, the robot operating system is ROS2 and the native operating system is Linux system as an example for explanation:

[0092] The compilation system of ROS2 is different from that of pure Linux. In the prior art, if you want to port the Linux applications accumulated by the team to the ROS2 system, you need to make major modifications to the original applications. Through the above-mentioned robot control system structure proposed in this application, as well as the communication protocol between the various parts, the zero-cost porting of the original Linux application is realized, the original code can be used directly, and a large number of mature algorithms and software tools of ROS2 can still be reused. In the future, if you want to port it to other systems, you only need to modify the message forwarding service, which greatly reduces the cost of porting.

[0093] In some embodiments of the present application, a robot is also provided, which includes the robot control and system introduced in the above embodiments.

[0094] The robot in this embodiment can be various types of robots, such as a home companion robot, a transport robot, a cleaning robot, etc.

[0095] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0096] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0097] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0098] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A robot control system, characterized in that: include: Lower computer and upper computer; The lower computer is used to obtain data from various sensors and decode and calculate the data to obtain message data to be uploaded, upload the message data to be uploaded to the upper computer according to a set format, and obtain instructions from the upper computer and perform control operations according to the instructions; The host computer is provided with a native system application, a message transfer service and a robot operating system ROS application, and the native system is different from the robot operating system; The lower computer communicates messages with the native system application in the upper computer through a first communication protocol. The message transfer service is responsible for communication adaptation between the native system application and the ROS application, and realizes communication interaction between the native system application and the ROS application through message format conversion.

2. The system according to claim 1, characterized in that The message transfer service performs message communication with the native system application through a second communication protocol, and the message transfer service performs message communication with the ROS application through a third communication protocol; The second communication protocol adopts an inter-process communication IPC communication protocol; The third communication protocol adopts the communication mechanism of ROS.

3. The system according to claim 2, characterized in that The first communication protocol includes at least one of the following: Serial peripheral interface SPI communication protocol and universal asynchronous receiver and transmitter UART communication protocol; The second communication protocol includes a local socket communication protocol; The third communication protocol includes at least one of the following: Topic, service, and action.

4. The system according to claim 1, characterized in that The setting format is a binary code format; The process of the native system application sending a message to the message forwarding service includes: The native system application converts the message in binary code format into a Json string, and sends the Json string to the message relay service.

5. The system according to claim 1, characterized in that The host computer is provided with a sending queue, a receiving queue, a sending thread and a receiving thread. When the native system application needs to send a command message to the lower computer, the command message is placed in the sending queue; the sending thread reads the sending queue at a set frequency, and if the sending queue is empty, all 0 data is sent; if the sending queue is not empty, the command message is read from the sending queue and sent.

6. The system according to claim 5, characterized in that The host computer is also provided with a message processor, which, after receiving a message not containing all zeros uploaded by the lower computer, puts the message into the receiving queue and wakes up the receiving thread; The receiving thread cyclically reads the receiving queue. If the receiving queue is empty, the receiving thread is blocked until it is awakened next time. If the receiving queue is not empty, the message is read from the receiving queue and reported to the native system application.

7. The system according to claim 3, characterized in that The SPI communication protocol includes the following fields: Message header, version number, action number, parameters and check digit, wherein the message header field contains fixed content, the version number field contains the version information of the message, the action number field contains the unique number of the message instruction issued by the host computer, the parameter field contains the uploaded parameter value, and the check digit contains the check value of the entire message.

8. The system according to claim 7, characterized in that The SPI communication protocol also The following fields are included: Action type and / or action hold flag, wherein the action type field includes the command type number corresponding to the message instruction sent by the host computer, different command type numbers correspond to different motion mechanisms and sensors on the robot, and the action hold flag is used to indicate whether the motor maintains the current position or unloads the force after the action is completed.

9. The system according to any one of claims 1 to 8, characterized in that: Also includes: A cloud server that provides background services for the robot operation, wherein the cloud server communicates with the native system application.

10. A robot, characterized in that: A robot control system comprising any one of claims 1-9.

Images