System for operating robot and robot device

Abstract

The invention discloses a system for operating a robot and a robot apparatus. The system comprises a user mode robot subsystem configured to be responsible for functions associated with a robot; the host operating system is configured to be responsible for providing a user interface and interaction for the robot subsystem, the robot subsystem and the host operating system are coupled to the same kernel to multiplex the kernel, and the robot subsystem can access system services of the host operating system. And the host operating system can access the system service of the robot subsystem. According to the invention, kernel multiplexing and resource sharing can be realized, and meanwhile, system redundancy is reduced.

Description

Technical Field

[0001] The present invention relates to the field of robot operating systems, and in particular to a system and a robot device for operating a robot. Background Art

[0002] Traditional robot operating systems generally use dual systems, such as the Linux-like Robot Operating System (ROS) for processing SLAM (Simultaneous Localization and Mapping) / motion control and character control, and the Android operating system (Android) for processing human-computer interaction.

[0003] There are currently two solutions to achieve dual systems. One is a dual system based on single-chip full virtualization, that is, an operating system with fully virtualized hardware resources. However, full virtualization requires support from the operating system driver layer and the middle layer, which results in large performance loss, difficulty in GPU / IO resource reuse, and incomplete compatibility of the display system (X11, etc.), which is not suitable for robot operating system application scenarios. The other is a dual system based on dual-chip cooperation, that is, a 1+1 solution, which is relatively simple but increases the difficulty of dual-machine communication.

[0004] In summary, whether it is a virtualized dual system implemented based on a single chip or a dual system implemented based on dual chips, there is additional overhead in communication and synchronization, resulting in more redundancy in the operating system. Summary of the invention

[0005] The present invention provides a system and a robot device for operating a robot, realizing core reuse and hardware resource sharing while reducing system redundancy.

[0006] In one aspect of the present invention, a system for operating a robot is provided. The system includes a user-mode robot subsystem configured to be responsible for functions associated with the robot; and a host operating system configured to be responsible for providing a user interface and interaction to the robot subsystem, wherein the robot subsystem and the host operating system are both coupled to the same kernel to reuse the kernel, the robot subsystem can access the system services of the host operating system, and the host operating system can access the system services of the robot subsystem.

[0007] In another aspect of the present invention, a robot device is provided, which includes: a single chip; and a single operating system, including the system for operating the robot according to the above and the kernel.

[0008] According to the technical solution of the present invention, by setting up a host operating system and a robot subsystem, and the two systems reuse one kernel, the waste of software and hardware resources caused by multiple kernels is avoided. At the same time, the host operating system and the robot subsystem are respectively configured with different functions to avoid system redundancy caused by resource duplication. The host operating system and the robot subsystem indirectly complete the corresponding functions through each other's system services, so that the system can control the other system to achieve the specified function when the corresponding function is not configured, thereby realizing resource sharing between the host operating system and the robot subsystem. Compared with the existing single-system solution, the host operating system and the robot subsystem of the present invention share resources without the need for virtualization operations, effectively reducing system performance loss. Compared with the existing dual-system solution, the host operating system and the robot subsystem of the present invention are based on a single chip, that is, a single kernel to respectively realize the corresponding functions of the dual systems, effectively reducing the system redundancy caused by the dual systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is an architectural diagram of a system for operating a robot according to an embodiment of the present invention; Figure 2 A flowchart of a process in which a host operating system controls a robot in a system according to an embodiment of the present invention; Figure 3 A flowchart of a process of interaction between a robot subsystem and a user in a system according to an embodiment of the present invention; Figure 4 A flowchart of a process of displaying a picture by a robot subsystem in a system according to an embodiment of the present invention; Figure 5 The present invention is a flowchart of a process for implementing initialization in a system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0010] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0011] In the existing technology, the virtualized dual system based on a single chip is not suitable for the robot operating system because it has a great impact on the system performance. The dual system based on dual main control chips (i.e., the 1+1 solution) is relatively simple, but it increases the difficulty of dual-machine communication. In addition, the performance loss of the solution in the dual system based on a single chip (Linux + (Docker + Android)) is relatively large, GPU / IO resource reuse is difficult, and the display system cannot be perfectly compatible. In addition, in the 1+1 hybrid solution, in addition to the industrial control board, the robot may also be equipped with an Android development board to provide a development platform for developers, so that the applications they develop can be installed on the user's mobile terminal, and the developed applications can also be installed on the robot's development board, so that the user's mobile terminal can control the robot. In the 1+1 hybrid solution, 2 sets of hardware (industrial control board + Android development board) and 2 sets of operating systems (ROS + Android) are required.

[0012] In order to solve at least the above technical problems, the present disclosure provides a solution of single chip + single operating system + robot subsystem. According to the embodiment of the present disclosure, a robot subsystem (Robert Subsystem, RSS) solution based on Android is proposed, and the solution has the following characteristics. The kernel is fully reused to avoid the waste of software and hardware resources caused by multiple cores. The host operating system and the robot subsystem RSS are independent of each other and have their own responsibilities. For example, the host operating system is responsible for display and user interaction, and the robot subsystem RSS is responsible for complex robot-related functions. The host operating system and the robot subsystem RSS can access public services between each other, that is, there is a mechanism to ensure mutual cooperation to complete tasks. Compared with multi-operating system, multi-machine system and virtual machine system solutions, the redundancy of the entire system is minimal.

[0013] According to the embodiment of the present disclosure, the kernel Kernel refers to kernel components such as Linux / microkernel, which are responsible for task management, memory management, interrupt management and device drivers. The host operating system refers to systems such as Android, Ubuntu, Harmony, etc., which are responsible for the management of file systems, service frameworks, application frameworks, etc. The robot subsystem RSS is parallel to the host operating system and is independently responsible for device drivers, file systems, ROS service packages, ROS application packages, and ROS communication packages. According to the embodiment of the present disclosure, the robot subsystem RSS does not have the functions of a complete operating system and requires a fully functional host operating system to work properly. Compared with a robot system in the form of a virtual machine or a complete dual operating system, it is an improvement scheme. The robot subsystem RSS is a partial function of the robot operating system, not the robot operating system ROS itself.

[0014] According to the embodiment of the present disclosure, the concept of robot subsystem is introduced, and a unique single chip + single operating system + robot subsystem architecture is proposed. Compared with virtual machine multi-system, or software multi-system, or physical dual system, the robot subsystem is not an independent operating system, so there is no redundant kernel redundancy, file system redundancy, dual system communication and synchronization overhead in the dual system. According to the embodiment of the present disclosure, the overall power consumption of the architecture system is better.

[0015] According to an embodiment of the present disclosure, in the robot subsystem startup process, the kernel Kernel is started, and the basic software and hardware environment, such as CPU, memory, device driver and interrupt initialization, are initialized. Then, the host operating system is started, and user 0 (root user) is started. The root user is a super user in the Linux system and has full access to the system and can perform any operation, including modifying system files, installing software, managing user accounts, etc. Subsequently, the robot subsystem RSS initializes the user-state device driver, and the external device driver that the RSS needs to directly access and manage is initialized. The robot subsystem RSS initializes and starts the user-state file system. The robot subsystem RSS initializes and starts the ROS service package. The ROS service package depends on the device driver and file system and needs to be started later. The robot subsystem RSS initializes and starts the ROS application package. The ROS application package depends on the ROS service package and needs to be started later.

[0016] According to the disclosed embodiment, in the robot subsystem access and system services of the host operating system, the robot subsystem queries the system service permissions. The host operating system returns the system service permissions and opens non-sensitive services. If there is permission, the robot subsystem obtains the system service interface (WIFI). The system only has WIFI assigned to the host operating system. If the robot subsystem RSS wants to obtain WIFI communication services, it needs to obtain it indirectly from the host operating system. The robot subsystem executes the system service interface and obtains the execution result. The robot subsystem post-processes the execution result to change the internal state and motion state of the subsystem. The physical hardware interface is assigned to the host operating system. If the robot subsystem RSS wants to use the communication interface to complete the communication task, it needs to obtain the communication capability indirectly from the host operating system. The robot subsystem RSS needs to maintain the internal state of the communication interface (opened, closed, released, working, idle, etc.), and perform control processing and error processing according to the internal state.

[0017] According to the disclosed embodiment, when the host operating system accesses the system service of the robot subsystem, the host operating system queries the system service permission. The robot subsystem returns the system service permission and opens the non-sensitive part of the service (SLAM service, motion service, posture service, etc.). If the permission is granted, the host operating system obtains the system service interface (SLAM service). This interface is combined with the above Figure 3 The system service interface described is similar. Pay attention to distinguishing between master and slave management. For SLAM service, the robot subsystem RSS is the host, and for the communication interface, the host operating system is the host. The host operating system executes the system service interface and obtains the execution result. The host operating system post-processes the execution result and changes the internal state and motion state of the subsystem. For SLAM service, the robot subsystem RSS is the host, and RSS has the service state of maintaining SLAM. The host operating system has the SLAM client state and also needs to maintain a copy of the SLAM state synchronously.

[0018] According to the disclosed embodiment, when the host operating system renders the window system of the robot subsystem, the host operating system starts the robot virtual window application. The display service of the host operating system creates a rendering client. The robot subsystem obtains the rendering client. The host operating system has a rendering server. The robot subsystem obtains the rendering client. The rendering client needs to be routed to the rendering server to complete the rendering. The rendering server then returns the rendering result to the rendering client measured by the RSS. The robot subsystem renders the window system to the rendering client. The display service of the host operating system renders display Buffers. The host operating system displays the window system of the robot subsystem.

[0019] Hereinafter, the technical solution according to the present disclosure will be described with reference to specific embodiments and in conjunction with the accompanying drawings.

[0020] Figure 1 is an architectural diagram showing a system 100 for operating a robot according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a user-mode robot subsystem 102, a host operating system 104, and a kernel 106. The user-mode robot subsystem 102 is configured to be responsible for functions associated with the robot. The host operating system 104 is configured to be responsible for providing a user interface and interaction for the robot subsystem. The robot subsystem 102 and the host operating system 104 are both coupled to the same kernel 106 to reuse the kernel 106. The robot subsystem accesses the system services of the host operating system, and the host operating system accesses the system services of the robot subsystem. In this way, two systems reuse one kernel, avoiding the waste of software and hardware resources caused by multiple kernels.

[0021] In some embodiments, the robot subsystem 102 is coupled to a first resource 108 for controlling the robot through the kernel 106 , and the host operating system 104 is coupled to a second resource 110 for interacting with a user and a third resource 112 for displaying a screen through the kernel 106 .

[0022] In some embodiments, the host operating system 104 obtains a first system service of the robot subsystem 102 for controlling the first resource 108, and controls the first resource 108 to execute a control service instruction for controlling the robot through the first system service.

[0023] Figure 2 2 is a flowchart showing a process 200 of controlling a robot by a host operating system in a system according to an embodiment of the present disclosure. Figure 2 , the process 200 includes steps 202 to 210.

[0024] In step 202, the host operating system calls the first permission management service of the robot subsystem to query whether the host operating system has permission to access the first system service (such as SLAM service, motion control service or posture control service). If the host operating system has access permission, step 204 is executed.

[0025] In step 204, the host operating system obtains the interface of the first system service through the service management function called by the robot subsystem. Wherein, obtaining the interface of the first system service is to obtain an instance of the first system service. In some embodiments, if the first system service is a SLAM service, its interface can be correspondingly represented as SlamService vslamService = (SlamService) getSystemService (Context.SLAM_SERVICE).

[0026] In step 206, the host operating system uses the interface of the first system service to obtain a set of function interfaces for robot control related operations. If the first system service is a SLAM service, the host operating system can obtain a set of function interfaces for SLAM related operations.

[0027] When the host operating system needs to access the system service of the robot subsystem, the host operating system will create a client of the system service. At the same time, the service corresponding to the robot subsystem and the system service will respond to the access operation of the client, thereby providing corresponding system services (SLAM service, motion control service and posture control service) through communication between the service and the client.

[0028] In step 208, the host operating system carries the control parameters and calls the function interface to control the first resource of the robot subsystem to perform the corresponding operation. If the first system service is a SLAM service, the host operating system controls the SLAM sensor of the robot subsystem to perform the corresponding operation.

[0029] In step 210, the host operating system obtains the execution result and updates the control information of the host application.

[0030] If the first system service is a SLAM service, the host operating system can obtain the latest indoor positioning information from the robot subsystem, thereby updating the indoor positioning information in the host application. If the first system service is a posture control service, the host operating system can obtain the latest 6DOF (six degrees of freedom) posture information from the robot subsystem, thereby updating the posture information of the virtual digital human in the host application. If the first system service is a motion control service, the host operating system can obtain the latest speed / acceleration information from the robot subsystem, thereby updating the motion information of the game character in the host application.

[0031] In some embodiments, the robot subsystem 102 obtains the second system service of the host operating system 104 for controlling the second resource 110, and controls the second resource 110 to execute interactive service instructions for interacting with the user through the second system service.

[0032] Figure 3 3 is a flow chart showing a process 300 of interaction between a robot subsystem and a user in a system according to an embodiment of the present disclosure. Figure 3 , the process 300 includes steps 302 to 310.

[0033] In step 302 , the robot subsystem calls the second permission management service of the host operating system to query whether the robot subsystem has permission to access the second system service (such as Wifi communication). If the robot subsystem has access permission, step 304 is executed.

[0034] In step 304, the robot subsystem calls the service management function of the host operating system to obtain the interface of the Wifi communication service. Obtaining the interface of the Wifi communication service is to obtain an instance of the Wifi communication service. In some embodiments, the interface of the Wifi communication service can be represented as WifiService wifiService = (WifiService)getSystemService(Context.WIFI_SERVICE).

[0035] In step 306, the robot subsystem uses the interface of the Wifi communication service to obtain a set of function interfaces for network communication related operations.

[0036] When the robot subsystem needs to access the communication system service of the host operating system, the robot subsystem will create a client of the communication system service. At the same time, the host operating system and the service corresponding to the communication system service will respond to the access operation of the client, thereby providing the corresponding communication system service through communication between the service and the client.

[0037] In step 308, the robot subsystem calls the function interface to control the network communication interface of the host operating system to perform corresponding operations to access the external network.

[0038] In step 310, the robot subsystem obtains the execution result and updates the internal state of the network communication interface.

[0039] In some embodiments, the robot subsystem 102 obtains a rendering client used by the host operating system 104 to render images, and controls the third resource 112 to execute a display service instruction for displaying images through the rendering client.

[0040] In some embodiments, the third resource for displaying the screen includes a display device (such as a CVBS display screen or an HDMI display). Since the robot subsystem is not equipped with a display device, the window system of the robot subsystem needs to be displayed through the display device of the host operating system.

[0041] Figure 4 4 is a flow chart showing a process 400 of displaying a robot subsystem in a system according to an embodiment of the present disclosure. Figure 4 , the process 400 includes steps 402 to 406.

[0042] In step 402, the host operating system responds to the start instruction of the robot virtual window application and creates a rendering client and a rendering server through its display system service.

[0043] In step 404, the robot subsystem obtains the rendering client, routes the rendering client to the rendering server of the host operating system to complete the rendering, and the host operating system returns the rendering result to the rendering client of the robot subsystem through the rendering server. At the same time, the robot subsystem renders the window system to be displayed to the rendering client.

[0044] In step 406, the host operating system renders the rendering result to the physical display interface of the display device through its display system service, so that the display device configured by the host operating system can display the window system of the robot subsystem.

[0045] In some embodiments, after the kernel 106 and the host operating system 104 complete the initialization service, the robot subsystem 102 sequentially initializes the user-mode device driver, the user-mode file system, the ROS service package, and the ROS application package to complete the initialization instructions.

[0046] Figure 5 is a flow chart showing a process 500 of implementing initialization in a system according to an embodiment of the present disclosure. Figure 5 , the process 500 includes steps 502 to 508.

[0047] In step 502, the kernel responds to the initialization instruction and initializes the basic software and hardware environment, such as CPU, memory, device driver and interrupt initialization.

[0048] In step 504, the host operating system responds to the startup instruction and starts the root user. The root user is a super user in the Linux system and has full access rights to the system and can perform any operation, including modifying system files, installing software, and managing user accounts.

[0049] In step 506, the robot subsystem responds to the initialization instruction and initializes the user mode device driver, that is, initializes the external device driver that the robot subsystem needs to directly access and manage.

[0050] In step 508, the robot subsystem initializes and starts the user-mode file system, the ROS service package, and the ROS application package in sequence.

[0051] In another aspect of the present invention, a robot device is provided. The robot device includes a single chip and a single operating system. The single system includes the system and the kernel as described in the above embodiment. The system includes a host operating system and a user-mode robot subsystem.

[0052] In summary, the present invention provides a system and a robot device for operating a robot, wherein a host operating system and a robot subsystem are respectively arranged on the same kernel, so that the two systems reuse one kernel, thereby avoiding the waste of software and hardware resources caused by multiple kernels, and effectively reducing the system redundancy caused by the dual systems on the basis of ensuring the dual system functions. Resources used to implement different functions are coupled to the two systems respectively, thereby avoiding system redundancy caused by resource duplication. At the same time, the two systems can indirectly control the resource execution instructions of the other system through the system service corresponding to each resource, so that the system can control the resources coupled to the other system to implement the specified functions when the resources are not coupled. The resource sharing of the dual systems can be realized on one chip without the need for hardware virtualization operations, thereby effectively reducing the system performance loss.

[0053] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A system for operating a robot, characterized in that: include: A user-mode robot subsystem, configured to be responsible for functions associated with the robot; as well as A host operating system configured to provide a user interface and interaction to the robot subsystem, The robot subsystem and the host operating system are both coupled to the same kernel to reuse the kernel, the robot subsystem can access system services of the host operating system, and the host operating system can access system services of the robot subsystem.

2. The system according to claim 1, characterized in that The robot subsystem is coupled to a first resource for controlling a robot through the kernel, The host operating system is coupled to a second resource for interacting with a user and a third resource for displaying a screen through the kernel.

3. The system according to claim 2, characterized in that The host operating system obtains a first system service of the robot subsystem for controlling the first resource, and controls the first resource through the first system service to execute a control service instruction for controlling the robot.

4. The system according to claim 3, characterized in that The host operating system calls the first permission management service of the robot subsystem to query whether the host operating system has permission to access the first system service; as well as If so, call the service management function of the robot subsystem to obtain the first system service.

5. The system according to claim 3, characterized in that The host operating system receives control parameters input by the user through a host application for controlling the robot, and calls the first system service according to the control parameters to control the first resource to execute a control service instruction for controlling the robot.

6. The system according to claim 5, characterized in that The host operating system obtains the control execution result of the control service instruction through the first system service, and updates the control information in the host application according to the control execution result.

7. The system according to claim 2, characterized in that The robot subsystem obtains the second system service of the host operating system for controlling the second resource, and controls the second resource to execute an interactive service instruction for interacting with a user through the second system service.

8. The system according to claim 7, characterized in that The robot subsystem calls the second permission management service of the host operating system to query whether the robot subsystem has permission to access the second system service; as well as If so, call the service management function of the host operating system to obtain the second system service.

9. The system according to claim 7, characterized in that The robot subsystem obtains an operation function related to user interaction through the second system service, and calls the operation function to control the second resource to execute an interactive service instruction.

10. The system according to claim 7, characterized in that The robot subsystem obtains the interactive execution result of the interactive service instruction through the second system service, and updates the system interactive state according to the interactive execution result.

11. The system according to claim 2, characterized in that The robot subsystem obtains a rendering client used by the host operating system to render a picture, and controls a third resource through the rendering client to execute a display service instruction for displaying a picture.

12. The system according to claim 11, characterized in that The robot subsystem renders the window to be displayed to the rendering client, and routes the rendering client to the rendering server of the host operating system for rendering the screen, so as to render the window to be displayed to the third resource for output display.

13. The system according to claim 1, characterized in that After the kernel and the host operating system complete the initialization service, the robot subsystem sequentially initializes the user-mode device driver, the user-mode file system, the ROS service package, and the ROS application package to complete the initialization instruction.

14. A robot device, characterized in that: include: Single chip; as well as A single operating system comprising the system according to any one of claims 1 to 13 and the kernel.

Images