Robot heterogeneous controller, system and control method based on resource pooling

Abstract

The invention relates to the technical field of intelligent control, in particular to a robot heterogeneous controller, system and control method based on resource pooling, and the controller comprises a user interface which is connected with a robot; the plurality of hardware processors provide a plurality of computing capabilities; the system comprises a plurality of hardware processors, a data bus and a virtual machine resource pool manager, the virtual machine resource pool manager is connected with the plurality of hardware processors through the data bus, the virtual machine resource pool manager obtains a target task and operation data of a robot through a user interface, and a plurality of virtual operation systems are generated according to the target task; and the multiple virtual operating systems determine at least one computing task according to the target task and the running data, and call the multiple hardware processors to execute the at least one computing task. Therefore, the problems of poor scheduling flexibility, insufficient task processing capability and the like in the process of coping with diversified task execution of the robot based on a modular controller computing unit scheduling mechanism in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to a robot heterogeneous controller, system and control method based on resource pooling. Background Art

[0002] Humanoid robots often face the challenge of processing multiple tasks in parallel when performing tasks in their environments. This requires the real-time execution and scheduling of multiple algorithms and models across the entire chain, from perception to decision-making to motion execution. Therefore, heterogeneous controllers are a key technology for humanoid robot control systems and have become a key area of focus in the field of robotic control systems.

[0003] Heterogeneous controllers in related technologies modularly introduce CPUs (Central Processing Units) and GPUs (Graphics Processing Units). The CPU uses chips based on X86 or ARM architectures to perform computational and scheduling tasks such as motion control, which require high real-time performance and floating-point computing capabilities. The GPU uses massively parallel processing units to perform tasks such as perception and decision-making, which require massively parallel real-time processing of multimodal sensor data. However, modularized controller computing unit scheduling can only achieve isolation and decoupling between different processing programs. Users need to allocate and schedule the processing themselves, making it difficult to implement a scheduling interface that meets the different computing needs of different users for the robot, and unable to complete complex task allocation for multiple users. Summary of the Invention

[0004] The present application provides a robot heterogeneous controller, system and control method based on resource pooling to solve the problems of poor scheduling flexibility and insufficient task processing capability in the process of executing diverse tasks of robots based on the scheduling mechanism of modular controller computing units in related technologies.

[0005] The first aspect of the present application provides a robot heterogeneous controller based on resource pooling, including: a user interface, the user interface is connected to the robot; multiple hardware processors, the multiple hardware processors provide multiple computing capabilities; a data bus and a virtual machine resource pool manager, wherein the virtual machine resource pool manager is connected to the multiple hardware processors via the data bus, the virtual machine resource pool manager obtains the robot's target tasks and operating data through the user interface, generates multiple virtual operating systems according to the target tasks, the multiple virtual operating systems determine at least one computing task according to the target tasks and operating data, and call multiple hardware processors to execute the at least one computing task.

[0006] Optionally, multiple virtual operating systems run synchronously in isolation from each other.

[0007] Optionally, the computing task includes at least one of a motion control task, a perception task, and a decision-making task.

[0008] Optionally, the multiple virtual operating systems include a real-time virtual operating system and a non-real-time virtual operating system, wherein the real-time virtual operating system is used to execute motion control tasks, and the non-real-time virtual operating system is used to execute perception tasks and decision-making tasks.

[0009] Optionally, the multiple hardware processors include multiple first computing power units and multiple second computing power units, wherein the computing capabilities of the first computing power units are different from those of the second computing power units.

[0010] Optionally, the first computing unit provides high-speed serial floating-point computing capabilities, and the second computing unit provides large-scale parallel deep neural network inference computing capabilities.

[0011] Optionally, the data bus includes Ethernet, PCIe and IIC.

[0012] Optionally, the virtual machine resource pool manager is implemented based on an ARM platform using hardware virtualization technology.

[0013] A second aspect of the present application provides a heterogeneous robot controller resource pooling system, comprising: a robot and a robot heterogeneous controller based on resource pooling according to the above embodiment, wherein the heterogeneous controller controls the robot to perform a target task.

[0014] The third aspect of the present application provides a control method for a robot heterogeneous controller based on resource pooling. The method is applied to the robot heterogeneous controller based on resource pooling of the above embodiment, and includes the following steps: obtaining the target task and operating data of the robot; generating multiple virtual operating systems according to the target task; in the multiple virtual operating systems, determining at least one computing task according to the target task and operating data, and calling multiple hardware processors to execute the at least one computing task.

[0015] Therefore, this application has the following beneficial effects:

[0016] The virtual machine resource pool manager of the embodiment of the present application obtains the target tasks and operating data of the robot through the user interface, and can build multiple virtual operating systems for users to use based on the target tasks. Users only need to deploy their programs in a virtual system environment without considering the actual hardware interface and task scheduling. The virtual machine resource pool manager calls multiple hardware processors to process according to the computing task requirements generated by each operating system, which can support the robot to complete various tasks in a complex environment.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A block diagram of a robot heterogeneous controller based on resource pooling according to an embodiment of the present application;

[0020] Figure 2 A detailed block diagram of a robot heterogeneous controller based on resource pooling according to one embodiment of the present application;

[0021] Figure 3 This is a block diagram of a heterogeneous robot controller resource pooling system provided according to one embodiment of the present application;

[0022] Figure 4 This is a flowchart of a control method for a robot heterogeneous controller based on resource pooling according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes a robot heterogeneous controller based on resource pooling and its control method and system according to an embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides a robot heterogeneous controller based on resource pooling. The virtual machine resource pool manager obtains the target tasks and operation data of the robot through a user interface, and can build multiple virtual operating systems for users to use based on the target tasks. Users only deploy their programs in a virtual system environment without considering the actual hardware interface and task scheduling. The virtual machine resource pool manager calls multiple hardware processors for processing according to the computing task requirements generated by each operating system, which can support the robot to complete various tasks in a complex environment.

[0025] Figure 1 It is a block diagram of a robot heterogeneous controller based on resource pooling according to an embodiment of the present application.

[0026] like Figure 1 As shown, the resource pooling-based robot heterogeneous controller 10 includes: a user interface 101, multiple hardware processors 102, a data bus 103 and a virtual machine resource pool manager 104.

[0027] Among them, the user interface is connected to the robot; multiple hardware processors 102 provide multiple computing capabilities; the virtual machine resource pool manager 104 is connected to the multiple hardware processors 102 through the data bus 103, and the virtual machine resource pool manager 104 obtains the robot's target task and operating data through the user interface 101, generates multiple virtual operating systems according to the target task, and the multiple virtual operating systems determine at least one computing task according to the target task and operating data, and call multiple hardware processors 102 to execute at least one computing task.

[0028] It is understandable that the virtual machine resource pool manager 104 in the embodiment of the present application is implemented based on the ARM platform using hardware virtualization technology. Figure 2 As shown, the data bus 103 includes Ethernet, PCIe and IIC, and multiple hardware processors 102 are connected through different types of data buses 103 such as Ethernet, PCIe and IIC to form an integrated robot mobile terminal controller. The virtual machine resource pool manager 104 of the embodiment of the present application obtains the target task and operation data of the robot through the user interface 101, and can build multiple virtual operating systems for users to use based on the target task. It can support both real-time systems and non-real-time systems, taking into account the computing requirements of different types of tasks such as motion control, perception and decision-making. Users only deploy their programs in a virtual system environment without considering the actual hardware interface and task scheduling. The virtual machine resource pool manager 104 can automatically allocate to the corresponding hardware processor 102 for processing according to the computing requirements of the virtual operating system, and at the same time use data buses 103 with different bandwidth characteristics for data exchange in combination with different task requirements.

[0029] Therefore, the embodiment of the present application realizes system-level isolation of different users through the resource pool manager, and can support the isolation of real-time systems and non-real-time systems, covering cerebellum real-time tasks for motion control and brain tasks for perception and decision-making, and can support humanoid robots to complete various tasks in complex environments.

[0030] Specifically, if Figure 2 As shown, multiple virtual operating systems (OS1-OSn) are generated based on target task planning and configuration, targeting various real-time and non-real-time computing tasks such as motion control, perception, and decision-making. Users can deploy programs in isolated operating systems, which run synchronously in isolation. Based on the computing task requirements generated by each operating system, the virtual machine resource pool manager 104 distributes computing tasks to multiple hardware processors 102 via three data buses: Ethernet, PCIe, and IIC, and facilitates data exchange. The user interface 101 connects to the robot via multiple data buses, such as Ethernet, CAN, and serial ports, enabling real-time data exchange between sensors and actuators.

[0031] In an embodiment of the present application, the multiple hardware processors 102 include multiple first computing power units and multiple second computing power units.

[0032] For example, the first computing unit may be an X86, and the second computing unit may be a GPU. Figure 2 The first computing unit X86 and the second computing unit GPU have different computing capabilities. The first computing unit X86 provides high-speed serial floating-point computing capabilities, while the second computing unit GPU provides large-scale parallel deep neural network reasoning computing capabilities.

[0033] During the actual execution process, the hardware processor 102 of the embodiment of the present application can be flexibly configured, and can automatically provide the required computing power scale according to the task execution requirements of the entire controller system, and then configure the required number of X86 and GPU boards. The virtual machine resource pool manager 104 schedules the computing resources of X86 and GPU through three data buses: Ethernet, PCIe and IIC, and adopts corresponding data buses 103 for different computing tasks. Among them, the IIC data bus provides low-speed data interaction capabilities to obtain the system status of X86 and GPU, operation monitoring and other data. The Ethernet data bus provides high-speed data interaction capabilities, and can process X86 and GPU data at the same time through networking to achieve collaborative work of multiple processors. The PCIe data bus provides ultra-high-speed data interaction capabilities, and realizes the ultra-high-speed real-time computing task deployment and operation of the virtual operating system and the corresponding X86 or GPU unit in a point-to-point manner.

[0034] In an embodiment of the present application, multiple virtual operating systems run synchronously in an isolated manner, and the multiple virtual operating systems include a real-time virtual operating system and a non-real-time virtual operating system, wherein the real-time virtual operating system is used to execute motion control tasks, and the non-real-time virtual operating system is used to execute perception tasks and decision-making tasks.

[0035] It is understandable that the virtual machine resource pool manager 104 of the embodiment of the present application has an ARM processor hardware platform with a virtualized environment, supports the isolated operation of multiple virtual operating systems, supports direct user access through the virtual operating system interface, and has an isolated operating environment between virtual operating systems. The multiple virtual operating systems include real-time virtual operating systems and non-real-time virtual operating systems. Among them, the real-time virtual operating system has the interrupt capability of the real-time operating system and can provide users with a real-time thread scheduling method suitable for tasks such as motion control. The non-real-time virtual operating system has a scheduling method for large-scale parallel tasks such as perception and decision-making, supports real-time acquisition of multi-modal sensor data of robots, supports large-scale parallel deep neural network reasoning, can automatically allocate parallel processing units and obtain reasoning results in real time.

[0036] In the embodiment of the present application, computing tasks can be divided into perception tasks, decision-making tasks, and motion control tasks. Motion control tasks require high real-time and high serial processing capabilities and are assigned to real-time virtual operating systems corresponding to X86 computing resources. Perception and decision-making tasks require large-scale parallel processing capabilities and are assigned to non-real-time virtual operating systems corresponding to GPU computing resources.

[0037] It should be noted that the real-time and non-real-time virtual operating systems in the embodiment of the present application run in isolation and do not affect each other, and the real-time system runs in a high-priority manner.

[0038] According to the resource pooling-based robot heterogeneous controller proposed in the embodiment of the present application, the virtual machine resource pool manager obtains the robot's target tasks and operating data through the user interface, and can build multiple virtual operating systems for users to use based on the target tasks. Users only deploy their programs in a virtual system environment without having to consider the actual hardware interface and task scheduling. The virtual machine resource pool manager calls multiple hardware processors to process according to the computing task requirements generated by each operating system, which can support the robot to complete various tasks in a complex environment.

[0039] Next, the heterogeneous robot controller resource pooling system 20 proposed in accordance with the embodiment of the present application is described with reference to the accompanying drawings. Figure 3 As shown, it includes: a robot 201 and a robot heterogeneous controller 10 based on resource pooling of the above embodiment, and the heterogeneous controller 10 controls the robot 201 to perform the target task.

[0040] In addition, the embodiment of the present application also provides a control method for a robot heterogeneous controller based on resource pooling, which is applied to the robot heterogeneous controller based on resource pooling in the above embodiment, such as Figure 4 As shown, the following steps are included:

[0041] In step S101 , the target task and operation data of the robot are obtained.

[0042] In step S102 , a plurality of virtual operating systems are generated according to the target task.

[0043] In step S103, in the multiple virtual operating systems, at least one computing task is determined according to the target task and the operating data, and multiple hardware processors are called to execute the at least one computing task.

[0044] It should be noted that the explanation of the control method embodiment of the robot heterogeneous controller based on resource pooling in the embodiment of the present application can refer to the execution process of the above-mentioned robot heterogeneous controller based on resource pooling, which will not be repeated here.

[0045] According to the control method of the robot heterogeneous controller based on resource pooling proposed in the embodiment of the present application, the target tasks and operation data of the robot are obtained, and multiple virtual operating systems can be built for users to use based on the target tasks. Users only deploy their programs in a virtual system environment without considering the actual hardware interface and task scheduling. The virtual machine resource pool manager calls multiple hardware processors to process according to the computing task requirements generated by each operating system, which can support the robot to complete various tasks in a complex environment.

[0046] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0047] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0048] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0049] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, the steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement the method: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0050] A person skilled in the art may understand that all or part of the steps carried out in the method for implementing the above-mentioned embodiment may be completed by instructing the relevant hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0051] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A robot heterogeneous controller based on resource pooling, characterized in that: include: a user interface, the user interface being connected to the robot; a plurality of hardware processors, the plurality of hardware processors providing a variety of computing capabilities; A data bus and a virtual machine resource pool manager, wherein the virtual machine resource pool manager is connected to the multiple hardware processors via the data bus, the virtual machine resource pool manager obtains the target task and operating data of the robot via the user interface, generates multiple virtual operating systems based on the target task, the multiple virtual operating systems determine at least one computing task based on the target task and the operating data, and call the multiple hardware processors to execute the at least one computing task.

2. The resource pooling-based robot heterogeneous controller according to claim 1, characterized in that: The multiple virtual operating systems run synchronously in a mutually isolated manner.

3. The robot heterogeneous controller based on resource pooling according to claim 1 is characterized in that: The computing task includes at least one of a motion control task, a perception task, and a decision-making task.

4. The resource pooling-based robot heterogeneous controller according to claim 3, characterized in that: The multiple virtual operating systems include a real-time virtual operating system and a non-real-time virtual operating system, wherein the real-time virtual operating system is used to execute motion control tasks, and the non-real-time virtual operating system is used to execute perception tasks and decision-making tasks.

5. The robot heterogeneous controller based on resource pooling according to claim 1, characterized in that: The multiple hardware processors include multiple first computing units and multiple second computing units, wherein the computing capabilities of the first computing units are different from those of the second computing units.

6. The resource pooling-based robot heterogeneous controller according to claim 5, characterized in that: The first computing power unit provides high-speed serial floating-point computing capabilities, and the second computing power unit provides large-scale parallel deep neural network inference computing capabilities.

7. The robot heterogeneous controller based on resource pooling according to claim 1, characterized in that: The data bus includes Ethernet, PCIe and IIC.

8. The robot heterogeneous controller based on resource pooling according to claim 1, characterized in that: The virtual machine resource pool manager is implemented based on the ARM platform using hardware virtualization technology.

9. A heterogeneous robot controller resource pooling system, characterized in that: include: robot; The resource pooling-based robot heterogeneous controller according to any one of claims 1 to 8, wherein the heterogeneous controller controls the robot to perform a target task.

10. A control method for a robot heterogeneous controller based on resource pooling, characterized in that: The method is applied to the resource pooling-based heterogeneous robot controller according to any one of claims 1 to 8, wherein the method comprises the following steps: Obtain the robot's target tasks and operating data; generating a plurality of virtual operating systems according to the target tasks; In the multiple virtual operating systems, at least one computing task is determined according to the target task and the operating data, and multiple hardware processors are called to execute the at least one computing task.

Images