Robot driving control method, system and device, electronic equipment and storage medium

Abstract

The invention discloses a robot driving control method, system and device, electronic equipment and a storage medium. The robot driving control method comprises the following steps: loading and analyzing an OpenVX graph to obtain a plurality of nodes to be executed, a dependency relationship among the plurality of nodes and a binding relationship between the plurality of nodes and a plurality of cores; according to the dependency relationship and the binding relationship, allocating the plurality of nodes to a plurality of cores of the multi-core heterogeneous chip for execution; wherein each node is an operation algorithm, and the plurality of nodes at least comprise a control algorithm, a driving algorithm and a deep learning algorithm of the robot; the deep learning algorithm is used for performing deep learning to obtain optimal control parameters of the control algorithm; the control algorithm is used for acquiring sensing data output by robot sensing equipment and making a robot behavior decision according to the optimal control parameter and the sensing data; the driving algorithm is used for driving an execution mechanism of the robot to act according to the control algorithm operation result. The method is realized on the basis of a multi-core heterogeneous chip, and the size and cost of the robot can be reduced.

Description

Technical Field

[0001] This application relates to the technical field of robot control, and particularly relates to a robot driving and control method, system, device, electronic device, and storage medium. Background Art

[0002] A robot is an automated device that can perform autonomous or predefined tasks. It usually has a mechanical structure, electronic components, and a computer control system, etc., and can sense the environment, make decisions, and execute specific actions or tasks. The more intelligent a robot is, the more chips are usually required to achieve complex movements and decisions. This is because a highly intelligent robot needs more computing and processing power to sense, analyze, and respond to environmental changes in order to better execute tasks. The increase in the number of chips will lead to an increase in the volume and cost of the robot. Summary of the Invention

[0003] The purpose of this application is to propose a robot driving and control method, system, device, electronic device, and storage medium, which is implemented based on a multi-core heterogeneous chip, and is beneficial to reducing the volume and cost of the robot.

[0004] To achieve the above purpose, an embodiment of this application provides a robot driving and control method, and the method includes: Loading and parsing an OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and multiple cores of a multi-core heterogeneous chip; According to the dependency relationships and the binding relationships, allocating the multiple nodes to multiple cores of the multi-core heterogeneous chip for execution; Wherein, each node is an operation algorithm, and the multiple nodes at least include a control algorithm, a driving algorithm, and a deep learning algorithm of the robot; the deep learning algorithm is used to perform deep learning to obtain the optimal control parameters of the control algorithm; the control algorithm is used to obtain sensing data output by a robot sensing device, and make a robot behavior decision according to the optimal control parameters and the sensing data; the driving algorithm is used to drive the action of an execution mechanism of the robot according to the operation result of the control algorithm.

[0005] An embodiment of this application also provides a robot driving and control system, and the system includes: A graph loading module, configured to load and parse an OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and multiple cores of a multi-core heterogeneous chip; A node allocation module, configured to allocate the multiple nodes to multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships; Among them, each node is an operation algorithm, and the multiple nodes at least include a control algorithm, a driving algorithm, and a deep learning algorithm of the robot; the deep learning algorithm is used to perform deep learning to obtain the optimal control parameters of the control algorithm; the control algorithm is used to obtain the sensing data output by the robot sensing device, and make a robot behavior decision according to the optimal control parameters and the sensing data; the driving algorithm is used to drive the action of the robot actuator according to the operation result of the control algorithm.

[0006] An embodiment of the present application further provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned robot driving and controlling method is implemented.

[0007] An embodiment of the present application further provides a robot driving and controlling device, including a multi-core heterogeneous chip and the above-mentioned electronic device; The multi-core heterogeneous chip includes multiple cores; The electronic device is used to load and parse the OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and the multiple cores of the multi-core heterogeneous chip, and according to the dependency relationships and the binding relationships, allocate the multiple nodes to the multiple cores of the multi-core heterogeneous chip for execution.

[0008] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle control method is implemented.

[0009] An embodiment of the present application provides a robot driving and controlling method, system, device, electronic device, and storage medium. By constructing an OpenVX middleware, the OpenVX middleware is used to load and parse the OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and the multiple cores of the multi-core heterogeneous chip, and according to the dependency relationships and the binding relationships, allocate the multiple nodes to the multiple cores of the multi-core heterogeneous chip for parallel execution, that is, intelligent robot driving and controlling can be realized based on a single multi-core heterogeneous chip. Therefore, the number of required chips is reduced, and the volume and cost of the robot are reduced; at the same time, using a deep learning algorithm to optimize the control parameters of the control algorithm can improve the autonomous decision-making ability of the robot and make the robot control more intelligent. Description of the Drawings

[0010] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0011] Figure 1 This is a flowchart of a robot driving and control method in an embodiment of the present application.

[0012] Figure 2 This is a schematic diagram of the OpenVX middleware in an embodiment of the present application.

[0013] Figure 3 This is a control schematic diagram of a robot driving and control method in an embodiment of the present application.

[0014] Figure 4 This is a framework structure diagram of a robot driving and control system in an embodiment of the present application.

[0015] Figure 5 This is a circuit diagram of an intelligent robot product in an embodiment of the present application. Detailed Description of the Specific Embodiments

[0016] The detailed description of the accompanying drawings is intended to illustrate some current embodiments of the present application, rather than representing the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be completed by different embodiments intended to be included within the scope of the present application.

[0017] An embodiment of the present application provides a robot driving and control method. This embodiment method can be implemented based on an OpenVX middleware. Refer to Figures 1 - 2 , the method of this embodiment includes the following steps: Step S10, load and parse the OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and multiple cores of the multi-core heterogeneous chip.

[0018] Specifically, OpenVX includes five components: context, kernel, parameter, node, and graph. Among them, the context is the scope where all data objects and all framework docking exist. The context stores the reference count of all objects and performs garbage collection during the deconstruction process to release lost references. The kernel encapsulates various visual function interfaces, and can register callback functions or directly call the interfaces of visual functions. The parameter is the input and output passed to the function. The node is an operation algorithm or operation, and multiple nodes form a graph to complete a certain function. The node is only created from a certain graph and associated with a single graph. The graph is a set of nodes connected in a directed (only one-way) acyclic manner, and the nodes are connected together through data dependencies.

[0019] It should be noted that the operation of OpenVX is based on nodes. All nodes will be linked to the graph and finally run by the graph uniformly. Therefore, by loading and parsing the OpenVX graph, multiple nodes to be executed included in the OpenVX graph, the dependency relationships between multiple nodes, and the binding relationships between multiple nodes and multiple cores can be obtained. One node is bound to one core, and one core can be bound to multiple nodes, but each core only executes one node at a time.

[0020] Step S20: Allocate the multiple nodes to multiple cores of the multi-core heterogeneous chip according to the dependency relationship and the binding relationship.

[0021] Specifically, in robot applications, different algorithms / tasks may require different computing resources, and the computing resources of different types of cores are different. By correspondingly binding multiple nodes to multiple cores of the multi-core heterogeneous chip, appropriate nodes can be allocated to the corresponding cores for execution according to the nature and computing requirements of the nodes. In this way, parallel processing of tasks and effective utilization of the computing power of the multi-core heterogeneous chip can be achieved, and the execution efficiency and performance of the tasks can be improved.

[0022] In this embodiment, the multiple nodes at least include the control algorithm, drive algorithm, and deep learning algorithm of the robot. The deep learning algorithm is used to perform deep learning to obtain the optimal control parameters of the control algorithm. The control algorithm is used to obtain the sensing data output by the robot sensing device and make robot behavior decisions according to the optimal control parameters and the sensing data. The drive algorithm is used to drive the actuator of the robot according to the operation result of the control algorithm.

[0023] Specifically, referring to Figure 3 , the robot drive and control method of this embodiment mainly involves the control algorithm, drive algorithm, and deep learning algorithm of the robot; The control algorithm is the core algorithm for robot driving and control. It is responsible for formulating the optimal robot behavior decision based on the robot's sensing data and target tasks. The main function of the control algorithm is to analyze and process the sensing data to achieve the robot's autonomous decision-making ability. The control algorithm includes, but is not limited to, PID control, behavior trees, state machines, etc. By analyzing and judging the robot's sensing data, the control algorithm can determine which actions and behaviors the robot should take to achieve the predetermined control objectives. The robot's sensing data includes, for example, environmental data (temperature, humidity, light, etc. of the surrounding environment), obstacle data (obstacle position, speed, movement direction, acceleration, etc.), robot motion data (robot's own position, speed, movement direction, acceleration, etc.), robot pose data (posture, tilt, and rotation, etc.). The specific content of the robot's sensing data can be determined specifically in combination with the robot type and application scenario. By comprehensively using these sensor data, the robot can obtain comprehensive environmental and self-information and perform perception, analysis, and response in the robot's control and decision-making process. In this way, the robot can make intelligent decisions and actions based on the perceived data and better adapt to different task and environmental requirements.

[0024] The driving algorithm is used to drive the actions of the robot's actuators according to the operation results of the control algorithm. Specifically, the driving algorithm is responsible for converting the control instructions into action signals for the robot's actuators, enabling the robot to actually execute the corresponding actions according to the decisions of the control algorithm. The driving algorithm is, for example, a position loop, a speed loop, etc. The robot's actuators refer to the components or assemblies on the robot's body that are responsible for executing various actions and tasks. The specific type of actuator will vary according to different robot types and applications. The following are some common robot actuators: (1.1) Joints: Robots often use joints to achieve body movement, such as rotation, bending, or extension. Joints can be rotary joints, sliding joints, or spherical joints, etc. The joints of a robot are usually driven by motors and can control the joint position and speed. (1.2) Wheels or tracks: Some robots, such as ground mobile robots and unmanned vehicles, use wheels or tracks to achieve movement. These actuators can be driven by motors, and the movement of the robot can be controlled by controlling the speed and direction of the wheels. (1.3) Arms and grippers: Some robots are equipped with multi-joint robotic arms that can perform tasks such as grasping, placing, and operating objects. The robotic arm usually consists of multiple joints and actuators, and the movement of the arm can be controlled by controlling the position and force of the joints. Tools such as grippers or suction cups can also be equipped at the end of the robot's arm to adapt to different operation requirements. (1.4) Servos and servo systems: A servo is a special type of electric motor that can be used to drive various parts of a robot, such as eyes, mouth, fingers, etc.; A servo system is a closed-loop control system that can control an actuator to reach a specific position or angle, such as servos and stepper motors; The selection of the above-mentioned actuators depends on the type, function, and application scenario of the robot; When designing and building a robot, it is necessary to select a suitable actuator according to the actual needs to achieve the actions and tasks of the robot.

[0025] In robot applications, deep learning algorithms are usually applied to tasks such as image and speech processing, object detection and recognition, path planning, etc. In the robot driving and control method of this embodiment, a new application of deep learning algorithms is proposed, that is, to perform deep learning to obtain the optimal control parameters of the control algorithm (including but not limited to PID control, behavior trees, state machines, etc.), so that the control parameters of the control algorithm can be adaptively adjusted according to the application requirements and environmental changes of the robot, so as to improve the accuracy and adaptability of the control algorithm; For example, a neural network model can be constructed and trained to learn the complex mapping relationship between the input and output of the control algorithm, and the control parameters can be fitted to make the control algorithm reach the expected output; By using deep learning algorithms to optimize the control parameters, the control algorithm can be made more intelligent and adaptive, so as to better meet the needs of different scenarios and tasks; Such a method can improve the performance and adaptability of the control algorithm and reduce the need for manual adjustment, making robot control more intelligent.

[0026] In some embodiments, the multiple nodes further include a multimedia algorithm; The multimedia algorithm is used to obtain the image data output by the robot camera and process the image data; The control algorithm is used to make robot behavior decisions based on the optimal control parameters, the sensing data, and the operation results of the multimedia algorithm.

[0027] Specifically, the robot of this embodiment is equipped with a camera, mainly to obtain visual perception ability. A robot with visual perception ability can observe and understand the surrounding environment through the camera, so as to better adapt to and interact with the real world; The following are some common examples of multimedia algorithms for processing image data: (2.1) Image filtering: including mean filtering, median filtering, Gaussian filtering, etc., used to smooth images, remove noise, or enhance image details; (2.2) Edge detection: including Sobel operator, Canny edge detection, etc., used to detect the edges of objects in images; (2.3) Feature extraction: including Harris corner detection, SIFT feature extraction, etc., used to extract specific features in the image, such as corners, key points, etc.; (2.4) Image segmentation: including threshold-based and region-based segmentation algorithms, which are used to divide images into different regions or objects; (2.5) Image restoration and enhancement: including algorithms based on compensation, interpolation, denoising, etc., used to repair damaged images, enhance image clarity, or restore low-light images; The multimedia algorithms listed above are only a part of the many multimedia algorithms. Each algorithm has its specific purpose and application scenario. According to different needs, you can choose a suitable algorithm or combine multiple algorithms to process and analyze image data.

[0028] In some embodiments, the multiple cores include at least one Cortex-A core, two C66DSP cores, one C7DSP core, and one Cortex-R core.

[0029] The Cortex-A core is used to implement custom protocols and peripheral access; specifically, the Cortex-A core is a high-performance application processor core suitable for processing complex tasks and running operating systems. The Cortex-A core is used to implement custom protocols and peripheral access because these tasks usually require higher processing power and flexibility, and have more interactions with the operating system.

[0030] One of the C66DSP cores is bound to the node of the control algorithm, and the other C66DSP core is bound to the node of the drive algorithm; specifically, the C66DSP core is a digital signal processor core for signal processing and floating-point calculations, suitable for high-performance real-time computing; control algorithms usually require high-performance real-time computing, while drive algorithms require high-performance signal processing capabilities. The C66DSP core can provide high-speed and low-latency computing on these two nodes to meet the requirements of real-time control and drive algorithms.

[0031] The C7 DSP core is bound to the node of the deep learning algorithm; specifically, the C7 DSP core is a digital signal processor core specially used for digital signal processing and embedded computing, and is suitable for running complex mathematical operations; because deep learning algorithms usually require a large number of matrix operations and complex mathematical calculations, the C7 DSP core has high-performance parallel computing capabilities, which can accelerate the operation of deep learning algorithms and improve processing speed and efficiency.

[0032] The Cortex-R core is bound to the nodes of the multimedia algorithm; specifically, the Cortex-R core is an embedded processor core dedicated to real-time applications, suitable for real-time signal processing and embedded system control; since multimedia algorithms usually require real-time signal processing and embedded system control, the Cortex-R core has low latency and high real-time performance, making it suitable for handling the real-time requirements of multimedia algorithms.

[0033] Specifically, the multi-core heterogeneous chip contains different types of cores, and each core is responsible for a specific type of algorithm or task. This design can improve the performance and efficiency of the entire system, enabling different types of algorithms to run in parallel on different cores to achieve multi-task processing and real-time performance requirements.

[0034] In some embodiments, loading and parsing the OpenVX graph to obtain multiple nodes to be executed specifically includes: Obtain the current working scenario of the robot, and load the corresponding OpenVX graph according to the current working scenario of the robot; where the OpenVX graphs for different scenarios are different.

[0035] Specifically, different working scenarios of the robot may involve different vision tasks and processing requirements. To meet the requirements in different scenarios, different OpenVX graphs can be designed for each scenario. These graphs can connect different OpenVX image processing nodes in a specific order and parameter configuration to form an executable image processing flow; by loading different OpenVX graphs, the robot can perform corresponding image processing according to the current working scenario, that is, the robot can flexibly execute specific image processing tasks according to the requirements of different scenarios, thus providing more intelligent and adaptable functions.

[0036] Another embodiment of the present application provides a robot drive and control system. The system of this embodiment can be used to execute the method of the above embodiment. Refer to Figure 4 and the system of this embodiment includes: A graph loading module 1, configured to load and parse the OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and the multiple cores of the multi-core heterogeneous chip; A node allocation module 2, configured to allocate the multiple nodes to the multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships; Among them, each node is an operation algorithm, and the multiple nodes at least include the control algorithm, driving algorithm, and deep learning algorithm of the robot; the deep learning algorithm is used to perform deep learning to obtain the optimal control parameters of the control algorithm; the control algorithm is used to obtain the sensing data output by the robot sensing device, and make robot behavior decisions according to the optimal control parameters and the sensing data; the driving algorithm is used to drive the action of the robot's actuator according to the operation result of the control algorithm.

[0037] In some embodiments, the multiple nodes further include a multimedia algorithm; the multimedia algorithm is used to obtain the image data output by the robot camera and process the image data; the control algorithm is used to make robot behavior decisions according to the optimal control parameters, the sensing data, and the operation result of the multimedia algorithm.

[0038] In some embodiments, the multiple cores at least include one Cortex-A core, two C66 DSP cores, one C7 DSP core, and one Cortex-R core; The Cortex-A core is used to implement custom protocols and peripheral access; One of the C66 DSP cores is bound to the node of the control algorithm, and the other C66 DSP core is bound to the node of the driving algorithm; The C7 DSP core is bound to the node of the deep learning algorithm; The Cortex-R core is bound to the node of the multimedia algorithm.

[0039] In some embodiments, the graph loading module 1 is specifically configured to obtain the current working scenario of the robot and load the corresponding OpenVX graph according to the current working scenario of the robot; where the OpenVX graphs for different scenarios are different.

[0040] The vehicle control device of the above-described embodiments is merely illustrative. The modules described as separate components may or may not be physically separated. The components as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the vehicle control device in the embodiments.

[0041] It should be noted that the vehicle control device in the above embodiments corresponds to the vehicle control method in the above embodiments. Therefore, the parts not detailed in the vehicle control device in the above embodiments can be obtained by referring to the content of the vehicle control method in the above embodiments, and will not be elaborated here.

[0042] Another embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the robot driving and controlling method described in the above embodiment is implemented.

[0043] Among them, the electronic device can be understood as a device carrying the OpenVX middleware. The electronic device may further include a bus connecting different components (including the memory and the processor). The memory may include a computer-readable medium in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The memory may also include at least one program product having a set (for example, at least one) of program modules configured to perform the functions of the embodiments of the present application.

[0044] Another embodiment of the present application provides a robot driving and controlling device, which is characterized by including a multi-core heterogeneous chip and the electronic device described in the above embodiment; The multi-core heterogeneous chip includes multiple cores; The electronic device is configured to load and parse the OpenVX graph to obtain a plurality of nodes to be executed, the dependency relationships between the plurality of nodes, and the binding relationships between the plurality of nodes and the multiple cores of the multi-core heterogeneous chip, and allocate the plurality of nodes to the multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships.

[0045] Specifically, the electronic device is a hardware device carrying the OpenVX middleware. In order to reduce the number and area of chips, in a specific embodiment, the electronic device may be integrated with the multi-core heterogeneous chip into a main control processing chip, that is, one core in the main control processing chip serves as the OpenVX middleware, and is configured to load and parse the OpenVX graph to obtain a plurality of nodes to be executed, the dependency relationships between the plurality of nodes, and the binding relationships between the plurality of nodes and the other multiple cores of the multi-core heterogeneous chip, and allocate the plurality of nodes to the other multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships.

[0046] As Figure 5The figure shows a circuit diagram of an intelligent robot product including the main control processing chip. The intelligent robot product includes a main control processing chip, an FPGA / CPLD module, a power amplification module, and motors 1 to 3. The main control processing chip is connected to the FPGA / CPLD module through a high-speed bus, and the power amplification module is connected to motors 1 to 3 through an RS485 bus. Motors 1 to 3 are respectively different actuators of the intelligent robot product. Among them: The main control processing chip is used to control the operation and decision-making of the entire intelligent robot product, and communicate and interact with other modules; The FPGA / CPLD module is used to implement specific hardware logic functions, can process and transform input signals, and transfer the results to other modules; The power amplification module is used to enhance the output power of the motor to drive the movement of the actuator; Motors 1 to 3 are used to receive commands from the main control processing chip to achieve different actions and movements; Another embodiment of the present application further provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the vehicle control method as described in the above embodiment.

[0047] Specifically, the computer-readable storage medium may include: any entity or recording medium capable of carrying the computer program instructions, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0048] The various embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. A robot driving and controlling method, characterized in that, The method includes: Loading and parsing an OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and multiple cores of a multi-core heterogeneous chip; Allocating the multiple nodes to the multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships; Wherein, each node is an operation algorithm, and the multiple nodes at least include a control algorithm, a driving algorithm, and a deep learning algorithm of a robot; the deep learning algorithm is used to perform deep learning to obtain optimal control parameters of the control algorithm; the control algorithm is used to obtain sensing data output by a robot sensing device, and make a robot behavior decision according to the optimal control parameters and the sensing data; the driving algorithm is used to drive the action of an execution mechanism of the robot according to the operation result of the control algorithm.

2. The method according to claim 1, wherein The multiple nodes further include a multimedia algorithm; the multimedia algorithm is used to obtain image data output by a robot camera and process the image data; the control algorithm is used to make a robot behavior decision according to the optimal control parameters, the sensing data, and the operation result of the multimedia algorithm.

3. The method according to claim 2, characterized in that, The multiple cores at least include one Cortex-A core, two C66 DSP cores, one C7 DSP core, and one Cortex-R core; The Cortex-A core is used to implement a custom protocol and peripheral access; One of the C66 DSP cores is bound to the node of the control algorithm, and the other C66 DSP core is bound to the node of the driving algorithm; The C7 DSP core is bound to the node of the deep learning algorithm; The Cortex-R core is bound to the node of the multimedia algorithm.

4. The method according to any one of claims 1 to 3, characterized in that The loading and parsing the OpenVX graph to obtain multiple nodes to be executed specifically includes: Obtaining the current working scenario of the robot, and loading a corresponding OpenVX graph according to the current working scenario of the robot; wherein the OpenVX graphs for different scenarios are different.

5. A robot drive and control system, characterized in that, The system includes: A graph loading module, configured to load and parse an OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and multiple cores of a multi-core heterogeneous chip; A node allocation module, configured to allocate the multiple nodes to the multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships; Wherein, each node is an operation algorithm, and the multiple nodes at least include a control algorithm, a driving algorithm, and a deep learning algorithm of a robot; the deep learning algorithm is used to perform deep learning to obtain optimal control parameters of the control algorithm; the control algorithm is used to obtain sensing data output by a robot sensing device, and make a robot behavior decision according to the optimal control parameters and the sensing data; the driving algorithm is used to drive the action of an execution mechanism of the robot according to the operation result of the control algorithm.

6. The system according to claim 5, characterized in that, The multiple nodes further include a multimedia algorithm; the multimedia algorithm is used to obtain the image data output by the robot camera and process the image data; the control algorithm is used to make robot behavior decisions according to the optimal control parameters, the sensing data, and the operation result of the multimedia algorithm.

7. The system according to claim 6, wherein The multiple cores at least include one Cortex-A core, two C66 DSP cores, one C7 DSP core, and one Cortex-R core; The Cortex-A core is used to implement a custom protocol and peripheral access; One of the C66 DSP cores is bound to the node of the control algorithm, and the other C66 DSP core is bound to the node of the drive algorithm; The C7 DSP core is bound to the node of the deep learning algorithm; The Cortex-R core is bound to the node of the multimedia algorithm.

8. The system according to any one of claims 5 to 7, characterized in that, The graph loading module is specifically used to obtain the current working scenario of the robot and load the corresponding OpenVX graph according to the current working scenario of the robot; the OpenVX graphs for different scenarios are different.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the robot driving and controlling method according to any one of claims 1 to 4.

10. A robot driving and controlling device, characterized in that, It includes a multi-core heterogeneous chip and the electronic device according to claim 9; The multi-core heterogeneous chip includes multiple cores; The electronic device is used to load and parse the OpenVX graph to obtain multiple nodes to be executed, the dependency relationships between the multiple nodes, and the binding relationships between the multiple nodes and the multiple cores of the multi-core heterogeneous chip, and allocate the multiple nodes to the multiple cores of the multi-core heterogeneous chip for execution according to the dependency relationships and the binding relationships.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the robot driving and controlling method according to any one of claims 1 to 4.

Images