Robot control system

Abstract

The embodiment of the invention relates to the technical field of robots, in particular to a robot control system which comprises a central computing module, a computing cluster module, a microcontroller and a task scheduling module, the central computing module comprises a plurality of general computing units, and the computing cluster module comprises a first computing unit, a second computing unit and a third computing unit. And the task scheduling module is connected with the central computing module, the computing cluster module and the microcontroller. According to the embodiment of the invention, the task scheduling module is used for selecting the appropriate data processing and computing unit as the candidate processing unit according to the target task, the candidate processing unit is used for executing the computing processing operation corresponding to the target task, and the selected candidate processing unit focuses on executing the computing processing operation corresponding to the target task; different tasks can be flexibly adapted, the data processing speed is increased, calculation and storage resources are fully utilized, and therefore the task execution efficiency is improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the application relates to the technical field of robots, in particular to a robot control system. BACKGROUND

[0002] In the related art, the robot needs to realize the cooperative processing of tasks such as visual guidance, target recognition, 3D positioning and SLAM mapping, each task is processed by using an independent or associated module, however, this way will cause the problems of response delay, conflict of computing resources and storage resources between the modules, reduce the task execution efficiency, and even cause the robot task execution failure. SUMMARY

[0003] Therefore, an object of the embodiment of the application is to provide a robot control system, aiming at improving the situation of response delay, conflict of computing resources and storage resources when the robot processes tasks in the related art.

[0004] To solve the above technical problems, the embodiment of the application provides the following technical scheme. In a first aspect, the embodiment of the application provides a robot control system, comprising: a central computing module comprising a plurality of general-purpose computing units; a computing cluster module comprising a first computing unit, a second computing unit and a third computing unit; a microcontroller; a task scheduling module connected with the central computing module, the computing cluster module and the microcontroller, and configured to respond to a target task to be executed by the robot, select a target object as a candidate processing unit in the central computing module, the computing cluster module and the microcontroller according to the target task, and execute a computing processing operation corresponding to the target task by using the candidate processing unit, the target object being any one of the general-purpose computing units, the first computing unit, the second computing unit, the third computing unit and the microcontroller.

[0005] In some embodiments, the task scheduling module is configured to respond to a target task to be executed by the robot, select a target object as a candidate processing unit in the central computing module, the computing cluster module and the microcontroller according to the target task, and execute a computing processing operation corresponding to the target task by using the candidate processing unit, including: analyzing the target task to determine a reference task type of the target task; selecting any one or more of the general-purpose computing units, the first computing unit, the second computing unit, the third computing unit and the microcontroller as the candidate processing unit according to the reference task type, the candidate processing unit corresponding to the target task; executing the computing processing operation corresponding to the target task by using the candidate processing unit.

[0006] In some embodiments, the task scheduling module selects any one or more of the general computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller as the candidate processing unit according to the reference task type, including: in response to the reference task type being the first task type, selecting one or more of the general computing unit, the first computing unit, and the second computing unit as the candidate processing unit; in response to the reference task type being the second task type, selecting the third computing unit and the microcontroller as the candidate processing unit.

[0007] In some embodiments, the task scheduling module executes the computing processing operation corresponding to the target task by using the candidate processing unit, including: in response to the candidate processing unit including the general computing unit, the first computing unit, and the second computing unit, establishing a reference data channel between the general computing unit and the first computing unit; establishing a first communication connection between the first computing unit and the second computing unit, and establishing a first number of data channels based on the first communication connection to form a first data channel; scheduling the general computing unit, the first computing unit, and the second computing unit to execute the computing processing operation based on the reference data channel and the first data channel.

[0008] In some embodiments, the task scheduling module schedules the general computing unit, the first computing unit, and the second computing unit to execute the computing processing operation based on the reference data channel and the first data channel, including: sending a first instruction to the general computing unit, the first computing unit, and the second computing unit, so that the general computing unit and the first computing unit construct a map based on the received sensor data through the reference data channel to obtain an environment map, and the second computing unit obtains the environment map through the first data channel, identifies a target object in the environment map based on the environment map to obtain a semantic label corresponding to the target object, and the first computing unit obtains the semantic label through the first data channel and maps the semantic label to the position coordinates of the target object in the environment map to obtain a target map, the target map including a first-level semantic map and a second-level semantic map, the first-level semantic map representing position coordinate information of the target object, and the second-level semantic map representing label information of the target object.

[0009] In some embodiments, the second computing unit obtains the environment map through the first data channel, identifies the target object in the environment map based on the environment map, and obtains the semantic label corresponding to the target object, including: the second computing unit obtains the environment map based on the first data channel; the second computing unit switches the working mode to a dense mode, and identifies the target object in the environment map in the dense mode; The second computing unit determines a semantic label corresponding to the target object according to a type of the target object.

[0010] In some embodiments, the task scheduling module schedules the candidate processing unit to perform the computing processing operation corresponding to the target task, and further includes: In response to the candidate processing unit including the third computing unit and the microcontroller, the third computing unit and the microcontroller are connected via a second communication connection, a second number of data channels are established based on the second communication connection, and a second data channel is formed. The third computing unit and the microcontroller perform the computing processing operation based on the second data channel.

[0011] In some embodiments, the task scheduling module schedules the third computing unit and the microcontroller to perform the computing processing operation based on the second data channel, including: The second instruction is sent to the third computing unit and the microcontroller, so that the third computing unit obtains the target map, plans a motion path of a mechanical arm of the robot according to the target map, generates a target control instruction corresponding to the motion path, sends the target control instruction to the microcontroller, and controls the motor corresponding to the mechanical arm to drive the mechanical arm to move according to the target control instruction.

[0012] In some embodiments, the control system further includes a shared storage module, which is in communication connection with the computing cluster module and the microcontroller. The shared storage module is configured to store candidate data generated or used in the process of the computing cluster module and the microcontroller performing the computing processing operation corresponding to the target task. The candidate data includes instruction data, point cloud data, map data, and label data. The instruction data includes control instructions for the robot to perform the target task. The point cloud data includes point clouds of the environment in which the robot is located when performing the target task. The map data includes a map of the environment in which the robot is located when performing the target task. The label data includes semantic labels of objects in the environment in which the robot is located when performing the target task. In the process of performing the computing processing operation corresponding to the target task, the first computing unit, the second computing unit, the third computing unit, and the microcontroller store the generated data into the shared storage module or obtain the required data from the shared storage module.

[0013] In some embodiments, the shared storage module includes a first storage unit and a second storage unit. The first storage unit is configured to store the point cloud data and the map data generated in the process of the first computing unit performing the computing processing operation. The second storage unit is configured to store the label data generated in the process of the second computing unit performing the computing processing operation and the instruction data used in the process of the microcontroller performing the computing processing operation.

[0014] The robot control system provided by the embodiment of the present application comprises a central computing module, a computing cluster module, a microcontroller and a task scheduling module, the central computing module comprises a plurality of general-purpose computing units, the computing cluster module comprises a first computing unit, a second computing unit and a third computing unit, the task scheduling module is connected with the central computing module, the computing cluster module and the microcontroller, and is configured to select a target object as a candidate processing unit in the central computing module, the computing cluster module and the microcontroller according to a target task to be executed by the robot in response to the target task, and execute a computing processing operation corresponding to the target task by using the candidate processing unit, the target object being any one of the general-purpose computing units, the first computing unit, the second computing unit, the third computing unit and the microcontroller.

[0015] The embodiment of the present application selects a suitable data processing and computing unit as a candidate processing unit according to a target task by using the task scheduling module, executes a computing processing operation corresponding to the target task by using the candidate processing unit, the selected candidate processing unit focuses on executing the computing processing operation corresponding to the target task, can flexibly adapt to different tasks, improves the data processing speed, fully utilizes the computing and storage resources, and thus improves the task execution efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the prior art or the embodiments will be briefly introduced below. Obviously, the drawings described below only show some of the embodiments of the present application, and should not be regarded as a limitation to the protection scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0017] Figure 1 is a structural schematic diagram of a robot provided by some embodiments of the present application; Figure 2 is a structural schematic diagram of a robot control system provided by some embodiments of the present application; Figure 3a is a schematic diagram of the flow steps executed by the task scheduling module in some embodiments of the present application; Figure 3b is a schematic diagram of the data channel established between the first computing unit and the general-purpose computing unit, the second computing unit in some embodiments of the present application; Figure 4 is a schematic diagram of the data channel established between the third computing unit and the microcontroller in some embodiments of the present application; Figure 5 is a structural schematic diagram of a robot control system provided by some embodiments of the present application; Figure 6 is Figure 5A schematic diagram of the shared storage module in the robot control system shown in the embodiment.

[0018] Explanation of reference numerals in the attached figures: 101. Robot; 1011. Vision component; 1012. Main body; 1013. Grasping component; 1014. Walking component; 100. Robot control system; 110. Central computing module; 111. General-purpose computing unit; 120. Computing cluster module; 121. First computing unit; 122. Second computing unit; 123. Third computing unit; 130. Microcontroller; 140. Task scheduling module; 150. Shared storage module; 151. First storage unit; 152. Second storage unit. Detailed Implementation

[0019] To make the objectives and advantages of the embodiments of the present invention more readily understood, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The detailed description of the embodiments of the present invention in the accompanying drawings is not intended to limit the scope of protection claimed by the present invention, but only to illustrate selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, unless there is a conflict, the various technical features involved in the embodiments of the present invention described below can be combined with each other, and all are within the protection scope of the present invention. Furthermore, although functional modules are divided in the device or structural schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," "third," and other similar expressions used herein do not limit the data or execution order, but are only for illustrative purposes and to distinguish identical or similar items with substantially the same function and effect, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features.

[0021] Unless otherwise defined, the technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. It should be understood that the term "and / or" as used herein includes any and all combinations of one or more of the listed items.

[0022] Referring to Figure 1 , Figure 1 A structural schematic diagram of a robot provided by some embodiments of the present application is shown.

[0023] As shown in Figure 1 , the robot 101 includes a vision assembly 1011, a main body 1012, a grabbing assembly 1013, a walking assembly 1014, and a robot control system 100 (not shown), which is in communication connection with the vision assembly 1011, the grabbing assembly 1013, and the walking assembly 1014. The robot control system 100 is arranged at any suitable position on the robot 101, for example, in the main body 1012 of the robot 101. The vision assembly 1011 and the grabbing assembly 1013 are arranged in and fixed to the main body 1012, and the walking assembly 1014 is arranged at the lower end of the main body 1012 and fixed to the main body 1012. Figure 1

[0024] The vision assembly 1011 includes various sensors (such as a laser radar, a camera, etc.) for detecting and acquiring environmental data (such as point cloud data, image data, etc.) of the environment in which the robot 101 is located. The grabbing assembly 1013 includes a mechanical arm, a dexterous hand, and a connecting joint, the dexterous hand being fixedly connected to the mechanical arm through the connecting joint, and the grabbing assembly 1013 being configured to grab a corresponding object in response to a grabbing instruction. The walking assembly 1014 includes a leg arm, a leg piece, and a leg joint, the leg piece being fixedly connected to the leg arm through the leg joint, and the walking assembly 1014 being configured to walk forward and backward or stop in response to a walking instruction or a stop instruction. The robot control system 100 is configured to cooperatively control the vision assembly 1011, the grabbing assembly 1013, and the walking assembly 1014 to complete various robot tasks (such as a delivery task, a guiding task, etc.).

[0025] In embodiments of the present application, the robot 101 can be a mobile robot based on a SLAM system, such as a cleaning robot, a delivery robot, a pet robot, a carrying robot, a nursing robot, a remote monitoring robot, etc. It can be understood that the cleaning robot includes but is not limited to a sweeping robot, a dust-collecting robot, a mopping robot, a floor-washing robot, etc.

[0026] It should be understood that the robot 101 shown in Figure 1 is only illustrative, and does not impose any limitation on the structure, type, etc. of the robot in other embodiments. For example, in some other embodiments, the robot can include more or fewer components than the robot 101 shown in Figure 1 , or have a different configuration than the desktop computer shown in Figure 1 . ​

[0027] Referring to Figure 2 , Figure 2 A structural schematic diagram of a robot control system provided by some embodiments of the present application is shown.

[0028] Specifically, as shown in Figure 2 , the robot control system 100 includes a central computing module 110, a computing cluster module 120, a microcontroller 130, and a task scheduling module 140. The task scheduling module 140 is connected to the central computing module 110, the computing cluster module 120, and the microcontroller 130, for example, the task scheduling module 140 is in communication connection with the central computing module 110, the computing cluster module 120, and the microcontroller 130 through a wireless communication network (such as a Wi-Fi network, a Bluetooth network, or a Zigbee network, etc.).

[0029] The central computing module 110 includes a plurality of general-purpose computing units 111, and the task scheduling module 140 is connected to each general-purpose computing unit 111, for example, the task scheduling module 140 is in communication connection with the general-purpose computing unit 111 through a wireless communication network. It is easy to understand that the general-purpose computing unit 111 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or any other suitable component, etc. The general-purpose computing unit 111 is configured to perform computing and processing operations related to robot tasks, for example, computing and processing operations related to complex tasks (such as path planning, image recognition, task decision, etc.) with high complexity and low real-time requirements. One or more general-purpose computing units 111 can be invoked to perform computing and processing operations according to different types of robot tasks.

[0030] The computing cluster module 120 includes a first computing unit 121, a second computing unit 122, and a third computing unit 123, and the task scheduling module 140 is connected to the first computing unit 121, the second computing unit 122, and the third computing unit 123, for example, the task scheduling module 140 is in communication connection with the first computing unit 121, the second computing unit 122, and the third computing unit 123 through a wireless communication network. It can be understood that the first computing unit 121, the second computing unit 122, and the third computing unit 123 are configured to perform computing and processing operations related to robot tasks, and any one or more of the first computing unit 121, the second computing unit 122, and the third computing unit 123 can be invoked to perform computing and processing operations according to different types of robot tasks.

[0031] The first computing unit 121 can be a graphics processing unit (GPU) or any other suitable component. The first computing unit 121 is used to perform computational processing operations related to high-throughput tasks such as visual rendering, point cloud generation, and SLAM mapping.

[0032] The second computing unit 122 can be a neural processing unit (NPU) or any other suitable component. The second computing unit 122 is used to perform computational processing operations related to tasks such as object detection, semantic segmentation and feature extraction.

[0033] The third computing unit 123 can be a Behavior Processing Unit (BPU) or any other suitable component. The third computing unit 123 is used to perform computational processing operations related to tasks such as inverse kinematics calculation and joint trajectory planning of the robotic arm.

[0034] The microcontroller 130 is configured to perform computational processing operations related to tasks such as robot walking control and walking data monitoring. For example, it is used to perform computational processing operations related to tasks with high real-time requirements but low computational load (such as robot chassis motor drive control, sensor data acquisition, or emergency braking).

[0035] The task scheduling module 140 is configured to respond to the target task to be executed by the robot. Based on the target task, it selects a target object as a candidate processing unit from the central computing module 110, the computing cluster module 120, and the microcontroller 130. The candidate processing unit then performs the computational processing operation corresponding to the target task. The target object can be any one of the general computing unit 111, the first computing unit 121, the second computing unit 122, the third computing unit 123, and the microcontroller 130. The task scheduling module 140 is the core scheduling center of the robot control system 100, responsible for dynamically selecting an appropriate processing unit to execute the computational task based on the type, complexity, real-time nature, and resource status of the robot's target task.

[0036] For example, when the target task is SLAM mapping, one or more general computing units 111 of the central computing module 110, the first computing unit 121 and the second computing unit 122 of the computing cluster module 120 are selected as target objects from the central computing module 110, the computing cluster module 120 and the microcontroller 130, that is, one or more general computing units 111, the first computing unit 121 and the second computing unit 122 are candidate processing units. Then, the environmental data / sensor data detected by the vision component 1011 in the robot 101 are obtained, and the environmental data / sensor data are input to the general computing unit 111, the first computing unit 121 and the second computing unit 122. The general computing unit 111, the first computing unit 121 and the second computing unit 122 are used to process the environmental data / sensor data. For example, the first computing unit 121 is used to perform spatiotemporal alignment and noise reduction processing on the environmental data / sensor data, and the processed data is parsed to generate an environmental map. The second computing unit 122 reads the environmental map, identifies target objects in the environment (such as obstacles or pedestrians), and outputs semantic labels for each target object (such as pedestrian location, road boundary, obstacle type, etc.). These semantic labels are then mapped to the environmental map, and a hierarchical semantic map (e.g., the bottom layer is the environmental map, and the upper layer is the semantic object relationships) is constructed based on the semantic labels and the environmental map. The hierarchical semantic map is then sent to other units (such as the third computing unit 123 or the microcontroller 130) using the general-purpose computing unit 111. The hierarchical semantic map serves as a data reference for robot walking control (such as obstacle avoidance and path planning).

[0037] This invention utilizes a task scheduling module to select appropriate data processing and computing units as candidate processing units based on the target task. The candidate processing units then execute the computing operations corresponding to the target task. The selected candidate processing units focus on executing the computing operations corresponding to the target task, which can flexibly adapt to different tasks, improve data processing speed, make full use of computing and storage resources, and thus improve task execution efficiency.

[0038] Please see Figure 3a In some embodiments, the task scheduling module is configured to respond to the target task to be executed by the robot, and select a target object as a candidate processing unit from the central computing module, computing cluster module and microcontroller according to the target task, and use the candidate processing unit to perform the computing processing operation corresponding to the target task, specifically including but not limited to the following steps S31-S33: S31: Analyze the target task and determine the reference task type for the target task.

[0039] S32: Select one or more of the following as candidate processing units: general computing unit, first computing unit, second computing unit, third computing unit, and microcontroller, based on the reference task type.

[0040] S33: Utilize candidate processing units to perform computational processing operations corresponding to the target task.

[0041] In this embodiment, after the task scheduling module obtains the target task to be executed by the robot, it parses the target task and identifies the reference task type (e.g., "grasping task") corresponding to the target task based on key fields in the target task (e.g., the key field is "grasping toy A"). After identifying the reference task type of the target task, the task scheduling module selects one or more of the following as candidate processing units from the general computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller, based on the reference task type. The selected candidate processing unit matches the target task; for example, the performance, latency, power consumption, and real-time performance of the candidate processing unit are suitable for processing the target task. Understandably, the candidate processing unit corresponds to the target task, meaning it is used to perform the computational processing operations corresponding to the target task. The task scheduling module assigns the target task to the selected candidate processing unit, and the candidate processing unit performs the computational processing operations corresponding to the target task, such as data analysis, control command generation, image reasoning, or path planning. Understandably, the target task can be completed using a single unit processing independently or multiple units cooperating in computation.

[0042] In this embodiment, by parsing the target task, the task type is accurately identified, providing a basis for subsequent allocation of computing resources. Furthermore, through task type matching and multi-computing unit selection mechanisms, adaptive matching of computing resources and task requirements is achieved, improving the task scheduling flexibility and overall computing efficiency of the robot control system. This enables efficient execution of robot tasks, balancing real-time task response and complex computing capabilities, and enhancing the overall system performance.

[0043] For example, in some embodiments, the task scheduling module selects one or more of the general computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller as candidate processing units according to the reference task type, specifically including but not limited to the following steps S321-S322: S321: If the response reference task type is the first task type, select one or more general computing units, the first computing unit, and the second computing unit as candidate processing units.

[0044] S322: In response to the second task type, select the third computing unit and the microcontroller as candidate processing units.

[0045] In this embodiment, the task types include a first task type and a second task type. The first task type corresponds to tasks with large computational load, complex data, and the need for parallel processing, such as image recognition, path planning, and environmental modeling. The second task type corresponds to tasks with high real-time requirements and precise control requirements, such as robot low-level motion control, rapid response of sensor data, and coordination of robotic arms.

[0046] Specifically, when the reference task type is the first task type, the task scheduling module selects one or more general-purpose computing units, the first computing unit, and the second computing unit as candidate processing units. These units then perform the computational processing operations corresponding to the target task, thereby ensuring the processing performance and parallel efficiency of large-scale computations or complex algorithm tasks. When the reference task type is the second task type, the task scheduling module selects a third computing unit and a microcontroller as candidate processing units. These units then perform the computational processing operations corresponding to the target task, reducing task response latency, improving robot control precision, and ensuring the accuracy of robot motion control and real-time feedback.

[0047] In some embodiments, the task scheduling module utilizes candidate processing units to perform computational processing operations corresponding to the target task, specifically including but not limited to the following steps S331-S333: S331: The response candidate processing unit includes a general computing unit, a first computing unit and a second computing unit, and establishes a reference data channel between the general computing unit and the first computing unit.

[0048] S332: Establish a first communication connection between the first computing unit and the second computing unit, and establish a first number of data channels based on the first communication connection to form a first data channel.

[0049] S333: The scheduling general computing unit, the first computing unit and the second computing unit perform computing processing operations based on the reference data channel and the first data channel.

[0050] In this embodiment, the reference data channel is used to transmit data and instructions, enabling data sharing based on task division and providing a unified data or instruction transmission channel for subsequent parallel computing. The first data channel is used for high-speed data transmission and interaction of distributed task decomposition results. A reference data channel is established between the general computing unit and the first computing unit, and a first data channel is established between the first computing unit and the second computing unit, forming a stable and high-throughput computing data link. Then, the task scheduling module schedules the general computing unit, the first computing unit, and the second computing unit, and executes the computing processing operations corresponding to the target task based on the reference data channel and the first data channel. This realizes data interaction and collaborative computing among multiple units, improves the efficiency of cross-computing unit task collaboration, and enhances the computational parallelism and data processing efficiency of complex tasks.

[0051] For example, please see Figure 3b A reference data channel 201 is established between the general computing unit 111 and the first computing unit 121, and a first data channel 202 is established between the first computing unit 121 and the second computing unit 122. The first data channel 202 includes three data channels 2021.

[0052] In some embodiments, the task scheduling module schedules the general computing unit, the first computing unit, and the second computing unit to perform computational processing operations based on the reference data channel and the first data channel, specifically including but not limited to the following step S3331: S3331: Send a first instruction to the general-purpose computing unit, the first computing unit, and the second computing unit, so that the general-purpose computing unit and the first computing unit construct a map based on the received sensor data through the reference data channel to obtain an environmental map, and the second computing unit acquires the environmental map through the first data channel, identifies target objects in the environmental map based on the environmental map, obtains semantic labels corresponding to the target objects, and the first computing unit acquires the semantic labels through the first data channel and maps the semantic labels to the position coordinates of the target objects in the environmental map to obtain a target map.

[0053] Specifically, the task scheduling module sends a first instruction to the general-purpose computing unit, the first computing unit, and the second computing unit. This first instruction instructs the general-purpose computing unit, the first computing unit, and the second computing unit to work collaboratively, specifying their data communication paths / channels and computational division of labor. Based on received sensor data (such as data detected and collected by LiDAR, depth cameras, and webcams), the general-purpose computing unit and the first computing unit collaboratively construct an environmental map through a reference data channel. This environmental map serves as the basic geometric model of the robot's perceptual space, containing scene topology and depth information. Understandably, by utilizing the collaboration of multiple computing units, the accuracy and computational efficiency of the environmental map generation are improved.

[0054] For example, the second computing unit obtains the constructed environment map from the first computing unit through the first data channel, performs target detection and recognition operations on the environment map, identifies target objects in the environment, and generates corresponding semantic labels (such as "table", "obstacle", "person", "door", etc.) for each target object based on the recognition results. The first computing unit obtains the semantic labels of the target objects from the second computing unit through the first data channel, maps the semantic labels to the corresponding position coordinates of the target objects in the environment map, that is, associates the semantic labels with the position coordinates of the target objects in the environment map to generate the target map. The target map includes a first-level semantic map and a second-level semantic map. The first-level semantic map represents the position coordinate information of the target objects, and the second-level semantic map represents the semantic label information of the target objects. The target map is a multi-level map that combines spatial structure and semantic understanding, providing a high-level scene cognition reference for the robot's subsequent path planning, interaction understanding, and other tasks.

[0055] In some embodiments, the second computing unit acquires an environmental map through the first data channel, identifies target objects in the environmental map based on the environmental map, and obtains semantic labels corresponding to the target objects, including but not limited to the following steps S33311-S33313: S33311: The second computing unit acquires the environmental map based on the first data channel.

[0056] S33312: The second computing unit switches its working mode to dense mode, and identifies target objects in the environment map in dense mode.

[0057] S33313: The second calculation unit determines the semantic tag corresponding to the target object based on the type of the target object.

[0058] In this embodiment of the invention, the second computing unit includes a dense mode and a sparse mode. In dense mode, the second computing unit activates all processing elements (PEs) to handle dense matrix operations that are computationally intensive, involve complex data, and require parallel processing. In sparse mode, the second computing unit shuts down idle processing elements and focuses on handling feature sparse computations that require high real-time performance, precise control, and appropriate computational load.

[0059] In this embodiment, when identifying the semantic tags of target physical objects in the environment, the second computing unit obtains the environment map from the first computing unit based on the first data channel, and then switches the working mode to dense mode. In dense mode, fine-grained analysis is performed on the pixel points or voxel data in the environment map to accurately detect and identify target objects in the environment map. Based on the type of the identified target object (such as a person, table, obstacle or furniture), a semantic tag corresponding to the target object is generated according to a preset semantic library or classification model. The semantic tag corresponds one-to-one with the identification result of the target object.

[0060] In this embodiment, dense data analysis improves the accuracy of target recognition and spatial positioning precision, enhances semantic discrimination capabilities, and enables semantic understanding and classification of target objects in the environment, providing structured semantic information for the subsequent construction of semantic maps.

[0061] In some embodiments, the task scheduling module utilizes candidate processing units to perform computational processing operations corresponding to the target task, which specifically includes, but is not limited to, the following steps S334-S335: S334: The response candidate processing unit includes a third computing unit and a microcontroller. A second communication connection is established between the third computing unit and the microcontroller. A second number of data channels are established based on the second communication connection to form a second data channel.

[0062] S335: Schedules the third computing unit and the microcontroller to perform computing processing operations based on the second data channel.

[0063] In this embodiment, the second data channel is used to distribute real-time control parameters, execution feedback, or status monitoring data. After establishing the second data channel between the third computing unit and the microcontroller, the task scheduling module's third computing unit and the microcontroller perform computational processing operations corresponding to the target task based on the second data channel. Specifically, the third computing unit performs complex motion planning, path calculation, or control strategy generation, while the microcontroller performs real-time closed-loop control, actuator driving, and sensor data feedback. This enables real-time collaboration between the upper-level computing unit and the lower-level control unit, ensuring the stability and dynamic response capability of task execution, and achieving a high real-time and low-latency computational processing mechanism.

[0064] For example, please see Figure 4 A second data channel 203 is established between the third computing unit 123 and the microcontroller 130. The second data channel 203 includes three data channels 2031.

[0065] In some embodiments, the task scheduling module schedules the third computing unit and the microcontroller to perform computing processing operations based on the second data channel, specifically including but not limited to the following step S3351: S3351: Send the second instruction to the third computing unit and the microcontroller, so that the third computing unit can obtain the target map, plan the motion path of the robot's arm according to the target map, generate the target control instruction corresponding to the motion path, send the target control instruction to the microcontroller, and make the microcontroller control the motor corresponding to the robot arm to drive the robot arm to move according to the target control instruction.

[0066] For example, the task scheduling module sends a second instruction to the third computing unit and the microcontroller. This second instruction instructs the third computing unit and the microcontroller to work collaboratively and specifies their data communication path / data channel and computational division of labor. The third computing unit responds to the second instruction, acquires the target map, plans the motion path of the robot's arm based on the target map, generates target control instructions corresponding to the motion path, and sends the target control instructions to the microcontroller. The microcontroller controls the motors corresponding to the robot arm according to the target control instructions to drive the robot arm to move along the motion path. The third computing unit is responsible for high-level planning tasks (such as path planning and control instruction generation), while the microcontroller is responsible for executing low-level driving tasks (such as controlling the motors to drive the robot arm's movements). The two communicate through the second data channel, achieving a bidirectional closed loop of data and control flow.

[0067] In some embodiments, please refer to Figure 5 The robot control system 100 also includes a shared storage module 150, which is communicatively connected to the computing cluster module 120 and the microcontroller 130. The shared storage module 150 is also communicatively connected to the first computing unit 121, the second computing unit 122 and the third computing unit 123 of the computing cluster module 120.

[0068] The shared storage module 150 is configured to store candidate data generated or used by the computing cluster module 120 and the microcontroller 130 during the computational processing operations corresponding to the target task. The candidate data includes instruction data, point cloud data, map data, and label data. Instruction data includes control instructions related to the robot's execution of the target task, such as robotic arm control instructions. Point cloud data includes point clouds of the environment in which the robot is located when performing the target task, such as point clouds of the environment in front of the robot. Map data includes maps of the environment in which the robot is located when performing the target task, such as a grid map of the robot's environment. Label data includes semantic labels for various objects in the environment in which the robot is located when performing the target task, such as "person," "door," "wall," and "obstacle."

[0069] During the execution of computational processing operations corresponding to the target task, the first computing unit 121, the second computing unit 122, the third computing unit 123, and the microcontroller 130 store the generated or used data to the shared storage module 150. For example, the first computing unit 121 stores the constructed environment map to the shared storage module 150, and the third computing unit 123 stores the generated control commands to the shared storage module 150. Alternatively, they may retrieve the required data from the shared storage module 150. For example, the second computing unit 122 retrieves the environment map from the shared storage module 150, and the microcontroller 130 retrieves the control commands from the shared storage module 150.

[0070] In some embodiments, please refer to Figure 6 The shared storage module 150 includes a first storage unit 151 and a second storage unit 152. The first storage unit 151 is communicatively connected to the first computing unit 121, and the second storage unit 152 is communicatively connected to the second computing unit 122.

[0071] The first storage unit 151 is configured to store point cloud data and map data generated during the computational processing operation performed by the first computing unit 121, such as storing the three-dimensional point cloud obtained by the first computing unit 121 from the sensor and the environmental map generated by the first computing unit 121 from the environmental data.

[0072] The second storage unit 152 is configured to store tag data generated during the computational processing operation performed by the second computing unit 122, and to store instruction data used by the microcontroller 130 during the computational processing operation, such as semantic tags of objects in the environment identified by the second computing unit 122 and control instructions of the robotic arm used by the microcontroller 130.

[0073] In summary, the robot control system provided by the embodiments of the present invention includes a central computing module, a computing cluster module, a microcontroller, and a task scheduling module. The central computing module includes multiple general-purpose computing units, and the computing cluster module includes a first computing unit, a second computing unit, and a third computing unit. The task scheduling module is connected to the central computing module, the computing cluster module, and the microcontroller, and is configured to respond to the target task to be executed by the robot. Based on the target task, the module selects a target object as a candidate processing unit from the central computing module, the computing cluster module, and the microcontroller, and uses the candidate processing unit to perform the computational processing operation corresponding to the target task. The target object can be any one of the general-purpose computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller.

[0074] This invention utilizes a task scheduling module to select appropriate data processing and computing units as candidate processing units based on the target task. The candidate processing units then execute the computing operations corresponding to the target task. The selected candidate processing units focus on executing the computing operations corresponding to the target task, which can flexibly adapt to different tasks, improve data processing speed, make full use of computing and storage resources, and thus improve task execution efficiency.

[0075] Those skilled in the art will understand that the embodiments provided by this invention are merely illustrative. The order in which the steps in the methods of the embodiments are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The order can be adjusted, merged, and deleted according to actual needs. Modules or sub-modules, units or sub-units in the apparatus or system of the embodiments can be merged, divided, and deleted according to actual needs. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0076] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, and of course, it can also be implemented using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. It should be understood that the storage medium can be flash memory, hard disk, optical disk, register, magnetic surface memory, removable disk, CD-ROM, random access memory (RAM), read-only memory (ROM), electrically programmable ROM, and electrically erasable programmable ROM, etc.

[0077] It should be noted that the above embodiments are for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can understand that all or part of the processes of the above embodiments can be implemented by modifying the technical solutions described in the embodiments of the present invention, or by making equivalent substitutions for some of the technical features. It is understood that these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should be considered as equivalent changes and modifications made based on the embodiments of the present invention, all of which should fall within the scope of the claims of the present invention.

Claims

1. A robot control system, characterized in that, include: The central computing module includes multiple general-purpose computing units; The computing cluster module includes a first computing unit, a second computing unit, and a third computing unit; microcontroller; The task scheduling module, connected to the central computing module, the computing cluster module, and the microcontroller, is configured to respond to the target task to be executed by the robot. Based on the target task, it selects a target object as a candidate processing unit from the central computing module, the computing cluster module, and the microcontroller, and uses the candidate processing unit to perform computational processing operations corresponding to the target task. The target object can be any one of the general computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller.

2. The control system according to claim 1, characterized in that, The task scheduling module is configured to respond to the target task to be executed by the robot. Based on the target task, it selects a target object as a candidate processing unit from the central computing module, the computing cluster module, and the microcontroller. The module then uses the candidate processing unit to execute computational processing operations corresponding to the target task, including: Analyze the target task and determine the reference task type of the target task; Based on the reference task type, one or more of the general computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller are selected as candidate processing units, and the candidate processing units correspond to the target task. The candidate processing unit is used to perform computational processing operations corresponding to the target task.

3. The control system according to claim 2, characterized in that, The task scheduling module selects one or more of the general computing unit, the first computing unit, the second computing unit, the third computing unit, and the microcontroller as candidate processing units based on the reference task type, including: In response to the reference task type being the first task type, one or more of the general computing unit, the first computing unit, and the second computing unit are selected as candidate processing units; In response to the reference task type being the second task type, the third computing unit and the microcontroller are selected as candidate processing units.

4. The control system according to claim 2, characterized in that, The task scheduling module utilizes the candidate processing unit to perform computational processing operations corresponding to the target task, including: The response to the candidate processing unit includes the general computing unit, the first computing unit and the second computing unit, and establishes a reference data channel between the general computing unit and the first computing unit; A first communication connection is established between the first computing unit and the second computing unit, and a first number of data channels are established based on the first communication connection to form a first data channel; The general computing unit, the first computing unit, and the second computing unit are scheduled to perform the computing processing operation based on the reference data channel and the first data channel.

5. The control system according to claim 4, characterized in that, The task scheduling module schedules the general computing unit, the first computing unit, and the second computing unit to perform the computing processing operation based on the reference data channel and the first data channel, including: A first instruction is sent to the general-purpose computing unit, the first computing unit, and the second computing unit, so that the general-purpose computing unit and the first computing unit construct a map based on the received sensor data through the reference data channel to obtain an environmental map, and the second computing unit acquires the environmental map through the first data channel, identifies target objects in the environmental map based on the environmental map, obtains semantic tags corresponding to the target objects, and the first computing unit acquires the semantic tags through the first data channel and maps the semantic tags to the position coordinates of the target objects in the environmental map to obtain a target map. The target map includes a first-level semantic map and a second-level semantic map. The first-level semantic map represents the position coordinate information of the target objects, and the second-level semantic map represents the tag information of the target objects.

6. The control system according to claim 5, characterized in that, The second computing unit acquires the environment map through the first data channel, identifies target objects in the environment map based on the environment map, and obtains semantic tags corresponding to the target objects, including: The second computing unit acquires the environmental map based on the first data channel; The second computing unit switches its operating mode to dense mode and identifies target objects in the environment map in dense mode; The second calculation unit determines the semantic tag corresponding to the target object based on the type of the target object.

7. The control system according to claim 5, characterized in that, The task scheduling module utilizes the candidate processing unit to execute computational processing operations corresponding to the target task, and further includes: The candidate processing unit includes the third computing unit and the microcontroller. A second communication connection is established between the third computing unit and the microcontroller. A second number of data channels are established based on the second communication connection to form a second data channel. The third computing unit and the microcontroller are scheduled to perform the computing processing operation based on the second data channel.

8. The control system according to claim 7, characterized in that, The task scheduling module schedules the third computing unit and the microcontroller to perform the computing processing operation based on the second data channel, including: The third computing unit and the microcontroller are sent a second instruction to the third computing unit to obtain the target map, plan the motion path of the robot's arm based on the target map, generate a target control instruction corresponding to the motion path, send the target control instruction to the microcontroller, and enable the microcontroller to control the motor corresponding to the robot arm to drive the robot arm to move according to the target control instruction.

9. The control system according to any one of claims 1-8, characterized in that, The control system further includes a shared storage module, which is communicatively connected to the computing cluster module and the microcontroller. The shared storage module is configured to store candidate data generated or used by the computing cluster module and the microcontroller during the execution of computational processing operations corresponding to the target task. The candidate data includes instruction data, point cloud data, map data, and tag data. The instruction data includes control instructions for the robot to execute the target task. The point cloud data includes point clouds of the environment in which the robot is located when executing the target task. The map data includes a map of the environment in which the robot is located when executing the target task. The tag data includes semantic tags of objects in the environment in which the robot is located when executing the target task. In the process of performing computational processing operations corresponding to the target task, the first computing unit, the second computing unit, the third computing unit, and the microcontroller store the generated data in the shared storage module or obtain the required data from the shared storage module.

10. The control system according to claim 9, characterized in that, The shared storage module includes a first storage unit and a second storage unit. The first storage unit is configured to store point cloud data and map data generated during the computational processing operation of the first computing unit. The second storage unit is configured to store tag data generated during the computational processing operation of the second computing unit and instruction data used by the microcontroller during the computational processing operation.

Images