Robot control method, apparatus, and electronic device

Abstract

The embodiment of the application provides a kind of robot control method, device and electronic equipment, belong to robot technical field, this method includes: obtaining task information and task target to be executed;According to the skill of task information to be executed, carry out skill disassembly processing, obtain multiple independent atomic skills;According to atomic skill, carry out trajectory planning processing, obtain initial trajectory, and obtain the expected parameter of atomic skill;Obtain environment perception information, and according to environment perception information, carry out skill reuse adjustment processing, obtain target execution strategy;According to target execution strategy, expected parameter and environment perception information, trajectory replanning processing is carried out to initial trajectory, and target trajectory is obtained;According to target trajectory, control each atomic skill executes corresponding subtask to reach expected subtask target, until reach task target.The method is used to reach the effect of improving the flexibility and efficiency of the control task of robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and in particular to a robot control method and device and electronic equipment. BACKGROUND

[0002] With the development of robot technology, in order to complete complex tasks, it is usually necessary to divide the control system of the robot into a plurality of modular skill units, each of which is used to execute a corresponding control task.

[0003] However, the inventors have found in the process of researching and designing a robot control system that, in the process of executing complex tasks, the robot needs to cope with changes in its own position while also coping with complex and variable environmental information and changing task targets, etc. As the complexity of environmental changes and target changes increases, the robot's control execution capability is greatly reduced, resulting in low flexibility and efficiency of control task execution.

[0004] Therefore, there is an urgent need for a robot control method to solve the above technical problems. SUMMARY

[0005] The embodiments of the present application provide a robot control method, device and electronic equipment to improve the flexibility and efficiency of robot control tasks.

[0006] In a first aspect, the embodiments of the present application provide a robot control method, comprising:

[0007] obtaining task information to be executed and a task target;

[0008] performing skill decomposition processing according to the task information to be executed to obtain a plurality of independent atomic skills, each atomic skill being used to execute a subtask to achieve an expected subtask target, and all expected subtask targets being achieved to achieve the task target;

[0009] performing trajectory planning processing according to the atomic skills to obtain an initial trajectory, and obtaining expected parameters of the atomic skills;

[0010] obtaining environmental perception information, and performing skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy;

[0011] performing trajectory re-planning processing on the initial trajectory according to the target execution strategy, the expected parameters and the environmental perception information to obtain a target trajectory;

[0012] controlling each atomic skill to execute a corresponding subtask to achieve an expected subtask target according to the target trajectory, until the task target is achieved.

[0013] In a possible implementation, the to-be-executed task information includes a target position, a target pose, and robot state information; the task decomposition processing is performed according to the to-be-executed task information, to obtain a plurality of independent atomic skills, including: performing task analysis processing according to the target position, the target pose, and the robot state information, to obtain a plurality of subtask target description information; performing task decomposition processing according to the subtask target description information, to obtain a plurality of subtasks; and selecting corresponding function modules from pre-stored robot function modules according to the subtasks to determine the function modules as atomic skills, to obtain a plurality of independent atomic skills.

[0014] In a possible implementation, the trajectory planning processing is performed according to the atomic skills, to obtain an initial trajectory, including: determining trajectory starting point data according to the atomic skills; performing trajectory planning processing according to the trajectory starting point data, to obtain an actuator motion trajectory; and performing combination processing on actuator motion trajectories corresponding to the atomic skills, to obtain the initial trajectory.

[0015] In a possible implementation, the trajectory planning processing is performed according to the trajectory starting point data, to obtain an actuator motion trajectory, including: performing trajectory planning processing according to the trajectory starting point data by using a teleoperation method or a trajectory planning algorithm, to obtain an actuator motion trajectory.

[0016] In a possible implementation, the skill reuse adjustment processing is performed according to the environment perception information, to obtain a target execution strategy, including: determining reuse mode update information of the atomic skills according to the environment perception information; and adjusting calling information of the atomic skills according to the reuse mode update information of the atomic skills, to obtain the target execution strategy.

[0017] In a possible implementation, the trajectory replanning processing is performed on the initial trajectory according to the target execution strategy, the expected parameters, and the environment perception information, to obtain a target trajectory, including: determining default environment information according to the target execution strategy; performing comparison and calculation processing according to the default environment information and the environment perception information, to obtain trajectory correction information; and performing adjustment processing according to the trajectory correction information, the expected parameters, and the initial trajectory, to obtain the target trajectory.

[0018] In a possible implementation, the expected parameter includes an expected robot body pose matrix; accordingly, the adjusting processing according to the trajectory correction information, the expected parameter, and the initial trajectory to obtain the target trajectory includes: determining an initial robot body pose matrix according to the trajectory correction information and the initial trajectory; determining a target robot body pose matrix according to the expected robot body pose matrix and the initial robot body pose matrix; performing solving operation processing according to the target robot body pose matrix to obtain a robot end pose trajectory; determining a transition trajectory according to the robot end pose trajectory and the initial robot body pose matrix; and performing pose point-by-point merging processing according to the transition trajectory and the robot end pose trajectory to obtain the target trajectory.

[0019] In a possible implementation, the method further includes: during the process of controlling each atomic skill to perform a corresponding subtask according to the target trajectory to achieve an expected subtask target until the task target is achieved, acquiring environment change information in real time; when detecting that the environment change information and the environment perception information have information differences, returning to acquire the environment perception information, and performing skill reuse adjustment processing according to the environment perception information to obtain a target execution strategy.

[0020] In a second aspect, an embodiment of the present application provides a robot control device, including:

[0021] an acquisition module configured to acquire task information to be executed and a task target;

[0022] a skill disassembly module configured to perform skill disassembly processing according to the task information to be executed to obtain a plurality of independent atomic skills, each atomic skill being configured to execute a subtask to achieve an expected subtask target, and all expected subtask targets being achieved being the task target achieved;

[0023] a trajectory planning module configured to perform trajectory planning processing according to the atomic skills to obtain an initial trajectory, and acquire expected parameters of the atomic skills;

[0024] the acquisition module is further configured to acquire environment perception information, and perform skill reuse adjustment processing according to the environment perception information to obtain a target execution strategy;

[0025] the trajectory planning module is further configured to perform trajectory re-planning processing on the initial trajectory according to the target execution strategy, the expected parameters, and the environment perception information to obtain a target trajectory;

[0026] a skill execution module configured to control each atomic skill to perform a corresponding subtask according to the target trajectory to achieve an expected subtask target until the task target is achieved.

[0027] In a third aspect, an electronic device is provided, comprising: a memory, a processor;

[0028] The memory stores computer-executable instructions.

[0029] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to the first aspect and / or various possible implementation manners of the first aspect.

[0030] The robot control method, device and electronic device provided by the embodiments of the present application first perform skill decomposition processing according to task information to be executed, to obtain a plurality of independent atomic skills, then perform trajectory planning processing according to the atomic skills to obtain an initial trajectory, and simultaneously obtain expected parameters of the atomic skills. Then, environment perception information is obtained to perform skill reuse adjustment processing according to the environment perception information, to obtain a target execution strategy, so as to ensure that the robot controls the atomic skills to execute corresponding subtasks according to the target execution strategy, and to avoid errors. Furthermore, the initial trajectory is subjected to trajectory re-planning processing according to the target execution strategy, the expected parameters and the environment perception information, to achieve the purpose of correcting the trajectory, to obtain a target trajectory. Finally, the corresponding subtasks are executed according to the target trajectory to achieve the expected subtask target, until the task target is achieved. In this process, the target trajectory obtained by the trajectory re-planning processing can ensure that the task is efficiently and accurately completed. At the same time, the trajectory is corrected by collecting the perception environment information, to improve the flexibility of controlling the task execution. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0032] Figure 1 A scene schematic diagram of the robot control method provided by the present application;

[0033] Figure 2 A flowchart of the robot control method provided by the present application;

[0034] Figure 3 A process schematic diagram of trajectory re-planning provided by the present application;

[0035] Figure 4 A structure schematic diagram of the robot control device provided by the present application;

[0036] Figure 5 A structure schematic diagram of the electronic device provided by the present application.

[0037] The specific embodiments of the application have been shown by way of example in the above figures, and will be described in more detail hereafter. These figures and this written description are not intended to limit the scope of the inventive concept in any way, but rather to illustrate the inventive concept to one of ordinary skill in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0038] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same numbers are used in different drawings to represent the same or similar elements. The following detailed description is not intended to limit the application, as claimed, in any way. Rather, the following description is intended to describe the embodiments of the application in sufficient detail to enable one of ordinary skill in the art to practice the application, including making and using the application.

[0039] In the prior art, the control system of a robot is divided into a plurality of modular skill units, but each skill unit only adopts a fixed skill calling sequence for control instructions, and these calling sequences are all pre-set or pre-trained by a designer, so that the existing control method cannot be quickly adjusted according to changing task requirements, environmental changes or external intervention. For example, in a complex multi-task environment (such as the number of target objects to be grasped increasing from one to three, and the coordinates and poses of each target object being different), if the robot lacks the ability to generate appropriate task trajectories and calling sequences of different skill units to flexibly respond to changing task requirements or changing task environments, the control execution capability of the robot will be greatly reduced, resulting in problems such as low flexibility and efficiency of control task execution, for example, the robot control process is slow to react, the task execution period is lengthened, or only the initial target object grasping task can be completed.

[0040] To solve the above technical problems, the inventors propose the following inventive concept: by decomposing a task into a plurality of atomic skills, and configuring an initial trajectory and an environment adaptation mechanism for each atomic skill, i.e., obtaining environment change information and adjusting the calling sequence and calling mode of the atomic skills during task execution, while re-planning the trajectory, to ensure that the robot can flexibly respond to changes in the environment and improve the flexibility and efficiency of control task execution.

[0041] The technical solutions of the application and how the technical solutions of the application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the application will be described below with reference to the drawings.

[0042] Figure 1 A scene diagram of the robot control method provided by the application is shown in FIG. 1. Figure 1As shown, the specific application scenarios of the present application include: a robot 101 and an initial target object 102 and a new target object 103, wherein the robot 101 is installed with a server and a scanning device for environment information and target object pose data, such as a radar sensor or a depth camera, and the server can execute the relevant steps of the robot control method according to the environment information, and finally obtain the calling sequence of each functional module of the robot 101 and the movement trajectory of the actuator end, for example, first control the movement function module of the robot to move to the position where the initial target object 102 is located, and then control the grasping function module of the robot to grasp the initial target object 102, and finally complete the control task.

[0043] The initial target object 102 and the new target object 103 can be any object waiting for the robot 101 to perform corresponding actions. Figure 1 The obstacles and new obstacles in the robot 101 are obstacles that may be encountered during the execution of the control task. The new target object 103 represents a task demand change scenario encountered by the robot 101 during the execution of the control task, and the new obstacle represents an environment change scenario encountered by the robot 101 during the execution of the control task. When the robot 101 encounters a task demand change scenario and an environment change scenario during the execution of the corresponding control task, it can cope with it through the robot control method provided by the present application, so as to improve the flexibility and efficiency of the control task execution.

[0044] It should be noted that, Figure 1 The application scenario of the robot control method as shown is only one application scenario of target object grasping and robot movement, and the robot control method proposed in the present embodiment can also be applied to other application scenarios such as the scenario of robot moving off the track due to external force interference, and the present embodiment does not limit this.

[0045] Figure 2 The flowchart of the robot control method provided by the present application is shown in Figure 2 As shown, the execution subject of the method can be Figure 1 The service area in the robot 101 as shown, or other controllers or related computer devices with control functions and data processing functions, and the present embodiment does not particularly limit this.

[0046] As shown in Figure 1 The robot control method includes:

[0047] S201: Obtain task information and task target.

[0048] In this embodiment, the to-be-executed task information can be information related to the robot control task obtained based on the input condition of the task, for example: the to-be-executed task information can include task requirement information. The task target can be the execution result of the to-be-executed task to be executed by the robot.

[0049] S202: According to the to-be-executed task information, skill decomposition processing is performed to obtain a plurality of independent atomic skills, each atomic skill is used to execute a subtask to achieve an expected subtask target, and all expected subtask targets are achieved to achieve the task target.

[0050] In this embodiment, the skill decomposition processing can be a process of decomposing the to-be-executed task information into a plurality of independent subtasks and then associating the subtasks to corresponding atomic skills according to the task properties or task requirements of the subtasks. The atomic skill refers to an independent execution function module of the robot, for example, a movement function module, a grabbing function module or an interaction function module of the robot, etc. In this embodiment, the task target can be defined based on the input condition of the task, wherein the input condition of the task can include the position of the target object, the pose information of the target object and the current state information of the robot, and other information related to the task.

[0051] In an optional embodiment of the present application, the to-be-executed task information includes target position, target pose and robot state information; correspondingly, step S202 specifically includes:

[0052] S202a: According to the target position, the target pose and the robot state information, task analysis processing is performed to obtain a plurality of subtask target description information.

[0053] In this embodiment, the robot state information can be the movement speed, the current position and the current robot body pose information of the robot. According to the robot state information, the target position and the target pose, description information related to each subtask target and task process of the to-be-executed task can be obtained, for example: subtask 1 is to control the robot to move from point A to point B, and subtask 2 is to control the robot to grab the target object at point B.

[0054] S202b: According to the subtask target description information, task decomposition processing is performed to obtain a plurality of subtasks.

[0055] In this embodiment, the total to-be-executed task is composed of a plurality of subtasks, and at this time, task decomposition processing of the total to-be-executed task needs to be performed according to the subtask description information, which means that the corresponding subtask is determined according to each subtask description information, so as to be matched to the corresponding function module subsequently.

[0056] S202c: According to the subtask, the corresponding function module is selected from the pre-stored robot function module to determine as an atomic skill, so as to obtain a plurality of independent atomic skills.

[0057] In this embodiment, the selection of the corresponding function module from the pre-stored robot function module according to the subtask can be performed by matching the keyword related to the subtask with the pre-set keyword in each function module. For example, if the keyword is movement, grabbing or interaction, the corresponding movement function module, grabbing function module and interaction function module can be selected as the atomic skill, respectively.

[0058] In this embodiment, the task target is defined based on the input condition of the task, and the task target is decomposed into multiple subtasks such as a movement subtask, a grabbing subtask and an interaction subtask, so as to ensure that the atomic skill corresponding to each subtask can be independently executed and the expected target can be achieved. The pre-stored robot function module can be a function module pre-stored in the robot control system and capable of completing a corresponding control task, for example, a movement function module, a grabbing function module, an interaction function module and a performance function module. In this embodiment, each independent atomic skill can be used to complete the corresponding subtask, and after all the expected subtasks are executed, it is indicated that the task to be executed is completed, and the robot control process reaches the task target.

[0059] S203: Trajectory planning processing is performed according to the atomic skill, an initial trajectory is obtained, and expected parameters of the atomic skill are acquired.

[0060] In this embodiment, each decomposed atomic skill generally depends on certain expected parameters, and the expected parameters of the atomic skill can be data related to the task to be executed, such as sensor data, requirements and targets of the task and target objects. Therefore, while the initial trajectory is obtained, the sensor on the robot also needs to collect environment information related to the execution of each atomic skill. The environment information includes but is not limited to the default target position, the target pose, whether the dual arms use re-planning, whether the interpolation of the given trajectory is performed and the execution speed.

[0061] For example, when the robot executes the grabbing task, the position and pose of the target object need to be perceived; when the robot executes the playback task, the interpolation method and the playback speed of the action need to be acquired. The environment data acquired by the sensor (such as a laser radar, a depth camera and the like) can be combined with the initial trajectory, so as to form a more complete execution plan.

[0062] In some embodiments, the expected parameters of the atomic skill can be parameters output after the image recognition processing and the pose estimation operation are performed based on the environment information around the robot after the environment information around the robot is collected by the scanning device on the robot. For example, the expected parameters can include the changed object type and the pose change data of the target object.

[0063] For example, when the scanning device collects the information of the environment around the robot, and identifies that the target objects to be grasped change from one to three, the expected parameters can be the pose information and the item type information of the three target objects to be grasped.

[0064] In an optional embodiment of the present application, step S203 comprises:

[0065] S203a: determining the trajectory starting point data according to the atomic skill.

[0066] In this embodiment, when the atomic skill executes the corresponding subtask, the starting point position and the ending point position of the actuator controlled by the atomic skill can be extracted from the corresponding subtask target description information as the trajectory starting point data of the atomic skill.

[0067] S203b: performing trajectory planning processing according to the trajectory starting point data to obtain an actuator motion trajectory.

[0068] In this embodiment, the trajectory starting point data can be calculated by a trajectory planning module of the robot or in communication connection with the robot to perform trajectory planning processing to obtain an actuator motion trajectory. The actuator motion trajectory can be the motion trajectory of the end of the actuator controlled by the atomic skill.

[0069] In an optional embodiment of the present application, step S203b specifically comprises: performing trajectory planning processing according to the trajectory starting point data by a teleoperation method or a trajectory planning algorithm to obtain an actuator motion trajectory.

[0070] For example, in the atomic skill of “moving”, the trajectory planning algorithm can be an interpolation-based method, such as a cubic spline interpolation, to ensure that the trajectory is smooth and meets the kinematic constraints.

[0071] In this embodiment, to improve the accuracy of trajectory planning, when performing trajectory planning processing by the teleoperation method or the trajectory planning algorithm, the influencing factors need to be input into the planning module of the robot. The influencing factors include but are not limited to the current state of the robot, the target state, and the required time requirement.

[0072] S203c: combining the actuator motion trajectories corresponding to the atomic skills to obtain an initial trajectory.

[0073] In this embodiment, the combination processing can be the processing of the combined first and last ends of the multiple actuator motion trajectories in the order of execution.

[0074] In an optional embodiment of the present application, the combination processing can also be a process of combining and connecting the points on the multiple actuator motion trajectories in the order of the shortest path after calculating the shortest path by the shortest path planning algorithm when the task to be executed changes from a single target to multiple targets.

[0075] In this embodiment, the initial trajectory can be a movement route formed between the starting point of the first segment of the effector movement trajectory and the ending point of the last segment of the effector movement trajectory when the robot moves according to each segment of the effector movement trajectory.

[0076] In an optional embodiment of the present application, step S203 further comprises: when the initial trajectory is obtained by combining the effector movement trajectories corresponding to the atomic skills, recording the time stamp of each point on the initial trajectory to determine the default speed, and controlling the robot to execute the atomic skill according to the default speed.

[0077] In this embodiment, the time stamp is recorded to determine the default speed so that the robot can more accurately complete the corresponding subtask when the atomic skill is subsequently controlled. In addition, whether trajectory deviation occurs when the atomic skill is executed is detected.

[0078] S204: Obtain environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy.

[0079] In this embodiment, the environmental perception information can be collected by a scanning device such as a laser radar or a depth camera in a sensor on the robot to acquire environmental data around the robot. The environmental perception information is related to the execution of each atomic skill. The environmental perception information includes but is not limited to the position of the default target object, the posture of the target object, whether the robot's arms use re-planning, whether the initial trajectory is interpolated, and the execution speed of the effector. For example, when the robot executes the "grabbing" subtask, the position and posture of the target object need to be perceived; when the "playback" subtask is executed, the interpolation method and the playback speed of the robot action need to be perceived.

[0080] In an optional embodiment of the present application, step S204 specifically comprises:

[0081] S204a: Determine atomic skill reuse mode update information according to the environmental perception information.

[0082] S204b: Adjust the calling information of the atomic skill according to the atomic skill reuse mode update information to obtain a target execution strategy.

[0083] In this embodiment, the reuse mode of the atomic skill is adjusted in real time based on the environmental perception information. The target execution strategy can be information of a control instruction containing the calling mode, the calling sequence, and the calling number of the atomic skill.

[0084] For example, when the position of the target object changes, the "moving" subtask has not been completed, and the robot needs to be called again to execute the atomic skill associated with the subtask to move to the new position of the target object. When the robot executes the "grabbing" subtask, if a new target object appears in front of the robot based on the environment perception information, the robot updates the task execution strategy according to the environment perception information, that is, performs skill reuse adjustment processing to obtain a new calling sequence of atomic skills, such as repeatedly calling the atomic skill associated with the "grabbing" subtask multiple times. The robot can also be controlled to start multi-object grabbing to improve the task execution efficiency.

[0085] In an optional embodiment of the present application, the skill reuse adjustment processing in step S204 according to the environment perception information to obtain the target execution strategy comprises:

[0086] S204c: determining the reuse mode update information of the atomic skill according to the environment perception information.

[0087] In this embodiment, the environment perception information can include environment changes and pose changes of the target object, a matching index relationship between different environment changes and the calling mode of the atomic skill, and a matching index relationship between the target object change information and the calling mode of the atomic skill. The matching index relationship can be pre-set for the robot to use at any time, so that the robot can index the corresponding calling mode of the atomic skill according to the pre-set matching index relationship based on the environment perception information, and determine the corresponding calling mode of the atomic skill as the reuse mode update information of the atomic skill.

[0088] S204d: adjusting the calling information of the atomic skill according to the reuse mode update information of the atomic skill to obtain the target execution strategy.

[0089] In this embodiment, the reuse mode update information of the atomic skill can be the calling sequence and frequency of the atomic skill in the current changed environment. In this embodiment, the calling mode of the atomic skill corresponding to each step in the entire target task can be obtained by using the calling sequence and frequency of the atomic skill in the current changed environment, to form the atomic skill calling information of the entire target task. The target execution strategy is the atomic skill calling information of the target task.

[0090] S205: performing trajectory replanning processing on the initial trajectory according to the target execution strategy, the expected parameters, and the environment perception information to obtain a target trajectory.

[0091] In this embodiment, the trajectory replanning processing means that when the environment changes, the robot needs to replan the current initial trajectory to ensure the smooth progress of each subtask. Based on the environment changes, such as the position change of the target object, the initial trajectory is corrected in real time to make the robot adapt to the environment changes and adjust the execution sequence of the atomic skill and update the initial trajectory.

[0092] In this embodiment, the goal of trajectory re-planning is to enable the robot to adapt to environmental changes and adjust the execution strategy, and then modify the initial trajectory to obtain the corrected target trajectory.

[0093] Specifically, in an optional embodiment of the present application, step S205 includes:

[0094] S205a: determining default environment information according to the target execution strategy.

[0095] In this embodiment, the target execution strategy contains atomic skill invocation information of the target task, according to which the initial trajectory to be completed can be determined, and then the surrounding environment information is collected according to the initial trajectory, and finally the default environment information is obtained.

[0096] S205b: comparing and calculating the default environment information and the environment perception information to obtain trajectory correction information.

[0097] In this embodiment, the robot compares the detected environment perception information with the default environment information. If the default environment information is the same as the environment perception information, trajectory re-planning is not needed, otherwise, according to the default environment information of the atomic skill and the detected environment perception information, trajectory correction information is calculated, so as to obtain the target trajectory after real-time correction.

[0098] S205c: adjusting according to the trajectory correction information, the expected parameter and the initial trajectory to obtain the target trajectory.

[0099] In an optional embodiment of the present application, the expected parameter includes an expected robot body pose matrix. Correspondingly, step S205c includes: step c1: determining an initial robot body pose matrix according to the trajectory correction information and the initial trajectory.

[0100] In this embodiment, the initial robot body pose matrix can be a homogeneous transformation matrix calculated by the six-dimensional pose point of the initial trajectory.

[0101] Step c2: determining a target robot body pose matrix according to the expected robot body pose matrix and the initial robot body pose matrix.

[0102] In this embodiment, the target robot body pose matrix can be obtained by matrix multiplication operation using the expected robot body pose matrix and the initial robot body pose matrix.

[0103] In this embodiment, the expected robot body pose matrix can be determined according to target object feature data and pose change data, wherein the target object feature data and the pose change data can be generated after the robot automatically scans the surrounding environment and then is identified and processed.

[0104] In this embodiment, the target object feature data can be the object type of the target object, for example, the target object of the original task is a cup, and the robot identifies that the changed target object is a bottle of mineral water. The pose change data can be the position and attitude change of the target object. These two are expected parameters, which can provide a basis for the target grasping task of the robot, and the expected robot body pose matrix can be obtained through corresponding data matrix transformation operations.

[0105] Step c3: performing solving operation processing according to the target robot body pose matrix to obtain a robot end pose trajectory.

[0106] In this embodiment, the robot end pose trajectory can be a calculated trajectory six-dimensional pose of the solution of the target robot body pose matrix, and a to-be-executed trajectory obtained after point-by-point merging.

[0107] Step c4: determining a transition trajectory according to the robot end pose trajectory and the initial robot body pose matrix.

[0108] In this embodiment, the transition trajectory can be an interpolation between the coordinates of the end of the trajectory corresponding to the executed subtask and the start point of the trajectory corresponding to the next to-be-executed subtask, which is used to transition from the previous trajectory to the new trajectory.

[0109] Step c5: performing pose point-by-point merging processing according to the transition trajectory and the robot end pose trajectory to obtain a target trajectory.

[0110] Figure 3 The process diagram of the trajectory re-planning provided in this application is shown in the following figure.

[0111] As shown in the following figure, Figure 3 as the initial coordinates of the robot, as the expected target pose in the expected parameters described in the above steps and the new expected pose set through environmental perception or manually, , respectively. and respectively are the expected trajectory recorded in the above steps and the target trajectory after the re-planning processing, refers to an interpolation between the coordinates of the end of the trajectory corresponding to the executed subtask and the start point of the trajectory corresponding to the next to-be-executed subtask, which is used to transition from the previous trajectory to the trajectory of the next subtask, so that the robot can perform the trajectory of the next subtask through and achieve the desired atomic task. The trajectory re-planning process is to re-plan the initial trajectory to output a trajectory that can achieve the current task after a series of transformations The trajectory re-planning process uses the following calculation method:

[0112]

[0113] wherein, refers to the target robot body pose matrix calculated by the six-dimensional pose point of the target trajectory, refers to the target robot body pose relative to the recorded desired robot body pose matrix in the desired parameters, refers to the initial robot body pose matrix calculated by the six-dimensional pose point of the initial trajectory.

[0114] But since this homogeneous transformation matrix cannot be directly obtained, the calculation formula used in the trajectory re-planning process can be expanded to obtain the target robot body pose matrix solving formula:

[0115]

[0116] wherein refers to the homogeneous transformation matrix of the desired target coordinate relative to the robot coordinate, refers to the desired target coordinate relative to the desired robot body pose matrix in the recorded desired parameters. Then the target robot body pose matrix calculated by the six-dimensional pose point of the target trajectory The trajectory six-dimensional pose solved point by point is merged, that is, the target robot end pose matrix represented by the trajectory, and the transition trajectory can form the target trajectory.

[0117] In this embodiment, , and are all 1x6 matrices. , , and are all 4x4 matrices.

[0118] S206: Control each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved.

[0119] In this embodiment, after obtaining the target trajectory, the control of the robot is performed on each atomic skill to execute the corresponding subtask to cope with the control scenario of environmental changes. If no abnormality is detected in the subsequent process, the control of each atomic skill can continue to be performed according to the target trajectory to execute the corresponding subtask until the task goal is achieved.

[0120] Based on the above embodiment, the robot control method provided in an optional embodiment of the present application further includes the following steps:

[0121] A: In the process of controlling each atomic skill according to the target trajectory to execute the corresponding subtask to achieve the expected subtask goal until the task goal is achieved, real-time environmental change information is acquired.

[0122] B: When it is detected that the environmental change information and the environmental perception information have information differences, the environmental perception information is acquired, and skill reuse adjustment processing is performed according to the environmental perception information to obtain a target execution strategy.

[0123] In this embodiment, when it is detected that the environmental change information and the environmental perception information have information differences, data comparison can be performed to determine whether the environment in which the robot is located has changed. If it is detected that a change has occurred, step S204 is returned until the task goal is achieved. Otherwise, the robot continues to control the atomic skill according to the target trajectory to execute the corresponding subtask until the task goal is achieved.

[0124] In summary, the robot control method provided in the embodiments of the present application first performs skill disassembly processing according to the to-be-executed task information to obtain a plurality of independent atomic skills, then performs trajectory planning processing according to the atomic skills to obtain an initial trajectory, and records the expected parameters of the atomic skills. Then, the environmental perception information is acquired to perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy, so as to ensure that the robot controls the atomic skill according to the target execution strategy to execute the corresponding subtask and avoid errors. Furthermore, the initial trajectory is subjected to trajectory re-planning processing according to the target execution strategy, the expected parameters, and the environmental perception information to achieve the purpose of correcting the trajectory and obtain a target trajectory. Finally, each atomic skill is controlled according to the target trajectory to execute the corresponding subtask to achieve the expected subtask goal until the task goal is achieved. In this process, the target trajectory obtained through the trajectory re-planning processing can ensure that the task is efficiently and accurately completed.

[0125] Meanwhile, the trajectory is corrected through the collection of the perception environment information to improve the flexibility of the control task execution.

[0126] Figure 4 The structure diagram of the robot control device provided in the present application is as follows: Figure 4As shown, the robot control device provided in the embodiment includes an acquisition module 41, a skill disassembling module 42, a trajectory planning module 43, and a skill execution module 44.

[0127] The acquisition module 41 is configured to acquire task information to be executed and a task target.

[0128] The skill disassembling module 42 is configured to perform skill disassembling processing according to the task information to be executed, to obtain a plurality of independent atomic skills, each of which is used to execute a subtask to achieve an expected subtask target, and all the expected subtask targets are achieved to achieve the task target.

[0129] The trajectory planning module 43 is configured to perform trajectory planning processing according to the atomic skills, to obtain an initial trajectory, and to record expected parameters of the atomic skills.

[0130] The acquisition module 41 is further configured to acquire environment perception information, and to perform skill reuse adjustment processing according to the environment perception information, to obtain a target execution strategy.

[0131] The trajectory planning module 43 is further configured to perform trajectory re-planning processing on the initial trajectory according to the target execution strategy, the expected parameters, and the environment perception information, to obtain a target trajectory.

[0132] The skill execution module 44 is configured to control each atomic skill to execute a corresponding subtask to achieve an expected subtask target according to the target trajectory, until the task target is achieved.

[0133] In a possible implementation, the task information to be executed includes a target position, a target pose, and robot state information. Correspondingly, the skill disassembling module 42 is specifically configured to: perform task analysis processing according to the target position, the target pose, and the robot state information, to obtain a plurality of subtask target description information; perform task disassembling processing according to the subtask target description information, to obtain a plurality of subtasks; and determine corresponding function modules as atomic skills according to the subtasks from pre-stored robot function modules, to obtain a plurality of independent atomic skills.

[0134] In a possible implementation, the trajectory planning module 43 is specifically configured to: determine trajectory starting point data according to the atomic skills; perform trajectory planning processing according to the trajectory starting point data, to obtain an actuator motion trajectory; and perform combination processing on actuator motion trajectories corresponding to the atomic skills, to obtain the initial trajectory.

[0135] In a possible implementation, the trajectory planning module 43 is specifically configured to: perform trajectory planning processing according to the trajectory starting point data by a teleoperation method or a trajectory planning algorithm, to obtain an actuator motion trajectory.

[0136] In a possible implementation, the acquisition module 41 is further configured to determine the reuse mode update information of the atomic skill according to the environment perception information; and adjust the calling information of the atomic skill according to the reuse mode update information of the atomic skill to obtain the target execution strategy.

[0137] In a possible implementation, the trajectory planning module 43 is configured to determine default environment information according to the target execution strategy; perform comparison and calculation processing according to the default environment information and the environment perception information to obtain trajectory correction information; and perform adjustment processing according to the trajectory correction information, the expected parameter and the initial trajectory to obtain the target trajectory.

[0138] In a possible implementation, the expected parameter includes an expected trajectory and an expected robot body pose matrix. Correspondingly, the trajectory planning module 43 is further configured to determine an initial robot body pose matrix according to the trajectory correction information and the initial trajectory; determine a target robot body pose matrix according to the expected robot body pose matrix and the initial robot body pose matrix; perform solving operation processing according to the target robot body pose matrix to obtain a robot end pose trajectory; determine a transition trajectory according to the robot end pose trajectory and the initial robot body pose matrix; and perform pose point-by-point merging processing according to the transition trajectory and the robot end pose trajectory to obtain the target trajectory.

[0139] In a possible implementation, the acquisition module 41 is further configured to acquire environment change information in real time in a process of controlling each atomic skill to perform a corresponding subtask according to the target trajectory to achieve an expected subtask target, until a task target is achieved. When it is detected that the environment change information and the environment perception information have information differences, the environment perception information is acquired again, and skill reuse adjustment processing is performed according to the environment perception information to obtain the target execution strategy.

[0140] The robot control device provided in this embodiment can execute the method provided in the method embodiment, and has similar implementation principles and technical effects. Details are not described herein again.

[0141] Figure 5 A structural schematic diagram of an electronic device provided in this application is shown in FIG. 1. As shown in the figure, the electronic device provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected through a bus 504. Figure 5

[0142] In the specific implementation process, the at least one processor 501 executes the computer execution instructions stored in the memory 502, so that the at least one processor 501 executes the method described above.

[0143] ​The specific implementation process of the processor 501 can refer to the method embodiments described above, which have similar implementation principles and technical effects, and thus will not be described here again.

[0144] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0145] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0146] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0147] The present application also provides a computer program product, comprising a computer program, which is executed by a processor to implement the above method.

[0148] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when the processor executes the computer execution instructions, the above method is implemented.

[0149] The above-mentioned readable storage medium can be realized by any type of volatile or nonvolatile storage devices or their combinations, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0150] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0151] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0153] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0154] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0155] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0156] Finally, it should be noted that: those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed in the present application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A robot control method characterized by, The method comprises the following steps: acquiring task information to be executed and a task target; performing skill decomposition processing according to the task information to be executed, to obtain a plurality of independent atomic skills, each atomic skill being used to execute a subtask to achieve an expected subtask target, and all expected subtask targets being achieved to achieve the task target; performing combination processing on an execution trajectory of an executor corresponding to the atomic skill to obtain an initial trajectory, recording a time stamp of each point on the initial trajectory to determine a default speed, and controlling a robot to execute the atomic skill according to the default speed and acquiring an expected parameter of the atomic skill; acquiring environmental perception information in real time, and performing skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy, the environmental perception information including a position of a default target object, a posture of the target object, whether a double arm of the robot uses re-planning, whether interpolation is performed on the initial trajectory, and an execution speed of the executor; dynamically adjusting the initial trajectory according to the target execution strategy, the expected parameter, and the environmental perception information to obtain a target trajectory, the expected parameter including an expected robot body pose matrix; controlling each atomic skill to execute a corresponding subtask to achieve the expected subtask target according to the target trajectory, until the task target is achieved; the dynamically adjusting the initial trajectory according to the target execution strategy, the expected parameter, and the environmental perception information to obtain a target trajectory comprises: determining default environmental information according to the target execution strategy; performing comparison and calculation processing on the default environmental information and the environmental perception information to obtain trajectory correction information; performing adjustment processing on the trajectory correction information, the expected parameter, and the initial trajectory to obtain the target trajectory; the performing adjustment processing on the trajectory correction information, the expected parameter, and the initial trajectory to obtain the target trajectory comprises: determining an initial robot body pose matrix according to the trajectory correction information and the initial trajectory; determining a target robot body pose matrix according to an expected robot body pose matrix and the initial robot body pose matrix; performing conversion on a solution of the target robot body pose matrix to obtain a trajectory six-dimensional pose, and performing point-by-point merging on the trajectory to be executed to obtain a robot end pose trajectory, the target robot body pose matrix being obtained by matrix multiplication of the expected robot body pose matrix and the initial robot body pose matrix; determining a transition trajectory according to the robot end pose trajectory and the initial robot body pose matrix; performing point-by-point pose merging processing on the transition trajectory and the robot end pose trajectory to obtain the target trajectory; the performing skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy comprises: determining reuse mode update information of the atomic skill according to the environmental perception information; adjusting calling information of the atomic skill according to the reuse mode update information of the atomic skill to obtain the target execution strategy.

2. The method of claim 1, wherein, the task information to be executed includes target position, target posture, and robot state information. According to the task to be executed information, task decomposition processing is performed to obtain a plurality of independent atomic skills, including: According to the target position, target attitude and robot state information, task analysis processing is performed to obtain a plurality of sub-task target description information; According to the sub-task target description information, task decomposition processing is performed to obtain a plurality of sub-tasks; According to the sub-tasks, corresponding function modules are selected from the pre-stored robot function modules to determine the atomic skills, so as to obtain a plurality of independent atomic skills.

3. The method of claim 2, wherein, According to the atomic skills, trajectory planning processing is performed to obtain an initial trajectory, including: According to the atomic skills, the trajectory starting point data is determined; According to the trajectory starting point data, trajectory planning processing is performed to obtain an actuator motion trajectory; The actuator motion trajectories corresponding to each atomic skill are combined to obtain an initial trajectory.

4. The method of claim 3, wherein, According to the trajectory starting point data, trajectory planning processing is performed to obtain an actuator motion trajectory, including: According to the trajectory starting point data, trajectory planning processing is performed by a teleoperation method or a trajectory planning algorithm to obtain an actuator motion trajectory.

5. The method according to any one of claims 1 to 4, characterized in that, Further comprising: In the process of controlling each atomic skill according to the target trajectory to execute the corresponding sub-task to achieve the expected sub-task target until the task target is achieved, real-time environment change information is obtained; When detecting that the environment change information and the environment perception information have information differences, the environment perception information is returned, and skill reuse adjustment processing is performed according to the environment perception information to obtain a target execution strategy.

6. A robot control device characterized by comprising: Comprising: An acquisition module for acquiring task information to be executed and a task target; A skill decomposition module for performing skill decomposition processing according to the task information to be executed to obtain a plurality of independent atomic skills, each atomic skill being used to execute a sub-task to achieve an expected sub-task target, all expected sub-task targets being achieved when the task target is achieved; A trajectory planning module for combining the actuator running trajectories corresponding to the atomic skills to obtain an initial trajectory, recording the time stamp of each point on the initial trajectory to determine a default speed, and controlling the robot to execute the atomic skills according to the default speed and recording the expected parameters of the atomic skills; The acquisition module is also used to obtain real-time environment perception information and perform skill reuse adjustment processing according to the environment perception information to obtain a target execution strategy, the environment perception information including the position of the default target object, the attitude of the target object, whether the robot arms use re-planning, whether the initial trajectory is interpolated, and the execution speed of the actuator; The trajectory planning module is also used to dynamically adjust the initial trajectory according to the target execution strategy, the expected parameters and the environment perception information to obtain a target trajectory, the expected parameters including an expected robot body pose matrix; A skill execution module for controlling each atomic skill according to the target trajectory to execute the corresponding sub-task to achieve the expected sub-task target until the task target is achieved; The acquisition module is specifically configured to determine default environment information according to the target execution strategy, perform comparison and calculation processing on the default environment information and the environment perception information to obtain trajectory correction information, and perform adjustment processing on the trajectory correction information, the expected parameter and the initial trajectory to obtain a target trajectory. The trajectory planning module is specifically configured to determine an initial robot body pose matrix according to the trajectory correction information and the initial trajectory, determine a target robot body pose matrix according to an expected robot body pose matrix and the initial robot body pose matrix, obtain a trajectory six-dimensional pose through conversion of a solution of the target robot body pose matrix, and obtain a robot end pose trajectory by point-by-point merging of the trajectory and the to-be-executed trajectory, the target robot body pose matrix being obtained through matrix multiplication operation of the expected robot body pose matrix and the initial robot body pose matrix, determine a transition trajectory according to the robot end pose trajectory and the initial robot body pose matrix, and perform pose point-by-point merging processing on the transition trajectory and the robot end pose trajectory to obtain the target trajectory. The acquisition module is specifically configured to determine the reuse mode update information of the atomic skill according to the environment perception information. The calling information of the atomic skill is adjusted according to the reuse mode update information of the atomic skill to obtain a target execution strategy.

7. An electronic device, comprising: Comprise: Memory, processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method in any one of claims 1 to 5.

Images