Robot control method and device and electronic equipment

Abstract

The embodiment of the invention provides a robot control method and device and electronic equipment, and belongs to the technical field of robots. The method comprises the steps that to-be-executed task information and a task target are acquired; according to the to-be-executed task information, skill disassembling processing is carried out, and a plurality of independent atomic skills are obtained; trajectory planning processing is carried out according to the atomic skills, an initial trajectory is obtained, and expected parameters of the atomic skills are obtained; acquiring environment perception information, and performing skill reuse adjustment processing according to the environment perception information to obtain a target execution strategy; performing track re-planning processing on the initial track according to the target execution strategy, the expected parameter and the environment perception information to obtain a target track; and controlling each atomic skill to execute the corresponding sub-task according to the target trajectory so as to achieve an expected sub-task target until the task target is achieved. The method is used for achieving the effect of improving the flexibility and efficiency of the control task of the robot.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot control method, device and electronic equipment. Background Art

[0002] With the development of robotics technology, in order to complete complex tasks, it is usually necessary to divide the robot's control system into multiple modular skill units, each skill unit is used to perform a corresponding control task.

[0003] However, during the research and design of the robot control system, the inventors discovered that when the robot is performing complex tasks, while coping with changes in its own position, it also needs to cope with complex and changing environmental information and changing task objectives. As the complexity of environmental changes and target changes increases, the robot's control execution capability is greatly reduced, resulting in low flexibility and efficiency in the execution of control tasks.

[0004] Therefore, a robot control method is urgently needed to solve the above technical problems. Summary of the invention

[0005] The embodiments of the present application provide a robot control method, device and electronic device to achieve the effect of improving the flexibility and efficiency of the robot's control tasks.

[0006] In a first aspect, an embodiment of the present application provides a robot control method, comprising:

[0007] Obtain information about tasks to be performed and task objectives;

[0008] Perform skill decomposition processing according to the task information to be executed to obtain multiple independent atomic skills, each atomic skill is used to execute a subtask to achieve the expected subtask goal, and the task goal is achieved when all the expected subtask goals are achieved;

[0009] Performing trajectory planning processing according to the atomic skill to obtain an initial trajectory and acquire expected parameters of the atomic skill;

[0010] Acquire environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy;

[0011] Performing trajectory replanning processing on the initial trajectory according to the target execution strategy, the expected parameters and the environmental perception information to obtain a target trajectory;

[0012] According to the target trajectory, each atomic skill is controlled to execute the corresponding subtask to achieve the expected subtask goal until the task goal is achieved.

[0013] In one possible implementation, the information of the task to be executed includes a target position, a target posture and a robot state information; performing task decomposition processing based on the information of the task to be executed to obtain a plurality of independent atomic skills, including: performing task analysis processing based on the target position, the target posture and the robot state information to obtain a plurality of sub-task target description information; performing task decomposition processing based on the sub-task target description information to obtain a plurality of sub-tasks; selecting a corresponding functional module from pre-stored robot functional modules based on the sub-tasks to determine as an atomic skill, so as to obtain a plurality of independent atomic skills.

[0014] In a possible implementation, performing trajectory planning processing according to the atomic skill to obtain an initial trajectory includes: determining trajectory starting point data according to the atomic skill; performing trajectory planning processing according to the trajectory starting point data to obtain a segment of actuator motion trajectory; and combining the actuator motion trajectories corresponding to each atomic skill to obtain an initial trajectory.

[0015] In a possible implementation, performing trajectory planning processing according to the trajectory starting point data to obtain a segment of actuator motion trajectory includes: performing trajectory planning processing according to the trajectory starting point data through a remote operation method or a trajectory planning algorithm to obtain a segment of actuator motion trajectory.

[0016] In a possible implementation, the skill reuse adjustment processing is performed according to the environmental perception information to obtain a target execution strategy, including: determining the reuse mode update information of the atomic skill according to the environmental perception information; adjusting the calling information of the atomic skill according to the reuse mode update information of the atomic skill to obtain a target execution strategy.

[0017] In a possible implementation, the replanning of the initial trajectory according to the target execution strategy, the expected parameters, and the environmental perception information to obtain the target trajectory includes: determining default environmental information according to the target execution strategy; performing comparison and calculation processing according to the default environmental information and the environmental perception information to obtain trajectory correction information; and performing adjustment processing according to the trajectory correction information, the expected parameters, and the initial trajectory to obtain the target trajectory.

[0018] In a possible implementation, the expected parameters include an expected robot body pose matrix; accordingly, the adjustment processing is performed according to the trajectory correction information, the expected parameters and the initial trajectory to obtain the target trajectory, including: determining the initial robot body pose matrix according to the trajectory correction information and the initial trajectory; determining the target robot body pose matrix according to the expected robot body pose matrix and the initial robot body pose matrix; performing a solving operation according to the target robot body pose matrix to obtain a robot terminal pose trajectory; determining a transition trajectory according to the robot terminal pose trajectory and the initial robot body pose matrix; performing a pose point-by-point merging processing according to the transition trajectory and the robot terminal pose trajectory to obtain the target trajectory.

[0019] In a possible implementation, it also includes: in the process of controlling each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved, real-time acquisition of environmental change information; when it is detected that there is an information difference between the environmental change information and the environmental perception information, returning to the environmental perception information, and performing skill reuse adjustment processing according to the environmental perception information to obtain the target execution strategy.

[0020] In a second aspect, an embodiment of the present application provides a robot control device, comprising:

[0021] The acquisition module is used to obtain the task information and task objectives to be executed;

[0022] A skill decomposition module is used to perform skill decomposition processing according to the task information to be executed, and obtain multiple independent atomic skills. Each atomic skill is used to execute a subtask to achieve the expected subtask goal. When all the expected subtask goals are achieved, the task goal is achieved;

[0023] A trajectory planning module, used to perform trajectory planning processing according to the atomic skill, obtain an initial trajectory, and acquire expected parameters of the atomic skill;

[0024] The acquisition module is further used to acquire environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy;

[0025] The trajectory planning module is further used to perform trajectory replanning processing on the initial trajectory according to the target execution strategy, the expected parameters and the environmental perception information to obtain a target trajectory;

[0026] The skill execution module is used to control each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0028] The memory stores computer-executable instructions;

[0029] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0030] A robot control method, device and electronic device provided in an embodiment of the present application first performs skill disassembly processing according to the information of the task to be executed to obtain multiple independent atomic skills, then performs trajectory planning processing according to the atomic skills to obtain the initial trajectory, and obtains the expected parameters of the atomic skills at the same time. Then obtain environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain the target execution strategy, so as to ensure that the robot controls the atomic skills to execute the corresponding subtasks according to the target execution strategy to avoid errors. In addition, the initial trajectory is replanned according to the target execution strategy, the expected parameters and the environmental perception information to achieve the purpose of correcting the trajectory and obtain the target trajectory. Finally, each atomic skill is controlled to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved. In this process, the target trajectory obtained by the trajectory replanning process can ensure the efficient and accurate completion of the task. At the same time, the trajectory is corrected by collecting and correcting the perceived environmental information to improve the flexibility of the control task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0032] Figure 1 A schematic diagram of a scenario of the robot control method provided in this application;

[0033] Figure 2 A schematic diagram of the flow chart of the robot control method provided in this application;

[0034] Figure 3 Schematic diagram of the trajectory replanning process provided for this application;

[0035] Figure 4 A schematic diagram of the structure of the robot control device provided for this application;

[0036] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0037] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0038] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0039] In the prior art, the control system of the robot is divided into multiple modular skill units, but each skill unit only uses a fixed skill call sequence for the control instructions, and these call sequences are pre-set or pre-trained by the designer. Therefore, the existing control method cannot be quickly adjusted according to the changing task requirements, environmental changes or external interventions. For example: in a complex multi-task environment (such as the number of targets to be grasped increases from one to three, and the coordinates and postures of each target are different), if the robot lacks the ability to flexibly respond to the changing task requirements or the changing task environment by generating appropriate task trajectories and the call sequence of different skill units, the robot's control execution ability will be greatly reduced, resulting in problems such as low flexibility and efficiency in the execution of control tasks, such as the robot's control process being slow to respond, the task execution cycle being longer, or only being able to complete the task of grasping the initial target object.

[0040] In response to the above technical problems, the inventors proposed the following inventive concept: by decomposing the task into multiple atomic skills and configuring the initial trajectory and environmental adaptation mechanism for each atomic skill, that is, obtaining environmental change information, and adjusting the calling order and calling method of the atomic skills during the task execution, and re-planning the trajectory at the same time, ensuring that the robot can flexibly respond to changes in the environment, thereby improving the flexibility and efficiency of control task execution.

[0041] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0042] Figure 1 A schematic diagram of a scenario of the robot control method provided in this application, such as Figure 1As shown, the specific application scenarios of the present application include: a robot 101, an initial target object 102, and a newly added target object 103, wherein a server and a scanning device for environmental information and target object posture data, such as a radar sensor or a depth camera, are installed on the robot 101. The server can execute the relevant steps of the robot control method in sequence according to the environmental information, and finally obtain the calling sequence corresponding to each functional module of the robot 101 and the movement trajectory of the actuator end. For example, the robot's movement function module is first controlled to move to the position of the initial target object 102, and then the robot's grasping function module is controlled to grasp the initial target object 102, and finally the control task is completed.

[0043] The initial target object 102 and the newly added target object 103 can be any object, waiting for the robot 101 to perform corresponding actions. Figure 1 The obstacles and newly added obstacles are obstacles that may be encountered by the robot 101 during the execution of the control task. The newly added target 103 represents the task requirement change scenario encountered by the robot 101 during the execution of the control task, and the newly added obstacles may represent the environment change scenario encountered by the robot 101 during the execution of the control task. When the robot 101 encounters the task requirement change scenario and the environment change scenario when executing the corresponding control task, it can deal with it through the robot control method provided in the embodiment of the present application to improve the flexibility and efficiency of the control task execution.

[0044] It should be noted that Figure 1 The application scenario of the robot control method shown only represents an application scenario of target object grasping and robot movement. The robot control method proposed in this embodiment can also be applied to other application scenarios, such as when the robot is disturbed by external forces and its moving trajectory is deviated. This embodiment is not limited here.

[0045] Figure 2 A schematic diagram of the flow chart of the robot control method provided in this application, such as Figure 2 As shown, the execution subject of this method can be Figure 1 The service area in the robot 101 shown may also be other controllers or related computer devices with control functions and data processing functions, and this embodiment does not impose any particular limitation on this.

[0046] like Figure 1 As shown, the robot control method includes:

[0047] S201: Obtain information about tasks to be executed and task objectives.

[0048] In this embodiment, the task information to be performed may be information related to the robot control task obtained based on the input conditions of the task, for example, the task information to be performed may include task requirement information. The task target may be the execution result of the task to be performed by the robot.

[0049] S202: Perform skill decomposition processing according to the information of the task to be executed to obtain multiple independent atomic skills. Each atomic skill is used to execute a subtask to achieve the expected subtask goal. When all the expected subtask goals are achieved, the task goal is achieved.

[0050] In this embodiment, the skill disassembly process may be a process of disassembling the task information to be executed into multiple independent subtasks and then associating them with the corresponding atomic skills according to the nature of the subtasks or the task requirements. An atomic skill refers to an independently executed functional module of a robot, such as a robot's mobile functional module, a grasping functional module, or an interactive functional module. In this embodiment, the task objective may be defined based on the input conditions of the task, wherein the input conditions of the task may include information related to the task, such as the position of the target object, the posture information of the target object, and the current state information of the robot.

[0051] In an optional embodiment of the present application, the task information to be executed includes the target position, the target posture and the robot state information; accordingly, step S202 specifically includes:

[0052] S202a: Perform task analysis and processing according to the target position, target posture and robot state information to obtain multiple sub-task target description information.

[0053] In this embodiment, the robot state information may be the robot's moving speed, current position and current robot body posture information. Based on the robot state information, target position and target posture, descriptive information related to each subtask goal and task process of the task to be executed may be obtained. For example: Subtask 1 is to control the robot to move from point A to point B, and Subtask 2 is to control the robot to grab the target object at point B.

[0054] S202b: Decompose the task according to the subtask target description information to obtain multiple subtasks.

[0055] In this embodiment, the total task to be executed is composed of multiple subtasks. At this time, the total task to be executed needs to be decomposed according to the subtask description information, which means determining the corresponding subtask according to each subtask description information so as to match it to the corresponding functional module later.

[0056] S202c: Select corresponding function modules from pre-stored robot function modules according to the subtasks to determine them as atomic skills, so as to obtain multiple independent atomic skills.

[0057] In this embodiment, the corresponding function module can be selected from the pre-stored robot function modules according to the subtask by using the keyword related to the subtask to match the preset keyword in each function module. For example, if the keyword is movement, grasping or interaction, the corresponding movement function module, grasping function module and interaction function module can be selected as atomic skills respectively.

[0058] In this embodiment, the task goal is defined based on the input conditions of the task, and the task goal is decomposed into multiple subtasks such as moving subtasks, grasping subtasks, and interactive subtasks to ensure that the atomic skills corresponding to each subtask can be independently executed and achieve the expected goals. The pre-stored robot function module can be a function module that can complete the corresponding control task pre-stored in the robot control system, such as: moving function module, grasping function module, interactive function module and performance function module. In this embodiment, each independent atomic skill can be used to complete the corresponding subtask. After all the expected subtasks are executed, it means that the task to be executed has been completed, and the robot control process has achieved the task goal.

[0059] S203: Perform trajectory planning processing according to the atomic skill to obtain an initial trajectory and obtain expected parameters of the atomic skill.

[0060] In this embodiment, each disassembled atomic skill generally depends on certain expected parameters, which can be sensor data, task requirements, target objects and other data related to the task to be performed. Therefore, while obtaining the initial trajectory, the sensor on the robot also needs to collect environmental information related to the execution of each atomic skill. This environmental information includes but is not limited to the default target position, target posture, whether the arms use replanning, whether to interpolate the established trajectory, and the execution speed.

[0061] For example, when a robot performs a grasping task, it needs to sense the position and posture of the target object; when performing a playback task, it needs to obtain the interpolation method and playback speed of the action. Environmental data obtained by sensors (such as lidar, depth camera, etc.) can be combined with the initial trajectory to form a more complete execution plan.

[0062] In some embodiments, the expected parameters for obtaining the atomic skill may be parameters outputted after the robot's surrounding environment information is collected by a scanning device on the robot, and image recognition processing and posture estimation operations are performed based on the robot's surrounding environment information. For example, the expected parameters may include the object type and posture change data of the changed target object.

[0063] Exemplarily, when the scanning device collects information about the robot's surrounding environment and recognizes that the number of targets to be grasped changes from one to three, the expected parameters may be the position information and item type information of the three targets to be grasped.

[0064] In an optional embodiment of the present application, step S203 includes:

[0065] S203a: Determine trajectory starting point data according to the atomic skills.

[0066] In this embodiment, when an atomic skill executes a corresponding subtask, the starting point position and the end point position of the actuator controlled by the atomic skill can be extracted from the corresponding subtask target description information as the trajectory starting point data of the atomic skill.

[0067] S203b: Perform trajectory planning processing according to the trajectory starting point data to obtain a section of actuator motion trajectory.

[0068] In this embodiment, the trajectory starting point data can be calculated by a trajectory planning module that is built into the robot or is connected to the robot for trajectory planning processing to obtain a segment of actuator motion trajectory, which can be the motion trajectory of the actuator end controlled by the atomic skill.

[0069] In an optional embodiment of the present application, step S203b specifically includes: performing trajectory planning processing through a teleoperation method or a trajectory planning algorithm according to the trajectory starting point data to obtain a segment of actuator motion trajectory.

[0070] For example, in the atomic skill "Move", the trajectory planning algorithm can be based on an interpolation method, such as cubic spline interpolation, to ensure that the trajectory is smooth and complies with kinematic constraints.

[0071] In this embodiment, in order to improve the accuracy of trajectory planning, when performing trajectory planning through remote control method or trajectory planning algorithm, influencing factors need to be input into the planning module of the robot. The influencing factors include but are not limited to the current state of the robot, the target state and the required time requirements.

[0072] S203c: Combine the actuator motion trajectories corresponding to the atomic skills to obtain an initial trajectory.

[0073] In this embodiment, the combined processing may be a process of combining multiple segments of actuator motion trajectories in an end-to-end manner in the execution order.

[0074] In an optional embodiment of the present application, the combination processing can also be a process in which, after the task to be executed changes from a single target to multiple targets, the shortest path can be calculated by using the shortest path planning algorithm for each segment of the actuator motion trajectory, and then the points on the multiple segments of the actuator motion trajectory are combined and connected according to the shortest path.

[0075] In this embodiment, the initial trajectory may be a moving route formed from the starting point of the first actuator motion trajectory to the end point of the last actuator motion trajectory when the robot moves according to each actuator motion trajectory.

[0076] In an optional embodiment of the present application, step S203 also includes: when combining the actuator motion trajectories corresponding to each atomic skill to obtain the initial trajectory, recording the timestamp of each point on the initial trajectory to determine the default speed, and controlling the robot to execute the atomic skill according to the default speed.

[0077] In this embodiment, the timestamp is recorded to determine the default speed so that the robot can more accurately complete the corresponding subtask when controlling the atomic skill in the future, and detect whether the trajectory deviation occurs when the atomic skill is executed.

[0078] S204: Acquire environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy.

[0079] In this embodiment, environmental perception information can be collected by scanning devices on the robot, such as laser radar or depth cameras in sensors, to collect environmental data around the robot. And the environmental perception information is related to the execution of each atomic skill. Environmental perception information includes but is not limited to the position of the default target object, the posture of the target object, whether the robot's arms use replanning, whether the initial trajectory is interpolated, and the execution speed of the actuator. For example: when the robot is performing the "grasping" subtask, it needs to perceive the position and posture of the target object; when performing the "playback" subtask, it needs to perceive the interpolation method and playback speed of the robot's action.

[0080] In an optional embodiment of the present application, step S204 specifically includes:

[0081] S204a: Determine the atomic skill reuse method update information according to the environmental perception information.

[0082] S204b: Update the information of the atomic skill according to the reuse mode of the atomic skill to adjust the calling information of the atomic skill and obtain the target execution strategy.

[0083] In this embodiment, based on the environmental perception information, the reuse mode of the atomic skills is adjusted in real time, and the target execution strategy may be information of control instructions including the calling mode, calling sequence and calling times of the atomic skills.

[0084] For example, when the position of the target object changes, the "move" subtask has not been completed. At this time, the atomic skills related to the subtask need to be re-called to move the robot to the new position of the target object. When the robot executes the "grasp" subtask, if a new target object appears in front of the robot based on environmental perception information, the robot will update the task execution strategy based on the environmental perception information, that is, perform skill reuse adjustment processing to obtain a new atomic skill calling sequence, such as repeatedly calling the atomic skills related to the "grasp" subtask multiple times. The robot can also be controlled to start multi-object grasping to improve task execution efficiency.

[0085] In an optional embodiment of the present application, in step S204, skill reuse adjustment processing is performed according to the environment perception information to obtain a target execution strategy, including:

[0086] S204c: Determine the atomic skill reuse method update information according to the environmental perception information.

[0087] In this embodiment, environmental perception information may include changes in the environment and the posture of the target object. The matching index relationship between different environmental changes and the calling method of atomic skills, as well as the matching index relationship between the target object change information and the calling method of atomic skills can be pre-set so that the robot can use it at any time, so that the robot can index the corresponding calling method of atomic skills according to the pre-set matching index relationship through environmental perception information, and determine the corresponding atomic skill calling method as the atomic skill reuse method update information.

[0088] S204d: Update the information of the atomic skill according to the reuse mode of the atomic skill and adjust the calling information of the atomic skill to obtain the target execution strategy.

[0089] In this embodiment, the atomic skill reuse method update information can be the calling order and number of atomic skills in the current changing environment. In this embodiment, the calling method of the atomic skills corresponding to each step in the entire target task can be obtained by using the calling order and number of atomic skills in the current changing environment to form the atomic skill calling information of the entire target task. Among them, the target execution strategy is the atomic skill calling information of the target task.

[0090] S205: replanning the initial trajectory according to the target execution strategy, expected parameters and environmental perception information to obtain the target trajectory.

[0091] In this embodiment, trajectory replanning means that when the environment changes, the robot needs to replan the current initial trajectory to ensure the smooth progress of each subtask. Based on environmental changes, such as when the target position changes, the initial trajectory is corrected in real time to enable the robot to adapt to environmental changes and adjust the execution order of atomic skills and update the initial trajectory.

[0092] In this embodiment, the goal of trajectory replanning is to enable the robot to adapt to environmental changes and adjust the execution strategy, and then modify the initial trajectory to obtain a corrected target trajectory.

[0093] Specifically, in an optional embodiment of the present application, step S205 includes:

[0094] S205a: Determine default environment information according to the target execution strategy.

[0095] In this embodiment, the target execution strategy includes the atomic skill calling information of the target task. Based on this information, it can be determined that the initial trajectory must be completed, and then the surrounding environment information is collected according to the initial trajectory to finally obtain the default environment information.

[0096] S205b: Perform comparison and calculation based on the default environment information and the environment perception information to obtain trajectory correction information.

[0097] In this embodiment, the robot will compare the detected environmental perception information with the default environmental information. If the default environmental information is the same as the environmental perception information, there is no need to replan the trajectory. Otherwise, the robot will calculate the trajectory correction information based on the default environmental information of the atomic skill and the detected environmental perception information, so as to obtain the target trajectory after real-time correction.

[0098] S205c: Perform adjustment processing according to the trajectory correction information, the expected parameters and the initial trajectory to obtain the target trajectory.

[0099] In an optional embodiment of the present application, the expected parameters include an expected robot body pose matrix; accordingly, step S205c includes: step c1: determining an initial robot body pose matrix according to the trajectory correction information and the initial trajectory.

[0100] In this embodiment, the initial robot body posture matrix may be a homogeneous transformation matrix calculated from the six-dimensional posture points of the initial trajectory.

[0101] Step c2: Determine the target robot body pose matrix according to the expected robot body pose matrix and the initial robot body pose matrix.

[0102] In this embodiment, the target robot body pose matrix can be obtained by performing a matrix product operation on the desired robot body pose matrix and the initial robot body pose matrix.

[0103] In this embodiment, the desired robot body posture matrix can be determined based on the target object feature data and posture change data, wherein the target object feature data and posture change data can be generated after the robot automatically scans its surrounding environment and undergoes identification and data processing.

[0104] In this embodiment, the target feature data can be the object type of the target object. For example, the target object of the original task is a water cup, and the robot recognizes that the target object after the change is a bottle of mineral water. The posture change data can be the position and posture change of the target object. These two items can provide a basis for the robot's target grasping task as expected parameters. After the corresponding data matrix transformation operation, the expected robot body posture matrix can be obtained.

[0105] Step c3: Perform calculation and processing based on the target robot body pose matrix to obtain the robot terminal pose trajectory.

[0106] In this embodiment, the robot end posture trajectory can be a trajectory to be executed after converting the calculated solution of the target robot body posture matrix into a six-dimensional posture and merging point by point.

[0107] Step c4: Determine the transition trajectory based on the robot end pose trajectory and the initial robot body pose matrix.

[0108] In this embodiment, the transition trajectory may be an interpolation between the coordinates of the end of the trajectory corresponding to the executed subtask and the starting point of the trajectory corresponding to the next subtask to be executed, for transitioning from the previous trajectory to the new trajectory.

[0109] Step c5: Perform point-by-point merging processing based on the transition trajectory and the robot end pose trajectory to obtain the target trajectory.

[0110] Figure 3 Schematic diagram of the trajectory replanning process provided for this application.

[0111] like Figure 3 As shown, in this embodiment, As the initial coordinates of the robot, used for the inverse solution of the robot's kinematics, , They are the expected target pose in the expected parameters described in the above steps and the new expected pose set by environment perception or manual setting. and are the expected trajectory recorded in the above steps and the target trajectory after re-planning, Refers to the interpolation between the coordinates of the end of the trajectory corresponding to the executed subtask and the starting point of the trajectory corresponding to the next subtask to be executed, which is used to transition from the previous trajectory to the trajectory of the next subtask. The robot can then and Complete the expected atomic task. Among them, the trajectory replanning process is the initial trajectory After a series of transformations, the output is the trajectory that can complete the current task The trajectory replanning process uses the following calculation method:

[0112]

[0113] in, Refers to the target robot body pose matrix calculated from the six-dimensional pose points of the target trajectory. Refers to the target robot body pose relative to the recorded expected robot body pose matrix, Refers to the initial robot machine material matrix calculated from the six-dimensional pose points of the initial trajectory.

[0114] However, since this homogeneous transformation matrix cannot be obtained directly, the calculation formula used in trajectory replanning can be expanded to obtain the solution formula for the target robot body pose matrix:

[0115]

[0116] in Refers to the homogeneous transformation matrix of the desired target coordinates relative to the robot coordinates, It refers to the expected target coordinates relative to the expected robot body pose matrix in the recorded expected parameters. Then the target robot body pose matrix calculated by the six-dimensional pose points of the calculated target trajectory is calculated. The calculated six-dimensional trajectory poses are merged point by point to obtain the target robot terminal pose matrix The trajectory represented by , this trajectory and the transition trajectory can constitute the target trajectory.

[0117] In this embodiment, , and Both are 1×6 matrices. , , and Both are 4×4 matrices.

[0118] S206: Control each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved.

[0119] In this embodiment, after obtaining the target trajectory, each atomic skill of the robot is controlled to execute the corresponding subtask to cope with the control scenario of environmental changes. If no abnormality in the environmental perception information is detected in the subsequent process, each atomic skill can continue to be controlled according to the target trajectory to execute the corresponding subtask until the task goal is achieved.

[0120] Based on the above embodiment, a robot control method provided in an optional embodiment of the present application further includes the following steps:

[0121] A: While controlling each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal, until the task goal is achieved, real-time environmental change information is obtained.

[0122] B: When it is detected that there is an information difference between the environmental change information and the environmental perception information, it returns to obtain the environmental perception information, and performs skill reuse adjustment processing according to the environmental perception information to obtain the step of the target execution strategy.

[0123] In this embodiment, when the difference between the environmental change information and the environmental perception information is detected, data comparison can be performed to find out whether the environment in which the robot is located has changed. If a change is detected, the process returns to step S204 until the task goal is achieved. Otherwise, the robot continues to control the atomic skill to perform the corresponding subtask according to the target trajectory until the task goal is achieved.

[0124] In summary, the robot control method provided in the embodiment of the present application first performs skill disassembly processing according to the information of the task to be executed to obtain multiple independent atomic skills, and then performs trajectory planning processing according to the atomic skills to obtain the initial trajectory, and records the expected parameters of the atomic skills at the same time. Then obtain environmental perception information to perform skill reuse adjustment processing according to the environmental perception information to obtain the target execution strategy, ensuring that the robot controls the atomic skills to execute the corresponding subtasks according to the target execution strategy to avoid errors. In addition, the initial trajectory is replanned according to the target execution strategy, the expected parameters and the environmental perception information to achieve the purpose of correcting the trajectory and obtain the target trajectory. Finally, each atomic skill is controlled to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goals until the task goal is achieved. In this process, the target trajectory obtained by the trajectory replanning process can ensure the efficient and accurate completion of the task.

[0125] At the same time, the trajectory is corrected by collecting and sensing environmental information to improve the flexibility of control task execution.

[0126] Figure 4 A schematic diagram of the structure of the robot control device provided in this application, such as Figure 4As shown, the robot control device provided in this embodiment includes: an acquisition module 41, a skill disassembly module 42, a trajectory planning module 43 and a skill execution module 44.

[0127] The acquisition module 41 is used to acquire information about the task to be executed and the task target;

[0128] The skill decomposition module 42 is used to perform skill decomposition according to the information of the task to be executed, and obtain multiple independent atomic skills. Each atomic skill is used to execute a subtask to achieve the expected subtask goal. When all the expected subtask goals are achieved, the task goal is achieved;

[0129] The trajectory planning module 43 is used to perform trajectory planning processing according to the atomic skills, obtain the initial trajectory, and record the expected parameters of the atomic skills.

[0130] The acquisition module 41 is also used to acquire environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy.

[0131] The trajectory planning module 43 is further used to perform trajectory replanning processing on the initial trajectory according to the target execution strategy, expected parameters and environmental perception information to obtain the target trajectory.

[0132] The skill execution module 44 is used to control each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved.

[0133] In one possible implementation, the information of the task to be executed includes the target position, the target posture and the robot state information; accordingly, the skill disassembly module 42 is specifically used to: perform task analysis and processing according to the target position, the target posture and the robot state information to obtain a plurality of sub-task target description information; perform task disassembly processing according to the sub-task target description information to obtain a plurality of sub-tasks; and select the corresponding function module from the pre-stored robot function modules according to the sub-task to determine it as the atomic skill, so as to obtain a plurality of independent atomic skills.

[0134] In a possible implementation, the trajectory planning module 43 is specifically used to: determine the trajectory starting point data according to the atomic skill; perform trajectory planning processing according to the trajectory starting point data to obtain a section of actuator motion trajectory; and combine the actuator motion trajectories corresponding to each atomic skill to obtain an initial trajectory.

[0135] In a possible implementation, the trajectory planning module 43 is specifically used to: perform trajectory planning processing according to trajectory starting point data through a teleoperation method or a trajectory planning algorithm to obtain a section of actuator motion trajectory.

[0136] In a possible implementation, the acquisition module 41 is further specifically used to: determine the multiplexing mode update information of the atomic skill according to the environmental perception information; adjust the calling information of the atomic skill according to the multiplexing mode update information of the atomic skill to obtain the target execution strategy.

[0137] In a possible implementation, the trajectory planning module 43 is specifically used to: determine default environmental information according to the target execution strategy; perform comparison and calculation processing based on the default environmental information and environmental perception information to obtain trajectory correction information; perform adjustment processing based on the trajectory correction information, expected parameters and initial trajectory to obtain the target trajectory.

[0138] In one possible implementation, the expected parameters include an expected trajectory and an expected robot body pose matrix; accordingly, the trajectory planning module 43 is further specifically used to: determine the initial robot body pose matrix based on the trajectory correction information and the initial trajectory; determine the target robot body pose matrix based on the expected robot body pose matrix and the initial robot body pose matrix; perform solution operation processing based on the target robot body pose matrix to obtain the robot terminal pose trajectory; determine the transition trajectory based on the robot terminal pose trajectory and the initial robot body pose matrix; perform point-by-point pose merging processing based on the transition trajectory and the robot terminal pose trajectory to obtain the target trajectory.

[0139] In a possible implementation, the acquisition module 41 is further used to: in the process of controlling each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved, obtain the environment change information in real time. When it is detected that there is an information difference between the environment change information and the environment perception information, return to obtain the environment perception information, and perform skill reuse adjustment processing according to the environment perception information to obtain the step of the target execution strategy.

[0140] The robot control device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0141] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0142] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0143] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0144] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0145] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0146] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0147] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0148] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0149] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0150] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0151] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0154] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0155] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0156] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A robot control method, characterized in that: include: Obtain information about tasks to be performed and task objectives; Perform skill decomposition processing according to the task information to be executed to obtain multiple independent atomic skills, each atomic skill is used to execute a subtask to achieve the expected subtask goal, and the task goal is achieved when all the expected subtask goals are achieved; Performing trajectory planning processing according to the atomic skill to obtain an initial trajectory and acquire expected parameters of the atomic skill; Acquire environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy; Performing trajectory replanning processing on the initial trajectory according to the target execution strategy, the expected parameters and the environmental perception information to obtain a target trajectory; According to the target trajectory, each atomic skill is controlled to execute the corresponding subtask to achieve the expected subtask goal until the task goal is achieved.

2. The method according to claim 1, characterized in that The task information to be performed includes target position, target posture and robot status information; The task is decomposed according to the information of the task to be executed to obtain multiple independent atomic skills, including: Perform task analysis and processing according to the target position, target posture and robot state information to obtain multiple subtask target description information; Decomposing the task according to the subtask target description information to obtain multiple subtasks; According to the subtask, a corresponding function module is selected from pre-stored robot function modules to be determined as an atomic skill, so as to obtain a plurality of independent atomic skills.

3. The method according to claim 2, characterized in that The step of performing trajectory planning processing according to the atomic skill to obtain an initial trajectory includes: Determine trajectory starting point data according to the atomic skill; Perform trajectory planning processing according to the trajectory starting point data to obtain a section of actuator motion trajectory; The actuator motion trajectories corresponding to each atomic skill are combined and processed to obtain the initial trajectory.

4. The method according to claim 3, characterized in that: The process of performing trajectory planning processing according to the trajectory starting point data to obtain a segment of actuator motion trajectory includes: According to the trajectory starting point data, trajectory planning processing is performed through a remote operation method or a trajectory planning algorithm to obtain a section of actuator motion trajectory.

5. The method according to claim 1, characterized in that: The skill reuse adjustment process is performed according to the environmental perception information to obtain a target execution strategy, including: Determine, according to the environmental perception information, update information of the multiplexing mode of the atomic skill; The calling information of the atomic skill is adjusted according to the multiplexing mode update information of the atomic skill to obtain a target execution strategy.

6. The method according to claim 1, characterized in that The performing trajectory replanning processing on the initial trajectory according to the target execution strategy, the expected parameters and the environmental perception information to obtain the target trajectory includes: Determining default environment information according to the target execution strategy; Compare and calculate the default environment information and the environment perception information to obtain trajectory correction information; Adjustment processing is performed according to the trajectory correction information, the expected parameters and the initial trajectory to obtain a target trajectory.

7. The method according to claim 6, characterized in that The expected parameters include an expected robot body pose matrix; Accordingly, the adjusting process is performed according to the trajectory correction information, the expected parameters and the initial trajectory to obtain the target trajectory, including: Determining an initial robot body pose matrix according to the trajectory correction information and the initial trajectory; Determine a target robot body pose matrix according to the expected robot body pose matrix and the initial robot body pose matrix; Performing a solution operation according to the target robot body posture matrix to obtain the robot terminal posture trajectory; Determining a transition trajectory according to the robot end pose trajectory and the initial robot body pose matrix; The transition trajectory and the robot end posture trajectory are subjected to point-by-point posture merging processing to obtain the target trajectory.

8. The method according to any one of claims 1 to 7, characterized in that Also includes: In the process of controlling each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved, real-time acquisition of environmental change information; When it is detected that there is an information difference between the environmental change information and the environmental perception information, the step of obtaining the environmental perception information is returned, and skill reuse adjustment processing is performed according to the environmental perception information to obtain the target execution strategy.

9. A robot control device, characterized in that: include: The acquisition module is used to obtain the task information and task objectives to be executed; A skill decomposition module is used to perform skill decomposition processing according to the task information to be executed, and obtain multiple independent atomic skills. Each atomic skill is used to execute a subtask to achieve the expected subtask goal. When all the expected subtask goals are achieved, the task goal is achieved; A trajectory planning module, used to perform trajectory planning processing according to the atomic skill, obtain an initial trajectory, and record the expected parameters of the atomic skill; The acquisition module is further used to acquire environmental perception information, and perform skill reuse adjustment processing according to the environmental perception information to obtain a target execution strategy; The trajectory planning module is further used to perform trajectory replanning processing on the initial trajectory according to the target execution strategy, the expected parameters and the environmental perception information to obtain a target trajectory; The skill execution module is used to control each atomic skill to execute the corresponding subtask according to the target trajectory to achieve the expected subtask goal until the task goal is achieved.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

Images