Robot motion control method and device, robot, medium and program product

Abstract

The invention relates to a robot motion control method and device, a robot, a medium and a program product, the robot comprises a robot body, robot feet and mechanical arms, and the robot feet and the mechanical arms form motion branch chains with the robot body respectively; the method comprises the steps that second reference data are determined through a first controller according to obtained first reference data and current actual state estimation of the robot, the first reference data are used for representing an estimated expected state of the robot for executing a target task, and the second reference data are used for representing an actual expected state of the robot for executing the target task; according to the second reference data and the current actual state estimation of the robot, a control instruction of the joints of the motion branch chains is determined through a second controller; the robot is controlled to move according to the control instruction, the current situation of leg-arm separation control is broken through, control logic of a double-layer controller is adopted, and therefore the robot has high robustness and environment anti-interference capacity in the process of controlling the robot to execute a target task.

Description

Technical Field

[0001] The present disclosure relates to the field of robots, and in particular to a robot motion control method, device, robot, medium and program product. Background Art

[0002] Leg-arm composite robots can adapt to more complex terrains and working environments, and can replace humans to complete multi-contact and long-sequence operation tasks. Therefore, they are widely used in various scenarios such as home services, industrial inspections, and rescue and firefighting.

[0003] The movement of a leg-arm combined robot often has the characteristics of high nonlinearity, high degree of freedom, and mixed continuous and discrete systems. Therefore, the control of a leg-arm combined robot is relatively difficult. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a robot motion control method, device, robot, medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, a robot motion control method is provided, wherein the robot comprises a robot body, a robot foot and a robot arm, wherein the robot foot and the robot arm respectively form a motion branch chain with the robot body, and the method comprises: Determine, by a first controller, second reference data based on the acquired first reference data and the current actual state of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; Determining, by a second controller, control instructions for the joints of the kinematic branch chain according to the second reference data and the current actual state estimation of the robot; The robot is controlled to move according to the control instruction to perform the target task.

[0006] Optionally, the first reference data includes a first reference trajectory and state information of the end of the motion branch and the target object when performing the target task, and the second reference data is determined by the first controller according to the acquired first reference data and the current actual state estimation of the robot, including: The first controller determines the actual expected state of the robot in multiple time steps in the future based on the acquired first reference data and the current actual state estimation of the robot, and the second reference data includes the actual expected state of the robot in multiple time steps in the future.

[0007] Optionally, the control frequency of the first controller is lower than the control frequency of the second controller.

[0008] Optionally, the kinematic branch includes an end and a joint, the second reference data includes a second reference trajectory, joint reference data of the joint, and end reference data of the end, and determining the control instructions of the joints of the kinematic branch by the second controller according to the second reference data and the current actual state estimation of the robot includes: Determine, by a second controller, a target state of the robot at the next time step based on task priority and constraint priority according to the second reference data and the current actual state estimation of the robot; According to the target state, control instructions for the joints of the motion branch chain are determined.

[0009] Optionally, the task priority is determined in real time during the movement of the robot.

[0010] Optionally, the first controller includes a linear model, and the second controller includes a nonlinear model.

[0011] According to a second aspect of an embodiment of the present disclosure, a robot motion control device is provided, wherein the robot comprises a robot body, a robot foot and a robot arm, wherein the robot foot and the robot arm respectively form a motion branch chain with the robot body, and the device comprises: A first determination module is configured to determine second reference data through a first controller according to the acquired first reference data and the current actual state estimation of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; A second determination module is configured to determine, through a second controller, a control instruction for a joint of the motion branch chain according to the second reference data and a current actual state estimate of the robot; The control module is configured to control the movement of the robot according to the control instruction to perform the target task.

[0012] According to a third aspect of an embodiment of the present disclosure, a robot is provided, the robot comprising a robot body, a robot foot and a robot arm, the robot foot and the robot arm respectively forming a motion branch chain with the robot body, the robot further comprising: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Determine, by a first controller, second reference data based on the acquired first reference data and the current actual state of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; Determining, by a second controller, control instructions for the joints of the kinematic branch chain according to the second reference data and the current actual state estimation of the robot; The robot is controlled to move according to the control instruction to perform the target task.

[0013] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the robot motion control method provided in the first aspect of the present disclosure are implemented.

[0014] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the robot motion control method provided in the first aspect of the present disclosure.

[0015] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the robot foot and the robot arm are equivalent to the same motion branch chain connected to the robot body, thereby breaking the current situation of separate leg and arm control; in addition, the first controller is used to predict the actual expected state of the robot performing the target task based on the estimated expected state of the robot performing the target task and the current actual state of the robot; on this basis, the second controller is used to obtain the control instructions of the joints of the motion branch chain, and the robot movement is controlled according to the control instructions to complete the execution of the target task, that is, the control logic of the double-layer controller is adopted, so that the robot has strong robustness and environmental interference resistance in the process of controlling the robot to perform the target task.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] Figure 1 The figure is a flow chart of a robot motion control method according to an exemplary embodiment.

[0019] Figure 2 The figure is a flow chart of a robot motion control method according to an exemplary embodiment.

[0020] Figure 3 It is a schematic diagram of multiple parameters involved in a robot motion control method at different times according to an exemplary embodiment of the present disclosure.

[0021] Figure 4 The figure is a block diagram of a robot motion control device according to an exemplary embodiment.

[0022] Figure 5 The diagram is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0024] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0025] Figure 1 This is a flow chart of a robot motion control method according to an exemplary embodiment of the present disclosure, which can be applied to electronic devices or robots, and the electronic devices can be implemented in various forms. For example, the electronic devices described in the present disclosure may include mobile phones, tablet computers, laptop computers, desktop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, etc.

[0026] The robot described in the present disclosure includes a leg-arm composite robot, which is a multi-modal mobile manipulation robot that combines leg and arm mechanisms. The legs can also be called robot feet, and further, the leg-arm composite robot can be a multi-legged robot such as a bipedal robot, a quadruped robot, and a hexapod robot, and the arms can also be called mechanical arms, and further, the leg-arm composite robot can be a dual-arm robot or a single-arm robot, etc.

[0027] In the present disclosure, the robot may include a robot body, robot feet and a robot arm, and each robot foot and each robot arm respectively form a motion branch chain with the robot body. Further, in the following embodiments, the present disclosure takes the robot including a robot body, four robot feet and one robot arm as an example to explain and illustrate the present disclosure.

[0028] Reference Figure 1 , the robot motion control method may include the following steps: In step S11, the first controller determines the second reference data according to the acquired first reference data and the current actual state estimation of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; In step S12, the control instructions of the joints of the kinematic branch chain are determined by the second controller according to the second reference data and the current actual state estimation of the robot; In step S13, the robot is controlled to move according to the control instruction to perform the target task.

[0029] Through the above method, the robot foot and the robot arm are equivalent to the same motion branch chain connected to the robot body, thereby breaking the current situation of separate leg and arm control; in addition, the first controller is used to predict the actual expected state of the robot to perform the target task based on the estimated expected state of the robot to perform the target task and the current actual state of the robot; on this basis, the second controller is used to obtain the control instructions of the joints of the motion branch chain, and the robot movement is controlled according to the control instructions to complete the execution of the target task, that is, the control logic of the double-layer controller is adopted, so that the robot has strong robustness and environmental interference resistance in the process of controlling the robot to perform the target task.

[0030] In a possible approach, the target task is a mobile manipulation task with multiple contacts and long sequences, such as an automatic door opening task.

[0031] It is worth noting that each robot foot and each robotic arm respectively forms a motion branch with the robot body. For example, taking the robot including the robot body, four robot feet and one robotic arm as an example, a total of 5 motion branches are included, and the 5 motion branches are respectively formed by each robot foot and the robot body, and the motion branch formed by the robotic arm and the robot body. In the present disclosure, each motion branch may include a joint and an end. Among them, the movement of the joint can cause the state of the robot to change. When the motion branch includes a robot foot, the end may refer to the foot end of the robot foot. When the motion branch includes a robotic arm, the end may refer to the end of the robotic arm.

[0032] It is worth noting that the current actual state estimation of the robot is determined based on the raw information collected by the sensor in real time, such as the raw information collected by the IMU (Inertial Measurement Unit) in real time, and the raw information collected by the electronic joint goniometer in real time. Based on these raw information collected in real time, the current actual state estimation can be determined, and the actual state estimation describes the actual state of the robot. The actual state here includes, for example, the body posture of the robot, the end data of the end of the motion branch chain, and the joint data of the joint of the motion branch chain. For example, the end data includes the speed and angle of the end, and the joint data includes the angle and speed of the joint.

[0033] It is worth noting that the first reference data is used to characterize the estimated expected state of the robot when performing the target task, wherein the first reference data may include the first reference trajectory and the state information of the end of the motion branch and the target object when performing the target task.

[0034] In one embodiment of the present disclosure, the first reference trajectory may be a trajectory obtained by autonomous planning of the robot, which may include a body trajectory and a trajectory of the end, and further, the trajectory of the end may include a trajectory of the robot foot and a trajectory of the robot arm end. The determination of the first reference trajectory may refer to related technologies. In another embodiment of the present disclosure, the first reference trajectory may also be obtained by input from a robot operator.

[0035] The state information of the end of the motion branch and the target object is used to characterize whether the end of the motion branch is in contact with the target object. In the present disclosure, the target object is all the targets that the robot can contact in the process of completing the target task. For example, taking the target task of automatic door opening as an example, the door handle, the door panel surface and the ground are all considered as targets that the robot can contact in the process of completing the automatic door opening.

[0036] Among them, the second reference data is used to characterize the actual expected state of the robot performing the target task. The actual expected data is obtained by optimizing the estimated expected state in combination with the current actual state estimation of the robot. The specific optimization can refer to the following related embodiments, and this embodiment is not described in detail here. The second reference data may include body trajectory, joint data, and terminal data, etc. Further, the second reference data may include body trajectory, joint data, and terminal data that are respectively optimized for the body trajectory, joint data, and terminal data in the first reference data. Further, the body trajectory includes body momentum and body posture, key data may include joint angles, and terminal data may include displacement of the end of the robotic arm and speed of the end of the robotic arm, etc.

[0037] Figure 2is another flow chart of a robot motion control method according to an exemplary embodiment of the present disclosure. Figure 2 The present disclosure is further explained and illustrated.

[0038] In a possible manner, the step of determining the second reference data by the first controller based on the acquired first reference data and the current actual state estimation of the robot can be implemented in the following manner: the first controller determines the actual expected state of the robot in multiple time steps in the future based on the acquired first reference data and the current actual state estimation of the robot, and the second reference data includes the actual expected state of the robot in multiple time steps in the future.

[0039] As can be seen from the above content, the first reference data may include the first reference trajectory and state information. The second reference data may include the body trajectory, joint data and joint data, etc. The robot may determine the control instruction according to the actual expected state represented by the second reference data, so that the robot reaches the corresponding actual expected state after executing the control instruction.

[0040] It is worth noting that, in solving the second reference data, it is necessary to construct the optimization problem of the first controller and the corresponding preset constraints, which may include state space equations, equality constraints, state constraints, state initial constraints, joint angle limit constraints, joint torque limit constraints, and center of mass dynamics constraints. As an example, the optimization problem and preset constraints constructed by the first controller can be represented by the following formula: ; The first equation above is an optimization problem, and the others are some preset constraints. is the state space equation, is an equality constraint, is the state constraint, is the initial state constraint.

[0041] Further, is the final performance, is the dynamic performance, T is the preset duration, represents the state variable at time t, represents the control variable at time t. Among them, , is the robot's center of mass (body) momentum, is the center of mass pose, is the angle of each joint, is the displacement of the end, is the speed at the end, , In represents the contact force and contact torque at the corresponding end, represents the velocity of the joint, i represents the number of end points, and j represents the number of joints.

[0042] It is worth noting that when the state information represents that the terminal is in contact with the target object, a terminal contact force can be generated, and whether a contact torque is generated is related to the contact type between the defined terminal and the target object. The contact type of the terminal of the motion branch chain can include a point contact terminal that cannot be grasped and a surface contact terminal that can be grasped. The control variables corresponding to the point contact terminal that cannot be grasped include contact force, and the control variables corresponding to the surface contact terminal that can be grasped include contact force and contact torque. The contact type of the target object includes a graspable point contact target and an unclawed surface contact target. The control variables corresponding to the graspable point contact target include contact force, and the control variables corresponding to the unclawed surface contact target include contact force and contact torque. In addition, during implementation, the output of the contact torque can be 0 when no contact torque is generated.

[0043] It is worth noting that the joint angle limit constraint can be constructed by the equation that the joint angle should not exceed the maximum angle; the joint torque limit constraint can be constructed by the equation that the joint torque should not exceed the maximum torque; the construction of the center of mass dynamics constraint can refer to relevant technologies, which will not be elaborated in this embodiment.

[0044] In one embodiment of the present disclosure, the process of determining the second reference data of the present disclosure is exemplarily described by taking the first reference data as the trajectory of the fuselage as an example. Figure 3 is a schematic diagram of multiple parameters involved in a robot motion control method at different times according to an exemplary embodiment of the present disclosure. First of all, it is worth noting that Figure 3 The reference trajectory in refers to the first reference trajectory, such as the fuselage trajectory, and the reference trajectory corresponding to the first reference trajectory. Figure 3 The predicted output in may refer to the fuselage trajectory in the second reference data, Figure 3 The measurement output in is the actual state estimate of the robot, Figure 3 The predictive control quantity in can refer to the control instructions of the joints of the motion branch chain. Figure 3 The executed control quantity in refers to the control instructions of the joints of the executed motion chain. Figure 3 , the sampling time is a time step of the first controller, and the time step of the first controller is determined based on the control frequency of the controller. At the kth time, the actual expected state of the robot in the future multiple time steps is predicted based on the actual state estimate of the robot at the kth time and the trajectory of multiple time steps after the kth time in the reference trajectory. As an example, multiple time steps such as Figure 3The time step corresponding to the kth moment to the k+1th moment, the time step corresponding to the k+1th moment to the k+2th moment, the time step corresponding to the k+2th moment to the k+3th moment, and the time step corresponding to the k+Nth moment to the k+N+1th moment shown in , where N can be set according to actual conditions, and this embodiment will not be described in detail here.

[0045] In the above manner, the first controller predicts the state of the robot in multiple time steps in the future, providing a longer-term perspective, helping to avoid falling into a local optimal solution, and can quickly recover to the actual desired state when facing disturbances, which is conducive to more comprehensive planning and decision-making of the control system composed of the first controller, the second controller and the third controller.

[0046] In a possible manner, the step of determining the control instructions for the joints of the motion branch through the second controller according to the second reference data and the current actual state estimation of the robot may include: determining the target state of the robot in the next time step based on the task priority and the constraint priority through the second controller according to the second reference data and the current actual state estimation of the robot; and determining the control instructions for the joints of the motion branch of the motion branch according to the target state.

[0047] It is worth noting that the control frequency of the first controller is different from the control frequency of the second controller, and therefore, the control frequency of the first controller and the time step corresponding to the second controller are different. In a possible way, since the first controller determines the second reference data for the second controller in a manner taking into account multiple time steps in the future, it may take a long time, and the second controller needs to output a control instruction to control the robot motion, and has a high requirement for real-time performance. Therefore, the control frequency of the first controller can be set lower than the control frequency of the second controller to meet the real-time performance of the robot motion control. It should be worth noting that the control frequency is the frequency at which the controller performs an operation, and this frequency can determine the above-mentioned sampling time. As an example, the control frequency of the first controller can be 50 Hz -100 Hz, that is, the first controller performs the output of the second reference data once at a frequency of 50 Hz -100 Hz, and the control frequency of the second controller can be 500 Hz -1000 Hz, that is, the second controller performs the output of the control instruction once at a frequency of 50 0 Hz -1000 Hz.

[0048] The second reference data includes a second reference trajectory, joint reference data of the joint, and terminal reference data of the terminal. The second reference trajectory may include a body trajectory and a terminal trajectory, the joint reference data may include a joint angle and a joint velocity, etc., and the terminal reference data may include a terminal contact force, a terminal contact torque, and a terminal velocity, etc.

[0049] Among them, the target state can be the robot's center of mass (body) momentum, center of mass posture, joint angles, end displacement, etc.

[0050] The control instructions of the joints may include torque instructions, speed instructions and angle instructions of the joints. After the robot executes the control instructions, the robot can reach the desired target state. Figure 3 After the robot continuously executes the predicted control amount, the robot's own state is more described by the reference trajectory, that is, the robot's own state is closer to the state described by the first reference trajectory, so that the robot finally completes the execution of the target task.

[0051] In one embodiment of the present disclosure, in the process of the robot achieving the entire target task, it can be divided into multiple control cycles. Each control cycle involves multiple tasks and multiple constraints, such as tracking tasks related to the foot end, tracking tasks related to the robotic arm end, etc., such as friction constraints and slip constraints, etc. Therefore, in different control cycles, optimization problems involving multiple tasks and multiple constraints are involved. Different tasks and constraints can have different priorities in different control cycles. For example, in a control cycle where the robot needs to walk and does not need the robotic arm end to grasp any target, more attention should be paid to the tracking tasks related to the foot end. Therefore, the priority of the tracking tasks related to the foot end needs to be higher than the tracking tasks related to the robotic arm end.

[0052] In this way, since robot control requires real-time performance, the second controller that outputs control instructions only focuses on the situation at the next time point, without considering the more distant future, that is, predicting the target state of the robot at the next time step, and determining the control instructions of the joints of the motion branch based on the target state to adapt to the real-time performance required by robot control. In addition, by considering the priorities of different tasks and constraints, the robot can prioritize solving the optimization problems composed of high-level tasks and constraints, and on the basis of the obtained optimization results, stack the optimization results of subsequent levels layer by layer.

[0053] In a possible way, it can be seen from the above content that the first controller focuses on multiple time steps and the second controller focuses on a single time step. Therefore, the first controller can be set as a linear model and the second controller can be set as a nonlinear model. The linear model has low control difficulty and high calculation efficiency, and the nonlinear model can handle more complex linear relationships and has higher calculation accuracy. Therefore, the different advantages of linear models and nonlinear models are utilized, combined with the characteristics that the first controller and the second controller focus on different numbers of time steps, and the linear model and the nonlinear model are used in combination, thereby reducing the calculation amount and difficulty of the first controller and improving the accuracy of the second controller.

[0054] In a possible manner, during the movement of the robot, as the robot state changes, the task priority can be determined in real time, thereby adapting to the change of the robot state.

[0055] In a possible manner, when the second controller solves the control instructions of the joint, the tasks of executing each control cycle involve different constraints such as friction cone constraints, joint torque limit constraints, multi-rigid body dynamics models of floating bases, and non-slip constraints, etc. The equations of the constraints can be constructed with reference to related technologies. As an example, the priority of the constraints can be pre-configured to reduce the amount of data processed in real time, providing a basis for the real-time performance of the robot motion control.

[0056] In the present disclosure, taking the robot performing an end operation task, such as a grasping task, in a certain controller cycle as an example, the first priority tasks and constraints may include the position tracking of the end of the manipulator, the contact force tracking of the end of the manipulator, the multi-rigid body dynamics model of the floating base, the joint torque limit constraint, the quadruped friction cone constraint when the robot foot is in contact with the ground, and the non-slip constraint. The second priority tasks may include the body acceleration tracking, the quadruped swing trajectory tracking when the robot foot is not in contact with the ground, and the trajectory tracking of the joints of the motion branch where the manipulator is located. The third priority tasks include the quadruped contact force tracking when the robot foot is in contact with the ground.

[0057] Continue to refer to Figure 2 , the above-mentioned step of controlling the robot movement according to the control instruction may include: processing the control instruction through a third controller to obtain a joint torque instruction, and controlling the joint movement of the robot according to the joint torque instruction. Among them, the third controller is a bottom-level joint controller, which is an important component in the robot control system, and it is responsible for controlling the movement of each joint of the robot. Specifically, the third controller can use a proportional-integral-differential control method to realize the robot's motion control. The third controller processes the control instruction to obtain a joint torque instruction, and inputs the joint torque instruction to the motor of the corresponding joint, so that each motor outputs the corresponding joint torque at the corresponding time, realizes the robot motion control, and then enables the robot to complete the target task.

[0058] Figure 4 is a block diagram of a robot motion control device according to an exemplary embodiment. Figure 4 The robot comprises a robot body, a robot foot and a robot arm, wherein the robot foot and the robot arm respectively form a motion branch chain with the robot body, and the device 400 comprises: The first determination module 401 is configured to determine second reference data through a first controller according to the acquired first reference data and the current actual state estimation of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; A second determination module 402 is configured to determine, through a second controller, a control instruction for a joint of the motion branch chain according to the second reference data and a current actual state estimate of the robot; The control module 403 is configured to control the movement of the robot according to the control instruction to perform the target task.

[0059] Optionally, the first reference data includes a first reference trajectory and state information of an end of the motion branch and a target object when performing the target task, and the first determination module 301 is further configured to: The first controller determines the actual expected state of the robot in multiple time steps in the future based on the acquired first reference data and the current actual state estimation of the robot, and the second reference data includes the actual expected state of the robot in multiple time steps in the future.

[0060] Optionally, the control frequency of the first controller is lower than the control frequency of the second controller.

[0061] Optionally, the kinematic branch includes an end and a joint, the second reference data includes a second reference trajectory, joint reference data of the joint and end reference data of the end, and the second determination module 402 is further configured to: Determine, by a second controller, a target state of the robot at the next time step based on task priority and constraint priority according to the second reference data and the current actual state estimation of the robot; According to the target state, control instructions for the joints of the motion branch chain are determined.

[0062] Optionally, the task priority is determined in real time during the movement of the robot.

[0063] Optionally, the first controller includes a linear model, and the second controller includes a nonlinear model.

[0064] Regarding the device 400 in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the robot motion control method, and will not be elaborated here.

[0065] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the robot motion control method provided by the present disclosure are implemented.

[0066] Figure 5 is a block diagram of a robot according to an exemplary embodiment. Figure 5 The robot 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output interface 512 , a sensor component 514 , and a communication component 516 .

[0067] The processing component 502 generally controls the overall operation of the robot 500, such as operations associated with display, data communication, camera operation, and recording operation. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned robot motion control method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0068] The memory 504 is configured to store various types of data to support operations on the robot 500. Examples of such data include instructions for any application or method operating on the robot 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0069] The power supply assembly 506 provides power to various components of the robot 500. The power supply assembly 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the robot 500.

[0070] The multimedia component 508 includes a screen that provides an output interface between the robot 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the robot 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0071] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the robot 500 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0072] The input / output interface 512 provides an interface between the processing component 502 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0073] The sensor assembly 514 includes one or more sensors for providing various aspects of the status assessment of the robot 500. For example, the sensor assembly 514 can detect the open / closed state of the robot 500, the relative positioning of components, such as the display and keypad of the robot 500, the sensor assembly 514 can also detect the position change of the robot 500 or a component of the robot 500, the presence or absence of user contact with the robot 500, the orientation or acceleration / deceleration of the robot 500, and the temperature change of the robot 500. The sensor assembly 514 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0074] The communication component 516 is configured to facilitate wired or wireless communication between the robot 500 and other devices. The robot 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0075] In an exemplary embodiment, the robot 500 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described robot motion control method.

[0076] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the above instructions can be executed by a processor 520 for robot trajectory determination to complete the above robot motion control method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0077] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned robot motion control method when executed by the programmable device.

[0078] Those skilled in the art may also understand that the various illustrative logical blocks and steps listed in the embodiments of the present application may be implemented by electronic hardware, computer software, or a combination of the two. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the functions described for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0079] In the above detailed description, reference is made to the accompanying drawings, which illustrate specific aspects of the present disclosure that can be practiced by way of illustration. In this regard, terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating directions or representing positional relationships, can be used with reference to the orientation of the described figures. Since the components of the described device can be positioned in a plurality of different orientations, directional terms can be used for illustrative purposes rather than restrictive. It should be understood that other aspects can be utilized and structural or logical changes can be made without departing from the concepts of the present disclosure. Therefore, the following detailed description should not be considered in a limiting sense.

[0080] It should be understood that, unless otherwise specifically noted, the features of some embodiments of the various present disclosures described herein may be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of any two or more; similarly, "at least one of . . . " includes any one of the related listed items and any combination of any two or more.

[0081] It should be understood that, unless otherwise clearly specified and limited, the terms "joining", "attaching", "installing", "connecting", "connecting", "fixing" and the like used in the embodiments of the present disclosure should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly limited. For ordinary technicians in this field, the specific meanings of the above terms in this article can be understood according to specific circumstances.

[0082] In addition, the term "above" used in relation to a component, element or material layer formed "above" or located "above" a surface may be used herein to indicate that the component, element or material layer is "indirectly" positioned (e.g., placed, formed, deposited, etc.) on the surface such that one or more additional components, elements or layers are arranged between the surface and the component, element or material layer. However, the term "above" used in relation to a component, element or material layer formed "above" or located "above" a surface may also optionally have a specific meaning: the component, element or material layer is "directly" positioned (e.g., placed, formed, deposited, etc.) on the surface, such as in direct contact with the surface.

[0083] Although terms such as "first", "second" and "third" can be used herein to describe various components, parts, regions, layers or sections, these components, parts, regions, layers or sections are not limited to these terms. On the contrary, these terms are only used to distinguish one component, component, region, layer or section from another component, component, region, layer or section. Therefore, without departing from the teachings of each example, the first component, component, region, layer or section mentioned in the examples described herein may also be referred to as the second component, component, region, layer or section. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may expressly or implicitly include at least one of the features. In the description herein, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0084] It should be understood that spatially relative terms such as "above", "upper", "below", and "lower" are used herein to describe the relationship of one element to another element shown in the figures. In addition to the orientation depicted in the drawings, such spatially relative terms are also intended to include different orientations of the device in use or operation. For example, if the device in the drawings is turned over, the element described as being "above" or "upper" relative to another element will be "below" or "lower" relative to the other element. Therefore, depending on the spatial orientation of the device, the term "above" includes both upper and lower orientations. The device may have other orientations (e.g., rotated 90 degrees or in other orientations), and the spatially relative terms used herein should be interpreted accordingly.

[0085] In addition, the word "exemplary" is used herein to indicate serving as an example, instance, or diagram. Any aspect or design described as "exemplary" in this article is not necessarily understood to be advantageous compared to other aspects or designs. On the contrary, the use of the word exemplary is intended to present concepts in a specific way. As used herein, the term "or" is intended to represent an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X applies A or B" is intended to represent any one of the natural inclusive arrangements. That is, if X applies A; X applies B; or X applies both A and B, "X applies A or B" is satisfied under any of the aforementioned examples. In addition, unless otherwise specified or clearly pointed to a singular form from the context, the articles "one" and "an" as used in this application and the appended claims are generally understood to mean "one or more".

[0086] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art after reading and understanding the specification and drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components (e.g., elements, resources, etc.) described above, unless otherwise indicated, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific functions of the described components, even if the structure is not equivalent to the disclosed structure. In addition, although specific features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and beneficial to any given or specific application. In addition, with respect to "including", "having", "having", "having", or variations thereof used in a specific embodiment or claim, such terms are intended to be inclusive in a manner similar to the term "comprising".

[0087] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0088] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A robot motion control method, characterized in that: The robot comprises a robot body, a robot foot and a robot arm, wherein the robot foot and the robot arm respectively form a motion branch chain with the robot body, and the method comprises: Determine, by a first controller, second reference data based on the acquired first reference data and the current actual state of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; Determining, by a second controller, control instructions for the joints of the motion branch chain according to the second reference data and the current actual state estimation of the robot; The robot is controlled to move according to the control instruction to perform the target task.

2. The method according to claim 1, characterized in that The first reference data includes a first reference trajectory and state information of the end of the motion branch and the target object when performing the target task, and the second reference data is determined by the first controller according to the acquired first reference data and the current actual state estimation of the robot, including: The first controller determines the actual expected state of the robot in multiple time steps in the future based on the acquired first reference data and the current actual state estimation of the robot, and the second reference data includes the actual expected state of the robot in multiple time steps in the future.

3. The method according to claim 2, characterized in that The control frequency of the first controller is lower than the control frequency of the second controller.

4. The method according to claim 2, characterized in that: The kinematic branch chain includes an end and a joint, the second reference data includes a second reference trajectory, joint reference data of the joint, and end reference data of the end, and the second controller determines the control instructions of the joint of the kinematic branch chain according to the second reference data and the current actual state estimation of the robot, including: Determine, by a second controller, a target state of the robot at the next time step based on task priority and constraint priority according to the second reference data and the current actual state estimation of the robot; According to the target state, control instructions for the joints of the motion branch chain are determined.

5. The method according to claim 4, characterized in that The task priorities are determined in real time during the movement of the robot.

6. The method according to any one of claims 1 to 5, characterized in that: The first controller includes a linear model and the second controller includes a nonlinear model.

7. A robot motion control device, characterized in that: The robot comprises a robot body, a robot foot and a robot arm, wherein the robot foot and the robot arm respectively form a motion branch chain with the robot body, and the device comprises: A first determination module is configured to determine second reference data through a first controller according to the acquired first reference data and the current actual state estimation of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; A second determination module is configured to determine, through a second controller, a control instruction for a joint of the motion branch chain according to the second reference data and a current actual state estimate of the robot; The control module is configured to control the movement of the robot according to the control instruction to perform the target task.

8. A robot, characterized in that: The robot comprises a robot body, a robot foot and a robot arm, wherein the robot foot and the robot arm respectively form a motion branch chain with the robot body, and the robot further comprises: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Determine, by a first controller, second reference data based on the acquired first reference data and the current actual state of the robot, wherein the first reference data is used to characterize the estimated expected state of the robot performing the target task, and the second reference data is used to characterize the actual expected state of the robot performing the target task; Determining, by a second controller, control instructions for the joints of the motion branch chain according to the second reference data and the current actual state estimation of the robot; The robot is controlled to move according to the control instruction to perform the target task.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images