Robotic arm control method and device, computing device and storage medium

Abstract

Disclosed are a method and apparatus for controlling a robotic arm, a computing device, a computer-readable storage medium, and a computer program product. The method includes: obtaining robotic arm state information; determining, based on the robotic arm state information, a motion relationship between a joint space and an operation space of the robotic arm, wherein the joint space includes multiple joints and the operation space includes an end effector with multiple degrees of freedom; determining a robotic arm control strategy based on the motion relationship and a preset target task, wherein the robotic arm control strategy includes an acceleration layer-based robotic arm joint space control strategy; and controlling the robotic arm using the robotic arm control strategy.

Description

Technical Field

[0001] The present disclosure relates to the field of robotic arms, and more particularly to a robotic arm control method and apparatus, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In recent years, with the development of artificial intelligence and mechanical control technologies, robotic arms have played an increasingly important role in fields such as agricultural production and industrial manufacturing. However, most current robotic arm control methods are only designed for robotic arms with known structural information and are not suitable for robotic arms with unknown models (i.e., unknown structural parameters) or inaccurate models (i.e., inaccurate structural parameters). Furthermore, long-term industrial production processes can affect the initial model parameters of the robotic arm, causing, for example, offset, wear, and singularity. Consequently, model-based control methods cannot effectively respond to changes in the model, resulting in mission failure or damage to the robotic arm. Summary of the Invention

[0003] The present disclosure provides a robotic arm control method and apparatus, a computing device, a computer-readable storage medium, and a computer program product, thereby alleviating, mitigating, or even eliminating some or all of the above-mentioned problems and other possible problems.

[0004] According to one aspect of the present disclosure, a robotic arm control method is proposed, including: obtaining robotic arm state information; determining a motion relationship between a joint space and an operation space of the robotic arm based on the robotic arm state information, the joint space including multiple joints, and the operation space including an end effector with multiple degrees of freedom; determining a robotic arm control strategy based on the motion relationship and a preset target task, the robotic arm control strategy including an acceleration layer-based robotic arm joint space control strategy; and controlling the robotic arm using the robotic arm control strategy.

[0005] In some embodiments, obtaining the state information of the robotic arm includes: obtaining the joint angular velocity and joint angular acceleration of each joint in the joint space of the robotic arm; and obtaining the end effector velocity and end effector acceleration in the operating space of the robotic arm, the end effector velocity including the velocity component in the direction of each degree of freedom in the multiple degrees of freedom, and the end effector acceleration including the acceleration component in the direction of each degree of freedom in the multiple degrees of freedom.

[0006] In some embodiments, determining the motion relationship between the joint space and the operation space of the robotic arm based on the robotic arm state information includes: determining an estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information; and determining the motion relationship between the joint space and the operation space of the robotic arm based on the estimated Jacobian matrix.

[0007] In some embodiments, determining the estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information includes: using the gradient descent method to determine the first quantitative relationship between the joint angular acceleration of the robotic arm, the joint angular velocity of the robotic arm, the end effector velocity and the estimated Jacobian matrix; determining the Jacobian matrix estimation formula based on at least the first quantitative relationship; and determining the estimated Jacobian matrix according to the Jacobian matrix estimation formula.

[0008] In some embodiments, the Jacobian matrix estimation formula is determined at least based on the first quantitative relationship, including: based on the time lag error of the joint angular acceleration, correcting the first quantitative relationship to obtain a second quantitative relationship between the joint angular acceleration of the robotic arm, the joint angular velocity of the robotic arm, the end effector velocity and the estimated Jacobian matrix; and determining the Jacobian matrix estimation formula according to the second quantitative relationship.

[0009] In some embodiments, the robotic arm control strategy is determined based on the motion relationship and the preset target task, and the robotic arm control strategy includes a robotic arm joint space control strategy based on the acceleration layer, including: constructing a performance optimization target based on the robotic arm joint space acceleration layer; determining the control constraints of the robotic arm based on the motion relationship and the preset target task; constructing a control optimization scheme for the robotic arm based on the performance optimization target and the control constraints; and determining the robotic arm control strategy based on the control optimization scheme.

[0010] In some embodiments, constructing a performance optimization objective based on the robotic arm joint space acceleration layer includes: constructing an objective function with the target joint angular acceleration of the robotic arm as an independent variable, wherein the objective function involves the norm of the target joint angular acceleration; and constructing the performance optimization objective to minimize the objective function.

[0011] In some embodiments, determining the control constraints of the robotic arm based on the motion relationship and the preset target task includes: determining a first control constraint regarding the range of target joint angular acceleration and target joint angular velocity based on the preset target task; and determining a second control constraint based on the Jacobi equation based on the preset target task and the motion relationship.

[0012] In some embodiments, determining the second control constraint based on the Jacobi equation according to the preset target task and the motion relationship includes: obtaining state information of the operating space of the robotic arm; determining the error compensation of the operating space based on the target task and the state information of the operating space of the robotic arm; and determining the second control constraint based on the error compensation and the motion relationship.

[0013] In some embodiments, determining the error compensation of the operating space based on the target task and the state information of the operating space of the robotic arm includes: determining the target speed and target position of the end effector according to the target task; determining the current speed and current position of the end effector according to the state information of the operating space of the robotic arm; determining the speed compensation of the end effector based on the target speed and current speed of the end effector; determining the position compensation of the end effector based on the target position and current position of the end effector; and determining the error compensation of the operating space based on at least one of the speed compensation and position compensation of the end effector.

[0014] In some embodiments, constructing a control optimization scheme for the robotic arm according to the performance optimization objective and the control constraint condition includes constructing the robotic arm control optimization scheme as the following quadratic programming problem:

[0015]

[0016] Among them, formula (1) is the performance optimization goal, formulas (2)-(4) are control constraints, among which, is the desired end-effector position, is the desired end-effector velocity, is the desired end-effector acceleration, is the end effector position, is the end effector speed, is the position error compensation, is the compensation coefficient, is the speed error compensation, is the compensation coefficient, To estimate the Jacobian matrix, for The time derivative of is the joint angular velocity, is the joint angular acceleration, with the superscript represents the transpose operation, and are the lower and upper constraints of the joint angular velocity, respectively, and and are the lower and upper constraints of the joint angular acceleration respectively.

[0017] In some embodiments, determining the robotic arm control strategy based on the control optimization scheme includes: solving the control optimization scheme through a quadratic programming solver to obtain the target joint angular acceleration of the robotic arm; and determining the robotic arm control strategy based on the target joint angular acceleration.

[0018] In some embodiments, controlling the robotic arm using the robotic arm control strategy includes: determining a control signal according to the robotic arm control strategy; and controlling the robotic arm to perform a target task based on the control signal.

[0019] According to another aspect of the present disclosure, a robotic arm control device is proposed, including: an information acquisition module, configured to acquire robotic arm state information; a motion relationship determination module, configured to determine the motion relationship between the joint space and the operation space of the robotic arm based on the robotic arm state information, the joint space including multiple joints, and the operation space including an end effector with multiple degrees of freedom; a control strategy determination module, configured to determine the robotic arm control strategy based on the motion relationship and a preset target task, the robotic arm control strategy including a robotic arm joint space control strategy based on an acceleration layer; and a control module, configured to control the robotic arm using the robotic arm control strategy.

[0020] According to another aspect of the present disclosure, a computing device is proposed, comprising: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to execute a robotic arm control method according to some embodiments of the present disclosure.

[0021] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed, the robot arm control method according to some embodiments of the present disclosure is implemented.

[0022] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the robot arm control method according to some embodiments of the present disclosure.

[0023] In a robotic arm control method according to some embodiments of the present disclosure, first, robotic arm state information is acquired; then, based on the robotic arm state information, the kinematic relationship between the joint space and the operating space of the robotic arm is determined; then, based on the kinematic relationship and a preset target task, a robotic arm control strategy is determined; and finally, the robotic arm is controlled using the robotic arm control strategy. In this way, by acquiring robotic arm state information in real time and, based on the inverse kinematics process of the robotic arm, utilizing the kinematic relationship between the joint space and the operating space determined, the structural information of a robotic arm with an unknown or inaccurate model can be solved, and the variable parameters in the robotic arm control algorithm can be corrected in real time, thereby achieving control of the robotic arm. This provides important technical support for controlling robotic arms with unknown or inaccurate models. At the same time, utilizing the determined robotic arm control strategy, the robotic arm control method according to the present disclosure can effectively control robotic arms with unknown or inaccurate models to efficiently complete target tasks and effectively eliminate joint angle deviations and position errors generated during task execution. This is of great significance for solving the problem of robotic arm model changes caused by offset, wear, singularity, etc. in industrial production. In addition, the robotic arm control method according to the present disclosure also has the advantages of strong real-time performance, fast convergence speed, high calculation accuracy, and good robustness, which provides important technical advantages for the robotic arm in actual industrial production and applications.

[0024] These and other aspects of the application will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The various aspects, features and advantages of the present disclosure will be readily understood from the following detailed description and accompanying drawings, in which:

[0026] Figure 1 Schematically illustrates an example implementation environment of a robot arm control method according to some embodiments of the present disclosure;

[0027] Figure 2 Schematically shows a structural diagram of a robotic arm according to some embodiments of the present disclosure;

[0028] Figure 3 The following schematically shows a flow chart of a method for controlling a robotic arm according to some embodiments of the present disclosure;

[0029] Figure 4 Schematically illustrates a flowchart of the first sub-step of the robot arm control method according to some embodiments of the present disclosure;

[0030] Figure 5 Schematically illustrates a flow chart of the second sub-step of the robot arm control method according to some embodiments of the present disclosure;

[0031] Figure 6 Schematically illustrates a flow chart of the third sub-step of the robot arm control method according to some embodiments of the present disclosure;

[0032] Figure 7 Schematically illustrates a flowchart of the fourth sub-step of the robot arm control method according to some embodiments of the present disclosure;

[0033] Figures 8A-8E Schematically shows a control effect diagram of a robotic arm according to some embodiments of the present disclosure;

[0034] Figure 9 An example block diagram schematically illustrates a robotic arm control device according to some embodiments of the present disclosure; and

[0035] Figure 10 An example block diagram of a computing device according to some embodiments of the present disclosure is schematically shown.

[0036] It should be noted that the above drawings are merely schematic and illustrative and are not necessarily drawn to scale. DETAILED DESCRIPTION

[0037] Several embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings so that those skilled in the art can implement the present disclosure. The present disclosure can be embodied in many different forms and for many different purposes and should not be limited to the embodiments described herein. These embodiments are provided to make the present disclosure comprehensive and complete and to fully convey the scope of the present disclosure to those skilled in the art. The embodiments do not limit the present disclosure.

[0038] It will be understood that although the terms first, second, third, etc. may be used to describe various elements, components, and / or parts in this article, these elements, components, and / or parts should not be limited by these terms. These terms are only used to distinguish one element, component, or part from another element, component, or part. Therefore, the first element, component, or part discussed below may be referred to as the second element, component, or part without departing from the teachings of the present disclosure.

[0039] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to also include the plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "include" and / or "comprise" when used in this specification specify the presence of the features, wholes, steps, operations, elements and / or parts, but do not exclude the presence of one or more other features, wholes, steps, operations, elements, parts and / or their groups or add one or more other features, wholes, steps, operations, elements, parts and / or their groups. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0040] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the relevant art and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless expressly defined as such herein.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all promotional information and operations / steps, nor do they necessarily require execution in the order described. For example, some operations / steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0043] Those skilled in the art will understand that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present application, and therefore cannot be used to limit the scope of protection of the present application.

[0044] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, smart transportation, and automated control.

[0045] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, learning through demonstration, and active learning.

[0046] With the advancement of AI research and technology, AI is being studied and applied in a wide range of fields, including smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless and autonomous driving, drones, robots, smart healthcare, smart customer service, connected vehicles, autonomous driving, and smart transportation. We believe that as technology develops, AI will be applied in even more fields and play an increasingly important role.

[0047] Before introducing the embodiments of the present disclosure in detail, some related concepts are first explained for the sake of clarity.

[0048] 1. Robotic arm: refers to a complex system with high precision, multiple inputs and outputs, high nonlinearity, and strong coupling.

[0049] 2. Redundant Manipulator: This refers to a manipulator with more degrees of freedom than the minimum required for the task space. The redundant degrees of freedom of the redundant manipulator can be utilized to achieve additional tasks such as obstacle avoidance, singularity avoidance, joint limit avoidance, joint torque optimization, and increased maneuverability. Furthermore, since a human arm also has seven degrees of freedom, redundant manipulators are more practical from a bionics perspective.

[0050] 3. End effector: Any tool attached to the edge (joint) of a robot that performs a function. This may include robotic grippers, robotic tool changers, robotic collision sensors, robotic rotary joints, robotic pressure tools, compliance devices, robotic spray guns, robotic deburring tools, robotic arc welding guns, robotic electric welding guns, and so on. End effectors are often considered to be peripheral devices, robotic accessories, robotic tools, or end-of-arm tools.

[0051] 4. Inverse kinematics of the robotic arm: This refers to solving the corresponding robotic arm joint angle / angular velocity / angular acceleration information using the given end effector position / velocity / acceleration information.

[0052] 5. Jacobian Matrix: Generally, in vector calculus, the Jacobian matrix is ​​a matrix of first-order partial derivatives arranged in a specific pattern. Specifically, in the field of robotic arms, the Jacobian matrix represents the mapping relationship between the joint space and the manipulation space of the robotic arm (or, in other words, the relationship between the differential motion of the joints and the end effector). It can be thought of as the transmission ratio of motion velocity from the joint space to the manipulation space, and can also be used to represent the force transmission relationship between the joint space and the manipulation space.

[0053] 6. Quadratic Programming: In operations research, quadratic programming is a special type of optimization problem. Quadratic programming is the process of solving a special type of mathematical optimization problem—specifically, a (linearly constrained) quadratic optimization problem, which involves optimizing (minimizing or maximizing) a quadratic function of multiple variables, subject to linear constraints on those variables. Quadratic programming is a special type of nonlinear programming.

[0054] Figure 1 Schematically shows an example implementation environment 100 of a robot arm control method according to some embodiments of the present disclosure. Figure 1 As shown in , the implementation environment 100 may include a terminal device 110, and optionally may also include one or more robotic arms 120, and a network 130 for connecting the terminal device 110 and the robotic arms 120. In some embodiments, the terminal device 110 may be used to implement the robotic arm control method according to the present disclosure. For example, the terminal device 110 may be deployed with corresponding programs or instructions for executing the various methods provided by the present disclosure. Optionally, the robotic arms 120 and the terminal device 110 may also cooperate with each other to implement the various methods according to the present disclosure.

[0055] The terminal device 110 can be any type of mobile computing device, including a mobile computer (e.g., a personal digital assistant (PDA), laptop computer, notebook computer, tablet computer, netbook computer, etc.), a mobile phone (e.g., a cellular phone, smartphone, etc.), a wearable computing device (e.g., a smartwatch, a head-mounted device, including smart glasses, etc.), or other mobile devices. In some embodiments, the terminal device 110 can also be a stationary computing device, such as a desktop computer, a game console, or a smart TV.

[0056] The robotic arm 120 can be a single robotic arm or a robotic arm cluster. In addition, the robotic arm 120 can be a robotic arm with any degree of freedom. It should be understood that the robotic arm mentioned herein is typically a redundant robotic arm with seven degrees of freedom, but other embodiments are also possible.

[0057] Examples of the network 130 include a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and / or a combination of communication networks such as the Internet. The robotic arm 120 and the terminal device 110 may include at least one communication interface (not shown) capable of communicating via the network 130. Such a communication interface may be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless interface (such as an IEEE 802.11 wireless LAN (WLAN)), a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth™ interface, a Near Field Communication (NFC) interface, and the like.

[0058] like Figure 1 As shown in , the terminal device 110 may include a display screen 111 and a terminal application 112 that can interact with the terminal user via the display screen 111. The terminal device 110 can interact with the robotic arm 120, for example, via the network 130, such as sending data to or receiving data from it. The terminal application 112 can be a local application, a web application, or a lightweight application (e.g., a mobile app or WeChat app). If the terminal application 112 is a local application that requires installation, the terminal application 112 can be installed on the terminal device 110. If the terminal application 112 is a web application, the terminal application 112 can be accessed via a browser. If the terminal application 112 is a mini-app, the terminal application 112 can be opened directly on the terminal device 110 by searching for relevant information about the terminal application 112 (e.g., the name of the terminal application 112) or scanning a graphic code (e.g., a barcode, QR code, etc.), without having to install the terminal application 112.

[0059] Figure 2 The following schematically shows the structure of a robotic arm according to some embodiments of the present disclosure. Figure 2 As shown in FIG, the robot arm 120 includes joint 1, joint 2, joint 3, joint 4, joint 5, joint 6, joint 7, and an end effector. It should be understood that the number of joints included in the robot arm 120 includes, but is not limited to, seven. In some embodiments, the robot arm 120 is a seven-degree-of-freedom redundant robot arm.

[0060] Figure 1 An example implementation environment and Figure 2The structural diagram of the robotic arm is merely schematic, and the robotic arm control method according to the present disclosure is not limited to the example implementation environment and robotic arm shown. It should be understood that although the robotic arm 120 and the terminal device 110 are shown and described as separate structures in this document, they can be integrated. Optionally, all steps of the robotic arm control method according to some embodiments of the present disclosure can also be implemented on the robotic arm 120 side, or can also be implemented jointly on the terminal device 110 side and the robotic arm 120 side.

[0061] Figure 3 Schematically shows a flow chart of a method for controlling a robotic arm according to some embodiments of the present disclosure. In some embodiments, as Figure 1 As shown in , the robot arm control method according to the present disclosure can be executed on the terminal device 110 side. In other embodiments, the robot arm control method according to the present disclosure can also be executed by the robot arm 120 and the terminal device 110 in combination.

[0062] like Figure 3 As shown in , a robot arm control method according to some embodiments of the present disclosure may include:

[0063] S310, obtaining robot arm status information;

[0064] S320, determining a motion relationship between a joint space and an operation space of the robotic arm based on the robotic arm state information, wherein the joint space includes a plurality of joints, and the operation space includes an end effector having a plurality of degrees of freedom;

[0065] S330, determining a manipulator control strategy according to the motion relationship and the preset target task, wherein the manipulator control strategy includes a manipulator joint space control strategy based on an acceleration layer; and

[0066] S340: Control the robotic arm using the robotic arm control strategy.

[0067] Steps S310 - S340 are described in detail below.

[0068] At S310 , the robot arm status information is acquired.

[0069] In some embodiments, obtaining the state information of the robotic arm may include obtaining the joint angular velocity and the joint angular acceleration of each joint in the joint space of the robotic arm and obtaining the end effector velocity and the end effector acceleration in the operation space of the robotic arm.

[0070] In some embodiments, acquiring the robot arm state information may further include acquiring an end effector velocity component, an end effector acceleration component, a joint angular velocity component, and a joint angular acceleration component in each degree of freedom direction of the plurality of degrees of freedom.

[0071] In addition, in some embodiments, obtaining the robot arm state information may further include obtaining the joint angles of each joint in the joint space of the robot arm and obtaining the position of the end effector in the operation space of the robot arm. The present disclosure does not limit the obtained robot arm state information.

[0072] At S320 , based on the state information of the robotic arm, a motion relationship between a joint space and an operation space of the robotic arm is determined, where the joint space includes a plurality of joints, and the operation space includes an end effector with a plurality of degrees of freedom.

[0073] The motion relationship described in this article refers to the physical motion relationship in a general sense between the joint space and the operation space of the robotic arm, namely, the transmission relationship, geometric relationship, positional relationship, and the law of change over time.

[0074] In some embodiments, determining the kinematic relationship between the joint space of the robotic arm and the operating space based on the robotic arm state information may include: determining an estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information; and determining the kinematic relationship between the joint space of the robotic arm and the operating space based on the estimated Jacobian matrix. This disclosure does not limit the method for constructing the estimated Jacobian matrix.

[0075] In addition, in some embodiments, the kinematic relationship between the joint space of the robotic arm and the operating space can also be determined by a spatial mapping relationship corresponding to the robotic arm. Methods for solving such spatial mapping relationships include, but are not limited to, analytical methods, optimization methods, and iterative methods.

[0076] At S330 , a manipulator control strategy is determined according to the motion relationship and a preset target task, where the manipulator control strategy includes a manipulator joint space control strategy based on an acceleration layer.

[0077] The preset target mission described in this article refers to a pre-set mission plan for the robot arm, which the robot arm executes and completes according to the mission plan to achieve a certain function. Generally, the preset target mission is set by the user based on actual needs, application scenarios, and operating conditions.

[0078] In some embodiments, determining the robotic arm control strategy based on the motion relationship and preset target tasks may include: constructing a performance optimization target based on the spatial acceleration layer of the robotic arm joint; determining the control constraints of the robotic arm based on the motion relationship and the preset target tasks; constructing a control optimization scheme for the robotic arm based on the performance optimization target and the control constraints; and determining the robotic arm control strategy based on the control optimization scheme.

[0079] In addition, in some embodiments, the robotic arm control strategy may further include a robotic arm joint space control strategy based on a speed layer and a robotic arm joint space control strategy based on an angle layer. Therefore, determining the robotic arm control strategy may further include constructing a performance optimization target based on the robotic arm joint space speed layer and constructing a performance optimization target based on the robotic arm joint space angle layer. In this case, the control constraints of the robotic arm are also correspondingly adjusted to control constraints for the joint speed of the robotic arm and the joint angle of the robotic arm, respectively. The present disclosure does not limit the method for determining the robotic arm control strategy.

[0080] At S340 , the robotic arm is controlled using the robotic arm control strategy.

[0081] In some embodiments, controlling the robotic arm using the robotic arm control strategy may include: determining a control signal according to the robotic arm control strategy; and controlling the robotic arm to perform a target task based on the control signal. The control signal may include an electrical pulse signal, such that the robotic arm is controlled according to the electrical pulse signal. The control signal may also include various other types of signals.

[0082] In this way, by acquiring the state information of the manipulator in real time and based on the inverse kinematics process of the manipulator, using the determined motion relationship between the joint space and the operation space, the structural information of the manipulator with an unknown or inaccurate model can be solved, and the variable parameters in the manipulator control algorithm can be corrected in real time, thereby achieving control of the manipulator, which provides important technical support for the control of the manipulator with an unknown or inaccurate model. At the same time, using the determined manipulator control strategy, the manipulator control method according to the present invention can effectively control the manipulator with an unknown or inaccurate model to efficiently complete the target task, and effectively eliminate the joint angle deviation and position error generated when performing the task, which is of great significance for solving the situation in industrial production where the manipulator model changes due to offset, wear, singularity, etc. In addition, the manipulator control method according to the present invention also has the advantages of strong real-time performance, fast convergence speed, high calculation accuracy, and good robustness, which provides important technical advantages for the manipulator in actual industrial production and application.

[0083] Figure 4 The flowchart of the first sub-step S310 of the robot arm control method according to some embodiments of the present disclosure is schematically shown. Figure 4 As shown in , the first sub-step S310 - obtaining the robot arm status information - may include:

[0084] S310a, obtaining the joint angular velocity and joint angular acceleration of each joint in the joint space of the robotic arm; and

[0085] S310b, obtaining the end effector velocity and end effector acceleration in the operating space of the robotic arm, wherein the end effector velocity includes a velocity component in the direction of each degree of freedom in the multiple degrees of freedom, and the end effector acceleration includes an acceleration component in the direction of each degree of freedom in the multiple degrees of freedom.

[0086] Inverse kinematics for a robotic arm studies how, given a desired trajectory for the end effector in the robotic arm's operational space, the joints in the robotic arm's joint space must move so that the end effector's actual trajectory matches the desired trajectory. Therefore, it should be noted that robotic arm state information can also include the joint angles of each joint in the robotic arm's joint space and the position of the end effector in the robotic arm's operational space.

[0087] In some embodiments, in S310a and S310b, when obtaining the state information of the robotic arm, the state information of the robotic arm can be obtained directly through a sensor or by using the calculation result at the previous moment as the state information of the robotic arm; in other words, the state information of the robotic arm can be obtained in real time. In a specific embodiment, taking the acquisition of the joint angle, joint angular velocity, and joint angular acceleration of a joint in the joint space of the robotic arm as an example, the joint angle information, joint angular velocity information, and joint angular acceleration information of the joint of the robotic arm at the current moment can be obtained by a sensor installed at the position of the joint; alternatively, the state information of the joint of the robotic arm at the current moment can also be obtained by directly calling the state information calculated at the previous moment.

[0088] In some embodiments, in S310b, due to the vectorial nature of the end-effector velocity and acceleration, the end-effector velocity component in each of the multiple degrees of freedom can be first obtained, and then the end-effector velocity components in each of the multiple degrees of freedom can be synthesized into the end-effector velocity. Similarly, the end-effector acceleration component in each of the multiple degrees of freedom can be first obtained, and then the end-effector acceleration components in each of the multiple degrees of freedom can be synthesized into the end-effector acceleration. By obtaining the end-effector velocity and end-effector acceleration components in each of the multiple degrees of freedom, the initial range of the acquired manipulator state information can be expanded, thereby further improving the applicability and effectiveness of the acquired data.

[0089] Figure 5 The flowchart of the second sub-step S320 of the robot arm control method according to some embodiments of the present disclosure is schematically shown. Figure 5As shown in , the second sub-step S320—determining a motion relationship between a joint space and an operation space of the robot arm based on the robot arm state information, wherein the joint space includes a plurality of joints, and the operation space includes an end effector having a plurality of degrees of freedom—may include:

[0090] S320a, determining an estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information; and

[0091] S320b: Determine the motion relationship between the joint space and the operation space of the robotic arm according to the estimated Jacobian matrix.

[0092] In some embodiments, S320a - determining the estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information - may include the following steps: using the gradient descent method to determine the first quantitative relationship between the joint angular acceleration of the robotic arm, the joint angular velocity of the robotic arm, the end effector velocity and the estimated Jacobian matrix; determining the Jacobian matrix estimation formula based on at least the first quantitative relationship; and determining the estimated Jacobian matrix according to the Jacobian matrix estimation formula.

[0093] Further, in some embodiments, the Jacobian matrix estimation formula is determined at least based on the first quantitative relationship, including: based on the time lag error of the joint angular acceleration, correcting the first quantitative relationship to obtain a second quantitative relationship between the joint angular acceleration of the robotic arm, the joint angular velocity of the robotic arm, the end effector velocity and the estimated Jacobian matrix; and determining the Jacobian matrix estimation formula based on the second quantitative relationship.

[0094] In one embodiment, determining the estimated Jacobian matrix may specifically include the following steps:

[0095] 1) Obtain the end effector velocity and acceleration information as well as the joint angular velocity and angular acceleration information of the robot arm;

[0096] 2) Define the error function as ,in, Represents the velocity information of the end effector of the robot arm obtained by measurement; is the angular velocity information of each joint of the robotic arm; represents the estimated Jacobian matrix corresponding to the robotic arm; Represents the two-norm of a vector.

[0097] 3) For the error function e about By taking partial derivatives, a first quantitative relationship between the joint angular acceleration of the manipulator, the joint angular velocity of the manipulator, the end effector velocity, and the estimated Jacobian matrix can be obtained. This first quantitative relationship can be expressed as the following joint angular acceleration iterative formula based on the gradient descent method:

[0098]

[0099] in, Represents the angular acceleration information of each joint of the robotic arm; represents the convergence coefficient of the system, i.e., the iteration step size; is the transpose of a vector or matrix. Generally speaking, the larger the iteration step size, the faster the system converges. Therefore, the iteration step size can be designed or adjusted according to actual needs to achieve better iterative operation results. Optionally, you can also set a condition to exit the iteration so that the task stops when the condition is met.

[0100] 4) Multiply both sides of the joint angular acceleration iterative formula by , add joint angular acceleration compensation , the time lag error generated when the model is modified to solve the time-varying problem can obtain a second quantitative relationship between the joint angular acceleration of the manipulator, the joint angular velocity of the manipulator, the end effector velocity and the estimated Jacobian matrix. The second quantitative relationship can be expressed as the following iterative formula with the joint angular acceleration compensation term added:

[0101]

[0102] in, is the estimated Jacobian matrix; for The time derivative of Represents the acceleration information of the end effector of the manipulator. Specifically, because the state information of the manipulator changes in real time (time-varying), the joint angle acceleration compensation term is added , which represents the time lag error between the joint angular acceleration received by the sensor and the actual joint angular acceleration. This time lag error is inherent and unavoidable. Therefore, when the time lag error is taken into account and compensated for, the model becomes more accurate and more consistent with the actual state of the robot arm.

[0103] 5) In summary, the gradient-based solver obtains the estimated Jacobian matrix corresponding to the robotic arm as

[0104]

[0105] in, A superscript denotes a pseudo-inverse operation on a vector or matrix.

[0106] In some embodiments, S320b—determining the kinematic relationship between the joint space and the operational space of the robotic arm based on the estimated Jacobian matrix—can be based on the inverse kinematics of the robotic arm, i.e., using given end-effector position / velocity / acceleration values ​​to solve for the corresponding robotic arm joint angles / angular velocities / angular accelerations. In one embodiment, the iterative process for determining the estimated Jacobian matrix corresponding to the robotic arm can be as follows: first, an initial value of the estimated Jacobian matrix corresponding to the robotic arm at the current moment is obtained, and then this initial value is continuously substituted into the matrix formula to calculate the value of the estimated Jacobian matrix corresponding to the robotic arm at the next moment. Through continuous iteration and learning, the actual Jacobian matrix value gradually approaches the expected (true) Jacobian matrix value until the difference between the two falls below a predetermined threshold, at which point the iterative calculation ceases, thereby terminating the trajectory planning task. Alternatively, in one embodiment, the iterative process for determining the estimated Jacobian matrix corresponding to the robotic arm can be time-dependent, i.e., if the trajectory planning task ends at a predetermined time, the iterative process ceases. Therefore, this method of controlling the robotic arm by using the estimated Jacobian matrix corresponding to the robotic arm can have the advantages of strong real-time performance, fast convergence speed, high calculation accuracy, and good robustness, which provides important technical advantages for the robotic arm in actual industrial production and applications.

[0107] Figure 6 The flowchart of the third sub-step S330 of the robot arm control method according to some embodiments of the present disclosure is schematically shown. Figure 6 As shown in , the third sub-step S330 - determining a manipulator control strategy based on the motion relationship and the preset target task, wherein the manipulator control strategy includes a manipulator joint space control strategy based on an acceleration layer - may include:

[0108] S330a, constructing a performance optimization target based on the robotic arm joint space acceleration layer;

[0109] S330b, determining control constraints of the robotic arm based on the motion relationship and the preset target task;

[0110] S330c, constructing a control optimization solution for the robotic arm according to the performance optimization target and the control constraint conditions;

[0111] S330d: Determine the robotic arm control strategy based on the control optimization solution.

[0112] In some embodiments, the performance optimization objectives based on the joint-space acceleration layer of the robotic arm and the control constraints of the robotic arm can be flexibly set as needed. For example, in one embodiment, the range of motion of the robotic arm can be limited to avoid obstacles. Alternatively, in another embodiment, the bending angle of the robotic arm can be limited to prevent each joint from reaching its motion limit.

[0113] In some embodiments, S330a - constructing a performance optimization target based on the robotic arm joint space acceleration layer - may include the following steps: constructing an objective function with the target joint angular acceleration of the robotic arm as an independent variable, wherein the objective function involves the norm of the target joint angular acceleration; and constructing the performance optimization target to minimize the objective function. In one embodiment, considering the practicality of the objective function, the objective function may take the target joint angular acceleration of the robotic arm as an independent variable. Alternatively, in other embodiments, considering the practicality of the objective function, the objective function may include but is not limited to: a function with the target joint angular velocity of the robotic arm as an independent variable, a function with the target joint angle of the robotic arm as an independent variable, and the like.

[0114] In some embodiments, S330b - determining the control constraints of the robotic arm based on the motion relationship and the preset target task - may include the following steps: determining a first control constraint regarding the range of the target joint angular acceleration and the target joint angular velocity based on the preset target task; and determining a second control constraint based on the Jacobi equation based on the preset target task and the motion relationship. In a specific embodiment, the first control constraint may include: constraining the target joint angular velocity of the robotic arm within a preset joint angular velocity range, and constraining the target joint angular acceleration of the robotic arm within a preset joint angular acceleration range. Optionally, the first control constraint may also include: constraining the target joint angle of the robotic arm within a preset joint angle range.

[0115] Furthermore, in some embodiments, determining a second control constraint based on the Jacobi equation based on the preset target task and the motion relationship includes: obtaining state information of the manipulator's operating space; determining an error compensation for the operating space based on the target task and the state information of the manipulator's operating space; and determining the second control constraint based on the error compensation and the motion relationship. Because obtaining the state information of the manipulator's operating space has been described in detail above, it will not be repeated here.

[0116] Further, in some embodiments, determining the error compensation of the operating space based on the target task and the state information of the operating space of the robotic arm includes: determining the target speed and target position of the end effector according to the target task; determining the current speed and current position of the end effector according to the state information of the operating space of the robotic arm; determining the speed compensation of the end effector based on the target speed and current speed of the end effector; determining the position compensation of the end effector based on the target position and current position of the end effector; and determining the error compensation of the operating space based on at least one of the speed compensation and position compensation of the end effector.

[0117] In some embodiments, S330c—constructing a control optimization solution for the robotic arm based on the performance optimization objective and the control constraints—may include constructing the robotic arm control optimization solution as the following quadratic programming problem:

[0118]

[0119] Among them, formula (1) is the performance optimization goal, the symbol min represents minimization, formulas (2)-(4) are control constraints, the symbol st represents "so that", among them, is the desired end-effector position, is the desired end-effector velocity, is the desired end-effector acceleration, is the end effector position, is the end effector speed, is the position error compensation, is the compensation coefficient, is the speed error compensation, is the compensation coefficient, To estimate the Jacobian matrix, for The time derivative of is the joint angular velocity, is the joint angular acceleration, with the superscript represents the transpose operation, and are the lower and upper constraints of the joint angular velocity, respectively, and and are the lower and upper constraints of the joint angular acceleration, respectively. These compensation coefficients and the lower and upper constraints can be predetermined. In practical applications, the actual end effector position and end effector velocity of the manipulator should be consistent with the desired end effector position and desired end effector velocity, i.e. and Therefore, in order to meet the above conditions, the following error compensation items are designed ,in and are predetermined positive values. This robotic arm control optimization scheme can use the robotic arm execution trajectory planning problem as an equality constraint, and the Jacobian matrix in the robotic arm control optimization scheme can be estimated by the above data-driven algorithm, ultimately transforming it into an optimization problem based on quadratic programming.

[0120] In some embodiments, S330d—determining the robotic arm control strategy based on the control optimization solution—may include the following steps: solving the control optimization solution using a quadratic programming solver to obtain a target joint angular acceleration of the robotic arm; and determining the robotic arm control strategy based on the target joint angular acceleration. Alternatively, in one embodiment, the robotic arm control strategy may be determined by obtaining a target joint angular velocity of the robotic arm.

[0121] In some embodiments, for the designed quadratic programming-based control optimization solution, an example of a feasible quadratic programming solver is given below (but the present disclosure is not limited to this solver example). By using the Lagrangian projection method and the velocity compensation method, the estimated Jacobian matrix corresponding to the robotic arm can be equivalent to a piecewise linear projection equation system:

[0122]

[0123] in, is the time derivative of the joint angular acceleration; represents the convergence coefficient; represents the auxiliary variables of the projection equations, express The time derivative of . (i.e. function P ( x )) represents the projection function about physical constraints, which can be expressed as:

[0124]

[0125] in, ; ; Indicates the parameters that convert the angle constraint to the velocity layer.

[0126] In one embodiment, when there is a finite set of solutions to the above-mentioned system of equations, all solutions to the system of equations can be obtained, and then a set of solutions to the system of equations can be selected according to actual needs or according to set selection rules, so as to obtain an estimated Jacobian matrix corresponding to the robotic arm. Alternatively, in one embodiment, when there is an infinite set of solutions to the above-mentioned system of equations corresponding to the robotic arm, any set of solutions to the system of equations can be obtained, and the estimated Jacobian matrix corresponding to the robotic arm can be obtained through this set of solutions. Alternatively, in one embodiment, the constraints of the solutions to the system of equations can be set first, and then the solutions to the system of equations that meet the constraints can be obtained, and the estimated Jacobian matrix corresponding to the robotic arm can be obtained through this set of solutions. The present disclosure does not limit the method of solving equations.

[0127] Figure 7 The flowchart of the fourth sub-step S340 of the robot arm control method according to some embodiments of the present disclosure is schematically shown. Figure 7 As shown in , the fourth sub-step S340—controlling the robotic arm using the robotic arm control strategy—may include:

[0128] S340a, determining a control signal according to the robot arm control strategy; and

[0129] S340b: Control the robotic arm to perform a target task based on the control signal.

[0130] In some embodiments, in S340a and S340b, the robotic arm control strategy can be used to solve a problem using a solver to obtain a solution result; the solution result is converted into a required control signal and the control signal is transmitted to the robotic arm, so that the robotic arm is controlled according to the control signal. Furthermore, in some embodiments, the control signal can include an electrical pulse signal, which can drive the robotic arm to complete the control of the joint angular acceleration layer when the structural parameters are unknown, thereby driving the robotic arm to complete a specified task.

[0131] Figures 8A-8E The control effect diagram of the robotic arm according to some embodiments of the present disclosure is schematically shown. In order to demonstrate the actual system design process, a seven-degree-of-freedom robotic arm is used as an example for explanation. Figures 8A-8E As shown, the trajectory experiment of the seven-degree-of-freedom redundant manipulator of the disclosed method was simulated using MATLAB software. The specific modeling parameters are as follows (where superscript T is the transpose of a vector or matrix, and pi represents pi): the execution task is a circular path, the task execution time is 10 seconds, and the trajectory radius is rice, , , , , the lower limit of joint angular acceleration is rad / s 2 , the upper limit of joint angular acceleration is rad / s 2 , the lower limit of joint angular velocity is radians / second, the upper limit of joint angular velocity is rad / s, the initial joint angle is radians, the initial joint angular velocity is rad / s, , the estimated initial value of the Jacobian matrix is:

[0132] .

[0133] Under the above conditions, the control effect diagram of the seven-degree-of-freedom redundant manipulator is obtained. Specifically, Figure 8A The figure shows the motion process of the seven-degree-of-freedom redundant manipulator when executing a predetermined trajectory planning task; Figure 8A It can be seen that in three-dimensional space, the actual path trajectory of the seven-degree-of-freedom redundant manipulator can coincide well with the expected path trajectory, and its trajectory changes are in a smooth and continuous state, so that the given trajectory task is well executed, achieving a good control effect of the redundant manipulator, which reflects the effectiveness of the manipulator control method disclosed in the present invention. Figure 8B FIG shows a diagram of joint angle changes of a seven-degree-of-freedom redundant manipulator, where the clockwise direction is defined as positive and the counterclockwise direction is defined as negative (in other examples, the clockwise direction may be defined as negative and the counterclockwise direction may be defined as positive); Figure 8B It can be seen from the figure that the angle change trend of each joint of the seven-degree-of-freedom redundant robotic arm is stable, thus achieving extremely high control accuracy. Figure 8C FIG shows a graph showing changes in joint angular velocity of a seven-DOF redundant manipulator, where the clockwise direction is defined as positive and the counterclockwise direction is defined as negative (in other examples, the clockwise direction may be defined as negative and the counterclockwise direction may be defined as positive); FIG. Figure 8C It can be seen that the angular velocity fluctuations of each joint of the seven-degree-of-freedom redundant robotic arm are continuous and smooth. Because in actual industrial production and applications, if the joint angular velocity fluctuates violently, the redundant robotic arm will not be able to perform the task well, so effective control of the redundant robotic arm is achieved. Figure 8D FIG1 shows a graph showing changes in joint angular acceleration of a seven-degree-of-freedom redundant manipulator, where the clockwise direction is defined as positive and the counterclockwise direction is defined as negative (in other examples, the clockwise direction may be defined as negative and the counterclockwise direction may be defined as positive); FIG2 shows a graph showing changes in joint angular acceleration of a seven-degree-of-freedom redundant manipulator, where the clockwise direction is defined as positive and the counterclockwise direction is defined as negative (in other examples, the clockwise direction may be defined as negative and the counterclockwise direction may be defined as positive); Figure 8D It can be seen that the change of the angular acceleration of each joint of the seven-degree-of-freedom redundant manipulator is controlled within an appropriate range, that is, 0.5 rad / s 2Therefore, the angular acceleration of each joint changes slowly, which reflects the effectiveness of the robot arm control method disclosed in the present invention. Figure 8E The figure shows the position error variation of the end effector of the seven-DOF redundant manipulator. Figure 8E It can be seen that during the entire task execution process, the error of the redundant manipulator in executing the path planning task is well controlled within a certain range, among which the position error of the end effector on the X-axis, Y-axis and Z-axis is kept within 10 -3 The control accuracy achieved is at the meter level, demonstrating the high efficiency of the disclosed robotic arm control method. The control effect of the seven-degree-of-freedom redundant robotic arm has been demonstrated in actual redundant robotic arm operation experiments and industrial production and applications, meeting the requirements of most common tasks, such as drawing, object placement, welding, spraying, material inspection, and assembly.

[0134] Figure 9 An example block diagram of a robot arm control device 900 according to some embodiments of the present disclosure is schematically shown. Figure 9 The determining robot arm control device 900 shown in FIG may correspond to Figure 1 The terminal device 110 shown in .

[0135] like Figure 9 As shown in , the robot control device 900 may include an information acquisition module 910, a motion relationship determination module 920, a control strategy determination module 930, and a control module 940. The information acquisition module 910 is configured to obtain robot arm state information. The motion relationship determination module 920 is configured to determine the motion relationship between the joint space and the operation space of the robot arm based on the robot arm state information, the joint space including multiple joints, and the operation space including an end effector with multiple degrees of freedom. The control strategy determination module 930 is configured to determine the robot arm control strategy based on the motion relationship and the preset target task, the robot arm control strategy including a robot arm joint space control strategy based on the acceleration layer. The control module 940 is configured to control the robot arm using the robot arm control strategy.

[0136] It should be noted that the various modules described above can be implemented in software or hardware or a combination of both. Multiple different modules can be implemented in the same software or hardware structure, or one module can be implemented by multiple different software or hardware structures.

[0137] The robot arm control device provided by the present disclosure obtains the robot arm state information in real time and, based on the inverse kinematics process of the robot arm, utilizes the determined motion relationship between the joint space and the operation space to solve the structural information of the robot arm with an unknown or inaccurate model, and corrects the variable parameters in the robot arm control algorithm in real time, thereby realizing the control of the robot arm, which provides important technical support for the control of the robot arm with an unknown or inaccurate model. At the same time, using the determined robot arm control strategy, the robot arm control device according to the present disclosure can effectively control the robot arm with an unknown or inaccurate model to efficiently complete the target task, and effectively eliminate the joint angle deviation and position error generated when performing the task, which is of great significance for solving the situation in industrial production where the robot arm model changes due to offset, wear, singularity, etc. In addition, the robot arm control device according to the present disclosure also has the advantages of strong real-time performance, fast convergence speed, high calculation accuracy, and good robustness, which provides important technical advantages for the robot arm in actual industrial production and application.

[0138] Figure 10 Schematically illustrates an example block diagram of a computing device 1000 according to some embodiments of the present disclosure. The computing device 1000 may represent a device for implementing the various means or modules described herein and / or performing the various methods described herein. The computing device 1000 may be, for example, a server, a desktop computer, a laptop computer, a tablet, a smart phone, a smart watch, a wearable device, or any other suitable computing device or computing system, which may include various levels of devices ranging from full-resource devices with a large amount of storage and processing resources to low-resource devices with limited storage and / or processing resources. In some embodiments, the above description of the computing device 1000 may be a computer system or a computing system. Figure 9 The described robotic arm control apparatus 900 may be implemented in one or more computing devices 1000 , respectively.

[0139] like Figure 10 As shown in FIG, an example computing device 1000 includes a processing system 1001, one or more computer-readable media 1002, and one or more I / O interfaces 1003 that are communicatively coupled to each other. Although not shown, the computing device 1000 may also include a system bus or other data and command transmission system that couples the various components to each other. The system bus may include any one or a combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of a variety of bus architectures. Alternatively, other components such as control and data lines may also be included.

[0140] Processing system 1001 represents functionality that uses hardware to perform one or more operations. Thus, processing system 1001 is illustrated as including hardware elements 1004 that can be configured as processors, functional blocks, and the like. This can include hardware implementation as application-specific integrated circuits or other logic devices formed using one or more semiconductors. Hardware elements 1004 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor may be comprised of semiconductor(s) and / or transistors (e.g., an electronic integrated circuit (IC)). In such a context, processor-executable instructions may be electronically executable instructions.

[0141] Computer-readable medium 1002 is illustrated as including memory / storage 1005. Memory / storage 1005 represents memory / storage associated with one or more computer-readable media. Memory / storage 1005 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disks, magnetic disks, etc.). Memory / storage 1005 may include fixed media (e.g., RAM, ROM, fixed hard drives, etc.) as well as removable media (e.g., flash memory, removable hard drives, optical disks, etc.). For example, memory / storage 1005 may be used to store the first audio of the first category of users, the queued list of requests, and the like, as mentioned in the above embodiments. Computer-readable medium 1002 may be configured in various other ways, as further described below.

[0142] One or more I / O (input / output) interfaces 1003 represent functionality that allows a user to enter commands and information into the computing device 1000 and also allows information to be displayed to the user and / or sent to other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone (e.g., for voice input), a scanner, touch functionality (e.g., a capacitive or other sensor configured to detect physical touch), a camera (e.g., capable of detecting gestures without touch using visible or invisible wavelengths (such as infrared frequencies)), a network card, a receiver, and the like. Examples of output devices include a display device (e.g., a monitor or projector), a speaker, a printer, a tactile response device, a network card, a transmitter, and the like. For example, in the embodiments described above, the first category of users and the second category of users can use the input interfaces on their respective terminal devices to initiate requests and record audio and / or video, and can use the output interfaces to view various notifications, watch videos, or listen to audio, and the like.

[0143] The computing device 1000 also includes a robot control strategy 1006. The robot control strategy 1006 can be stored as a computer program instruction in the memory / storage device 1005, or can be hardware or firmware. The robot control strategy 1006 can be implemented together with the processing system 1001 and the like. Figure 9 The entire functions of each module of the robot arm control device 900 are described.

[0144] Various techniques may be described herein in the general context of software, hardware, elements, or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," etc. generally refer to software, firmware, hardware, or a combination thereof. A feature of the techniques described herein is that they are platform-independent, meaning that these techniques can be implemented on a variety of computing platforms with a variety of processors.

[0145] An implementation of the described modules and techniques may be stored on or transmitted across some form of computer-readable media. Computer-readable media may include various media accessible by the computing device 1000. By way of example and not limitation, computer-readable media may include "computer-readable storage media" and "computer-readable signal media."

[0146] "Computer-readable storage media" refers to media and / or devices capable of persistently storing information, as opposed to mere signal transmissions, carrier waves, or signals themselves. Thus, computer-readable storage media refers to non-signal-bearing media. Computer-readable storage media includes hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented in a method or technology suitable for storing information, such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVDs) or other optical storage devices, hard drives, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage devices, tangible media, or articles of manufacture suitable for storing the desired information and accessed by a computer.

[0147] "Computer-readable signal media" refers to signal-bearing media configured to transmit instructions to the hardware of computing device 1000, such as via a network. Signal media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, data signal, or other transport mechanism. Signal media also includes any information delivery media. By way of example and not limitation, signal media include wired media such as a wired network or direct connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0148] As previously described, hardware elements 1004 and computer-readable media 1002 represent instructions, modules, programmable device logic, and / or fixed device logic implemented in hardware that, in some embodiments, can be used to implement at least some aspects of the technology described herein. Hardware elements can include integrated circuits or systems on a chip, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon or components of other hardware devices. In this context, hardware elements can be considered processing devices that execute program tasks defined by the instructions, modules, and / or logic embodied by the hardware elements, as well as hardware devices for storing instructions for execution, such as the computer-readable storage media described previously.

[0149] The aforementioned combination may also be used to implement the various techniques and modules described herein. Therefore, software, hardware or program modules and other program modules may be implemented as one or more instructions and / or logic on some form of computer-readable storage medium and / or embodied by one or more hardware elements 1004. The computing device 1000 may be configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Therefore, for example, by using a computer-readable storage medium and / or hardware elements 1004 of a processing system, a module may be implemented as a module that can be executed by the computing device 1000 as software, at least in part, in hardware. Instructions and / or functions may be executed / operable by, for example, one or more computing devices 1000 and / or processing systems 1001 to implement the techniques, modules and examples described herein.

[0150] The techniques described herein may be supported by these various configurations of computing device 1000 and are not limited to the specific examples of the techniques described herein.

[0151] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer programs. For example, embodiments of the present disclosure provide a computer program product comprising a computer program carried on a computer-readable medium, the computer program including program code for executing at least one step of the method embodiments of the present disclosure.

[0152] In some embodiments of the present disclosure, one or more computer-readable storage media are provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed, the robot arm control method according to some embodiments of the present disclosure is implemented. The various steps of the robot arm control method according to some embodiments of the present disclosure can be converted into computer-readable instructions through programming and stored in a computer-readable storage medium. When such a computer-readable storage medium is read or accessed by a computing device or computer, the computer-readable instructions therein are executed by a processor on the computing device or computer to implement the method according to some embodiments of the present disclosure.

[0153] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0154] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment, or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed in a sequence other than as shown or discussed (including in a substantially simultaneous manner or in reverse order depending on the functions involved), which should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0155] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0156] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, it can be implemented using any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0157] Those skilled in the art will appreciate that all or part of the steps of the method of the above-described embodiment may be accomplished through hardware associated with program instructions, and the program may be stored in a computer-readable storage medium, which, when executed, includes executing one or a combination of the steps of the method embodiment.

[0158] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0159] Although the present disclosure has been described in conjunction with some embodiments, it is not intended to be limited to the specific forms set forth herein. On the contrary, the scope of the present disclosure is limited only by the appended claims. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined, and inclusion in different claims does not imply that a combination of features is not feasible and / or advantageous. The order of the features in the claims does not imply any specific order in which the features must work. Furthermore, in the claims, the word "comprising" does not exclude other elements, and the term "a" or "an" does not exclude a plurality. The reference numerals in the claims are provided merely as clear examples and should not be construed as limiting the scope of the claims in any way.

Claims

1. A method for controlling a robotic arm, comprising: Acquiring state information of the manipulator, wherein the state information of the manipulator includes state information of the joint space and state information of the operation space; Determining a motion relationship between a joint space and an operation space of the robotic arm based on the robotic arm state information, the joint space including a plurality of joints, and the operation space including an end effector having a plurality of degrees of freedom; Determining a manipulator control strategy based on the motion relationship and the preset target task, wherein the manipulator control strategy includes a manipulator joint space control strategy based on an acceleration layer; controlling the robotic arm using the robotic arm control strategy, Wherein, determining the robot arm control strategy according to the motion relationship and the preset target task includes: Constructing a performance optimization target based on the robotic arm joint space acceleration layer; Determining error compensation of the operating space based on the preset target task and state information of the operating space of the robotic arm; Determining control constraints of the robotic arm based on the preset target task, the error compensation, and the motion relationship; Constructing a control optimization scheme for the robotic arm according to the performance optimization target and the control constraint conditions; Based on the control optimization scheme, the robotic arm control strategy is determined.

2. The method according to claim 1, wherein obtaining the robot arm status information comprises: The joint angular velocity and the joint angular acceleration of each joint in the joint space of the robotic arm are obtained, as well as the end effector velocity and the end effector acceleration in the operating space of the robotic arm, wherein the end effector velocity includes a velocity component in the direction of each degree of freedom in the multiple degrees of freedom, and the end effector acceleration includes an acceleration component in the direction of each degree of freedom in the multiple degrees of freedom.

3. The method according to claim 1 or 2, wherein determining the motion relationship between the joint space and the operation space of the robotic arm based on the robotic arm state information comprises: determining an estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information, A motion relationship between a joint space and an operation space of the robotic arm is determined according to the estimated Jacobian matrix.

4. The method according to claim 3, wherein determining an estimated Jacobian matrix corresponding to the robotic arm based on the robotic arm state information comprises: Determining a first quantitative relationship between the joint angular acceleration of the manipulator, the joint angular velocity of the manipulator, the end effector velocity, and the estimated Jacobian matrix using a gradient descent method; Determining a Jacobian matrix estimation formula based at least on the first quantitative relationship; The estimated Jacobian matrix is ​​determined according to the Jacobian matrix estimation formula.

5. The method according to claim 4, wherein the determining of the Jacobian matrix estimation formula based at least on the first quantitative relationship comprises: Based on the time lag error of the joint angular acceleration, correcting the first quantitative relationship to obtain a second quantitative relationship between the joint angular acceleration of the manipulator, the joint angular velocity of the manipulator, the end effector velocity, and the estimated Jacobian matrix; According to the second quantitative relationship, a Jacobian matrix estimation formula is determined.

6. The method according to claim 1 or 2, wherein the constructing of a performance optimization target based on the robotic arm joint space acceleration layer comprises: Construct an objective function with the norm of the manipulator acceleration and the target joint angle acceleration as independent variables, wherein the objective function involves the target joint angle The performance optimization goal is constructed as minimizing the objective function.

7. The method according to claim 1 or 2, wherein determining the control constraint conditions of the robotic arm based on the preset target task, the error compensation, and the motion relationship comprises: Determining a first control constraint condition regarding a range of a target joint angular acceleration and a target joint angular velocity according to the preset target task; A second control constraint condition based on a Jacobi equation is determined according to the error compensation and the kinematic relationship.

8. The method according to claim 1 or 2, wherein determining the error compensation of the operating space based on the preset target task and the state information of the operating space of the manipulator comprises: Determining a target speed and a target position of the end effector according to the preset target task; Determining the current speed and current position of the end effector according to the state information of the operating space of the robotic arm; determining a velocity compensation for the end effector based on a target velocity and a current velocity of the end effector; determining a position compensation of the end effector based on a target position and a current position of the end effector; An error compensation of the operating space is determined based on at least one of a velocity compensation and a position compensation of the end effector.

9. The method according to claim 8, wherein constructing a control optimization scheme for the robotic arm according to the performance optimization target and the control constraint condition comprises: The robot control optimization scheme is formulated as the following quadratic programming problem: Among them, formula (1) is the performance optimization goal, formulas (2)-(4) are control constraints, and r d ∈R m is the desired end-effector position, is the desired end-effector velocity, is the desired end-effector acceleration, r a is the end effector position, is the end effector velocity, κ(r a -r d ) is the position error compensation, κ is the compensation coefficient, is the speed error compensation, η is the compensation coefficient, To estimate the Jacobian matrix, for The time derivative of is the joint angular velocity, is the joint angular acceleration, the superscript T represents the transposition operation, and are the lower and upper constraints of the joint angular velocity, respectively, and and are the lower and upper constraints of the joint angular acceleration respectively.

10. The method according to claim 9, wherein determining the robotic arm control strategy based on the control optimization solution comprises: Solving the control optimization scheme by a quadratic programming solver to obtain the target joint angular acceleration of the robotic arm; The robotic arm control strategy is determined based on the target joint angular acceleration.

11. The method according to claim 1 , wherein controlling the robotic arm using the robotic arm control strategy comprises: Determining a control signal according to the robotic arm control strategy; The robotic arm is controlled to perform a target task based on the control signal.

12. A robotic arm control device, comprising: An information acquisition module is configured to acquire state information of the manipulator, wherein the state information of the manipulator includes state information of the joint space and state information of the operation space; a motion relationship determination module configured to determine a motion relationship between a joint space and an operation space of the robotic arm based on the robotic arm state information, the joint space including a plurality of joints, and the operation space including an end effector having a plurality of degrees of freedom; A control strategy determination module is configured to determine a manipulator control strategy based on the motion relationship and a preset target task, the manipulator control strategy including a manipulator joint space control strategy based on an acceleration layer, wherein determining the manipulator control strategy based on the motion relationship and the preset target task includes: constructing a performance optimization target based on the manipulator joint space acceleration layer; determining an error compensation for the operation space based on the preset target task and state information of the operation space of the manipulator; determining a control constraint condition of the manipulator based on the preset target task, the error compensation, and the motion relationship; constructing a control optimization scheme for the manipulator based on the performance optimization target and the control constraint condition; and determining the manipulator control strategy based on the control optimization scheme; and A control module is configured to control the robotic arm using the robotic arm control strategy.

13. A computing device comprising: memory and processor, A computer program is stored in the memory, and when the computer program is executed by the processor, it causes the processor to perform the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-readable instructions thereon, wherein the computer-readable instructions implement the method according to any one of claims 1 to 11 when executed.

15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images