Data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, electronic equipment and a storage medium. According to the starting and ending states of the system and an initial response time matrix, the joint state and the control force of each joint in the system at each moment are determined, and the initial response time matrix comprises the motion type of each joint in the system at each moment; determining a total cost function corresponding to the joint according to the motion type of the joint; according to the total cost function corresponding to all the joints and the joint state and the control force at each moment, a dynamic energy function of the system is constructed; recalculating the initial response time matrix by adopting a dynamic energy function of the system to obtain an updated response time matrix; based on the updated response time matrix, new joint states and control forces of all joints in the system at all moments are determined, and the new joint states and control forces are used for controlling movement of all the joints in the system. The joint state and the control force obtained by the method can improve the movement effect of the system.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] As an important branch of robotics technology, the design and motion control of bionic robotic arms are inspired by biology and are designed to simulate the movement ability of human arms. In order to achieve precise and flexible operations, bionic robotic arms are usually composed of multiple joints. The coordinated movement of multiple joints refers to the coordinated work of multiple joints in a certain rhythm and sequence to enable the bionic robotic arm to complete more complex multi-degree-of-freedom operations; compliant movement is mainly used when the bionic robotic arm interacts with external objects or the environment, so that the robotic arm has a certain degree of elasticity and can maintain stability when subjected to external forces or interference. This is similar to the compliance of the human arm - automatically adjusting the posture and force distribution when contacting an object to avoid excessive rigid operation.

[0003] Among them, the joint trajectory can be understood as the angle change of the joint at each moment, that is, the movement state of the joint at each moment. However, the existing robot arm trajectory generation scheme or motion control scheme cannot effectively handle complex joint motion patterns, especially in scenarios that require coordinated motion and smooth motion. The generated trajectory often cannot take into account both simulation and efficiency, resulting in poor results when the system moves along the trajectory. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device, electronic device and storage medium, which can take into account both simulation and efficiency, thereby improving the effect of system movement.

[0005] The present application provides a data processing method, including:

[0006] Determine the joint state and control force of each joint in the system at each moment according to the system's start and end states and the initial response time matrix, where the initial response time matrix includes the motion type of each joint in the system at each moment;

[0007] According to the movement type of the joint, determine the total cost function corresponding to the joint;

[0008] Construct the system's dynamic energy function based on the total cost function corresponding to all joints, the joint states and control forces at each moment;

[0009] The initial response time matrix is ​​recalculated using the system's dynamic energy function to obtain an updated response time matrix, which includes the new motion types of each joint in the system at each moment;

[0010] Based on the updated response time matrix, the new joint state and control force of each joint in the system at each moment are determined, and the new joint state and control force are used to control the movement of each joint in the system.

[0011] The present application also provides a data processing device, including:

[0012] An initial unit, used to determine the joint state and control force of each joint in the system at each moment according to the system's start and end states and an initial response time matrix, wherein the initial response time matrix includes the motion type of each joint in the system at each moment;

[0013] The cost unit is used to determine the total cost function corresponding to the joint according to the motion type of the joint;

[0014] The dynamics unit is used to construct the dynamics energy function of the system based on the total cost function corresponding to all joints, the joint states and control forces at each moment;

[0015] A recalculation unit is used to recalculate the initial response time matrix using the system's dynamic energy function to obtain an updated response time matrix, where the updated response time matrix includes a new motion type of each joint in the system at each moment;

[0016] The determination unit is used to determine the new joint state and control force of each joint in the system at each moment based on the updated response time matrix, and the new joint state and control force are used to control the movement of each joint in the system.

[0017] In some embodiments, the initialization unit is used to:

[0018] Using the simulation motion equation, based on the start and end states and the initial response time matrix, the joint state and control force of the target joint in the system at the target time are determined;

[0019] If the motion type of the target joint at the target time is a passive type, the joint state of the target joint at the target time is corrected according to the compliance correction equation.

[0020] In some embodiments, the simulation motion equation also includes a multi-joint coordination equation, which is used to calculate the sum of the control force term and the multi-joint influence term based on the single torque term; the single torque term includes the product of the angular acceleration and the moment of inertia of the target joint at the target time, and the moment of inertia describes the inertia of the target joint during rotational motion; the control force term includes the control force of the target joint at the target time; the multi-joint influence term includes the interaction torque exerted on the target joint by all other joints at the target time.

[0021] In some embodiments, the simulated motion equations include control equations, and the data processing device is further configured to:

[0022] A control equation is constructed, and the control equation is used to calculate the sum of a weighted state term and a weighted force term according to a derivative of the joint state; the weighted state term includes a weighted result of weighting the joint state of the target joint at a target time using a state law weight, and the state law weight describes the movement of the target joint when it is in a specific joint state; the weighted force term includes a weighted result of weighting the control force of the target joint at the target time using a force law weight, and the force law weight describes the movement of the target joint when it is subjected to a specific force.

[0023] In some embodiments, the joint state includes angle, angular velocity and angular acceleration, and the data processing device is further used for:

[0024] A compliance correction equation is constructed, and the compliance correction equation is used to calculate the sum of the inertia term, the damping term and the elastic term according to the reaction force term; the inertia term includes the addition result of adding the angular acceleration correction value of the target joint at the target time by the inertia weight, and the inertia weight describes the inertial motion of the target joint; the damping term includes the addition result of adding the angular velocity correction value of the target joint at the target time by the damping weight, and the damping weight describes the damped motion of the target joint; the elastic term includes the addition result of adding the angle correction value of the target joint at the target time by the elastic weight, and the elastic weight describes the elastic recovery motion of the target joint;

[0025] According to the compliance correction equation, the joint state of the target joint at the target time is corrected, including:

[0026] The angular acceleration of the target joint at the target time is corrected according to the angular acceleration correction value obtained by the compliance correction equation;

[0027] The angular velocity of the target joint at the target moment is corrected according to the angular velocity correction value obtained by the compliance correction equation;

[0028] The angle of the target joint at the target moment is corrected according to the angle correction value obtained by the compliance correction equation.

[0029] In some embodiments, the joint state includes angle, angular velocity and angular acceleration, and the joint state and control force of each joint in the system at each moment are determined based on the start and end states and the initial response time matrix using the simulation motion equation, including:

[0030] If the motion type of the target joint in the system at the target time is active, the active joint coupling function is used, combined with the simulation motion equation, based on the updated response time matrix to calculate the joint state and control force of the target joint at the target time;

[0031] If the motion type of the target joint in the system at the target time is passive, the passive joint coupling function is used, combined with the simulation motion equation, to calculate the joint state and control force of the target joint at the target time based on the updated response time matrix.

[0032] In some embodiments, the data processing apparatus is further configured to:

[0033] An active joint coupling function is constructed, and the active joint coupling function is used to calculate the sum of the first monomer inertia term, the first coupling inertia term, the first centrifugal force data, and the first gravity data according to the sum of the first input force and the first contact force; the first monomer inertia term includes the addition result of the first monomer inertia weight to the angular acceleration of the target joint at the target time, and the first monomer inertia weight describes the individual inertial motion of the target joint; the first coupling inertia term includes the addition result of the first coupling inertia weight to the angular acceleration of other joints at the target time, and the first coupling inertia weight describes the inertial motion of the target joint when it is affected by other joints; the first centrifugal force data describes the motion of the target joint when it is affected by the centrifugal force, and the first gravity data describes the motion of the target joint when it is affected by gravity.

[0034] In some embodiments, the data processing apparatus is further configured to:

[0035] A passive joint coupling function is constructed, and the passive joint coupling function is used to calculate the sum of the second monomer inertia term, the second coupling inertia term, the second centrifugal force data, and the second gravity data according to the sum of the second input force and the second contact force; the second monomer inertia term includes the addition result of the second monomer inertia weight to the angular acceleration of the target joint at the target time, and the second monomer inertia weight describes the individual inertial motion of the target joint; the second coupling inertia term includes the addition result of the second coupling inertia weight to the angular acceleration of other joints at the target time, and the second coupling inertia weight describes the inertial motion of the target joint when it is affected by other joints; the second centrifugal force data describes the motion of the target joint when it is affected by the centrifugal force, and the second gravity data describes the motion of the target joint when it is affected by gravity.

[0036] In some embodiments, the cost unit is used to:

[0037] According to the motion type of the target joint at the target time, determine the cost function corresponding to the target joint at the target time;

[0038] According to the cost function corresponding to the target joint at all times, the total cost function corresponding to the target joint is obtained.

[0039] In some embodiments, determining a cost function corresponding to the target joint at the target time according to the motion type of the target joint at the target time includes:

[0040] If the motion type of the target joint at the target time is active motion, then the cost function corresponding to the target joint at the target time is determined to be a time cost function, and the time cost function is used to calculate the time cost by integrating the time according to the target time;

[0041] If the motion type of the target joint at the target time is passive motion, the cost function corresponding to the target joint at the target time is determined to be an energy cost function, which is used to calculate the energy cost based on the integral of the square of the control force of the target joint at the target time at the target time.

[0042] In some embodiments, the kinetic unit is used to:

[0043] Obtaining a set of physical constraints of the system, the set of physical constraints including weighted physical constraints of each joint in the system, the weighted physical constraints including physical constraints weighted by constraint strength weights;

[0044] The system's dynamic energy function is constructed based on the set of physical constraints, the total cost function corresponding to all joints, the joint states and control forces at each moment.

[0045] In some embodiments, the physical constraints include strict constraints and weighted loose constraints, and the weighted loose constraints include loose constraints weighted by penalty weights.

[0046] In some embodiments, the cost unit is also used to:

[0047] Obtain an optimization function, where the optimization objectives include a multi-joint total cost function and a throwing distance function; the multi-joint total cost function is used to calculate the overall performance cost of the system according to the joint states of each joint at each moment; the throwing distance function is used to calculate the flying distance of the thrown object after being thrown by the throwing joint according to the joint states of the throwing joints in the system;

[0048] Add the optimization function to the total cost function corresponding to the joint.

[0049] An embodiment of the present application also provides an electronic device, including a memory storing multiple instructions; the processor loads instructions from the memory to execute the steps in any data processing method provided in the embodiment of the present application.

[0050] An embodiment of the present application also provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps in any data processing method provided in the embodiment of the present application.

[0051] The embodiments of the present application can determine the joint state and control force of each joint in the system at each moment according to the system's start and end states and the initial response time matrix, provide initial conditions for the dynamic optimization of the system, and ensure that the movement of each joint at the initial moment meets the requirements of the overall action. The initial response time matrix includes the movement type of each joint in the system at each moment; according to the movement type of the joint, determine the total cost function corresponding to the joint, which comprehensively considers factors such as the joint's movement energy consumption, stability, response time, etc., and is used to evaluate the system's movement efficiency. Since different types of motion (active or passive) have different effects on the cost, the cost function is optimized according to the motion type, so that the control of the bionic robotic arm is both flexible and energy-saving; the system's dynamic energy function is constructed based on the total cost function corresponding to all joints, the joint state and control force at each moment; the system's dynamic energy function is used to recalculate the initial response time matrix to obtain an updated response time matrix, thereby optimizing the motion state and motion type of each joint to ensure that the system achieves the optimal motion effect at different action stages. The updated response time matrix includes the new motion type of each joint in the system at each moment; based on the updated response time matrix, the new joint state and control force of each joint in the system at each moment are determined, and the new joint state and control force are used to control the motion of each joint in the system. In this process, the active and passive joints can be flexibly switched according to task requirements, and coordinated and compliant motion can be optimized synchronously, thereby achieving efficient action execution of the system.

[0052] This application combines the characteristics of active and passive joints, coordination and compliant movement of the bionic manipulator, and through dynamic adjustment and optimization of the system's motion state, ensures the simulation effect and control efficiency in complex operation scenarios. Therefore, this solution can take into account both simulation and efficiency, thereby improving the effect of system movement. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1a It is a scenario schematic diagram of the data processing method provided in the embodiment of the present application;

[0055] Figure 1b It is a flowchart of a data processing method provided in an embodiment of the present application;

[0056] Figure 2a is a schematic diagram of the application of the data processing method provided in the embodiment of the present application in a robotic arm throwing scenario;

[0057] Figure 2b is a schematic diagram of the application of the data processing method provided in the embodiment of the present application in a robotic arm throwing scenario;

[0058] Figure 2c is a schematic diagram of applying the data processing method provided in an embodiment of the present application to the movement of a humanoid robot;

[0059] Figure 3 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0060] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0062] Embodiments of the present application provide a data processing method, device, electronic device, and storage medium.

[0063] The data processing device can be integrated into an electronic device, which can be a mechanical arm, a robot or other automated equipment containing joints. The device integrates multiple joint elements and a control system to perform precise motion control tasks.

[0064] For example, the electronic device can be a service robot, a medical robot, a bionic robot, an automated production line equipment, and so on.

[0065] For example, refer to Figure 1a , the electronic device can be a two-degree-of-freedom robotic arm, the system formed by the robotic arm can include a shoulder joint, a wrist joint and a palm at the end, wherein the distance between the shoulder joint and the wrist joint is L1, and the distance between the wrist joint and the palm is L2. Now the robotic arm is applied in an object throwing scenario to calculate the joint state and control force required by the robotic arm to throw a small ball at each moment.

[0066] First, according to the starting and ending states of the manipulator and the initial response time matrix T, the joint state x and control force u of the shoulder joint and wrist joint at each time t are determined. The initial response time matrix includes the movement type of the shoulder joint and wrist joint at each time; according to the movement type of the shoulder joint and wrist joint, the total cost function corresponding to the shoulder joint and wrist joint is determined; according to the total cost function corresponding to all joints, the joint state and control force at each time, the dynamic energy function of the manipulator is constructed; using the dynamic energy function of the manipulator, the initial response time matrix is ​​recalculated to obtain the updated response time matrix T*, and the updated response time matrix includes the new movement type of the shoulder joint and wrist joint at each time; based on the updated response time matrix, the new joint state and control force of the shoulder joint and wrist joint at each time are determined, and the new joint state and control force are used to control the movement of each joint in the system. For example, it can be determined that at a certain moment, the state of the shoulder joint at this time is x1, and the torque it generates is τ1, and the state of the wrist joint at this time is x1, and the torque it generates is τ2. Finally, according to the obtained joint states and control forces of all the joints, the joints in the system are controlled to move, for example, from the shoulder joint to the wrist joint in sequence, so that the palm at the end obtains a larger linear velocity, and finally the palm releases the object, so that the object is thrown.

[0067] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0068] In this embodiment, a data processing method is provided, such as Figure 1b As shown, the specific process of the data processing method can be as follows:

[0069] 110. According to the system's starting and ending states and the initial response time matrix, the joint state and control force of each joint in the system at each moment are determined. The initial response time matrix includes the movement type of each joint in the system at each moment.

[0070] Before generating the trajectory of the robot arm, the starting and ending states of each joint are input first to ensure that the constraints of the starting and ending points are clearly set during the trajectory generation process. This state information provides important boundary conditions for trajectory planning, allowing the generated trajectory to transition smoothly and accurately between the specified initial and final states.

[0071] Therefore, the starting and ending states of the system include the joint states of each joint in the system at the starting time t = 0 and the ending time t = tf, which can be recorded as x0 and x tf , or x(0) and x(tf).

[0072] Among them, the joint state may include the joint angle (ie, position), angular velocity (ie, speed), angular acceleration (ie, acceleration), force / torque, motion type, etc.

[0073] Among them, the types of movement can be divided into active movement and passive movement. Active movement refers to the movement of the joint driven by power, and passive movement refers to the passive movement of the joint along with other joints.

[0074] The initial response time matrix includes the motion type of each joint in the system at each moment. Each row of the initial response time matrix T represents a different joint (such as shoulder joint, elbow joint, wrist joint, etc.), each column represents a certain time period, and the value of each element represents the type switching time of the joint at that moment. The type switching time refers to the response time of the motion type switching. For example, it can be switched from the passive state to the active state in the first second, or it can be switched from the active state to the passive state in the third second.

[0075] In some embodiments, the initial response time matrix can optimize the amount of data by introducing time period division, so as to more effectively describe the switching of the motion type of each joint of the system within the time period. Therefore, the initial response time matrix may include the switching time of the motion type of each joint in the system within each time period, that is, each row in the matrix represents a node, and each column represents a time interval, for example, it may be a time interval of 0.5 seconds, and each element represents the switching time of the motion type. If the value is 0, it means that no switching of the motion type occurs within the time interval.

[0076] For example, referring to the following table, Table 1, we can see that in the first time period, the first node is at t 11 A type switch occurs, and the second node is at t 21 A type switch has occurred; if the value of the element is 0, it indicates that the state type of the node remains unchanged during this time period.

[0077]

[0078] Table 1

[0079] In some embodiments, in order to represent the motion of the human body, the optimal control rate may be used to calculate the joint trajectory, so step 110 includes:

[0080] Using the simulation motion equation, based on the start and end states and the initial response time matrix, the joint state and control force of the target joint in the system at the target time are determined;

[0081] If the motion type of the target joint at the target time is a passive type, the joint state of the target joint at the target time is corrected according to the compliance correction equation.

[0082] Among them, the use of the simulation motion equation can be both the trajectory generation process involved in step 110 of this solution and the trajectory optimization process involved in step 150, that is, the principles of step 110 and step 150 are similar.

[0083] Among them, the simulation motion equation is a mathematical model that describes the motion of each joint of the robotic arm, and the compliance correction equation is a mathematical model that describes how the passive joint produces a compliant response under the torque applied by the outside world or the active joint.

[0084] In some embodiments, the angular acceleration of each joint (such as shoulder, elbow, etc.) is jointly determined by the torque of the joint itself and the interaction torque between adjacent joints, so as to achieve coordinated overall motion. Therefore, the simulation motion equation can include a multi-joint coordination equation, which describes the interaction between multiple joints to simulate the motion and control of a manipulator or a robot system. Different from the motion equation of a single joint, the multi-joint coordination equation needs to consider the dynamic behavior of multiple joints at the same time, as well as their coupling in the whole system. Such a model is particularly important in the motion control of a manipulator, because the motion of multiple joints is usually coordinated, and the interaction between joints will affect the overall system performance.

[0085] For example, the multi-joint coordination equation is as follows:

[0086]

[0087] Among them, the multi-joint coordination equation is used to calculate the torque term of the single joint Calculate the control force term τ i (t,T) and multi-joint influence terms The single moment term includes the angular acceleration of the target joint i at the target time t and moment of inertia I i The product of the moment of inertia describes the inertia of the target joint during rotational motion; the control force term includes the control force τ of the target joint at the target time i (t,T); The multi-joint influence term includes the interaction torque of the target joint subjected to all other joints j at the target time, N is the total number of joints in the system. Among them, F ij It is the interaction torque between joint i and joint j, reflecting how the movement of one joint affects other joints through torque.

[0088] Among them, angular acceleration is the time derivative of angular velocity, which represents the rate of change of angular velocity with time, that is, the degree of acceleration or deceleration of rotation, and angular velocity is the time derivative of angle.

[0089] In some embodiments, in order to more accurately describe and optimize the motion of each joint of the robotic arm, the trajectory of the joint can be calculated in combination with the joint state and force conditions. Therefore, a control equation can be constructed and used as a simulation motion equation. For example, the control equation is as follows:

[0090]

[0091] x is the state of the joint, which can include angle θ, angular velocity Angular acceleration is the time derivative of x, representing the rate of change of x; u is the input external force, u max is the maximum external force input, x goal It is the final state reached by the joint at the final moment tf, that is, the final angle, angular velocity and angular acceleration.

[0092] Therefore, the governing equations are used to determine the derivatives of the joint states Calculate the sum of the weighted state term Ax and the weighted force term Bu; the weighted state term includes the weighted result of weighting the joint state x of the target joint at the target moment using the state law weight A, and the state law weight A describes the movement of the target joint when it is in a specific joint state; the weighted force term includes the weighted result of weighting the control force u of the target joint at the target moment using the force law weight B, and the force law weight B describes the movement of the target joint when it is subjected to a specific force.

[0093] In some embodiments, in order to achieve the compliance of the system (i.e., the response characteristics of the joint when subjected to external or internal forces), a compliance correction equation can be used to correct the obtained trajectory so that the trajectory conforms to the compliant response. The compliance correction equation can model the passive joint as an inertia-damping-spring system, which is used to describe how the passive joint produces a compliant response under the torque applied by the external or active joint. This equation focuses more on describing the dynamic characteristics of the passive joint under compliant control, rather than the dynamic coupling relationship of the entire system. This mechanism can prevent the robot arm from excessive rigid response when encountering external force impact, thereby helping the robot arm to move more smoothly under external interference.

[0094] Therefore, step 110 also includes:

[0095] The compliance correction equation is constructed as follows;

[0096]

[0097] where the compliance correction equation is used to calculate the reaction force term τ pr Calculating the inertia term Damping term With elastic term K p Δθp The inertia term includes the inertia weight B p Correction value of the angular acceleration of the target joint at the target time The inertia weight describes the inertial motion of the target joint; the damping term includes the damping weight D p Correction value of the angular velocity of the target joint at the target time The result of adding the addition, the damping weight describes the target damping motion; the elastic term includes the elastic weight K p The angle correction value Δθ of the target joint at the target time p The elastic weight describes the elastic recovery motion of the target joint.

[0098] Then, in some embodiments, the joint state of the target joint at the target time is corrected according to the compliance correction equation, including:

[0099] The angular acceleration of the target joint at the target time is corrected according to the angular acceleration correction value obtained by the compliance correction equation;

[0100] The angular velocity of the target joint at the target moment is corrected according to the angular velocity correction value obtained by the compliance correction equation;

[0101] The angle of the target joint at the target moment is corrected according to the angle correction value obtained by the compliance correction equation.

[0102] Therefore, when the passive joint of the robot arm (such as the wrist) is subjected to a sudden external force (such as contact with an object), the inertia term will allow the joint to naturally buffer the external force, the damping term will further slow down its speed, and the elastic term will generate a restoring force to try to restore the joint to the reference position.

[0103] There is interaction between active joints and passive joints in the system, that is, there is not only independent behavior of each joint in the system, but also dynamic coupling relationship between them. Therefore, in order to show flexibility similar to human joints and achieve more natural movement and buffering, in some embodiments, the simulation motion equation is used to determine the joint state and control force of each joint in the system at each moment based on the start and end state and initial response time matrix, including:

[0104] If the motion type of the target joint in the system at the target time is active, the active joint coupling function is used, combined with the simulation motion equation, based on the updated response time matrix to calculate the joint state and control force of the target joint at the target time;

[0105] If the motion type of the target joint in the system at the target time is passive, the passive joint coupling function is used, combined with the simulation motion equation, to calculate the joint state and control force of the target joint at the target time based on the updated response time matrix.

[0106] When using these coupling functions, the simulation motion equation is the core to help calculate the state and control force of the joint. Whether it is active or passive type, these coupling functions need to be combined with the simulation motion equation for calculation so that the simulation motion equation can model the state change of the joint according to the dynamic characteristics of the system (such as the inertia of the joint, external load, constraint, etc.).

[0107] Therefore, in some embodiments, the method further comprises:

[0108] The active joint coupling function is constructed as follows:

[0109]

[0110] Among them, the active joint coupling function is used to calculate the first input force τ a With the first contact force τ ac The first monomer inertia term is calculated by summing The first coupled inertia term The first centrifugal force data C a , the first gravity data G a The first monomer inertia term includes the first monomer inertia weight M a The angular acceleration of the target joint a at the target time The result of the addition is that the first monomer inertia weight describes the inertial motion of the target joint alone; the first coupled inertia term includes the first coupled inertia weight M ap Angular acceleration of other joints p at the target time The result of the addition is that the first coupling inertia weight describes the inertial movement of the target joint when it is affected by other joints; the first centrifugal force data weight describes the movement of the target joint when it is affected by centrifugal force, and the first gravity data describes the movement of the target joint when it is affected by gravity.

[0111] In some embodiments, the method further comprises:

[0112] Construct the passive joint coupling function as follows:

[0113]

[0114] Among them, the passive joint coupling function is used to calculate the second input force τ p , the second contact force τ pc The second monomer inertia term is calculated by summing The second coupled inertia term The second centrifugal force data C p , the second gravity data G p The second monomer inertia term includes the second monomer inertia weight M p The angular acceleration of the target joint p at the target time The result of the addition, the second monomer inertia weight M pa Describes the inertial motion of the target joint alone; the second coupled inertia term includes the second coupled inertia weight M pa Angular acceleration of other joints a at the target time The result of the addition is that the second coupling inertia weight describes the inertial movement of the target joint when it is affected by other joints; the second centrifugal force data weight describes the movement of the target joint when it is affected by centrifugal force, and the second gravity data describes the movement of the target joint when it is affected by gravity.

[0115] Among them, the contact force is the force that describes the contact between the external environment and the joint. For example, the contact torque can better simulate the joint response under the influence of external force.

[0116] 120. According to the movement type of the joint, determine the total cost function corresponding to the joint.

[0117] The total cost function is the sum of the costs required for the system to perform a specific operation or achieve a goal, including costs in multiple aspects. In some embodiments, the total cost function can be a weighted combination of multiple cost functions, for example, it can be a weighted combination of various cost functions such as time, energy, error, throwing distance, power, etc. This application introduces a total cost function to measure system performance and guide the optimization process.

[0118] In some embodiments, different cost functions may be selected for optimization calculations according to the motion type (active or passive) of the target joint at the target moment.

[0119] The total cost function of a joint can be defined around its unique motion characteristics, thereby optimizing system performance and reducing unnecessary resource consumption.

[0120] For example, in some embodiments, the total cost function of an active joint (a joint whose motion type is active motion) generally focuses on the consumption of control force, power, and energy. Since the active joint can be controlled by an external driving force, its total cost function may include kinetic energy dissipation, response time, and control errors during operation.

[0121] For example, in some embodiments, passive joints (joints with passive motion) are usually controlled by springs or dampers, and passive joints usually do not directly apply control forces. The cost function mainly focuses on the stability and natural response caused by passive forces. The total cost function can focus on factors such as joint position offset, vibration amplitude, and damping consumption.

[0122] In some embodiments, the total cost function is a weighted sum of various costs paid by the system or joint at each moment, that is, the immediate costs are accumulated to obtain the long-term cost.

[0123] Therefore, step 120 includes:

[0124] According to the motion type of the target joint at the target time, determine the cost function corresponding to the target joint at the target time;

[0125] According to the cost function corresponding to the target joint at all times, the total cost function corresponding to the target joint is obtained.

[0126] For example, when the motion type of the target joint at the target moment is active motion, the control goal is to make the joint perform the specified task (e.g., move to the target position) within the specified time. In this case, time is usually considered as the main cost factor, with the goal of minimizing the time required to perform the task and improving system efficiency.

[0127] For example, when the motion type of the target joint at the target moment is passive motion, the control of the system is mainly determined by the external environment or the natural dynamics of the system, and the control force is not directly applied by the system. At this time, the movement of the joint is usually driven by external force, elastic restoring force or other natural reaction force, so the control energy becomes the focus of optimization.

[0128] Therefore, in some embodiments, determining the cost function corresponding to the target joint at the target time according to the motion type of the target joint at the target time includes:

[0129] If the motion type of the target joint at the target time is active motion, then the cost function corresponding to the target joint at the target time is determined to be a time cost function, and the time cost function is used to calculate the time cost by integrating the time according to the target time;

[0130] If the motion type of the target joint at the target time is passive motion, the cost function corresponding to the target joint at the target time is determined to be an energy cost function, which is used to calculate the energy cost based on the integral of the square of the control force of the target joint at the target time at the target time.

[0131] For example, the time cost function can be Jt = ∫1dt, and the energy cost function can be Ju = ∫u 2 dt.

[0132] In some embodiments, the total cost function may also be:

[0133] J(kt,ku)=kt*Jt+ku*Ju

[0134] Among them, kt is the time weight, ku is the energy weight, Jt is the time function, and J is the energy function. By adjusting the weights, a balance is achieved between time efficiency and energy efficiency. For example, when one weight is large, the other weight is adjusted to be smaller.

[0135] In some embodiments, an optimization function can be introduced to evaluate the performance of the system when performing a specific task, and the goal is to minimize or maximize the value of the function, thereby improving the system performance. For example, in robotic arm control, the optimization function can be used to adjust the state and control force of the joint to ensure that the system can achieve the best balance between multiple goals. Therefore, the method also includes:

[0136] Get the optimization function and add it to the total cost function corresponding to the joint.

[0137] Among them, the optimization function can be used to measure various factors, such as energy consumption, time consumption, stability, accuracy, load, etc. By adjusting the control input (such as joint angle, speed, acceleration, etc.), the value of the optimization function can be minimized (or maximized) to achieve the desired motion effect.

[0138] For example, the optimization function may include a multi-joint total cost function, a throwing distance function, etc. Among them, the multi-joint total cost function is used to calculate the overall performance cost of the system according to the joint state of each joint at each moment; the throwing distance function is used to calculate the flying distance of the thrown object after being thrown by the throwing joint according to the joint state of the throwing joint in the system.

[0139] Among them, the multi-joint total cost function is as follows:

[0140]

[0141] The multi-joint total cost function can evaluate the performance of each joint i according to its state (such as torque, angle, angular velocity and angular acceleration). i , and then use the weight α i To adjust the influence of each joint on the whole, it is used to optimize the behavior of a single joint in a multi-joint system.

[0142] 130. The dynamic energy function of the system is constructed based on the total cost function corresponding to all joints, the joint state and control force at each moment.

[0143] The dynamic energy function is the sum of energy distribution and conversion within a system, including kinetic energy (determined by mass and velocity) and potential energy (related to position or height). It is used to reflect the dynamic characteristics of the system according to the changes in kinetic energy and potential energy at different positions or states. The dynamic energy function is used to describe the energy state of the system in physics, engineering, and control theory. In the control of multi-joint systems, the dynamic energy function can be composed of the system's kinetic energy, potential energy, and energy terms related to the control input.

[0144] In some embodiments, the kinetic energy function is used to analyze the motion of the system in the phase space. For example, it may be the sum of the kinetic energy of the system and the potential energy of the system, which is used to describe the energy conservation of the system and the kinetic characteristics of the phase space.

[0145] There are many kinds of dynamic energy functions, for example, Hamiltonian, Lagrangian, generalized force-momentum energy function, etc. Hamiltonian is a function that describes the total energy of the system. Specifically, Hamiltonian can be used to solve control optimization problems. For example, in the optimal control problem, the Hamiltonian is optimized to determine how to select the control input to minimize the cost function (such as energy, time, error, etc.).

[0146] Among them, the Hamiltonian of the system can be constructed based on the total cost function corresponding to all joints, the joint state and control force at each moment:

[0147] H(x,u,t,T)=L(x,u,t,)

[0148] Among them, T is the initial response time matrix, L is the total cost function, and the Hamiltonian includes the joint state x and control force u at each time t.

[0149] In some embodiments, the Hamiltonian of the system can be constructed:

[0150] H(x,u,Λ,t,T)=L(x,u,t,)+Λ T (t)G(x,u,t,T)

[0151] Among them, G is the set of physical constraints, Λ T (t) is the adjoint vector, which is the Lagrange multiplier for the equality and inequality constraints.

[0152] In some embodiments, a dynamic energy function of the system is constructed based on the total cost function corresponding to all joints, the joint states and control forces at each moment, including:

[0153] Obtaining a set of physical constraints of the system, the set of physical constraints including weighted physical constraints of each joint in the system, the weighted physical constraints including physical constraints weighted by constraint strength weights;

[0154] The system's dynamic energy function is constructed based on the set of physical constraints, the total cost function corresponding to all joints, the joint states and control forces at each moment.

[0155] In some embodiments, the physical constraints include strict constraints and weighted loose constraints, where the weighted loose constraints P j (g j (x,u)) includes the penalty weight P j Weighted loose constraint g j (x,u).

[0156] So we get the Hamiltonian:

[0157] Ha(T)=H+∑ j＝1 M P j (g j (x,u)

[0158] 140. The initial response time matrix is ​​recalculated using the system's dynamic energy function to obtain an updated response time matrix, which includes the new motion type of each joint in the system at each moment.

[0159] For example, when recalculating, the partial derivative of Hamiltonian H to the input external force u can be made zero, indicating that u has reached an extreme value, that is, u is in an optimal state. (H includes the cost function L and the system constraint G. In order to minimize the cost function L or maximize the benefit, H must be insensitive to changes in u, that is, H is optimal when u is at an extreme value.

[0160] 150. Based on the updated response time matrix, the new joint state and control force of each joint in the system at each moment are determined, and the new joint state and control force are used to control the movement of each joint in the system.

[0161] Similar to step 110, details are omitted here.

[0162] From the above, it can be seen that the embodiments of the present application can determine the joint state and control force of each joint in the system at each moment according to the start and end states of the system and the initial response time matrix, and the initial response time matrix includes the motion type of each joint in the system at each moment; determine the total cost function corresponding to the joint according to the motion type of the joint; construct the system's dynamic energy function according to the total cost function corresponding to all joints, the joint state and control force at each moment; use the system's dynamic energy function to recalculate the initial response time matrix to obtain an updated response time matrix, and the updated response time matrix includes the new motion type of each joint in the system at each moment; based on the updated response time matrix, determine the new joint state and control force of each joint in the system at each moment, and the new joint state and control force are used to control the motion of each joint in the system.

[0163] Therefore, this solution uses different types of dynamic energy functions to improve the accuracy, stability, efficiency and flexibility of the system, optimize the motion control and task execution effect of the multi-joint system, and provide strong support for the application of robot systems in complex tasks. Specifically, by using Hamiltonian, the dynamic characteristics of multi-degree-of-freedom mechanical systems can be accurately described, helping to optimize the system's motion trajectory, joint angles and other key information; introducing time cost function and energy cost function can help to optimize the execution time and energy consumption of tasks at the same time, achieve multi-objective optimization, and improve the overall performance of the system; the introduction of elastic terms enables the system to show flexibility in contact and interaction, adapt to changes in external forces, and improve its ability to work in collaboration with the environment. Thereby improving the effect of system movement.

[0164] The method described in the above embodiment will be further described in detail below.

[0165] In this embodiment, a robotic arm is taken as an example to describe the method of the embodiment of the present application in detail.

[0166] The specific algorithm flow of a data processing method is as follows:

[0167] Settings: Set the initial state x0 at t = 0, the final state xg at t = tf, the performance index L(x, t) (operation cost function, such as time cost, energy consumption, etc.), and the constraint condition set ({g j (x,u)} j＝1 M Defines a series of constraints that the robot state x and input external force u need to satisfy, M is the number of constraints);

[0168] Initialization: assign the initial state x0 to q0, assign the target state xg to qg, and initialize the response time matrix T;

[0169] The classification constraints are: key constraints (such as joint angle limits) and non-key constraints (constraints that allow small deviations, such as smooth motion, etc., which can allow a certain degree of violation, but require the application of a penalty function P j );

[0170] For each non-critical constraint g j Define the penalty function P j ;

[0171] Function EvaluateMotion(T)

[0172] Input: Response time matrix T

[0173] Initialization: Create an empty set Θ, Θ˙, Θ˙˙, T (used to store the angle θ, angular velocity θ˙, angular acceleration θ˙˙, and torque τ of all joints)

[0174] For each joint k, numbered from N to 1 (i.e. from the end joint to the root joint), do:

[0175] {Determine the state (active or passive) of joint k at each moment according to the response time matrix T;

[0176] Select the corresponding cost function according to the state of the joint (active or passive);

[0177] Calculate the control torque τ of joint k at each moment k (t,T), used to achieve the desired active or passive motion;

[0178] τ k (t,T) is added to the set T to record the torque configuration of joint k;

[0179] Calculate the angle θ k (t,T), angular velocity θ˙ k (t,T), angular acceleration And store it into the corresponding set Θ,Θ˙,Θ˙˙;

[0180] }end for

[0181] {Use the set Θ, Θ˙, Θ˙˙ and T to calculate the Hamiltonian H (dynamic energy function), which integrates the state, control input and cost of the system;

[0182] Calculate the augmented Hamiltonian Ha(T) (constraint dynamics energy function): by introducing a penalty function P (constraint strength weight) in H j , forming the augmented Hamiltonian Ha(T)=H+∑ j＝1 M P j (gj (x,u)), which is used for optimal control calculation after comprehensively considering non-critical constraint deviations.

[0183] }Return: Ha(T), Θ, Θ˙, Θ˙˙, and T;

[0184] By maximizing the augmented Hamiltonian Ha(T), the optimal response time matrix T* (i.e., the optimal setting of the switching timing of each joint) is found;

[0185] For each joint k, numbered from N to 1 (i.e. from the end joint to the root joint) do: {Based on the optimal response time matrix T*, generate the optimal motion trajectory q corresponding to the angle, angular velocity and angular acceleration for joint k k (t), q k (t)˙,q k (t)˙Formula 3;

[0186] Use T' to correct the passive trajectory according to formula 14;

[0187] Based on the optimal response time matrix T*, the optimal control torque τk(t) is calculated, active 12 and passive 13;

[0188] }end for

[0189] The optimal trajectory is achieved by using a combination of methods such as impedance control, admittance control or whole body dynamics control.

[0190] Based on this algorithm, this application carried out experiments including dynamic parabolic motion of the robot and standing experiments of the humanoid robot.

[0191] Here, we take the robot throwing an object as an example. This experiment is conducted in a vertical gravity field. Figure 2a , only joints 1 (shoulder), 23 (elbow), and 3 (wrist) of the robot arm are used to throw the object. The robot's proximal joints (shoulder and elbow) move in sequence to obtain a larger linear velocity, and the distal joint (wrist) moves last to throw the object to minimize the effect of this action on the acceleration of the proximal joints. Figure 2b ,When the distal joint completes the throwing task, multiple joints of the robotic arm are in a passive state to buffer the impact of dynamic motion on the joints.

[0192] Here, the dynamic standing motion of a humanoid robot is used as an example. This application selects a humanoid robot, which weighs about 80 kg and has a large load capacity. Therefore, an efficient motion trajectory generation algorithm is essential for achieving motion control tasks. Figure 2cThe driving torque of the hip joint and leg joint is limited to 200Nm and 800N respectively. Due to the large inertia of the upper body, the leg-hip link of the robot must form a composite optimal motion trajectory during the configuration adjustment of the lower body, otherwise the leg and hip joints of the robot will bear a large load.

[0193] This algorithm targets the above two situations, taking into account both simulation and efficiency. As can be seen from the above, the embodiment of the present application can improve the effect of system movement.

[0194] In order to better implement the above method, the embodiment of the present application also provides a data processing device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0195] For example, in this embodiment, the method of the embodiment of the present application will be described in detail by taking the specific integration of the data processing device in the robotic arm as an example.

[0196] For example, Figure 3 As shown, the data processing device may include an initialization unit 310, a cost unit 320, a dynamics unit 330, a recalculation unit 340, and a determination unit 350, as follows:

[0197] (i) Initial unit 310.

[0198] The initial unit 310 is used to determine the joint state and control force of each joint in the system at each moment according to the system's start and end states and the initial response time matrix, where the initial response time matrix includes the motion type of each joint in the system at each moment.

[0199] In some embodiments, the initialization unit 310 is used to:

[0200] Using the simulation motion equation, based on the start and end states and the initial response time matrix, the joint state and control force of the target joint in the system at the target time are determined;

[0201] If the motion type of the target joint at the target time is a passive type, the joint state of the target joint at the target time is corrected according to the compliance correction equation.

[0202] In some embodiments, the simulation equations of motion include control equations, and the initialization unit 310 is further configured to:

[0203] A control equation is constructed, and the control equation is used to calculate the sum of a weighted state term and a weighted force term according to a derivative of the joint state; the weighted state term includes a weighted result of weighting the joint state of the target joint at a target time using a state law weight, and the state law weight describes the movement of the target joint when it is in a specific joint state; the weighted force term includes a weighted result of weighting the control force of the target joint at the target time using a force law weight, and the force law weight describes the movement of the target joint when it is subjected to a specific force.

[0204] In some embodiments, the simulation motion equation also includes a multi-joint coordination equation, which is used to calculate the sum of the control force term and the multi-joint influence term based on the single torque term; the single torque term includes the product of the angular acceleration and the moment of inertia of the target joint at the target time, and the moment of inertia describes the inertia of the target joint during rotational motion; the control force term includes the control force of the target joint at the target time; the multi-joint influence term includes the interaction torque exerted on the target joint by all other joints at the target time.

[0205] In some embodiments, the joint state includes angle, angular velocity and angular acceleration, and the data processing device is further used for:

[0206] A compliance correction equation is constructed, and the compliance correction equation is used to calculate the sum of the inertia term, the damping term and the elastic term according to the reaction force term; the inertia term includes the addition result of adding the angular acceleration correction value of the target joint at the target time by the inertia weight, and the inertia weight describes the inertial motion of the target joint; the damping term includes the addition result of adding the angular velocity correction value of the target joint at the target time by the damping weight, and the damping weight describes the damped motion of the target joint; the elastic term includes the addition result of adding the angle correction value of the target joint at the target time by the elastic weight, and the elastic weight describes the elastic recovery motion of the target joint;

[0207] According to the compliance correction equation, the joint state of the target joint at the target time is corrected, including:

[0208] The angular acceleration of the target joint at the target time is corrected according to the angular acceleration correction value obtained by the compliance correction equation;

[0209] The angular velocity of the target joint at the target moment is corrected according to the angular velocity correction value obtained by the compliance correction equation;

[0210] The angle of the target joint at the target moment is corrected according to the angle correction value obtained by the compliance correction equation.

[0211] In some embodiments, the joint state includes angle, angular velocity and angular acceleration, and the joint state and control force of each joint in the system at each moment are determined based on the start and end states and the initial response time matrix using the simulation motion equation, including:

[0212] If the motion type of the target joint in the system at the target time is active, the active joint coupling function is used, combined with the simulation motion equation, based on the updated response time matrix to calculate the joint state and control force of the target joint at the target time;

[0213] If the motion type of the target joint in the system at the target time is passive, the passive joint coupling function is used, combined with the simulation motion equation, to calculate the joint state and control force of the target joint at the target time based on the updated response time matrix.

[0214] In some embodiments, the data processing apparatus is further configured to:

[0215] An active joint coupling function is constructed, and the active joint coupling function is used to calculate the sum of the first monomer inertia term, the first coupling inertia term, the first centrifugal force data, and the first gravity data according to the sum of the first input force and the first contact force; the first monomer inertia term includes the addition result of the first monomer inertia weight to the angular acceleration of the target joint at the target time, and the first monomer inertia weight describes the individual inertial motion of the target joint; the first coupling inertia term includes the addition result of the first coupling inertia weight to the angular acceleration of other joints at the target time, and the first coupling inertia weight describes the inertial motion of the target joint when it is affected by other joints; the first centrifugal force data describes the motion of the target joint when it is affected by the centrifugal force, and the first gravity data describes the motion of the target joint when it is affected by gravity.

[0216] In some embodiments, the data processing apparatus is further configured to:

[0217] A passive joint coupling function is constructed, and the passive joint coupling function is used to calculate the sum of the second monomer inertia term, the second coupling inertia term, the second centrifugal force data, and the second gravity data according to the sum of the second input force and the second contact force; the second monomer inertia term includes the addition result of the second monomer inertia weight to the angular acceleration of the target joint at the target time, and the second monomer inertia weight describes the individual inertial motion of the target joint; the second coupling inertia term includes the addition result of the second coupling inertia weight to the angular acceleration of other joints at the target time, and the second coupling inertia weight describes the inertial motion of the target joint when it is affected by other joints; the second centrifugal force data describes the motion of the target joint when it is affected by the centrifugal force, and the second gravity data describes the motion of the target joint when it is affected by gravity.

[0218] (ii) Cost unit 320.

[0219] The cost unit 320 is used to determine a total cost function corresponding to a joint according to the movement type of the joint.

[0220] In some embodiments, the cost unit is used to:

[0221] According to the motion type of the target joint at the target time, determine the cost function corresponding to the target joint at the target time;

[0222] According to the cost function corresponding to the target joint at all times, the total cost function corresponding to the target joint is obtained.

[0223] In some embodiments, determining a cost function corresponding to the target joint at the target time according to the motion type of the target joint at the target time includes:

[0224] If the motion type of the target joint at the target time is active motion, then the cost function corresponding to the target joint at the target time is determined to be a time cost function, and the time cost function is used to calculate the time cost by integrating the time according to the target time;

[0225] If the motion type of the target joint at the target time is passive motion, the cost function corresponding to the target joint at the target time is determined to be an energy cost function, which is used to calculate the energy cost based on the integral of the square of the control force of the target joint at the target time at the target time.

[0226] In some embodiments, the cost unit is also used to:

[0227] Obtain an optimization function, where the optimization objectives include a multi-joint total cost function and a throwing distance function; the multi-joint total cost function is used to calculate the overall performance cost of the system according to the joint states of each joint at each moment; the throwing distance function is used to calculate the flying distance of the thrown object after being thrown by the throwing joint according to the joint states of the throwing joints in the system;

[0228] Add the optimization function to the total cost function corresponding to the joint.

[0229] (iii) Dynamics unit 330.

[0230] The dynamics unit 330 is used to construct a dynamics energy function of the system according to the total cost function corresponding to all joints, the joint states and control forces at each moment.

[0231] Obtaining a set of physical constraints of the system, the set of physical constraints including weighted physical constraints of each joint in the system, the weighted physical constraints including physical constraints weighted by constraint strength weights;

[0232] The system's dynamic energy function is constructed based on the set of physical constraints, the total cost function corresponding to all joints, the joint states and control forces at each moment.

[0233] In some embodiments, the physical constraints include strict constraints and weighted loose constraints, and the weighted loose constraints include loose constraints weighted by penalty weights.

[0234] (iv) Recalculating unit 340.

[0235] The recalculation unit 340 is used to recalculate the initial response time matrix using the system's dynamic energy function to obtain an updated response time matrix, where the updated response time matrix includes new motion types of each joint in the system at each moment.

[0236] (V) Determine unit 350.

[0237] The determination unit 350 is used to determine the new joint state and control force of each joint in the system at each moment based on the updated response time matrix, and the new joint state and control force are used to control the movement of each joint in the system.

[0238] In specific implementation, each of the above units can be implemented as an independent entity, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of each of the above units can refer to the previous method embodiments, which will not be repeated here.

[0239] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function, and works together with other related parts to achieve the predetermined function, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0240] As can be seen from the above, the data processing device of this embodiment is composed of an initial unit, which is used to determine the joint state and control force of each joint in the system at each moment according to the start and end state of the system and the initial response time matrix, and the initial response time matrix includes the motion type of each joint in the system at each moment; the cost unit determines the total cost function corresponding to the joint according to the motion type of the joint; the dynamics unit is used to construct the dynamics energy function of the system according to the total cost function corresponding to all joints, the joint state and control force at each moment; the recalculation unit uses the dynamics energy function of the system to recalculate the initial response time matrix using the dynamics energy function of the system to obtain an updated response time matrix, and the updated response time matrix includes the new motion type of each joint in the system at each moment; the determination unit determines the new joint state and control force of each joint in the system at each moment based on the updated response time matrix, and the new joint state and control force are used to control the motion of each joint in the system. Thus, the embodiment of the present application can improve the effect of system motion.

[0241] An embodiment of the present application also provides an electronic device, which may be a robotic arm, a robot, or other automated equipment including joints.

[0242] In this embodiment, the electronic device of this embodiment is a robotic arm as an example for detailed description, for example, Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0243] The electronic device may include one or more processing core processors 410, one or more computer-readable storage media memories 420, a power supply 430, an input module 440, a communication module 450, a joint element 460, and a feedback element 470, etc. It will be understood by those skilled in the art that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0244] The processor 410 is the control center of the electronic device. It uses various interfaces and lines to connect each part of the entire electronic device. By running or executing software programs and / or modules stored in the memory 420, and calling data stored in the memory 420, it executes each function of the electronic device and processes data, thereby performing overall detection of the electronic device. In some embodiments, the processor 410 may include one or more processing cores; in some embodiments, the processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 410.

[0245] The memory 420 can be used to store software programs and modules. The processor 410 executes each functional application and data processing by running the software programs and modules stored in the memory 420. The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 420 may also include a memory controller to provide the processor 410 with access to the memory 420.

[0246] The electronic device also includes a power supply 430 for supplying power to each component. In some embodiments, the power supply 430 can be logically connected to the processor 410 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 430 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0247] The electronic device may further include an input module 440, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0248] The electronic device may further include a communication module 450. In some embodiments, the communication module 450 may include a wireless module. The electronic device may perform short-range wireless transmission through the wireless module of the communication module 450, thereby providing the user with wireless broadband Internet access. For example, the communication module 450 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0249] The electronic device may include a plurality of joint elements 460 for driving the motion of the mechanical arm. Each joint element includes a control unit (CE), a parallel elastic element (PEE) and other execution components. Specifically, the control unit (CE) is used to receive a control signal from the processor 410 and output a corresponding drive instruction to achieve precise motion control of the joint. The parallel elastic element (PEE) is used to absorb and release the energy generated when the joint is passively stretched or subjected to external force, so as to reduce the mechanical stress of the system and enhance the flexibility and impact resistance of the joint.

[0250] In addition, in some embodiments, each joint element may also include an additional feedback element 470, such as a torque sensor or a position encoder, for collecting information of each joint element, such as real-time position, speed, angle, angular velocity, angular acceleration, force conditions, etc., in order to update the response time matrix and optimize the control strategy, so that the joint movement can be dynamically adjusted according to the movement requirements of the simulated person, thereby improving the response accuracy and flexible control capabilities of the system.

[0251] Those skilled in the art will appreciate that, in some embodiments, the component design in the joint element may be adjusted according to the specific task requirements of the robotic arm, by adding vibration suppression units, damping control elements, etc., to adapt to more complex trajectories and load environments, thereby achieving more efficient and more sophisticated motion control effects.

[0252] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 410 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 420 according to the following instructions, and the processor 410 will run the application programs stored in the memory 420, thereby realizing each function, as follows:

[0253] Determine the joint state and control force of each joint in the system at each moment according to the system's start and end states and the initial response time matrix, where the initial response time matrix includes the motion type of each joint in the system at each moment;

[0254] According to the movement type of the joint, determine the total cost function corresponding to the joint;

[0255] Construct the system's dynamic energy function based on the total cost function corresponding to all joints, the joint states and control forces at each moment;

[0256] The initial response time matrix is ​​recalculated using the system's dynamic energy function to obtain an updated response time matrix, which includes the new motion types of each joint in the system at each moment;

[0257] Based on the updated response time matrix, the new joint state and control force of each joint in the system at each moment are determined, and the new joint state and control force are used to control the movement of each joint in the system.

[0258] The specific implementation of each of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0259] It can be seen from the above that the embodiments of the present application can improve the effect of system movement.

[0260] A person skilled in the art will appreciate that all or part of the steps in each method of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0261] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any data processing method provided in the embodiment of the present application. For example, the instructions can execute the following steps:

[0262] Determine the joint state and control force of each joint in the system at each moment according to the system's start and end states and the initial response time matrix, where the initial response time matrix includes the motion type of each joint in the system at each moment;

[0263] According to the movement type of the joint, determine the total cost function corresponding to the joint;

[0264] Construct the system's dynamic energy function based on the total cost function corresponding to all joints, the joint states and control forces at each moment;

[0265] The initial response time matrix is ​​recalculated using the system's dynamic energy function to obtain an updated response time matrix, which includes the new motion types of each joint in the system at each moment;

[0266] Based on the updated response time matrix, the new joint state and control force of each joint in the system at each moment are determined, and the new joint state and control force are used to control the movement of each joint in the system.

[0267] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0268] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method provided in each optional implementation of the trajectory generation aspect or motion control aspect provided in the above embodiments.

[0269] Since the instructions stored in the storage medium can execute the steps in any data processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any data processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0270] The above is a detailed introduction to a data processing method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: include: Determine the joint state and control force of each joint in the system at each moment according to the start and end states of the system and an initial response time matrix, wherein the initial response time matrix includes the motion type of each joint in the system at each moment; Determining a total cost function corresponding to the joint according to the motion type of the joint; Constructing a dynamic energy function of the system according to the total cost function corresponding to all the joints, the joint states and the control forces at each moment; The initial response time matrix is ​​recalculated using the dynamic energy function of the system to obtain an updated response time matrix, wherein the updated response time matrix includes a new motion type of each joint in the system at each moment; Based on the updated response time matrix, a new joint state and control force of each joint in the system at each moment are determined, and the new joint state and control force are used to control the movement of each joint in the system.

2. The data processing method according to claim 1, characterized in that: Determining the joint state and control force of each joint in the system at each moment according to the start and end states of the system and the initial response time matrix includes: Using a simulation motion equation, based on the start and end states and the initial response time matrix, determining the joint state and control force of the target joint in the system at the target time; If the motion type of the target joint at the target time is a passive type, the joint state of the target joint at the target time is corrected according to the compliance correction equation.

3. The data processing method according to claim 2, characterized in that: The simulation motion equation also includes a multi-joint coordination equation, which is used to calculate the sum of the control force term and the multi-joint influence term based on the single torque term; the single torque term includes the product of the angular acceleration and the moment of inertia of the target joint at the target time, and the moment of inertia describes the inertia of the target joint during rotational motion; the control force term includes the control force of the target joint at the target time; the multi-joint influence term includes the interaction torque exerted on the target joint by all other joints at the target time.

4. The data processing method according to claim 2, characterized in that: The simulation motion equation includes a control equation, and the method further includes: A control equation is constructed, wherein the control equation is used to calculate the sum of a weighted state term and a weighted force term based on a derivative of a joint state; the weighted state term includes a weighted result of weighting the joint state of the target joint at a target moment using a state law weight, and the state law weight describes the movement of the target joint when it is in a specific joint state; the weighted force term includes a weighted result of weighting the control force of the target joint at the target moment using a force law weight, and the force law weight describes the movement of the target joint when it is subjected to a specific force.

5. The data processing method according to claim 2, characterized in that: The joint state includes angle, angular velocity and angular acceleration, and the method further includes: Constructing a compliance correction equation, the compliance correction equation is used to calculate the sum of an inertia term, a damping term and an elastic term according to a reaction force term; the inertia term includes an addition result of adding an angular acceleration correction value of the target joint at a target time by an inertia weight, and the inertia weight describes the inertial motion of the target joint; the damping term includes an addition result of adding an angular velocity correction value of the target joint at a target time by a damping weight, and the damping weight describes the damped motion of the target joint; the elastic term includes an addition result of adding an angle correction value of the target joint at a target time by an elastic weight, and the elastic weight describes the elastic recovery motion of the target joint; The step of correcting the joint state of the target joint at the target time according to the compliance correction equation includes: Correcting the angular acceleration of the target joint at the target time according to the angular acceleration correction value obtained by the compliance correction equation; Correcting the angular velocity of the target joint at the target time according to the angular velocity correction value obtained by the compliance correction equation; The angle of the target joint at the target time is corrected according to the angle correction value obtained by the compliance correction equation.

6. The data processing method according to claim 2, characterized in that: The joint state includes angle, angular velocity and angular acceleration. The simulation motion equation is used to determine the joint state and control force of each joint in the system at each moment based on the start and end states and the initial response time matrix, including: If the motion type of the target joint in the system at the target time is an active type, the active joint coupling function is used in combination with the simulation motion equation to calculate the joint state and control force of the target joint at the target time based on the update response time matrix; If the motion type of the target joint in the system at the target time is a passive type, the passive joint coupling function is used in combination with the simulation motion equation to calculate the joint state and control force of the target joint at the target time based on the update response time matrix.

7. The data processing method according to claim 6, characterized in that: The method further comprises: An active joint coupling function is constructed, wherein the active joint coupling function is used to calculate the sum of a first monomer inertia term, a first coupling inertia term, a first centrifugal force data, and a first gravity data according to the sum of a first input force and a first contact force; the first monomer inertia term includes a result of adding a first monomer inertia weight to an angular acceleration of the target joint at a target time, and the first monomer inertia weight describes the individual inertial motion of the target joint; the first coupling inertia term includes a result of adding a first coupling inertia weight to the angular acceleration of other joints at the target time, and the first coupling inertia weight describes the inertial motion of the target joint when affected by other joints; the first centrifugal force data describes the motion of the target joint when affected by centrifugal force, and the first gravity data describes the motion of the target joint when affected by gravity.

8. The data processing method according to claim 6, characterized in that: The method further comprises: A passive joint coupling function is constructed, wherein the passive joint coupling function is used to calculate the sum of a second monomer inertia term, a second coupling inertia term, a second centrifugal force data, and a second gravity data according to the sum of a second input force and a second contact force; the second monomer inertia term includes a result of adding a second monomer inertia weight to an angular acceleration of the target joint at a target time, and the second monomer inertia weight describes the individual inertial motion of the target joint; the second coupling inertia term includes a result of adding a second coupling inertia weight to the angular acceleration of other joints at the target time, and the second coupling inertia weight describes the inertial motion of the target joint when affected by other joints; the second centrifugal force data describes the motion of the target joint when affected by centrifugal force, and the second gravity data describes the motion of the target joint when affected by gravity.

9. The data processing method according to claim 1, characterized in that: Determining the total cost function corresponding to the joint according to the movement type of the joint includes: Determining a cost function corresponding to the target joint at the target time according to the motion type of the target joint at the target time; According to the cost functions corresponding to the target joint at all times, a total cost function corresponding to the target joint is obtained.

10. The data processing method according to claim 9, characterized in that: Determining the cost function corresponding to the target joint at the target time according to the motion type of the target joint at the target time includes: If the motion type of the target joint at the target time is active motion, determining that the cost function corresponding to the target joint at the target time is a time cost function, the time cost function is used to calculate the time cost by integrating the time according to the target time; If the movement type of the target joint at the target time is passive movement, then the cost function corresponding to the target joint at the target time is determined to be an energy cost function, and the energy cost function is used to calculate the energy cost based on the integral of the square of the control force of the target joint at the target time at the target time.

11. The data processing method according to claim 1, characterized in that: The method of constructing the dynamic energy function of the system according to the total cost function corresponding to all the joints, the joint states and the control forces at each moment, includes: Acquire a set of physical constraints of the system, wherein the set of physical constraints includes weighted physical constraints of each joint in the system, and the weighted physical constraints include physical constraints weighted by constraint strength weights; The dynamic energy function of the system is constructed according to the set of physical constraint conditions, the total cost function corresponding to all the joints, the joint states and the control forces at each moment.

12. The data processing method according to claim 11, characterized in that: The physical constraint conditions include strict constraint conditions and weighted loose constraint conditions, and the weighted loose constraint conditions include loose constraint conditions weighted by penalty weights.

13. A data processing device, characterized in that: include: An initial unit, used to determine the joint state and control force of each joint in the system at each moment according to the starting and ending states of the system and an initial response time matrix, wherein the initial response time matrix includes the motion type of each joint in the system at each moment; A cost unit, used for determining a total cost function corresponding to the joint according to the motion type of the joint; A dynamics unit, used to construct a dynamics energy function of the system according to the total cost function corresponding to all the joints, the joint states and the control forces at each moment; A recalculation unit, configured to recalculate the initial response time matrix using a dynamic energy function of the system to obtain an updated response time matrix, wherein the updated response time matrix includes a new motion type of each joint in the system at each moment; A determination unit is used to determine a new joint state and control force of each joint in the system at each moment based on the updated response time matrix, and the new joint state and control force are used to control the movement of each joint in the system.

14. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps in the data processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the data processing method according to any one of claims 1 to 12.

Images