Robotic arm motion control methods, devices, equipment, robots and media

Abstract

This disclosure provides a method, apparatus, device, robot, and medium for motion control of a robotic arm, relating to the fields of sensor and robotics technology. The method includes: predicting the singular value decomposition (SVD) result of the Jacobian matrix using a deep learning model to obtain a set of direction vectors and corresponding singular value sequences; dividing the direction vectors into two subsets corresponding to non-singular directions and near-singular directions, respectively, based on the singular value sequences and a sensitivity threshold; decomposing the input vector of the desired velocity onto the two subsets to obtain a first direction component and a second direction component; performing scaling processing on the second direction component; and calculating the velocity of each joint in the joint space based on the first direction component and the scaled second direction component. This disclosure enables stable and precise motion control of the robotic arm when it approaches a singular configuration, and reduces computational complexity while ensuring control effectiveness, thereby reducing resource consumption and improving computational efficiency.

Description

Technical Field

[0001] This disclosure relates to the fields of sensor and robotics technology, and to a method for controlling the motion of a robot arm, a device for controlling the motion of a robot arm, an electronic device, a robot, and a computer-readable storage medium. Background Technology

[0002] When performing high-precision tasks, the end effector of a robotic arm typically needs to achieve continuous, smooth, and stable motion control in three-dimensional space. Such control usually relies on the Jacobian matrix to characterize the mapping relationship between joint space and task space.

[0003] However, when the robotic arm operates in a singular configuration, the Jacobian matrix may exhibit rank deficiency, meaning that some singular values ​​of the Jacobian matrix approach or even become zero, leading to the Jacobian matrix being non-invertible or having an excessively large condition number. In this case, the joint velocity solutions obtained by inverting the Jacobian matrix will lose stability, easily causing drastic changes or even divergence in joint velocities, resulting in uncontrolled movement of the end effector, and in severe cases, potentially endangering operational safety or damaging equipment.

[0004] To avoid the aforementioned problems, traditional methods such as damped least squares (DLS) are widely used. This method introduces a damping factor when calculating the generalized inverse time. When the singular value is below a set threshold, the damping factor is used to suppress its dominant effect on the global solution, thereby suppressing numerical divergence.

[0005] While the DLS method is effective in handling singular problems, its "global damping" mechanism can impair the system's resolution and sensitivity in non-singular directions, leading to a decrease in overall control accuracy. This is especially true in task space regions far from singular points, where unnecessary obstruction can affect the robot arm's dynamic performance and operational range. Furthermore, the DLS method involves high computational complexity in matrix decomposition and generalized inverse solving, which can easily result in excessive resource consumption and decreased computational efficiency.

[0006] Therefore, there is an urgent need to provide a new motion control scheme for robotic arms that can not only maintain stability and accuracy when approaching unusual configurations, but also reduce computational complexity. Summary of the Invention

[0007] This disclosure provides a method, apparatus, device, robot, and medium for controlling the motion of a robot arm, which can achieve stable and precise motion control when the robot arm approaches a singular configuration, and reduce computational complexity while ensuring control effect, thereby reducing resource consumption and improving computational efficiency.

[0008] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure.

[0009] According to a first aspect of this disclosure, a method for controlling the motion of a robot arm is provided, applied to the main controller of a robot, the method comprising:

[0010] By using a deep learning model to predict the singular value decomposition results of the original Jacobian matrix representing the mapping relationship between the joint space and the task space of the robot arm, a set of direction vectors and a sequence of singular values ​​associated with each direction vector in the set of direction vectors are obtained.

[0011] Based on the singular value sequence and sensitivity threshold, determine the first subset of the direction vector set corresponding to non-singular directions and the second subset corresponding to near-singular directions;

[0012] The input vector representing the desired velocity in the task space is decomposed on the first subset and the second subset respectively to obtain the first directional component and the second directional component;

[0013] The second direction component is scaled, and the velocity of each joint in the joint space is obtained based on the first direction component and the scaled second direction component.

[0014] In one exemplary embodiment of this disclosure, predicting the singular value decomposition result of the original Jacobian matrix representing the mapping relationship between the robot arm joint space and the task space using a deep learning model includes:

[0015] Obtain joint angle data representing the current joint configuration of the robot arm;

[0016] The joint angle data is input into a pre-trained deep learning model to obtain the singular value decomposition result of the original Jacobian matrix corresponding to the current joint configuration.

[0017] In one exemplary embodiment of this disclosure, the deep learning model includes:

[0018] The input layer is used to receive joint angle data;

[0019] A shared feature extraction layer is used to extract features from joint angle data to obtain joint feature vectors;

[0020] A multi-branch output layer is used to predict the singular value decomposition results of the original Jacobian matrix based on the joint feature vectors.

[0021] In one exemplary embodiment of this disclosure, the predicted singular value decomposition result includes the singular direction probability corresponding to each direction vector in the direction vector set and the singular value sequence associated with the direction vector set;

[0022] The multi-branch output layer includes:

[0023] The classification output layer is used to obtain the singular orientation probability corresponding to each orientation vector in the orientation vector set based on the joint feature vector;

[0024] The regression output layer is used to obtain a sequence of singular values ​​associated with the set of direction vectors based on the joint feature vectors.

[0025] In one exemplary embodiment of this disclosure, the method further includes:

[0026] Acquire joint angle data representing various joint configurations of the robot arm;

[0027] Calculate the true Jacobian matrix corresponding to each joint configuration, and perform singular value decomposition on each true Jacobian matrix to obtain the set of direction vectors and the sequence of true singular values ​​associated with each direction vector in the set of direction vectors.

[0028] Based on the teacher-student model framework, a deep learning model is trained using a set of direction vectors and a sequence of true singular values ​​associated with each direction vector in the set.

[0029] In one exemplary embodiment of this disclosure, the joint angle data includes raw joint angle data and target joint angle data;

[0030] Obtain joint angle data representing various joint configurations of the robotic arm, including:

[0031] Raw joint angle data for various joint configurations were obtained by sampling within the joint constraints of the robotic arm;

[0032] Data augmentation is performed on the original joint angle data to obtain the target joint angle data.

[0033] In one exemplary embodiment of this disclosure, data augmentation is performed on each raw joint angle data to obtain target joint angle data, including:

[0034] Gaussian noise is added to each of the original joint angle data to obtain the target joint angle data.

[0035] In one exemplary embodiment of this disclosure, data augmentation is performed on each raw joint angle data to obtain target joint angle data, including:

[0036] Inverse kinematics is performed on the continuous trajectory of the robot arm in the task space to obtain the sequence of adjacent joint angles;

[0037] By combining adjacent joint angle sequences with the original joint angle data, the target joint angle data is obtained.

[0038] In one exemplary embodiment of this disclosure, training a deep learning model using a set of direction vectors and a sequence of true singular values ​​associated with each direction vector in the set includes:

[0039] A loss function is constructed based on the set of direction vectors and the sequence of true singular values ​​associated with each direction vector in the set;

[0040] The loss function is used to iteratively train the deep learning model.

[0041] In one exemplary embodiment of this disclosure, a loss function is constructed based on a set of direction vectors and a sequence of true singular values ​​associated with each direction vector in the set of direction vectors, including:

[0042] Based on the true and predicted values ​​of each direction vector in the direction vector set, a classification loss for singular direction prediction is constructed.

[0043] Based on the true singular value sequence and the predicted singular value sequence associated with each directional vector, a regression loss for singular value prediction is constructed.

[0044] Based on the predicted singular value sequence associated with each directional vector, the reconstructed Jacobian matrix is ​​obtained, and the reconstruction loss of the Jacobian matrix is ​​constructed based on the reconstructed Jacobian matrix and the true Jacobian matrix.

[0045] A loss function is constructed based on the classification loss from singular orientation prediction, the regression loss from singular value prediction, and the reconstruction loss from the Jacobian matrix.

[0046] In one exemplary embodiment of this disclosure, the loss function is:

[0047]

[0048] in, L For loss function, As the first weight, It is a true sequence of singular values. To predict singular value sequences, The regression loss for singular value prediction is represented by the L2 norm. and The differences between them; As the second weight, The classification loss is for predicting singular orientations. For the predicted first j The probability that a direction belongs to a near-singular direction, i.e., the predicted value of each direction vector. For the indicator function, i.e. the true value of each direction vector, when the... j A singular value Less than The value is 1 if it is true, and 0 otherwise. The sensitivity threshold, The largest singular value in the true singular value sequence. Let m be the cross-entropy function, and m be the number of singular values. As the third weight, The reconstruction loss is the Jacobian matrix. For the first i Joint configuration The calculated true Jacobian matrix, To reconstruct the Jacobian matrix, and The first i The task space matrix and joint space matrix are obtained by decomposing the real Jacobian matrix. To predict the singular value matrix formed by the singular values, denoted by Frobenius norm, used to measure the difference between the reconstructed Jacobian matrix and the true Jacobian matrix.

[0049] In one exemplary embodiment of this disclosure, obtaining the velocity of each joint in the joint space based on the first direction component and the scaled second direction component includes:

[0050] The singular values ​​corresponding to each directional vector in the second subset are boosted to obtain the adjusted singular value matrix.

[0051] A safe Jacobian matrix is ​​constructed based on the task space matrix, joint space matrix, and adjusted singular value matrix obtained by decomposing the original Jacobian matrix.

[0052] Based on the generalized inverse of the safe Jacobian matrix, the velocity of each joint is calculated by combining the first directional component and the scaled second directional component.

[0053] In one exemplary embodiment of this disclosure, the singular values ​​corresponding to each direction vector in the second subset are boosted to obtain an adjusted singular value matrix, including:

[0054] The singular values ​​corresponding to each direction vector in the second subset are raised to a singular value threshold; wherein the singular value threshold is determined based on a sensitivity threshold.

[0055] The adjusted singular value matrix is ​​obtained based on the singular values ​​corresponding to each directional vector in the first subset and the singular value thresholds corresponding to each directional vector in the second subset.

[0056] In one exemplary embodiment of this disclosure, the velocity of each joint is calculated based on the generalized inverse of the secure Jacobian matrix, combined with the first directional component and the scaled second directional component, including:

[0057] Calculate the corrected velocity in the task space based on the first directional component and the scaled second directional component;

[0058] The velocities of each joint are calculated based on the generalized inverse of the safety Jacobian matrix and the corrected velocity.

[0059] In one exemplary embodiment of this disclosure, calculating the corrected velocity in the task space based on the first directional component and the scaled second directional component includes:

[0060]

[0061] in, To correct the speed, For the first direction component, This is the scaled second direction component.

[0062] In one exemplary embodiment of this disclosure, the velocity of each joint is calculated based on the generalized inverse of the safety Jacobian matrix and the corrected velocity, including:

[0063]

[0064] in, For the speed of each joint, For a safe Jacobian matrix, For the generalized inverse of the safe Jacobian matrix, To correct the speed.

[0065] In one exemplary embodiment of this disclosure, scaling processing is performed on the second direction component, including:

[0066] The scaling factor is calculated based on the ratio between the singular values ​​of each directional vector in the second subset and the singular value threshold; where the singular value threshold is determined based on the sensitivity threshold.

[0067] The second direction component is scaled according to the scaling factor to obtain the scaled second direction component.

[0068] In one exemplary embodiment of this disclosure, scaling processing is performed on the second direction component according to a scaling factor to obtain the scaled second direction component, including:

[0069] Determine the scaling factor corresponding to each direction vector in the second subset, and construct a scaling matrix based on each scaling factor;

[0070] Based on the scaling matrix and the second direction component, the scaled second direction component is obtained.

[0071] In one exemplary embodiment of this disclosure, obtaining the scaled second direction component based on the scaling matrix and the second direction component includes:

[0072] The scaled second direction component is calculated using the scaling matrix, the second direction component, and the gain matrix.

[0073] In one exemplary embodiment of this disclosure, the scaled second direction component is calculated using a scaling matrix, a second direction component, and a gain matrix, including:

[0074]

[0075] in, This is the scaled second-direction component. This is the second direction component before scaling. The direction vector in the second subset. This is the transpose of the direction vectors in the second subset. Let be the input vector representing the desired velocity in the task space. For scaling matrices, This is the gain matrix, used to adjust the control strength of each direction vector in the second subset.

[0076] In one exemplary embodiment of this disclosure, the method further includes:

[0077] When the singular values ​​corresponding to each directional vector in the second subset are greater than or equal to the singular value threshold, the second directional component is retained.

[0078] In one exemplary embodiment of this disclosure, determining a first subset of the set of direction vectors corresponding to non-singular directions and a second subset corresponding to near-singular directions, based on a sequence of singular values ​​and a sensitivity threshold, includes:

[0079] Discriminant conditions are constructed based on sensitivity thresholds;

[0080] Based on the singular values ​​in the singular value sequence that do not meet the discrimination criteria, determine the first subset of the direction vector set corresponding to non-singular directions;

[0081] Based on the singular values ​​in the singular value sequence that satisfy the discrimination condition, determine the second subset of the direction vector set corresponding to the near-singular direction.

[0082] In one exemplary embodiment of this disclosure, a discrimination condition is constructed based on a sensitivity threshold, including:

[0083] Determine the largest singular value in a sequence of singular values;

[0084] The singular value threshold is determined based on the maximum singular value and the sensitivity threshold, and the discrimination condition is constructed using the singular value threshold.

[0085] In one exemplary embodiment of this disclosure, determining the singularity threshold based on the maximum singularity and a sensitivity threshold includes:

[0086] Obtain the adjustment factor; where the adjustment factor is calculated based on the singular orientation probability obtained by inputting the joint angle data representing the current joint configuration of the robot arm into the deep learning model;

[0087] The singular value threshold is determined based on the adjustment factor, the maximum singular value, and the sensitivity threshold, and the discrimination criteria are constructed using the singular value threshold.

[0088] In one exemplary embodiment of this disclosure, the discrimination criteria include:

[0089]

[0090] in, κ is the singular value threshold, and κ is the adjustment factor. The sensitivity threshold, , The largest singular value in the singular value sequence. The first singular value in the sequence i A singular value.

[0091] In one exemplary embodiment of this disclosure, the input vector representing the desired velocity in the task space is decomposed on a first subset and a second subset respectively to obtain a first directional component and a second directional component, including:

[0092] Construct the projection Jacobian matrix based on the first subset, and decompose the input vector to obtain the first directional component based on the projection Jacobian matrix;

[0093] Extract the target direction vector corresponding to the task space from the second subset;

[0094] The input vector is decomposed along the target direction vector to obtain the second direction component.

[0095] In one exemplary embodiment of this disclosure, the first directional component is:

[0096]

[0097] in, For the first direction component, For the projected Jacobian matrix, For the generalized inverse of the projected Jacobian matrix, The input vector represents the desired velocity in the task space.

[0098] In one exemplary embodiment of this disclosure, constructing a projected Jacobian matrix based on a first subset includes:

[0099] Remove the direction vectors corresponding to the second subset from the task space matrix obtained by decomposing the original Jacobian matrix, and retain the direction vectors corresponding to the first subset;

[0100] Remove the singular values ​​corresponding to the direction vectors in the second subset from the singular value matrix obtained by decomposing the original Jacobian matrix, and retain the singular values ​​corresponding to the direction vectors in the first subset;

[0101] Based on the joint space matrix obtained by decomposing the original Jacobian matrix, the direction vectors in the retained first subset, and the singular values ​​corresponding to the direction vectors in the first subset, the projected Jacobian matrix is ​​constructed.

[0102] According to a second aspect of this disclosure, a motion control method for a robotic arm is applied to a joint actuator of a robot, the method comprising:

[0103] Receive the velocity of each joint in the joint space of the robot arm, and drive the corresponding joint to move according to the velocity of each joint;

[0104] The velocities of each joint in the joint space of the robot arm are obtained according to the robot arm motion control method of the first aspect of this disclosure.

[0105] According to a third aspect of this disclosure, a robot arm motion control device is provided, applied to the main controller of a robot, the device comprising:

[0106] The Singular Value Decomposition (SVD) module is used to perform singular value decomposition on the Jacobian matrix representing the mapping relationship between the joint space and the task space, to obtain a set of direction vectors and a sequence of singular values ​​associated with each direction vector in the set of direction vectors.

[0107] The singular direction determination module is used to determine, based on the singular value sequence and the sensitivity threshold, the first subset of the direction vector set corresponding to non-singular directions and the second subset corresponding to near-singular directions.

[0108] The desired velocity decomposition module is used to decompose the input vector representing the desired velocity in the task space on the first subset and the second subset respectively to obtain the first directional component and the second directional component.

[0109] The joint velocity generation module is used to perform scaling processing on the second direction component and obtain the velocity of each joint in the joint space based on the first direction component and the scaled second direction component.

[0110] According to a fourth aspect of this disclosure, a motion control device for a robot arm is provided, applied to a joint actuator of a robot, the device comprising:

[0111] The joint motion module is used to receive the velocity of each joint in the joint space of the robot arm and drive the corresponding joint motion according to the velocity of each joint.

[0112] The velocities of each joint in the joint space of the robot arm are obtained according to the robot arm motion control method of the first aspect of this disclosure.

[0113] According to a fifth aspect of this disclosure, an electronic device is provided, comprising:

[0114] Processor; and

[0115] A memory that stores computer-readable instructions, which, when executed by a processor, implement the method as described in the above embodiments.

[0116] According to a sixth aspect of this disclosure, a robot is provided, comprising:

[0117] Processor; and

[0118] A memory that stores computer-readable instructions, which, when executed by a processor, implement the method as described in the above embodiments.

[0119] In one exemplary embodiment of this disclosure, the robot includes either a humanoid robot or a dual-arm robot.

[0120] According to a seventh aspect of this disclosure, a computer-readable storage medium is provided that stores computer program code instructions, which, when invoked by a robot's processor, cause the robot to perform the method as described in the above embodiments.

[0121] As can be seen from the above technical solution, this disclosure possesses at least one of the following advantages and positive effects:

[0122] This disclosure provides a motion control method for a robotic arm. By using a deep learning model to predict the singular value decomposition results of the original Jacobian matrix, it can directly obtain the set of direction vectors and the sequence of singular values ​​corresponding to the current joint configuration without performing complex matrix operations. Therefore, compared to traditional methods that rely on real-time singular value decomposition, this method can significantly shorten the computation path, reduce computational complexity, thereby reducing resource consumption and improving computational efficiency. Simultaneously, it improves the real-time response capability of the main controller when performing high-precision tasks, avoiding a decline in the dynamic following performance of the robotic arm due to excessive numerical computation burden, thus enhancing the overall stability of motion control.

[0123] Furthermore, by utilizing singular value sequences and sensitivity thresholds, the set of direction vectors is divided into two subsets: non-singular directions and near-singular directions. This allows for differentiated processing of different directions in the mapping relationship between joint space and task space, ensuring that only near-singular directions are suppressed when approaching singular configurations, while maintaining the resolution and sensitivity of non-singular directions. Additionally, by decomposing the desired velocity in the task space into a first subset and a second subset, and scaling the velocity components corresponding to the second subset, the amplification effect of small singular value directions on the joint velocity solution can be effectively reduced when approaching singular states. This ensures that the final synthesized joint velocity remains within a stable range, thereby maintaining smooth joint motion and ensuring control accuracy even when approaching singular configurations. Attached Figure Description

[0124] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0125] Figure 1 A system architecture diagram is shown that can be applied to the robot arm motion control method in the embodiments of this disclosure.

[0126] Figure 2 A flowchart illustrating a motion control method for a robotic arm according to an embodiment of this disclosure is shown.

[0127] Figure 3 A schematic diagram of a process for singularly dividing a direction vector is shown in an embodiment of this disclosure.

[0128] Figure 4 A schematic diagram of a process for decomposing a desired velocity according to an embodiment of this disclosure is shown.

[0129] Figure 5 A schematic diagram of a process for performing scaling on a second direction vector is shown in an embodiment of this disclosure.

[0130] Figure 6 A schematic flowchart illustrating the calculation of the speed of each joint is shown in an embodiment of this disclosure.

[0131] Figure 7 A schematic diagram of a deep learning model pre-training process is shown in an embodiment of this disclosure.

[0132] Figure 8 The comparison results of the traditional control method and the present solution in terms of real-time performance and stability are shown in the embodiments of this disclosure under a unified test environment.

[0133] Figure 9 A block diagram of a robot arm motion control device according to an embodiment of the present disclosure is shown.

[0134] Figure 10 A block diagram of another robotic arm motion control device according to an embodiment of this disclosure is shown.

[0135] Figure 11 A schematic diagram of a humanoid robot according to an embodiment of this disclosure is shown.

[0136] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown.

[0137] Figure 13 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0138] In this disclosure, the terms "first" and "second" are used for description only and do not indicate relative importance or imply the number of technical features. Therefore, the features referred to as "first" or "second" may explicitly or implicitly include at least one of those features. "A plurality of" means at least two, unless otherwise expressly defined.

[0139] First, the relevant terms used in the exemplary embodiments of this disclosure will be explained:

[0140] Deep learning model: A machine learning model based on a multi-layer neural network structure, which approximates nonlinear mapping relationships by extracting features and learning parameters from a large number of training samples.

[0141] Joint space: A parameter space composed of the independent angular variables of each joint of a robot arm, used to describe the robot arm's degrees of freedom and its internal configuration. Any point in the joint space corresponds to the joint configuration of the robot arm at a certain moment, usually represented by joint angle data.

[0142] Task space: refers to the space encompassing the position and orientation of the robot arm's end effector in three-dimensional space, used to describe the robot arm's motion state in the actual operating environment. The velocity vector in the task space represents the motion velocity that the end effector should achieve in the desired task.

[0143] Jacobian matrix: A matrix that describes the mapping relationship between the joint space and task space of a robot arm. The matrix elements depend on the kinematic parameters of the robot arm and the current joint angle.

[0144] The original Jacobian matrix refers to the standard Jacobian matrix without any numerical correction or stabilization, which fully reflects the mapping relationship between joint angular velocity and end effector velocity. In the example implementation of this disclosure, the deep learning model makes predictions based on singular value decomposition of the original Jacobian matrix.

[0145] The secure Jacobian matrix is ​​a modified matrix obtained by lifting or replacing singular values ​​that meet certain conditions, based on the singular value decomposition results of the original Jacobian matrix. This matrix is ​​used to constrain pseudo-inverse operations when approaching singular configurations, preventing joint velocity divergence caused by small singular values, thereby ensuring numerical stability.

[0146] Projected Jacobian matrix: refers to the equivalent Jacobian matrix formed by projecting the mission space velocity onto a specified subset of directions based on the original Jacobian matrix decomposition result. In the exemplary embodiments of this disclosure, the projected Jacobian matrix can be used to retain only the velocity components in directions that are not close to singularities, thereby suppressing instabilities caused by singular directions while ensuring control accuracy.

[0147] Approaching Singular Direction: In the singular value decomposition of the Jacobian matrix, the direction of motion corresponding to the direction vector with a singular value less than the singular value threshold reflects the degree of freedom of the robot arm that is weak in motion capability or susceptible to singularity under the current configuration.

[0148] Non-singular direction: In the singular value decomposition of the Jacobian matrix, the direction of motion corresponding to the direction vector whose singular value is greater than or equal to the singular value threshold represents the degree of freedom of the robot arm to maintain good motion capability in the current configuration.

[0149] Joint configuration: refers to the overall posture determined by the angles of all joints of the robot arm, and is the physical manifestation of a specific point in the joint space.

[0150] Joint angle data: refers to the real-time angle measurements of each joint of the robot arm.

[0151] Teacher-student model framework: A knowledge distillation training method for deep learning models, where the teacher model is typically a high-precision but computationally complex reference model, and the student model is a target model with fewer parameters and faster inference speed. During training, the student model learns the output distribution or intermediate features of the teacher model, achieving a balance between accuracy and efficiency.

[0152] Figure 1 A system architecture diagram is shown that can be applied to the robot arm motion control method in the embodiments of this disclosure. For example... Figure 1 As shown, the system architecture 100 may include a terminal device 101, a robot 102, a network 103, and a server 104.

[0153] The terminal device 101 includes, but is not limited to, desktop computers, laptops, smartphones, and tablets. The terminal device 101 is equipped with a graphical user interface, providing visualization capabilities to display the operating status and joint motion information of the robot 102. Furthermore, the terminal device 101 also provides parameter configuration functions, allowing users to set model parameters, sensitivity thresholds, and control strategy execution modes related to the robot arm's motion control, and to collaborate with the server 104 to complete the visualization analysis and management of training data.

[0154] Robot 102 is equipped with various sensors, such as inertial measurement units and joint angle encoders, to collect its own joint space data and task space data in real time. It is important to note that Robot 102 includes at least a main controller and joint actuators. The main controller runs the deployed deep learning model, receives sensor sampling data, and predicts the singular value decomposition (SVD) results of the original Jacobian matrix. This yields a set of direction vectors and a sequence of singular values ​​associated with each direction vector in the set. Based on the singular value sequence and a sensitivity threshold, it partitions the direction subsets, performs directional decomposition and scaling of the desired velocity vector, and finally obtains the velocity of each joint in the joint space. The joint actuators receive velocity commands from the main controller and drive the corresponding joints to perform movements according to their respective velocities, thus achieving real-time control of the robot arm.

[0155] Server 104 is equipped with a training module for robot arm motion control. It can receive and preprocess sensor sampling data from robot 102 and train a deep learning model capable of predicting the singular value decomposition results of the original Jacobian matrix. After training, server 104 can deploy the deep learning model to robot 102 for real-time prediction and generation of joint velocities during operation.

[0156] Network 103 serves as the medium for providing a communication link between terminal device 101, robot 102, and server 104. Network 103 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc. It should be understood that... Figure 1 The number and type of terminal devices, robots, networks, and servers shown are merely illustrative. Depending on implementation needs, there can be any number and type of terminal devices, robots, networks, and servers; for example, the servers could be cloud servers.

[0157] Through the coordinated operation of various parts in the system architecture 100, a complete process can be realized, from sensor data acquisition and deep learning model prediction of the singular value decomposition results of the Jacobian matrix to the joint actuator executing the joint speed, thereby significantly improving the real-time performance and control stability of the robotic arm in complex tasks.

[0158] This disclosure provides an example implementation of a robot arm motion control method, applied to the main controller of a robot. (Reference) Figure 2 As shown, the method may include the following steps S201 to S204:

[0159] Step S201: The singular value decomposition result of the original Jacobian matrix representing the mapping relationship between the joint space and the task space of the robot arm is predicted by the deep learning model, and the set of direction vectors and the singular value sequence associated with each direction vector in the set of direction vectors are obtained.

[0160] Step S202: Based on the singular value sequence and the sensitivity threshold, determine the first subset of the direction vector set corresponding to non-singular directions and the second subset corresponding to near-singular directions;

[0161] Step S203: Decompose the input vector representing the desired velocity in the task space on the first subset and the second subset respectively to obtain the first directional component and the second directional component;

[0162] Step S204: Perform scaling processing on the second direction component, and obtain the velocity of each joint in the joint space based on the first direction component and the scaled second direction component.

[0163] The robot arm motion control method provided in the exemplary embodiments of this disclosure directly predicts the singular value decomposition results of the original Jacobian matrix through a deep learning model. This enables the rapid acquisition of the set of direction vectors and the sequence of singular values ​​corresponding to the current joint configuration without the need for complex real-time computation, thereby improving the real-time performance of robot arm control. Furthermore, by comparing with a sensitivity threshold, the direction vectors are divided into non-singular and near-singular directions, and only the velocity components in the near-singular directions are scaled to avoid joint velocity divergence while maintaining control accuracy in the non-singular directions. Therefore, this method can still achieve smooth and stable motion control even in near-singular configurations, improving reliability and robustness under complex tasks and boundary configurations.

[0164] The robot arm motion control method in this example embodiment will now be described in detail.

[0165] In step S201, the singular value decomposition result of the original Jacobian matrix representing the mapping relationship between the joint space and the task space of the robot arm is predicted by the deep learning model, and the set of direction vectors and the singular value sequence associated with each direction vector in the set of direction vectors are obtained.

[0166] In an exemplary embodiment of this disclosure, the deep learning model can output the singular value decomposition result of the original Jacobian matrix representing the mapping relationship between the joint space and the task space during the inference phase, based on the mapping relationship learned during the training phase.

[0167] For the original Jacobian matrix, the direction vector set is a set of orthogonal direction vectors obtained through singular value decomposition, used to represent the main motion directions that the robot arm can achieve in joint space and task space. The sequence of singular values ​​associated with each direction vector in the direction vector set refers to the singular value corresponding to each direction vector, which is used to characterize the magnitude of the robot arm's motion capability in the corresponding direction. A larger singular value indicates a stronger motion capability in that direction, while a smaller singular value indicates a weaker motion capability in that direction, or even close to a singular configuration.

[0168] For example, joint angle data representing the current joint configuration of the robot arm can be obtained, and the joint angle data can be input into a pre-trained deep learning model to obtain the singular value decomposition result of the original Jacobian matrix corresponding to the current joint configuration. The deep learning model can be a convolutional neural network, a recurrent neural network, a graph neural network, a Transformer network, etc., and this disclosure does not limit the specific type of deep learning model.

[0169] In some example implementations, the deep learning model may include an input layer, a shared feature extraction layer, and a multi-branch output layer. Specifically, the input layer receives joint angle data of the current joint configuration of the robot arm. For example, the joint angle data is an n-dimensional vector that can fully represent the current joint configuration. The shared feature extraction layer extracts features from the joint angle data to obtain joint feature vectors that reflect the characteristics of the joint configuration. For instance, the shared feature extraction layer may include two fully connected layers, each with a predetermined number of nodes. After the fully connected computation, normalization and nonlinear activation functions, such as layer normalization and ReLU (Rectified Linear Unit) functions, are introduced to perform step-by-step mapping and normalization of the input data, thereby extracting stable and discriminative feature representations. The joint feature vectors obtained after processing by the shared feature extraction layer can serve as a unified feature representation for shared use by the multi-branch output layer. The multi-branch output layer predicts the singular value decomposition results of the original Jacobian matrix based on the joint feature vectors to directly obtain the structured information required for motion control.

[0170] The predicted singular value decomposition results can include the singular direction probabilities corresponding to each direction vector in the direction vector set, as well as the singular value sequence associated with the direction vector set. Correspondingly, the multi-branch output layer can include a classification output layer and a regression output layer. Specifically, the classification output layer is used to obtain the singular direction probabilities corresponding to each direction vector in the direction vector set based on the joint feature vectors, in order to further distinguish between near-singular directions and non-near-singular directions. The regression output layer is used to obtain the singular value sequence associated with the direction vector set based on the joint feature vectors, thereby characterizing the magnitude of motion capability in each direction.

[0171] It should be noted that the set of direction vectors obtained in step S201 is an inherent orthogonal basis structure of singular value decomposition. The deep learning model does not directly output specific direction vectors, but rather characterizes the attribute information of the set of direction vectors by predicting the singular direction probabilities and singular value sequences corresponding to each direction vector, and thereby distinguishes between near-singular directions and non-singular directions.

[0172] For example, after predicting the singular value decomposition results of the original Jacobian matrix using a deep learning model, we have:

[0173] (1)

[0174] in, The original Jacobian matrix, The task space matrix has a spatial dimension of . m , Let be the joint space matrix, with spatial dimensions of . n Both are orthogonal matrices, and the column vectors of each spatial matrix are the main directions of motion that can be realized in the corresponding space, and together they form a set of direction vectors; The singular value matrix is ​​the sequence of singular values ​​{σ1,σ2,…σ} predicted by a deep learning model. m The structure can be written as:

[0175] Σ = diag (σ1,σ2,…σ m ), σ1≥σ2≥…≥σ m ≥0 (2)

[0176] in, diag () denotes the diagonalization operation, where σ1, σ2, ..., σ3 are arranged sequentially on the diagonal of the singular value matrix. m All other elements are zero, σ1, σ2, ..., σ m These are all called singular values, and are non-negative real numbers, arranged in descending order.

[0177] Deep learning models can output the singular value decomposition results of the original Jacobian matrix in one go during the inference phase, thereby avoiding the large amount of computation required for real-time numerical decomposition. This allows the robot's main controller to directly use the singular value prediction results for velocity decomposition and scaling when performing motion control, ensuring control stability and overall real-time performance under near-singular configurations.

[0178] In step S202, based on the singular value sequence and the sensitivity threshold, a first subset corresponding to non-singular directions and a second subset corresponding to near-singular directions are determined in the set of direction vectors.

[0179] In this exemplary implementation, the set of direction vectors is classified into two categories: non-singular directions and near-singular directions, using a sequence of singular values ​​and a sensitivity threshold. Specifically, the magnitude of the singular values ​​reflects the strength of the robot's motion capability in each direction. When the singular value is sufficiently large, the direction is determined to have stable motion capability and is classified as a non-singular direction; conversely, when the singular value is too small or close to zero, the direction is determined to have a singularity risk and is classified as a near-singular direction.

[0180] This distinction facilitates differentiated treatment of different directions in subsequent control, ensuring control accuracy in non-singular directions while suppressing near-singular directions, thereby improving the overall stability and reliability of control.

[0181] In some example implementations, references Figure 3 As shown, step S202 may further include steps S301 to S303:

[0182] Step S301: Construct discrimination conditions based on sensitivity thresholds.

[0183] For example, the maximum singular value in the singular value sequence can be determined first, and this maximum singular value can be used as a reference value, such as σ1 in formula (2), which is the maximum singular value. Then, the singular value threshold is determined based on the maximum singular value and the sensitivity threshold, and the discrimination condition for direction vector classification is constructed using the singular value threshold.

[0184] The sensitivity threshold controls the tolerance for the magnitude of singular values, and its value directly affects the range of directions approaching singularities and the subsequent control effect. Specifically, when the sensitivity threshold approaches 1, the discrimination condition restricts singular values ​​more strictly, allowing only directions close to the largest singular value to be judged as non-singular directions. This results in higher numerical stability, but reduced motion flexibility. Conversely, when the sensitivity threshold approaches 0, the tolerance for smaller singular values ​​increases, and more directions are retained to maintain motion capability, but the risk of numerical instability also increases.

[0185] In practical applications, the sensitivity threshold can be adjusted experimentally, set according to the different accuracy and stability requirements of the task. For example, a larger sensitivity threshold can be selected for tasks requiring higher stability, while a smaller sensitivity threshold can be selected for tasks requiring strong motion flexibility. Furthermore, a dynamic adjustment method can be used to adaptively optimize the sensitivity threshold based on the current joint configuration or task state of the robotic arm, further improving the system's control performance and adaptability in complex environments.

[0186] For example, if the singular value corresponding to a certain direction is less than the product of the maximum singular value and the sensitivity threshold, the direction is determined to be a near-singular direction; otherwise, it is determined to be a non-singular direction. By setting different sensitivity thresholds, the system's sensitivity to singular directions can be flexibly adjusted to adapt to different accuracy requirements and control stability requirements.

[0187] For example, when determining the singular value threshold based on the maximum singular value and the sensitivity threshold, an adjustment factor can be introduced to dynamically adjust the sensitivity of the discrimination criteria. This adjustment factor is calculated based on the singular orientation probabilities obtained by inputting joint angle data representing the current joint configuration of the robot arm into a deep learning model. It can adaptively adjust the discrimination criteria in conjunction with the current joint configuration of the robot arm, ensuring that the system maintains a reasonable discrimination standard under different joint configurations. This avoids the problem of a single sensitivity threshold being too lenient or too strict under complex postures.

[0188] Specifically, the regulation factor κ is defined as:

[0189] (3)

[0190] in, This represents the probability of a singular direction.

[0191] Furthermore, the singular value threshold is determined based on the adjustment factor, the maximum singular value, and the sensitivity threshold, and the discrimination criteria are constructed using the singular value threshold.

[0192] For example, the discrimination criteria can be:

[0193] (4)

[0194] in, κ is the singular value threshold, and κ is the adjustment factor. The sensitivity threshold, , To predict the maximum singular value in the obtained singular value sequence, For the predicted singular value sequence, the first... i There are singular values. This criterion can be used to determine whether a direction vector belongs to a near-singular direction.

[0195] In this embodiment, the introduction of the adjustment factor allows the singular value threshold to not only depend on the maximum singular value and the sensitivity threshold, but also to adjust with changes in the singular direction probability, thereby more accurately reflecting the singularity risk under the current joint configuration. Based on this, the discrimination condition can more accurately distinguish between near and non-near singular directions, improving the reliability of singular direction detection. Furthermore, when the adjustment factor is applied to the calculation of the singular value threshold, it can more effectively identify and constrain components corresponding to near singular directions while maintaining the accuracy of components in non-near singular directions, avoiding abnormal amplification in joint velocity calculations, thus improving the stability and safety of overall motion control. Therefore, by introducing the adjustment factor, the singular direction discrimination condition can possess better adaptability and accuracy, providing a more stable foundation for subsequent velocity decomposition and scaling.

[0196] Step S302: Based on the singular values ​​in the singular value sequence that do not meet the discrimination conditions, determine the first subset of the direction vector set corresponding to non-singular directions.

[0197] For example, for satisfying The singular value of the direction vector corresponding to the singular value is determined to be a non-singular direction. This type of direction can maintain the original velocity component during the control process, thereby ensuring motion accuracy.

[0198] Step S303: Based on the singular values ​​in the singular value sequence that satisfy the discrimination condition, determine the second subset in the direction vector set corresponding to the singular direction.

[0199] For example, for satisfying The singular value, the direction vector corresponding to the singular value is determined to be close to the singular direction. This type of direction will be identified and its velocity component will be scaled during the control process to avoid abnormal amplification of joint velocity in pseudo-inverse calculation, and to ensure the stability and reliability of the overall motion.

[0200] This embodiment can achieve joint determination based on the magnitude of singular values ​​and the probability of singular directions, thereby effectively distinguishing between non-singular directions and near-singular directions in the set of direction vectors, providing a reliable basis for direction division for subsequent robot arm motion control.

[0201] In step S203, the input vector representing the desired velocity in the task space is decomposed on the first subset and the second subset respectively to obtain the first directional component and the second directional component.

[0202] This step can separate the velocity components that are close to the singular direction while maintaining the velocity components that are not close to the singular direction, laying the foundation for subsequent differential processing. It avoids the numerical instability caused by the proximity to the singular direction and ensures the control accuracy of the main motion direction.

[0203] In some example implementations, references Figure 4 As shown, step S203 may further include steps S401 to S403:

[0204] Step S401: Construct a projection Jacobian matrix based on the first subset, and decompose the input vector into the first direction component according to the projection Jacobian matrix.

[0205] The first subset corresponds to the set of direction vectors that are not close to the singular direction. In the example implementation of this disclosure, the first subset is used to construct the projection Jacobian matrix, so that the input vector is preserved in the direction that is not close to the singular direction. Through decomposition processing based on the projection Jacobian matrix, the components corresponding to the direction that are not close to the singular direction can be extracted from the input vector as the first direction component, providing a basis for subsequent stable motion control.

[0206] For example, to construct a projected Jacobian matrix containing only non-singular directions, the task space matrix and the singular value matrix can be filtered based on the singular value decomposition results of the original Jacobian matrix. Specifically, the direction vectors corresponding to the second subset are removed from the task space matrix obtained by decomposing the original Jacobian matrix, while the direction vectors corresponding to the first subset are retained, thus obtaining the truncated task space matrix. Simultaneously, the singular values ​​corresponding to the direction vectors in the second subset are removed from the singular value matrix obtained by decomposing the original Jacobian matrix, while the singular values ​​corresponding to the direction vectors in the first subset are retained, thus obtaining the truncated singular value matrix.

[0207] Finally, based on the joint space matrix obtained by decomposing the original Jacobian matrix, the direction vectors in the retained first subset (i.e., the truncated task space matrix), and the singular values ​​corresponding to the direction vectors in the first subset (i.e., the truncated singular value matrix), the projected Jacobian matrix is ​​constructed.

[0208] Referring to formula (1), all column vectors corresponding to near-singular directions can be removed from the task space matrix U, and the corresponding singular values ​​can be removed from the singular value matrix Σ. For example, assuming k near-singular directions are removed, the remaining m-k non-near-singular directions can be used to obtain the truncation matrix. and .

[0209] Received and The effective information corresponding to non-singular directions in the original singular value decomposition results is retained, while the column vectors and singular values ​​related to near-singular directions have been removed. Based on the truncated task space matrix and singular value matrix, a projected Jacobian matrix can be further constructed to avoid joint velocity divergence caused by near-singular directions in numerical calculations, thereby providing a stable mathematical basis for the motion control of the robotic arm.

[0210] For example, the constructed projection Jacobian matrix for:

[0211] (5)

[0212] in, To retain only the task space matrix corresponding to the direction vectors in the first subset, To be a singular value matrix that retains only the non-zero singular values ​​corresponding to the direction vectors in the first subset. It is the joint space matrix, which is the same as the joint space matrix in formula (1).

[0213] In the exemplary embodiments of this disclosure, by constructing a projected Jacobian matrix, components corresponding to near-singular directions can be eliminated, effectively preventing the amplification of joint velocities in pseudo-inverse calculations and ensuring numerical stability. Furthermore, the projected Jacobian matrix can be used to calculate the safe-to-execute velocity components in the task space and ignore movements that may lead to singularities, thereby concentrating the velocity decomposition in the task space on directions not close to singularities. This ensures the smoothness and controllability of the robot arm's movements and improves the stability and reliability of the overall recovery process.

[0214] Furthermore, the input vector can be decomposed into its first directional component based on the projected Jacobian matrix, i.e.:

[0215] (6)

[0216] in, For the first direction component, For the projected Jacobian matrix, For the generalized inverse of the projected Jacobian matrix, such as the right pseudo-inverse of the projected Jacobian matrix, we have: ,matrix It contains only derivatives of non-zero singular values. The input vector represents the desired velocity in the task space.

[0217] Step S402: Extract the target direction vector corresponding to the task space from the second subset.

[0218] It is understandable that the second subset includes direction vectors corresponding to the task space and joint space. Therefore, the target direction vector corresponding to the task space is extracted from the second subset as follows: ,in, m The dimension of the task space. k This indicates the number of directions that are close to the singular.

[0219] Step S403: Decompose the input vector on the target direction vector to obtain the second direction component, i.e.:

[0220] (7)

[0221] in, For the second direction component, Let the target direction vector be... This is the transpose of the target direction vector. The input vector represents the desired velocity in the task space.

[0222] In this embodiment, by extracting the target direction vector of the task space from the second subset and decomposing the input vector on the target direction vector, the velocity components corresponding to near-singular directions can be identified and separated separately. This avoids numerical instability caused by mixing such components with normal motion directions, and provides a clear processing object for subsequent scaling and constraint of velocity components near-singular directions, thereby improving the controllability and stability of overall motion control.

[0223] In step S204, scaling is performed on the second direction component, and the velocity of each joint in the joint space is obtained based on the first direction component and the scaled second direction component.

[0224] Understandably, the second directional component corresponds to a near-singular direction. Near-singular directions tend to amplify velocity during calculations, thus requiring scaling to limit its amplitude and prevent joint velocity instability. The scaled second directional component, combined with the first directional component, ensures the accuracy of velocity components not near-singular directions while effectively suppressing velocity components near-singular directions, thereby achieving stable and reliable joint velocities and ensuring smooth and consistent overall motion control.

[0225] In some example implementations, references Figure 5 As shown, scaling processing can be performed on the second direction component according to steps S501 and S502, specifically:

[0226] Step S501: Calculate the scaling factor based on the ratio between the singular values ​​corresponding to each directional vector in the second subset and the singular value threshold; wherein the singular value threshold is determined based on the sensitivity threshold.

[0227] In this example implementation, the scaling factor is used to adjust the velocity component, and the singular value threshold is determined based on the sensitivity threshold to help distinguish between near-singular directions and non-singular directions.

[0228] For each direction vector in the second subset, the corresponding singular values ​​are relatively small, and directly using them for joint velocity calculations may lead to numerical instability. Therefore, by calculating the ratio of each singular value to a singular value threshold, a scaling factor can be obtained, and the amplitude of the second direction component can be constrained accordingly. The introduction of this scaling factor effectively avoids the velocity amplification problem caused by proximity to singular directions while maintaining the accuracy of the main motion components, thus providing a stable numerical basis for subsequent joint velocity calculations.

[0229] For example, the singular value threshold can be based on the sensitivity threshold. The maximum singular value in the singular value sequence The calculations show that: For example, a moderating factor κ can be introduced, combined with a sensitivity threshold, to determine the singular value threshold, resulting in: Of course, the singular value threshold can be set according to actual needs, and this disclosure does not limit this.

[0230] Furthermore, for example, the calculated scaling factor could be:

[0231] (8)

[0232] in, For the first i Scaling factors corresponding to each direction vector The first singular value in the sequence i A singular value, The singular value threshold is preset. Of course, this disclosure is not limited to the calculation method shown in formula (8). When the calculation method of the singular value threshold changes, the calculation formula (8) of the scaling factor can also be adapted and adjusted to ensure that the relationship between the singular value and the singular value threshold can still be correctly reflected under different implementation conditions, thereby maintaining effective constraints on the velocity components close to the singular direction.

[0233] Step S502: Perform scaling processing on the second direction component according to the scaling factor to obtain the scaled second direction component.

[0234] Specifically, the scaling factor corresponding to each direction vector in the second subset is determined, and a scaling matrix is ​​constructed based on each scaling factor, i.e.:

[0235] (9)

[0236] in, For scaling matrices, diag () indicates the diagonalization operation. k This indicates the number of directions approaching the singular. It's important to note that when... When the value approaches 0, the corresponding scaling factor approaches 0, which greatly weakens the velocity components near the singular direction.

[0237] Then, based on the scaling matrix and the second directional component, the scaled second directional component is obtained. The scaling matrix consists of scaling factors corresponding to each near-singular direction, used to constrain the magnitude of the second directional component in different directions.

[0238] For example, the scaled second-direction component can be calculated using the scaling matrix, the second-direction component, and the gain matrix, i.e.:

[0239] (10)

[0240] in, This is the scaled second-direction component. This is the second direction component before scaling. The direction vector in the second subset. This is the transpose of the direction vectors in the second subset. Let be the input vector representing the desired velocity in the task space. For scaling matrices, This is the gain matrix, used to adjust the control strength of each direction vector in the second subset.

[0241] Formula (10) can not only limit the amplitude of velocity components in near-singular directions, but also flexibly adjust the scaling effect using the gain matrix, thereby ensuring that stable and reliable velocity components can be obtained under different working conditions, providing a solid foundation for the subsequent synthesis of joint velocities.

[0242] Additionally, it should be noted that when the singular values ​​corresponding to each direction vector in the second subset are greater than or equal to the singular value threshold, the second direction component is retained.

[0243] For example, when When the scaling factor is 1, the velocity component is not scaled in that direction, thus ensuring that the original amplitude of such directional components is maintained in the joint velocity calculation. This avoids excessive suppression of unnecessary directions, ensuring numerical stability while taking into account the accuracy and integrity of motion control.

[0244] Further, refer to Figure 6 As shown, after obtaining the scaled second direction component, the velocities of each joint in the joint space can be obtained according to steps S601 to S603, specifically:

[0245] Step S601: The singular values ​​corresponding to each direction vector in the second subset are boosted to obtain the adjusted singular value matrix.

[0246] Specifically, the singular values ​​in the second subset are relatively small. If used directly for joint velocity calculation, they may cause velocity amplification during pseudo-inverse operations, resulting in numerical instability. Therefore, these small singular values ​​can be raised to a level no lower than a preset threshold to avoid the adverse effects of excessively small singular values.

[0247] For example, the singular values ​​corresponding to each direction vector in the second subset can be boosted to a singular value threshold. For instance, a fixed singular value threshold can be set; when a singular value is less than this threshold, it is directly boosted to the threshold to ensure the numerical stability of the adjusted singular value matrix. Alternatively, the singular values ​​can be dynamically boosted using an adjustment factor. This involves calculating the adjustment factor based on the singular direction probabilities corresponding to the current joint configuration, and then adaptively adjusting the boosting magnitude of the singular values ​​to better suit the needs of actual working conditions.

[0248] Then, based on the singular values ​​corresponding to each direction vector in the first subset and the singular value thresholds corresponding to each direction vector in the second subset, the adjusted singular value matrix is ​​obtained.

[0249] For example, there are:

[0250] (11)

[0251] in, This is the adjusted singular value matrix. The singular value threshold, These are the singular values ​​corresponding to the directional vectors in the first subset.

[0252] As can be seen from formula (11), the adjusted singular value matrix not only retains the relative relationship of the original singular value distribution, but also ensures that all singular values ​​are within the controllable range, thereby effectively avoiding joint velocity divergence in subsequent pseudo-inverse operations and improving the stability and reliability of the overall control process.

[0253] Step S602: Based on the task space matrix, joint space matrix, and adjusted singular value matrix obtained from the decomposition of the original Jacobian matrix, a safe Jacobian matrix is ​​constructed, which is:

[0254] (12)

[0255] in, For a safe Jacobian matrix, , The task space matrix and joint space matrix are obtained by decomposing the original Jacobian matrix. This is the adjusted singular value matrix.

[0256] As can be seen from formula (12), the reconstructed safe Jacobian matrix retains the directional information of the original Jacobian matrix, only adjusting the... The presence of excessively small singular values ​​effectively avoids the instability of pseudo-inverse calculations caused by excessively small singular values ​​at the numerical level, preventing abnormal amplification of joint velocities. Furthermore, since the task space matrix U and the joint space matrix V remain unchanged, the safety Jacobian matrix still fully preserves the spatial mapping direction information reflected by the original Jacobian matrix, ensuring the accuracy of the mapping relationship between the task space and the joint space. Therefore, it can balance numerical stability and mapping accuracy, enabling the robotic arm to achieve smooth and reliable speed control even near singular configurations, thereby improving the overall stability and safety of motion control.

[0257] Step S603: Based on the generalized inverse of the secure Jacobian matrix, the velocity of each joint is calculated by combining the first directional component and the scaled second directional component.

[0258] Specifically, after obtaining the safe Jacobian matrix, the correction velocity in the task space can be calculated based on the first directional component and the scaled second directional component, i.e.:

[0259] (13)

[0260] in, To correct the speed, For the first direction component, This is the scaled second direction component.

[0261] Combining formulas (6) and (10), formula (13) can also be written as:

[0262] (14)

[0263] Next, based on the corrected velocity calculated from the generalized inverse of the safety Jacobian matrix, the velocity of each joint is calculated, i.e.:

[0264] (15)

[0265] in, For the speed of each joint, For a safe Jacobian matrix, For the generalized inverse of the safe Jacobian matrix, To correct the speed.

[0266] In this step, firstly, the first directional component and the scaled second directional component are combined to form a corrected velocity. This ensures that the velocity decomposition in the task space retains the accurate components in directions not close to singularities while imposing amplitude constraints on directions close to singularities, thus preventing velocity divergence. Secondly, the generalized inverse of the safe Jacobian matrix is ​​used for velocity mapping. This maintains the original mapping direction information while ensuring the stability of the numerical calculation, making the joint velocity calculation reliable even in near-singular configurations. Finally, the obtained joint velocities satisfy the motion requirements of the task space while avoiding the instability caused by singularities, thereby achieving smooth control and consistent motion of the robotic arm under complex working conditions.

[0267] In some example implementations, references Figure 7 As shown, before predicting the singular value decomposition result of the original Jacobian matrix using a deep learning model, the deep learning model can be pre-trained according to steps S701 to S703, specifically:

[0268] Step S701: Obtain joint angle data representing various joint configurations of the robot arm.

[0269] The joint angle data includes raw joint angle data and target joint angle data. For example, raw joint angle data for various joint configurations can be obtained by sampling within the joint limits of the robot arm. That is, sampling is performed within the motion limits of each joint of the robot arm to obtain raw joint angle data covering multiple joint configurations.

[0270] For example, the sampling of joint angle data can be performed uniformly within the joint limit range, that is:

[0271] (16)

[0272] in, This represents the joint angle data vector, which is the raw joint angle data. n This indicates the number of joints in the robot arm. These represent the minimum and maximum angle limits for each joint, respectively. The joint angle data acquired in this way can cover various joint configurations of the robotic arm within physical constraints, providing ample sample support for subsequent calculation of the real Jacobian matrix and training of deep learning models.

[0273] After collecting the original joint angle data, data augmentation can be performed on the original joint angle data, such as by adding noise perturbation, interpolation expansion or random perturbation, to obtain the target joint angle data for training, thereby improving the model's generalization ability under different joint configurations.

[0274] When performing data augmentation on the raw joint angle data, for example, Gaussian noise can be added to each raw joint angle data to obtain the target joint angle data, such as:

[0275] q noise = q + N (0,δ) (17)

[0276] in, q This represents the raw joint angle data. N (0,δ) represents a Gaussian random variable with a mean of 0 and a standard deviation of δ, such as δ = 0.01 rad. q noise The target joint angle data after adding noise.

[0277] By superimposing Gaussian noise on the joint angle data during training, the deep learning model can still output stable Jacobian matrix singular value decomposition results even with slight perturbations in the input data, thus enhancing the model's robustness. The noise perturbation is equivalent to data augmentation of the joint angle data, enriching the distribution of training samples and preventing the model from overfitting to data under ideal conditions, thereby improving its adaptability in real-world scenarios. Furthermore, since robot joint angle sensors typically exhibit measurement bias or jitter, adding noise introduces this uncertainty during the training phase, making the model more closely resemble actual working conditions after deployment. When the robot arm operates close to a singular configuration, the model can also predict the stability of the direction vector set and singular value sequence under small perturbations, ensuring the reliability of joint velocity decomposition and scaling calculations, and ultimately improving the overall stability of motion control.

[0278] For example, in addition to sampling the original joint angle data within the joint limits, richer training samples can be obtained based on the trajectory generation method in the task space. Specifically, inverse kinematics can be solved for the continuous trajectory of the robot arm in the task space to obtain the adjacent joint angle sequence corresponding to the continuous trajectory. Subsequently, the adjacent joint angle sequence is combined with the sampled original joint angle data to form target joint angle data with a wider coverage and smoother configuration transition.

[0279] The target joint angle data generated in this example can supplement the continuous change features between adjacent joint configurations during actual movement while preserving the diversity of the original data, thereby further improving the training effect and generalization ability of the model in dynamic task scenarios.

[0280] Step S702: Calculate the true Jacobian matrix corresponding to each joint configuration, and perform singular value decomposition on each true Jacobian matrix to obtain the set of direction vectors and the true singular value sequence associated with each direction vector in the set of direction vectors.

[0281] Specifically, the joint angle data obtained in step S701 can be substituted into the kinematic model of the robot arm to calculate the corresponding true Jacobian matrix, denoted as... The true Jacobian matrix can characterize the mapping relationship between the joint space and the task space, reflecting the correspondence between the joint angular velocity and the end effector velocity under a specific joint configuration.

[0282] Furthermore, singular value decomposition is performed on each true Jacobian matrix to obtain the task space matrix, joint space matrix, and singular value matrix. This yields a set of direction vectors, namely, an orthogonal basis set composed of the column vectors of the task space and joint space matrices, and a sequence of true singular values ​​corresponding to each direction vector. Based on this, each direction vector can be classified and labeled according to a sensitivity threshold and the maximum singular value in the true singular value sequence. This allows for the labeling of singular directions while extracting the direction vector set and the true singular value sequence, providing a reference for subsequent model training and control strategies.

[0283] The singular value decomposition process provides an accurate set of direction vectors and a sequence of true singular values ​​for the training of subsequent deep learning models, thereby ensuring the accuracy and reliability of the model in predicting the results of singular value decomposition.

[0284] Step S703: Based on the teacher-student model framework, the deep learning model is trained using the set of direction vectors and the sequence of true singular values ​​associated with each direction vector in the set of direction vectors.

[0285] By training the deep learning model within a teacher-student model framework, the prediction accuracy and deployment efficiency of the model for singular value decomposition results can be improved. Specifically, a loss function is constructed based on a set of direction vectors and the sequence of true singular values ​​associated with each direction vector in the set, and the deep learning model is iteratively trained using the loss function.

[0286] For example, multiple constraints can be integrated when constructing the loss function. For instance, a classification loss for singular direction prediction can be constructed based on the true and predicted values ​​of each direction vector in the direction vector set, to measure the accuracy of singular direction prediction. A regression loss for singular value prediction can be constructed based on the true and predicted singular value sequences associated with each direction vector, to reflect the model's accuracy in singular value prediction. Further, a reconstructed Jacobian matrix is ​​obtained based on the predicted singular value sequences associated with each direction vector, and a reconstruction loss for the Jacobian matrix is ​​constructed based on the reconstructed and true Jacobian matrices to ensure consistency in the overall mapping relationship of the prediction results. Finally, a loss function is constructed based on the classification loss for singular direction prediction, the regression loss for singular value prediction, and the reconstruction loss for the Jacobian matrix, to guide the parameter updates of the deep learning model.

[0287] During training, a knowledge distillation strategy can be employed. This involves minimizing the KL (Kullback–Leibler Divergence) divergence between the output distributions of the teacher and student models to achieve efficient knowledge transfer. This allows the student model to maintain low computational complexity while still achieving predictive performance close to that of the teacher model. Furthermore, during deployment, the student model can be quantized, such as by compressing and optimizing the trained model and using 16-bit floating-point numbers for computation. This enables fast inference with a target latency of less than 0.5ms, meeting the application requirements of real-time control.

[0288] In some example implementations, the loss function can be constructed as follows:

[0289] (18)

[0290] in, L For loss function, As the first weight, It is a true sequence of singular values. To predict singular value sequences, The regression loss for singular value prediction is represented by the L2 norm. and The differences between them; As the second weight, The classification loss is for predicting singular orientations. For the predicted first j The probability that a direction belongs to a near-singular direction, i.e., the predicted value of each direction vector. For the indicator function, i.e. the true value of each direction vector, when the... j A singular value Less than The value is 1 if it is true, and 0 otherwise. The sensitivity threshold, The largest singular value in the true singular value sequence. Let m be the cross-entropy function, and m be the number of singular values. As the third weight, The reconstruction loss is the Jacobian matrix. For the first i Joint configuration The calculated true Jacobian matrix, To reconstruct the Jacobian matrix, and The first i The task space matrix and joint space matrix are obtained by decomposing the real Jacobian matrix. To predict the singular value matrix formed by the singular values, denoted by Frobenius norm, used to measure the difference between the reconstructed Jacobian matrix and the true Jacobian matrix.

[0291] By collecting joint angle data for various joint configurations and calculating the corresponding true Jacobian matrix and singular value decomposition results, a wide range of accurately labeled training samples can be provided for deep learning models. Within the teacher-student model framework, training the model using a set of direction vectors and the true singular value sequence not only ensures the model's accuracy in singular direction prediction and singular value regression but also improves the generalization ability and lightweight performance of the student model through knowledge transfer. Therefore, the trained reinforcement learning model maintains high prediction accuracy under different joint configurations while meeting the computational efficiency and response speed requirements of real-time control, thus laying the foundation for stable motion control of robotic arms.

[0292] The robot arm motion control method proposed in the exemplary embodiments of this disclosure firstly preserves the corresponding velocity components in non-singular directions, ensuring high control accuracy and motion consistency even when the robot arm is far from singular configurations. Secondly, for components approaching singular directions, a dynamic scaling mechanism based on the singular value magnitude smoothly adjusts the velocity components as the singular value changes, avoiding sudden joint velocity changes caused by excessively small singular values ​​and achieving a smooth transition in the motion process. Furthermore, by introducing a safe Jacobian matrix and projection operations, the numerical instability problem that may arise near singular points in traditional pseudo-inverses is avoided, ensuring the stability and controllability of the computation. Overall, this method maintains control accuracy in non-singular directions while smoothing velocity components approaching singular directions, and improves numerical stability through an improved Jacobian matrix construction, thereby making the motion control of the robot arm more stable and reliable under complex working conditions.

[0293] For example, suppose a robot arm's end effector needs to move in a near-singular direction, such as performing an operation in a straight line where the robot arm is fully extended. Traditional methods (such as damped least squares) would uniformly apply damping to all velocity components, which, while avoiding numerical instability caused by singularities, also suppresses motion in non-singular directions, causing the overall motion to deviate from the expected path. In contrast, the method disclosed herein keeps the velocity components constant in non-singular directions (such as the rotational direction), thus ensuring control accuracy in these directions. However, in near-singular directions (such as the extension direction), scaling is applied according to the ratio of the singular value to the singular value threshold, causing the velocity in that direction to gradually decrease as it approaches a singularity, while maintaining its direction. This not only avoids instability caused by velocity divergence but also ensures that the end effector moves along the expected path, achieving a balance between numerical stability and path-keeping.

[0294] In summary, the method provided in this disclosure can restrict only the necessary directions when singularity risks occur, enabling the robotic arm to maintain a reasonable motion trajectory and control performance when approaching singular configurations.

[0295] This disclosure also provides another robotic arm motion control method applied to a robot's joint actuator. The method may include: receiving the velocities of each joint in the robot arm's joint space, and driving the corresponding joints to move according to their respective velocities. The velocities of each joint in the robot arm's joint space are determined according to... Figure 2 The robot arm motion control method shown is obtained.

[0296] Specifically, the joint actuator receives the velocities of each joint in the robot arm's joint space from the main controller and, based on the received velocity information, issues drive commands to the actuator motors or actuation units of each joint. For example, the joint actuator adjusts the voltage, current, and other drive signals of the actuator motors to match the output speed and torque of the actuator motors with the received velocity information, thereby ensuring that the actual motion state of the joints remains consistent with the target speed. During this process, the joint actuator can also incorporate feedback information to adjust the operating state of the actuator motors in real time to maintain the stability of the closed-loop control.

[0297] In this way, the synchronous movement of each joint in the time dimension can be achieved, enabling the robot arm to complete the movement along the expected speed trajectory, ensuring that the overall movement process in the task space is stable and coherent, and has good controllability and coordination.

[0298] In some example implementations, the robot arm motion control method provided in this disclosure can be executed in the form of a real-time control loop during the online deployment phase, such as a loop period of Δt = 2 ms. The method may include the following steps:

[0299] First, sensors collect real-time data on the robot arm's joint angles as control input. Then, the joint angle data is fed into a pre-trained deep learning model to obtain predictive outputs, including singular value estimates and singular orientation probabilities.

[0300] Based on this, a physical constraint correction mechanism can be introduced to dynamically adjust the sensitivity threshold according to the magnitude and rate of change of the real-time task error, as follows:

[0301] (19)

[0302] in, for t The sensitivity threshold at any given moment is used in real-time control to determine the criteria for approaching singular directions. The baseline sensitivity threshold represents the basic threshold of the system under error-free or static conditions. This is the task error vector. for The length of the mold, This is a proportionality coefficient, used for control. right The degree of influence, when the error is large, can be increased by... This tightens the discrimination criteria, thereby enhancing the suppression of singular directions; The differential coefficient represents the rate of change of the control error. right The degree of influence, when the error changes rapidly, improves This helps to quickly suppress potential numerical instability.

[0303] Next, the predicted singular values ​​are safety-boosted to ensure they do not fall below a dynamic threshold, preventing numerical instability caused by excessively small singular values. Further, a safety Jacobian matrix is ​​reconstructed based on the task space matrix U, the joint space matrix V, and the boosted singular value matrix. After obtaining the safety Jacobian matrix, the task space velocity is decomposed by projecting the input vector of the task velocity onto both non-singular and near-singular directions, thereby calculating the corrected velocity components. Subsequently, the generalized inverse of the safety Jacobian matrix is ​​used to map the corrected task space velocity onto the joint space, obtaining the velocity commands for each joint.

[0304] Through the above steps, task space decomposition, joint velocity calculation, and numerical safety assurance can be achieved, ensuring that the robotic arm avoids numerical divergence caused by singularities during real-time operation, while maintaining the stability and accuracy of the motion trajectory. Finally, the calculated joint velocity commands are sent to the joint actuators, enabling the robotic arm to execute the corresponding motion control.

[0305] In some example implementations, such as the online deployment of a 7-DOF (Degree of Freedom) robotic arm, it shows significant advantages in both computational efficiency and control performance compared to traditional singular value decomposition-based implementations.

[0306] Referring to Tables 1 and 2, the hardware and software baseline configurations required to implement this solution are illustrated below:

[0307] Table 1

[0308]

[0309] As shown in Table 1, the robotic arm itself is a robot arm with 7 degrees of freedom, and its serial number device identifier is HU_D03_03_001. The main controller is an Intel Xeon E3-1280v6 processor with a main frequency of 3.9GHz. During operation, Turbo Boost is disabled and the frequency is locked to ensure real-time stability. The real-time operating system is Ubuntu 18.04, loaded with the PREEMPT_RT real-time kernel, kernel version number 4.14.12-rt10.

[0310] In terms of sensor configuration, an ATI-Mini40 six-dimensional force sensor is used, with a sampling frequency of 1kHz and a measurement accuracy of ±0.1N. The motion capture system adopts the OptiTrack Prime13, equipped with 8 cameras, and has a positioning accuracy better than ±0.1mm.

[0311] Table 2

[0312]

[0313] As shown in Table 2, in terms of control algorithms, traditional methods use Jacobian matrix singular value decomposition, while this scheme uses neural network prediction combined with physical correction. Both methods have a fixed control cycle of 2ms (500Hz). In the task scenario, both methods employ a pick-place task and are compared and verified using the same physical model (elastic constraints). The pick-place task refers to the robot arm (end-effector) needing to complete the operation of "pick – move – place".

[0314] For example, during the experimental testing, the robot arm's end effector was first set to move along a helical trajectory. To simulate a high-load operating environment, a matrix calculation task was injected in the background to maintain the CPU load at approximately 80%. During the test, the fluctuations in the control cycle were recorded using a PCIe (Peripheral Component Interconnect Express)-1588 time synchronization card to evaluate the real-time performance and cycle maintenance capability of this solution under high-load conditions.

[0315] Furthermore, the angle of the 5th joint of the robotic arm can be fixed at q5=0 to construct a theoretically singular configuration. Subsequently, a high-speed motion command along the Z-axis is applied to the end effector of the robotic arm, such as... v z =0.8 m / s², enabling the robotic arm to traverse a singular region. During the test, a laser tracker was used to measure the end-effector position error, and strain gauges at the joints were used to collect joint vibration data to further evaluate the stability performance of this design in singular regions.

[0316] Specifically, in traditional control methods, real-time singular value decomposition of the Jacobian matrix corresponding to the joint configuration is required, resulting in a computational complexity of O(n log n). Taking the 7-DOF robotic arm as an example, the single decomposition delay is approximately 450 μs, which will occupy a large proportion of the calculation time under high-speed control cycles and put a significant burden on CPU (Central Processing Unit) resources.

[0317] To address the aforementioned issues, this solution introduces a Neural Network (NN) prediction method to replace online singular value decomposition. The forward computation complexity of the model is O(n). O (1) The parameter size is reduced to less than 50KB through lightweight model compression. When deployed on an embedded platform, the single-run latency is reduced to 180μs, which is about 60% lower than the latency of traditional control methods, thus meeting the real-time requirement of a 2ms control cycle. At the same time, the CPU load is reduced from about 65% to 11%, significantly freeing up computing resources.

[0318] like Figure 8 As shown, under a unified testing environment, real-time performance and stability comparison experiments were conducted between the traditional control method and the proposed scheme. Specifically:

[0319] In terms of real-time performance, Figure 8 The data is summarized in Table 3. Statistical analysis shows that this scheme significantly outperforms traditional control methods in four performance metrics: single calculation time, control cycle failure rate, average CPU load, and computational power consumption.

[0320] Table 3

[0321]

[0322] As shown in Table 3, the average single-cycle calculation time of this scheme is 38±3ms, which is significantly lower than the 458±18ms of the traditional control method. The control cycle failure rate of this scheme is 0%, while that of the traditional control method is 12.6%. Furthermore, the average CPU load on the main thread of this scheme is only 11.8±0.7%, compared to 65.4±2.1% of the traditional control method, representing an 83% reduction in average CPU load. Therefore, this scheme can significantly shorten the computation path, reduce the processing burden, and improve the real-time response capability of the main controller during task execution.

[0323] Furthermore, in traditional control methods, during a real-time loop with a control cycle of 2ms, the CPU spends 93% of its time calculating the control algorithm, leaving almost no margin for handling additional tasks. This solution, however, significantly reduces computation time, freeing up nearly five times the computational capacity within the same 2ms control cycle. This idle computing power can be used to handle additional control tasks, making the system more efficient and secure.

[0324] Regarding control stability, traditional control methods rely on fixed sensitivity thresholds to distinguish between singular and non-singular directions. This is prone to abrupt classification changes when singular values ​​approach the threshold, leading to direction jumps and affecting the continuity and stability of global control. This proposed solution introduces singular direction probabilities into the neural network prediction results and dynamically adjusts them, achieving smooth correction of singular values ​​and thus avoiding the abrupt changes in fixed threshold judgments under boundary conditions.

[0325] Similarly, in terms of controlling stability, Figure 8 The data is summarized in Table 4. Statistical analysis shows that this scheme significantly reduces the values ​​of position tracking error, joint speed fluctuation, and peak mechanical impact force compared to traditional control methods.

[0326] Table 4

[0327]

[0328] As shown in Table 4, for the joint velocity fluctuation index, the proposed solution has a value of 0.07±0.01 rad / s, which is below the safety threshold of 0.1 rad / s, while the traditional control method has a value of 0.32±0.04 rad / s. The position tracking error in the proposed solution is 0.42±0.1 mm, which is below the error threshold of 1.0 mm, while the traditional control method has an error of 1.82±0.3 mm. The peak mechanical impact force in the proposed solution is 2.1±0.3 N, which is below the safety threshold of 3.0 N, while the traditional method has an impact force of 5.6±0.6 N. Therefore, the proposed solution can effectively reduce joint velocity fluctuation, end-effector position overshoot, and instantaneous impact force in near-single-configuration control scenarios, ensuring the stability and safety of the motion control process.

[0329] In summary, this solution not only significantly reduces latency and resource consumption in terms of computational efficiency, but also improves numerical stability and motion continuity, thus better meeting the control requirements of robotic arms in complex tasks and near-singular configurations.

[0330] In an exemplary embodiment of this disclosure, a robot arm motion control device is also provided, which is applied to the robot's main controller. (Reference) Figure 9 As shown, the first type of robot arm motion control device 900 includes a singular value decomposition module 901, a singular direction determination module 902, a desired velocity decomposition module 903, and a joint velocity generation module 904, wherein:

[0331] The singular value decomposition module 901 is used to perform singular value decomposition on the Jacobian matrix representing the mapping relationship between the joint space and the task space to obtain a set of direction vectors and a sequence of singular values ​​associated with each direction vector in the set of direction vectors.

[0332] The singular direction determination module 902 is used to determine, based on the singular value sequence and the sensitivity threshold, a first subset of the direction vector set corresponding to non-singular directions and a second subset corresponding to near-singular directions.

[0333] The desired velocity decomposition module 903 is used to decompose the input vector representing the desired velocity in the task space on the first subset and the second subset respectively to obtain the first directional component and the second directional component.

[0334] The joint velocity generation module 904 is used to perform scaling processing on the second direction component and obtain the velocity of each joint in the joint space based on the first direction component and the scaled second direction component.

[0335] The specific details of each module in the above-mentioned robot arm motion control device have been described in detail in the corresponding robot arm motion control method, so they will not be repeated here.

[0336] In an exemplary embodiment of this disclosure, another motion control device for a robot arm is also provided, which is applied to the joint actuators of a robot. (See reference...) Figure 10 As shown, the second type of robot arm motion control device 1000 includes a joint motion module 1001, wherein:

[0337] The joint motion module 1001 is used to receive the speed of each joint in the joint space of the robot arm and drive the corresponding joint motion according to the speed of each joint.

[0338] The speeds of each joint in the joint space of the robot arm are based on, for example, in the embodiments of this disclosure. Figure 2 The robot arm motion control method shown is obtained.

[0339] The specific details of each module in the above-mentioned robot arm motion control device have been described in detail in the corresponding robot arm motion control method, so they will not be repeated here.

[0340] In an exemplary embodiment of this disclosure, a robot is also provided. The robot includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, implement the described method. The robot includes either a humanoid robot or a dual-arm robot. (Reference) Figure 11 The diagram shows a humanoid robot, including a robotic arm 1100.

[0341] refer to Figure 12 As shown, an electronic device capable of implementing the above method is also provided. The electronic device 1200 includes a processor 1201 and a memory 1202. The memory 1202 stores computer-readable instructions, which, when executed by the processor 1201, implement the method of this disclosure.

[0342] In an exemplary embodiment of this disclosure, a computer-readable storage medium is also provided, having stored thereon computer program code instructions that, when invoked by a robot's processor, cause the robot to perform the method described in the embodiments.

[0343] refer to Figure 13 As shown, a program product 1300 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0344] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0345] Finally, the above preferred embodiments are only used to illustrate the technical solutions of this application and are not restrictive. Although this application has been described in detail, those skilled in the art should understand that changes in form and detail can be made without departing from the scope defined by the claims of this application. The dimensions in the drawings are not related to the specific physical object, and the physical object dimensions can be arbitrarily changed.

Claims

1. A robot arm motion control method characterized by, A main controller applied to the robot, the method comprising: predicting, by a deep learning model, a singular value decomposition result of an original Jacobian matrix representing a mapping relationship between a joint space and a task space of a robot arm, obtaining a direction vector set and a singular value sequence associated with each direction vector in the direction vector set; determining, based on the singular value sequence and a sensitivity threshold, a first subset of the direction vector set corresponding to a non-near-singular direction and a second subset corresponding to a near-singular direction; decomposing an input vector representing a desired velocity in the task space on the first subset and the second subset respectively, obtaining a first direction component and a second direction component; performing scaling processing on the second direction component, and obtaining a velocity of each joint in the joint space based on the first direction component and the scaled second direction component.

2. The robot arm motion control method according to claim 1, characterized by, The singular value decomposition result of the original Jacobian matrix representing the mapping relationship between the joint space and the task space of the robot arm predicted by the deep learning model comprises: obtaining joint angle data representing a current joint configuration of the robot arm; inputting the joint angle data into the pre-trained deep learning model to obtain a singular value decomposition result of an original Jacobian matrix corresponding to the current joint configuration.

3. The robot arm motion control method according to claim 2, characterized by, The deep learning model comprises: an input layer for receiving the joint angle data; a shared feature extraction layer for performing feature extraction on the joint angle data to obtain a joint feature vector; a multi-branch output layer for predicting the singular value decomposition result of the original Jacobian matrix according to the joint feature vector.

4. The robot arm motion control method according to claim 3, characterized by, The predicted singular value decomposition result includes a singular direction probability corresponding to each direction vector in the direction vector set and a singular value sequence associated with the direction vector set; The multi-branch output layer comprises: a classification output layer for obtaining the singular direction probability corresponding to each direction vector in the direction vector set according to the joint feature vector; a regression output layer for obtaining the singular value sequence associated with the direction vector set according to the joint feature vector.

5. The robot arm motion control method according to claim 1, wherein, The method further comprises: obtaining joint angle data representing a plurality of joint configurations of the robot arm; calculating a real Jacobian matrix corresponding to each joint configuration, and performing singular value decomposition on each real Jacobian matrix to obtain a direction vector set and a real singular value sequence associated with each direction vector in the direction vector set; training the deep learning model based on a teacher-student model framework using the direction vector set and the real singular value sequence associated with each direction vector in the direction vector set.

6. The robot arm motion control method according to claim 5, wherein, The joint angle data includes original joint angle data and target joint angle data; The obtaining of the joint angle data representing a plurality of joint configurations of the robot arm comprises: sampling within the joint limits of the robot arm to obtain original joint angle data of a plurality of joint configurations; performing data augmentation on each original joint angle data to obtain the target joint angle data.

7. The robot arm motion control method according to claim 6, wherein, The data augmentation on each original joint angle data to obtain the target joint angle data comprises: Gaussian noise is added to each of the original joint angle data to obtain the target joint angle data.

8. The robot arm motion control method according to claim 6, wherein, The data augmentation is performed on each of the original joint angle data to obtain the target joint angle data, including: Inverse kinematics is solved for a continuous trajectory of the robot arm in the task space to obtain a sequence of adjacent joint angles; The target joint angle data is obtained by combining the sequence of adjacent joint angles and the original joint angle data.

9. The robot arm motion control method according to claim 5, wherein, The deep learning model is trained by using the set of direction vectors and the sequence of real singular values associated with each direction vector in the set of direction vectors, including: A loss function is constructed based on the set of direction vectors and the sequence of real singular values associated with each direction vector in the set of direction vectors; The deep learning model is iteratively trained by using the loss function.

10. The robot arm motion control method according to claim 9, wherein, The loss function is constructed based on the set of direction vectors and the sequence of real singular values associated with each direction vector in the set of direction vectors, including: A classification loss for singular direction prediction is constructed according to the real values and predicted values of each direction vector in the set of direction vectors; A regression loss for singular value prediction is constructed according to the sequence of real singular values and the sequence of predicted singular values associated with each direction vector; A reconstructed Jacobian matrix is obtained based on the sequence of predicted singular values associated with each direction vector, and a reconstruction loss for the Jacobian matrix is constructed according to the reconstructed Jacobian matrix and the real Jacobian matrix; The loss function is constructed according to the classification loss for singular direction prediction, the regression loss for singular value prediction, and the reconstruction loss for the Jacobian matrix.

11. The robot arm motion control method according to claim 10, wherein, The loss function is: wherein, L is a loss function, is a first weight, is a sequence of real singular values, is a sequence of predicted singular values, is a regression loss of singular value prediction, indicating the difference between and ; is a second weight, is a classification loss of singular direction prediction, is a probability that the predicted j th direction belongs to the proximity of singular direction, i.e., the predicted value of each direction vector, is an indicator function, i.e., the true value of each direction vector, which is 1 when the j th singular value is less than , otherwise 0, is a sensitivity threshold, is the maximum singular value in the sequence of real singular values, is a cross-entropy function, and m is the number of singular values; is a third weight, is a reconstruction loss of Jacobian matrix, is a real Jacobian matrix calculated from the i th joint configuration , is a reconstructed Jacobian matrix, and are the task space matrix and joint space matrix respectively decomposed from the i th real Jacobian matrix, is a singular value matrix composed of predicted singular values, indicates the Frobenius norm, which is used to measure the difference between the reconstructed Jacobian matrix and the real Jacobian matrix.

12. The robot arm motion control method of claim 1, wherein, The velocities of each joint in the joint space are obtained based on the first direction component and the scaled second direction component, including: The singular values corresponding to each direction vector in the second subset are lifted to obtain an adjusted singular value matrix; A safety Jacobian matrix is constructed based on the task space matrix, the joint space matrix obtained by decomposing the original Jacobian matrix, and the adjusted singular value matrix; The velocities of each joint are calculated based on the generalized inverse of the safety Jacobian matrix, the first direction component, and the scaled second direction component.

13. The robot arm motion control method according to claim 12, wherein, The singular values corresponding to each direction vector in the second subset are lifted to obtain an adjusted singular value matrix, including: The singular values corresponding to each direction vector in the second subset are lifted to a singular value threshold; wherein the singular value threshold is determined based on a sensitivity threshold; The adjusted singular value matrix is obtained according to the singular values corresponding to each direction vector in the first subset and the singular value threshold corresponding to each direction vector in the second subset.

14. The robot arm motion control method of claim 12, wherein, The velocities of each joint are calculated based on the generalized inverse of the safety Jacobian matrix, the first direction component, and the scaled second direction component, including: A corrected velocity in the task space is calculated according to the first direction component and the scaled second direction component; The velocities of each joint are calculated according to the generalized inverse of the safety Jacobian matrix and the corrected velocity.

15. The robot arm motion control method according to claim 14, wherein, The calculating a modified velocity in the task space according to the first direction component and the scaled second direction component comprises: wherein, is the modified velocity, is the first directional component, is the scaled second directional component.

16. The robot arm motion control method of claim 14, wherein, The calculating a velocity of each joint according to the generalized inverse of the safety Jacobian matrix and the modified velocity comprises: wherein is the velocity of each of the joints, is the safety Jacobian matrix, is the generalized inverse of the safety Jacobian matrix, is the modified velocity.

17. The robot arm motion control method of claim 1, wherein, The scaling processing performed on the second direction component comprises: A scaling factor is calculated according to a ratio between a singular value corresponding to each direction vector in the second subset and a singular value threshold, wherein the singular value threshold is determined based on a sensitivity threshold; A scaled second direction component is obtained by performing scaling processing on the second direction component according to the scaling factor.

18. The robot arm motion control method of claim 17, wherein, The scaling processing performed on the second direction component according to the scaling factor to obtain a scaled second direction component comprises: A scaling matrix is constructed according to scaling factors corresponding to each direction vector in the second subset; The scaled second direction component is obtained based on the scaling matrix and the second direction component.

19. The robot arm motion control method of claim 18, wherein, The scaled second direction component is obtained based on the scaling matrix and the second direction component, comprising: The scaled second direction component is calculated by using the scaling matrix, the second direction component and a gain matrix.

20. The robot arm motion control method of claim 19, wherein, The scaled second direction component is calculated by using the scaling matrix, the second direction component and a gain matrix, comprising: wherein is the scaled second direction component, is the unscaled second direction component, is a direction vector in the second subset, is the transpose of a direction vector in the second subset, is an input vector of desired velocities in the task space, is a scaling matrix, is a gain matrix for adjusting the control strength of each direction vector in the second subset.

21. The robot arm motion control method of claim 17, wherein, The method further comprises: When a singular value corresponding to each direction vector in the second subset is greater than or equal to a singular value threshold, the second direction component is retained.

22. The robot arm motion control method of claim 1, wherein, The first subset corresponding to a non-approaching singular direction and the second subset corresponding to an approaching singular direction in the direction vector set are determined based on the singular value sequence and a sensitivity threshold, comprising: A discrimination condition is constructed based on the sensitivity threshold; The first subset corresponding to a non-approaching singular direction in the direction vector set is determined according to singular values in the singular value sequence that do not satisfy the discrimination condition; The second subset corresponding to an approaching singular direction in the direction vector set is determined according to singular values in the singular value sequence that satisfy the discrimination condition.

23. The robot arm motion control method of claim 22, wherein, The discrimination condition is constructed based on the sensitivity threshold, comprising: A maximum singular value in the singular value sequence is determined; A singular value threshold is determined based on the maximum singular value and the sensitivity threshold, and the discrimination condition is constructed by using the singular value threshold.

24. The robot arm motion control method of claim 23, wherein, The singular value threshold is determined based on the maximum singular value and the sensitivity threshold, comprising: An adjustment factor is obtained, wherein the adjustment factor is calculated according to a singular direction probability obtained by inputting joint angle data representing a current joint configuration of the robot arm into a deep learning model; The singular value threshold is determined according to the adjustment factor, the maximum singular value and the sensitivity threshold, and the discrimination condition is constructed by using the singular value threshold.

25. The robot arm motion control method of claim 24, wherein, The discrimination condition comprises: wherein is a threshold for singular values, and is a threshold for sensitivity, , is a maximum singular value in the sequence of singular values, is the kth singular value in the sequence of singular values. i is the kth singular value in the sequence of singular values.

26. The robot arm motion control method of claim 1, wherein, The input vector representing the desired velocity in the task space is decomposed on the first subset and the second subset respectively to obtain the first direction component and the second direction component, comprising: constructing a projected Jacobian matrix based on the first subset, and decomposing the input vector based on the projected Jacobian matrix to obtain the first direction component; extracting a target direction vector corresponding to the task space from the second subset; decomposing the input vector on the target direction vector to obtain the second direction component.

27. The robot arm motion control method of claim 26, wherein, the first direction component is: wherein is a first direction component, is a projected Jacobian matrix, is a generalized inverse of the projected Jacobian matrix, is an input vector of desired velocities in the task space.

28. The robot arm motion control method of claim 26, wherein, the constructing the projected Jacobian matrix based on the first subset comprises: eliminating direction vectors in the second subset from a task space matrix obtained by decomposing the original Jacobian matrix, and retaining direction vectors in the first subset; eliminating singular values corresponding to direction vectors in the second subset from a singular value matrix obtained by decomposing the original Jacobian matrix, and retaining singular values corresponding to direction vectors in the first subset; constructing the projected Jacobian matrix based on a joint space matrix obtained by decomposing the original Jacobian matrix, the direction vectors in the first subset retained, and the singular values corresponding to the direction vectors in the first subset.

29. A robot arm motion control method, characterized by, applied to a joint driver of the robot, the method comprises: receiving velocities of joints in a joint space of a robot arm, and driving corresponding joint movements according to the velocities of the joints; wherein the velocities of the joints in the joint space of the robot arm are obtained according to the robot arm movement control method of any one of claims 1-28.

30. A robot arm motion control device, characterized by, applied to a main controller of the robot, the device comprises: a singular value decomposition module configured to perform singular value decomposition on a Jacobian matrix representing a mapping relationship between a joint space and a task space, to obtain a set of direction vectors and a sequence of singular values associated with each direction vector in the set of direction vectors; a singular direction determination module configured to determine, based on the sequence of singular values and a sensitivity threshold, a first subset of the set of direction vectors corresponding to non-approaching singular directions and a second subset of the set of direction vectors corresponding to approaching singular directions; an expected velocity decomposition module configured to decompose an input vector representing an expected velocity in the task space on the first subset and the second subset respectively, to obtain a first direction component and a second direction component; a joint velocity generation module configured to perform scaling processing on the second direction component, and to obtain velocities of joints in the joint space based on the first direction component and the scaled second direction component.

31. A robot arm motion control device, characterized by, applied to a joint driver of the robot, the device comprises: a joint movement module configured to receive velocities of joints in a joint space of a robot arm, and to drive corresponding joint movements according to the velocities of the joints; wherein the velocities of the joints in the joint space of the robot arm are obtained according to the robot arm movement control method of any one of claims 1-28.

32. An electronic device, comprising: comprise: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method of any one of claims 1-29.

33. A robot characterized by comprise: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method of any one of claims 1-29. a memory having computer readable instructions stored thereon that, when executed by the processor, implement the method of any of claims 1-29.

34. The robot of claim 33, wherein, The robot comprises any one of a humanoid robot and a dual-arm robot.

35. A computer readable storage medium, characterized in that, The computer readable storage medium has computer program code instructions stored thereon that, when invoked by a processor of the robot, cause the robot to perform the method of any of claims 1-29.

Images