Mechanical arm gravity dynamic compensation control method and system based on multi-modal prediction

Abstract

The invention relates to a mechanical arm gravity dynamic compensation control method and system based on multi-modal prediction, and the method comprises the steps: collecting load information data, including a joint torque value, a tail end acceleration value and a joint angle value, of a mechanical arm in a current operation period through a multi-modal sensor; secondly, preprocessing the data to generate a standardized data set; the trained deep learning model is utilized to predict the load dynamic characteristics of the next operation cycle, and the load mass distribution and the gravity center position change trend are accurately grasped; a gravity compensation torque value is calculated according to a prediction result by combining a mechanical arm dynamic model, so that compensation is more accurate; the gravity compensation torque value is fed back to a control system in real time, the output torque of a driving motor is adjusted, and mechanical arm gravity dynamic compensation control is achieved; according to the scheme, the movement precision of the mechanical arm in the operation process can be effectively improved, positioning deviation and operation errors caused by gravity are reduced, and it is ensured that the mechanical arm stably and efficiently operates in the complex load environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control and robotics, and particularly relates to a mechanical arm gravity dynamic compensation control method and system based on multi-modal prediction. BACKGROUND

[0002] In the field of industrial automation production, the mechanical arm as a kind of key automation equipment is widely used in material handling, part assembly, welding and many other production links. During the operation of the mechanical arm, its load condition is complex and changeable, and the load gravity has a significant influence on the motion accuracy and stability of the mechanical arm. Accurate compensation control of the mechanical arm gravity has become one of the core technologies to ensure the efficient and accurate operation of the mechanical arm.

[0003] At present, some traditional methods calculate the gravity compensation value based on the fixed mechanical arm dynamics model by theoretically analyzing the structure and load of the mechanical arm in the design stage. However, in actual production scenarios, the load carried by the mechanical arm often has uncertainty and dynamic variability. For example, in the assembly line, the weights and centers of gravity of different batches of parts may differ, and the posture of the load may also change during the movement of the mechanical arm.

[0004] These traditional gravity compensation methods based on fixed models cannot adapt to the dynamic changes of the load in real time. When the dynamic changes of the load exceed the preset range of the model, the accuracy of the gravity compensation decreases significantly, resulting in deviation of the motion of the mechanical arm, which further affects the production efficiency and product quality, and cannot meet the requirements of modern industrial automation production for high precision and high stability of the mechanical arm SUMMARY

[0005] The main purpose of the present application is to provide a mechanical arm gravity dynamic compensation control method and system based on multi-modal prediction, which can improve the accuracy of the gravity compensation of the mechanical arm, thereby improving the motion accuracy of the mechanical arm.

[0006] To achieve the above-mentioned purpose, the embodiment of the present application provides a mechanical arm gravity dynamic compensation control method based on multi-modal prediction, which comprises: acquiring load information data collected by a multi-modal sensor in a current operation period of a mechanical arm, the load information data including joint torque values recorded by a torque sensor, mechanical arm end acceleration values recorded by an acceleration sensor, and joint angle values recorded by an angle sensor; preprocessing the load information data to generate a standardized data set corresponding to the operation state of the mechanical arm, the standardized data set containing torque change curves, acceleration change curves and angle change curves in time series; input the standardized data set into the trained deep learning model, and predict the load dynamic characteristics of the robot arm in the next operation cycle based on the deep learning model, the load dynamic characteristics including load mass distribution characteristics and load center of gravity position change characteristics; According to the prediction result of the load dynamic characteristics, the gravity compensation torque value of each joint of the robot arm in the next operation cycle is calculated, which is obtained by combining the load dynamic characteristics through the robot arm dynamics model; The gravity compensation torque value is fed back to the control system of the robot arm in real time, and the output torque of the driving motor of each joint of the robot arm is adjusted to realize the gravity dynamic compensation control of the robot arm in the next operation cycle.

[0007] In summary, by using the technical solution of the present application, the load information data of the robot arm in the current operation cycle is collected through multi-modal sensors, including joint torque value, end acceleration value and joint angle value, which comprehensively reflects the running state of the robot arm. The data is preprocessed to generate a standardized data set, which provides high-quality input for the deep learning model. The trained deep learning model is used to predict the load dynamic characteristics in the next operation cycle, accurately grasp the load mass distribution and center of gravity position change trend. According to the prediction result, the gravity compensation torque value is calculated by combining the robot arm dynamics model, so that the compensation is more accurate. The gravity compensation torque value is fed back to the control system in real time, and the output torque of the driving motor is adjusted to realize the gravity dynamic compensation control of the robot arm. This effectively improves the motion accuracy of the robot arm in operation, reduces the positioning deviation and operation error caused by gravity, improves the production efficiency and product quality, and ensures the stable and efficient operation of the robot arm in complex load environment. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced.

[0009] Figure 1 is a scene diagram of the robot arm gravity dynamic compensation control method based on multi-modal prediction in the embodiments of the present application; Figure 2 The flowchart of the robot arm gravity dynamic compensation control method based on multi-modal prediction is provided for the embodiments of the present application; Figure 3 The flowchart of the load dynamic characteristics prediction is provided for the embodiments of the present application; Figure 4 The flowchart of the local feature generation is provided for the embodiments of the present application; Figure 5 The flowchart of the global feature generation is provided for the embodiments of the present application; Figure 6This is another flowchart illustrating the load dynamic characteristic prediction provided in an embodiment of this application; Figure 7a A schematic diagram illustrating the training process of the machine learning model provided in this application embodiment; Figure 7b A schematic flowchart for training set expansion provided in the embodiments of this application. Figure 8 A schematic diagram illustrating the process of generating gravity compensation torque values ​​provided in this application embodiment; Figure 9 A schematic diagram of the structure of a multimodal prediction-based dynamic gravity compensation control system for a robotic arm provided in an embodiment of this application; Figure 10 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0010] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0011] This application provides a method and system for dynamic gravity compensation control of a robotic arm based on multimodal prediction, which will be described in detail below.

[0012] Taking an automotive parts assembly workshop as an example, this workshop contains a large number of parts of different specifications and shapes that need to be assembled. The robotic arm undertakes the important task of grasping, transporting, and assembling these parts. The workshop environment is complex, and the weight and center of gravity of the parts vary greatly depending on the type. Moreover, during the assembly process, the robotic arm's movement posture is constantly changing, and the load also changes dynamically.

[0013] like Figure 1 As shown, in this scenario, the robotic arm is equipped with torque sensors, acceleration sensors, and angle sensors. Torque sensors are installed at the joints of the robotic arm to record the torque exerted on the joints in real time during movement. For example, when grasping heavy engine parts, the torque exerted on the joints increases significantly, and the torque sensor can accurately detect this change. Acceleration sensors are located at the end effector of the robotic arm and measure the acceleration of the end effector during the handling of parts. When the robotic arm starts or stops rapidly, the acceleration sensor can accurately capture changes in acceleration. Angle sensors are distributed at each joint to record the angle values ​​of the joints and determine the posture of the robotic arm. For example, when inserting parts into specific positions, precise control of the joint angles is crucial, and angle sensors provide accurate angle information.

[0014] The data processing center receives load information data from various sensors. The data processing center centrally processes these data to generate a standardized data set corresponding to the operating state of the robot arm. Then, the data processing center inputs the standardized data set into a deep learning model that has been trained. The model predicts the load dynamic characteristics of the robot arm in the next operating cycle. Based on the prediction result of the load dynamic characteristics, the data processing center calculates the gravity compensation torque value of each joint of the robot arm in the next operating cycle and transmits the gravity compensation torque value to the control system of the robot arm in real time. The control system of the robot arm adjusts the output torque of the joint drive motor according to the received gravity compensation torque value, thereby realizing gravity dynamic compensation control of the robot arm in the next operating cycle and ensuring that the robot arm can accurately complete the assembly task of the parts and avoid assembly errors caused by gravity.

[0015] Reference Figure 2 , Figure 2 is a flowchart of a robot arm gravity dynamic compensation control method based on multi-modal prediction provided by the embodiments of the present application. The execution subject of the method can be a computer device (which can serve as a data processing center). The computer device can be a computer device or a cluster composed of multiple computer devices. The computer device can be a terminal device or a server, etc. The robot arm gravity dynamic compensation control method based on multi-modal prediction provided by the embodiments of the present application specifically includes:

[0016] S10: Obtain load information data collected by multi-modal sensors in the current operating cycle of the robot arm. The load information data includes joint torque values recorded by torque sensors, end acceleration values of the robot arm recorded by acceleration sensors, and joint angle values recorded by angle sensors.

[0017] In the present application, multi-modal sensors refer to multiple sensors of different functional types, which are used to work cooperatively and collect information related to the load of the robot arm from multiple dimensions.

[0018] The torque sensor is a device specially used for measuring the size of the torque acting on the joint of the robot arm. The joint torque value refers to the size of the torsional force that the joint of the robot arm needs to overcome in order to maintain a specific state of the load, such as static, uniform motion or variable speed motion. For example, when the robot arm grasps a heavy cargo, in order to keep the cargo stable and achieve the expected motion, the joint needs to output a larger torque. This torque value can be accurately measured by the torque sensor.

[0019] In this application, the joint torque value refers to the torsional force required for the mechanical arm joint to maintain a specific state of the load (such as static, uniform motion or variable speed motion), which is a key parameter for measuring the force condition of the mechanical arm joint. The joint torque value changes with the weight, center of gravity position of the load and the change of the motion posture of the mechanical arm. For example, when the mechanical arm grabs heavy objects, in order to keep the heavy objects stable and achieve the predetermined motion, the joint needs to output a larger torque, which is accurately measured by the torque sensor. It reflects the strength of the load acting on the joint, helps to understand the force condition of each joint of the mechanical arm, and has important significance for analyzing the load condition of the mechanical arm and optimizing the motion control strategy. Accurate acquisition of joint torque value can ensure that the mechanical arm moves according to the expected posture, and avoid motion deviation and task failure caused by improper torque.

[0020] The acceleration sensor is mainly used to detect the acceleration of the end of the mechanical arm during operation.

[0021] In this application, the end acceleration value of the mechanical arm is a physical quantity that describes the speed change of the end of the mechanical arm, which directly reflects the dynamic characteristics of the end of the mechanical arm during motion. When the mechanical arm performs a task, whether it is starting, stopping, or accelerating, decelerating, the end acceleration value will change accordingly. For example, when the mechanical arm quickly grabs goods, the end quickly reaches a certain speed from static, at this time the acceleration value is large; when approaching the target position to prepare to stop, the end will decelerate, and the size and direction of the acceleration value will also change accordingly. This value not only reflects the change of the motion state of the mechanical arm itself, but also is closely related to the inertia of the load. Through monitoring and analysis of the end acceleration value of the mechanical arm, the influence of the load on the motion of the mechanical arm can be deeply understood, which provides a key basis for precise regulation and control of the motion state of the mechanical arm, helps to optimize the motion planning, avoids problems such as load shaking and inaccurate positioning caused by improper acceleration, and thus improves the stability and accuracy of the motion of the mechanical arm.

[0022] The angle sensor is used to record the angle value of the joint of the mechanical arm, which determines the relative position relationship of each joint of the mechanical arm in space, and different joint angle combinations determine the overall posture of the mechanical arm. For example, in the process of accurately placing goods to a specific position of the goods shelf, accurate control of the joint angle is very important, and the angle sensor can provide accurate angle information to ensure that the mechanical arm moves according to the predetermined trajectory.

[0023] In an embodiment, the torque sensor can adopt a strain gauge torque sensor, which works on the principle that strain gauges will deform when subjected to force, causing a change in resistance value, and the torque size is calculated by measuring the change in resistance value. The acceleration sensor can be a MEMS acceleration sensor, which is made by silicon micromachining technology, and measures acceleration by detecting the inertial force generated by the mass block under acceleration. The angle sensor can use an optical encoder, which converts the rotation angle of the robot joint into a digital signal output through the principle of photoelectric conversion. These sensors collect data in real time according to the set frequency, and transmit the load information data to the data processing unit through high-speed data transmission lines.

[0024] S20: Preprocessing the load information data to generate a standardized data set corresponding to the operating state of the robot arm, the standardized data set containing torque change curves, acceleration change curves and angle change curves over time.

[0025] In this application, preprocessing is the process of preliminary processing of load information data collected from multi-modal sensors, the purpose is to improve data quality, making it more suitable for subsequent analysis and processing. Because the data collected by different types of sensors may differ in dimension, numerical range and data format, and may contain noise or abnormal values and other interference information, preprocessing is to eliminate these differences and interference, making the data consistent and reliable. The standardized data set is a data set formed by arranging according to a specific standard after preprocessing, which contains torque change curves, acceleration change curves and angle change curves over time.

[0026] In this application, the torque change curve is a curve that presents the relationship between joint torque value and time. It takes time as the horizontal axis and joint torque value as the vertical axis, and intuitively displays the dynamic changes of the torque of each joint over time during the operation of the robot arm. For example, during the process of the robot arm grabbing and carrying objects, as the action proceeds, such as different stages of initial grabbing, moving, placing, etc., the joints need to overcome different resistance, and the torque value will change accordingly. These changes are clearly reflected on the torque change curve. By analyzing the curve, the load condition of the robot arm at different times and the force trend of the joints can be understood, helping technicians understand the working state of the robot arm and timely detect abnormal torque fluctuations, providing strong data support for predicting load dynamic characteristics, and helping to adjust the control strategy in advance to ensure stable and efficient operation of the robot arm.

[0027] The acceleration change curve is used to depict the evolution of the acceleration value of the end of the mechanical arm in the time dimension, with time as the horizontal coordinate and the acceleration value of the end of the mechanical arm as the vertical coordinate. The curve reflects the degree of change in the motion state of the end of the mechanical arm. When the mechanical arm is rapidly started, suddenly stopped, or drastically changes the direction of motion, the acceleration value will fluctuate significantly, which is manifested as a large fluctuation in the curve. For example, in the material handling scene, when the mechanical arm quickly approaches the material and quickly leaves the material placement point, the acceleration change curve can clearly show the rapid change of acceleration in the two processes.

[0028] The angle change curve is used to represent the trajectory of the change of the joint angle value of the mechanical arm with time, with time as the horizontal axis and the joint angle value as the vertical axis. It embodies the posture adjustment of each joint of the mechanical arm during operation. Different task requirements will cause different changes in the angle of the joints of the mechanical arm, such as in the assembly task, the joints of the mechanical arm need to be accurately adjusted to align the installation position of the parts, and the angle change curve will present the corresponding accurate fluctuation.

[0029] In an embodiment, first, the collected joint torque value, end acceleration value and joint angle value are data cleaned, and by setting reasonable data threshold and statistical analysis method, abnormal data points obviously deviating from the normal range are removed. Then, a normalization method is used to map the data of different dimensions to the interval [0, 1], for example, for the joint torque value, the normalization is performed by the formula (X - Xmin) / (Xmax - Xmin), where X is the original data value, and Xmin and Xmax are the minimum and maximum values of the data set, respectively. For time series data, a moving average filtering method is used for smoothing to remove noise interference. Specifically, a fixed length time window is set, the average value of the data in the window is calculated, and the average value is used to replace the data point at the center position of the window, thereby obtaining the smoothed time series data. Finally, the processed data is plotted into the torque change curve, the acceleration change curve and the angle change curve in chronological order, to generate a standardized data set.

[0030] S30: inputting the standardized data set into the trained deep learning model, and predicting the load dynamic characteristics of the mechanical arm in the next operation cycle based on the deep learning model, the load dynamic characteristics including load mass distribution characteristics and load center of gravity position change characteristics.

[0031] In this application, the deep learning model is a model based on an artificial neural network architecture, which has strong data processing and pattern recognition capabilities. It can automatically extract complex features and patterns in data through learning a large amount of historical data, thereby establishing a mapping relationship between input data and output results.

[0032] In the embodiments of the present application, the standardized data set is taken as input, and the deep learning model aims to predict the load dynamic characteristics of the robot arm in the next operation cycle. The load mass distribution characteristic describes the distribution of the load mass on the robot arm, for example, whether the load is concentrated at one end of the robot arm or evenly distributed within the working range of the robot arm, which is crucial for analyzing the load weight distribution borne by each joint of the robot arm. The load center of gravity position change characteristic reflects the change of the position of the load center of gravity in space over time as the robot arm moves, which is of key significance to the balance control and motion planning of the robot arm. By predicting these characteristics through the deep learning model, the dynamic change trend of the load can be understood in advance, providing prospective guidance for the control of the robot arm, so that the robot arm can better adapt to the change of the load and improve the precision and stability of motion control.

[0033] In an embodiment, the deep learning model can adopt a combination of Transformer architecture and convolutional neural network (CNN). First, the CNN layer extracts local features from the time series curves in the standardized data set through sliding convolution operation of the convolution kernel on the data, capturing local patterns and features in the data. Then, the extracted local features are input into the Transformer architecture, which can effectively handle long-distance dependencies in time series data through its self-attention mechanism to generate global feature representation. Finally, the global feature representation is mapped to the prediction results of the load mass distribution characteristic and the load center of gravity position change characteristic through the fully connected layer. In the training process, a large amount of historical operation data of the robot arm is used to continuously adjust the parameters of the model through the back propagation algorithm, so that the prediction error of the model gradually decreases until the preset accuracy requirement is met.

[0034] S40: According to the prediction results of the load dynamic characteristics, the gravity compensation torque values of each joint of the robot arm in the next operation cycle are calculated, which are obtained through the robot arm dynamics model combined with the load dynamic characteristics.

[0035] In this application, the mechanical arm dynamics model is based on the principle of mechanics, considering the structure of the mechanical arm, mass distribution, joint motion and external force, etc. The prediction results of load dynamic characteristics, i.e. load mass distribution characteristics and load center of gravity position change characteristics, provide key basis for calculating the gravity compensation torque value. The gravity compensation torque value refers to the additional torque required by each joint to offset the influence of gravity on the movement of the mechanical arm, so that the mechanical arm can move accurately according to the expected trajectory and attitude. For example, when the load center of gravity position changes, it will cause the distribution and size of the gravity acting on each joint of the mechanical arm to change. By calculating the gravity compensation torque value, the driving torque of each joint can be adjusted to maintain the balance and stable movement of the mechanical arm. Accurate calculation of the gravity compensation torque value is crucial to improve the motion accuracy and control performance of the mechanical arm, which can ensure that the mechanical arm can accurately complete the task under different loads and motion attitudes, and avoid position deviation and motion instability caused by gravity factors.

[0036] In an embodiment, the Lagrange dynamics method is used to establish the dynamics model of the mechanical arm. The mechanical arm is regarded as a system composed of multiple rigid bodies connected by joints. According to the Lagrange equation, the mass distribution and center of gravity position of the load are considered to calculate the generalized force of each joint of the mechanical arm. When calculating the gravity compensation torque value, the predicted load dynamic characteristics are substituted into the dynamics model, and the gravity compensation torque value of each joint in the next operation period is obtained by solving the corresponding dynamics equation. Specifically, according to the load mass distribution characteristics, the equivalent mass borne by each joint is determined, and the force arm of gravity acting on each joint is calculated combined with the load center of gravity position change characteristics, and then the gravity compensation torque value of each joint is obtained through the torque calculation formula (torque = force x force arm). This calculation method based on the dynamics model can fully consider the actual structure of the mechanical arm and the load characteristics, and improve the accuracy of the gravity compensation torque value calculation.

[0037] Firstly, the dynamics model of the mechanical arm is established. The Lagrange dynamics method is used to regard the mechanical arm as a system composed of multiple rigid bodies connected by joints. For a mechanical arm with n joints, the mass, inertia tensor and joint variable of each rigid body are defined. According to the Lagrange equation (where the kinetic energy of the system is the potential energy of the system), the dynamics equation of the mechanical arm is derived.

[0038] The system kinetic energy T is the sum of the kinetic energy of each rigid body, which is related to the velocity and angular velocity of each rigid body, and the velocity and angular velocity are related to the joint variables and their derivatives. The system potential energy V is mainly composed of the gravitational potential energy, which is related to the position of each rigid body and the gravity of the load. By taking the partial derivative of L with respect to the joint variables and their derivatives, and combining the definition of generalized force, the dynamics equation of the robot arm is obtained:

[0039] wherein, is the joint torque vector, is the joint variable vector, and are the joint velocity vector and joint acceleration vector, respectively, is the inertia matrix, is the Coriolis force and centrifugal force matrix, is the gravity vector.

[0040] Next, the load dynamic characteristics are combined. The load dynamic characteristics include the load mass distribution characteristics and the load center of gravity position change characteristics. According to the load mass distribution characteristics, the equivalent load mass borne by each joint is determined. For example, by analyzing the distance between the load and each joint and the force transmission relationship, the load mass is proportionally distributed to the corresponding rigid body of each joint, thereby correcting the inertia matrix .

[0041] For the load center of gravity position change characteristics, it is converted into an additional torque acting on each joint. With the change of the load center of gravity position, the direction and size of the gravity acting on the joint will change, resulting in dynamic gravity effect. Through kinematic relationship, the velocity and acceleration of the load center of gravity at different times are calculated, and combined with the load mass, the additional torque of each joint due to the change of the load center of gravity position is calculated by using Newton-Euler equation.

[0042] Finally, the gravity compensation torque value is calculated. In each running period, according to the current load dynamic characteristics, the parameters in the dynamics equation of the robot arm are updated. Then, according to the desired motion state of the robot arm (such as static, uniform motion or specific trajectory motion), the gravity compensation torque value required to be applied to each joint to offset the gravity effect is calculated. For example, when the robot arm is expected to remain static, the joint acceleration and the joint velocity are zero, at this time the gravity compensation torque value is mainly used to balance the gravity vector and the additional torque due to the load dynamic characteristics. In this way, the gravity compensation torque value of each joint of the robot arm in the next running period can be accurately calculated, providing accurate basis for realizing the gravity dynamic compensation control of the robot arm.

[0043] S50: feeding the gravity compensation torque value to the control system of the robot arm in real time, adjusting the output torque of the driving motor of each joint of the robot arm, so as to realize the dynamic gravity compensation control of the robot arm in the next operation cycle.

[0044] In the present application, the control system of the robot arm is responsible for receiving various control signals and accurately controlling the movement of the robot arm according to these signals. The gravity compensation torque value is fed back to the control system of the robot arm in real time, so that the control system adjusts the output torque of the driving motor of each joint of the robot arm according to this information. The driving motor is a device that provides power for the joints of the robot arm. By changing the output torque, the joints of the robot arm rotate accordingly, thereby realizing the movement of the robot arm. The dynamic gravity compensation control of the robot arm in the next operation cycle can be realized by adjusting the output torque of the driving motor, so that the robot arm can compensate for the influence of gravity in real time during operation and maintain a stable and accurate movement state. For example, when the load changes and the gravity acting on the robot arm changes, timely adjustment of the output torque of the driving motor can ensure that the robot arm still moves according to the predetermined trajectory, improve the working efficiency and movement precision of the robot arm, and meet the high-precision requirements in actual production.

[0045] In an embodiment, with reference to Figure 3 , step S30 can specifically include the following process: S301: extracting time series features from the standardized data set, the time series features including torque change rate, acceleration change rate and angle change rate.

[0046] In the present application, the time series features are feature parameters that can reflect the trend and law of data change over time, which are mined from the standardized data set. The torque change rate represents the degree of change of the joint torque value per unit time, reflecting the speed of change of the joint torque during the operation of the robot arm. For example, at the moment when the robot arm grabs the goods, the joint torque may increase rapidly, and the torque change rate can quantify the degree of this rapid change, helping to analyze the dynamic process of the influence of the load on the joint torque. The acceleration change rate is the change amount of the end acceleration of the robot arm per unit time, which shows the degree of change of the movement state of the end of the robot arm. For example, when the robot arm starts or stops quickly, the acceleration change rate can reflect the degree of change of the speed, which helps to understand the change of the movement state of the robot arm and the dynamic influence of the inertia of the load on the acceleration. The angle change rate describes the change of the joint angle of the robot arm per unit time, which is used to analyze the change of the rotation speed of the joint of the robot arm, and is used to determine the speed and process of the posture adjustment of the robot arm.

[0047] In an embodiment, for the moment of force change curve, the moment of force change rate can be obtained by calculating the ratio of the difference of the moment of force values of adjacent time points and the time interval. In order to reduce the influence of noise on the calculation result, the moment of force change curve can be smoothed first, for example, using the Savitzky-Golay filtering method, which realizes smoothing by polynomial fitting of data in a local window. For the acceleration change curve and the angle change curve, the same calculation method is used to obtain the acceleration change rate and the angle change rate, respectively. At the same time, in order to ensure the accuracy and stability of the calculation, the size of the time interval can be reasonably adjusted according to the characteristics of the data and the actual needs.

[0048] S302: input the time series feature into the first convolutional layer of the deep learning model to extract a local feature vector.

[0049] In this application, the first convolutional layer of the deep learning model is a part specially used to extract local features from input data. The convolutional layer can automatically capture local patterns and features in the data through sliding convolution operation of the convolution kernel on the input data. The local feature vector is the result obtained after the convolutional layer processes the time series feature, which contains key information of the time series data in the local range. For the time series feature composed of the moment of force change rate, the acceleration change rate and the angle change rate, the convolution kernel will perform convolution operation on the feature data in each local time window during the sliding process, thereby extracting the combined features of these features in the local time and dimension.

[0050] In an embodiment, the size of the convolution kernel of the first convolutional layer can be selected according to the characteristics of the data and the experimental results, for example, a 3x1 convolution kernel. The time series feature is arranged in a two-dimensional matrix form in a certain order as the input of the convolutional layer. The convolution kernel slides on the input matrix row by row, performs convolution operation on the data in each local time window, and accumulates the results of the convolution operation to obtain a local feature map. Here, the convolution operation can be understood as the operation of multiplying the corresponding elements of the convolution kernel and the data in the local time window and then summing them up. Then, the local feature map is subjected to nonlinear activation processing, for example, using the ReLU (Rectified Linear Unit) function, whose expression is f(x) = max(0, x). Through the ReLU function, the nonlinear expression ability of the feature can be enhanced, so that the model can learn more complex feature relationships and generate a preliminary local feature vector. Finally, the preliminary local feature vector is input to the pooling layer, and the most representative local feature vector is extracted through the max-pooling operation, which is used for global feature generation of the subsequent long short-term memory network layer. The max-pooling operation selects the maximum value in the local region as the output, which can reduce the data dimension while retaining the key features, and reduce the computational complexity.

[0051] In an embodiment, referring to Figure 4 , step S302 can specifically include the following steps: S3021: Obtain time series features in the standardized dataset, including moment change rate, acceleration change rate and angle change rate.

[0052] In an embodiment, moment, acceleration and angle data are read in time sequence from the standardized dataset. For moment data, moment change rate is obtained by calculating the ratio of the difference between adjacent time point moment values and the time interval. For example, assuming that the moment values at time t1 and t2 (t2 > t1) are M1 and M2 respectively, and the time interval is Δt = t2 - t1, then the moment change rate is (M2 - M1) / Δt. In order to improve the accuracy of calculation, smoothing processing can be performed on the moment data, such as using the moving average method, calculating the average value in a short time window, and then calculating the change rate. For acceleration and angle data, the same method is used to calculate the acceleration change rate and the angle change rate respectively. In this way, time series features are accurately obtained from the standardized dataset, laying a foundation for subsequent feature extraction and analysis.

[0053] S3022: Divide the time series features into a plurality of local time windows, each local time window containing fixed length time series data.

[0054] In this application, each local time window contains fixed length time series data, and such division method helps to capture the features and patterns of data in local time period. For example, when analyzing the running state of a mechanical arm, different local time windows may correspond to different motion stages of the mechanical arm, such as start-up stage, stable motion stage or stop stage. By processing the data in each local time window separately, features related to specific motion stages can be extracted more accurately, providing more targeted information for subsequent model training and prediction. Moreover, the fixed length time window setting makes the data consistent and comparable, which is conducive to model learning and analyzing the rules in the data.

[0055] In an embodiment, a suitable local time window length is determined according to the characteristics of the motion of the robot arm and the sampling frequency of the data. For example, if the data sampling frequency is 100 Hz, that is, 100 data points are collected per second, the local time window length can be set to 50 data points, corresponding to a time length of 0.5 seconds. According to this length, a fixed length data segment is sequentially cut off from the starting position of the time series feature as a local time window. For example, for the time series of the moment of force change rate, the first 1 to 50 data points are cut off as the first local time window, the 51st to 100th data points are cut off as the second local time window, and so on. In this way, the entire time series feature is divided into multiple local time windows with a fixed length, providing a suitable data structure for subsequent convolution operations.

[0056] S3023: For each local time window, a sliding convolution operation is performed on the time series data by the convolution kernel of the first convolution layer to generate a local feature map.

[0057] In an embodiment, it is assumed that the convolution kernel size is 3x1 and the step size is 1. For the time series data (such as three-dimensional data composed of the moment of force change rate, the acceleration change rate, and the angle change rate) in a local time window, the convolution kernel starts from the upper left corner of the data, performs multiplication operation with the corresponding 3 data points, and then accumulates the results to obtain a value in the local feature map. Next, the convolution kernel moves one data point to the right according to the step size, and repeats the above operation until the entire width of the local time window is traversed. Then, the convolution kernel moves down one row (in the case of three-dimensional data), and continues the sliding convolution operation, finally generating a local feature map related to the local time window. In this way, the convolution operation is performed on the time series data in each local time window to extract the local features therein and generate the corresponding local feature map.

[0058] S3024: The local feature map is subjected to a nonlinear activation process to generate a preliminary local feature vector.

[0059] In an embodiment, the ReLU function is used to perform nonlinear activation processing on the local feature map. For each element x in the local feature map, the output value y = max (0, x) after ReLU function processing. In this way, all elements in the local feature map are processed by the ReLU function to obtain a new vector, that is, a preliminary local feature vector. Through this nonlinear activation processing, not only the expression ability of the model to the nonlinear features in the data can be enhanced, but also the gradient vanishing problem can be alleviated to some extent, which is conducive to the training and convergence of the model, and lays a foundation for subsequent generation of more representative local feature vectors and accurate prediction of load dynamic characteristics.

[0060] S3025: input the preliminary local feature vector into a pooling layer, and extract the most representative local feature vector by a max-pooling operation, for subsequent global feature generation of a long short-term memory network layer.

[0061] In this application, the max-pooling operation is a method in the pooling layer, which selects the maximum value in the local region as the output. Inputting the preliminary local feature vector into the pooling layer and performing the max-pooling operation can extract the most representative features from the preliminary local feature vector, reduce the data dimension, reduce the calculation amount, and highlight the important feature information in the data.

[0062] In an embodiment, it is assumed that the preliminary local feature vector is divided into a plurality of non-overlapping local regions, each with a size of 2x2 (which can be adjusted according to actual conditions). In each local region, the values of the four elements are compared, and the maximum value is selected as the output of the region. For example, for the elements [a, b, c, d] in a 2x2 local region, the output value after the max-pooling operation is max (a, b, c, d). By performing the max-pooling operation on all such local regions in the preliminary local feature vector, a vector with reduced dimension but retaining key features is obtained, i.e., the most representative local feature vector.

[0063] S303: input the local feature vector into the long short-term memory network layer of the deep learning model to generate a global feature vector.

[0064] In this application, the long short-term memory network layer (LSTM) is a special network structure in the deep learning model that is good at processing long-term dependencies of time series data. Although the local feature vector contains key features of the time series data in the local region, in order to comprehensively understand the information contained in the entire time series, the LSTM layer is needed to integrate these local features. The global feature vector is the result generated by the LSTM layer after processing the local feature vector, which integrates the information of the time series at different time steps and can reflect the overall features and long-term trends of the data.

[0065] In an embodiment, with reference to Figure 5 , step S303 can be implemented in the following way: S3031: arrange the local feature vector in time sequence to form a local feature sequence.

[0066] In the present application, the position of each local feature vector in the sequence corresponds to its order on the time axis of the operation of the robot arm by arranging in chronological order. In this way, the LSTM layer can process the local features in chronological order, understand the evolution of the robot arm operating state over time, and then more accurately generate the global feature vector, ultimately improving the prediction accuracy of the dynamic characteristics of the load.

[0067] In an embodiment, it is assumed that a series of local feature vectors have been obtained, for example, the local feature vectors v1, v2, v3, …, vn are generated in chronological order during a complete operation of the robot arm. These vectors are arranged in sequence [v1, v2, v3, …, vn], where v1 corresponds to the local feature at the early stage of the robot arm operation, and vn corresponds to the local feature at the later stage. Through this arrangement in chronological order, a local feature sequence is formed as input to the long short-term memory network layer, so that the LSTM layer performs subsequent feature processing and analysis based on the time dimension.

[0068] S3032: input the local feature sequence into the long short-term memory network layer, and capture the long-term dependence in the time sequence through the memory unit of the network layer.

[0069] In the present application, the memory unit in the long short-term memory network layer (LSTM) is the core component that can effectively process the long-term dependence of time series data. When the local feature sequence is input into the LSTM layer, the memory unit selectively remembers and forgets the input local features through its unique gating mechanism, thereby capturing the long-term dependence in the time sequence. This long-term dependence reflects the correlation of the robot arm operating state over a long time span, for example, how the load changes at different stages of the robot arm affect the subsequent operating state.

[0070] In an embodiment, the memory unit includes an input gate, a forget gate, and an output gate. When a local feature vector in the local feature sequence is input, the input gate determines how much information of the current input local feature can enter the memory unit; the forget gate determines how much previously stored information in the memory unit needs to be retained; and the output gate determines the information output to the next layer according to the current state of the memory unit. For example, when the load state of the robot arm changes slowly, the forget gate may retain more previous memory information, while the input gate allows new local feature information to enter to update the state of the memory unit, thereby accurately capturing the long-term dependence in the time sequence.

[0071] S3033: update the state value of the memory unit at each time step, the state value including the activation values of the input gate, the forget gate, and the output gate.

[0072] In the present application, in the process of processing the local feature sequence by the long short-term memory network layer (LSTM), each time step corresponds to processing a local feature vector in the local feature sequence. At each time step, the memory unit updates its state value according to the current input local feature vector and the state value at the last time, wherein the state value includes the activation values of the input gate, the forgetting gate and the output gate. The activation values of these gates determine the processing method of the memory unit to the information, the input gate activation value controls the degree of current input information entering the memory unit, the forgetting gate activation value determines the proportion of retaining the memory information at the last time, and the output gate activation value determines the information content output by the memory unit to the next layer. By continuously updating these state values, the memory unit can dynamically adapt to the input local feature sequence, accurately capture the long-term dependence in the time sequence, and continuously adjust its state to reflect the change of the robot arm running state over time.

[0073] In an embodiment, it is assumed that the local feature vector input at the current time step is xt, the state value of the memory unit at the last time is Ct-1, and the hidden state is ht-1. The activation value it of the input gate is calculated by a linear transformation including a weight matrix and a Sigmoid function: where Wix and Wih are weight matrices, and bi is a bias term. The activation value ft of the forgetting gate is calculated in a similar way: The activation value ot of the output gate is also calculated by a linear transformation and a Sigmoid function: According to these activation values, the memory unit updates its state value Ct, thereby dynamically processing the input local feature vector at each time step and capturing the long-term dependence in the time sequence.

[0074] S3034: generating a hidden state vector at the current time step based on the state value of the memory unit.

[0075] In the present application, the hidden state vector is a vector generated by the long short-term memory network layer (LSTM) at each time step according to the state value of the memory unit, which integrates the current input local feature and the historical information stored in the memory unit. The hidden state vector contains the key feature representation of the robot arm running state at the current time step, which reflects both the information carried by the current local feature vector and the historical information captured by the memory unit through long-term dependence. By generating the hidden state vector, the LSTM layer can effectively transmit the state information of the memory unit to the next layer, providing an important intermediate representation for subsequent generation of global feature vectors, which helps to comprehensively and accurately describe the features of the robot arm running state in the time sequence, thereby improving the prediction ability of the load dynamic characteristics.

[0076] In an embodiment, the hidden state vector ht of the current time step is generated based on the state value Ct of the memory cell and the activation value ot of the output gate. First, a nonlinear transformation is performed on the state value Ct of the memory cell, for example, by a tanh function: Ct' = tanh (Ct). Then, the hidden state vector ht is obtained by multiplying the transformed memory cell state value Ct' by the output gate activation value ot: The hidden state vector ht generated in this way not only contains important information related to the current time step in the memory cell, but also selectively outputs features valuable for subsequent processing through the control of the output gate, providing a key intermediate step for generating the global feature vector.

[0077] S3035: The hidden state vectors of all time steps are spliced to form a global feature vector, which is used to output the load dynamic characteristic prediction result of the fully connected layer of the deep learning model.

[0078] In an embodiment, it is assumed that n hidden state vectors h1, h2, …, hn of time steps are obtained in the process of processing the local feature sequence. These hidden state vectors are spliced in order to form a higher-dimensional vector H = [h1, h2, …, hn], and this vector H is the global feature vector. Then the global feature vector H is input to the fully connected layer of the deep learning model, and the fully connected layer processes it through linear transformation and nonlinear activation function to further extract high-level features, and finally outputs the prediction result of the load dynamic characteristic, such as the load mass distribution characteristic and the load center of gravity position change characteristic, to help the robot better cope with the load dynamic change and realize accurate gravity compensation control.

[0079] S304: Based on the global feature vector, the fully connected layer of the deep learning model outputs the load dynamic characteristic of the robot in the next operation cycle, including the load mass distribution characteristic and the load center of gravity position change characteristic.

[0080] In this application, the fully connected layer is the part of the deep learning model that integrates the features extracted by the previous layers and outputs the final prediction result. The global feature vector contains long-term dependency information in the time series, and the fully connected layer processes the global feature vector through linear transformation and nonlinear activation function to further extract high-level features, thereby generating the prediction result of the load dynamic characteristic.

[0081] In an embodiment, with reference to Figure 6 Step S304 can be specifically implemented in the following manner: S3041: Obtain the global feature vector, which contains long-term dependency information in the time series.

[0082] In this application, the long-term dependency information in the global feature vector reflects the correlation between the running states of the robot arm at different time points, such as how the early changes in the load state affect the subsequent movement, and the interaction of factors such as joint angle, torque, and acceleration over a long period of time. By obtaining this global feature vector, the deep learning model can comprehensively understand the dynamic changes in the robot arm running process.

[0083] S3042: input the global feature vector into the first hidden layer of the fully connected layer, and generate an intermediate feature representation through linear transformation and nonlinear activation function.

[0084] In this application, the first hidden layer of the fully connected layer further processes the input global feature vector, and generates an intermediate feature representation through linear transformation and nonlinear activation function. Linear transformation maps the global feature vector to a new feature space by multiplying the global feature vector by a weight matrix and adding a bias term, changing the dimension and representation of the feature. Nonlinear activation function introduces nonlinear factors on this basis, enhances the expression ability of the model, and enables the model to learn more complex feature relationships.

[0085] In an embodiment, assuming that the global feature vector is x, the weight matrix of the first hidden layer is W1, and the bias term is b1. First, linear transformation is performed to obtain Then, the result y of linear transformation is input to a nonlinear activation function such as a ReLU function to obtain an intermediate feature representation z = ReLU (y). Through such operations, the global feature vector is processed by linear transformation and nonlinear activation in the first hidden layer to generate a more representative intermediate feature representation.

[0086] S3043: input the intermediate feature representation into the second hidden layer of the fully connected layer to further extract high-level features.

[0087] In this application, the intermediate feature representation contains a certain degree of abstract features after processing by the first hidden layer, but in order to more accurately predict the load dynamic characteristics, it is necessary to further mine and refine the features in the second hidden layer. By performing linear transformation and nonlinear activation operations again, the second hidden layer can capture more complex and higher-level feature relationships, which are crucial for accurately describing the load mass distribution characteristics and load center of gravity position change characteristics, and help to improve the prediction accuracy of the model for load dynamic characteristics.

[0088] In an embodiment, assuming that the intermediate feature representation is z, the weight matrix of the second hidden layer is W2, and the bias term is b2. First, linear transformation is performed to obtain Then, the a is input to a nonlinear activation function, such as a sigmoid function, to obtain a high-level feature representation s = Sigmoid(a). Through this series of operations, the second hidden layer extracts higher-level features from the intermediate feature representation, which can more accurately reflect the relationship between the load dynamic characteristics and the robot operating state, and provide key support for finally generating an accurate load dynamic characteristic prediction result.

[0089] S3044: Based on the high-level features, a load dynamic characteristic prediction result is generated through an output layer of the fully connected layer, and the load dynamic characteristics include load mass distribution characteristics and load center of gravity position change characteristics.

[0090] In this application, the output layer of the fully connected layer generates a load dynamic characteristic prediction result based on the high-level features extracted by the second hidden layer. The output layer maps the high-level features to the dimension space of the load dynamic characteristics through linear transformation, so as to obtain the predicted values of the load mass distribution characteristics and the load center of gravity position change characteristics. These prediction results are learned by the deep learning model according to the time series data of the robot operating state, and reflect the prediction of the future dynamic change of the load.

[0091] In an embodiment, let the high-level feature representation be s, the weight matrix of the output layer be W3, and the bias term be b3. Through linear transformation, the prediction result is obtained where p includes the predicted values of the load mass distribution characteristics and the load center of gravity position change characteristics. For example, the prediction result p can be a vector, in which part of the elements represent the related parameters of the load mass distribution, and the other part of the elements represent the predicted coordinates or parameters of the load center of gravity position change. In this way, the output layer of the fully connected layer generates a load dynamic characteristic prediction result based on the high-level features, and provides key decision information for the actual operation control of the robot.

[0092] Reference Figure 7a The training process of the deep learning model of the embodiment of the application includes the following steps: S60: Obtain load information data of the robot in a plurality of historical operating periods, and the load information data includes joint torque values recorded by the torque sensor, end acceleration values of the robot recorded by the acceleration sensor, and joint angle values recorded by the angle sensor.

[0093] In this application, the load information data in a plurality of historical operating periods refers to a data set collected by the torque sensor, the acceleration sensor and the angle sensor to reflect the load state of the robot in a series of past operating periods.

[0094] In an embodiment, in order to obtain these data, a suitable sampling frequency can be set during the long-term operation of the robot arm, for example, 50 times per second, to ensure that subtle changes in the operation state of the robot arm can be captured.

[0095] S61: labeling the load information data to generate a training data set, and the labeling information in the training data set includes load mass distribution characteristics and load center of gravity position change characteristics.

[0096] In an embodiment, the detailed physical parameters of the load carried by the robot arm in each historical operation period are first obtained, which can be obtained by consulting the product specification of the load, measuring the mass of the load using a weighing device, and obtaining the coordinates of the center of gravity of the load using three-dimensional laser scanning technology. Based on these load physical parameters, the theoretical load mass distribution characteristics and the theoretical load center of gravity position change characteristics of the robot arm in each historical operation period are calculated using mechanical analysis software or theoretical calculation formula. For example, according to the shape, mass of the load and the motion posture of the robot arm, the change of the center of gravity of the load is determined by the centroid calculation formula. Then, these theoretical load dynamic characteristics are used as labeling information, and are associated with the corresponding load information data to form a preliminary training data set.

[0097] S62: dividing the training data set into a training subset and a validation subset, training the initial deep learning model using the training subset, and evaluating the performance of the trained deep learning model using the validation subset.

[0098] In this application, the training subset is used to train the initial deep learning model, and the model adjusts its parameters in the process, learns the relationship between the load information data and the labeling information in the training data set, and gradually optimizes the performance of the model. The validation subset is used to evaluate the performance of the model after training, which can test the prediction ability of the model on unseen data and help to judge whether the model is overfitting or underfitting.

[0099] In an embodiment, the training data set is divided into a training subset and a validation subset in a ratio of 7:3 using stratified sampling method. In this way, the training subset and the validation subset can have similarity in data distribution, i.e. containing data samples of various load conditions and robot arm operation states. When training the initial deep learning model using the training subset, a stochastic gradient descent algorithm is used to update the model parameters. Each time a batch of data is randomly selected from the training subset as a training batch, and the model adjusts the parameters according to the error between the prediction results of the batch of data and the labeling information. During the training process, the model is evaluated periodically using the validation subset, and the prediction error of the model on the validation subset is calculated, such as mean square error (MSE), whose calculation formula is Where yi is the actual value, Here, n is the predicted value, and n is the number of samples. The model is considered to have achieved good training results when the prediction error on the validation subset no longer decreases significantly.

[0100] S63: Based on the performance evaluation results, adjust the hyperparameters of the deep learning model until the prediction error of the deep learning model meets the preset accuracy requirements.

[0101] In this application, the hyperparameters of the deep learning model are parameters pre-set before model training, which determine the model's structure and training method. Performance evaluation results reflect the model's current predictive ability, and adjusting the hyperparameters based on this can optimize model performance. For example, hyperparameters such as the number of layers in the neural network, the number of neurons, and the learning rate all affect the model's complexity and learning speed.

[0102] In one embodiment, a hyperparameter search space can be established, including different learning rates (e.g., 0.001, 0.01, 0.1), the number of neural network layers (e.g., 2, 3, 4 layers), and the number of neurons (e.g., 10, 20, 30). A random search algorithm is used to randomly select hyperparameter combinations from this search space for model training and validation. After each training iteration, the prediction error of the model on the validation subset is recorded. Based on the results of multiple trials, the hyperparameter combination with the smallest prediction error and meeting the preset accuracy requirements is selected as the final model hyperparameter settings.

[0103] In one embodiment, reference Figure 7b In this embodiment of the application, step S610 can be implemented in the following way: S610: Obtain the load physical parameters of the robotic arm in each historical operating cycle, the load physical parameters including the load mass value and the load center of gravity position coordinates.

[0104] In this application, the physical parameters of the load are crucial parameters describing the inherent properties and spatial position characteristics of the load. The load mass directly determines the weight carried by the robotic arm; different load masses cause the joints of the robotic arm to experience varying degrees of force. For example, when the robotic arm handles large equipment, the load mass is large, and the joints need to withstand greater torque to maintain balance and achieve movement; while when handling small parts, the load mass is small, and the torque on the joints is relatively small. The coordinates of the load's center of gravity determine the position of the equivalent point of application of the load's gravity in space, which changes with the movement of the robotic arm and the change of the load's own posture. For example, when the robotic arm grasps an irregularly shaped object and changes its posture, the position of the load's center of gravity will change accordingly.

[0105] In an embodiment, for a regular-shaped load with uniform mass distribution, the load mass value can be obtained by consulting the product specification manual, and the load barycenter position coordinates can be calculated according to the geometric shape. For an irregularly-shaped load or a load with non-uniform mass distribution, a high-precision weighing device is used to directly measure the load mass value. For the acquisition of the load barycenter position coordinates, a suspension method can be used, that is, the load is suspended by a thin line, and under the action of gravity, the intersection point of the extension line of the suspension line is marked to determine the position of the barycenter in the two-dimensional plane. The suspension point is changed several times to finally determine the barycenter position coordinates in the three-dimensional space. Advanced three-dimensional laser scanning technology can also be used to obtain three-dimensional point cloud data of the load, and the load barycenter position coordinates can be accurately calculated by processing these data through an algorithm.

[0106] S611: Based on the load physical parameters, the theoretical load dynamic characteristics of the mechanical arm in each historical running period are calculated, including the theoretical load mass distribution characteristics and the theoretical load barycenter position change characteristics.

[0107] In this application, the theoretical load dynamic characteristics are characteristics reflecting the dynamic changes of the load during the operation of the mechanical arm, which are obtained through theoretical analysis and calculation based on the load physical parameters. The theoretical load mass distribution characteristics describe the theoretical distribution of the load mass on the mechanical arm. For example, if the load mass is concentrated at one end of the mechanical arm, the joints near the end will bear more load pressure. The theoretical load barycenter position change characteristics reflect the theoretical position change of the load barycenter in space with the movement of the mechanical arm, which plays a key role in the balance control and motion planning of the mechanical arm. By calculating the theoretical load dynamic characteristics, accurate labeling information can be provided for the training data set, so that the deep learning model can learn the internal rules of the load dynamic changes, thereby improving the accuracy and reliability of the model in predicting the load dynamic characteristics.

[0108] S612: The theoretical load dynamic characteristics are associated with the corresponding load information data as labeling information to generate a preliminary training data set.

[0109] In an embodiment, the load information data (joint torque value, end acceleration value, joint angle value) in each historical running period is one-to-one corresponding to the corresponding theoretical load mass distribution characteristics and theoretical load center of gravity position change characteristics, indexed by time sequence. A database management system can be used to store these data, creating a data table, where each row represents a data record of a historical running period, and each column corresponds to different load information data and annotation information. For example, the first column stores the joint torque value, the second column stores the end acceleration value, the third column stores the joint angle value, and the fourth and fifth columns store the related parameters of the theoretical load mass distribution characteristics and the theoretical load center of gravity position change characteristics, respectively. Such a data storage structure facilitates data reading and processing during subsequent model training.

[0110] S613: performing data enhancement processing on the preliminary training data set to generate an expanded training data set, the data enhancement processing including data smoothing processing and noise injection processing.

[0111] In this application, data enhancement processing aims to increase the diversity and richness of data by a series of operations on the preliminary training data set, thereby improving the generalization ability of the deep learning model.

[0112] In an embodiment, for data smoothing processing, a Gaussian smoothing filter is used to process the data. According to the characteristics of the data and the noise level, a suitable Gaussian kernel size is selected. For example, for the time series data of joint torque value, assuming that the noise of the data is mainly concentrated in the high frequency part, a Gaussian kernel with a size of 5 is selected, and its weight distribution conforms to the Gaussian function. The Gaussian kernel is convolved with the time series of joint torque value to smooth each data point, making the change between adjacent data points more gradual and effectively removing noise. Similar methods are used for acceleration and angle data smoothing.

[0113] S614: dividing the expanded training data set into a training subset and a validation subset for training and evaluation of the deep learning model.

[0114] In an embodiment, stratified sampling method is adopted for the division. According to the distribution of different load conditions and robot operating states in the extended training dataset, the dataset is divided into a training subset and a validation subset in a certain proportion (such as 7:3). In this way, the training subset and the validation subset can be ensured to have similarity in data distribution, that is, both subsets contain data samples of various different load masses, center of gravity positions, and robot motion states. In the model training process, the training subset is used to perform multiple iterations of training on the model, and after each iteration, the performance of the model is evaluated using the validation subset to observe the changes in the prediction error, accuracy, and other indicators of the model on the validation subset. According to the evaluation results, the training strategy of the model is adjusted, such as adjusting the learning rate and optimizing the network structure, until the performance of the model on the validation subset reaches a satisfactory level, ensuring that the model can accurately predict the load dynamic characteristics in actual application.

[0115] In an embodiment, reference is made to Figure 8 , step S50 can be specifically implemented by the following manner: S501: based on the load mass distribution characteristics in the load dynamic characteristics, the static gravity compensation torque value of each joint of the robot arm is calculated.

[0116] In this application, the load mass distribution characteristics reflect the distribution of the load mass on the robot arm. The static gravity compensation torque value refers to the compensation torque required to be applied by each joint to offset the torque generated by the joint due to the action of the load gravity when the robot arm is in a static or uniform motion state. For example, when the load mass is concentrated at one end of the robot arm, the joints near that end need to bear greater static gravity, and accordingly, greater static gravity compensation torque is needed to maintain balance. By accurately analyzing the load mass distribution characteristics, the static gravity compensation torque value required by each joint can be more accurately calculated, thereby effectively offsetting the influence of gravity on the joints of the robot arm and improving the stability and positioning accuracy of the robot arm in a static or uniform motion state.

[0117] In an embodiment, according to the structural model of the robot arm, the robot arm is divided into multiple rigid links, each link corresponding to a joint. Based on the load mass distribution characteristics, the equivalent load mass borne by each joint is determined. For example, for a robot arm with six joints, by analyzing the distance between the load and each joint and the force transmission relationship, the load mass is proportionally distributed to each joint using statics principles. Then, according to the distance from each joint to the equivalent center of mass of the load and the gravitational acceleration, the static gravity compensation torque value of each joint is calculated. The specific calculation formula is: static gravity compensation torque value = equivalent load mass borne by the joint × gravitational acceleration × distance from the joint to the equivalent center of mass of the load.

[0118] S502: Calculate dynamic gravity compensation torque values of each joint of the robot arm based on the load center of gravity position change characteristic in the load dynamic characteristic.

[0119] In this application, the load center of gravity position change characteristic describes the dynamic change of the load center of gravity in space as the robot arm moves. The dynamic gravity compensation torque value is the torque required to be applied to each joint to compensate for the additional force on the joint due to the change in the load center of gravity position. When the robot arm moves, the change in the load center of gravity position will cause the direction and magnitude of the gravity force on the joint to change, resulting in dynamic gravity effects. For example, when the robot arm starts, stops or changes direction quickly, the inertia of the load center of gravity will cause the joint to be subjected to additional force. By calculating the dynamic gravity compensation torque value based on the load center of gravity position change characteristic, the dynamic gravity effect can be effectively eliminated during the movement of the robot arm, ensuring the smoothness and accuracy of the movement of the robot arm, and avoiding movement deviation caused by dynamic gravity change.

[0120] In an embodiment, a dynamic model of the robot arm is established, considering the kinematic parameters of the robot arm (such as joint angle, angular velocity, angular acceleration) and the load center of gravity position change characteristic. The load center of gravity position change is converted into the torque acting on each joint using Lagrange equation or Newton-Euler equation. Specifically, first, the velocity and acceleration of the load center of gravity at different times are calculated according to the kinematic relationship of the robot arm. Then, combined with the load mass and the change of the center of gravity position, the dynamic gravity compensation torque value of each joint due to the change of the load center of gravity position is calculated through the dynamic equation. For example, when the robot arm rotates, the centrifugal force and Coriolis force of the load center of gravity will generate additional torque on the joint, which can be accurately calculated through the dynamic model and taken as part of the dynamic gravity compensation torque value. The dynamic gravity compensation torque value calculated in this way can compensate for the change in the force on the joint due to the change in the load center of gravity position in real time, ensuring the accuracy and stability of the movement of the robot arm.

[0121] S503: Add the static gravity compensation torque value and the dynamic gravity compensation torque value to obtain the gravity compensation torque value of each joint of the robot arm in the next operation cycle.

[0122] In one embodiment, before the start of each operating cycle, the static gravity compensation torque value and the dynamic gravity compensation torque value are calculated based on the load mass distribution characteristics and load center of gravity position change characteristics predicted in the previous cycle. For each joint, the corresponding static gravity compensation torque value and dynamic gravity compensation torque value are simply added together, i.e.: Gravity compensation torque value = Static gravity compensation torque value + Dynamic gravity compensation torque value. For example, for joint i of the robotic arm, the calculated static gravity compensation torque value M_static_i and dynamic gravity compensation torque value M_dynamic_i are added together to obtain M_gravity_compensation_i = M_static_i + M_dynamic_i, which is the gravity compensation torque value of joint i in the next operating cycle. This comprehensively calculated gravity compensation torque value can more comprehensively reflect the influence of load gravity on the joint, providing an accurate basis for subsequent adjustment of the output torque of the drive motors of each joint of the robotic arm, thereby achieving more precise dynamic gravity compensation control.

[0123] In one embodiment, step S503 can be implemented as follows: A: The gravity compensation torque value is filtered to generate a gravity compensation torque signal that meets the input requirements of the robotic arm control system.

[0124] In one embodiment, a Butterworth low-pass filter is used to filter the gravity compensation torque value. A suitable cutoff frequency is selected based on the bandwidth requirements and noise frequency characteristics of the robotic arm control system. For example, if the robotic arm control system primarily focuses on low-frequency signals and the noise is mainly concentrated in the high-frequency range, a Butterworth low-pass filter with a cutoff frequency of 10Hz can be selected.

[0125] B: The gravity compensation torque signal is sent to the control system of the robotic arm, instructing the drive motors of each joint of the robotic arm to adjust the output torque according to the gravity compensation torque signal.

[0126] In one embodiment, the gravity compensation torque signal is transmitted to the control system of the robotic arm via a high-speed, reliable communication link. The controller in the control system samples and quantizes the received gravity compensation torque signal, converting it into a digital signal. Then, the controller, according to a preset control algorithm, parses the digital signal into specific control commands for the drive motors of each joint. For example, the controller can use a proportional-integral-derivative (PID) control algorithm to calculate the control quantity that needs adjustment based on the difference between the gravity compensation torque signal and the current motor output torque. By adjusting the input voltage or current of the drive motor, the output torque of the drive motor is changed.

[0127] Correspondingly, in order to better implement the above method, the embodiment of the present application also provides a mechanical arm gravity dynamic compensation control system 9 based on multi-modal prediction. As shown in Figure 9 The mechanical arm gravity dynamic compensation control system 80 based on multi-modal prediction includes:

[0128] The acquisition module 801 is configured to acquire load information data collected by a multi-modal sensor in a current operation period of the mechanical arm, the load information data including a joint torque value recorded by a torque sensor, an end acceleration value of the mechanical arm recorded by an acceleration sensor, and a joint angle value recorded by an angle sensor. The preprocessing module 802 is configured to preprocess the load information data to generate a standardized data set corresponding to the operation state of the mechanical arm, the standardized data set including a torque change curve, an acceleration change curve, and an angle change curve in a time sequence. The load prediction module 803 is configured to input the standardized data set into a deep learning model trained to predict load dynamic characteristics of the mechanical arm in a next operation period based on the deep learning model, the load dynamic characteristics including load mass distribution characteristics and load center of gravity position change characteristics. The compensation calculation module 804 is configured to calculate gravity compensation torque values of each joint of the mechanical arm in the next operation period according to the prediction results of the load dynamic characteristics, the gravity compensation torque values being derived from the load dynamic characteristics through a mechanical arm dynamics model. The feedback module 805 is configured to feed back the gravity compensation torque values to a control system of the mechanical arm in real time to adjust the output torque of the driving motor of each joint of the mechanical arm, so as to realize gravity dynamic compensation control of the mechanical arm in the next operation period.

[0129] The implementation of each module can be specifically referred to the method embodiments described above, and will not be repeated here. The technical effects of each module and device are described in the foregoing method embodiments.

[0130] As shown in Figure 10 The embodiment of the present application also provides a computer device 90 including a processor 901 and a memory 902, wherein the memory 902 stores a computer program, and when the computer program is executed by the processor 901, the processor 901 executes the steps of the method described above.

[0131] The above disclosure is only the preferred embodiment of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. A method for dynamic gravity compensation control of a robotic arm based on multimodal prediction, characterized in that, Includes the following steps: The load information data collected by the robotic arm through multimodal sensors during the current operating cycle is acquired. The load information data includes the joint torque value recorded by the torque sensor, the end-effector acceleration value recorded by the acceleration sensor, and the joint angle value recorded by the angle sensor. The load information data is preprocessed to generate a standardized dataset corresponding to the operating state of the robotic arm. The standardized dataset includes torque change curves, acceleration change curves, and angle change curves in a time series. The standardized dataset is input into the trained deep learning model, and the load dynamic characteristics of the robotic arm in the next operating cycle are predicted based on the deep learning model. The load dynamic characteristics include load mass distribution characteristics and load center of gravity position change characteristics. Based on the predicted results of the load dynamic characteristics, the gravity compensation torque value of each joint of the robotic arm in the next operating cycle is calculated. The gravity compensation torque value is obtained by combining the robotic arm dynamic model with the load dynamic characteristics. The gravity compensation torque value is fed back to the control system of the robotic arm in real time, and the output torque of the drive motor of each joint of the robotic arm is adjusted to realize the dynamic gravity compensation control of the robotic arm in the next operating cycle.

2. The method according to claim 1, characterized in that, The standardized dataset is input into the trained deep learning model, and the load dynamics of the robotic arm in the next operating cycle are predicted based on the deep learning model, including: Time series features are extracted from the standardized dataset, including the rate of change of torque, the rate of change of acceleration, and the rate of change of angle. The time series features are input into the first convolutional layer of the deep learning model to extract local feature vectors. The local feature vectors are input into the long short-term memory network layer of the deep learning model to generate global feature vectors; Based on the global feature vector, the fully connected layer of the deep learning model outputs the load dynamic characteristics of the robotic arm in the next operating cycle. The load dynamic characteristics include load mass distribution characteristics and load center of gravity position change characteristics.

3. The method according to claim 2, characterized in that, The training process of the deep learning model includes the following steps: The load information data of the robotic arm in multiple historical operating cycles is acquired. The load information data includes the joint torque value recorded by the torque sensor, the end-effector acceleration value recorded by the acceleration sensor, and the joint angle value recorded by the angle sensor. The load information data is labeled to generate a training dataset. The labeled information in the training dataset includes load quality distribution characteristics and load center of gravity position change characteristics. The training dataset is divided into a training subset and a validation subset. The initial deep learning model is trained using the training subset, and the performance of the trained deep learning model is evaluated using the validation subset. Based on the performance evaluation results, the hyperparameters of the deep learning model are adjusted until the prediction error of the deep learning model meets the preset accuracy requirements.

4. The method according to claim 3, characterized in that, The load information data is labeled to generate a training dataset, including: The load physical parameters of the robotic arm in each historical operating cycle are obtained, including the load mass value and the coordinates of the load center of gravity. Based on the physical parameters of the load, the theoretical load dynamic characteristics of the robotic arm in each historical operating cycle are calculated. The theoretical load dynamic characteristics include the theoretical load mass distribution characteristics and the theoretical load center of gravity position change characteristics. The theoretical load dynamic characteristics are used as annotation information and associated with the corresponding load information data to generate a preliminary training dataset. The initial training dataset is subjected to data augmentation processing to generate an expanded training dataset. The data augmentation processing includes data smoothing and noise injection processing. The extended training dataset is divided into a training subset and a validation subset for training and evaluating deep learning models.

5. The method according to claim 1, characterized in that, Based on the predicted results of the load dynamic characteristics, the gravity compensation torque values ​​of each joint of the robotic arm in the next operating cycle are calculated, including: Based on the load mass distribution characteristics in the load dynamic characteristics, the static gravity compensation torque values ​​of each joint of the robotic arm are calculated. Based on the load center of gravity position change characteristics in the load dynamic characteristics, the dynamic gravity compensation torque values ​​of each joint of the robotic arm are calculated. The static gravity compensation torque value and the dynamic gravity compensation torque value are added together to obtain the gravity compensation torque value of each joint of the robotic arm in the next operating cycle.

6. The method according to claim 5, characterized in that, The gravity compensation torque value is fed back to the control system of the robotic arm in real time, including: The gravity compensation torque value is filtered to generate a gravity compensation torque signal that meets the input requirements of the robotic arm control system. The gravity compensation torque signal is sent to the control system of the robotic arm, instructing the drive motors of each joint of the robotic arm to adjust their output torque according to the gravity compensation torque signal.

7. The method according to any one of claims 2 to 6, characterized in that, The first convolutional layer of the deep learning model extracts local feature vectors through the following steps: Obtain time-series features from a standardized dataset, including the rate of change of torque, the rate of change of acceleration, and the rate of change of angle. The time series features are divided into multiple local time windows, each containing time series data of a fixed length. For each local time window, a sliding convolution operation is performed on the time series data through the convolution kernel of the first convolutional layer to generate a local feature map; The local feature map is subjected to nonlinear activation processing to generate a preliminary local feature vector; The initial local feature vector is input into the pooling layer, and the most representative local feature vector is extracted through max pooling operation, which is then used for global feature generation in the subsequent long short-term memory network layer.

8. The method according to claim 7, characterized in that, The local feature vectors are input into the Long Short-Term Memory (LSTM) network layer of the deep learning model to generate global feature vectors. Includes the following steps: The local feature vectors are arranged in chronological order to form a local feature sequence; The local feature sequence is input into a long short-term memory network layer, and the long-term dependencies in the time series are captured through the memory units of the network layer. At each time step, the state value of the memory cell is updated, and the state value includes the activation values ​​of the input gate, forget gate, and output gate; Based on the state value of the memory unit, the hidden state vector of the current time step is generated; The hidden state vectors of all time steps are concatenated to form a global feature vector, which is used to predict the load dynamic characteristics of the fully connected layer output of the deep learning model.

9. The method according to claim 8, characterized in that, The step of using the fully connected layer of a deep learning model to output the load dynamic characteristics of the robotic arm in the next operating cycle based on the global feature vector includes the following steps: Obtain the global feature vector, which contains long-term dependency information in the time series; The global feature vector is input into the first hidden layer of the fully connected layer, and intermediate feature representations are generated through linear transformation and nonlinear activation functions. The intermediate feature representation is input into the second hidden layer of the fully connected layer to further extract high-level features; Based on the high-level features, load dynamic characteristics prediction results are generated through the output layer of the fully connected layer. The load dynamic characteristics include load mass distribution characteristics and load center of gravity position change characteristics.

10. A dynamic gravity compensation control system for a robotic arm based on multimodal prediction, characterized in that, The system includes: The acquisition module is used to acquire load information data collected by the robotic arm through multimodal sensors during the current operating cycle. The load information data includes joint torque values ​​recorded by torque sensors, end-effector acceleration values ​​recorded by acceleration sensors, and joint angle values ​​recorded by angle sensors. The preprocessing module is used to preprocess the load information data to generate a standardized dataset corresponding to the operating state of the robotic arm. The standardized dataset includes torque change curves, acceleration change curves, and angle change curves over a time series. The load prediction module is used to input the standardized dataset into the trained deep learning model and predict the load dynamic characteristics of the robotic arm in the next operating cycle based on the deep learning model. The load dynamic characteristics include load mass distribution characteristics and load center of gravity position change characteristics. The compensation calculation module is used to calculate the gravity compensation torque value of each joint of the robotic arm in the next operating cycle based on the prediction results of the load dynamic characteristics. The gravity compensation torque value is obtained by combining the robotic arm dynamic model with the load dynamic characteristics. The feedback module is used to feed back the gravity compensation torque value to the control system of the robotic arm in real time, and adjust the output torque of the drive motors of each joint of the robotic arm to realize the dynamic gravity compensation control of the robotic arm in the next operating cycle.

Images