Trajectory smoothing control method, device and equipment of mechanical arm and storage medium

Abstract

The application relates to a trajectory smoothing control method and device of a mechanical arm, equipment and a storage medium, which are applied to a control unit in a mechanical arm control system, the system further comprises a mechanical arm and a camera unit, the method comprises the following steps: in the process of executing a preset task, first state data of the mechanical arm is collected based on the camera unit; the first state data is input into a target end-to-end model for processing, and a first action instruction is output; the mechanical arm is controlled to move based on the first action instruction, after the mechanical arm completes the movement, the first state data is updated, a new action instruction is regenerated based on the updated first state data, the mechanical arm is controlled to move based on the new action instruction, and the process is repeated until the mechanical arm completes the preset task; or after the mechanical arm executes the preset task for a time period longer than a preset time period, it is determined that the preset task fails. According to the application, the smooth control of the trajectory of the mechanical arm can be realized.

Description

Technical Field

[0001] This application relates to the field of robotic arm control technology, and in particular to a method, apparatus, device and storage medium for smoothing the trajectory of a robotic arm. Background Technology

[0002] With the development of industrial automation technology, robotic arms are widely used in precision operation scenarios such as grasping, assembly, and welding. The smoothness of the trajectory motion directly determines the working accuracy, operational stability, and service life of the robotic arm.

[0003] Currently, most robotic arm trajectory control methods employ pre-programmed open-loop control. Pre-programmed open-loop control involves pre-writing fixed motion trajectory instructions into the control system, and the robotic arm executes the motion according to the preset instructions. However, this method cannot cope with sudden situations such as load changes, minor external disturbances, and mechanical wear during the motion process, which can easily lead to trajectory deviation and sudden changes in action, resulting in insufficient smoothness of the robotic arm's motion trajectory.

[0004] Therefore, how to achieve smooth control of the robotic arm's trajectory has become an urgent problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, device, equipment, and storage medium for smooth control of the trajectory of a robotic arm, which can achieve smooth control of the trajectory of the robotic arm.

[0006] In a first aspect, this application provides a trajectory smoothing control method for a robotic arm, applied to a control unit in a robotic arm control system. The system further includes: a robotic arm and a camera unit. The method includes:

[0007] During the execution of the preset task, the first state data of the robotic arm is collected based on the camera unit;

[0008] The first state data is input into the target end-to-end model for processing, and the first action command is output; the target end-to-end model is obtained by training the preset end-to-end model based on the training state dataset of the robotic arm;

[0009] The robotic arm is controlled to move based on the first action command. After the robotic arm completes the movement, the first state data is updated. A new action command is regenerated based on the updated first state data. The robotic arm is controlled to move based on the new action command until the robotic arm completes the preset task; or, if the time taken for the robotic arm to execute the preset task is greater than the preset time, the preset task is determined to have failed.

[0010] Secondly, this application provides a trajectory smoothing control device for a robotic arm, applied to a control unit in a robotic arm control system. The system further includes: a robotic arm and a camera unit. The device includes:

[0011] The data acquisition module is used to acquire the first state data of the robotic arm based on the camera unit during the execution of a preset task;

[0012] The instruction generation module is used to input the first state data into the target end-to-end model for processing and output the first action instruction; the target end-to-end model is obtained by training a preset end-to-end model based on the training state dataset of the robotic arm;

[0013] The robotic arm control module is used to control the robotic arm to move based on the first action command. After the robotic arm completes the movement, it updates the first state data, regenerates a new action command based on the updated first state data, and controls the robotic arm to move based on the new action command until the robotic arm completes the preset task; or, if the time taken for the robotic arm to execute the preset task is greater than the preset time, it is determined that the preset task has failed.

[0014] Thirdly, this application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method described above.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0016] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described above.

[0017] The aforementioned method, device, equipment, and storage medium for smooth trajectory control of the robotic arm, when performing a preset task, collects real-time state data (i.e., first state data) of the robotic arm based on a camera unit; inputs the first state data into a target end-to-end model for processing and outputs a first action command; then, the robotic arm moves according to the first action command, and after the robotic arm completes a single movement, updates the state data and generates a new command again, dynamically corrects the movement deviation through iterative feedback, avoids trajectory deviation and sudden movement, and finally achieves smooth trajectory control of the robotic arm. Attached Figure Description

[0018] Figure 1 An application environment diagram for a trajectory smoothing control method for a robotic arm provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the structure of a robotic arm control system provided in an embodiment of this application;

[0020] Figure 3A flowchart illustrating a trajectory smoothing control method for a robotic arm provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of another robotic arm control system provided in an embodiment of this application;

[0022] Figure 5 A structural block diagram of a trajectory smoothing control device for a robotic arm provided in an embodiment of this application;

[0023] Figure 6 An internal structural diagram of a computer device provided in an embodiment of this application;

[0024] Figure 7 An internal structural diagram of another computer device provided in an embodiment of this application;

[0025] Figure 8 This is an internal structural diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] The following explains some of the technical terms or phrases used in this application:

[0028] End-to-end model: This is a deep learning model architecture whose core feature is that it directly receives raw input data, performs autonomous learning and feature extraction within the model, and outputs the final target result without the need for manual design and intervention in multiple independent intermediate processing steps.

[0029] Model inference cycle: refers to the complete time interval from the input of data to be processed to the output of the final inference result of the model, also known as "inference time" or "single inference duration".

[0030] Servo frequency refers to the number of times a servo system (e.g., a robotic arm) responds to or adjusts to control commands per unit time. The unit is usually Hertz (Hz), and it essentially reflects the dynamic response speed and control accuracy of the servo system.

[0031] The trajectory smoothing control method for the robotic arm provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a communication network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0032] It should be explained that the terminal 102 or the server 104 can execute any of the implementation methods described in the trajectory smoothing control method of the robotic arm provided in the embodiments of this application, which will not be repeated here.

[0033] The computer device described in the embodiments of this application may include at least one of terminal 102 or server 104.

[0034] Please see Figure 2 , Figure 2 This is a schematic diagram of a robotic arm control system provided in an embodiment of this application; the robotic arm control system may include: a robotic arm, a camera unit, and a control unit, wherein:

[0035] The robotic arm is the execution terminal of the system, responsible for receiving motion commands from the control unit and completing physical movements. It is the core carrier for achieving task objectives. For example, according to the command stream output by the control unit, the robotic arm drives the joint motors to complete precise displacement, turning, grasping and other actions to perform preset tasks (such as object grasping, precision assembly, etc.). In addition, the robotic arm can also collect its own body state data in real time (such as joint angles, end pose, and movement speed) and send it back to the control unit to provide input basis for model inference.

[0036] The camera unit is the "eyes" of the system, responsible for collecting environmental information about the robotic arm and providing environmental status input to the control unit. For example, the camera unit can capture images of the robotic arm's working area in real time, capturing environmental changes (e.g., the position, posture, and relative distance of an object to the robotic arm); and transmit the collected image data or extracted environmental feature data (e.g., object coordinates) to the control unit.

[0037] The control unit is the "brain" of the system. It is responsible for integrating perception data (e.g., the body state data of the robotic arm and the image data collected by the camera unit), running model reasoning, executing smoothing optimization algorithms (i.e., the trajectory smoothing control method of the robotic arm provided in the embodiments of this application), and issuing action commands. It is the key to realizing trajectory smoothing control.

[0038] Please see Figure 3 , Figure 3 This is a flowchart illustrating a trajectory smoothing control method for a robotic arm, provided in an embodiment of this application. The method can be applied to a control unit in a robotic arm control system. The system further includes a robotic arm and a camera unit. The method includes the following steps:

[0039] S101. During the execution of the preset task, the first state data of the robotic arm is collected based on the camera unit.

[0040] The preset tasks can be preset in advance or defaulted. For example, the preset tasks can be one of the following: material grabbing and handling tasks, precision assembly tasks, workpiece sorting tasks, and trajectory reproduction tasks. The control unit can include at least one of the following: industrial control computer, embedded controller, and computer host. The camera unit can include at least one of the following: 2D industrial camera, 3D depth camera, vision sensor, and high-speed camera. In addition, the control unit can also be configured with computing acceleration components such as GPU to speed up the data processing speed of the system.

[0041] Specifically, at the instant the preset task begins execution, the camera unit is simultaneously activated, and a data transmission channel is established between the camera unit and the robotic arm to ensure that the timestamps of the two types of data (environmental data and the robotic arm's physical state data) are aligned. That is, the environmental information and the robotic arm's physical state data at the same point in time can correspond. Then, the camera unit is controlled to capture images of the robotic arm's working area in real time, capturing environmental change details at each point in time to obtain image data. Environmental data is extracted from the image data. For example, assuming the camera unit is a 2D camera, the 2D camera can output real-time images of the working area, and environmental data around the robotic arm can be extracted from the real-time images. Or, assuming the camera unit is a 3D depth camera, the 3D depth camera can directly output three-dimensional point cloud data of the environment, and environmental data can be extracted from the three-dimensional point cloud data. The environmental data may include at least one of the following: basic information of the target object (e.g., appearance features, initial pose), and relative position change information between the robotic arm and the target object. The target object is the object that the robotic arm needs to operate on in the preset task.

[0042] Then, while the camera unit collects environmental data, the control unit can also obtain the robotic arm's body state information at the same point in time (e.g., joint angles of each joint and pose of the end effector). Finally, the environmental data and the robotic arm's body state information can be integrated to obtain the first state data. Specifically, each frame of environmental data collected by the camera unit can be timestamped with a precise time stamp (e.g., at the millisecond level) to record the specific time point of environmental data collection; at the same time, the body state information can be timestamped with the same precision; by matching the timestamps, environmental data and body state information at the same point in time can be filtered out to form a "one-to-one" data pair, thereby obtaining the first state data.

[0043] Please see Figure 4 , Figure 4 The schematic diagram of another robotic arm control system provided in this application embodiment shows that, in addition to the control unit, robotic arm, and camera unit, the robotic arm control system may also include a worktable; the robotic arm can perform preset tasks on the worktable.

[0044] S102. Input the first state data into the target end-to-end model for processing and output the first action command; the target end-to-end model is obtained by training the preset end-to-end model based on the training state dataset of the robotic arm.

[0045] The preset end-to-end model can be any kind of imitation learning model, such as the Behavior Cloning (BC) model, the Deep Deterministic Policy Gradient (DDPG) model, or the Generative Adversarial Imitation Learning (GAIL) model.

[0046] In addition, the training state dataset is a continuous motion trajectory data of the robotic arm from the start of a task to the completion of the task. This trajectory is divided into several consecutive "time steps", each time step corresponds to a set of independent data, and the data of all time steps are arranged in order to form the training state dataset. Specifically, the data of each time step includes: environmental change data observed by the camera unit, the robotic arm's own state information, and the robotic arm's current action information.

[0047] The environmental change data observed by the camera unit is the real-time scene information of the environment in which the robotic arm is located, as "seen" by the camera unit. This includes things like the positional shift of the target object, changes in the distance between the robotic arm's end effector and the object, and the positional changes of obstacles within the work area. This type of information reflects the current environmental state and serves as the external basis for the model to determine "what action to take."

[0048] The robotic arm's own state information refers to the state information that the robotic arm "senses." The core of this information is the real-time joint angles of each joint, but it can also include parameters such as the end effector's pose and motor speed. This type of information reflects the robotic arm's current state and is the internal basis for the model to determine "its current position."

[0049] The current motion information of the robotic arm refers to the specific operational instructions executed by the robotic arm at the current time step. It has two common forms of expression: one is the change in joint angle (e.g., joint 1 rotates by 5°), and the other is the motion parameters of the end effector (e.g., the end effector moves forward 2cm and downward 1cm). This type of information represents the optimal action corresponding to the current "environment-body" state and is the core output target that the model needs to learn.

[0050] The purpose of the training state dataset is to provide "demonstration samples" for the model. By inputting a training state dataset containing multiple datasets into the model, the model essentially learns the mapping relationship of "what action should be performed in what environment and in what state." Through learning from a large amount of this type of training data, the model can imitate the ability to complete the same task.

[0051] In some embodiments, before inputting the first state data into the target end-to-end model for processing and outputting the first action command, the above method further includes:

[0052] A1. Determine the basic task loss function corresponding to the preset end-to-end model;

[0053] A2. Obtain the regularization coefficient and the temporal difference smoothing loss function;

[0054] A3. Determine the training loss function based on the regularization coefficient, the temporal difference smoothing loss function, and the basic task loss function;

[0055] A4. Train the preset end-to-end model based on the training loss function and the training state dataset to obtain the target end-to-end model.

[0056] Among them, the basic task loss function is a loss function used to measure the deviation between the model's predicted value and the true value, and is used to ensure that the model completes the core task with accuracy; the temporal difference smoothing loss function is a loss function used to constrain the differences in output actions between adjacent time steps, suppress action jumps, and make the robotic arm's movement more continuous and smooth.

[0057] Specifically, we can first determine the core task that the pre-defined end-to-end model will perform, and then select a base task loss function that is suitable for the core task. For example, assuming the core task is "predicting the joint angle / end-effector pose of the robotic arm," which is a regression task, we can choose the mean squared error (MSE) loss function as the base task loss function. The MSE loss function quantifies the "degree of deviation of the predicted value from the true value" by calculating the sum of the squares of the differences between the predicted value and the true value. The larger the loss value, the greater the prediction deviation of the joint angle / end-effector pose, and the lower the accuracy of the robotic arm when performing the action. As another example, assuming the core task is "determining whether the action is correct," which is a classification task, we can choose the cross-entropy loss function as the base task loss function. The cross-entropy loss function has a stronger penalty for "misclassification" and can quickly guide the model to distinguish between correct and incorrect actions. For example, when the robotic arm performs a grasping action, if the model misclassifies it as "correct" but the actual grasping fails, the cross-entropy loss will increase significantly, forcing the model to adjust its parameters to correct the judgment.

[0058] Next, we can obtain the regularization coefficient and the temporal difference smoothing loss function. Specifically, the regularization coefficient can be preset or defaulted. In addition, in order to penalize the abrupt changes in the output actions of adjacent time intervals, we can introduce a loss function constraint for the smoothness of actions, namely the temporal difference smoothing loss function. Specifically, the temporal difference smoothing loss function can be the MSE loss function. If the difference between actions in adjacent time intervals is large (abrupt change), the value of the MSE loss function is large, and the model will be penalized; if the actions are continuous and smooth, the value of the MSE loss function is small, and the model is not penalized, thereby smoothing the actions.

[0059] In some embodiments, the regularization coefficient can be 0.5.

[0060] It's important to explain that the regularization coefficient λ can also be obtained experimentally. Specifically, we can first determine the range of values ​​for the regularization coefficient, for example, 0.1 to 1.0. This is because if λ is too small (e.g., less than 0.1): the weight of the smoothing loss is extremely low, and the model prioritizes task accuracy, but issues like action jumps and jitter cannot be improved; if λ is too large (e.g., greater than 1.0): the weight of the smoothing loss is too high, and the model's actions are too smooth, but this may deviate from the correct trajectory, leading to task execution failure (e.g., failing to grasp the target object). Next, using the intermediate value of 0.5 as a center, we select several gradient values ​​within the range for comparative experiments, for example, testing five values: 0.1, 0.3, 0.5, 0.7, and 0.9.

[0061] Keep the model structure, training dataset, optimizer, and other parameters completely consistent;

[0062] Five sets of models were trained separately, and the prediction error, action variance, and task success rate of each set of models were recorded.

[0063] Plot three performance curves with λ as the horizontal axis and "prediction error", "action variance", and "task success rate" as the vertical axes to visually observe the trends of these metrics. Select the optimal balance point: the λ value that achieves the best balance between task accuracy and action smoothness.

[0064] The model must have a task success rate greater than or equal to a preset threshold (e.g., 90%) to ensure that the core task can be completed.

[0065] Under the premise of meeting the task success rate, select the λ value corresponding to the minimum action variance. For example, if the test finds that when λ=0.5, the task success rate is 95% and the action variance is 0.02; when λ=0.7, the task success rate is 92% and the variance is 0.015; then λ=0.5 is preferred, balancing accuracy and smoothness.

[0066] If the optimal range is between two test values ​​(e.g., 0.3~0.5), the gradient can be further subdivided within this range (e.g., 0.3, 0.35, 0.45, 0.5), and the above experimental steps can be repeated to obtain a more accurate value of λ.

[0067] Furthermore, the training loss function can be determined based on the regularization coefficient, the temporal difference smoothing loss function, and the basic task loss function. The specific training loss function is as follows:

[0068] ;

[0069] in, Represents the training loss function; Represents the loss function of the basic task; Represents the regularization coefficient; This represents the time-difference smoothing loss function; This represents the predicted action instruction of the model at time t; This represents the predicted action instruction of the model at time t-1.

[0070] Finally, the preset end-to-end model can be trained based on the training loss function and the training state dataset to obtain the target end-to-end model. Specifically, the training state dataset is input into the preset end-to-end model in batches for processing, and the corresponding predicted action command data is output. Then, the task loss value between the predicted action command data and the real action command data and the temporal smoothing loss of the predicted action command data in adjacent time steps are calculated using the training loss function, and the total loss value is obtained by weighting. The total loss gradient is determined based on the total loss value, and the gradient descent algorithm is used to backpropagate the total loss gradient and update the various parameters of the model until the total loss converges (no longer decreases significantly), or the "task completion rate and action smoothness" of the preset end-to-end model on the preset validation set reach the preset standard. At this time, the preset end-to-end model is used as the target end-to-end model.

[0071] Both the preset validation set and the preset standard can be preset in advance or left as default.

[0072] Thus, by first anchoring the basic task loss to ensure core operational capabilities, then introducing temporal difference smoothing loss and regularization coefficients to constrain action continuity, and finally integrating dual-objective loss functions for training, the step-by-step design achieves source-level collaborative optimization of "task completion accuracy" and "action trajectory smoothness". This allows the final trained target model to directly output high-precision, low-jitter action commands without relying on additional processing modules, combining practicality and robustness, and effectively solving the pain point of traditional solutions where task execution and action smoothness are difficult to balance.

[0073] In some embodiments, the first state data is input into the target end-to-end model for processing, and a first action command is output, including:

[0074] B1. Input the first state data into the target end-to-end model for processing and output the reference action command;

[0075] B2. Obtain the previous action instruction output by the target end-to-end model before the reference action instruction, and obtain the historical action instruction;

[0076] B3. Determine the first action instruction based on the preset exponential moving average formula, the preset smoothing factor, historical action instructions, and reference action instructions;

[0077] The specific formula for the preset exponential moving average is as follows:

[0078] ;

[0079] in, Indicates the first action instruction; Indicates the preset smoothing factor; Indicates historical action commands; Indicates a reference action instruction.

[0080] The preset exponential moving average formula and the preset smoothing factor can both be preset in advance or defaulted to; specifically, the preset smoothing factor can be denoted as... ,like If the value is too small (e.g., less than 0.5), the smoothing effect is not obvious; if If the value is too large (e.g., greater than 0.95), it will produce a significant sense of lag. Therefore, the preset smoothing factor can be set to a range of 0.5 to 0.95.

[0081] Specifically, the previous action instruction output by the target end-to-end model before the reference action instruction is obtained to obtain the historical action instruction. Specifically, the reference timestamp corresponding to the reference action instruction is obtained, and the time range of the "previous time step" is determined based on the reference timestamp. For example, assuming the reference timestamp t1 = 100ms and the model inference cycle is 100ms, then the previous time step is t0 = t1 - 100ms = 0ms. Then, the control unit can read the action instruction of the previous time step, i.e., the historical action instruction, from the local cache.

[0082] In some embodiments, the preset smoothing factor can be set to a value range of 0.75 to 0.9. When the preset smoothing factor is within this value range, it can effectively filter out high-frequency jitter components and control the control delay within 100ms, thus meeting the dynamic grasping requirements.

[0083] In some embodiments, the above-mentioned robotic arm control system may further include an EMA filter (Exponential Moving Average Filter), which can perform the above steps B1 to B3 after the target end-to-end model outputs the motion command and before the motion command is sent to the control unit.

[0084] In this way, the reference action command is output through the target end-to-end model, and the historical action command is retrieved. The first action command is calculated by combining the preset exponential moving average formula. Based on the smoothing constraints in the model training stage, "double smoothing" is further achieved through EMA post-processing in the inference stage. This not only relies on the model to ensure the matching degree between the action command and the current environment and the body state, ensuring the accuracy of task execution, but also uses the weighted constraints of historical action commands to effectively filter out possible small jumps in the reference action command. This makes the final output first action command more continuous in the time dimension and the trajectory more stable. At the same time, the preset smoothing factor can flexibly adjust the weight ratio of historical actions and reference actions to adapt to the "accuracy first" or "smoothness first" requirements in different task scenarios, significantly reducing the risk of jitter during the operation of the robotic arm and improving control stability.

[0085] S103. Control the robotic arm to move based on the first action command. After the robotic arm completes the movement, update the first state data, regenerate a new action command based on the updated first state data, and control the robotic arm to move based on the new action command until the robotic arm completes the preset task; or, if the time taken for the robotic arm to execute the preset task is greater than the preset time, determine that the preset task has failed to be executed.

[0086] The preset time can be preset in advance or set to the default value.

[0087] Specifically, the first action command can be issued to the robotic arm, which then begins a movement. After the robotic arm completes this movement, real-time status data of the robotic arm can be collected by the camera unit. The first status data can be updated using the real-time status data, and the updated first status data can be input into the target end-to-end model to obtain a new action command. The new action command is then issued to the robotic arm, and the robotic arm continues to move. The above steps are repeated until the robotic arm completes the preset task; or, if the time taken for the robotic arm to perform the preset task exceeds the preset time, the preset task is determined to have failed.

[0088] In some embodiments, controlling the robotic arm to move based on a first action command includes:

[0089] C1. Determine the task motion space corresponding to the preset task, and the allowable motion space of the robotic arm;

[0090] C2. Determine the target scaling factor based on the task action space and the allowed action space; the target scaling factor is used to normalize and map the action commands output by the target end-to-end model into action values ​​that the robotic arm can execute.

[0091] C3. Determine the actual action value based on the target proportional coefficient and the first action command;

[0092] C4. Control the robotic arm to move according to the actual motion values.

[0093] The first action instruction is a normalized value, which ranges from -1 to 1.

[0094] Specifically, the specific requirements of the preset task can be analyzed to extract the minimum range of motion (i.e., task motion space) required to complete the task. For example, if the preset task is "material grasping task within a 10cm range", the task motion space is the actual physical range that the end effector of the robotic arm needs to move in the grasping task. For example, the joint angle adjustment range is -15° to +15°, and the displacement range of the end effector of the robotic arm is 0 to 10cm. In other words, the task motion space is the minimum necessary range of motion that the robotic arm can meet the task requirements.

[0095] Next, you can obtain the hardware parameter manual of the robotic arm and obtain the limit range of motion of each joint (i.e. the maximum action boundary that the robotic arm can physically reach). This is the hardware safety threshold of the robotic arm. For example, the limit rotation range of the robotic arm joint is -90° to +90°, and the maximum displacement range of the robotic arm end is 0 to 50cm. This range is the allowable action space.

[0096] Furthermore, a target proportional coefficient can be determined based on the task motion space and the allowed motion space. Then, the actual motion value can be determined based on the target proportional coefficient and the first motion command. Specifically, the first motion command can be adjusted based on the target proportional coefficient to obtain the actual motion value. For example, assuming the preset task is "material grasping within a 10cm range", the corresponding task motion space is set to a robotic arm end-effector displacement of 0~10cm, and the first motion command is 0.2, then:

[0097] ;

[0098] in, Indicates the actual action value; Indicates the target proportion coefficient; This represents the maximum value of the task action space; This represents the minimum value of the task action space; This indicates the span of the value range from -1 to 1 (i.e., the target numerical range mentioned later); This represents the offset (making the midpoint of the normalized value, 0, correspond to the midpoint of the task action space). This indicates the first action command; finally, the robotic arm can be controlled to move according to the actual action value. For example, if the actual action value is 6cm and the direction of movement corresponding to the actual action value is the direction of the material, the robotic arm can be controlled to move precisely 6cm in the direction of the material, and the displacement deviation is controlled within the preset accuracy threshold (e.g., ±0.1cm).

[0099] As can be seen, in this embodiment, to address the sensitivity issue of mapping between model output and physical motion, the target scaling factor of the motion space has been adjusted and optimized. The method of directly mapping the normalized values ​​of the model output to the limit physical joint angles of the robotic arm is abandoned. Instead, a limited physical mapping range is set based on the actual workspace required for the task (i.e., the task motion space). Taking a grasping task with a displacement of 10cm as an example, if [ [1,1] Mapping to the limit range of 100cm, a tiny noise of 0.01 in the model output will cause a physical jitter of 1cm. However, after limiting the mapping range to 15cm (leaving a reasonable margin), the same noise will only cause a displacement fluctuation of 1.5mm. By reducing the target scaling factor, the amplification factor of the prediction noise in the physical world can be significantly reduced, thereby improving the fineness of control from a geometric perspective. In addition, the target scaling factor can be adaptively adjusted for different tasks and is not a fixed value.

[0100] In this way, by first defining the task motion space that matches the preset task, and then calculating the target ratio coefficient accordingly, the normalized instructions output by the model are converted into actual motion values ​​to drive the movement of the robotic arm. This not only significantly reduces the amplification effect of the model output noise from a geometric perspective, but also improves the fineness of control and the smoothness of the motion.

[0101] In addition, the scaling factor can be adaptively adjusted for different tasks to meet diverse operational needs. It can also work in synergy with the smoothing loss constraint in the training phase and the EMA filtering in the inference phase to achieve high-precision and stable operation of the robotic arm.

[0102] In some embodiments, determining the target scaling factor based on the task action space and the allowed action space includes:

[0103] D1. Determine the reference motion space based on the task motion space; the motion space range of the reference motion space is larger than the task motion space.

[0104] D2. Obtain the target numerical range corresponding to the target end-to-end model;

[0105] D3. Determine the target scaling factor based on the target value range, reference motion space, and allowable motion space.

[0106] Specifically, the task motion space refers to the minimum necessary range of motion required to complete a preset task. It is determined by the technical requirements of the task itself and has no redundancy. For example, for a 10cm material grasping task, the robotic arm's end effector only needs to move within a displacement range of 0-10cm to complete the grasping. This 0-10cm range is the task motion space. By adding appropriate margins to the task motion space, a reference motion space can be obtained. Specifically, for tasks without a clear directional preference (e.g., grasping, handling), a symmetrical margin can be added, increasing the margin proportionally at the upper and lower limits of the task motion space. For example, assuming the task motion space is 0-10cm with a span of 10cm, taking a 25% unilateral margin Δ = 25% × 10cm = 2.5cm, then the reference motion space is... 2.5cm~12.5cm; or, for tasks with clear directional restrictions, only add margin in specific directions. For example, if an assembly task requires the end effector of the robotic arm to move within a 5~8cm range (task motion space) and is only allowed to adjust in the direction away from the part (to avoid collision), then a 2cm margin can be added to the upper limit side and no margin is added to the lower limit side, resulting in a reference motion space of 5~10cm.

[0107] It needs to be explained that the reason for adding the margin is to compensate for deviations in actual execution. Because the robotic arm is affected by factors such as sensor noise, transmission mechanism backlash, and environmental disturbances during movement, there is a slight deviation between the actual motion value and the theoretical value. If the task motion space is directly used as the mapping reference, even a small deviation could cause the motion value to exceed the range, resulting in task failure. For example, if the target displacement for a grasping task is 10cm, and the robotic arm actually moves 10.2cm, the task motion space (0~10cm) will be considered out of range. The reference motion space (-2.5~12.5cm) can accommodate this deviation, ensuring the task is completed normally.

[0108] Next, the target numerical range corresponding to the target end-to-end model can be obtained. Specifically, since the target end-to-end model usually normalizes the output action instructions during the training phase, its default target numerical range is generally a standardized interval. [1, 1]; This range is determined by the data preprocessing strategy during training. During training, the historical motion data of the robotic arm (such as joint angles and displacements) will be mapped to [ The predicted values ​​output by the model after backpropagation optimization will naturally fall within the interval [1,1].

[0109] It should be explained that in practical applications, the target numerical range can also be changed: if the motion data is not normalized during model training, but physical quantities such as joint angles and displacements are directly used as training targets, then the target numerical range is the actual fluctuation range of the corresponding physical quantity (for example, the joint angle [-90°, 90°]); if it is necessary to adapt to special high-precision tasks, a custom numerical range can also be set by adjusting the activation function of the model output layer.

[0110] Finally, the target scaling factor can be determined based on the target value range, the reference action space, and the allowed action space.

[0111] In this way, by first expanding the reference motion space with a margin based on the task motion space, and then combining the target numerical range with the allowable motion space of the robotic arm to determine the target proportional coefficient, the design can improve the fault tolerance and dynamic adjustment space of task execution through the margin of the reference motion space. At the same time, it can accurately calculate the proportional coefficient that takes into account both control accuracy and equipment safety by relying on the numerical range benchmark and hardware threshold constraints. This reduces the amplification effect of model output noise from the source and achieves high-precision and low-shake operation of the robotic arm.

[0112] In some embodiments, determining the target scaling factor based on the target numerical range, the reference motion space, and the allowed motion space includes:

[0113] When the allowable motion space is greater than or equal to the reference motion space, the target scaling factor is determined based on the reference motion space and the target value range; or...

[0114] When the range of the allowed motion space is smaller than the reference motion space, the target scaling factor is determined based on the allowed motion space and the target value range.

[0115] If the allowed motion space is greater than or equal to the reference motion space, the target proportional coefficient can be determined based on the reference motion space and the target value range: first, determine the first span corresponding to the target value range, determine the second span corresponding to the reference motion space, and use the second span divided by the first span to obtain the target proportional coefficient; for example, if the target value range is -1 to 1, and the reference motion space is 0 to 15cm, then the first span is 1 - (-1) = 2; the second span is 15 - 0 = 15 (cm), and the target proportional coefficient is 15 / 2 = 7.5 (cm).

[0116] If the allowed motion space is smaller than the reference motion space, the target ratio coefficient can be determined based on the allowed motion space and the target value range. Specifically, the third span corresponding to the allowed motion space can be determined, and the target ratio coefficient can be obtained by dividing the third span by the first span.

[0117] It should be explained that after determining the target scaling factor, the corresponding offset b can also be calculated.

[0118] Thus, by dynamically selecting the mapping reference to determine the target scaling factor based on the relationship between the allowed motion space and the reference motion space, the core advantage is that it adapts to task requirements and attenuates the amplification effect of model output noise through the reference motion space, while using the allowed motion space as the hardware safety baseline to ensure that the robotic arm always operates within a safe range, thus balancing control accuracy and equipment safety.

[0119] In some embodiments, after inputting the first state data into the target end-to-end model for processing and outputting the first action command, and before controlling the robotic arm to move based on the first action command, the method further includes:

[0120] E1. Obtain the next action instruction output by the target end-to-end model after the first action instruction, and obtain the second action instruction;

[0121] E2. Obtain the first position of the robotic arm at the current moment;

[0122] E3. Determine the first action position based on the first position and the first action instruction;

[0123] E4. Determine the second action position based on the first position and the second action instruction;

[0124] E5. Based on the first position, the first action position, and the second action position, determine N interpolation positions; N is a positive integer.

[0125] E6. Control the robotic arm to move according to N interpolation positions, the first action position, and the second action position.

[0126] Specifically, the first timestamp corresponding to the first action command can be obtained, and the time range of the "next time step" can be determined based on the first timestamp. For example, assuming the first timestamp t2 = 200ms and the model inference cycle is 100ms, the next time step is t3 = t2 + 100ms = 300ms. Then, the control unit can receive the action command of the target end-to-end model in the next time step, thereby obtaining the second action command.

[0127] Then, the first position of the robotic arm at the current moment can be obtained. Specifically, each joint of the robotic arm is equipped with a servo encoder, which can record the rotation angle of the joint in real time. The current angle value of each joint encoder is read from the servo encoder, for example, 30° for joint 1, -15° for joint 2, 45° for joint 3, etc. The forward kinematics model of the robotic arm is called, and the angle of each joint is substituted into the preset model formula to calculate the three-dimensional spatial coordinates of the end effector of the robotic arm (for example, X=10.5cm, Y=5.2cm, Z=8.8cm). The three-dimensional spatial coordinates are the first position of the robotic arm.

[0128] Then, based on the first position and the first action command, the first action position can be determined. Specifically, the meaning of the first action command can be analyzed to clarify the displacement change ΔP=(ΔX, ΔY, ΔZ) in Cartesian space corresponding to the first action command. ΔP is converted from the actual action value. For example, assuming the actual action value is "move 6.75cm in the positive X-axis direction", its corresponding ΔP is: ΔX=6.75cm, ΔY=0, ΔZ=0. The first position is obtained by superimposing the coordinates of the first position and the displacement change ΔP. For example, assuming the first position P1=(10cm, 5cm, 8cm), the first action command corresponds to ΔX=6.75cm, ΔY=0, ΔZ=0; then the first action position Ptarget=(16.75cm, 5cm, 8cm).

[0129] Next, based on the first position and the second action command, the second action position can be determined. Specifically, the method for determining the second action position can be the same as that for determining the first action position, and will not be repeated here. Then, based on the first position, the first action position, and the second action position, N interpolation positions can be determined. Finally, based on the N interpolation positions, the first action position, and the second action position, the robotic arm can be controlled to move. Specifically, the first action position can be defined as the starting target point of the movement, the second action position as the final target point of the movement, and the N interpolation positions are the midpoints between the starting target point and the final target point. Then, based on the movement speed of the robotic arm (e.g., v = 2cm / ...), the movement can be controlled to... s), calculate the motion time interval Δt between each difference position in the N interpolation positions, and generate a time-ordered motion command sequence; first, send the first action command to the robotic arm to control the robotic arm to move to the first action position P1. After reaching the position, trigger position verification to confirm that the accuracy meets the requirements; then, send the action commands in the motion command sequence in sequence according to the motion time interval Δt, drive the robotic arm to move smoothly from P1 to the first interpolation position Pc1, and verify again after reaching the position; repeat the above steps, pass through all interpolation positions in sequence, and finally reach the second action position P2; during the movement, the controller can dynamically adjust the movement speed by feeding back position information in real time through the servo encoder to ensure that the trajectory fits the interpolation path.

[0130] Thus, by first obtaining the two-stage action commands continuously output by the model, calculating the corresponding action position and generating the interpolated position, and then controlling the movement of the robotic arm in sequence, the core advantage is that by connecting the continuous commands of the model and introducing the interpolated trajectory, the movement path of the robotic arm can be made smoother and more continuous, avoiding the jitter and impact caused by command jumps. At the same time, combined with real-time position feedback, the accuracy of action execution is ensured, improving the motion stability and control precision in precision operation scenarios.

[0131] In some embodiments, determining N interpolation positions based on a first position, a first action position, and a second action position includes:

[0132] F1. Obtain the model inference cycle corresponding to the target end-to-end model, and the servo frequency corresponding to the robotic arm;

[0133] F2. Determine the number of interpolations based on the model inference cycle and servo frequency; the number of interpolations is N.

[0134] F3. Based on the number of interpolations and the first position, interpolate between the first action position and the second action position to obtain N interpolation positions.

[0135] Specifically, 100 sets of test data can be input into the target end-to-end model, and the start time and command output time of each inference can be recorded. The average time of a single inference can be calculated, and this average value is the model inference cycle. For example, assuming 100 consecutive tests with a total time of 10 seconds, the model inference cycle T = 10 / 100 = 0.1 (s). Next, the equipment manual of the robotic arm can be obtained, and the servo frequency can be obtained from the equipment manual. The servo frequency determines the motion control accuracy of the robotic arm. The higher the frequency, the shorter the single control cycle and the smoother the motion. For example, the common servo frequency of industrial robotic arms is 500Hz.

[0136] Then, the number of interpolations can be determined based on the model inference cycle and the servo frequency. Specifically, the single control time of the robotic arm can be equal to the reciprocal of the servo frequency. For example, assuming the servo frequency is 500Hz, the single control time is 0.002s. The number of interpolations can be determined based on the model inference cycle and the single control time. Specifically, the number of interpolations can be obtained by dividing the model inference cycle by the single control time. For example, if the model inference cycle is 0.1s, 0.1s / 0.002s-1=499, which means the number of interpolations is 499.

[0137] Finally, interpolation can be performed between the first action position and the second action position based on the number of interpolations and the first position to obtain N interpolation positions. Specifically, a preset interpolation method can be used to insert N values ​​between the first action position and the second action position based on the first position, that is, N interpolation positions. The preset interpolation method can include one of the following: linear interpolation method, polynomial interpolation method, circular interpolation method, spline interpolation method.

[0138] To illustrate, assuming the preset interpolation method is linear interpolation, the specific interpolation formula is as follows:

[0139] ;

[0140] in, Represents the coordinates of the i-th interpolation position out of N interpolation positions; The coordinates represent the current position of the robotic arm (i.e., the first position); The coordinates represent the position of the first action.

[0141] As can be seen, the core advantage of the design that first obtains the model inference cycle and the servo frequency of the robotic arm, then determines the number of interpolations and finally generates the corresponding interpolation positions is that the interpolation density accurately matches the servo control frequency and the model command output interval, fills the frequency gap between low-frequency model commands and high-frequency servo control, eliminates the step phenomenon in the robotic arm's movement, and ensures that the motion trajectory is continuous and smooth.

[0142] In some embodiments, to address the frequency mismatch between the low-frequency inference of the model (e.g., 10Hz) and the high-frequency servo of the robotic arm (e.g., 500Hz), the aforementioned robotic arm control system adopts a multi-threaded architecture, adding an intermediate interpolation controller between the model inference thread and the robotic arm's servo communication thread. The specific implementation logic is as follows:

[0143] Model inference thread: responsible for running the target end-to-end model and outputting sparse target points (corresponding to action instructions) at a frequency of 10Hz (each inference takes about 100ms).

[0144] Intermediate layer interpolation controller: Receives continuous target points output from the end-to-end target model, combines them with the current position of the robotic arm, and performs high-density interpolation calculations between adjacent target points using a preset interpolation method;

[0145] Servo communication thread: Sends interpolated position commands to the robotic arm at a control frequency of 500Hz (i.e., every 2ms).

[0146] The above-mentioned trajectory smoothing control method for a robotic arm, when performing a preset task, collects real-time state data (i.e., first state data) of the robotic arm based on a camera unit; inputs the first state data into the target end-to-end model for processing and outputs a first action command; then, the robotic arm moves according to the first action command. After the robotic arm completes a single movement, it updates the state data and generates a new command again. Through iterative feedback, it dynamically corrects motion deviations, avoids trajectory offsets and sudden changes in movement, and finally achieves smooth control of the robotic arm trajectory.

[0147] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0148] Based on the same inventive concept, this application also provides a trajectory smoothing control device for a robotic arm. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the trajectory smoothing control device for a robotic arm provided below can be found in the limitations of the trajectory smoothing control method for a robotic arm described above, and will not be repeated here.

[0149] like Figure 5 As shown, this application embodiment provides a trajectory smoothing control device 500 for a robotic arm, applied to a control unit in a robotic arm control system. The system further includes: a robotic arm and a camera unit. The trajectory smoothing control device 500 for the robotic arm includes:

[0150] The acquisition module 501 is used to acquire the first state data of the robotic arm based on the camera unit during the execution of the preset task;

[0151] The instruction generation module 502 is used to input the first state data into the target end-to-end model for processing and output the first action instruction; the target end-to-end model is obtained by training a preset end-to-end model based on the training state dataset of the robotic arm;

[0152] The robotic arm control module 503 is used to control the robotic arm to move based on the first action command. After the robotic arm completes the movement, it updates the first state data, regenerates a new action command based on the updated first state data, and controls the robotic arm to move based on the new action command until the robotic arm completes the preset task; or, if the time taken for the robotic arm to execute the preset task is greater than the preset time, it is determined that the preset task has failed to be executed.

[0153] In some embodiments, the trajectory smoothing control device 500 of the robotic arm is further specifically used for:

[0154] Determine the basic task loss function corresponding to the preset end-to-end model;

[0155] Obtain the regularization coefficient and the temporal difference smoothing loss function;

[0156] The training loss function is determined based on the regularization coefficient, the temporal difference smoothing loss function, and the basic task loss function;

[0157] The target end-to-end model is obtained by training a pre-defined end-to-end model based on the training loss function and the training state dataset.

[0158] In some embodiments, in inputting first state data into a target end-to-end model for processing and outputting a first action command, the command generation module 502 is specifically used for:

[0159] The first state data is input into the target end-to-end model for processing, and reference action instructions are output.

[0160] Obtain the previous action instruction output by the target end-to-end model before the reference action instruction, and obtain the historical action instruction;

[0161] The first action instruction is determined based on a preset exponential moving average formula, a preset smoothing factor, historical action instructions, and reference action instructions. The preset exponential moving average formula is as follows:

[0162] ;

[0163] in, Indicates the first action instruction; Indicates the preset smoothing factor; Indicates historical action commands; Indicates a reference action instruction.

[0164] In some embodiments, in controlling the robotic arm to move based on a first action command, the robotic arm control module 503 is specifically used for:

[0165] Determine the task motion space corresponding to the preset task, as well as the allowable motion space of the robotic arm;

[0166] Based on the task action space and the allowed action space, the target scaling factor is determined; the target scaling factor is used to normalize and map the action commands output by the target end-to-end model into action values ​​that the robotic arm can execute.

[0167] The actual action value is determined based on the target proportional coefficient and the first action command;

[0168] The robotic arm is controlled to move based on the actual motion values.

[0169] In some embodiments, the robotic arm control module 503 is specifically used for determining the target scaling factor based on the task motion space and the allowed motion space:

[0170] The reference action space is determined based on the task action space; the action space range of the reference action space is larger than the task action space.

[0171] Obtain the target numerical range corresponding to the target end-to-end model;

[0172] The target scaling factor is determined based on the target value range, reference action space, and allowed action space.

[0173] In some embodiments, the robotic arm control module 503 is specifically used for determining the target proportional coefficient based on the target numerical range, the reference motion space, and the allowed motion space:

[0174] When the allowable motion space is greater than or equal to the reference motion space, the target scaling factor is determined based on the reference motion space and the target value range; or...

[0175] When the range of the allowed motion space is smaller than the reference motion space, the target scaling factor is determined based on the allowed motion space and the target value range.

[0176] In some embodiments, the trajectory smoothing control device 500 of the robotic arm is further specifically used for:

[0177] Obtain the next action instruction output by the target end-to-end model after the first action instruction, and thus obtain the second action instruction;

[0178] Obtain the current position of the robotic arm;

[0179] Determine the first action position based on the first position and the first action command;

[0180] Determine the second action position based on the first position and the second action command;

[0181] Based on the first position, the first action position, and the second action position, determine N interpolation positions; N is a positive integer;

[0182] The robotic arm is controlled to move based on N interpolation positions, the first action position, and the second action position.

[0183] In some embodiments, the trajectory smoothing control device 500 of the robotic arm is further specifically used for determining N interpolated positions based on the first position, the first action position, and the second action position:

[0184] Obtain the model inference cycle corresponding to the target end-to-end model, and the servo frequency corresponding to the robotic arm;

[0185] The number of interpolations is determined based on the model inference cycle and servo frequency; the number of interpolations is N.

[0186] Based on the number of interpolations and the first position, interpolation is performed between the first action position and the second action position to obtain N interpolation positions.

[0187] Each module in the aforementioned trajectory smoothing control device 500 for the robotic arm can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0188] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to the trajectory smoothing control of the robotic arm. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the aforementioned robotic arm trajectory smoothing control method.

[0189] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the steps in the aforementioned method for smoothing the trajectory control of the robotic arm. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen; the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs or touchpads set on the casing of the computer device, or external keyboards, touchpads or mice, etc.

[0190] Those skilled in the art will understand that Figure 6 or Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0191] In some embodiments, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0192] In some embodiments, such as Figure 8 The diagram shows the internal structure of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described method embodiments.

[0193] In some embodiments, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0195] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0196] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for smoothing the trajectory of a robotic arm, characterized in that, A control unit applied in a robotic arm control system, the system further including: a robotic arm and a camera unit, the method comprising: During the execution of a preset task, the first state data of the robotic arm is collected based on the camera unit; The first state data is input into the target end-to-end model for processing, and the first action command is output; the target end-to-end model is obtained by training a preset end-to-end model based on the training state dataset of the robotic arm; The robotic arm is controlled to move based on the first action command. After the robotic arm completes the movement, the first state data is updated, and a new action command is regenerated based on the updated first state data. The robotic arm is controlled to move based on the new action command until the robotic arm completes the preset task; or, the preset task is determined to have failed if the time taken for the robotic arm to perform the preset task is greater than the preset time. The method further includes, before inputting the first state data into the target end-to-end model for processing and outputting the first action command: Determine the basic task loss function corresponding to the preset end-to-end model; Obtain the regularization coefficient and the temporal difference smoothing loss function; The training loss function is determined based on the regularization coefficient, the temporal difference smoothing loss function, and the basic task loss function; The preset end-to-end model is trained based on the training loss function and the training state dataset to obtain the target end-to-end model; The step of inputting the first state data into the target end-to-end model for processing and outputting the first action command includes: The first state data is input into the target end-to-end model for processing, and a reference action command is output. Obtain the previous action instruction output by the target end-to-end model before the reference action instruction to obtain the historical action instruction; The first action instruction is determined based on the preset exponential moving average formula, the preset smoothing factor, the historical action instructions, and the reference action instructions.

2. The method according to claim 1, characterized in that, The specific formula for the preset exponential moving average is as follows: ； Among them, the This indicates the first action instruction; the The preset smoothing factor is represented; This indicates the historical action instruction; the This indicates the reference action instruction.

3. The method according to claim 1 or 2, characterized in that, The step of controlling the robotic arm to move based on the first action command includes: Determine the task motion space corresponding to the preset task, and the allowed motion space of the robotic arm; Based on the task action space and the allowed action space, a target scaling factor is determined; the target scaling factor is used to normalize and map the action instructions output by the target end-to-end model into action values ​​that the robotic arm can execute. The actual action value is determined based on the target proportional coefficient and the first action command; The robotic arm is controlled to move according to the actual motion values.

4. The method according to claim 3, characterized in that, The step of determining the target scaling factor based on the task action space and the allowed action space includes: A reference action space is determined based on the task action space; the action space range of the reference action space is larger than the task action space. Obtain the target numerical range corresponding to the target end-to-end model; The target scaling factor is determined based on the target numerical range, the reference action space, and the allowed action space.

5. The method according to claim 4, characterized in that, The step of determining the target scaling factor based on the target numerical range, the reference motion space, and the allowed motion space includes: When the range of the allowed motion space is greater than or equal to the reference motion space, a target scaling factor is determined based on the reference motion space and the target value range; or, When the range of the allowed motion space is smaller than the reference motion space, a target scaling factor is determined based on the allowed motion space and the target value range.

6. The method according to claim 1 or 2, characterized in that, After inputting the first state data into the target end-to-end model for processing and outputting the first action command, and before controlling the robotic arm to move based on the first action command, the method further includes: Obtain the next action instruction output by the target end-to-end model after the first action instruction, and obtain the second action instruction; Obtain the first position of the robotic arm at the current moment; The first action position is determined based on the first position and the first action command; The second action position is determined based on the first position and the second action command; Based on the first position, the first action position, and the second action position, N interpolation positions are determined; where N is a positive integer. The robotic arm is controlled to move based on the N interpolation positions, the first action position, and the second action position.

7. The method according to claim 6, characterized in that, The step of determining N interpolation positions based on the first position, the first action position, and the second action position includes: Obtain the model inference cycle corresponding to the target end-to-end model, and the servo frequency corresponding to the robotic arm; The number of interpolations is determined based on the model inference cycle and the servo frequency; the number of interpolations is N. Based on the number of interpolations and the first position, interpolation is performed between the first action position and the second action position to obtain N interpolation positions.

8. A trajectory smoothing control device for a robotic arm, used to perform the method as described in any one of claims 1-7, characterized in that, A control unit used in a robotic arm control system, the system further including: a robotic arm and a camera unit, the device comprising: The acquisition module is used to acquire the first state data of the robotic arm based on the camera unit during the execution of a preset task; The instruction generation module is used to input the first state data into the target end-to-end model for processing and output the first action instruction; the target end-to-end model is obtained by training a preset end-to-end model based on the training state dataset of the robotic arm; The robotic arm control module is used to control the robotic arm to move based on the first action command, update the first state data after the robotic arm completes the movement, regenerate a new action command based on the updated first state data, and control the robotic arm to move based on the new action command until the robotic arm completes the preset task; or, if the time taken for the robotic arm to perform the preset task is greater than the preset time, it is determined that the preset task has failed. Specifically, before inputting the first state data into the target end-to-end model for processing and outputting the first action command, the device is further configured to: Determine the basic task loss function corresponding to the preset end-to-end model; Obtain the regularization coefficient and the temporal difference smoothing loss function; The training loss function is determined based on the regularization coefficient, the temporal difference smoothing loss function, and the basic task loss function; The preset end-to-end model is trained based on the training loss function and the training state dataset to obtain the target end-to-end model; Specifically, in the step of inputting the first state data into the target end-to-end model for processing and outputting the first action command, the command generation module is used for: The first state data is input into the target end-to-end model for processing, and a reference action command is output. Obtain the previous action instruction output by the target end-to-end model before the reference action instruction to obtain the historical action instruction; The first action instruction is determined based on the preset exponential moving average formula, the preset smoothing factor, the historical action instructions, and the reference action instructions.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images