Network training method and device, electronic equipment and computer readable storage medium

Abstract

This application provides a network training method, apparatus, electronic device, and computer-readable storage medium. The method includes: acquiring real-time joint angle vectors of a robotic arm in a simulation environment; calling a joint planning network to map the real-time joint angle vectors to obtain joint angle increment vectors; determining the first joint angle vector of the robotic arm at the next moment based on the joint angle increment vector and the real-time joint angle vector, and determining the first end-effector position corresponding to the first joint angle vector; determining a distance loss value based on the first end-effector position and the first target position of a preset target point in the simulation environment, and determining a velocity smoothness loss value based on the joint angle increment vector; determining a target loss gradient based on the distance loss value and the velocity smoothness loss value; and adjusting the network parameters of the joint planning network based on the target loss gradient. This application can improve the accuracy and smoothness of robotic arm joint planning.

Description

Technical Field

[0001] This application relates to the field of robotic arm joint planning, and more particularly to a network training method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] The core objective of robotic arm joint planning is to generate a collision-free trajectory from the starting position to the target position, while satisfying the robotic arm's own kinematic and dynamic constraints, as well as the geometric constraints posed by obstacles in the workspace. The significance of this research lies in improving specific performance indicators of the robotic arm's motion by optimizing the trajectory, such as minimizing execution time, reducing energy consumption, or ensuring smooth motion to reduce impact on the mechanical structure. Therefore, joint planning technology is fundamental to determining the motion performance, safety, and environmental adaptability of robotic arm systems, and is an essential technical support for achieving automated and intelligent robotic operations.

[0003] In related technologies, joint planning for robotic arms relies on kinematic modeling of the robotic arm and solving for the joint angles of the robotic arm through analytical methods or numerical iteration methods. This results in low accuracy and smoothness in joint planning for robotic arms. Summary of the Invention

[0004] This application provides a network training method, apparatus, electronic device, and computer-readable storage medium that can improve the accuracy and smoothness of robotic arm joint planning.

[0005] The technical solution of this application embodiment is implemented as follows: This application provides a network training method, the method comprising: Obtain the real-time joint angle vectors of the robotic arm in the simulation environment, and perform the following processing until the training termination condition is met to obtain the trained joint planning network: The joint planning network is invoked to map the real-time joint angle vector to obtain the joint angle increment vector. Based on the joint angle increment vector and the real-time joint angle vector, the first joint angle vector of the robotic arm at the next moment is determined, and the first end position corresponding to the first joint angle vector is determined. The distance loss value is determined based on the first end position and the first target position of the preset target point in the simulation environment, and the velocity smoothness loss value is determined based on the joint angle increment vector; The target loss gradient is determined based on the distance loss value and the velocity smoothness loss value; The network parameters of the joint planning network are adjusted based on the target loss gradient, and the first joint angle vector is determined as the real-time joint angle vector.

[0006] This application provides a network training device, including: The mapping processing module is used to obtain the real-time joint angle vectors of the robotic arm in the simulation environment and perform the following processing until the training end condition is met to obtain the trained joint planning network: calling the joint planning network to perform mapping processing on the real-time joint angle vectors to obtain the joint angle increment vectors; The first determining module is used to determine the first joint angle vector of the robotic arm at the next moment based on the joint angle increment vector and the real-time joint angle vector, and to determine the first end position corresponding to the first joint angle vector. The second determining module is used to determine the distance loss value based on the first end position and the first target position of the preset target point in the simulation environment, and to determine the velocity smoothness loss value based on the joint angle increment vector; The third determining module is used to determine the target loss gradient based on the distance loss value and the velocity smoothness loss value; The parameter adjustment module is used to adjust the network parameters of the joint planning network based on the target loss gradient, and to determine the first joint angle vector as the real-time joint angle vector.

[0007] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the network training method provided in the embodiments of this application.

[0008] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which are executed by a processor to implement the network training method provided in this application.

[0009] This application provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the network training method provided in this application.

[0010] The embodiments of this application have the following beneficial effects: Using the embodiments of this application, the real-time joint angle vectors of the robotic arm in the simulation environment are obtained, and the following processing is performed until the training termination condition is met, resulting in a trained joint planning network: The joint planning network is called to map the real-time joint angle vectors to obtain joint angle increment vectors. Based on the joint angle increment vectors and the real-time joint angle vectors, the first joint angle vector of the robotic arm at the next moment is determined, and the first end position corresponding to the first joint angle vector is determined. A mapping relationship from the current joint state of the robotic arm to the joint state at the next moment is established. This mapping relationship is implemented through a neural network, avoiding the complexity and high computational cost of traditional inverse kinematics solutions, and providing a technical foundation for realizing high-speed and high-precision movement of the robotic arm. Then, based on the first end position and the first target position of the preset target point in the simulation environment, the distance loss value is determined, and the speed smoothness loss value is determined based on the joint angle increment vector. Then, based on the distance loss value and the speed smoothness loss value, the target loss gradient is determined, and then the network parameters of the joint planning network are adjusted based on the target loss gradient, and the first joint angle vector is determined as the real-time joint angle vector. This enables the joint planning network to efficiently learn a nonlinear strategy that maps real-time joint angle vectors to optimal joint angle increment vectors. As a result, in subsequent inference applications, the trained joint planning network can directly output joint angle increment vectors that balance target point approximation and motion smoothness, thus improving the accuracy and smoothness of robotic arm joint planning. Attached Figure Description

[0011] Figure 1 This is a schematic diagram illustrating the application mode of the network training method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 3 This is a first flowchart illustrating the network training method provided in this application embodiment; Figure 4 This is a second flowchart illustrating the network training method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the process for evaluating a trained joint planning network according to an embodiment of this application; Figure 6 This is a schematic diagram of the joint planning network training process provided in an embodiment of this application; Figure 7 This is a flowchart illustrating the joint planning method provided in an embodiment of this application; Figure 8 This is a schematic diagram of the seven-axis robotic arm provided in an embodiment of this application; Figure 9 This is a schematic diagram of the process for training and evaluating the joint planner neural network provided in an embodiment of this application; Figure 10This is a schematic diagram of the simulation environment provided in the embodiments of this application; Figure 11 This is a schematic diagram of the joint planner neural network provided in an embodiment of this application; Figure 12 This is a schematic diagram of the gradient calculation graph provided in an embodiment of this application; Figure 13 This is a schematic diagram of continuous trajectory frames provided in an embodiment of this application.

[0012] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0015] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0016] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0017] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0018] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0019] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0020] 1) Seven-axis robotic arm: It is a multi-degree-of-freedom mechanical device composed of multiple links connected by joints. It is a seven-axis redundant degree-of-freedom robotic arm with seven rotary joints.

[0021] 2) Joint planning network: It is a multi-layer perceptron (MLP) consisting of an input layer, two hidden layers and an output layer, used to realize the mapping from the current joint angle vector of the robotic arm to the joint angle increment vector.

[0022] This application provides a network training method, apparatus, electronic device, and computer-readable storage medium that can improve the accuracy and smoothness of robotic arm joint planning.

[0023] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following will describe exemplary applications when the device is implemented as a server.

[0024] See Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the network training method provided in the embodiments of this application, for example. Figure 1 The system involves server 200, network 300, and terminal 400. Terminal 400 connects to server 200 through network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0025] During the training of the joint planning network, the terminal 400 sends its real-time joint angle vectors in the simulation environment to the server 200. The server 200 obtains the real-time joint angle vectors and performs the following processes until the training termination condition is met, resulting in the trained joint planning network: The joint planning network is invoked to map the real-time joint angle vectors to obtain the joint angle increment vector; based on the joint angle increment vector and the real-time joint angle vector, the first joint angle vector of the robotic arm at the next moment is determined, and the first end-effector position corresponding to the first joint angle vector is determined; based on the first end-effector position and the first target position of a preset target point in the simulation environment, the distance loss value is determined, and the velocity smoothness loss value is determined based on the joint angle increment vector; based on the distance loss value and the velocity smoothness loss value, the target loss gradient is determined; based on the target loss gradient, the network parameters of the joint planning network are adjusted, and the first joint angle vector is determined as the real-time joint angle vector. The robotic arm can be a seven-axis robotic arm used for tasks such as target tracking and motion planning, i.e., the terminal 400.

[0026] After obtaining the trained joint planning network, it needs to be evaluated to determine its predictive performance. During the evaluation, server 200 generates test target points in a simulation environment and determines the second joint angle vector of the robotic arm when the joint planning network training is complete. The trained joint planning network is then invoked, and based on the second joint angle vector and the test target points, joint planning is performed for a preset number of steps to obtain the robotic arm's motion trajectory. The planning effect is evaluated based on the robotic arm's motion trajectory, and an evaluation result is obtained. If the evaluation result is satisfactory, the trained joint planning network can be deployed to terminal 400. For example, in a target tracking scenario, terminal 400 performs object tracking and grasping based on the trained joint planning network. Alternatively, in a human-computer interaction scenario, terminal 400 performs tasks such as serving tea or water and folding clothes based on the trained joint planning network.

[0027] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a terminal or a server. Figure 2 The illustrated electronic device includes at least one processor 410, a memory 450, and at least one network interface 420. The various components of the electronic device are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0028] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0029] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0030] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0031] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0032] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0033] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0034] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A network training device 455 stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: mapping processing module 4551, first determination module 4552, second determination module 4553, third determination module 4554, and parameter adjustment module 4555. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0035] The network training method provided in this application will be described in conjunction with exemplary applications and implementations of the server devices provided in the embodiments of this application.

[0036] The following describes the network training method provided in the embodiments of this application. For example, in order to facilitate understanding of the network training method provided in the embodiments of this application, the description is based on the scenario of applying it to the joint planning network of a training robotic arm.

[0037] As mentioned above, the electronic device implementing the network training method of this application embodiment can be a terminal, a server, or a combination of both. The following explanation uses an electronic device as a server as an example to illustrate the network training method provided in this application embodiment. See also... Figure 3 , Figure 3 This is a first flowchart illustrating the network training method provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0038] In step 301, the real-time joint angle vector of the robotic arm in the simulation environment is obtained.

[0039] Here, the robotic arm can be a seven-axis robotic arm. A seven-axis robotic arm is a multi-degree-of-freedom mechanical device composed of multiple links connected by joints; it is a seven-axis redundant degree-of-freedom robotic arm with seven rotary joints. The simulation environment is used to simulate the physical characteristics and kinematic behavior of the robotic arm. It is built based on the open-source simulation engine (Nvidia Newton). The simulation engine can instantiate the robotic arm model based on the imported robot description file (Unified Robot Description Format, URDF) and simulate its motion in three-dimensional space. This simulation environment has differentiability, meaning that the functional relationship between physical quantities (such as position and velocity) in the simulation environment and input parameters (such as joint angles) is differentiable.

[0040] The real-time joint angle vector is a set of values ​​used to describe the current state of each joint of the robotic arm. The dimension of the real-time joint angle vector is equal to the number of joints of the robotic arm (7 in this embodiment). Each component in the real-time joint angle vector corresponds to the rotation angle value of a joint, which together uniquely determine the posture of the robotic arm in the joint space at that moment.

[0041] After obtaining the real-time joint angle vectors, the following processing can be performed repeatedly until the training termination condition is met, resulting in the trained joint planning network.

[0042] In step 302, the joint planning network is invoked to map the real-time joint angle vectors to obtain the joint angle increment vectors.

[0043] The joint planning network is a multi-layer perceptron (MLP) consisting of an input layer, two hidden layers, and an output layer, used to map the current joint angle vector of a robotic arm to the joint angle increment vector.

[0044] In some embodiments, the joint planning network includes a first hidden layer, a second hidden layer, and an output layer. The joint planning network is invoked to map real-time joint angle vectors to obtain joint angle increment vectors. This can be achieved through the following steps: The first parameter matrix and first bias vector in the first hidden layer are used to perform a linear operation on the real-time joint angle vectors to obtain a first representation vector; the first representation vector is then subjected to a nonlinear mapping to obtain a second representation vector. The second parameter matrix and second bias vector in the second hidden layer are used to perform a linear operation on the second representation vector to obtain a third representation vector; the third representation vector is then subjected to a nonlinear mapping to obtain a fourth representation vector. Finally, the third parameter matrix and third bias vector in the output layer are used to perform a linear operation on the fourth representation vector to obtain a fifth representation vector; the fifth representation vector is then normalized to obtain the joint angle increment vector.

[0045] Here, the network dimension of the first hidden layer is 64, the dimension of the real-time joint angle vector is 7, and the first parameter matrix in the first hidden layer is represented as follows: The first bias vector in the first hidden layer is represented as In the first hidden layer, a linear operation is performed on the real-time joint angle vector based on the first parameter matrix and the first bias vector to obtain the first representation vector. For example, the first representation vector is determined by the following formula (1): (1) in, Denotes the first representation vector. This represents the input to the joint planning network, i.e., the real-time joint angle vector.

[0046] The first representation vector is nonlinearly mapped by the activation function (ReLU) to obtain the second representation vector. For example, the second representation vector is determined by the following formula (2): (2) in, Represents the second representation vector. This represents the activation function.

[0047] The network dimension of the second hidden layer is 64, and the second parameter matrix in the second hidden layer is represented as follows: The second bias vector in the second hidden layer is represented as In the second hidden layer, a linear operation is performed on the second representation vector based on the second parameter matrix and the second bias vector to obtain the third representation vector. For example, the third representation vector is determined by the following formula (3): (3) in, Represents the third representation vector. This represents the second representation vector.

[0048] The activation function (ReLU) is called to perform nonlinear mapping on the third representation vector, that is, to perform the processing in the above formula (2) on the third representation vector to obtain the fourth representation vector.

[0049] The output layer has a network dimension of 7, and the third parameter matrix in the output layer is represented as follows: The third bias vector in the output layer is represented as In the output layer, a linear operation is performed on the fourth representation vector based on the third parameter matrix and the third bias vector to obtain the fifth representation vector. For example, the fifth representation vector is determined by the following formula (4): (4) in, Represents the fifth representation vector. This represents the fourth representation vector.

[0050] The fifth representation vector is normalized by calling the normalization function (Softmax) to map it to the interval [-0.1, 0.1], thus obtaining the joint angle increment vector. The joint angle increment vector is a set of values ​​used to describe the angle changes of each joint of the robotic arm. Each component in the joint angle increment vector corresponds to the angle change value of a joint. For example, the joint angle increment vector is determined by the following formula (5): (5) in, Represents the joint angle increment vector. Represents the normalization function. This represents the fifth representation vector.

[0051] In this embodiment, the first hidden layer, the second hidden layer, and the output layer of the joint planning network are called respectively to map the real-time joint angle vectors to obtain the joint angle increment vector. This enables layer-by-layer feature extraction and dimensional transformation of the real-time joint angle vectors, mapping low-dimensional joint state information to a high-dimensional feature space. This allows for learning and characterizing the complex intrinsic relationship between the current joint state of the robotic arm and the optimal motion direction. The complex inverse kinematics problem, which requires iterative solution, is transformed into a matrix operation process with extremely low computational cost, providing a computational foundation for achieving real-time and rapid response in joint planning.

[0052] Continue to refer to Figure 3 In step 303, based on the joint angle increment vector and the real-time joint angle vector, the first joint angle vector of the robotic arm at the next moment is determined, and the first end position corresponding to the first joint angle vector is determined.

[0053] Here, the real-time joint angle vector is incrementally updated based on the joint angle increment vector to obtain the joint angle vector of the robotic arm at the next moment, which is also the first joint angle vector. The first end effector position is the position of the end effector of the robotic arm when the joint angle vector is the first joint angle vector.

[0054] In some embodiments, determining the first end-effector position corresponding to the first joint angle vector can be achieved through the following steps: obtaining the kinematic model parameters of the robotic arm, and constructing homogeneous transformation matrices corresponding to each joint based on the first joint angle vector and the kinematic model parameters; performing cascade multiplication on each homogeneous transformation matrix based on the joint connection order of the robotic arm to obtain the end-effector pose matrix; and performing coordinate analysis on the end-effector pose matrix to obtain the first end-effector position.

[0055] Here, the kinematic model parameters are a set of numerical parameters used to describe the fixed geometric relationships of each link in the robotic arm, including link length, link torsion angle, and link offset. The first joint angle vector and the kinematic model parameters are calculated using a preset kinematic transformation formula to obtain the homogeneous transformation matrix corresponding to each joint. Each homogeneous transformation matrix includes a set of numerical values ​​describing rotational transformation and a set of numerical values ​​describing translational transformation, used to define the spatial pose relationship between the coordinate system of the next link connected by a joint and the coordinate system of the previous link.

[0056] The joint connection sequence is the physical connection order from the robot arm's base to the end effector. Based on this sequence, the homogeneous transformation matrices of each joint are multiplied sequentially. The homogeneous transformation matrix of the first joint closest to the robot arm base is multiplied, along with the homogeneous transformation matrix of the second joint closest to the first joint. This result is then multiplied with the result of the homogeneous transformation matrix of the third joint closest to the second joint, yielding an updated result. This process is repeated until the multiplication of the homogeneous transformation matrix of the last joint (the joint connecting to the end effector) is complete, resulting in the end effector pose matrix. The end effector pose matrix describes the spatial orientation and position of the robot arm's end effector relative to the robot arm's base coordinate system.

[0057] The end effector pose matrix is ​​analyzed by coordinate analysis, which involves extracting numerical components representing spatial position from the end effector pose matrix and extracting translations corresponding to the three principal axes of the base coordinate system from these numerical components. These three translations constitute the three-dimensional spatial coordinates of the end effector center point in the base coordinate system, i.e., the first end effector position.

[0058] In this embodiment, based on the first joint angle vector and kinematic model parameters of the robotic arm at the next moment, homogeneous transformation matrices corresponding to each joint are constructed. Based on the joint connection order of the robotic arm, cascaded multiplication of each homogeneous transformation matrix is ​​performed to obtain the end-effector pose matrix. The end-effector pose matrix is ​​then analyzed to obtain the first end-effector position. This achieves the construction of a homogeneous transformation matrix representing the relative position and attitude of each joint. The homogeneous transformation matrices are then cascaded multiplied according to the physical connection order of the robotic arm from the base to the end-effector. The final end-effector pose matrix contains the position and attitude information of the end effector relative to the base coordinate system, thereby accurately calculating the coordinates of the robotic arm end-effector in three-dimensional space. This provides a computational basis for subsequently determining the distance loss value between the position of the robotic arm end-effector and the position of the target point.

[0059] Continue to refer to Figure 3 In step 304, the distance loss value is determined based on the first end position and the first target position of the preset target point in the simulation environment, and the velocity smoothness loss value is determined based on the joint angle increment vector.

[0060] Here, the preset target point is a target point (which can be in the form of a square, circle, etc.) randomly generated in the simulation environment for training the joint planning network, and the first target position is the position of the preset target point in the simulation environment. The Euclidean distance between the first end position and the first target position is calculated to obtain the distance loss value. For example, the distance loss value is determined by the following formula (6): (6) in, express Distance loss value at time step express The first end position of time, Indicates the location of the first target.

[0061] In some embodiments, the joint angle increment vector includes a first increment value corresponding to each joint angle. Determining the speed smoothness loss value based on the joint angle increment vector can be achieved through the following steps: determining a first number of joints of the robotic arm and determining the historical increment vector of the previous moment, wherein the historical increment vector includes the historical increment value corresponding to each joint angle; based on the first number, summing and averaging the differences between the first increment value and the historical increment value corresponding to each joint angle to obtain a first processing result; when the first processing result is less than or equal to a first preset threshold, determining the speed smoothness loss value as a preset value; when the first processing result is greater than the first preset threshold, smoothing the first processing result based on a first preset coefficient to obtain a second processing result, and determining the second processing result as the speed smoothness loss value.

[0062] Here, taking a seven-axis robotic arm as an example, the number of joints in the robotic arm is 7, that is, the first quantity is 7. The joint planning network is called to map the joint angle vector of the robotic arm at the previous time step, to obtain the joint angle increment vector of the robotic arm at the previous time step, that is, the historical increment vector of the previous time step.

[0063] Since each joint of the robotic arm corresponds to a joint angle, meaning the number of joints and the number of joint angles are the same, the number of joint angles is also known as the first quantity. The differences between the first incremental value corresponding to the first quantity of joint angles and the historical incremental values ​​are summed and averaged to obtain the first processing result. For example, the first processing result is represented as... ,in, Indicates the first The joint angles are in The first increment value at time 1, Indicates the first Each joint angle is Historical increment value at any given moment Indicates the first quantity.

[0064] The first preset threshold is a speed smoothness threshold. When the first processing result is less than or equal to the first preset threshold, the speed smoothness loss value is determined as the preset value, which can be 0. The first preset coefficient is a smoothing coefficient. When the first processing result is greater than the first preset threshold, the first processing result is smoothed based on the first preset coefficient to obtain the second processing result, and the second processing result is directly determined as the speed smoothness loss value. For example, the speed smoothness loss value is determined by the following formula (7): (7) in, express The velocity smoothness loss value at time step Indicates the first The joint angles are in The first increment value at time 1, Indicates the first The joint angles are in Historical increment value at any given moment Indicates the first quantity. This represents the first preset coefficient. This indicates the first preset threshold.

[0065] In this embodiment, the difference between the first incremental value and the historical incremental value corresponding to each joint angle is summed and averaged to obtain a first processing result. When the first processing result is less than or equal to a first preset threshold, the speed smoothness loss value is determined as a preset value. When the first processing result is greater than the first preset threshold, the first processing result is smoothed based on a first preset coefficient to obtain a second processing result, and the second processing result is determined as the speed smoothness loss value. This realizes the speed smoothness loss value that quantifies the continuity of joint angular velocity during the movement of the robotic arm. The speed smoothness loss value can be directly used to calculate the target loss gradient, thereby guiding the parameters of the joint planning network to be optimized in the direction of generating more continuous and smoother joint angle increment vectors, so as to improve the smoothness of the actual movement of the robotic arm and reduce impact and vibration.

[0066] Continue to refer to Figure 3 In step 305, the target loss gradient is determined based on the distance loss value and the velocity smoothness loss value.

[0067] Here, the target loss value is determined based on the distance loss value and the velocity smoothness loss value. Backpropagation is performed on the target loss value to obtain the set of partial derivatives of the target loss function at the network parameters of the joint planning network, and the set of partial derivatives is determined as the target loss gradient.

[0068] In some embodiments, see Figure 4 , Figure 4 This is a schematic diagram of the second process of the network training method provided in the embodiments of this application. Figure 3 Step 305 shown can be achieved through... Figure 4 Steps 3051 to 3054 are implemented, and will be explained in detail below.

[0069] In step 3051, the distance loss value and the velocity smoothness loss value are fused to obtain the target loss value.

[0070] Here, based on the first weighting coefficient corresponding to the distance loss value and the second weighting coefficient corresponding to the velocity smoothness loss value, the distance loss value and the velocity smoothness loss value are weighted and summed to achieve fusion processing and obtain the target loss value. For example, the target loss value is determined using the following formula (8): (8) in, Indicates the target loss value. This represents the distance loss value. This represents the speed smoothness loss value. and These represent the first weighting coefficient and the second weighting coefficient, respectively.

[0071] In step 3052, a first objective parameter with a gradient and a second objective parameter without a gradient are determined from the network parameters.

[0072] Here, the network parameters of the joint planning network include real-time joint angles (joint_q), real-time joint angular velocities (joint_qd), initial joint angles (joint_q_start), initial joint angular velocities (joint_qd_start), poses of links and end effectors (body_q), velocities of links and end effectors (body_qd), joint type (joint_type), joint axis (joint_axis), parent / child joints (joint_parent / joint_child), parent / child joint transformation matrices (joint_X_p / joint_X_c), joint degrees of freedom (joint_dof_dim), and the centroid positions of each link (body_com). The real-time joint angular velocities, poses of links and end effectors, and velocities of links and end effectors are determined to be differentiable parameters, i.e., the first objective parameters with gradients. The remaining network parameters are determined to be non-differentiable parameters, i.e., the second objective parameters without gradients.

[0073] In step 3053, a gradient calculation graph is constructed based on the first target parameter, the second target parameter, and the preset differential operator.

[0074] Here, the first node corresponding to the first objective parameter and the second node corresponding to the second objective parameter in the gradient computation graph are interconnected. The preset differential operator is the gradient function operator, including the differential operator of multiplication (kernel0) and the differential operator of exponentiation / root extraction (kernel1).

[0075] In some embodiments, the gradient computation graph is constructed based on the first target parameter, the second target parameter, and the preset differential operator, which can be achieved through the following steps: constructing a first node based on the first target parameter and constructing a second node based on the second target parameter; constructing a directed acyclic graph based on the first node, the second node, and the preset differential operator, and determining the directed acyclic graph as the gradient computation graph.

[0076] Here, a data structure for a first node corresponding to the first target parameter is created. This data structure for the first node is configured with a storage unit for storing the current value of the first target parameter, and an identifier for enabling gradient calculation is set. This identifier is used to indicate that this node is a gradient node that needs to calculate and store its corresponding gradient value during gradient backpropagation.

[0077] Create a data structure for the second node corresponding to the second objective parameter. This data structure for the second node is configured with a storage unit for storing the value of the second objective parameter, and a flag is set to disable gradient calculation. This flag is used to indicate that the node is a constant node during gradient backpropagation and does not participate in the calculation and updating of gradient values.

[0078] The first and second nodes are defined as input nodes in the gradient computation graph. Preset differential operators are instantiated as computation nodes in the gradient computation graph. Directed connections are established between input nodes and computation nodes, resulting in a graph structure composed of nodes and directed edges, i.e., a directed acyclic graph (DAG). This DAG is then defined as the gradient computation graph. This gradient computation graph contains no closed loops, represents the data flow and computation order, and clarifies the dependencies between operations: from one input node to one computation node, or from the output of one computation node to the input of another. See the example for reference. Figure 12 , Figure 12 This is a schematic diagram of the gradient calculation graph provided in an embodiment of this application. For example... Figure 12 As shown, the nodes corresponding to the first target parameters "real-time joint angular velocity", "pose of link and end effector" and "velocity of link and end effector" are the first nodes, and the nodes corresponding to the remaining second target parameters are the second nodes.

[0079] In this embodiment, a first node is constructed based on a first target parameter with a gradient, and a second node is constructed based on a second target parameter without a gradient. A directed acyclic graph is constructed based on the first node, the second node, and a preset differential operator, and the directed acyclic graph is determined as the gradient calculation graph. This allows the gradient calculation graph to clearly and completely record the dependency path of the target loss value on each first target parameter, and to automatically and accurately solve the partial derivatives of the target loss function at each trainable parameter (first target parameter). This structurates the gradient solution process and provides the necessary data structure foundation for the subsequent automatic gradient calculation using the backpropagation algorithm.

[0080] Continue to refer to Figure 4 In step 3054, in the gradient calculation graph, the partial derivatives of the target loss function at each first target parameter are calculated layer by layer based on the target loss value, and the partial derivatives are combined into the target loss gradient.

[0081] Here, the output node representing the final calculation result is located in the gradient computation graph. This output node stores the calculation result of the target loss function, i.e., the target loss value. Using this output node as the starting point for backpropagation, gradient calculation and propagation are performed layer by layer along the reverse direction of the connection relationships in the gradient computation graph, towards the input nodes (i.e., the first and second nodes) and the computation nodes (i.e., the differential operators). For any computation node in the gradient computation graph, the partial derivatives of its output gradient with respect to each of the first target parameters at its input are calculated using the predefined differential rules of that computation node. A set of partial derivatives is constructed based on these partial derivatives, and this set of partial derivatives is determined as the target loss gradient.

[0082] In this embodiment, the distance loss value and the velocity smoothness loss value are first fused to obtain the target loss value. Then, a gradient calculation graph is constructed based on the first target parameter with gradient, the second target parameter without gradient, and a preset differential operator in the network parameters. Finally, in the gradient calculation graph, the partial derivatives of the target loss function at each first target parameter are calculated layer by layer based on the target loss value, and the partial derivatives are combined to form the target loss gradient. This realizes the reverse propagation of the target loss value layer by layer in the gradient calculation graph, which can automatically and accurately solve the partial derivatives of the target loss function with respect to each trainable parameter (first target parameter), and combine these partial derivatives to form the target loss gradient. This provides a definite direction and quantitative basis for subsequent optimization algorithms such as gradient descent to update the network parameters.

[0083] Continue to refer to Figure 3 In step 306, the network parameters of the joint planning network are adjusted based on the target loss gradient, and the first joint angle vector is determined as the real-time joint angle vector.

[0084] Here, the network parameters of the joint planning network are adjusted based on the target loss gradient to achieve a comprehensive update of the network parameters, resulting in the trained joint planning network at the current time step. For example, the updated network parameters of the joint planning network are determined using the following formula (9): (9) in, express The network parameters of the joint planning network are updated over time. express Network parameters of the joint planning network at different times. This represents the gradient of the target loss.

[0085] The first joint angle vector of the robotic arm at the next moment is determined as the real-time joint angle vector, so that the above training process can be repeated based on the real-time joint angle vector if the training termination condition is not met.

[0086] In step 307, it is determined whether the training termination condition has been met.

[0087] Here, the training is considered complete when the number of training steps (which refers to the process of adjusting the network parameters of the joint planning network once) reaches the preset step threshold or when the end of the robotic arm reaches the vicinity of the target point.

[0088] If the training termination condition is not met, proceed to step 302 to continue training the joint planning network; if the training termination condition is determined to be met, proceed to step 308.

[0089] In step 308, the trained joint planning network is obtained.

[0090] Based on steps 301 to 308 above, a joint planning network is constructed with the real-time joint angle vector of the robotic arm as input and the joint angle increment vector as output. The target loss gradient, determined by the distance loss between the robotic arm end position and the target point and the velocity smoothness loss of the joint angle increment vector, is used to iteratively train the joint planning network. This enables the joint planning network to efficiently learn a nonlinear strategy that maps the real-time joint angle vector to the optimal joint angle increment vector. In subsequent inference applications, the trained joint planning network can directly output the joint angle increment vector that takes into account both the approach to the target point and the smoothness of the motion process, thereby improving the accuracy and smoothness of the robotic arm joint planning and achieving real-time and generalizable joint planning results that do not depend on a specific robotic arm model.

[0091] In some embodiments, the trained joint planning network needs to be evaluated before it can be deployed. See also Figure 5 , Figure 5 This is a flowchart illustrating the evaluation of a trained joint planning network provided in an embodiment of this application. The evaluation process can be performed through... Figure 5 Steps 310 to 312 are implemented, and will be explained in detail below.

[0092] In step 310, test target points are generated in the simulation environment, and the second joint angle vector of the robotic arm is determined when the joint planning network training is completed.

[0093] Here, one or more target points that did not appear during the training phase are randomly generated in the simulation environment, i.e., test target points. The moment when the joint planning network training is completed is obtained, and the real-time joint angle vector of the robotic arm at that moment is determined as the second joint angle vector.

[0094] In step 311, the trained joint planning network is invoked, and joint planning processing is performed for a preset number of steps based on the second joint angle vector and the test target point to obtain the robotic arm motion trajectory.

[0095] Here, joint planning is an iterative computation process. In a single joint planning iteration, the second joint angle vector and the position of the test target point are combined to form a composite input data that meets the input format requirements of the joint planning network. The trained joint planning network is then invoked to perform a forward computation on this composite input data. That is, based on the trained weight parameters in the trained joint planning network, a nonlinear transformation is performed on the composite input data to obtain the target joint angle vector of the robotic arm at the next moment. This target joint angle vector is then determined as the second joint angle vector used in the next joint planning iteration.

[0096] Repeat the single joint planning process described above until the preset number of steps is reached. Record and store the joint angle vectors generated in each joint planning process in the order of iteration, forming an ordered sequence containing all intermediate joint state data from the initial state to the final planned state. Perform motion trajectory analysis on this ordered sequence to obtain the motion trajectory of the robotic arm from the current position to the vicinity of the test target point.

[0097] In step 312, the planning effect is evaluated based on the robotic arm's motion trajectory, and the evaluation results are obtained.

[0098] Here, the target distance between the end effector of the robotic arm and the test target point is determined based on the robotic arm's motion trajectory, as well as the change in angular velocity of the robotic arm at multiple adjacent time points. The planning effect of the trained joint planning network is evaluated by combining the target distance and the change in angular velocity.

[0099] In some embodiments, the evaluation of the planning effect based on the robotic arm's motion trajectory can be achieved through the following steps: determining the second end position corresponding to the last trajectory point of the robotic arm's motion trajectory, and determining the target distance between the second end position and the second target position of the test target point; determining the angular velocity change of the robotic arm in multiple adjacent moments based on the robotic arm's motion trajectory, and averaging each angular velocity change to obtain a third processing result; when the target distance is less than a second preset threshold and the third processing result is less than a third preset threshold, the evaluation result is determined to be passed.

[0100] Here, the final trajectory point of the robotic arm's motion is located, and the position of the end effector at that moment is defined as the second end position. The coordinate difference between the second end position and the second target position of the test target point is processed using the Euclidean norm to obtain the target distance. Within each two adjacent moments in the robotic arm's motion trajectory, the absolute value of the change in angular velocity is calculated, and the arithmetic mean of these absolute values ​​is taken to obtain a scalar value, which is defined as the third processing result. This third processing result is used to quantitatively evaluate the smoothness of the robotic arm's motion trajectory.

[0101] The second preset threshold is used to evaluate positioning accuracy, and the third preset threshold is used to evaluate motion smoothness. When the target distance is less than the second preset threshold and the third processing result is less than the third preset threshold, it indicates that the robotic arm's motion trajectory meets the preset positioning accuracy and motion smoothness, and the evaluation result is determined to be passed.

[0102] For example, the second preset threshold can be set to 1.0 cm, the third preset threshold can be set to 0.1 rad / s, the target distance is 0.64 cm, and the third processing result is 0.064 rad / s. At this time, if the target distance is determined to be less than the second preset threshold and the third processing result is less than the third preset threshold, then the evaluation result is determined to be passed.

[0103] In this embodiment, the second joint angle vector of the robotic arm is determined when the joint planning network training is completed. The trained joint planning network is then invoked, and joint planning processing with a preset number of steps is performed based on the second joint angle vector and the test target point to obtain the robotic arm's motion trajectory. The planning effect is evaluated based on the robotic arm's motion trajectory, thereby achieving an objective and quantitative evaluation of the planning capability of the trained joint planning network. This more realistically reflects the comprehensive performance of the trained joint planning network when facing new planning tasks, thus providing reliable technical support for confirming whether it meets the preset performance requirements and whether it is suitable for subsequent actual deployment.

[0104] In some embodiments, when the target distance is greater than or equal to a second preset threshold, or when the third processing result is greater than or equal to a third preset threshold, the evaluation result is determined to be an evaluation failure. At this time, the real-time joint angle vector of the robotic arm in the simulation environment is obtained again, and the above training process is repeated on the trained joint planning network based on the real-time joint angle vector until the evaluation result is an evaluation success.

[0105] In this embodiment, the target distance between the second end position corresponding to the last trajectory point of the robotic arm's motion trajectory and the second target position of the test target point is determined. Based on the robotic arm's motion trajectory, the angular velocity change of the robotic arm in multiple adjacent moments is determined, and the angular velocity change is averaged to obtain a third processing result. When the target distance is less than a second preset threshold and the third processing result is less than a third preset threshold, the evaluation result is determined to be passed. This establishes a dual evaluation system that takes into account both the accuracy of the planning result and the smoothness of the motion, which can more comprehensively evaluate the overall performance of the joint planning network and ensure that the generated robotic arm motion trajectory can not only accurately reach the target point, but also that the robotic arm has a low acceleration change during the motion.

[0106] Example, reference Figure 6 , Figure 6 This is a schematic diagram of the joint planning network training process provided in this application embodiment. The real-time joint angle vectors of the robotic arm in the simulation environment are obtained, and the following processes are performed until the training termination condition is met, resulting in the trained joint planning network: The joint planning network 320 is invoked to map the real-time joint angle vector 321 to obtain the joint angle increment vector; Based on the joint angle increment vector and the real-time joint angle vector 321, the first joint angle vector of the robotic arm at the next moment is determined, and the first end-effector position corresponding to the first joint angle vector is determined; Based on the first end-effector position and the first target position of a preset target point in the simulation environment, the distance loss value is determined, and the velocity smoothness loss value is determined based on the joint angle increment vector; Based on the distance loss value and the velocity smoothness loss value, the target loss gradient is determined; Based on the target loss gradient, the network parameters of the joint planning network are adjusted, and the first joint angle vector is determined as the real-time joint angle vector.

[0107] In some embodiments, the network training method provided in this application can be applied to tasks such as target tracking and motion planning. The real-time joint angle vectors of the robotic arm in a simulation environment are obtained, and the following processing is performed until the training termination condition is met, resulting in a trained joint planning network: The joint planning network is invoked to map the real-time joint angle vectors to obtain joint angle increment vectors. Based on the joint angle increment vectors and the real-time joint angle vectors, the first joint angle vector of the robotic arm at the next moment is determined, and the first end-effector position corresponding to the first joint angle vector is determined, establishing a mapping relationship from the current joint state of the robotic arm to the joint state at the next moment. This mapping relationship is implemented through a neural network, avoiding the complexity and high computational cost of traditional inverse kinematics solutions, providing a technical foundation for achieving high-speed, high-precision motion of the robotic arm. Then, based on the first end-effector position and the first target position of a preset target point in the simulation environment, a distance loss value is determined, and based on the joint angle increment vector, a velocity smoothness loss value is determined. Then, based on the distance loss value and the velocity smoothness loss value, a target loss gradient is determined, and the network parameters of the joint planning network are adjusted based on the target loss gradient, and the first joint angle vector is determined as the real-time joint angle vector. This enables the joint planning network to efficiently learn a nonlinear strategy that maps real-time joint angle vectors to optimal joint angle increment vectors. As a result, in subsequent inference applications, the trained joint planning network can directly output joint angle increment vectors that balance target point approximation and motion smoothness, thus improving the accuracy and smoothness of robotic arm joint planning.

[0108] The joint planning method provided in the embodiments of this application is described below. See also: Figure 7 , Figure 7 This is a flowchart illustrating the joint planning method provided in the embodiments of this application, which will be combined with... Figure 7 The steps shown are explained.

[0109] In step 701, the trained joint planning network is invoked to map the real-time joint angle vectors of the robotic arm in the simulation environment to obtain the target joint angle increment vector.

[0110] Here, the trained joint planning network includes a first target hidden layer, a second target hidden layer, and a target output layer. The parameter matrix and bias vector from the first target hidden layer are used to perform linear operations on the real-time joint angle vectors to obtain the first target representation vector. This first target representation vector is then subjected to nonlinear mapping to obtain the second target representation vector. The parameter matrix and bias vector from the second target hidden layer are then used to perform linear operations on the second target representation vector to obtain the third target representation vector. This third target representation vector is then subjected to nonlinear mapping to obtain the fourth target representation vector. Finally, the parameter matrix and bias vector from the target output layer are used to perform linear operations on the fourth target representation vector to obtain the fifth target representation vector. This fifth target representation vector is then normalized to obtain the target joint angle increment vector. That is, the process of determining the target joint angle increment vector in step 701 is the same as the process of determining the joint angle increment vector in step 301 above, and will not be repeated here.

[0111] In step 702, based on the target joint angle increment vector and the real-time joint angle vector, the first target joint angle vector of the robotic arm at the next moment is determined, and the target end position corresponding to the first target joint angle vector is determined.

[0112] Here, the real-time joint angle vector is incrementally updated based on the target joint angle increment vector to obtain the joint angle vector of the robotic arm at the next moment, which is also the first target joint angle vector. Based on the first target joint angle vector and the kinematic model parameters of the robotic arm, homogeneous transformation matrices corresponding to each joint are constructed. Based on the joint connection order of the robotic arm, cascaded multiplication operations are performed on each homogeneous transformation matrix to obtain the target end-effector pose matrix. Coordinate analysis is performed on the target end-effector pose matrix to obtain the target end-effector position. That is, the process of determining the target end-effector position in step 702 is the same as the process of determining the first end-effector position in step 302 above, and will not be repeated here.

[0113] Based on steps 701 to 702, the current joint angle vector of the robotic arm is obtained in real time as the input of the trained joint planning network. By utilizing the forward inference capability of the trained joint planning network, the joint angle increment vector that takes into account both the approach of the target point and the smoothness of the motion process is output. Thus, the corresponding end position of the robotic arm is calculated by using forward kinematics. This provides a technical means to achieve accurate and real-time motion control of the robotic arm in dynamic or uncertain environments, and improves the accuracy and smoothness of the robotic arm's motion.

[0114] The following will describe an exemplary application of the network training method provided in the embodiments of this application in the scenario of training a joint planner neural network for a robotic arm.

[0115] A reasonable joint angle planning scheme is crucial for robotic arms to accurately complete tasks such as target tracking and motion planning. Traditional robotic arm joint planning methods rely on kinematic modeling of the robotic arm and solve the joint angles through analytical or numerical iterative methods. These methods have problems such as slow solution speed, low solution accuracy, uneven joint angle changes, and complex solutions for redundant arms.

[0116] Seven-axis robotic arms, due to their high degrees of freedom and larger solution space, have become the mainstream choice for projects such as humanoid robots or manipulating robotic arms. Achieving precise control of a seven-axis robotic arm is fundamental to accomplishing the aforementioned tasks. (For example, see reference.) Figure 8 , Figure 8 This is a schematic diagram of a seven-axis robotic arm provided in an embodiment of this application. Traditional joint planning methods for seven-axis robotic arms rely on inverse kinematics solutions, which can be divided into analytical methods and numerical iteration methods.

[0117] Analytical methods, also known as geometric methods, simplify complex nonlinear equations by utilizing the specific geometry of the robotic arm (such as a ball-and-arm structure), thereby deriving analytical expressions for joint variables. Some common seven-axis robotic arms introduce additional joints in their wrist or shoulder, creating specific geometric constraints that transform their inverse kinematics problem into an equivalent six-axis problem plus an optimization problem with a redundant joint. However, analytical methods suffer from drawbacks such as dependence on specific structures and complex solutions.

[0118] Numerical iterative methods do not depend on the specific geometry of the robotic arm; instead, they solve inverse kinematics problems through iterative optimization, making them suitable for robotic arms without explicit geometric constraints. Common numerical iterative methods include the Jacobian matrix method (e.g., gradient projection method, damped least squares method), Newton's iteration method, and intelligent optimization algorithms (e.g., genetic algorithms, particle swarm optimization). Taking the commonly used Jacobian matrix method as an example, the joint angles are iteratively updated by constructing the Jacobian matrix of the robotic arm until the end-effector pose error is less than a threshold. Numerical iterative methods suffer from problems such as high computational cost, difficulty in local convergence, and low accuracy.

[0119] This application proposes a network training method to address the problems existing in related technologies, which includes the following improvements compared to related technologies: Based on a simulation environment with differentiable dynamics, a multilayer perceptron neural network (MLPN) is built as a joint planner (the joint planning network in the above embodiment). A loss function is constructed, and the gradients of relevant physical quantities in the simulation environment are calculated. Finally, the parameters of the joint planner are directly updated through backpropagation, achieving efficient training of the joint planner. This joint planner has the ability to plan the shortest path to the target point, and the planning speed is fast, the planning result is accurate and unique, and the planned trajectory is smooth. It has a significant improvement effect on achieving high real-time and high-precision control of robotic arms, and therefore has broad application prospects.

[0120] In this embodiment, the differentiable gradient of the target tracking loss function is directly used for training the neural network, which can achieve convergence of the neural network with very few training iterations. Subsequently, the neural network, which serves as the joint planner, outputs the joint angle planning results with extremely low inference time, effectively improving the real-time response capability of the joint planner and solving the problems of complex, slow, and poor real-time performance of traditional methods.

[0121] The embodiments of this application are applicable to robotic arms of any structure. They do not require adjustments to the joint planner training and inference process based on the shape and structure of the robotic arm, thus exhibiting stronger generalization and applicability. This solves the problem of insufficient generalization of traditional methods that rely on specific structures.

[0122] In this embodiment, the joint planner is trained in a differentiable simulation environment, where the backpropagation gradients of each physical quantity have high-precision analytical results, which are more accurate than the numerical integration results in traditional simulators. Furthermore, the joint angles are planned using a rapidly converging neural network, which can converge the planning error to a smaller range. Therefore, it has higher accuracy than traditional methods and solves the problem of insufficient accuracy in traditional methods.

[0123] In this embodiment, the neural network joint planner based on differentiable gradients is trained using the gradient descent method and can output a continuous and smooth sequence of joint angles, improving the smoothness of motion planning results and solving the problem of unsmooth planning results in traditional methods.

[0124] The training and evaluation processes of the joint planner neural network are described below. First, a simulation environment is constructed, consisting of a seven-axis robotic arm with a fixed base and random target points (in block form). Based on this, a multilayer perceptron neural network with two hidden layers is constructed as the joint planner. The input to this neural network is the real-time joint angle vector, and the output is the joint angle change vector (the joint angle increment vector in the above embodiment). Then, a target loss function is constructed, which includes the real-time distance loss between the robotic arm's end-effector position and the target point position, and the velocity smoothness loss. The real-time loss is calculated using the target loss function, and the loss gradient is calculated. Through a computational graph composed of various variables and differentiable physical quantities in the differentiable simulation environment, the loss gradient is directly backpropagated to update the joint planner. Finally, after training for a few steps, untrained target points (the test target points in the above embodiment) are generated, and the joint planner is invoked to perform planning inference and evaluate the planning effect.

[0125] Example, reference Figure 9 , Figure 9 This is a schematic flowchart illustrating the training and evaluation of a joint planner neural network provided in an embodiment of this application. The electronic device implementing the network training and evaluation method of this embodiment can be a server, combined with... Figure 9 The steps shown illustrate the training and evaluation process of the joint planner neural network provided in the embodiments of this application.

[0126] In step 901, a simulation environment for robotic arm motion planning is constructed.

[0127] Here, a simulation environment for the robotic arm motion planning task is built based on the open-source simulation engine (Nvidia Newton). First, the robot description file (Unified Robot Description Format, URDF) of the robotic arm is imported into the simulation engine. The simulation engine then instantiates the robotic arm in the simulation environment based on the entity joint information and mesh textures in the description file. The scale of this robotic arm is consistent with the real environment and includes a coordinate transformation matrix tree between the centroids of each joint and arm segment. For an example, refer to [reference needed]. Figure 10 , Figure 10 This is a schematic diagram of the simulation environment provided in the embodiments of this application. The simulation environment 1001 includes a robotic arm 1002 and a target point 1003.

[0128] Then, the interaction logic in the simulation environment was constructed. During the training phase, before each round of training, the position of a target point was randomly initialized within a feasible range. A mechanism was set to reset the environment after the number of training steps exceeded a certain threshold or the end effector of the robotic arm reached the vicinity of the target point. The joint planner neural network (the joint planning network in the above embodiment) receives the real-time joint angle vector (joint_q) of the robotic arm and outputs the change in joint angle (delta_q) at the next moment. This change is used as an increment to update the joint angle at the next moment. Then, the state of the robotic arm is updated through forward kinematics. Finally, a loss function is constructed based on the end effector position of the robotic arm and the change in joint angle, and the joint planner neural network is updated. Then, the next round of training begins.

[0129] During the inference phase, the joint planner neural network is no longer updated. Multiple target points that did not appear in the training phase are randomly generated in the simulation environment. Then, the simulation is planned within the same maximum number of steps as in the training phase to evaluate the effect of each round of robotic arm motion planning.

[0130] In step 902, a neural network-based robotic arm joint planner is constructed.

[0131] Here, a multilayer perceptron neural network with two hidden layers is used as the joint planner. For example, refer to [reference needed]. Figure 11 , Figure 11 This is a schematic diagram of the joint planner neural network provided in an embodiment of this application. The input of the joint planner neural network is a real-time joint angle vector with a dimension of 7. The dimensions of the first hidden layer 1101 and the second hidden layer 1102 are both 64, and the dimension of the final output joint angle change vector is 7.

[0132] The forward pass of the joint planner neural network can be described as follows: Let the parameter matrix from the input layer to the first hidden layer (the first hidden layer in the above embodiment) be... The bias vector is The parameter matrix from the first hidden layer to the second hidden layer is: The bias vector is The parameter matrix from the second hidden layer network (the second hidden layer in the above embodiment) to the output layer is: The bias vector is For example, the representation vector generated in the first hidden layer network (the first representation vector in the above embodiment) is determined by the following formula (1): (1) in, This represents the representation vector generated in the first hidden layer of the network. This represents the input to the joint planner neural network, which is the real-time joint angle vector of the robotic arm.

[0133] The representation vector generated in the first hidden layer network is nonlinearly mapped using an activation function (ReLU). For example, the mapped representation vector (the second representation vector in the above embodiment) is determined by the following formula (2): (2) in, Represents the mapped representation vector. This represents the activation function.

[0134] The mapped representation vector is then passed through a second hidden layer network and nonlinearly mapped by an activation function, in the same manner as described above, ultimately yielding the representation vector in the output layer. Finally, the representation vector in the output layer is processed by a normalization function (Softmax), and the final output joint angle change vector is uniformly mapped to the interval [-0.1, 0.1]. The joint angle change vector is used to control the magnitude of each change in the joints of the robotic arm. For example, the joint angle change vector (the joint angle increment vector in the above embodiment) is determined by the following formula (5): (5) in, Represents the vector of joint angle changes. Represents the normalization function. This represents the representation vector in the output layer (the fifth representation vector in the above embodiment).

[0135] In step 903, a loss function based on real-time distance and velocity smoothness is constructed.

[0136] Here, a loss function that balances the moving target and motion smoothness is constructed. This loss function consists of two parts: real-time distance loss and velocity smoothness loss. The real-time distance loss represents the Euclidean distance between the real-time position of the robotic arm's end effector and the position of the target point. The larger the Euclidean distance, the larger the real-time distance loss. For example, the real-time distance loss is determined by the following formula (6): (6) in, express Real-time distance loss at any given moment (distance loss value in the above embodiments). express The three-dimensional spatial position of the end effector of the robotic arm at any given time in the coordinate system of the robotic arm base (the first end effector position in the above embodiment). This indicates the location of the target point (the first target location in the above embodiment).

[0137] The velocity smoothness loss is expressed as the average absolute value of the changes in the joints within adjacent time intervals. For example, the velocity smoothness loss is determined by the following formula (7): (7) in, express The speed smoothness loss at any given moment (the speed smoothness loss value in the above embodiment). Indicates the first The joint angles are in The change in time (the first increment value in the above embodiment). Represents the smoothing coefficient. Indicates the speed smoothness threshold. The degree of freedom of the robotic arm (the first number in the above embodiments) is 7 in this embodiment.

[0138] For example, the total loss is determined by the following formula (8): (8) in, This represents the total loss (the target loss value in the above embodiments). and These represent the weighting coefficients for the two types of losses mentioned above.

[0139] In step 904, the differentiable gradient of the loss function is calculated and the joint planner neural network is trained.

[0140] Here, during the training phase, the update algorithm for the joint planner neural network is the backpropagation (BP) algorithm. For example, the network parameters of the joint planner neural network are updated using the following formula (9): (9) in, express The network parameters of the joint planner neural network at each time step. express The network parameters of the joint planner neural network at each time step. This represents the gradient of the loss function (the target loss gradient in the above embodiment).

[0141] In a differentiable simulation environment, the gradient of the loss function can be propagated among the variables (network parameters of the joint planner neural network) along a differentiable gradient computation graph. The gradient computation graph is automatically generated by the differentiable engine; an example is available in the reference. Figure 12 , Figure 12This is a schematic diagram of the gradient calculation graph provided in the embodiments of this application. The gradient calculation graph 1201 includes information related to joint angles such as real-time joint angle (joint_q), real-time joint angular velocity (joint_qd), initial joint angle (joint_q_start), and initial joint angular velocity (joint_qd_start), as well as different differential operators kernel0 and kernel1. kernel0 represents the differential operator for multiplication, while kernel1 represents the differential operator for exponentiation / root extraction.

[0142] The input to the differential operator kernel0 is information related to the joint angles. According to the internal algorithm of the simulation environment, the information of each link of the robotic arm is obtained by linearly combining and transforming the relevant information of the joint angles. kernel_0 is the differential operator kernel of the linear transformation algorithm, which facilitates the automatic differential solver built into the simulation engine to calculate the gradient between the relevant information of the joint angles.

[0143] The loss function is wrapped in computational kernels (kernel0 and kernel1) within a differentiable environment, and a gradient computation graph is constructed using a shared gradient pool. The gradient of the loss function can then flow directly within this graph. Since the gradient of the loss function originates from the solver of the differentiable engine, rather than from iterative integration through continuous interaction with the environment, the joint planner neural network can accurately obtain prior information about the robotic arm's trajectory over a future period without actually executing actions or updating the simulation environment. This allows the joint planner neural network to learn more precise control strategies.

[0144] In step 905, the motion trajectory is planned and evaluated using a joint planner neural network.

[0145] Here, after the joint planner neural network is trained, without updating the joint planner neural network, 10 target points that did not appear in the training phase are randomly generated in the simulation environment. The joint planner neural network is used to plan the trajectory from the end of the robotic arm to these target points. The training effect is evaluated by estimating the distance between the end of the trajectory and each target point (the distance between the end position of the robotic arm after movement and the position of the target point; the smaller the distance, the better the effect of the end of the robotic arm reaching the specified target point) and the average angular velocity change value (the average value of the change of angular velocity in adjacent time moments during the entire movement; the smaller the value, the smoother and more compliant the movement of the robotic arm).

[0146] Example, reference Figure 13 , Figure 13This is a schematic diagram of a continuous trajectory frame provided in an embodiment of this application. After evaluation, the average distance between the end of the trajectory and each target point is 0.64 cm, and the average angular velocity change is 0.064 rad / s, meeting the requirements for planning control accuracy and velocity smoothness. Furthermore, the average trajectory generation time is 0.59 s, significantly better than traditional numerical iterative planning methods (where the average distance is between 0.5 cm and 1.0 cm, the average angular velocity change is less than 0.1 rad / s, and the trajectory generation time is between 0.8 s and 10 s).

[0147] In the aforementioned scenario of training a joint planner neural network for a robotic arm, the high-precision gradients of a differentiable simulation environment were used to efficiently train the neural network, achieving a comprehensive improvement in four dimensions: real-time performance, generalization ability, accuracy, and smoothness. This not only meets the real-time requirements of high-speed operations with extremely low inference time but also possesses strong generalization capabilities due to its independence from specific robotic arm structures. Simultaneously, the planning accuracy of the joint planner was improved, ensuring that it outputs continuous and smooth motion trajectories, thereby enhancing the motion quality and operational efficiency of the robotic arm.

[0148] The following description continues to illustrate the exemplary structure of the network training device 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules stored in the network training device 455 in the memory 450 may include: a mapping processing module 4551, used to acquire the real-time joint angle vector of the robotic arm in the simulation environment, and perform the following processing until the training end condition is met to obtain the trained joint planning network: calling the joint planning network to perform mapping processing on the real-time joint angle vector to obtain the joint angle increment vector; a first determination module 4552, used to determine the first joint angle vector of the robotic arm at the next moment based on the joint angle increment vector and the real-time joint angle vector, and determine the first end position corresponding to the first joint angle vector; a second determination module 4553, used to determine the distance loss value based on the first end position and the first target position of the preset target point in the simulation environment, and determine the speed smoothness loss value based on the joint angle increment vector; a third determination module 4554, used to determine the target loss gradient based on the distance loss value and the speed smoothness loss value; and a parameter adjustment module 4555, used to adjust the network parameters of the joint planning network based on the target loss gradient, and determine the first joint angle vector as the real-time joint angle vector.

[0149] In some embodiments, the joint planning network includes a first hidden layer, a second hidden layer, and an output layer. The mapping processing module 4551 is further configured to call the first parameter matrix and the first bias vector in the first hidden layer to perform a linear operation on the real-time joint angle vector to obtain a first representation vector, and perform nonlinear mapping processing on the first representation vector to obtain a second representation vector; call the second parameter matrix and the second bias vector in the second hidden layer to perform a linear operation on the second representation vector to obtain a third representation vector, and perform nonlinear mapping processing on the third representation vector to obtain a fourth representation vector; call the third parameter matrix and the third bias vector in the output layer to perform a linear operation on the fourth representation vector to obtain a fifth representation vector, and perform normalization processing on the fifth representation vector to obtain a joint angle increment vector.

[0150] In some embodiments, the first determining module 4552 is further configured to obtain the kinematic model parameters of the robotic arm, and construct homogeneous transformation matrices corresponding to each joint based on the first joint angle vector and the kinematic model parameters; perform cascade multiplication on each homogeneous transformation matrix based on the joint connection order of the robotic arm to obtain the end-effector pose matrix; and perform coordinate analysis on the end-effector pose matrix to obtain the first end-effector position.

[0151] In some embodiments, the joint angle increment vector includes a first increment value corresponding to each joint angle. The second determining module 4553 is further configured to determine a first number of joints of the robotic arm and determine a historical increment vector at the previous moment, wherein the historical increment vector includes a historical increment value corresponding to each joint angle. Based on the first number, the difference between the first increment value and the historical increment value corresponding to each joint angle is summed and averaged to obtain a first processing result. When the first processing result is less than or equal to a first preset threshold, the speed smoothness loss value is determined as a preset value. When the first processing result is greater than the first preset threshold, the first processing result is smoothed based on a first preset coefficient to obtain a second processing result, and the second processing result is determined as the speed smoothness loss value.

[0152] In some embodiments, the third determining module 4554 is further configured to fuse the distance loss value and the velocity smoothness loss value to obtain the target loss value; determine the first target parameter with gradient and the second target parameter without gradient from the network parameters; construct a gradient calculation graph based on the first target parameter, the second target parameter and a preset differential operator, wherein the first node corresponding to the first target parameter and the second node corresponding to the second target parameter in the gradient calculation graph are interconnected; in the gradient calculation graph, calculate the partial derivatives of the target loss function at each first target parameter layer by layer based on the target loss value, and combine the partial derivatives into the target loss gradient.

[0153] In some embodiments, the third determining module 4554 is further configured to construct a first node based on a first target parameter and a second node based on a second target parameter; construct a directed acyclic graph based on the first node, the second node and a preset differential operator, and determine the directed acyclic graph as a gradient calculation graph.

[0154] In some embodiments, the parameter adjustment module 4555 is further configured to generate test target points in a simulation environment after obtaining the trained joint planning network, and determine the second joint angle vector of the robotic arm when the joint planning network training is completed; call the trained joint planning network, and perform joint planning processing for a preset number of steps based on the second joint angle vector and the test target points to obtain the robotic arm motion trajectory; evaluate the planning effect based on the robotic arm motion trajectory to obtain the evaluation result.

[0155] In some embodiments, the parameter adjustment module 4555 is further configured to determine the second end position corresponding to the end trajectory point of the robotic arm's motion trajectory, and determine the target distance between the second end position and the second target position of the test target point; based on the robotic arm's motion trajectory, determine the change in angular velocity of the robotic arm in multiple adjacent moments, and average each change in angular velocity to obtain a third processing result; when the target distance is less than a second preset threshold and the third processing result is less than a third preset threshold, the evaluation result is determined to be an evaluation pass.

[0156] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the network training method provided in this application.

[0157] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the network training method provided in this application. For example, ... Figure 3 The network training method is shown.

[0158] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0159] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0160] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0161] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0162] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A network training method, characterized in that, The method includes: Obtain the real-time joint angle vectors of the robotic arm in the simulation environment, and perform the following processing until the training termination condition is met to obtain the trained joint planning network: The joint planning network is invoked to map the real-time joint angle vector to obtain the joint angle increment vector. Based on the joint angle increment vector and the real-time joint angle vector, the first joint angle vector of the robotic arm at the next moment is determined, and the first end position corresponding to the first joint angle vector is determined. The distance loss value is determined based on the first end position and the first target position of the preset target point in the simulation environment, and the velocity smoothness loss value is determined based on the joint angle increment vector; The target loss gradient is determined based on the distance loss value and the velocity smoothness loss value; The network parameters of the joint planning network are adjusted based on the target loss gradient, and the first joint angle vector is determined as the real-time joint angle vector.

2. The method according to claim 1, characterized in that, The joint planning network includes a first hidden layer, a second hidden layer, and an output layer. The process of calling the joint planning network to map the real-time joint angle vector to obtain the joint angle increment vector includes: The first parameter matrix and the first bias vector in the first hidden layer are called to perform a linear operation on the real-time joint angle vector to obtain a first representation vector, and the first representation vector is subjected to a nonlinear mapping process to obtain a second representation vector. The second parameter matrix and the second bias vector in the second hidden layer are called to perform a linear operation on the second representation vector to obtain a third representation vector, and the third representation vector is then subjected to a nonlinear mapping process to obtain a fourth representation vector. The third parameter matrix and the third bias vector in the output layer are called to perform a linear operation on the fourth representation vector to obtain the fifth representation vector. The fifth representation vector is then normalized to obtain the joint angle increment vector.

3. The method according to claim 1, characterized in that, Determining the first end position corresponding to the first joint angle vector includes: Obtain the kinematic model parameters of the robotic arm, and construct the homogeneous transformation matrix corresponding to each joint based on the first joint angle vector and the kinematic model parameters; Based on the joint connection sequence of the robotic arm, a cascaded multiplication operation is performed on each homogeneous transformation matrix to obtain the end-effector pose matrix; The first end position is obtained by performing coordinate analysis on the end pose matrix.

4. The method according to claim 1, characterized in that, The joint angle increment vector includes a first increment value corresponding to each joint angle, and the step of determining the velocity smoothness loss value based on the joint angle increment vector includes: Determine the first number of joints of the robotic arm and determine the historical increment vector of the previous moment, wherein the historical increment vector includes the historical increment value corresponding to each joint angle; Based on the first quantity, the difference between the first incremental value and the historical incremental value corresponding to each joint angle is summed and averaged to obtain the first processing result; When the first processing result is less than or equal to the first preset threshold, the speed smoothness loss value is determined as the preset value; When the first processing result is greater than the first preset threshold, the first processing result is smoothed based on the first preset coefficient to obtain the second processing result, and the second processing result is determined as the speed smoothness loss value.

5. The method according to claim 1, characterized in that, Determining the target loss gradient based on the distance loss value and the velocity smoothness loss value includes: The distance loss value and the velocity smoothness loss value are fused together to obtain the target loss value; Determine a first objective parameter with a gradient and a second objective parameter without a gradient from the network parameters; A gradient computation graph is constructed based on the first target parameter, the second target parameter, and a preset differential operator. The first node corresponding to the first target parameter and the second node corresponding to the second target parameter in the gradient computation graph are interconnected. In the gradient calculation graph, the partial derivatives of the target loss function at each of the first target parameters are calculated layer by layer based on the target loss value, and the partial derivatives are combined to form the target loss gradient.

6. The method according to claim 5, characterized in that, The construction of the gradient calculation graph based on the first target parameter, the second target parameter, and the preset differential operator includes: A first node is constructed based on the first target parameter, and a second node is constructed based on the second target parameter; A directed acyclic graph is constructed based on the first node, the second node, and the preset differential operator, and the directed acyclic graph is determined as the gradient calculation graph.

7. The method according to any one of claims 1 to 6, characterized in that, After obtaining the trained joint planning network, the method further includes: Test target points are generated in the simulation environment, and the second joint angle vector of the robotic arm is determined when the joint planning network training is completed; The trained joint planning network is invoked, and joint planning processing is performed for a preset number of steps based on the second joint angle vector and the test target point to obtain the robotic arm motion trajectory. The planning effect was evaluated based on the robotic arm's motion trajectory, and the evaluation results were obtained.

8. The method according to claim 7, characterized in that, The evaluation of the planning effect based on the robotic arm's motion trajectory yields the following evaluation results: Determine the second end position corresponding to the last trajectory point of the robotic arm's motion trajectory, and determine the target distance between the second end position and the second target position of the test target point; Based on the motion trajectory of the robotic arm, the angular velocity change of the robotic arm in multiple adjacent moments is determined, and the angular velocity changes are averaged to obtain a third processing result. When the target distance is less than the second preset threshold and the third processing result is less than the third preset threshold, the evaluation result is determined to be an evaluation pass.

9. A network training device, characterized in that, The device includes: The mapping processing module is used to obtain the real-time joint angle vectors of the robotic arm in the simulation environment and perform the following processing until the training end condition is met to obtain the trained joint planning network: calling the joint planning network to perform mapping processing on the real-time joint angle vectors to obtain the joint angle increment vectors; The first determining module is used to determine the first joint angle vector of the robotic arm at the next moment based on the joint angle increment vector and the real-time joint angle vector, and to determine the first end position corresponding to the first joint angle vector. The second determining module is used to determine the distance loss value based on the first end position and the first target position of the preset target point in the simulation environment, and to determine the velocity smoothness loss value based on the joint angle increment vector; The third determining module is used to determine the target loss gradient based on the distance loss value and the velocity smoothness loss value; The parameter adjustment module is used to adjust the network parameters of the joint planning network based on the target loss gradient, and to determine the first joint angle vector as the real-time joint angle vector.

10. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the network training method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the network training method according to any one of claims 1 to 8.

Images