Control method and device for cooperative movement of multiple mechanical arms

Abstract

The invention provides a control method and device for cooperative movement of multiple mechanical arms. The control method and device can be applied to the field of mechanical control and the field of intelligent manufacturing. The control method for cooperative movement of the multiple mechanical arms comprises the steps that mechanical arm pose information of the multiple mechanical arms and workpiece pose information of a workpiece are processed through a first prediction sub-model in a prediction model, expected stress information of the workpiece is obtained, and the expected stress information represents stress needed by the workpiece in the expected workpiece pose; the workpiece pose information comprises an expected workpiece pose; second prediction sub-models, corresponding to the multiple mechanical arms, in the prediction model are used for processing the expected stress information and the workpiece pose information, and action operation instructions, matched with the expected stress information, of the multiple mechanical arms are obtained; and the corresponding mechanical arms are controlled through the multiple action operation instructions, so that the carrying task is executed on the workpiece, the force / position coupling effect in the movement process of the multiple mechanical arms is reduced, and the stability of cooperative movement of the multiple mechanical arms is facilitated.

Description

Technical Field

[0001] The present invention relates to the fields of mechanical control and intelligent manufacturing, and more particularly to a control method and device for cooperative movement of multiple robotic arms. Background Art

[0002] In industrial automation, intelligent manufacturing, and precision assembly, multiple robotic arms move cooperatively to transport workpieces to target positions, thereby completing workpiece assembly. Related technologies consider the actions of multiple robotic arms. During the movement process of multiple robotic arms, complex force / position coupling effects will occur, resulting in low stability of the cooperative movement of multiple robotic arms. Summary of the Invention

[0003] In view of the above problems, the present invention provides a control method and device for cooperative movement of multiple robotic arms.

[0004] According to a first aspect of the present invention, there is provided a control for cooperative movement of multiple robotic arms, including: processing the robotic arm pose information of multiple robotic arms and the workpiece pose information of a workpiece by using a first prediction sub-model in a prediction model to obtain expected force information of the workpiece, wherein the multiple robotic arms are used to cooperatively transport the workpiece, the expected force information represents the force required for the workpiece to be in an expected workpiece pose, and the workpiece pose information includes the expected workpiece pose; processing the expected force information and the workpiece pose information by using second prediction sub-models respectively corresponding to the multiple robotic arms in the prediction model to obtain action operation instructions respectively corresponding to the multiple robotic arms that match the expected force information, wherein the prediction model is trained based on an activation function; and controlling the respective corresponding robotic arms by using the multiple action operation instructions to perform a transportation task on the workpiece.

[0005] According to an embodiment of the present invention, processing the robotic arm pose information of multiple robotic arms and the workpiece pose information of a workpiece by using a first prediction sub-model in a prediction model to obtain expected force information of the workpiece includes: processing the multiple robotic arm pose information and the workpiece pose information by using a convolutional layer of the first prediction sub-model to obtain a directed graph, the directed graph includes a sub-graph representing the robotic arm pose information and a global node representing the workpiece pose information, and the global node is connected to the sub-graph; and inputting the directed graph into an attention layer of the first prediction sub-model to obtain the expected force information.

[0006] According to an embodiment of the present invention, inputting the directed graph into an attention layer of the first prediction sub-model to obtain expected force information includes: inputting the directed graph into an attention layer of the first prediction sub-model to obtain local node features respectively corresponding to the multiple robotic arms; performing information aggregation on the multiple local node features based on the global node to obtain a global feature; and predicting the expected force information based on the global feature.

[0007] According to an embodiment of the present invention, the sub-graph includes local nodes and sub-graph directed edges. The local nodes represent the joint pose information of the joints in the robotic arm. The robotic arm includes multiple joints, and the sub-graph directed edges represent the association relationships between the multiple joint pose information.

[0008] According to an embodiment of the present invention, the directed graph further includes cross-sub-graph directed edges. The multiple robotic arms include a first robotic arm and a second robotic arm, and the cross-sub-graph directed edges represent the association relationships between the first target joint pose information of the first robotic arm and the second target joint pose information of the second robotic arm.

[0009] According to an embodiment of the present invention, the robotic arm pose information includes the current robotic arm pose and the desired robotic arm pose, and the workpiece pose information further includes the current workpiece pose.

[0010] According to an embodiment of the present invention, the incentive function includes a team incentive sub-function, and the above method further includes: when the workpiece training pose of the workpiece satisfies the target workpiece pose condition, determining a preset team reward value as the team incentive sub-function value corresponding to the team incentive sub-function. The workpiece training pose is the workpiece pose after the robotic arm executes the training action operation instruction.

[0011] According to an embodiment of the present invention, the incentive function further includes an individual incentive sub-function, and the above method further includes: when the robotic arm training pose of the robotic arm satisfies the target robotic arm pose condition, determining a preset arrival incentive value as the first individual incentive sub-function value. The robotic arm training pose is the robotic arm pose after the robotic arm executes the training action operation instruction; when the robotic arm training pose represents a collision between multiple robotic arms, determining a preset collision incentive value as the second individual incentive sub-function value; and when the training running time satisfies the time threshold condition, determining a preset timeout incentive value as the third individual incentive sub-function value. The training running time represents the time when multiple robotic arms respectively execute their training action operation instructions. The individual incentive sub-function value corresponding to the individual incentive sub-function includes at least one of the following: the first individual incentive sub-function value, the second individual incentive sub-function value, and the third individual incentive sub-function value.

[0012] According to an embodiment of the present invention, the desired force information includes at least one of the following: the force and torque acting on the center of mass of the workpiece.

[0013] The second aspect of the present invention provides a control device for collaborative movement of multiple robotic arms, including: a first processing module, configured to process the robotic arm pose information of multiple robotic arms and the workpiece pose information of a workpiece by using a first prediction sub-model in a prediction model to obtain the desired force information of the workpiece, where the multiple robotic arms are used to collaboratively carry the workpiece, the desired force information represents the force required for the workpiece to be in the desired workpiece pose, and the workpiece pose information includes the desired workpiece pose; a second processing module, configured to process the desired force information and the workpiece pose information respectively by using second prediction sub-models corresponding to the multiple robotic arms to obtain respective action operation instructions of the multiple robotic arms that match the desired force information, where the prediction model is trained based on an activation function; and a control module, configured to use the multiple action operation instructions to control the respective corresponding robotic arms to perform a carrying task on the workpiece

[0014] The third aspect of the present invention provides an electronic device, including: one or more processors; a memory, configured to store one or more computer programs, where the above one or more processors execute the above one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present invention further provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0016] The fifth aspect of the present invention further provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0017] According to an embodiment of the present invention, by processing the robotic arm pose information of multiple robotic arms and the workpiece pose information of a workpiece by using a first prediction sub-model in a prediction model, the desired force information of the workpiece is obtained; then, by using a second prediction sub-model in the prediction model to process the desired force information and the workpiece pose information, the respective action operation instructions of the multiple robotic arms that match the desired force information of the workpiece are accurately predicted, so that the forces exerted by the multiple robotic arms on the workpiece can be balanced, and the interaction between the multiple robotic arms is reduced. Therefore, by using the multiple action operation instructions to control the respective corresponding robotic arms to perform a carrying task on the workpiece, the force / position coupling effect during the movement of the multiple robotic arms is reduced, which is beneficial to the stability of the collaborative movement of the multiple robotic arms. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Through the following description of the embodiments of the present invention with reference to the drawings, the above content and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0019] Figure 1Shows an application scenario diagram of the control method for cooperative movement of multiple robotic arms according to an embodiment of the present invention;

[0020] Figure 2 Shows a flowchart of the control method for cooperative movement of multiple robotic arms according to an embodiment of the present invention;

[0021] Figure 3 Shows a schematic diagram of a prediction model according to an embodiment of the present invention;

[0022] Figure 4 Shows the iterative return values of the prediction model training according to an embodiment of the present invention;

[0023] Figure 5 Shows the success rate of the prediction model training according to an embodiment of the present invention;

[0024] Figure 6 Shows a schematic diagram of the control method for cooperative movement of multiple robotic arms according to an embodiment of the present invention;

[0025] Figure 7 Shows a structural block diagram of the control device for cooperative movement of multiple robotic arms according to an embodiment of the present invention;

[0026] Figure 8 Shows a block diagram of an electronic device that can be used to implement the control method for cooperative movement of multiple robotic arms according to an embodiment of the present invention. Detailed implementation manners

[0027] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present invention. However, it is obvious that one or more embodiments can be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0028] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning that those skilled in the art usually understand this expression (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0031] In industrial automation, intelligent manufacturing, and precision assembly, multiple robotic arms move collaboratively to transport workpieces to target positions, thereby completing workpiece assembly. The related technology considers the movements of multiple robotic arms. During the movement process, complex force / position coupling effects will occur in multiple robotic arms, resulting in low stability of the collaborative movement of multiple robotic arms.

[0032] In view of this, embodiments of the present invention provide a control method for the collaborative movement of multiple robotic arms, including: processing the robotic arm pose information of multiple robotic arms and the workpiece pose information of a workpiece using a first prediction sub-model in a prediction model to obtain the expected force information of the workpiece, where the multiple robotic arms are used to collaboratively transport the workpiece, the expected force information represents the force required for the workpiece to be in the expected workpiece pose, and the workpiece pose information includes the expected workpiece pose; processing the expected force information and the workpiece pose information respectively using second prediction sub-models corresponding to the multiple robotic arms to obtain action operation instructions corresponding to each of the multiple robotic arms that match the expected force information, where the prediction model is trained based on an activation function; and using the multiple action operation instructions to control the corresponding robotic arms respectively to perform a transportation task on the workpiece.

[0033] Figure 1 The application scenario diagram of the control method for the collaborative movement of multiple robotic arms according to an embodiment of the present invention is shown.

[0034] As Figure 1 shown, the application scenario diagram of the control method for the collaborative movement of multiple robotic arms includes a first robotic arm 110, a second robotic arm 120, and a workpiece 130.

[0035] The first robotic arm 110 and the second robotic arm 120 move collaboratively to transport the workpiece 130 from the ground to the target position.

[0036] According to an embodiment of the present invention, the actions of each of the multiple robotic arms at the next moment can be predicted based on the current action sequences of the multiple robotic arms. However, there is usually a force / position coupling effect when the multiple robotic arms collaboratively transport the workpiece 130, and the predicted actions of the robotic arms may also cause deviations in the smoothness of transporting the workpiece 130.

[0037] In the present invention, the motion operation instructions of the first robotic arm 110 and the second robotic arm 120 are deduced based on the force information of the workpiece 130, thereby effectively reducing the force / position coupling effect and improving the smoothness of transporting the workpiece 130.

[0038] The pose of the workpiece 130 on the ground is the current pose, and the desired force information of the workpiece 130 at the target position can be that the force acting on the centroid of the workpiece is F and the torque is M. According to the desired force information and the workpiece pose information, the motion operation instructions of multiple robotic arms that match the desired force information are obtained.

[0039] The robotic arm control algorithms of the first robotic arm 110 and the second robotic arm 120 are the same and can be controlled by the same server. The control method for the coordinated motion of multiple robotic arms can also be executed by this server. For example, the robotic arm control algorithm can be fuzzy control, sliding mode control, adaptive control, PID control (Proportional-Integral-Derivative control), etc.

[0040] Inputting the motion operation instructions into the robotic arm control algorithm can achieve smooth control of the first robotic arm 110 and / or the second robotic arm 120.

[0041] There are no restrictions on the number of joints, motor types, end effectors, etc. of the first robotic arm 110 and the second robotic arm 120.

[0042] It should be understood that Figure 1 the number of robotic arms in

[0043] In some embodiments, Figure 2 a flowchart of the control method for the coordinated motion of multiple robotic arms according to an embodiment of the present invention is shown.

[0044] As Figure 2 shown, the control method for the coordinated motion of multiple robotic arms in this embodiment includes operations S210 to S230.

[0045] In operation S210, the desired force information of the workpiece is obtained by processing the robotic arm pose information of multiple robotic arms and the workpiece pose information using the first prediction submodel in the prediction model.

[0046] In operation S220, the motion operation instructions of multiple robotic arms that match the desired force information are obtained by processing the desired force information and the workpiece pose information respectively using the second prediction submodels corresponding to the multiple robotic arms in the prediction model.

[0047] In operation S230, the respective robotic arms are controlled using multiple motion operation instructions to perform the transportation task on the workpiece.

[0048] According to an embodiment of the present invention, multiple robotic arms are used to cooperate in handling workpieces. The multiple robotic arms can handle the workpiece to a target position of the target workpiece to complete the assembly or fitting between the workpiece and the target workpiece.

[0049] For example, the workpiece can be an engine of a vehicle, and the target workpiece can be a vehicle frame, and there is a target position for installing the engine on the vehicle frame. The end effectors (such as grippers) of the multiple robotic arms are used to cooperate in handling the engine so as to smoothly handle the engine to the target position of the vehicle frame, and then replace the end effector of the robotic arm with a welding gun, and use the welding guns of the multiple robotic arms respectively to install the engine on the vehicle frame.

[0050] According to an embodiment of the present invention, the robotic arm pose information can include a reference robotic arm pose, joint poses, link poses, end effector poses, desired robotic arm poses, etc.

[0051] The reference robotic arm pose can also be called the initial pose in the initial coordinate system, and can be used as a reference position and attitude of the robotic arm in the working space.

[0052] The joint poses can be the positions and attitudes of the joints in the robotic arm.

[0053] The link poses can be the relative positions and attitudes of the structural parts between the joints in the robotic arm with respect to the initial coordinate system.

[0054] The end effector can be the end of the robotic arm. For example, the end effector can be a fixture, a welding gun, etc. The end effector pose can be the position and attitude of the end effector with respect to the initial coordinate system.

[0055] The desired robotic arm pose can be the position and attitude that the robotic arm needs to reach in order to place the workpiece in the desired workpiece pose.

[0056] According to an embodiment of the present invention, the desired force information characterizes the force required for the workpiece to be in the desired workpiece pose. The workpiece pose information includes the desired workpiece pose.

[0057] For example, the desired force information can be a supporting force of 5 Newtons upward to lift the workpiece by 5 centimeters.

[0058] According to an embodiment of the present invention, the prediction model is trained based on an activation function.

[0059] According to an embodiment of the present invention, the prediction model can include a first prediction sub-model and multiple second prediction sub-models. The first prediction sub-model and the second prediction sub-models can be the same or different machine learning models.

[0060] For example, the first prediction sub-model can be a deep learning model, and the second prediction sub-model can be a random forest model. The deep learning model can learn the relationships between multiple robotic arms from the robotic arm pose information of the multiple robotic arms, and use this relationship and the workpiece pose information to obtain the expected force information of the workpiece. The random forest model can determine the action operation instructions of the robotic arm from multiple candidate action operation instructions according to the expected force information and the workpiece pose information.

[0061] For example, the first prediction sub-model and the second prediction sub-model can be neural network models. However, the second prediction sub-models of the multiple robotic arms can form a distributed policy network. Through the multiple second prediction sub-models, they can collaborate and learn by sharing robotic arm pose information, etc. Not only the movement of this robotic arm is considered, but also the movements of other robotic arms are considered to achieve the coordinated movement of multiple robotic arms, and collisions between multiple robotic arms can be reduced.

[0062] According to an embodiment of the present invention, by using the first prediction sub-model in the prediction model to process the robotic arm pose information of multiple robotic arms and the workpiece pose information of the workpiece, the expected force information of the workpiece is obtained; then using the second prediction sub-model in the prediction model to process the expected force information and the workpiece pose information, the action operation instructions of each of the multiple robotic arms that match the expected force information of the workpiece are accurately predicted, which can make the forces exerted by the multiple robotic arms on the workpiece reach balance, and the interaction between the multiple robotic arms is reduced. Therefore, using the multiple action operation instructions to control their respective corresponding robotic arms to perform a handling task on the workpiece can reduce the force / position coupling effect during the movement of multiple robotic arms, which is beneficial to the stability of the coordinated movement of multiple robotic arms.

[0063] According to an embodiment of the present invention, using the first prediction sub-model in the prediction model to process the robotic arm pose information of multiple robotic arms and the workpiece pose information of the workpiece to obtain the expected force information of the workpiece includes: using the convolutional layer of the first prediction sub-model to process the multiple robotic arm pose information and the workpiece pose information to obtain a directed graph, the directed graph includes a sub-graph representing the robotic arm pose information and a global node representing the workpiece pose information, and the global node is connected to the sub-graph; and inputting the directed graph into the attention layer of the first prediction sub-model to obtain the expected force information.

[0064] According to an embodiment of the present invention, the first prediction sub-model can be a Graph Attention Network (GAT). For example, the first prediction sub-model can include a convolutional layer, an attention layer, a normalization layer, a fully connected layer, etc.

[0065] According to an embodiment of the present invention, the sub-graph can include multiple nodes, and the nodes represent the joint pose information of the robotic arm.

[0066] The first prediction sub-model can enhance the expressive power of the model by introducing an attention mechanism to weight the neighbors of nodes in the directed graph.

[0067] According to an embodiment of the present invention, the role of the global node can be to globally model the relationship with the nodes in the directed graph, optimize the information flow within the global scope of the directed graph, and avoid the deviation of local information of the sub-graph.

[0068] According to an embodiment of the present invention, inputting the directed graph into the attention layer of the first prediction sub-model to obtain the expected force information includes: inputting the directed graph into the attention layer of the first prediction sub-model to obtain the local node features corresponding to each manipulator; aggregating the information of multiple local node features based on the global node to obtain the global feature; and predicting the expected force information based on the global feature.

[0069] The first prediction sub-model can include three layers of attention layers. Feature extraction is performed on the directed graph through three layers of attention layers to obtain the local node features corresponding to each manipulator. Each node aggregates information through the input pose information of multiple manipulators and the workpiece pose information to obtain the node aggregation information. The features of the global node are updated through the node aggregation information.

[0070] The node update equation of the graph attention network is:

[0071] (1)

[0072] represents the feature vector of node i in the (l + 1)-th layer of the attention layer, K represents the number of linear projection heads, and the value of K can be 3. Concat represents concatenating the outputs of K independent attention heads. Each attention head generates a feature vector, and the final feature representation is obtained after concatenation. ELU represents the Exponential Linear Unit, an activation function; represents the learnable weight matrix in the k-th attention head, which is used to perform a linear transformation on the node features of the l-th layer of the attention layer; represents the feature vector of node j in the l-th layer of the attention layer; represents the normalized attention weight of node i to neighbor node j in the l-th layer of the attention layer and the k-th attention head, and is specifically defined as:

[0073] (2)

[0074] (3)

[0075] exp represents the exponential function, which is used to convert the attention score into a positive number; denotes the unnormalized attention weights, which are calculated by Equation (3); denotes the sum of the unnormalized attention weights for all neighbors j′ of node i (including node i itself) to achieve normalization; denotes the Rectified Linear Unit, an activation function; denotes the learnable weight vector used to calculate the attention weights in the l-th attention layer and the k-th attention head; denotes the feature vector of node i in the l-th attention layer. Through Equations (1), (2), and (3), GAT learns an average value of the neighborhood features of each node and performs sparse weighting based on the importance of each neighborhood.

[0076] The second prediction sub-model can be a Long Short-Term Memory (LSTM) network. For example, the distributed policy network consists of n Long Short-Term Memory networks, where n represents the number of robotic arms. The Long Short-Term Memory network can extract a fixed-size sequence of robotic arm poses from the current and historical poses of the robotic arm.

[0077] According to an embodiment of the present invention, the expected force information includes at least one of the following: the force and torque acting on the centroid of the workpiece.

[0078] Each second prediction sub-model separately extracts information from the expected force information and the workpiece pose information, and outputs an action operation instruction that matches the expected force information.

[0079] For example, the robotic arm pose information of multiple robotic arms and the workpiece pose information of the workpiece can be sequences of robotic poses and workpiece poses arranged in time series respectively. The sequence of robotic poses and the sequence of workpiece poses can form a 256-dimensional vector. After extracting the state, through 3 fully connected layers with the tanh activation function (hyperbolic tangent function) in the first prediction sub-model, a 12-dimensional vector is output. The 12-dimensional vector is the input of the second prediction sub-model, that is, the expected force information and the workpiece pose information.

[0080] The second prediction sub-model can first predict the action values for each robotic arm, and then determine the action operation instructions based on the action values of each robotic arm. The structures of each second prediction sub-model are the same, but the network parameters are different.

[0081] According to an embodiment of the present invention, to perform an assembly task on the workpiece, each of the n robotic arms needs to have a corresponding sequence of action operation instructions. The sequence of action operation instructions is the action operation instructions corresponding to the action values arranged based on the motion sequence. The expected force information can provide a pre-correction for the action values, so that the predicted action operation instructions can make the coordinated movement of multiple robotic arms more stable.

[0082] According to an embodiment of the present invention, a sub-graph includes local nodes and sub-graph directed edges. The local nodes represent the joint pose information of the joints in the robotic arm. The robotic arm includes multiple joints, and the sub-graph directed edges represent the association relationship between the multiple joint pose information.

[0083] According to an embodiment of the present invention, the association relationship between the multiple joint pose information can be a spatial position relationship, a dependency relationship of handling tasks, etc. For example, the movement directions between the pose of joint A and the pose of joint B in the robotic arm are different.

[0084] According to an embodiment of the present invention, the directed graph further includes cross-sub-graph directed edges. The multiple robotic arms include a first robotic arm and a second robotic arm, and the cross-sub-graph directed edges represent the association relationship between the first target joint pose information of the first robotic arm and the second target joint pose information of the second robotic arm.

[0085] According to an embodiment of the present invention, the association relationship between the first target joint pose information of the first robotic arm and the second target joint pose information of the second robotic arm can be a spatial position relationship, a dependency relationship of handling tasks. The cross-sub-graph directed edges can also represent the communication relationship between the multiple robotic arms, the relative movement between the robotic arms, collision constraints, etc.

[0086] According to an embodiment of the present invention, the robotic arm pose information includes the current robotic arm pose and the desired robotic arm pose, and the workpiece pose information further includes the current workpiece pose.

[0087] Figure 3 A schematic diagram of a prediction model according to an embodiment of the present invention is shown.

[0088] As Figure 3 shown, the prediction model includes a first prediction sub-model and multiple second prediction sub-models. The first prediction sub-model can be a graph attention network. By inputting the workpiece pose information and the robotic arm pose information into the first prediction sub-model, the desired force information is obtained.

[0089] The workpiece pose information can include the current workpiece pose and the desired workpiece pose. The robotic arm pose information can include the current robotic arm pose and the desired robotic arm pose.

[0090] The second prediction sub-model can be an LSTM. The n robotic arms correspond to n LSTMs. For example, the n robotic arms respectively correspond to LSTM 1 ……LSTM n .

[0091] By inputting the workpiece pose information and the desired force information into the LSTM 1 , the first action operation instruction of the first robotic arm is obtained. By inputting the workpiece pose information and the desired force information into the LSTM n, the nth action operation instruction of the nth robotic arm is obtained.

[0092] According to an embodiment of the present invention, the incentive function includes a team incentive sub-function, and the above method further includes: when the workpiece training pose of the workpiece satisfies the target workpiece pose condition, determining a preset team reward value as the team incentive sub-function value corresponding to the team incentive sub-function, where the workpiece training pose is the workpiece pose after the robotic arm executes the training action operation instruction.

[0093] According to an embodiment of the present invention, in order to pursue cooperative behavior, rewards are given to all robotic arms only when the workpiece successfully reaches the desired workpiece pose. Based on the team incentive sub-function as follows:

[0094] (4)

[0095] p am ,p bm respectively represent the current workpiece pose and the desired workpiece pose at both ends of the workpiece, where ,δ w is the team incentive sub-function threshold.

[0096] When m = 1, p a1 and p b1 respectively correspond to the current workpiece pose and the desired workpiece pose at one end of the workpiece.

[0097] When m = 2, p a2 and p b2 respectively correspond to the current workpiece pose and the desired workpiece pose at the other end of the workpiece.

[0098] According to an embodiment of the present invention, the team incentive function focuses on multiple robotic arms transporting the workpiece to the desired workpiece pose, which can promote the cooperation between multiple second prediction sub-models and reduce conflicts. According to an embodiment of the present invention, the incentive function further includes an individual incentive sub-function, and the above method further includes: when the robotic arm training pose of the robotic arm satisfies the target robotic arm pose condition, determining a preset arrival incentive value as the first individual incentive sub-function value, where the robotic arm training pose is the robotic arm pose after the robotic arm executes the training action operation instruction; when the robotic arm training pose represents a collision between multiple robotic arms, determining a preset collision incentive value as the second individual incentive sub-function value; and when the training running time satisfies the time threshold condition, determining a preset timeout incentive value as the third individual incentive sub-function value, where the training running time represents the time when multiple robotic arms respectively execute their training action operation instructions, and the individual incentive sub-function value corresponding to the individual incentive sub-function includes at least one of the following: the first individual incentive sub-function value, the second individual incentive sub-function value, and the third individual incentive sub-function value.

[0099] According to an embodiment of the present invention, the target robotic arm pose condition may include that the distance between the position of the robotic arm training pose and the expected position of the expected robotic arm pose is less than a distance threshold, the pose of the robotic arm training pose reaches the expected robotic arm pose, and when the running direction of the robotic arm conforms to the direction of the expected robotic arm pose.

[0100] According to an embodiment of the present invention, the time threshold condition may be that the training running time exceeds the time threshold.

[0101] Individual incentive sub - function The formula is as follows:

[0102] (5)

[0103] represents when the distance between the position of the robotic arm training pose and the expected position of the expected robotic arm pose is less than the distance threshold δ L , the obtained position - reaching incentive value; represents the preset collision incentive value obtained when a collision occurs between multiple robotic arms; represents when a collision occurs between multiple robotic arms. represents when the training running time exceeds the time threshold T, the obtained preset timeout incentive value; represents the pose - reaching incentive value obtained when the pose of the robotic arm training pose reaches the expected robotic arm pose; approaching the goal represents when the pose of the robotic arm training pose reaches the expected robotic arm pose. represents the direction incentive value obtained when the running direction of the robotic arm conforms to the direction of the expected robotic arm pose. represents the direction of the expected robotic arm pose.

[0104] According to an embodiment of the present invention, the preset reaching incentive value may include at least one of the position - reaching incentive value, the pose - reaching incentive value, and the direction incentive value.

[0105] According to an embodiment of the present invention, the individual incentive sub - function may be an optimization objective function for multiple second prediction sub - models, and the team incentive sub - function may be for the collaboration between the first prediction sub - model and multiple second prediction sub - models in the prediction model to achieve global optimization.

[0106] According to an embodiment of the present invention, by measuring the individual performance of the second prediction sub - model through the individual incentive sub - function, its own behavioral performance can be optimized. Combining with the team incentive sub - function, both the individual learning of the second prediction sub - model and the collaboration between the first prediction sub - model and multiple second prediction sub - models can be achieved, making the prediction of the action operation instructions of multiple robotic arms more accurate.

[0107] Figure 4 Shows the iterative return values of the prediction model training according to an embodiment of the present invention.

[0108] As Figure 4 shown, "Our method" may refer to the result obtained by training the prediction model using the activation function of the present invention, and "Baseline method" refers to the result obtained by training the prediction model using the baseline method of directly controlling each robotic arm. As the number of training steps increases, the iterative return values of the training of the present invention tend to converge to a stable value, and the result is better than the baseline method.

[0109] Figure 5 Shows the success rate of the prediction model training according to an embodiment of the present invention.

[0110] As Figure 5 shown, "Our method" refers to the result obtained by training the prediction model using the activation function of the present invention, and "Baseline method" refers to the result obtained by training the prediction model using the baseline method of directly controlling each robotic arm. As the number of training steps increases, the success rate of the training of the present invention increases and converges near 0.95, and the result is better than the baseline method.

[0111] In some embodiments, Figure 6 Shows a schematic diagram of the control method for the cooperative movement of multiple robotic arms according to an embodiment of the present invention.

[0112] As Figure 6 shown, in the offline training stage, the prediction model 620 is trained based on the activation function using the respective simulated robotic arm pose information of multiple simulated robotic arms in the simulation environment 610 and the simulated workpiece pose information of the simulated workpiece. The activation function can optimize the parameters of the prediction model 620.

[0113] In the online control stage, the prediction model 620 is deployed to the server 630. The robotic arm pose information can be obtained from the sensors installed on the robotic arm, or the workpiece pose information can be obtained by using an image acquisition device or the sensors installed on the workpiece. Among them, the desired workpiece pose and the desired robotic arm pose can be obtained from the client 650.

[0114] The server 630 is used to process the robotic arm pose information of multiple robotic arms and the workpiece pose information of the workpiece using the first prediction sub-model in the prediction model 620 to obtain the desired force information of the workpiece; and use the second prediction sub-models corresponding to the multiple robotic arms in the prediction model 620 to process the desired force information and the workpiece pose information respectively to obtain the action operation instructions of the multiple robotic arms that match the desired force information.

[0115] The server 630 is further configured to send the action operation instruction to the position controller 640 of the robotic arm.

[0116] The position controller 640 is configured to control the movement of the end effector of the robotic arm and send the robotic arm pose information and the task feedback result of the robotic arm to the client 650. The task feedback result may include workpiece pose information, workpiece pose error value, etc.

[0117] The client 650 (such as a control interface) is configured to send a target task (such as a specified grasping position) or a cancellation instruction.

[0118] In some embodiments, Figure 7 The structural block diagram of the control device for the cooperative movement of multiple robotic arms according to an embodiment of the present invention is shown.

[0119] Such as Figure 7 As shown, the control device 700 for the cooperative movement of multiple robotic arms in this embodiment includes a first processing module 710, a second processing module 720, and a control module 730.

[0120] The first processing module 710 is configured to process the robotic arm pose information of multiple robotic arms and the workpiece pose information of the workpiece by using the first prediction sub-model in the prediction model to obtain the expected force information of the workpiece. Among them, the multiple robotic arms are used to cooperatively carry the workpiece, and the expected force information represents the force required for the workpiece to be in the expected workpiece pose. The workpiece pose information includes the expected workpiece pose. In one embodiment, the first processing module 710 may be configured to execute the operation S210 described above, which will not be elaborated here.

[0121] The second processing module 720 is configured to process the expected force information and the workpiece pose information respectively by using the second prediction sub-models corresponding to the multiple robotic arms in the prediction model to obtain the action operation instructions corresponding to the multiple robotic arms that match the expected force information. Among them, the prediction model is trained based on an activation function, and the activation function includes an individual activation sub-function and a team activation sub-function. In one embodiment, the second processing module 720 may be configured to execute the operation S220 described above, which will not be elaborated here.

[0122] The control module 730 is configured to control the corresponding robotic arms respectively by using the multiple action operation instructions to perform a carrying task on the workpiece. In one embodiment, the control module 730 may be configured to execute the operation S230 described above, which will not be elaborated here.

[0123] According to an embodiment of the present invention, the first processing module 710 includes a first processing sub-module and an input sub-module. The first processing sub-module is configured to process multiple robotic arm pose information and workpiece pose information by using the convolutional layer of the first prediction sub-model to obtain a directed graph, where the directed graph includes a sub-graph representing the robotic arm pose information and a global node representing the workpiece pose information, and the global node is connected to the sub-graph. The input sub-module is configured to input the directed graph into the attention layer of the first prediction sub-model to obtain the expected force information.

[0124] According to an embodiment of the present invention, the input sub-module includes an input unit, an information aggregation unit, and a prediction unit. The input unit is configured to input the directed graph into the attention layer of the first prediction sub-model to obtain local node features corresponding to multiple robotic arms respectively; the information aggregation unit is configured to perform information aggregation on the multiple local node features based on the global node to obtain a global feature; the prediction unit is configured to predict the expected force information based on the global feature.

[0125] According to an embodiment of the present invention, the sub-graph includes local nodes and sub-graph directed edges, the local nodes represent the joint pose information of the joints in the robotic arm, the robotic arm includes multiple joints, and the sub-graph directed edges represent the association relationship between the multiple joint pose information.

[0126] According to an embodiment of the present invention, the directed graph further includes cross-sub-graph directed edges, the multiple robotic arms include a first robotic arm and a second robotic arm, and the cross-sub-graph directed edges represent the association relationship between the first target joint pose information of the first robotic arm and the second target joint pose information of the second robotic arm.

[0127] According to an embodiment of the present invention, the robotic arm pose information includes the current robotic arm pose and the expected robotic arm pose, and the workpiece pose information further includes the current workpiece pose.

[0128] According to an embodiment of the present invention, the activation function includes a team incentive sub-function, and the above device further includes a first determination module. The first determination module is configured to, when the workpiece training pose of the workpiece meets the target workpiece pose condition, determine a preset team reward value as the team incentive sub-function value corresponding to the team incentive sub-function, and the workpiece training pose is the workpiece pose after the robotic arm executes the training action operation instruction.

[0129] According to an embodiment of the present invention, the excitation function further includes an individual excitation sub-function, and the above device further includes a second determination module, a third determination module, and a fourth determination module. The second determination module is configured to determine a preset arrival excitation value as the first individual excitation sub-function value when the robotic arm training pose of the robotic arm satisfies the target robotic arm pose condition, where the robotic arm training pose is the robotic arm pose after the robotic arm executes the training action operation instruction; the third determination module is configured to determine a preset collision excitation value as the second individual excitation sub-function value when the robotic arm training pose represents a collision between multiple robotic arms; and the fourth determination module is configured to determine a preset timeout excitation value as the third individual excitation sub-function value when the training running time satisfies the time threshold, where the training running time represents the time when multiple robotic arms respectively execute their own training action operation instructions, and the individual excitation sub-function value corresponding to the individual excitation sub-function includes at least one of the following: the first individual excitation sub-function value, the second individual excitation sub-function value, and the third individual excitation sub-function value.

[0130] According to an embodiment of the present invention, the expected force information includes at least one of the following: the force and torque acting on the centroid of the workpiece.

[0131] According to an embodiment of the present invention, any multiple of the first processing module 710, the second processing module 720, and the control module 730 can be combined and implemented in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first processing module 710, the second processing module 720, and the control module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable means such as hardware or firmware by integrating or packaging circuits, or can be implemented in any one of the three implementation manners of software, hardware, and firmware or in an appropriate combination of any several of them. Alternatively, at least one of the first processing module 710, the second processing module 720, and the control module 730 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0132] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0133] Figure 8 The block diagram of the electronic device that can be used to implement the control method for the cooperative movement of multiple robotic arms according to the embodiments of the present invention is shown.

[0134] The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0135] As Figure 8 shown, the device includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory ROM 802 or a computer program loaded from a storage unit 808 into a random access memory RAM 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An I / O interface 805 is also connected to the bus 804.

[0136] Multiple components in the electronic device are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0137] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the virtual avatar driving method. For example, in some embodiments, the virtual avatar driving method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the virtual avatar driving method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the control method for the coordinated movement of multiple robotic arms in any other suitable manner (e.g., by means of firmware).

[0138] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code for implementing the methods of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0140] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0143] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. Among them, the server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0144] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present invention can be combined or / and combined in various ways, even if such combinations or combinations are not explicitly recited in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recited in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0145] Although the present invention has been shown and described with reference to specific exemplary embodiments of the present invention, those skilled in the art should understand that various changes in form and detail can be made to the present invention without departing from the spirit and scope of the present invention as defined by the appended claims and their equivalents. Therefore, the scope of the present invention should not be limited to the above embodiments, but should be determined not only by the appended claims but also by the equivalents of the appended claims.

Claims

1. A control method for coordinated motion of multiple robotic arms, characterized in that: The method comprises: Processing the manipulator arm posture information of the plurality of manipulator arms and the workpiece posture information of the workpiece using the first prediction sub-model in the prediction model to obtain the expected force information of the workpiece, wherein the plurality of manipulator arms are used to collaboratively carry the workpiece, the expected force information represents the force required for the workpiece to be in the expected workpiece posture, and the workpiece posture information includes the expected workpiece posture; Using the second prediction sub-models in the prediction model corresponding to each of the plurality of robotic arms to respectively process the expected force information and the workpiece posture information to obtain the respective action operation instructions of the plurality of robotic arms that match the expected force information, wherein the prediction model is obtained based on the training of the activation function; and The plurality of action operation instructions are used to control the corresponding robot arms respectively to perform the transport task on the workpiece.

2. The method according to claim 1, characterized in that The method of using the first prediction sub-model in the prediction model to process the robot arm posture information of the plurality of robot arms and the workpiece posture information of the workpiece to obtain the expected force information of the workpiece includes: Processing the plurality of the robot arm posture information and the workpiece posture information using the convolutional layer of the first prediction sub-model to obtain a directed graph, wherein the directed graph includes a sub-graph representing the robot arm posture information and a global node representing the workpiece posture information, wherein the global node is connected to the sub-graph; and The directed graph is input into the attention layer of the first prediction sub-model to obtain the expected force information.

3. The method according to claim 2, characterized in that The step of inputting the directed graph into the attention layer of the first prediction sub-model to obtain the expected force information comprises: Inputting the directed graph into the attention layer of the first prediction sub-model to obtain local node features corresponding to each of the plurality of robotic arms; Aggregating information of the features of the plurality of local nodes based on the global node to obtain a global feature; and The expected force information is obtained based on the global feature prediction.

4. The method according to claim 2, characterized in that: The subgraph includes local nodes and subgraph directed edges, the local nodes represent joint pose information of the joints in the robotic arm, the robotic arm includes multiple joints, and the subgraph directed edges represent the association relationship between the multiple joint pose information.

5. The method according to claim 2, characterized in that: The directed graph also includes a cross-subgraph directed edge, the multiple robotic arms include a first robotic arm and a second robotic arm, and the cross-subgraph directed edge represents the association relationship between the first target joint pose information of the first robotic arm and the second target joint pose information of the second robotic arm.

6. The method according to claim 1, characterized in that The robot arm posture information includes the current robot arm posture and the expected robot arm posture, and the workpiece posture information also includes the current workpiece posture.

7. The method according to claim 1, characterized in that The incentive function includes a team incentive sub-function, and the method further includes: When the workpiece training posture of the workpiece meets the target workpiece posture condition, the preset team reward value is determined as the team incentive sub-function value corresponding to the team incentive sub-function, and the workpiece training posture is the workpiece posture after the robot arm executes the training action operation instruction.

8. The method according to claim 7, characterized in that The excitation function also includes an individual excitation sub-function, and the method further includes: In the case where the manipulator arm training posture of the manipulator arm satisfies the target manipulator arm posture condition, determining the preset arrival excitation value as the first body excitation sub-function value, the manipulator arm training posture being the manipulator arm posture after the manipulator arm executes the training action operation instruction; In the case where the training posture of the robot arm represents a collision between a plurality of the robot arms, determining a preset collision excitation value as a second individual excitation sub-function value; and When the training running time meets the time threshold condition, the preset timeout incentive value is determined as the third individual incentive sub-function value. The training running time represents the time for the multiple robotic arms to respectively execute their respective training action operation instructions. The individual incentive sub-function value corresponding to the individual incentive sub-function includes at least one of the following: the first individual incentive sub-function value, the second individual incentive sub-function value and the third individual incentive sub-function value.

9. The method according to claim 1, characterized in that: The expected force information includes at least one of the following: force and moment acting on the center of mass of the workpiece.

10. A control device for coordinated motion of multiple robotic arms, characterized in that: The device comprises: A first processing module is used to process the robot arm posture information of multiple robot arms and the workpiece posture information of the workpiece by using the first prediction sub-model in the prediction model to obtain the expected force information of the workpiece, wherein the multiple robot arms are used to collaboratively carry the workpiece, the expected force information represents the force required for the workpiece to be in the expected workpiece posture, and the workpiece posture information includes the expected workpiece posture; A second processing module is used to process the expected force information and the workpiece posture information respectively by using the second prediction sub-models corresponding to the plurality of the robotic arms in the prediction model to obtain the action operation instructions of the plurality of robotic arms that match the expected force information, wherein the prediction model is obtained by training based on an incentive function, and the incentive function includes an individual incentive sub-function and a team incentive sub-function; and The control module is used to use the plurality of action operation instructions to respectively control the corresponding robotic arms to perform the transport task on the workpiece.

Images