Multi-manipulator collaborative motion planning method

Abstract

The present invention provides a multi-manipulator collaborative motion planning method, which can be applied to the field of multi-manipulator collaborative control. The method comprises: inputting the action values ​​and observation values ​​of each of the multiple manipulators into an encoder of an action prediction model to obtain the intermediate action values ​​and reference intermediate action values ​​of each of the multiple manipulators, wherein the observation value represents the state information after the manipulator performs an action operation, and the reference intermediate action value is the action value of the manipulator when the multiple manipulators are in a preset state; inputting the intermediate action values, reference intermediate action values, observation values, and partial observable values ​​of each of the multiple manipulators into decoders corresponding to the multiple manipulators respectively to obtain the target action values ​​of each of the multiple manipulators, and the partial observable values ​​are the state information of the manipulators in a preset state; controlling the multiple manipulators to move according to their respective target action values ​​to perform an assembly task on a cylinder head of an engine, wherein an excitation function comprises sub-excitation functions corresponding to each stage in the assembly task.

Description

Technical Field

[0001] The present invention relates to the field of robot arm collaborative control, and more specifically, to a multi-robot arm collaborative motion planning method. Background Art

[0002] In the industrial production process, industrial robots replace manual labor in production activities, especially the assembly line production mode, which has gradually become the main development trend of industrial production and manufacturing. Compared with single-arm robots, multi-arm robots can complete some more difficult collaborative tasks and can operate larger target loads.

[0003] In the process of realizing the concept of the present invention, the inventors found that there are at least the following problems in the related technology: there is coupling in the grasping motion during the collaboration of multiple robotic arms, resulting in poor stability of the collaborative motion and low scene adaptability. Summary of the invention

[0004] In view of this, the present invention provides a multi-robot arm collaborative motion planning method.

[0005] One aspect of the present invention provides a method for collaborative motion planning of multiple robotic arms, including: inputting the action values ​​and observation values ​​of each of the multiple robotic arms into an encoder of a motion prediction model to obtain intermediate action values ​​and reference intermediate action values ​​of each of the multiple robotic arms, wherein the action value represents the action operation performed by the robotic arm, the observation value represents the state information of the robotic arm after performing the action operation, and the reference intermediate action value is the action value of the robotic arm when the multiple robotic arms are in a preset state; inputting the intermediate action values, reference intermediate action values, observation values, and partial observable values ​​corresponding to the reference intermediate action values ​​of each of the multiple robotic arms into decoders corresponding to the multiple robotic arms respectively to obtain target action values ​​of the multiple robotic arms, the motion prediction model includes a decoder, and the partial observable values ​​are the state information of the robotic arms in a preset state; controlling the multiple robotic arms to move according to their respective target action values ​​to perform an assembly task on the cylinder head of the engine, wherein the assembly task includes multiple stages, the motion prediction model is obtained based on training of an excitation function, and the excitation function includes sub-excitation functions corresponding to each stage in the assembly task.

[0006] Another aspect of the present invention provides a multi-robotic arm collaborative motion planning device, including: a first input module, used to input the action values ​​and observation values ​​of each of the multiple robotic arms into the encoder of the action prediction model, to obtain the intermediate action values ​​and reference intermediate action values ​​of each of the multiple robotic arms, the action value represents the action operation performed by the robotic arm, the observation value represents the state information of the robotic arm after performing the action operation, and the reference intermediate action value is the action value of the robotic arm when the multiple robotic arms are in a preset state; a second input module, used to input the intermediate action values, reference intermediate action values, observation values, and partial observable values ​​corresponding to the reference intermediate action values ​​of each of the multiple robotic arms into the decoders corresponding to the multiple robotic arms, to obtain the target action values ​​of the multiple robotic arms, the action prediction model includes a decoder, and the partial observable values ​​are the state information of the robotic arms in a preset state; a control module, used to control the multiple robotic arms to move according to their respective target action values ​​to perform an assembly task on the cylinder head of the engine, wherein the assembly task includes multiple stages, the action prediction model is obtained based on the training of the excitation function, and the excitation function includes sub-excitation functions corresponding to each stage in the assembly task.

[0007] Another aspect of the present invention provides an electronic device, comprising:

[0008] one or more processors;

[0009] a memory for storing one or more programs,

[0010] When one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0011] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0012] Another aspect of the present invention provides a computer program product, which includes computer executable instructions, and the instructions are used to implement the above method when being executed.

[0013] According to an embodiment of the present invention, by inputting the action values ​​and observation values ​​of each of the multiple mechanical arms into the encoder of the action prediction model, the intermediate action values ​​and reference intermediate action values ​​of each of the multiple mechanical arms are obtained. Since the reference intermediate action value is the action value of the mechanical arms when the multiple mechanical arms are in a preset state, the intermediate action values, reference intermediate action values, partial observable values, and reference observable values ​​corresponding to the reference intermediate action values ​​of each of the multiple mechanical arms are respectively input into the decoders corresponding to the multiple mechanical arms to obtain the target action values ​​of each of the multiple mechanical arms, so that the target action values ​​of each of the multiple mechanical arms can make the coordinated motion more stable. At the same time, only the partial observable values ​​of a single mechanical arm need to be obtained, and the target action value of the mechanical arm can also be accurately predicted, so that more applicable scenarios are available. Therefore, the multiple mechanical arms are controlled to move according to their respective target action values ​​to assemble the cylinder head of the engine, so that the stability of the coordinated motion of the multiple mechanical arms is improved and the motion planning method is applicable to more scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0015] Figure 1 A system architecture diagram showing a method for multi-robot coordinated motion planning according to an embodiment of the present invention is shown;

[0016] Figure 2 A flowchart of a multi-manipulator collaborative motion planning method according to an embodiment of the present invention is shown;

[0017] Figure 3a A schematic diagram of an action prediction model according to an embodiment of the present invention is shown;

[0018] Figure 3b A schematic diagram showing a target action value of a single robotic arm according to an embodiment of the present invention is shown;

[0019] Figure 3c A schematic diagram showing an intermediate action value and a reference intermediate action value of a single robot arm according to an embodiment of the present invention is shown;

[0020] Figure 4 A visualization result diagram of a multi-manipulator collaborative assembly platform and a simulation platform according to an embodiment of the present invention is shown;

[0021] Figure 5a Iteration return values ​​of the action prediction model training corresponding to the camshaft installation process according to an embodiment of the present invention are shown;

[0022] Figure 5b The success rate of the action prediction model training corresponding to the camshaft installation process according to an embodiment of the present invention is shown;

[0023] Figure 5c The iterative return value of the action prediction training model corresponding to the valve cover installation process according to an embodiment of the present invention is shown;

[0024] Figure 5d The success rate of the action prediction model training corresponding to the valve cover installation process according to an embodiment of the present invention is shown;

[0025] Figure 6 A block diagram of a multi-manipulator collaborative motion planning device according to an embodiment of the present invention is shown;

[0026] Figure 7 A block diagram of an electronic device suitable for implementing a multi-manipulator collaborative motion planning method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0032] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0033] There is coupling in the grasping motion during the collaboration of multiple robotic arms, resulting in poor stability of the collaborative motion and low scene adaptability.

[0034] An embodiment of the present invention provides a method for collaborative motion planning of multiple robotic arms, including: inputting the action values ​​and observation values ​​of each of the multiple robotic arms into an encoder of a motion prediction model to obtain intermediate action values ​​and reference intermediate action values ​​of each of the multiple robotic arms, wherein the action value represents the action operation performed by the robotic arm, the observation value represents the state information after the robotic arm performs the action operation, and the reference intermediate action value is the action value of the robotic arm when the multiple robotic arms are in a preset state; inputting the intermediate action values, reference intermediate action values, observation values, and partial observable values ​​corresponding to the reference intermediate action values ​​of each of the multiple robotic arms into decoders corresponding to the multiple robotic arms, respectively, to obtain target action values ​​of the multiple robotic arms, the motion prediction model includes a decoder, and the partial observable values ​​are the state information of the robotic arms in a preset state; controlling the multiple robotic arms to move according to their respective target action values ​​to perform an assembly task on the cylinder head of the engine, wherein the assembly task includes multiple stages, the motion prediction model is obtained based on training of an excitation function, and the excitation function includes sub-excitation functions corresponding to each stage in the assembly task.

[0035] Figure 1 A system architecture diagram is shown to which a multi-robot collaborative motion planning method according to an embodiment of the present invention can be applied.

[0036] It should be noted that Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0037] like Figure 1As shown, the system architecture 100 according to this embodiment may include a first robotic arm 101, a second robotic arm 102, a depth camera 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first robotic arm 101, the second robotic arm 102, the depth camera 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, and the like.

[0038] The user can use the first robotic arm 101 , the second robotic arm 102 , and the depth camera 103 to interact with the server 105 through the network 104 to receive or send messages, etc.

[0039] The first robotic arm 101 and the second robotic arm 102 are equipped with a controller, a sensor, etc. The controller is used to control the motor of the robotic arm to rotate so that the joints and the grippers of the robotic arm reach the action value. The controller can be connected to the server 105 to use the controller to control multiple robotic arms to move according to their respective target action values. The environmental information of the first robotic arm 101 and the second robotic arm 102 is collected. For example, the environmental information may include action information, state information, assembly environment, etc. The environmental information of each robotic arm can be represented by a partial observable value.

[0040] At the same time, the degrees of freedom of the first mechanical arm 101 and the second mechanical arm 102 are not limited, and the first mechanical arm 101 and the second mechanical arm 102 cooperate to perform the assembly task on the cylinder head of the engine. The cylinder head of the engine needs to be assembled with a camshaft and a valve cover.

[0041] The multi-robotic arm collaborative control platform may include a first robotic arm 101 , a second robotic arm 102 , a depth camera 103 , and the like.

[0042] The depth camera 103 is used to photograph the cylinder head, camshaft and valve cover during the assembly process to obtain an assembly process image, thereby obtaining position information of the robot arm, cylinder head, camshaft and valve cover.

[0043] For example, the depth camera 103 captures an image of the first mechanical arm 101 and the second mechanical arm 102 cooperatively gripping a camshaft, and the image can be input into the target detection model to obtain the spatial position information of the first mechanical arm 101 and the second mechanical arm 102 and the position information of the camshaft. The spatial position information of the first mechanical arm 101 and the second mechanical arm 102 can be used as their respective observation values, and the position information that can be obtained by the first mechanical arm 101 and the second mechanical arm 102 in a preset state can be used as their respective partial observable values ​​corresponding to the reference intermediate action value.

[0044] The server 105 can be used to input the dynamic value and observation value of the first robotic arm 101 and the second robotic arm 102 respectively gripping the camshaft into the encoder of the action prediction model to obtain the intermediate action value and reference intermediate action value of the first robotic arm 101 and the second robotic arm 102 respectively; input multiple respective intermediate action values, reference intermediate action values, observation values, and partial observable values ​​into the decoders corresponding to the first robotic arm 101 and the second robotic arm 102 respectively to obtain the respective target action values ​​of the first robotic arm 101 and the second robotic arm 102 at the next moment, and control the first robotic arm 101 and the second robotic arm 102 to move according to their respective target action values, thereby completing the installation of the cylinder head.

[0045] It should be noted that the multi-mechanical arm collaborative motion planning method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the multi-mechanical arm collaborative motion planning device provided in the embodiment of the present invention can generally be set in the server 105. The multi-mechanical arm collaborative motion planning method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first mechanical arm 101, the second mechanical arm 102, the depth camera 103 and / or the server 105. Correspondingly, the multi-mechanical arm collaborative motion planning device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first mechanical arm 101, the second mechanical arm 102, the depth camera 103 and / or the server 105. Alternatively, the multi-mechanical arm collaborative motion planning method provided in the embodiment of the present invention can also be performed by the first mechanical arm 101, the second mechanical arm 102, the depth camera 103, or by other terminal devices different from the first mechanical arm 101, the second mechanical arm 102, and the depth camera 103. Correspondingly, the multi-robotic arm collaborative motion planning device provided in an embodiment of the present invention can also be set in the first robotic arm 101, the second robotic arm 102, and the depth camera 103, or in other terminal devices different from the first robotic arm 101, the second robotic arm 102, and the depth camera 103.

[0046] It should be understood that Figure 1 The number of robotic arms, depth cameras, networks, and servers in the system is only . Depending on the implementation needs, there can be any number of robotic arms, depth cameras, networks, and servers.

[0047] Figure 2 A flow chart of a multi-manipulator collaborative motion planning method according to an embodiment of the present invention is shown.

[0048] like Figure 2 As shown, the method includes operations S210 to S230.

[0049] In operation S210, the action values ​​and observation values ​​of the plurality of robotic arms are input into an encoder of a motion prediction model to obtain intermediate action values ​​and reference intermediate action values ​​of the plurality of robotic arms.

[0050] According to an embodiment of the present invention, the action prediction model may be a machine learning algorithm, such as a neural network model or a deep learning model.

[0051] According to an embodiment of the present invention, the encoder may be composed of multiple layers, such as convolutional layers, fully connected layers, or recurrent layers.

[0052] According to an embodiment of the present invention, the action value represents the action operation performed by the robotic arm, the observation value represents the state information of the robotic arm after performing the action operation, and the reference intermediate action value is the action value of the robotic arm when multiple robotic arms are in a preset state.

[0053] For example, the action value may be an instruction for executing the action operation of the robot arm, such as the robot arm moving speed, moving distance, grasping force, grasping opening and closing distance, joint angle, joint angular velocity, joint angular acceleration, etc. The action value may generally be generated by the controller of the robot arm.

[0054] For example, the observation value can be the spatial position of each joint and the posture of each joint after the robot arm performs the action operation. The observation value can usually be collected by sensors and can be used to evaluate the effect of the action operation.

[0055] According to an embodiment of the present invention, the intermediate action value and the reference intermediate action value can both be predicted values ​​of the action value. The reference intermediate action value can be a predicted value of the subsequent collaborative state of multiple robotic arms predicted based on the current action value. For example, the action value can represent the starting action of the current assembly task, the intermediate action value can represent the next action after the starting action, and the reference intermediate action value can represent the action when multiple robotic arms are in a clamping state. The preset state can be a clamping state, a lifting state, a placement state, etc. The preset state is not specifically limited and can be changed according to the progress of the assembly task.

[0056] In operation S220, the intermediate action values, reference intermediate action values, observation values, and partial observable values ​​corresponding to the reference intermediate action values ​​of the multiple robotic arms are respectively input into the decoders corresponding to the multiple robotic arms to obtain the target action values ​​of the multiple robotic arms.

[0057] According to an embodiment of the present invention, the action prediction model includes a decoder. The decoder is a structure corresponding to the encoder. The decoder can be composed of multiple layers, such as convolutional layers, fully connected layers, or recurrent layers.

[0058] According to an embodiment of the present invention, some observable values ​​are state information of a robot arm in a preset state. Some observable values ​​may include action information, state information, assembly environment information, etc. of a single robot arm in a preset state. For example, some observable values ​​may be observation values ​​of a single robot arm in a preset state. Some observable values ​​can reduce the action coupling of multi-robot arm operations in the robot arm motion space and improve the stability of the assembly process.

[0059] Although more information is obtained during the multi-machine assembly process, which is more beneficial for predicting actions, sensors need to be deployed in real application scenarios, and environmental noise will also make the information unreliable. However, the present invention only needs to obtain the partial observable values ​​of a single mechanical arm, and can also accurately predict the target action value of the mechanical arm, making the method applicable to more scenarios and reducing the computational complexity.

[0060] Figure 3a A schematic diagram of an action prediction model according to an embodiment of the present invention is shown.

[0061] like Figure 3a As shown, the action values ​​of the n robotic arms are a1, a2, ...a n , the observation values ​​of the n robotic arms are o1, o2, ... o n n is an integer greater than or equal to 1. Set the action values ​​a1, a2, ...a n And the observations o1, o2, ... o n Input the encoder of the action prediction model to obtain the intermediate action values ​​v1, v2, ...v of multiple robotic arms n .

[0062] For example, the intermediate action value v1, the observation value o1, and the reference intermediate action value v c , and the reference intermediate action value v c The corresponding partial observable value o c Input the decoder 1 corresponding to the first robot arm to obtain the target action value a1 of the first robot arm ’ . Partially observable values ​​o c It may include n observation values ​​of the first robotic arm being in a preset state.

[0063] Similarly, the intermediate action value v of the nth robot arm is n , Observation value o n , reference intermediate action value v c , and the reference intermediate action value v c The corresponding partial observable value o c Input the decoder 1 corresponding to the nth robot arm to obtain the target action value a of the nth robot arm n ’ .

[0064] For example, the reference action values ​​v of multiple robotic arms c , the intermediate action value v1 of the first robot, the observed value o1, and the reference intermediate action value v c The corresponding partial observable value o c Input the decoder 1 corresponding to the first robot arm to obtain the target action value a1 of the first robot arm ’ .

[0065] Similarly, the reference intermediate action values ​​v of multiple robotic arms are c , the intermediate action value v of the nth robot n , Observation value o n , and the reference intermediate action value v c The corresponding partial observable value o c Input the decoder 1 corresponding to the nth robot arm to obtain the target action value a of the nth robot arm n ’ .

[0066] Figure 3b A schematic diagram showing target action values ​​of a single robotic arm according to an embodiment of the present invention is shown.

[0067] like Figure 3b As shown in the figure, i is an integer greater than 1 and less than n. N represents the current robot arm. Taking the i-th robot arm as an example, the intermediate action value v of the i-th robot arm is i , Observation value o i , reference intermediate action value v c , some observable values ​​o c can affect the target action value a i ’ .

[0068] Figure 3c A schematic diagram showing intermediate action values ​​and reference intermediate action values ​​of a single robot arm according to an embodiment of the present invention is shown.

[0069] like Figure 3c As shown, taking the dual robotic arms as an example, i is the second robotic arm and i-1 is the first robotic arm. i is the action value and observation value o of the current robot arm i , a- i is the action value and observation value o- of the other robot arm i , some observable values ​​o c Both can affect the intermediate action value v of the second robot i , reference intermediate action value v c .

[0070] For example, each robot arm is provided with information to be observed to obtain the observation value of each robot arm. The observation value may include: the current joint position, the joint position in the previous time step, and the range of the opening width of the robot arm gripper, etc. The information to be observed may be obtained by the controller of the robot arm. The action value may include the displacement of the joint angle, the opening width of the robot arm gripper, etc.

[0071] In operation S230 , the plurality of robot arms are controlled to move according to their respective target motion values ​​to perform an assembly task on a cylinder head of an engine.

[0072] According to an embodiment of the present invention, the assembly task includes multiple stages, the action prediction model is obtained by training based on an excitation function, and the excitation function includes sub-excitation functions corresponding to each stage in the assembly task.

[0073] According to an embodiment of the present invention, to perform an assembly task on a cylinder head of an engine, each robot arm needs to have a corresponding action sequence. The action sequence is a target action value arranged based on the order of motion. The target action value can be predicted using an action prediction model. The partial observable value corresponding to the reference intermediate action value can associate the information between multiple robot arms, so that the predicted target action value can make the coordinated motion of multiple robot arms more stable.

[0074] According to an embodiment of the present invention, by inputting the action values ​​and observation values ​​of each of the multiple mechanical arms into the encoder of the action prediction model, the intermediate action values ​​and reference intermediate action values ​​of each of the multiple mechanical arms are obtained. Since the reference intermediate action value is the action value of the mechanical arms when the multiple mechanical arms are in a preset state, the intermediate action values, reference intermediate action values, partial observable values, and reference observable values ​​corresponding to the reference intermediate action values ​​of each of the multiple mechanical arms are respectively input into the decoders corresponding to the multiple mechanical arms to obtain the target action values ​​of each of the multiple mechanical arms, so that the target action values ​​of each of the multiple mechanical arms can make the coordinated motion more stable. At the same time, only the partial observable values ​​of a single mechanical arm need to be obtained, and the target action value of the mechanical arm can also be accurately predicted, so that more applicable scenarios are available. Therefore, the multiple mechanical arms are controlled to move according to their respective target action values ​​to assemble the cylinder head of the engine, so that the stability of the coordinated motion of the multiple mechanical arms is improved and the motion planning method is applicable to more scenarios.

[0075] According to an embodiment of the present invention, the assembly task includes the following stages: a stage of grabbing parts to be assembled, a stage of lifting parts to be assembled, a stage of translating parts to be assembled, and a stage of placing parts to be assembled. The parts to be assembled include a camshaft and a valve cover, and the robotic arm includes a clamp.

[0076] According to an embodiment of the present invention, the stage of grabbing the parts to be assembled can be a process in which multiple robotic arms cooperate to grab the parts to be assembled. For example, the first robotic arm grabs the left end of the part to be assembled, the second robotic arm grabs the right end of the part to be assembled, and the first robotic arm and the second robotic arm both reach the preset grabbing position and adjust the grabbing force to complete the process.

[0077] According to an embodiment of the present invention, the stage of lifting the part to be assembled can be a process in which multiple mechanical arms cooperate to lift the part to be assembled. For example, the first mechanical arm and the second mechanical arm simultaneously lift the part to be assembled in space so that the part to be assembled is raised to a preset height from the cylinder head of the engine.

[0078] According to an embodiment of the present invention, the stage of moving the part to be assembled may be a process in which multiple robotic arms cooperate to move the part to be assembled to the cylinder head of the engine. For example, the first robotic arm and the second robotic arm simultaneously translate the part to be assembled in space so that the part to be assembled is moved to the cylinder head of the engine.

[0079] According to an embodiment of the present invention, the stage of placing the parts to be assembled can be viewed as a process in which multiple mechanical arms cooperate to stably place the parts to be assembled on the cylinder head of the engine. For example, the first mechanical arm and the second mechanical arm simultaneously reduce the gripping force and increase the opening and closing distance of the gripper to stably place the parts to be assembled on the cylinder head of the engine.

[0080] According to an embodiment of the present invention, controlling multiple robotic arms to move according to their respective target action values ​​to assemble the cylinder head of the engine includes: utilizing a multi-robotic arm collaborative assembly platform to control multiple robotic arms to move according to their respective target action values ​​to assemble the parts to be assembled to the installation position of the cylinder head.

[0081] According to an embodiment of the present invention, a multi-robotic arm collaborative assembly platform includes controllers for respective multiple robotic arms.

[0082] For example, the controllers of the multiple robotic arms in the multi-robotic arm collaborative assembly platform control the multiple robotic arms so that they move according to their respective target action values, thereby assembling the parts to be assembled to the installation position of the cylinder head.

[0083] According to an embodiment of the present invention, the multi-robot collaborative assembly platform further includes a depth camera, and the above method further includes:

[0084] The assembly process images taken by the depth camera are input into the target detection model to obtain the position detection results, which include the position information of the robot arm, cylinder head and parts to be assembled.

[0085] According to an embodiment of the present invention, the number of depth cameras is not limited, and each robotic arm can be equipped with a depth camera. The depth camera can capture the assembly process image of the cylinder head, camshaft and valve cover of the engine during the assembly task in real time. The assembly process image is input into the target detection model, and the position information of the robotic arm, cylinder head, camshaft and valve cover can be identified in real time. The position information of the robotic arm, cylinder head and parts to be assembled can be used as part of the partial observable value of a single robotic arm to generate a target action value suitable for the scene.

[0086] According to an embodiment of the present invention, the target detection model may be a YOLOv5 (You Only Look Once Version 5, real-time object detection model) network model.

[0087] For example, use a depth camera to take 100 images of the assembly process at different angles, and mark the area with parts in each assembly process image. Parts may include a robotic arm, a cylinder head, a camshaft, and a valve cover. All the marked images are used as a data set to input into the YOLOv5 network training to obtain a trained YOLOv5 network.

[0088] For example, the parts may include a robot arm, a cylinder head, a camshaft, and a valve cover. The depth camera collects the parts information, the object detection model determines the position of the parts in the image, and then calculates the three-dimensional position of the parts in the robot arm base coordinate system. Finally, the motion prediction model outputs the moving position of the robot arm joint and the opening and closing position of the gripper, thereby completing the assembly task of the engine cylinder head.

[0089] According to an embodiment of the present invention, the action prediction model is obtained based on the following training method: the simulated action values, simulated observation values ​​and simulated partially observable values ​​of each of the multiple simulated robotic arms are obtained from a simulation platform, the simulation platform is used to simulate the assembly task process under a multi-robotic arm collaborative assembly platform, and the simulation platform includes simulated robotic arms corresponding to each of the multiple robotic arms, a simulated cylinder head corresponding to the cylinder head of the engine, and simulated parts to be assembled corresponding to the parts to be assembled; the action prediction model is trained using the simulated action values, simulated observation values ​​and simulated partially observable values ​​of each of the multiple simulated robotic arms.

[0090] According to an embodiment of the present invention, the simulated partial observable value may be an observed value of a single simulated robotic arm in a preset state.

[0091] According to an embodiment of the present invention, the simulated parts to be assembled include a simulated camshaft and a simulated valve cover.

[0092] According to an embodiment of the present invention, a simulated valve cover is randomly placed at an initial position on a simulation platform, a depth camera identifies the positions of a simulated cylinder head and a simulated valve cover in real time, and the valve cover is installed at the installation position on the cylinder head in three stages through a trained deep network model.

[0093] According to an embodiment of the present invention, the effectiveness of the multi-manipulator collaborative motion planning method for assembly tasks can be verified through simulation platform experiments and multi-manipulator collaborative assembly platform experiments. In the simulation platform, a simulation physics engine and a manipulator equipped with a gripper are used. In order to achieve the transformation from simulation to reality, a trained motion prediction model is used as a planner for the manipulator, providing the target motion value of the manipulator. The communication between all the information obtained by the sensor and the joint control command is established in the robot operating system.

[0094] Simulating the actual assembly process on the simulation platform can reduce the number of trial and error in actual applications and save resources.

[0095] According to an embodiment of the present invention, training an action prediction model using simulated action values, simulated observation values ​​and simulated partially observable values ​​of multiple simulated robotic arms includes: inputting the simulated action values, simulated observation values ​​and simulated partially observable values ​​of the ith simulated robotic arm into an actor network corresponding to the ith simulated robotic arm in the action prediction model to obtain a predicted action of the ith simulated robotic arm, where i is an integer greater than or equal to 1; inputting the predicted action of the ith simulated robotic arm into a critic network corresponding to the ith simulated robotic arm in the action prediction model to obtain a value of the predicted action; and updating the parameters of the actor network according to the value of the predicted action.

[0096] According to an embodiment of the present invention, MASAC (Multi-Agent Soft Actor-Critic) is used to train the motion prediction model in a simulation platform. In each training process, the simulation environment starts with a camshaft or valve cover randomly distributed on a desktop, the dual robotic arms are reset to a predefined initial posture, and the model parameters are iteratively updated to maximize the excitation function, thereby obtaining a trained motion prediction model.

[0097] For example, if there are two robotic arms, the action prediction model includes the target actor network of the current robotic arm and the actor network of the other robotic arm, as well as the target critic networks of the two current robotic arms and the critic networks of the two other robotic arms. The actor network is used to input the current state and output the action during the interaction between the robotic arm and the environment. The two critic networks simultaneously evaluate the actions of the actor network to reduce overestimation. The target actor network and the two target critic networks obtain the results through soft updating and finally output the optimal strategy. The agents corresponding to each robotic arm (actor network and critic network) are not affected by other agents. When one agent obtains a feasible strategy, the other agents will continue to learn, thereby improving the collaboration efficiency between the two robotic arms.

[0098] The actor network of each robot is responsible for generating actions for that robot. This allows each robot to make independent decisions based on its own state and task requirements. The critic network of each robot evaluates the performance of the actor network. This approach allows each robot to learn independently based on its own experience and improve the targeting of the strategy. Although each robot has an independent network, it can share environmental state information and reward signals for better coordination.

[0099] Figure 4 A visualization result diagram of a multi-robotic arm collaborative assembly platform and a simulation platform according to an embodiment of the present invention is shown.

[0100] like Figure 4 As shown, the simulation platform and the multi-robot collaborative assembly platform ( Figure 4 There is a corresponding relationship between the real platform). For example, the positions of the camera (depth camera), robotic arm, camshaft, valve cover, and cylinder head are all one-to-one. At the same time, the functions that the camera and robotic arm can be configured with are also the same. The depth camera can be miniature, which is convenient for multiple robotic arms to perform assembly tasks.

[0101] First, the simulated action values, simulated observation values, and simulated partial observable values ​​of the two simulated robotic arms in the simulation platform are input into the actor networks corresponding to the two simulated robotic arms in the action prediction model to obtain the predicted actions of the simulated robotic arms. The predicted actions of the two simulated robotic arms are input into the critic networks corresponding to the two simulated robotic arms in the action prediction model to obtain the values ​​of the predicted actions; the parameters of the actor networks are updated according to the values ​​of the predicted actions.

[0102] The two simulated manipulators perform their respective predicted actions and obtain the state information corresponding to the predicted action (state extraction), that is, the predicted observation value. And update the state. After taking the predicted action and predicted observation value as the current action value and observation value, the current action value and observation value are input into the action prediction model again (experience playback) to obtain the next predicted action value (store the predicted action value in the experience pool and comment on the value of the predicted action). The composition of the actor network and the critic network is called an intelligent agent.

[0103] The state s1 and action a1 of simulated robot arm 1 are input into the action network and evaluation network corresponding to robot arm 1 to train the intelligent agent corresponding to robot arm 1; the state s2 and action a2 of simulated robot arm 2 are input into the action network and evaluation network corresponding to robot arm 2 to train the intelligent agent corresponding to robot arm 2.

[0104] In actual application, the current action values ​​and current observation values ​​of robot arm 1 and robot arm 2 are input into the trained action prediction model to obtain the target action values ​​of robot arm 1 and robot arm 2.

[0105] According to an embodiment of the present invention, the incentive function includes a grabbing sub-incentive function corresponding to the stage of grabbing the parts to be assembled, and the grabbing sub-incentive function is as follows:

[0106] (1);

[0107] r grasp represents the reward for grabbing the parts to be assembled, l represents the number of robotic arms, s t is the state at time t during the assembly process, t is greater than 0, w g is the weight of the crawling stage, p gi is the coordinate position of the gripper of the i-th robot arm, p ci is the gripping position of the gripper of the i-th robotic arm corresponding to the i-th simulated robotic arm on the part to be assembled.

[0108] According to an embodiment of the present invention, the incentive function further includes a promotion sub-incentive function corresponding to the stage of promoting the parts to be assembled, and the promotion sub-incentive function is as follows:

[0109] (2);

[0110] r lift Represents the reward for the stage of lifting the parts to be assembled. In the stage of lifting the parts to be assembled, multiple robotic arms collaborate to lift the parts to be assembled to the target height. Two parts of rewards are set respectively, one is the first distance error from the height of the parts to be assembled to the target height during the lifting process, and the other is the first tilt error of the parts to be assembled during the lifting process, w l1 is the reward weight of the first distance, w l2is the reward weight of the first tilt error, p l is the height of the parts to be assembled during the lifting process, p lt is the target height, θ l is the inclination angle of the assembly to be assembled on the vertical plane, θ lmax It is the maximum inclination angle allowed for the assembly on the vertical plane.

[0111] According to an embodiment of the present invention, the excitation function further includes a translation sub-excitation function corresponding to the stage of translating the to-be-assembled parts, and the translation sub-excitation function is as follows:

[0112] (3);

[0113] r trans Represents the reward for the translation phase of the assembly part. The translation phase is to use multiple robotic arms to translate the assembly part to the target translation position of the cylinder head. Two parts of rewards are set respectively, one is the second distance error from the center of mass position to the target translation position during the translation process of the assembly part, and the other is the second tilt error of the assembly part during the translation process. t1 is the reward weight of the second distance error, w t2 is the reward weight of the second tilt error, p t is the center of mass position of the parts to be assembled, p tt is the target translation position of the part to be assembled, θ t is the horizontal inclination angle of the part to be assembled, θ tmax is the maximum allowable horizontal tilt angle.

[0114] According to an embodiment of the present invention, the incentive function further includes a placement sub-incentive function corresponding to the stage of placing the parts to be assembled, and the placement sub-incentive function is as follows:

[0115] (4);

[0116] r place Indicates the reward for placing the parts to be assembled, w p is the weight of the stage of placing the parts to be assembled, p pi To use the gripper of the i-th robot arm to grasp the coordinate positions of the two ends of the cover to be assembled, p ci The clamp of the i-th robot arm is used to place the two ends of the assembly to be assembled on the cylinder head;

[0117] The activation function is as follows:

[0118] (5).

[0119] Figure 5a Iteration return values ​​of the motion prediction model training corresponding to the camshaft installation process according to an embodiment of the present invention are shown.

[0120] like Figure 5a As shown in the figure, with the increase of the number of training steps, the iterative return value of the action prediction model training corresponding to the camshaft installation process tends to converge and reach a stable value.

[0121] Figure 5b The success rate of the action prediction model training corresponding to the camshaft installation process according to an embodiment of the present invention is shown.

[0122] like Figure 5b As shown in the figure, with the increase of training steps, the success rate of the action prediction model training corresponding to the camshaft installation process also increases. Since the present invention adopts four stages of grasping stage sub-excitation function, lifting stage sub-excitation function, translation sub-excitation function and placement sub-excitation function, the action of the robot arm is more staged, so the training success rate is obviously close to 1, and the high accuracy of the camshaft installation process can be maintained in multiple scenarios.

[0123] Figure 5c The iterative return value of the action prediction training model corresponding to the valve cover installation process according to an embodiment of the present invention is shown.

[0124] like Figure 5c As shown in the figure, with the increase of the number of training steps, the iterative return value of the action prediction model training corresponding to the valve cover installation process tends to converge and reach a stable value.

[0125] Figure 5d The success rate of the action prediction model training corresponding to the valve cover installation process according to an embodiment of the present invention is shown.

[0126] like Figure 5d As shown, as the number of training steps increases, the success rate of the action prediction model training corresponding to the valve cover installation process also increases. Since the present invention adopts four stages of grasping stage sub-excitation function, lifting stage sub-excitation function, translation sub-excitation function and placement sub-excitation function, the action of the robot arm is more staged, so the training success rate is obviously close to 1, and the high accuracy of the valve cover installation process can be maintained in multiple scenarios.

[0127] Figure 6 A block diagram of a multi-manipulator collaborative motion planning device according to an embodiment of the present invention is shown.

[0128] like Figure 6 As shown, the multi-robotic arm collaborative motion planning device 600 includes a first input module 610 , a second input module 620 and a control module 630 .

[0129] The first input module 610 is used to input the action values ​​and observation values ​​of each of the multiple robotic arms into the encoder of the action prediction model to obtain the intermediate action values ​​and reference intermediate action values ​​of each of the multiple robotic arms, wherein the action value represents the action operation performed by the robotic arm, the observation value represents the state information after the robotic arm performs the action operation, and the reference intermediate action value is the action value of the robotic arm when the multiple robotic arms are in a preset state. In one embodiment, the first input module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0130] The second input module 620 is used to input the intermediate action values, reference intermediate action values, observation values, and partial observable values ​​corresponding to the reference intermediate action values ​​of the multiple robotic arms into the decoders corresponding to the multiple robotic arms, respectively, to obtain the target action values ​​of the multiple robotic arms, the action prediction model includes a decoder, and the partial observable values ​​are state information of the robotic arms in a preset state. In one embodiment, the second input module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0131] The control module 630 is used to control the multiple robot arms to move according to their respective target action values ​​to perform an assembly task on the cylinder head of the engine, wherein the assembly task includes multiple stages, and the action prediction model is obtained by training based on an incentive function, and the incentive function includes a sub-incentive function corresponding to each stage in the assembly task. In one embodiment, the control module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0132] According to an embodiment of the present invention, the assembly task includes the following stages: a stage of grabbing parts to be assembled, a stage of lifting parts to be assembled, a stage of translating parts to be assembled, and a stage of placing parts to be assembled. The parts to be assembled include a camshaft and a valve cover, and the robotic arm includes a clamp.

[0133] According to an embodiment of the present invention, the control module includes a control submodule. The control submodule is used to control multiple robotic arms to move according to their respective target action values ​​using a multi-robotic arm collaborative assembly platform to assemble the parts to be assembled to the installation position of the cylinder head.

[0134] According to an embodiment of the present invention, the multi-robot collaborative assembly platform further includes a depth camera, and the above device further includes a position detection module. The position detection module is used to input the assembly process image captured by the depth camera into the target detection model to obtain a position detection result, and the position detection result includes the position information of the robot arm, the cylinder head and the parts to be assembled.

[0135] According to an embodiment of the present invention, the action prediction model is obtained based on the following training method: the simulated action values, simulated observation values ​​and simulated partially observable values ​​of each of the multiple simulated robotic arms are obtained from a simulation platform, the simulation platform is used to simulate the assembly task process under a multi-robotic arm collaborative assembly platform, and the simulation platform includes simulated robotic arms corresponding to each of the multiple robotic arms, a simulated cylinder head corresponding to the cylinder head of the engine, and simulated parts to be assembled corresponding to the parts to be assembled; the action prediction model is trained using the simulated action values, simulated observation values ​​and simulated partially observable values ​​of each of the multiple simulated robotic arms.

[0136] According to an embodiment of the present invention, training an action prediction model using simulated action values, simulated observation values ​​and simulated partially observable values ​​of multiple simulated robotic arms includes: inputting the simulated action values, simulated observation values ​​and simulated partially observable values ​​of the ith simulated robotic arm into an actor network corresponding to the ith simulated robotic arm in the action prediction model to obtain a predicted action of the ith simulated robotic arm, where i is an integer greater than or equal to 1; inputting the predicted action of the ith simulated robotic arm into a critic network corresponding to the ith simulated robotic arm in the action prediction model to obtain a value of the predicted action; and updating the parameters of the actor network according to the value of the predicted action.

[0137] According to an embodiment of the present invention, the excitation function includes a grasping sub-excitation function corresponding to the grasping stage of the parts to be assembled, and the grasping sub-excitation function is shown in formula (1).

[0138] According to an embodiment of the present invention, the incentive function further includes a promotion sub-incentive function corresponding to the stage of promoting the parts to be assembled, and the promotion sub-incentive function is shown in formula (2).

[0139] According to an embodiment of the present invention, the excitation function further includes a translation sub-excitation function corresponding to the stage of translating the to-be-assembled parts, and the translation sub-excitation function is shown in formula (3).

[0140] According to an embodiment of the present invention, the incentive function further includes a placement sub-incentive function corresponding to the stage of placing the parts to be assembled, and the placement sub-incentive function is shown in formula (6). The incentive function is shown in formula (5).

[0141] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0142] For example, any multiple of the first input module 610, the second input module 620 and the control module 630 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the first input module 610, the second input module 620 and the control module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the first input module 610 , the second input module 620 , and the control module 630 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be executed.

[0143] It should be noted that the multi-robot collaborative motion planning device part in the embodiment of the present invention corresponds to the multi-robot collaborative motion planning method part in the embodiment of the present invention. The description of the multi-robot collaborative motion planning device part specifically refers to the multi-robot collaborative motion planning method part, which will not be repeated here.

[0144] Figure 7A block diagram of an electronic device suitable for implementing a multi-manipulator collaborative motion planning method according to an embodiment of the present invention is shown.

[0145] Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0146] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0147] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0148] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0149] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0150] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0151] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0152] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0153] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the robotic arm collaborative motion planning method provided by the embodiment of the present invention.

[0154] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0155] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0156] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0158] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A multi-manipulator collaborative motion planning method, characterized in that: The method comprises: Inputting the action values ​​and observation values ​​of each of the plurality of robotic arms into the encoder of the action prediction model, obtaining the intermediate action values ​​and reference intermediate action values ​​of each of the plurality of robotic arms, wherein the action value represents the action operation performed by the robotic arm, the observation value represents the state information of the robotic arm after performing the action operation, and the reference intermediate action value is the action value of the robotic arm when the plurality of robotic arms are in a preset state; Input the intermediate action values, the reference intermediate action values, the observation values, and the partial observable values ​​corresponding to the reference intermediate action values ​​of the respective robotic arms into the decoders corresponding to the respective robotic arms, respectively, to obtain the target action values ​​of the respective robotic arms, the action prediction model includes the decoders, and the partial observable values ​​are state information of the robotic arms in the preset state; Control the multiple robotic arms to move according to their respective target action values ​​to perform an assembly task on the cylinder head of the engine, wherein the assembly task includes multiple stages, and the action prediction model is obtained by training based on an excitation function, and the excitation function includes sub-excitation functions corresponding to each stage in the assembly task.

2. The method according to claim 1, characterized in that The assembly task The following phases are included: The mechanical arm comprises a stage of grabbing parts to be assembled, a stage of lifting parts to be assembled, a stage of translating parts to be assembled and a stage of placing parts to be assembled. The parts to be assembled include a camshaft and a valve cover, and the mechanical arm includes a clamp.

3. The method according to claim 1, characterized in that The controlling the plurality of mechanical arms to move according to the respective target action values ​​to assemble the cylinder head of the engine comprises: A multi-robotic arm collaborative assembly platform is utilized to control the plurality of robotic arms to move according to their respective target action values, so as to assemble the parts to be assembled to the installation position of the cylinder head.

4. The method according to claim 3, characterized in that The multi-robot collaborative assembly platform further includes a depth camera, and the method further includes: The assembly process image captured by the depth camera is input into the target detection model to obtain a position detection result, wherein the position detection result includes position information of the robot arm, the cylinder head and the parts to be assembled.

5. The method according to claim 1, characterized in that The action prediction model is obtained based on the following training method: Acquire simulated action values, simulated observation values, and simulated partially observable values ​​of each of the plurality of simulated robotic arms from a simulation platform, wherein the simulation platform is used to simulate an assembly task process under a multi-robotic arm collaborative assembly platform, and the simulation platform includes the simulated robotic arms corresponding to each of the plurality of robotic arms, the simulated cylinder head corresponding to the cylinder head of the engine, and the simulated parts to be assembled corresponding to the parts to be assembled; The action prediction model is trained using the simulated action values, the simulated observation values ​​and the simulated partial observable values ​​of each of the plurality of simulated robotic arms.

6. The method according to claim 5, characterized in that The training of the action prediction model using the simulated action values, the simulated observation values ​​and the simulated partial observable values ​​of each of the plurality of simulated manipulators comprises: Inputting the simulated action value, the simulated observation value and the simulated partial observable value of the i-th simulated mechanical arm into the actor network corresponding to the i-th simulated mechanical arm in the action prediction model to obtain the predicted action of the i-th simulated mechanical arm, where i is an integer greater than or equal to 1; Inputting the predicted action of the i-th simulated robotic arm into the critic network corresponding to the i-th simulated robotic arm in the action prediction model to obtain the value of the predicted action; Parameters of the actor network are updated according to the value of the predicted action.

7. The method according to claim 2, characterized in that The incentive function includes a grabbing sub-incentive function corresponding to the grabbing stage of the parts to be assembled, and the grabbing sub-incentive function is as follows: r grasp represents the reward for the stage of grabbing the parts to be assembled, l represents the number of robotic arms, s t is the state at time t during the assembly process, t is greater than 0, w g is the weight of the crawling stage, p gi is the coordinate position of the gripper of the i-th robot arm, p ci It is the clamping position of the clamper of the i-th robotic arm corresponding to the i-th simulated robotic arm on the part to be assembled.

8. The method according to claim 2, characterized in that: The incentive function also includes a lifting sub-incentive function corresponding to the lifting stage of the to-be-assembled parts, and the lifting sub-incentive function is as follows: r lift represents the reward in the stage of lifting the to-be-assembled part. In the stage of lifting the to-be-assembled part, the plurality of the robotic arms cooperate to lift the to-be-assembled part to the target height. Two rewards are set respectively. One is the first distance error from the height of the to-be-assembled part to the target height during the lifting process, and the other is the first tilt error of the to-be-assembled part during the lifting process. l1 is the reward weight of the first distance, w l2 is the reward weight of the first tilt error, p l is the height of the part to be assembled during the lifting process, p lt is the target height, θ l is the inclination angle of the part to be assembled on the vertical plane, θ lmax It is the maximum inclination angle allowed for the part to be assembled on the vertical plane.

9. The method according to claim 2, characterized in that: The excitation function also includes a translation sub-excitation function corresponding to the translation stage of the to-be-assembled parts, and the translation sub-excitation function is as follows: r trans represents the reward of the translation stage of the part to be assembled. The translation stage is to use multiple robotic arms to translate the part to be assembled to the target translation position of the cylinder head. Two parts of rewards are set respectively. One part is the second distance error from the center of mass position to the target translation position during the translation of the part to be assembled, and the other part is the second tilt error of the part to be assembled during the translation of the part to be assembled. t1 is the reward weight of the second distance error, w t2 is the reward weight of the second tilt error, p t is the center of mass position of the part to be assembled, p tt is the target translation position of the part to be assembled, θ t is the horizontal inclination angle of the part to be assembled, θ tmax is the maximum allowable horizontal tilt angle.

10. The method according to claim 9, characterized in that The incentive function also includes a placement sub-incentive function corresponding to the stage of placing the parts to be assembled, and the placement sub-incentive function is as follows: r place represents the reward for the stage of placing the parts to be assembled, w p is the weight of the stage of placing the parts to be assembled, p pi The coordinate positions of the two ends of the cover to be assembled are grasped by the gripper of the i-th robot arm, p ci Using the gripper of the i-th mechanical arm to place the two ends of the to-be-assembled component at the mounting positions on the cylinder head; The activation function is as follows: 。

Images