An autonomous control method and system for a robotic arm applicable to grasping occluded objects

Abstract

The present invention discloses a method for autonomous control of a robotic arm suitable for grasping occluded objects. S1. Initialize the robotic arm and collect RGB color image data and Depth depth image data; S2. Observe and identify the currently collected scene; S3. Use the visible information of scene objects in matrix form to determine whether the i-th target object is visible; S4. According to the centroid coordinates of the object to be operated, the network outputs a direction for the robotic arm to push the object, and at the same time gives decision information according to the reward function; S5. The robotic arm performs a grasping or pushing action according to the received centroid coordinate information of the object to be operated; S6. The robotic arm performs a pushing action to push away obstacles and search for the target object until the target object is visible and unoccluded; after the target object is visible and unoccluded, the robotic arm grasps the target object, returns and places it on the table. This method can achieve the grasping of laterally occluded target objects and can be applied to various scenarios such as unmanned stores, hospital pharmacies, and warehouses.

Description

Technical Field

[0001] The present invention relates to the technical field of robotics, and specifically refers to a robotic arm autonomous control method and system suitable for grasping occluded objects. Background Art

[0002] The autonomous decision-making of robotic arms has important applications in the fields of human-computer interaction, service robots, etc., and has received extensive attention from researchers. Among them, searching for specific target objects from cluttered interfering objects and separating them from the sundries is a complex and meaningful task. For example, taking a bottle of seasoning from a household cabinet, taking an item from a warehouse shelf, or taking a bottle of beverage from a store display cabinet. This requires the robot to interact with the environment multiple times because the target object may not be visible and may also be occluded by other interfering objects. Most of the existing work on grasping target objects assumes that the interfering objects can be removed from the environment to facilitate the robot to discover the target object, and adopts a top-down grasping method, assuming that the objects are basically on the same plane along the camera axis, with many limitations and unable to meet the actual situations listed above. In view of the problem of searching for and grasping target objects in the horizontal direction, it is of great practical significance to develop a robotic arm autonomous control method and system suitable for grasping occluded objects. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention proposes a robotic arm autonomous control method and system suitable for grasping occluded objects. The system completes the pushing and grasping operations of the corresponding objects through image acquisition, twin network recognition, reinforcement learning algorithm to select target objects, reinforcement learning algorithm to execute action decisions, and robotic arm trajectory planning. It integrates multiple technologies such as instance segmentation, neural network, 6-axis robotic arm control, and trajectory planning. It can realize the grasping of horizontally occluded target objects and can be applied to various scenarios such as unmanned stores, hospital pharmacies, and warehouses.

[0004] To solve the above technical problems, the technical solution of the present invention is as follows:

[0005] A robotic arm autonomous control system suitable for grasping occluded objects includes an image data acquisition module, an object selection module, an action decision module, and a robotic arm grasping module;

[0006] The image data acquisition module includes:

[0007] A Realsense D435 depth camera for collecting RGB color image data and Depth depth image data;

[0008] The object selection module is used to input the Depth depth image of the scene and the scene foreground segmentation mask of the Depth depth image, and use the Double DQN reinforcement learning algorithm to select the object to be interacted with;

[0009] An action decision-making module, which is used to input a local Depth depth image and a mask of the object to be interacted with in the local Depth depth image, and uses the Double DQN reinforcement learning algorithm to determine the action to be executed by the robotic arm;

[0010] A robotic arm grasping module, which is used to perform actions on the object to be operated.

[0011] Among them, the RGB color image is used for the Siamese network to identify the target object, and the Depth depth image is used for foreground instance segmentation, and for the object selection module and the action decision-making module to select the target object and make decisions on the actions to be executed by the robotic arm.

[0012] Preferably, the robotic arm grasping module includes a UR10 six-axis robotic arm, a Robotiq two-finger gripper, and a six-axis torque sensor. The UR10 six-axis robotic arm and the two-finger gripper are responsible for grasping or pushing the object to be operated, and the six-axis torque sensor is used to measure the force of the two-finger gripper closing.

[0013] Preferably, the depth camera is installed at the end of the UR10 six-axis robotic arm.

[0014] The present invention also discloses a control method for the robotic arm autonomous control system applicable to grasping occluded objects according to the above, including the following steps:

[0015] S1. Initialize the robotic arm, move it to the observation pose, and collect RGB color image data and Depth depth image data;

[0016] S2. Observe and identify the currently collected scene:

[0017] Use the Depth depth image data for foreground segmentation and instance segmentation of the scene;

[0018] Use the pre-trained Siamese network to identify the RGB color image data under the segmentation mask to obtain the visible information of the scene objects in matrix form;

[0019] S3. Use the visible information of the scene objects in matrix form to judge whether the i-th target object is visible, that is, judge whether there is a target object;

[0020] S4. According to the centroid coordinates of the object to be operated, intercept a local depth image centered on this object, combine it with the binary image of the instance mask of this object, input it into the Double DQN decision network, the network outputs a direction for the robotic arm to push the object, and continue to transfer the centroid coordinates to the robotic arm, and at the same time give decision information according to the reward function;

[0021] S5. The robotic arm performs a grasping or pushing action according to the center-of-gravity coordinate information of the object to be operated received.

[0022] S6. The robotic arm performs a pushing action to push aside obstacles and search for the target object until the target object is visible and unobstructed. After the target object is visible and unobstructed, the robotic arm grasps the target object, returns, and places it on the table.

[0023] Preferably, in the step S2,

[0024] The method of using Depth depth image data for foreground segmentation of the scene is as follows:

[0025] The background behind the scene with a depth value less than the threshold ξ is removed using the depth value size condition, and then the background around the scene with horizontal and vertical gradients being negative values is removed using the gradient condition of the depth value, obtaining a foreground mask composed of all objects in the scene.

[0026] The method of using Depth depth image data for instance segmentation of the scene is: if there is no marked point in the neighborhood of a pixel point, then assign a new label to this pixel point.

[0027] If there is a marked point in the neighborhood of a pixel point, query the depth values of the marked points in the neighborhood and determine whether there is a pixel point with a difference in depth value from the central pixel less than the threshold η.

[0028] If there is a marked point in the neighborhood with a difference in depth value less than the threshold η, take the point with the smallest difference and assign its label to the central pixel point; otherwise, assign a new label to the central pixel point.

[0029] Preferably, the visible information of the scene objects in matrix form, that is, there is a landmark matrix EFM, and the expression is as follows:

[0030]

[0031] Among them, n rows represent n objects and have a fixed order, m columns represent m instance masks of instance segmentation and do not have a fixed order. The value of the i-th row and the j-th column of the matrix is 1, EFM(i, j) = 1, indicating that the j-th segmentation instance is recognized as the i-th object by the siamese network.

[0032] Preferably, in the step S3, the method for determining whether the i-th target object is visible is as follows:

[0033] That is, to determine whether there is:

[0034] If the target object is visible, then determine whether it is blocked, that is, determine whether there is a depth value smaller than the instance itself in the neighborhood of the instance mask.

[0035] If the target object is invisible, the depth image of the scene and the binary image of the scene foreground are used as the state and input into the Double DQN decision network, and the network outputs the centroid coordinates of an instance.

[0036] Preferably, the reward function reward of the Double DQN decision network is described as follows:

[0037] First, define the horizontal distribution of the foreground mask, that is, compress an n×m matrix M into an m-dimensional row vector V.

[0038]

[0039] And after performing the kth action, there is V k = V ∩ V k-1 The operation of taking the intersection makes V k become smaller and smaller. Therefore, V k actually represents the unexplored foreground area, and there must be ∑V k ≤ ΣV k-1 ; The calculation of the horizontal distribution I of the object instance is the same as that of V, but there is no need to take the intersection as the action progresses; the reward is obtained from the following formula:

[0040]

[0041] where I is the horizontal distribution of the object to be operated. The above formula shows that when the selected object is in the unexplored foreground, the selection of the object is considered effective; while when the selected object is in the explored foreground, the selection of the object is considered a failure.

[0042] Preferably, in step S5, when the signal of the centroid coordinates comes from the object selection module, the grasping action is executed; when the signal of the centroid coordinates comes from the action decision module, the pushing action is executed.

[0043] The present invention has the following characteristics and beneficial effects:

[0044] Adopting the above technical solution, multi-sensors are used for information collection, neural network technology is used to identify the scene, reinforcement learning algorithm is used to select the object to be operated and make action decisions, and finally a series of actions are completed through robotic arm path planning; the system adopts a modular design, with clear division of labor and clear hierarchical structure, which can efficiently explore the target object and accurately identify the target object; there is no need to remove obstacles when exploring and searching for the target object, and it has certain practical value in specific scenarios. Description of the Drawings

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0046] Figure 1 It is a schematic flowchart of the method according to the embodiment of the present invention. Detailed implementation manners

[0047] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0048] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "center", "longitudinal", "transverse", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "mounted", "connected", "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0050] This embodiment discloses a robotic arm autonomous control system applicable to grasping occluded objects, as Figure 1 shown, including an image data acquisition module, an object selection module, an action decision module, and a robotic arm grasping module;

[0051] The image data acquisition module includes:

[0052] The Realsense D435 depth camera is used to collect RGB color image data and Depth depth image data;

[0053] The object selection module is used to input the Depth depth image of the scene and the scene foreground segmentation mask of the Depth depth image, and use the Double DQN reinforcement learning algorithm to select the object to be interacted with;

[0054] The action decision module is used to input the local Depth depth image and the mask of the object to be interacted with in the local Depth depth image, and use the Double DQN reinforcement learning algorithm to determine the action to be performed by the robotic arm;

[0055] The robotic arm grasping module is used to perform actions on the object to be operated.

[0056] It should be noted that in this embodiment, the object selection module is a Q network loaded with U-Net as the reinforcement learning algorithm; the action decision module is a Q network loaded with a CNN convolutional network as the reinforcement learning algorithm; the robotic arm grasping module consists of a UR10 robotic arm and a ROS robot operating system.

[0057] Among them, the RGB color image is used for the twin network to realize the recognition of the target object, and the Depth depth image is used for foreground instance segmentation, and for the object selection module and the action decision module to realize the selection of the target object and the decision of the robotic arm to execute the action.

[0058] Preferably, the robotic arm grasping module includes a UR10 six-axis robotic arm, a Robotiq two-finger gripper, and a six-axis torque sensor. The UR10 six-axis robotic arm and the two-finger gripper are responsible for grasping or pushing the object to be operated, and the six-axis torque sensor is used to measure the force of the two-finger gripper closing.

[0059] Further, the depth camera is installed at the end of the UR10 six-axis robotic arm.

[0060] This embodiment also provides a control method for the robotic arm autonomous control system applicable to occluded object grasping according to the above, including the following steps:

[0061] S1. Initialize the robotic arm to move it to the observation pose and collect RGB color image data and Depth depth image data;

[0062] S2. Observe and recognize the currently collected scene:

[0063] Use the Depth depth image data for foreground segmentation and instance segmentation of the scene

[0064] The method for foreground segmentation of a scene using Depth depth image data is as follows:

[0065] Remove the background behind the scene with depth values less than the threshold ξ using the depth value size condition, and then remove the background around the scene with negative horizontal and vertical gradients of the depth value using the depth value gradient condition to obtain a foreground mask composed of all objects in the scene;

[0066] The method for instance segmentation of a scene using Depth depth image data is as follows: If there are no labeled points in the neighborhood of a pixel, assign a new label to this pixel;

[0067] If there are labeled points in the neighborhood of a pixel, query the depth values of the labeled points in the neighborhood and determine whether there are pixel points with a difference in depth value from the central pixel less than the threshold η;

[0068] If there are labeled points in the neighborhood with a difference in depth value less than the threshold η, take the point with the smallest difference and assign its label to the central pixel; otherwise, assign a new label to the central pixel.

[0069] Use a pre-trained Siamese network to identify the RGB color image data under the segmentation mask to obtain the visible information of the scene objects in matrix form;

[0070] Specifically, the visible information of the scene objects in matrix form, that is, there is a landmark matrix EFM, and the expression is as follows:

[0071]

[0072] Among them, n rows represent n objects and have a fixed order, m columns represent m instance masks of instance segmentation and do not have a fixed order. The value of the i-th row and the j-th column of the matrix is 1, EFM(i,j) = 1, indicating that the j-th segmentation instance is recognized as the i-th object by the Siamese network.

[0073] S3. Use the visible information of the scene objects in matrix form to determine whether the i-th target object is visible, that is, determine whether there is a target object, that is, determine whether there is:

[0074] That is, determine whether there is:

[0075] S31 If the target object is visible, then determine whether it is occluded, that is, determine whether there is a depth value smaller than the instance itself in the neighborhood of the instance mask;

[0076] S311 If the target object is occluded, select the instance closest to the target object as the candidate object to be operated on and determine whether it is occluded;

[0077] If the candidate object is not occluded, the object becomes the object to be operated output by the object selection module and is provided to the action decision module in the form of the centroid coordinates of the instance mask;

[0078] If the candidate object is occluded, the object is removed, and the process of S311 is repeated;

[0079] If the target object is not occluded, the object becomes the object to be operated output by the object selection module and is provided to the robotic arm grasping module in the form of the centroid coordinates of the instance mask

[0080] If the target object is not visible, the depth image of the scene and the binary image of the foreground of the scene are used as the state and input into the Double DQN decision network, and the network outputs the center coordinates of an instance.

[0081] Among them, the reward function reward of the Double DQN decision network is described as follows:

[0082] First, define the horizontal distribution of the foreground mask, that is, compress an n×m matrix M into an m-dimensional row vector V,

[0083]

[0084] And after performing the kth action, there is V k = V ∩ V k-1 , and the intersection operation makes V k become smaller and smaller. Therefore, V k actually represents the unexplored foreground area, and there must be ΣV k ≤ ΣV k-1 ; the calculation of the horizontal distribution I of the object instance is the same as that of V, but there is no need to take the intersection as the action progresses; the reward is obtained from the following formula:

[0085]

[0086] Among them, I is the horizontal distribution of the object to be operated. The above formula indicates that when the selected object is in the unexplored foreground, the selection of the object is considered valid; while when the selected object is in the explored foreground, the selection of the object is considered a failure.

[0087] S4. According to the centroid coordinates of the object to be operated, a local depth image is intercepted with the object as the center, combined with the binary image of the instance mask of the object, and input into the Double DQN decision network. The network outputs a direction for the robotic arm to push the object, and the centroid coordinates are passed to the robotic arm continuously, and decision information is given according to the reward function;

[0088] Among them, its reward function reward is described as follows:

[0089]

[0090] The above formula indicates that when the pushing action reduces the unexplored foreground area in the scene, the decision-making pushing action is considered effective; while when the pushing action cannot change the distribution of the unexplored foreground, the decision-making pushing action is considered a failure.

[0091] S5. The robotic arm performs a grasping or pushing action according to the received center-of-gravity coordinate information of the object to be operated;

[0092] When the signal of the center-of-gravity coordinate comes from the object selection module, a grasping action is performed; when the signal of the center-of-gravity coordinate comes from the action decision-making module, a pushing action is performed.

[0093] Specifically, when the information comes from the object selection module, a grasping action is performed; when the signal comes from the action decision-making module, a pushing action is performed;

[0094] The center-of-gravity coordinate of the object instance segmentation mask is the pixel coordinate [x, y]. Using the Depth depth image and combining with the camera internal parameters: the center pixel coordinate [ppx, ppy] and the pixel focal length [fx, fy], the three-dimensional space coordinate coord of this point relative to the camera center point can be obtained cam =[depth(x,y),cam x ,cam y T , where the latter two items can be calculated according to the following formula:

[0095]

[0096] Through the hand-eye calibration of the camera, the hand-eye matrix That is, the transformation matrix between the camera coordinate system and the end-effector coordinate system of the robotic arm. Then, through the TF coordinate system in ROS, the transformation matrix between the camera coordinate system and the world coordinate system can be obtained Therefore, the world coordinate of the center point on the surface of the object to be operated

[0097] After obtaining the world coordinate, through the OMPL algorithm and IK inverse solution algorithm of the MoveIt motion library in ROS, the motion information of the robotic arm in the joint space is obtained. If the planning is successful, the grasping or pushing action is performed. If the planning fails, the algorithm is run again until the planning is successful;

[0098] S6. The robotic arm performs a pushing action to push away the obstacle and search for the target object until the target object is visible and unobstructed; after the target object is visible and unobstructed, the robotic arm grasps the target object, returns and places it on the table.

[0099] ​The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions, and variations to these embodiments including components still fall within the protection scope of the present invention.

Claims

1. A control method for an autonomous control system of a robotic arm applicable to grasping occluded objects, characterized in that, The system includes an image data acquisition module, an object selection module, an action decision-making module, and a robotic arm grasping module; The image data acquisition module includes: A Realsense D435 depth camera for acquiring RGB color image data and Depth depth image data; An object selection module for inputting the Depth depth image of the scene and the scene foreground segmentation mask of the Depth depth image, and using the Double DQN reinforcement learning algorithm to select the object to be interacted with; An action decision-making module for inputting the local Depth depth image and the mask of the object to be interacted with in the local Depth depth image, and using the Double DQN reinforcement learning algorithm to determine the action to be executed by the robotic arm; A robotic arm grasping module for performing actions on the object to be operated; The specific steps are as follows: S1. Initialize the robotic arm to move it to the observation pose and acquire RGB color image data and Depth depth image data; S2. Observe and identify the currently acquired scene: Use the Depth depth image data for foreground segmentation and instance segmentation of the scene; The method for using the Depth depth image data for foreground segmentation of the scene is: Remove the background behind the scene with a depth value less than the threshold ξ using the depth value size condition, and then remove the background around the scene with negative horizontal and vertical gradients of the depth value using the depth value gradient condition to obtain the foreground mask composed of all objects in the scene; The method for using the Depth depth image data for instance segmentation of the scene is: If there is no marked point in the neighborhood of a pixel point, assign a new label to this pixel point; If there is a marked point in the neighborhood of a pixel point, query the depth values of the marked points in the neighborhood and determine whether there is a pixel point with a difference in depth value from the central pixel less than the threshold η; If there is a marked point in the neighborhood with a depth value difference less than the threshold η, take the point with the smallest difference and assign its marked label to the central pixel point; otherwise, assign a new label to the central pixel point; Use the pre-trained Siamese network to identify the RGB color image data under the segmentation mask to obtain the visible information of the scene objects in matrix form, that is, there is a landmark matrix EFM, and the expression is as follows: Among them, n rows represent n objects and have a fixed order, m columns represent m instance segmentation masks, which do not have a fixed order. The value of the i-th row and the j-th column of the matrix is 1, EFM(i,j)=1, indicating that the j-th segmentation instance is recognized as the i-th object by the Siamese network; S3. Use the visible information of the scene objects in matrix form to judge whether the i-th target object is visible, that is, judge whether there is a target object: That is, it is judged whether there is: S31 If the target object is visible, judge whether it is occluded, that is, judge whether there is a depth value smaller than the instance itself in the neighborhood of the instance mask; S311 If the target object is occluded, select the instance closest to the target object as the candidate object to be operated and judge whether it is occluded; S3111 If the candidate object is not occluded, then this object becomes the object to be operated output by the object selection module and is provided to the action decision-making module in the form of the centroid coordinates of the instance mask; S3112 If the candidate object is occluded, then this object is removed, and the S311 process is repeated; S312 If the target object is not occluded, then this object becomes the object to be operated output by the object selection module and is provided to the robotic arm grasping module in the form of the centroid coordinates of the instance mask; S32 If the target object is not visible, then the depth image of the scene and the binary image of the foreground of the scene are used as the state and input into the Double DQN decision network, and the network outputs the center coordinates of an instance; Among them, the reward function reward of the Double DQN decision network is described as follows: First, define the horizontal distribution of the foreground mask, that is, compress an n×m matrix M into an m-dimensional row vector V, And after performing the k-th action, there is V k = V ∩ V k-1 , and the operation of taking the intersection makes V k become smaller and smaller. Therefore, V k actually represents the unexplored foreground area, and there must be ∑V k ≤ ∑V k-1 ; The calculation of the horizontal distribution I of the object instance is the same as that of V, but there is no need to take the intersection as the action progresses; The reward is obtained from the following formula: where I is the horizontal distribution of the instance of the object to be operated. The above formula indicates that when the selected object is in the unexplored foreground, the selection of the object is considered valid; while when the selected object is in the explored foreground, the selection of the object is considered a failure; S4. According to the centroid coordinates of the object to be operated, a local depth image is intercepted with this object as the center, combined with the binary image of the instance mask of this object, and input into the Double DQN decision network. The network outputs a direction for the robotic arm to push the object, and the centroid coordinates are continuously passed to the robotic arm. At the same time, decision-making information is given according to the reward function; The reward function reward of the Double DQN decision network is described as follows: The above formula indicates that when the pushing action reduces the unexplored foreground area in the scene, the decision-making pushing action is considered valid; while when the pushing action cannot change the distribution of the unexplored foreground, the decision-making pushing action is considered a failure; S5. The robotic arm performs a grasping or pushing action according to the received centroid coordinate information of the object to be operated; S6. The robotic arm performs a pushing action to push away the obstacle and search for the target object until the target object is visible and unoccluded; after the target object is visible and unoccluded, the robotic arm grasps the target object, returns and places it on the table.

2. The control method of a robotic arm autonomous control system applicable to grasping occluded objects according to claim 1, characterized in that, The robotic arm grasping module includes a UR10 six-axis robotic arm, a Robotiq two-finger gripper, and a six-axis torque sensor. The UR10 six-axis robotic arm and the two-finger gripper are responsible for grasping or pushing the object to be operated, and the six-axis torque sensor is used to measure the force for the two-finger gripper to close.

3. The control method of a robotic arm autonomous control system applicable to grasping occluded objects according to claim 2, wherein, The depth camera is installed at the end of the UR10 six-axis robotic arm.

4. The control method of a robotic arm autonomous control system applicable to grasping occluded objects according to claim 3, characterized in that, In step S5, when the signal of the centroid coordinates comes from the object selection module, a grasping action is performed; when the signal of the centroid coordinates comes from the action decision-making module, a pushing action is performed.

Images