Obstacle-avoiding grabbing method and device for mechanical arm

Abstract

The invention relates to the technical field of mechanical arm visual control, in particular to a mechanical arm obstacle avoidance grabbing method and device which comprises an obtaining module, a processing module, a control module, a mechanical arm with mechanical arm joints and connecting rods connected with each other and a clamping jaw installed at the tail end of the mechanical arm. The acquisition module comprises a depth camera used for acquiring target object and obstacle information under a camera coordinate system, and the depth camera is installed beside a clamping jaw at the tail end of the mechanical arm and is orthogonal to the clamping jaw coordinate system; the method has the beneficial effects that the target object and obstacle information is obtained through the depth camera, then the obstacle information is converted to the base coordinate system in consideration of the possibility that the joint connecting rod of the mechanical arm collides with the obstacle in the operation process, and the models of the obstacle and the joint connecting rod of the mechanical arm are established respectively; collision detection is carried out on each planned path point, so that the operation safety is improved; and finally, the position over the target object serves as the middle point of the path, and the tail end grabbing posture of the mechanical arm is determined.

Description

Technical Field

[0001] This invention relates to the field of robotic arm vision control technology, specifically to a robotic arm obstacle avoidance and grasping method and device. Background Technology

[0002] Obstacle avoidance and grasping, as well as planning, are important issues in the field of industrial robotic arms. They generally revolve around a process of perception, modeling, planning, and control, which involves acquiring information about the target object and obstacles, establishing an equivalent model, planning the movement path, and controlling the robotic arm to reach the position and complete the grasping action.

[0003] Existing technical solutions generally do not consider the application scenarios where obstacles exist, and directly issue the target object's location information for grasping operations. The gripping posture of the robotic arm's end effector is usually consistent with its initial state, which can lead to planning failures and unreachable target points. In addition, in some obstacle avoidance scenarios, only the obstacle is modeled without considering the joints and links of the robotic arm, and only collision detection between the obstacle and the end effector is performed. This can result in collisions between the intermediate links and joints and the obstacle during joint rotation, damaging the robotic arm and causing grasping failure.

[0004] Therefore, considering the use scenarios with obstacles for grasping operations, this invention provides a robotic arm obstacle avoidance grasping method and device, which can model obstacles and the joints of the robotic arm itself in a base coordinate system to ensure the safety of the robotic arm during movement. At the same time, the grasping posture of the robotic arm end is determined by taking the top of the target object as the midpoint of the path, thereby improving the grasping success rate. Summary of the Invention

[0005] To address the problems in existing technologies, this invention proposes a robotic arm obstacle avoidance and grasping method and device, which solves the problem of low grasping success rate in existing robotic arms.

[0006] The technical solution adopted by this invention to solve its technical problem is: a robotic arm obstacle avoidance and grasping method, the method comprising:

[0007] The target object information in the camera coordinate system is obtained using a depth camera. Position: T c x, T c y、T c z, Length, Width, Height: T c l, T c w、T c h;

[0008] Obstacle information in the camera coordinate system is obtained using a depth camera. Location: O c x、O c y、O c z, Length, width and height: O c l、O c w、O ch;

[0009] If the target object exists, perform information transformation from the camera coordinate system (c-frame) to the base coordinate system (b-frame);

[0010] If the obstacle exists, perform information transformation from the camera coordinate system to the base coordinate system, and establish an equivalent model in the base coordinate system;

[0011] Model each joint link based on the described robotic arm structural model;

[0012] Based on the collision detection results, and considering the position of the robotic arm end and the position of the target object, the gripper is controlled to stop directly above the target object before it moves down to perform a gripping operation on the target object.

[0013] For the target object, perform information transformation from the camera coordinate system to the base coordinate system, including:

[0014] If the target object exists, first transform it from the camera coordinate system to the gripper coordinate system (g system):

[0015]

[0016] In the formula, T g x, T g y、T g z represents the position of the target object in the gripper coordinate system, and T represents the position of the target object in the gripper coordinate system. g l, T g w、T g h represents the length, width, and height of the obstacle in the gripper coordinate system, respectively. This is the homogeneous matrix from the camera coordinate system to the gripper coordinate system;

[0017] Then transform from the gripper coordinate system to the base coordinate system:

[0018]

[0019] In the formula, T b x, T b y、T b z represents the position of the target object in the base coordinate system, and T represents the position of the target object in the base coordinate system. b l, T b w、T b h represents the length, width, and height of the obstacle in the base coordinate system. is the homogeneous matrix from the gripper coordinate system to the base coordinate system.

[0020] Specifically, if the obstacle exists, the process involves information transformation from the camera coordinate system to the base coordinate system and the establishment of an equivalent model in the base coordinate system, including:

[0021] If the obstacle exists, first transform it from the camera coordinate system to the gripper coordinate system:

[0022]

[0023] In the formula, O g x、O g y、O g z represents the position of the obstacle in the gripper coordinate system, O g l、O g w、O g h represents the length, width, and height of the obstacle in the gripper coordinate system;

[0024] Then transform from the gripper coordinate system to the base coordinate system:

[0025]

[0026] In the formula, O b x、O b y、O b z represents the position of the obstacle in the base coordinate system, and O represents the position of the obstacle in the base coordinate system. b l、O b w、O b h represents the length, width, and height of the obstacle in the base coordinate system.

[0027] Specifically, the homogeneous matrix from the camera coordinate system to the gripper coordinate system Obtain it through the following methods:

[0028] Determine the camera installation location, obtain 3D models of the camera, gripper, and robotic arm, and customize the connecting rod between the camera and gripper;

[0029] Ensure that the gripper coordinate system and the camera coordinate system are orthogonally installed.

[0030] The homogeneous matrix is ​​determined based on the lengths of each link of the robotic arm, the location of the joint centers, the xyz rotation directions, and the distance between the camera coordinate center and the end effector of the robotic arm.

[0031] Specifically, determine the position of the beginning and end of each joint in the base coordinate system, the joint radius, and establish a cylindrical model;

[0032] Determine the positions of the beginning and end of each link in the base coordinate system, as well as the link length, width, and thickness, and establish a cube model.

[0033] Specifically, the collision detection includes:

[0034] Collision detection between each link and an obstacle;

[0035] Collision detection between links.

[0036] Specifically, based on the position of the robotic arm's end effector and the target object's position, the gripper is controlled to stop directly above the target object before descending to grasp it, including:

[0037] Based on the position of the robotic arm's end effector and the position of the target object, a movement path is generated to move directly above the target object.

[0038] Collision detection is performed on each path point within the joint space. If the shortest distance between different links and obstacles is less than 5 mm, or the target angle that a certain joint needs to reach exceeds its maximum limit angle, a new movement path is generated based on the position of the robotic arm end and the position of the target object, and the robotic arm is controlled to move along the new path to directly above the target object. If a collision occurs in two consecutive planning attempts, the target point is reported as unreachable.

[0039] Once it reaches directly above the target, it issues a downward gripping command to grab the target.

[0040] Specifically, based on the position of the robotic arm's end effector and the position of the target object, a movement path is generated directly above the target object, including:

[0041] Target information in the base coordinate system includes location: T b x, T b y、T b z, Length, Width, Height: T b l, T b w、T b h, then the position directly above the target object is The orientation is 0, π, 0 (representing a 180° rotation around the y-axis of the base coordinate system);

[0042] After reaching directly above the target, issue a downward probe command at position T. b x, T b y、T b z, attitude 0, π, 0, while simultaneously detecting the state of the robotic arm's end effector in real time, and performing a grasping operation after reaching the downward probe position.

[0043] A robotic arm obstacle avoidance and grasping device includes an acquisition module, a processing module, a control module, a robotic arm with joints and links connected to each other, and a gripper installed at the end of the robotic arm.

[0044] The acquisition module includes a depth camera, used to acquire target and obstacle information in the camera coordinate system. The depth camera is installed next to the gripper at the end of the robotic arm and is orthogonal to the gripper coordinate system.

[0045] The processing module, including an industrial computer, is used for:

[0046] The camera coordinate system information is transformed to the base coordinate system through a homogeneous transformation matrix, which is determined based on the link length of the robotic arm joint, the center position of the robotic arm joint, and the relative position of the camera and the gripper.

[0047] Establish a cube model of the obstacle and a base coordinate system model of the robotic arm joints and links;

[0048] The control module, including an industrial computer processor, is used for:

[0049] Based on the real-time angles of the robotic arm joints and the structural dimensions of the links, a movement path is generated for the gripper to move directly above the target object.

[0050] Collision detection is performed on links and robotic arm joints at each path point; if a collision is found, the path is replanned.

[0051] The control system coordinates the rotation of the robotic arm joints to drive the linkage, causing the gripper to descend to the target object position (Xb, Yb, Zb) to perform grasping. The gripper is installed at the end of the adjacent robotic arm joint, and its posture is determined by the rotation angle of the adjacent robotic arm joint.

[0052] The beneficial effects of this invention are as follows: The obstacle avoidance and grasping method and device of this invention first acquires information about the target object and obstacles using a depth camera. Then, considering the possibility of collisions between the robotic arm's joints and links and obstacles during operation, the obstacle information is transformed into a base coordinate system, and models of the obstacles and the robotic arm's joints and links are established respectively. Collision detection is performed at each planned path point, improving operational safety. Finally, the position directly above the target object is used as the midpoint of the path to determine the grasping posture of the robotic arm's end effector, improving the grasping success rate. This solves the problems in the prior art where collisions occur during operation, the target point is unreachable, and grasping fails because the prior art does not consider obstacles, only considers collision detection between the robotic arm's end effector and obstacles, or the grasping posture of the robotic arm's end effector is consistent with the initial state. Attached Figure Description

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] Figure 1 A schematic diagram of the process module structure of the industrial control computer for a robotic arm obstacle avoidance and grasping method and device provided by the present invention;

[0055] Figure 2 A schematic diagram of the obstacle avoidance and grasping process module structure of a robotic arm in a robotic arm obstacle avoidance and grasping method and device provided by the present invention;

[0056] Figure 3 A schematic diagram of the orthogonal mounting structure of the gripper and camera in a robotic arm obstacle avoidance and grasping method and device provided by the present invention;

[0057] Figure 4 This invention provides a schematic diagram of the three-dimensional model structure of the robotic arm body in a robotic arm obstacle avoidance and grasping method and device.

[0058] Figure 5A schematic diagram of the connection structure of a section of the robotic arm joint and connecting rod in a robotic arm obstacle avoidance and grasping method and device provided by the present invention;

[0059] Figure 6 A schematic diagram of the mechanical arm joint and linkage connection structure of another section of the robotic arm obstacle avoidance and grasping method and device provided by the present invention;

[0060] Figure 7 This is a schematic diagram of the robotic arm body modeling process structure in the obstacle avoidance and grasping method and device provided by the present invention.

[0061] In the diagram: 1. Robotic arm joint; 2. Linkage; 3. Gripper. Detailed Implementation

[0062] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0063] like Figures 1-7 As shown, the present invention provides the following technical solution:

[0064] A robotic arm obstacle avoidance and grasping method, the method comprising:

[0065] The target object information in the camera coordinate system is obtained using a depth camera. Position: T c x, T c y、T c z, Length, Width, Height: T c l, T c w、T c h;

[0066] Obstacle information in the camera coordinate system is obtained using a depth camera. Location: O c x、O c y、O c z, Length, width and height: O c l、O c w、O c h;

[0067] If the target object exists, perform information transformation from the camera coordinate system (c-frame) to the base coordinate system (b-frame);

[0068] If the obstacle exists, perform information transformation from the camera coordinate system to the base coordinate system, and establish an equivalent model in the base coordinate system;

[0069] Model each joint link based on the described robotic arm structural model;

[0070] Based on the collision detection results, and considering the position of the robotic arm end and the position of the target object, the gripper is controlled to stop directly above the target object before it moves down to perform a gripping operation on the target object.

[0071] For the target object, perform information transformation from the camera coordinate system to the base coordinate system, including:

[0072] If the target object exists, first transform it from the camera coordinate system to the gripper coordinate system (g system):

[0073]

[0074] In the formula, T g x, T g y、T g z represents the position of the target object in the gripper coordinate system, and T represents the position of the target object in the gripper coordinate system. g l, T g w、T g h represents the length, width, and height of the obstacle in the gripper coordinate system, respectively. It is the homogeneous matrix from the camera coordinate system to the gripper coordinate system.

[0075] Then transform from the gripper coordinate system to the base coordinate system:

[0076]

[0077] In the formula, T b x, T b y、T b z represents the position of the target object in the base coordinate system, and T represents the position of the target object in the base coordinate system. b l, T b w、T b h represents the length, width, and height of the obstacle in the base coordinate system. This is the homogeneous matrix from the gripper coordinate system to the base coordinate system;

[0078] If the obstacle exists, perform information transformation from the camera coordinate system to the base coordinate system and establish an equivalent model in the base coordinate system, including:

[0079] If the obstacle exists, first transform it from the camera coordinate system to the gripper coordinate system:

[0080]

[0081] In the formula, O g x、O g y、O g z represents the position of the obstacle in the gripper coordinate system, O g l、O g w、O g h represents the length, width, and height of the obstacle in the gripper coordinate system;

[0082] Then transform from the gripper coordinate system to the base coordinate system:

[0083]

[0084] In the formula, O b x、O b y、O b z represents the position of the obstacle in the base coordinate system, and O represents the position of the obstacle in the base coordinate system. b l、O b w、O b h represents the length, width, and height of the obstacle in the base coordinate system;

[0085] Homogeneous matrix from camera coordinate system to gripper coordinate system Obtain it through the following methods:

[0086] Determine the camera installation location, obtain 3D models of the camera, gripper, and robotic arm, and customize the connecting rod between the camera and gripper;

[0087] Ensure that the gripper coordinate system and the camera coordinate system are orthogonally installed.

[0088] The homogeneous matrix is ​​determined based on the lengths of each link of the robotic arm, the location of the joint centers, the xyz rotation directions, and the distance between the camera coordinate center and the end effector of the robotic arm.

[0089] Determine the positions of the beginning and end of each joint in the base coordinate system and the joint radius, and establish a cylindrical model;

[0090] Determine the positions of the beginning and end of each link in the base coordinate system, the link length, the link width, and the link thickness, and establish a cube model;

[0091] The collision detection includes: collision detection between each link and an obstacle; and collision detection between links.

[0092] Based on the position of the robotic arm's end effector and the target object's position, the gripper is controlled to stop directly above the target object before descending to grasp it, including:

[0093] Based on the position of the robotic arm's end effector and the position of the target object, a movement path is generated to move directly above the target object.

[0094] Collision detection is performed on each path point within the joint space. If the shortest distance between different links and obstacles is less than 5 mm, or the target angle that a certain joint needs to reach exceeds its maximum limit angle, a new movement path is generated based on the position of the robotic arm end and the position of the target object, and the robotic arm is controlled to move along the new path to directly above the target object. If a collision occurs in two consecutive planning attempts, the target point is reported as unreachable.

[0095] Once it reaches directly above the target, it issues a downward gripping command to grab the target.

[0096] Based on the position of the robotic arm's end effector and the target object's position, a movement path is generated directly above the target object, including:

[0097] Target information in the base coordinate system includes location: T b x, T b y、T b z, Length, Width, Height: T b l, T b w、T b h, then the position directly above the target object is The orientation is 0, π, 0 (representing a 180° rotation around the y-axis of the base coordinate system);

[0098] After reaching directly above the target, issue a downward probe command at position T. b x, T b y、T b z, attitude 0, π, 0, while simultaneously detecting the state of the robotic arm's end effector in real time, and performing a grasping operation after reaching the downward probe position.

[0099] A robotic arm obstacle avoidance and grasping device includes an acquisition module, a processing module, a control module, a robotic arm with joint 1 and connecting rod 2 connected to each other, and a gripper 3 installed at the end of the robotic arm.

[0100] The acquisition module includes a depth camera, used to acquire information about the target object and obstacles in the camera coordinate system. The depth camera is installed next to the end gripper 3 of the robotic arm and is orthogonal to the coordinate system of the gripper 3.

[0101] The processing module, including an industrial computer, is used for:

[0102] The camera coordinate system information is transformed to the base coordinate system through a homogeneous transformation matrix. The homogeneous transformation matrix is ​​determined based on the length of the link 2 of the robotic arm joint 1, the center position of the robotic arm joint 1, and the relative position of the camera and the gripper 3.

[0103] Establish a cube model of the obstacle and a base coordinate system model of the robotic arm joint 1 and link 2;

[0104] The control module, including an industrial computer processor, is used for:

[0105] Based on the real-time angle of the robotic arm joint 1 and the structural dimensions of the link 2, a movement path is generated from the gripper 3 to directly above the target object.

[0106] Collision detection is performed on link 2 and robotic arm joint 1 at each path point. If a collision occurs, the path is replanned.

[0107] The control robot arm joint 1 rotates in coordination with the drive link 2, causing the gripper 3 to descend to the target object position (Xb, Yb, Zb) to perform grasping. The gripper 3 is installed at the end of the adjacent robot arm joint 1, and its posture is determined by the rotation angle of the adjacent robot arm joint 1.

[0108] When using, refer to Figure 1 The diagram shown is a schematic diagram of the process module structure of an industrial control computer for a robotic arm obstacle avoidance and grasping method and device provided in an embodiment of this application; as shown... Figure 1 As shown, the robotic arm grasping application scenarios include: industrial control computer, robotic arm and depth camera;

[0109] In dynamic field-of-view grasping scenarios, the depth camera can be fixedly mounted on the end effector of the robotic arm, meaning that the position of the depth camera changes as the robotic arm moves.

[0110] Industrial control computers can be terminal devices with data processing capabilities, such as personal computers, laptops, NX boards, and tablets.

[0111] The robotic arm can be a Ruiman series six-axis robotic arm, and the end effector of the robotic arm is a time-sensitive gripper 3. The target object to be grasped can be common daily necessities, such as beverage bottles, shower gel, plastic boxes, etc.

[0112] The depth camera can be an Orbbec medium-light camera, which can obtain distance information between the surrounding environment and the depth camera, i.e., depth information.

[0113] The industrial control computer is connected to the robotic arm and the depth camera. The depth camera is used to collect information on the target object and obstacles in the grasping area and send the collected data to the industrial control computer. The industrial control computer performs coordinate system transformation and modeling on the data, and performs obstacle avoidance path planning to obtain a movement path. It controls the robotic arm to grasp the target object along the planned movement trajectory to achieve accurate grasping of the target object.

[0114] The following will explain the implementation principle and corresponding beneficial effects of the robotic arm obstacle avoidance and grasping method provided in this application, using a scenario with both obstacles and a target object.

[0115] by Figure 1 The industrial computer shown is the execution unit. In this embodiment, the processor uses an NX board. (Refer to...) Figure 2 As shown, a robotic arm grasping method is provided, the method comprising:

[0116] The target object information in the camera coordinate system is obtained using a depth camera. Position: T c x, T c y、T c z, Length, Width, Height: T c l, Tc w、T c h; Obstacle information, location: O c x、O c y、O c z, Length, width and height: O c l、O c w、O c h is transmitted to the processor via the bus.

[0117] Figure 3 The diagram shows the orthogonal installation of the gripper and camera. The position of the camera coordinate system origin in the gripper's 3-coordinate system is: The camera coordinate system can be obtained by rotating the camera coordinate system counterclockwise by 90° around the Z-axis, and therefore the homogeneous matrix can be determined. for:

[0118]

[0119] Will By pre-writing the information to the processor and combining it with the information from the depth camera, the following coordinate transformation can be completed.

[0120] Transform the target object from the camera coordinate system to the gripper 3 coordinate system:

[0121]

[0122] In the formula, T g x, T g y、T g z represents the position of the target object in the gripper's 3-coordinate system, and T represents the position of the target object in the gripper's 3-coordinate system. g l, T g w、T g h represents the length, width, and height of the obstacle in the gripper's 3-coordinate system.

[0123] Transform the obstacle from the camera coordinate system to the gripper 3 coordinate system:

[0124]

[0125] In the formula, O g x、O g y、O g z represents the position of the obstacle in the gripper's 3-coordinate system, O g l、O g w、O g h represents the length, width, and height of the obstacle in the gripper's 3-coordinate system.

[0126] Figure 4This is a schematic diagram of a three-dimensional model of the robotic arm body provided in an embodiment of the present invention. The robotic arm includes 6 robotic arm joints 1 and 5 links 2. The base coordinate system coincides with the coordinate system of the robotic arm base. According to the robotic arm DH parameter table and the real-time angle of the robotic arm joint 1 (which is transmitted back to the processor by the robotic arm in real time), the homogeneous matrix from the gripper 3 coordinate system to the base coordinate system can be obtained by multiplying the transfer matrix.

[0127]

[0128] In the formula, This represents the transfer matrix for transforming from coordinate system A to coordinate system B.

[0129] Next, the target object is transformed from the gripper's 3-coordinate system to the base coordinate system:

[0130]

[0131] In the formula, T b x, T b y、T b z represents the position of the target object in the base coordinate system, and T represents the position of the target object in the base coordinate system. b l, T b w、T b h represents the length, width, and height of the obstacle in the base coordinate system.

[0132] Transform the obstacle from the gripper's 3D coordinate system to the base coordinate system:

[0133]

[0134] In the formula, O b x、O b y、O b z represents the position of the obstacle in the base coordinate system, and O represents the position of the obstacle in the base coordinate system. b l、O b w、O b h represents the length, width, and height of the obstacle in the base coordinate system.

[0135] By combining the position, length, width, and height of the obstacle in the base coordinate system, its cube model in the base coordinate system can be established for subsequent collision detection.

[0136] Figure 5 This is a schematic diagram of the connection structure of one section of the robotic arm joint and connecting rod provided in an embodiment of the present invention. For robotic arm joint 1, its center position, radius, and height are determined. For connecting rod 2, its center position, length, width, and height are determined. The following modeling is performed using one section of connected robotic arm joint 1 and connecting rod 2 as an example:

[0137] Robotic arm joint 1:

[0138] Find the coordinate system closest to its center position, which is coordinate system 4, and its center position coincides with the origin of coordinate system 4, that is, its center position is available in coordinate system 4.

[000] T This means that, combined with the real-time angle of joint 1 of the robotic arm, it can be expressed in the base coordinate system as follows:

[0139]

[0140] By combining the structural dimensions of the robotic arm joint 1, a cylindrical model of the robotic arm joint 1 in the base coordinate system can be established.

[0141] Link 2:

[0142] Find the coordinate system closest to its center position, let's call it coordinate system 3, and combine it with... Figure 6 Based on the structural dimensions of link 2 (measured data from actual conditions), its center position in coordinate system 3 can be determined as [xyz]. T =[0 55.25-43] T Based on the real-time angle of joint 1 of the robotic arm, its center position in the base coordinate system can be expressed as:

[0143]

[0144] The cube model of link 2 can be established by adding the length, width, height, and center position in the base coordinate system;

[0145] The remaining robotic arm joints 1 and 2 can all be modeled as cylinders and cubes using this method.

[0146] After the processor has created the obstacle model and the models of robotic arm joint 1 and link 2, it takes the current position of the robotic arm's end effector as the starting point and the position directly above the target object. With the orientation 0, π, 0 (representing a 180° rotation around the y-axis of the base coordinate system), a movement path is generated to be directly above the target. Collision detection is performed at each path point, including collision detection between each link 2 and obstacles, and collision detection between each link 2. If a collision occurs at a path point, the movement path is regenerated based on the position of the robotic arm end and the position of the target, and the robotic arm is controlled to move along the new path to be directly above the target.

[0147] After reaching directly above the target, a further downward probe command is issued at position T. b x, T b y、T b z, posture π, 0, 0, while simultaneously detecting the state of the robotic arm's end effector in real time, and performing a grasping operation after reaching the downward probe position.

[0148] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of protection claimed by the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A robotic arm obstacle avoidance and grasping method, characterized in that: The method includes: The target object information in the camera coordinate system is obtained using a depth camera. Position: T c x, T c y、T c z, Length, width and height: T c l, T c w、T c h; Obstacle information in the camera coordinate system is obtained using a depth camera. Location: O c x、O c y、O c z, Length, width and height: O c l、O c w、O c h; If the target object exists, perform information transformation from the camera coordinate system (c-frame) to the base coordinate system (b-frame); If the obstacle exists, perform information transformation from the camera coordinate system to the base coordinate system, and establish an equivalent model in the base coordinate system; Model each joint link based on the described robotic arm structural model; Based on the collision detection results, and considering the position of the robotic arm end and the position of the target object, the gripper is controlled to stop directly above the target object before it moves down to perform a gripping operation on the target object. For the target object, perform information transformation from the camera coordinate system to the base coordinate system, including: If the target object exists, first transform it from the camera coordinate system to the gripper coordinate system (g system): In the formula, T g x, T g y、T g z represents the position of the target object in the gripper coordinate system, and T represents the position of the target object in the gripper coordinate system. g l, T g w、T g h represents the length, width, and height of the obstacle in the gripper coordinate system, respectively. It is the homogeneous matrix from the camera coordinate system to the gripper coordinate system. Then transform from the gripper coordinate system to the base coordinate system: In the formula, T b x, T b y、T b z represents the position of the target object in the base coordinate system, and T represents the position of the target object in the base coordinate system. b l, T b w、T b h represents the length, width, and height of the obstacle in the base coordinate system. is the homogeneous matrix from the gripper coordinate system to the base coordinate system.

2. The obstacle avoidance and grasping method for a robotic arm according to claim 1, characterized in that: If the obstacle exists, perform information transformation from the camera coordinate system to the base coordinate system and establish an equivalent model in the base coordinate system, including: If the obstacle exists, first transform it from the camera coordinate system to the gripper coordinate system: In the formula, O g x、O g y、O g z represents the position of the obstacle in the gripper coordinate system, O g l、O g w、O g h represents the length, width, and height of the obstacle in the gripper coordinate system, respectively; Then transform from the gripper coordinate system to the base coordinate system: In the formula, O b x、O b y、O b z represents the position of the obstacle in the base coordinate system, O b l、O b w、O b h represents the length, width, and height of the obstacle in the base coordinate system.

3. The obstacle avoidance and grasping method for a robotic arm according to claims 1 and 2, characterized in that: Homogeneous matrix from camera coordinate system to gripper coordinate system Obtain it through the following methods: Determine the camera installation location, obtain 3D models of the camera, gripper, and robotic arm, and customize the connecting rod between the camera and gripper; Ensure that the gripper coordinate system and the camera coordinate system are orthogonally installed. The homogeneous matrix is ​​determined based on the lengths of each link of the robotic arm, the location of the joint centers, the xyz rotation directions, and the distance between the camera coordinate center and the end effector of the robotic arm.

4. The obstacle avoidance and grasping method for a robotic arm according to claim 1, characterized in that: Determine the positions of the beginning and end of each joint in the base coordinate system and the joint radius, and establish a cylindrical model; Determine the positions of the beginning and end of each link in the base coordinate system, as well as the link length, width, and thickness, and establish a cube model.

5. The obstacle avoidance and grasping method for a robotic arm according to claim 1, characterized in that: The collision detection includes: Collision detection between each link and an obstacle; Collision detection between links.

6. The obstacle avoidance and grasping method for a robotic arm according to claim 1, characterized in that: Based on the position of the robotic arm's end effector and the target object's position, the gripper is controlled to stop directly above the target object before descending to grasp it, including: Based on the position of the robotic arm's end effector and the position of the target object, a movement path is generated to move directly above the target object. Collision detection is performed on each path point within the joint space. If the shortest distance between different links and obstacles is less than 5 mm, or the target angle that a certain joint needs to reach exceeds its maximum limit angle, a new movement path is generated based on the position of the robotic arm end and the position of the target object, and the robotic arm is controlled to move along the new path to directly above the target object. If a collision occurs in two consecutive planning attempts, the target point is reported as unreachable. Once it reaches directly above the target, it issues a downward gripping command to grab the target.

7. The obstacle avoidance and grasping method for a robotic arm according to claim 1, characterized in that: Based on the position of the robotic arm's end effector and the target object's position, a movement path is generated directly above the target object, including: Target information in the base coordinate system includes location: T b x, T b y、T b z, Length, width and height: T b l, T b w、T b h, then the position directly above the target is T. b x, T b y、 The orientation is 0, π, 0 (representing a 180° rotation around the y-axis of the base coordinate system); After reaching directly above the target, issue a downward probe command at position T. b x, T b y、T b z, attitude 0, π, 0, while simultaneously detecting the state of the robotic arm's end effector in real time, and performing a grasping operation after reaching the downward position.

8. A robotic arm obstacle avoidance and grasping device, characterized in that: It includes an acquisition module, a processing module, a control module, a robotic arm with interconnected joints 1 and links 2, and a gripper 3 installed at the end of the robotic arm; The acquisition module includes a depth camera, used to acquire information about the target object and obstacles in the camera coordinate system. The depth camera is installed next to the end gripper 3 of the robotic arm and is orthogonal to the coordinate system of the gripper 3. The processing module, including an industrial computer, is used for: The camera coordinate system information is transformed to the base coordinate system through a homogeneous transformation matrix. The homogeneous transformation matrix is ​​determined based on the length of the link 2 of the robotic arm joint 1, the center position of the robotic arm joint 1, and the relative position of the camera and the gripper 3. Establish a cube model of the obstacle and a base coordinate system model of the robotic arm joint 1 and link 2; The control module, including an industrial computer processor, is used for: Based on the real-time angle of the robotic arm joint 1 and the structural dimensions of the link 2, a movement path is generated from the gripper 3 to directly above the target object. Collision detection is performed on link 2 and robotic arm joint 1 at each path point. If a collision occurs, the path is replanned. The control robot arm joint 1 rotates in coordination with the drive link 2, causing the gripper 3 to descend to the target object position (Xb, Yb, Zb) to perform grasping. The gripper 3 is installed at the end of the adjacent robot arm joint 1, and its posture is determined by the rotation angle of the adjacent robot arm joint 1.

Images