Intelligent grabbing method of humanoid double-arm robot

Abstract

The invention belongs to the technical field of humanoid double-arm robots, and particularly relates to an intelligent grabbing method of a humanoid double-arm robot, which comprises the following steps: acquiring an RGB image and a depth image of a target object through a global camera, performing alignment processing, detecting the target object by using a coarse positioning model, and determining pre-grabbing positions of double arms; global point cloud data in a working space are obtained in real time through a global camera, and path planning is performed by adopting an improved artificial potential field algorithm, so that two arms respectively reach pre-grabbing positions; acquiring an RGB image and a depth image of a target object through a double-arm tail end camera, performing alignment processing, detecting the RGB image of a target camera by using a fine positioning model, and finding a feasible target grabbing attitude by using a pre-trained 6D grabbing attitude prediction model; and the two arms are controlled to execute the grabbing action according to the target grabbing posture and placed at the preset position. The method can adapt to various complex scenes, and diversified grabbing is achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of humanoid dual-arm robots, and in particular relates to an intelligent grasping method of a humanoid dual-arm robot. Background Art

[0002] In the field of modern automation and robotics, the demand for intelligent grasping technology is growing, especially in the application scenarios of object sorting in complex environments. However, existing single-arm robots often face problems such as low operating efficiency, insufficient flexibility and high risk of collision when handling multi-object grasping tasks. These technical bottlenecks make it particularly difficult to achieve efficient and accurate object grasping in dynamic environments. The introduction of a dual-arm system can significantly improve the flexibility and efficiency of grasping tasks. The collaborative work of the two arms can grasp objects at the same time, reduce operation time, and show greater adaptability when handling complex object layouts. However, the motion coordination of the two arms also brings additional challenges, especially in dynamic environments. How to effectively avoid collisions between the two arms and interference with other dynamic obstacles in the environment has become a key issue for intelligent grasping systems. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and to provide an intelligent grasping method for a humanoid dual-arm robot. Through a phased coarse positioning and fine positioning method, the system can adapt to various complex scenarios and achieve diversified grasping.

[0004] Technical solution: The intelligent grasping method of the humanoid dual-arm robot of the present invention comprises the following steps: S1, target coarse positioning stage: obtain the RGB image and depth image of the target object through the global camera and perform alignment processing; use the coarse positioning model to detect the RGB image of the target object, extract the coarse positioning information of the target object, calculate the point cloud data of the coarse positioning information based on the alignment processing, calculate the point cloud centroid and point normal vector of the target object based on the point cloud data and convert them to the robot world coordinate system, and determine the pre-grasping position of the two arms; S2, double-arm motion stage: construct the kinematic model of the double arms and envelop the double-arm joints; obtain the global point cloud data in the workspace in real time through the global camera, pre-process the global point cloud data, and retain the obstacle point cloud; combine the kinematic model and coordinate transformation to calculate the distance between the double-arm joints and the obstacle point cloud in real time; calculate the repulsive field and gravitational field in the joint space, and use the improved artificial potential field algorithm for path planning so that the double arms can reach the pre-grasping position respectively; S3, target precise positioning stage: when the two arms reach the pre-grasping position, the RGB image and depth image of the target object are obtained through the cameras at the end of the two arms and aligned, the RGB image of the target object is detected using the precise positioning model, the precise positioning information of the target object is extracted, and the point cloud data of the precise positioning information is calculated based on the alignment process; the RGB image and depth image processed by the precise positioning model are input into the pre-trained 6D grasping posture prediction model to generate an unordered grasping posture sequence, the grasping posture with the highest score is selected, and the selected grasping posture is inversely kinematically solved. If there is no solution, the posture with the second highest score is selected until a feasible target grasping posture is found; S4, object placement stage: control the arms to perform the grasping action according to the target grasping posture and place it at the preset position; determine whether there is an ungrasped target object. If so, repeat S1-S3 until all grasping tasks are completed.

[0005] To further improve the above technical solution, the target rough positioning stage obtains the pre-grasping position of the target object through the following steps: The RGB image and depth image of the target object are acquired through the global camera, and the RGB image and the depth image are aligned to obtain the depth value corresponding to each RGB pixel in the RGB image and generate point cloud data; Use the coarse positioning model to detect targets in RGB images and obtain the coarse prediction box of the target object; Extracting point cloud data of the corresponding area in the depth image according to the coarse prediction frame; Perform central processing on the point cloud data to obtain the point cloud centroid and point cloud normal vector of the point cloud data; The point cloud normal vector and the center of the rough prediction box are converted to the robot world coordinate system, and the pre-grasping position of the target object is determined by offsetting a preset distance along the direction of the point cloud normal vector; The joint angles corresponding to the pre-grasping position are obtained through inverse kinematics calculation of the robot arm.

[0006] Further, The rough prediction box of a target object is expressed as: ,in, and Respectively represent the upper left corner coordinates and lower right corner coordinates of the rough prediction box; The point cloud data of the area corresponding to the coarse prediction box in the depth image It is expressed as: ; Centralize the point cloud data to obtain the corresponding point cloud centroid and centralized point cloud data , expressed as: , , calculate the centralized point cloud data The covariance matrix of , the calculation formula is: ,in, is the transpose of the centralized point cloud data, solving the covariance matrix The eigenvalue of and the eigenvector , the calculation formula is: , assuming that the eigenvector corresponding to the largest eigenvalue is , then the feature vector is the normal vector of the point cloud data ; The point cloud normal vector and the center of the rough prediction box The conversion formula from the camera coordinate system to the robot world coordinate system is: , ,in, express Coordinate system to The rotation transformation matrix of the coordinate system, express Coordinate system to The offset vector of the coordinate system; Set the rough positioning result as the pre-grasping position of the robot arm , the pre-grab position is through the prediction box center Along the point cloud normal vector The calculation formula is as follows: ; Get the pre-fetch position Finally, the corresponding joint angles are calculated using the inverse kinematics of the robotic arm: .

[0007] Furthermore, the double-arm movement phase includes: Based on the kinematic model of the two arms, the structure of the robot arm is simplified, the joint points of the robot arm are determined, virtual joint points are set at the center of the connecting rod between the adjacent joints of the robot arm, and a spherical bounding box is used to envelop each joint point to obtain a spherical envelope of the two arms; The global point cloud data is acquired in real time through the global camera, and the global point cloud data is processed, including: using straight-through filtering to limit the point cloud data to the workspace of the dual-arm robot; using voxel downsampling to reduce the density of the point cloud data; removing the point cloud data of the robot body through conditional filtering, and only retaining the obstacle point cloud in the workspace; According to the spherical envelope of the arms and the kinematic model of the robotic arm, the position of each joint point of the arms in the world coordinate system is calculated. Through the position of the joint points, the distance between the joint points of the arms and the distance between the obstacle point cloud and the joint points of the arms are calculated; A repulsive field is established based on the distance between the obstacle point cloud and each joint point of the two arms. The Jacobian matrix 𝐽 is used to map the repulsive force from the Cartesian space to the joint space. The gravitational field is calculated based on the difference between the current joint angle of the two arms and the target joint angle. The gravitational field and the repulsive field are synthesized to obtain the total force field acting on the two arms. The joint angles of both arms are updated iteratively through the total force field to obtain the motion paths of both arms.

[0008] Furthermore, for each point in the point cloud data , the straight-through filtering is expressed as: in, and The range set for the filter; The point cloud data of the robot body is removed by conditional filtering, including: The joint position is taken as the center, and the distance threshold is set to filter out points whose distance from the reference point is less than the threshold: .

[0009] Furthermore, the distance between the joints of the two arms is calculated by the following formula: , in, Left arm joints and right arm The position coordinates of each joint in the world coordinate system; The distance between each obstacle point cloud and each joint point of the arms is calculated using the following formula: in, They represent the obstacle point cloud and the two arms The position coordinates of the joint points, The value is L or R, representing the left arm and the right arm respectively; A repulsive field is established based on the calculated distance. The calculation formula of the repulsive field is as follows: in, is the repulsion coefficient, The safety threshold is set; Total repulsive field The calculation formula is: ; The derivation of the repulsion function is: ; Calculate the Jacobian matrix of the robot Map the repulsive force to the joint space: ; Calculate the gravitational field, the calculation formula is as follows: , in, and They are the current joint angle and target joint angle of the robot arm respectively; Update the robot arm joint angles; in, is the iteration step length.

[0010] Beneficial effects: Compared with the prior art, the advantages of the present invention are: the system can adapt to various complex scenes and realize diversified grasping through the staged coarse positioning and fine positioning methods. The coarse positioning stage realizes the precise positioning of the target object in three-dimensional space through point cloud data and feature extraction (such as point cloud centroid and normal vector). This method provides a reliable initial position for subsequent precise grasping; in the fine positioning stage, based on the selected grasping posture, the corresponding robot arm joint angle is calculated through inverse kinematics and solved to ensure that the robot arm can accurately perform the grasping task.

[0011] The improved artificial potential field method effectively avoids collisions between the two arms and conflicts with dynamic obstacles in the environment, ensuring the safety of operation; the gravitational field guides the robotic arm to the target position, while the repulsive field ensures that collisions are avoided during movement. The design of the repulsive field effectively avoids collisions between joints and conflicts with obstacles by setting safety thresholds and using smooth exponential functions; the use of the Jacobian matrix ensures the effective conversion of control force in the joint space, and the iterative update of the joint angle ensures the stability and goal orientation of the movement.

[0012] The dual-arm system can perform grasping tasks simultaneously, significantly shortening working time and improving operational efficiency. It can be applied to dual-arm robot control in complex environments, especially for scenarios that require simultaneous obstacle avoidance and target positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Schematic diagram of the coordinate system of the dual-arm robot in the present invention; Figure 2 It is a schematic diagram of the target rough positioning process in the present invention; Figure 3 It is a schematic diagram of the result of point cloud preprocessing in the present invention; Figure 4 It is a specific flow chart of the double-arm movement stage in the present invention; Figure 5 It is a schematic diagram of obtaining a precise prediction frame in the target precise positioning stage of the present invention; Figure 6It is a schematic diagram of the processing of the precise prediction frame in the target precise positioning stage of the present invention; Figure 7 is a schematic diagram of the target grasping posture in the present invention; Figure 8 It is a specific flow chart of the target precise positioning stage in the present invention; Fig. 9 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0014] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0015] Embodiment 1: The present invention divides the dual-arm intelligent grasping operation process into a target coarse positioning stage, a target fine positioning stage, and a dual-arm movement stage. The target coarse positioning stage uses global camera information to provide the robot arm with position information for target fine positioning; the target fine positioning stage uses a 6D grasping posture estimation method to obtain the grasping posture of each arm; the dual-arm movement stage is based on an improved artificial potential field method for planning and control, solving the problem of dynamic obstacle avoidance and collision between the dual arms during the movement of the dual-arm robot. The specific process of the dual-arm intelligent grasping operation process is as follows: Fig. 9 shown.

[0016] Step 1: Establish the mutual conversion of coordinate systems on the dual-arm robot; The dual-arm robot coordinate system of the present invention includes a left arm, a right arm, a global camera, a left arm end camera, and a right arm end camera. Figure 1 As shown. Among them, is the world coordinate system, is the left arm base coordinate system, is the right arm base coordinate system, is the coordinate system of the left arm end, is the right arm base coordinate system, is the global camera coordinate system, is the camera coordinate system at the end of the left arm, is the camera coordinate system at the end of the right arm, where the coordinate system at the end of the left arm is converted to the world coordinate system as an example: In the above formula, express Coordinate system to The rotation transformation matrix of the coordinate system, express Coordinate system to The offset vector of the coordinate system.

[0017] Step 2: Set the joint angles for the arms to start working and to place , , , .

[0018] Step 3: Use the global depth camera to obtain RGB image information and depth information respectively, use the interface provided by the camera to align the RGB image and depth image, and realize the mutual conversion between the RGB image pixel coordinates and the depth point cloud position information.

[0019] Step 4: The robot enters the target coarse positioning stage, and uses the coarse positioning model (the coarse positioning model uses the YOLOv8 model) and point cloud computing to obtain the coarse positioning results of all objects to be grasped, that is, the pre-grasping positions of the two arms. The target coarse positioning process is as follows: Figure 2 shown.

[0020] The specific steps are: 1. Use the trained YOLOv8 model to detect the input image (RGB image) and obtain the Prediction box of the object to be grasped in, and Respectively represent the coordinates of the upper left corner and lower right corner of the prediction box; 2. Use image alignment to extract point cloud data aligned with the prediction box in the depth image Among them, each point cloud data Expressed as three-dimensional coordinates with the camera coordinate system as the origin ; 3. Centralize the point cloud data and calculate the corresponding point cloud centroid within the prediction box and centralized point cloud data , the calculation formulas are as follows: 4. Calculate the covariance matrix of the centralized point cloud data , the calculation formula is as follows: in, It is the transpose of the centralized point cloud data; 5. Solve the eigenvalues ​​of the covariance matrix and the eigenvector , the calculation formula is as follows: 6. Point cloud normal vector Calculate, assuming that the eigenvector corresponding to the largest eigenvalue is , then the feature vector is the normal vector of the point cloud data ; 7. Point cloud normal vector and the prediction box center The conversion formula from the camera coordinate system to the dual-arm robot world coordinate system is: 8. Determine the pre-grasping position and set the rough positioning result as the target position of the robot arm , the location has the center of the prediction box The normal vector of the point cloud along the area The calculation formula is as follows: 9. After obtaining the target position, use the inverse kinematics of the robot arm to calculate the corresponding joint angle: Step 5: Both arms enter the motion phase, and the improved artificial potential field method is used to control both arms to reach their respective pre-grasping positions.

[0021] The specific process of the double-arm movement phase is as follows Figure 4 As shown, the steps are as follows: 1. Based on the dual-arm kinematic model, the robot model is simplified, the joint points of the robot are determined, virtual joint points are set at the center of the connecting rod between adjacent joints of the robot, and a spherical bounding box is used to envelop each joint point; 2. Use the global depth camera to obtain all point cloud data in the camera in real time. Due to the large amount of original point cloud data, the use of original point cloud data for calculation cannot meet the real-time requirements of the dual-arm robot. Therefore, the PCL library is used to process the point cloud to reduce the amount of calculation.

[0022] a. Through filtering, set the filtering range, for each point in the point cloud , the condition of straight-through filtering can be expressed as: in, and The range set for the filter is then processed in the same way for other dimensions to filter the point cloud data beyond the workspace of the dual-arm robot.

[0023] b. Voxel downsampling filtering divides the point cloud data after straight-through filtering into a uniform voxel grid to reduce the density of the point cloud. While retaining the shape of the point cloud, it effectively reduces the amount of point cloud data and improves the calculation speed.

[0024] c. Filtering the point cloud of the robot body. During the movement of the two arms, the global camera will collect the point cloud of the robot body. It is necessary to further filter the point cloud of the robot body and only retain the point cloud data of obstacles that enter the workspace of the two-arm robot. Through the kinematic model and coordinate transformation, the position of each joint of each robot arm in the camera coordinate system is calculated, taking the left arm joint as an example.

[0025] The calculation formula is as follows: , in, For the left arm The transformation matrix of the joints in the left arm base coordinate system is: The left arm The position of each joint in the camera coordinate system.

[0026] Use conditional filtering to The joint position is taken as the center, and a distance threshold is set to filter out points whose distance from the reference point is less than the threshold.

[0027] After point cloud processing, the global camera only retains the obstacle information that exists in the workspace of the dual-arm robot. Figure 3 shown.

[0028] 3. Calculate the positions of all joints of the arms in the corresponding base coordinate system through the spherical envelope and kinematic model of the arms, and then transform them into the world coordinate system through coordinate transformation to calculate the distances of all joints of the arms in the world coordinate system. The calculation method is as follows: , in, Left arm joints and right arm The position coordinates of each joint in the world coordinate system.

[0029] At the same time, calculate the distance between each point cloud and all joints of the dual robotic arms in the world coordinate system , in, They represent the first Point cloud and robotic arm The position coordinates of the joints.

[0030] 4. The traditional artificial potential field algorithm is based on the Cartesian coordinate system and needs to be inversely converted into the joint space. The calculation time is long, the efficiency is low, and it is easy to fall into the local minimum. To improve the above problems, an artificial potential field algorithm based on the joint space is proposed: a repulsive field is established according to the distance calculated above. The calculation formula of the repulsive field is as follows: , , in, is the repulsion coefficient; Total repulsive field The calculation formula is: ; The derivation of the repulsion function is: ; because To calculate the repulsive force of each joint of the robot in Cartesian space, the Jacobian matrix of the robot needs to be calculated Map the repulsive force to the joint space: .

[0031] 5. Calculate the gravitational field. The calculation formula is as follows: , in, and They are the current joint angle and target joint angle of the robot arm respectively; 6. Update the joint angle of the robot arm; in, is the iteration step length; 7. Repeat steps 3 to 6 until the robot arm reaches the target joint angle.

[0032] Step 6: In the target precise positioning stage, the two arms use the 6D grasping posture prediction algorithm to obtain the precise grasping posture of the two arms in the pre-grasping posture; the specific process of the target precise positioning stage is as follows Figure 8 As shown, the steps are as follows: 1. When the robot reaches the pre-grasping position, the configured end camera is called to obtain RGB information and depth information in real time; 2. Call the trained fine positioning model to obtain a more accurate prediction frame of the object to be grasped. The coarse positioning is based on the image collected by the global camera. The coarse positioning effect is as follows: Figure 5 As shown in the figure, only the approximate edge range can be selected. Precise positioning can directly segment the work object and generate the recommended grasping posture, such as Figure 7 As shown, the precise positioning model adopts the YOLOv8 model, processes the RGB image and depth image of the terminal camera, expands the prediction frame, and only retains the data in the prediction frame after expansion. Expansion generally refers to the edge expansion and sparse processing of the image. The expansion described in the present invention also performs edge prediction on the image processed by YOLOv8, which is the predicted data, reducing the interference of irrelevant information and the calculation amount of the posture prediction model, such as Figure 6 As shown; 3. Input the processed RGB image and depth image into the pre-trained grasp prediction model GraspNet model (GraspNet is an existing model, which is an efficient convolutional neural network for real-time detection of low-power device grasping. The model can generate better grasping postures through a small amount of sample training). Generate a large number of disordered 6D grasping posture sequences of the gripper, sort them according to the grasping posture scores, and take the grasping posture with the highest score as the target grasping posture, such as Figure 7 As shown; 4. Perform an inverse solution on the end grasping posture. If there is no inverse solution, it means that the robot arm cannot reach the target end posture. Then take the grasping posture with the second highest score as the target grasping posture, until the target grasping posture has an inverse solution or all grasping postures in the posture sequence are traversed.

[0033] Although the YOLOv8 model is used in both the coarse positioning stage and the fine positioning stage, their application focuses are different: Coarse positioning stage: YOLOv8 is mainly used for rapid object detection to obtain the approximate location of the object, and the preliminary grasping position is calculated through point cloud data. At this time, the YOLOv8 model outputs a rough prediction box with relatively low positioning accuracy. This is achieved by configuring the YOLOv8 model as follows: In the coarse positioning stage, a smaller input size is used, larger Anchor Boxes are used, and looser NMS and confidence thresholds are set.

[0034] Precision positioning stage: YOLOv8 is mainly used to detect the target object more accurately, and the depth image and 6D grasping posture prediction model are used to generate accurate grasping posture. At this time, YOLOv8 is not only used to detect the position of the object, but also helps calculate and optimize the grasping posture, which requires higher accuracy. This is achieved by configuring the YOLOv8 model as follows: a larger input size is used in the precision positioning stage, more refined Anchor Boxes are used, and a stricter threshold is set in the precision positioning stage to ensure accurate positioning.

[0035] Step 7: Move both arms to the target grasping posture and control the gripper to complete the object grasping.

[0036] Step 8: The arms enter the motion phase and reach the placement position, completing the object placement.

[0037] Step 9: Determine whether there are still objects that need to be grasped. If so, repeat steps 3 to 8. Otherwise, end the dual-arm intelligent grasping task.

[0038] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. An intelligent grasping method for a humanoid dual-arm robot, characterized in that: The following steps are involved: S1, target coarse positioning stage: obtain the RGB image and depth image of the target object through the global camera and align them; Use the coarse positioning model to detect the RGB image of the target object, extract the coarse positioning information of the target object, calculate the point cloud data of the coarse positioning information based on the alignment process, calculate the point cloud centroid and point normal vector of the target object based on the point cloud data and convert them to the robot world coordinate system, and determine the pre-grasping position of the two arms; S2, double-arm motion stage: construct the kinematic model of the double arms and envelop the joints of the double arms; The global point cloud data in the workspace is acquired in real time through the global camera, and the global point cloud data is preprocessed to retain the obstacle point cloud; the distance between the double arm joints and the obstacle point cloud is calculated in real time by combining the kinematic model and coordinate transformation; The repulsive field and the gravitational field are calculated in the joint space, and the improved artificial potential field algorithm is used for path planning so that the two arms can reach the pre-grasping positions respectively; S3, target precise positioning stage: when the two arms reach the pre-grasping position, the RGB image and depth image of the target object are obtained through the cameras at the end of the two arms and aligned, the RGB image of the target object is detected using the precise positioning model, the precise positioning information of the target object is extracted, and the point cloud data of the precise positioning information is calculated based on the alignment process; the RGB image and depth image processed by the precise positioning model are input into the pre-trained 6D grasping posture prediction model to generate an unordered grasping posture sequence, the grasping posture with the highest score is selected, and the selected grasping posture is inversely kinematically solved. If there is no solution, the posture with the second highest score is selected until a feasible target grasping posture is found; S4, object placement stage: control the arms to perform the grasping action according to the target grasping posture and place it at the preset position; determine whether there is an ungrasped target object. If so, repeat S1-S3 until all grasping tasks are completed.

2. The intelligent grasping method of the humanoid dual-arm robot according to claim 1, characterized in that: The target rough positioning stage obtains the pre-grasp position of the target object through the following steps: The RGB image and depth image of the target object are acquired through the global camera, and the RGB image and the depth image are aligned to obtain the depth value corresponding to each RGB pixel in the RGB image and generate point cloud data; Use the coarse positioning model to detect targets in RGB images and obtain the coarse prediction box of the target object; Extracting point cloud data of the corresponding area in the depth image according to the coarse prediction frame; Perform central processing on the point cloud data to obtain the point cloud centroid and point cloud normal vector of the point cloud data; The point cloud normal vector and the center of the rough prediction box are converted to the robot world coordinate system, and the pre-grasping position of the target object is determined by offsetting a preset distance along the direction of the point cloud normal vector; The joint angles corresponding to the pre-grasping position are obtained through inverse kinematics calculation of the robot arm.

3. The intelligent grasping method of the humanoid dual-arm robot according to claim 2, characterized in that: No. The rough prediction box of a target object is expressed as: ,in, and Respectively represent the upper left corner coordinates and lower right corner coordinates of the rough prediction box; The point cloud data of the area corresponding to the coarse prediction box in the depth image It is expressed as: ; Centralize the point cloud data to obtain the corresponding point cloud centroid and centralized point cloud data , expressed as: , , calculate the centralized point cloud data The covariance matrix of , the calculation formula is: ,in, is the transpose of the centralized point cloud data, solving the covariance matrix The eigenvalue of and the eigenvector , the calculation formula is: , assuming that the eigenvector corresponding to the largest eigenvalue is , then the feature vector is the normal vector of the point cloud data ; The point cloud normal vector and the center of the rough prediction box The conversion formula from the camera coordinate system to the robot world coordinate system is: , ,in, express Coordinate system to The rotation transformation matrix of the coordinate system, express Coordinate system to The offset vector of the coordinate system; Set the rough positioning result as the pre-grasping position of the robot arm , the pre-grab position is through the prediction box center Along the point cloud normal vector The calculation formula is as follows: ; Get the pre-fetch position Finally, the corresponding joint angles are calculated using the inverse kinematics of the robotic arm: 。 4. The intelligent grasping method of a humanoid dual-arm robot according to claim 1, characterized in that: The two-arm movement phase includes: Based on the kinematic model of the two arms, the structure of the robot arm is simplified, the joint points of the robot arm are determined, virtual joint points are set at the center of the connecting rod between the adjacent joints of the robot arm, and a spherical bounding box is used to envelop each joint point to obtain a spherical envelope of the two arms; The global point cloud data is acquired in real time through the global camera, and the global point cloud data is processed, including: using straight-through filtering to limit the point cloud data to the workspace of the dual-arm robot; using voxel downsampling to reduce the density of the point cloud data; removing the point cloud data of the robot body through conditional filtering, and only retaining the obstacle point cloud in the workspace; According to the spherical envelope of the arms and the kinematic model of the robotic arm, the position of each joint point of the arms in the world coordinate system is calculated. Through the position of the joint points, the distance between the joint points of the arms and the distance between the obstacle point cloud and the joint points of the arms are calculated; A repulsive field is established based on the distance between the obstacle point cloud and each joint point of the two arms. The Jacobian matrix 𝐽 is used to map the repulsive force from the Cartesian space to the joint space. The gravitational field is calculated based on the difference between the current joint angle of the two arms and the target joint angle. The gravitational field and the repulsive field are synthesized to obtain the total force field acting on the two arms. The joint angles of both arms are updated iteratively through the total force field to obtain the motion paths of both arms.

5. The intelligent grasping method of the humanoid dual-arm robot according to claim 4, characterized in that: For each point in the point cloud data , the straight-through filtering is expressed as: in, and The range set for the filter; The point cloud data of the robot body is removed by conditional filtering, including: The joint position is taken as the center, and the distance threshold is set to filter out points whose distance from the reference point is less than the threshold: .

6. The intelligent grasping method of a humanoid dual-arm robot according to claim 4, characterized in that: The distance between the joints of the two arms is calculated using the following formula: , in, Left arm joints and right arm The position coordinates of each joint in the world coordinate system; The distance between each obstacle point cloud and each joint point of the arms is calculated using the following formula: in, They represent the obstacle point cloud and the two arms The position coordinates of the joint points, The value is L or R, representing the left arm and the right arm respectively; A repulsive field is established based on the calculated distance. The calculation formula of the repulsive field is as follows: in, is the repulsion coefficient, The safety threshold is set; Total repulsive field The calculation formula is: ; The derivation of the repulsion function is: ; Calculate the Jacobian matrix of the robot Map the repulsive force to the joint space: ; Calculate the gravitational field, the formula is as follows: , in, and They are the current joint angle and target joint angle of the robot arm respectively; Update the robot arm joint angle; in, is the iteration step length.

Images