A lightweight robot arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis

Abstract

This invention discloses a lightweight robotic arm grasping method and system based on point cloud bird's-eye view manifold projection and negative space analysis, belonging to the fields of computer vision and robot control technology. The method includes: obtaining a transformation matrix through hand-eye calibration using an RGB-D camera; using the lightweight YOLOv8 network for initial target localization and constructing a 3D truncated cone to initially filter the point cloud; using the RANSAC algorithm to peel off the background plane to extract the sparse point cloud of the target; constructing a geometric manifold projection model to reduce the 3D point cloud to a 2D bird's-eye view (BEV) plane, defining the data vacuum region caused by object occupancy as the "negative space"; and reconstructing the 6-DoF grasping pose of the target by analyzing the centroid and principal axis yaw angle of the negative space contour based on image moment theory. This invention transforms complex 3D feature matching into 2D geometric analysis, with computational complexity at only linear levels. It can achieve extremely fast and robust grasping pose estimation on industrial edge devices without the need for a GPU, significantly reducing hardware costs and improving adaptability to various working conditions.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and robot control technology, specifically relating to a lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space geometric analysis. Background Technology

[0002] In industrial automated production, robotic arm vision grasping is a core component for complex operations such as sorting and assembly. Currently, the mainstream technical approach generally employs deep learning models (such as PointNetGPD) to extract features from raw 3D point clouds. While this type of solution achieves good recognition accuracy under ideal data conditions, it reveals significant shortcomings in practical industrial applications.

[0003] The core problem facing such solutions lies in the contradiction between computing power requirements and hardware costs. Deep neural networks involve massive numbers of parameters and high-dimensional convolutional operations, which typically require high-performance, high-power GPUs. This directly increases the system's construction cost and limits the deployment of algorithms on low-power industrial embedded CPUs or edge devices. Even in environments with sufficient computing power, the random sampling and complex evaluation processes for large-scale point clouds are often time-consuming (usually exceeding 500 ms). The computational latency is difficult to match the high-frequency operation of modern production lines, easily leading to idle robotic arms and hindering overall production efficiency.

[0004] Fluctuations in the quality of raw data are also a key factor affecting the stability of existing algorithms. Industrial environments are extremely complex. Strong reflections from the metallic luster of object surfaces, drastic changes in ambient lighting, and system noise from the sensors themselves often result in extremely sparse point cloud data with incomplete contours or massive outliers. Existing feature matching and registration logic is prone to drift in positioning accuracy or even convergence failure when faced with such non-ideal data, making it difficult to maintain high-strength grasping stability.

[0005] In summary, the current field of industrial robot vision urgently needs a lightweight grasping solution that can break free from the constraints of high-performance hardware and has strong anti-interference capabilities in harsh working conditions, in order to solve the problem of balancing computing power cost, real-time performance and positioning accuracy in complex industrial environments. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis. This addresses the issues of existing 3D vision grasping technologies, such as high reliance on hardware computing power, poor robustness in sparse point cloud environments, and insufficient real-time performance. This invention projects a 3D point cloud manifold onto a 2D plane and utilizes the "negative space" formed by the object to be grasped for geometric analysis, aiming to achieve extremely fast and robust grasping pose estimation in edge computing environments.

[0007] The technical solution adopted in this invention is as follows:

[0008] A lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis includes the following steps:

[0009] S1: Initialize the system and complete the hand-eye calibration of the robotic arm and camera: First, set up a calibration board in the industrial operation scene, and acquire multiple sets of calibration board images in different poses using an industrial RGB-D camera mounted on the robotic arm. Extract feature points from the calibration board, combine them with the end-effector pose data fed back by the robotic arm, and then use classic... The calibration model is then subjected to nonlinear optimization to obtain the homogeneous transformation matrix between the camera coordinate system and the robot arm's base coordinate system. This step unifies the coordinate systems of the perception space and the execution space, providing a precise reference for subsequent grasping point transformations.

[0010] S2: Acquire multimodal scene data and achieve rapid coarse target localization: Drive the vision sensor in real time to acquire color images (RGB) and raw 3D point cloud data of the current scene. Use the pre-trained lightweight target detection network YOLOv8 to perform real-time inference on the color images, identify the category of the target to be grasped, and output the corresponding 2D bounding box to achieve rapid coarse target localization;

[0011] S3: Extracting the region of interest (ROI) from the point cloud and performing background stripping on the supporting plane: Based on the 2D bounding box obtained in S2, a 3D frustum is constructed using the camera intrinsic matrix projection. The ROI point cloud inside the frustum is extracted from the original point cloud. Subsequently, the RANSAC (Random Sample Consensus) algorithm is used to identify the geometric features of the supporting plane, fit the implicit equation of the table plane, and delete the point cloud set belonging to this plane. Finally, a clean target point cloud containing only the outline of the target object is extracted.

[0012] S4: Perform manifold projection transformation to establish the object's negative space geometry model: Construct a manifold projection model and vertically project the 3D target point cloud onto the 2D bird's-eye view (BEV) plane. During the projection process, according to the preset projection resolution... The mapping function for rasterizing spatial coordinates. as follows:

[0013] ;

[0014] In the generated binarized scene occupancy map, the area where the point cloud falls is defined as the background area and assigned a value of 1; while the "data vacuum" area in physical space that is occupied by objects and causes a lack of point cloud feedback is defined as the core analysis object—the "negative space"—and assigned a value of 0. This step transforms the complex 3D point cloud feature matching into stable 2D hole contour parsing;

[0015] S5: Analyzing Negative Space Image Moment Features and Solving 6-DoF for Pose Grabbing: Morphological processing is performed on the scene occupancy map, eliminating noise artifacts from the sensor through closing operations, and completing the negative space contour. Subsequently, based on image moment theory, geometric analysis is performed on the connected components of the negative space to calculate the centroid coordinates of the target in the BEV plane. and planar rotation yaw angle (Yaw) Specifically, the covariance matrix is ​​constructed using the second-order central moments, and the angle is solved using the following formula:

[0016] ;

[0017] Finally, the inverse transformation of the projection function is used to restore the two-dimensional centroid to the three-dimensional coordinate system, and combined with the support plane height information, a complete 6-DoF grasping pose matrix is ​​constructed.

[0018] S6: Generate automated path planning instructions and drive the robotic arm to perform grasping: Transform the solved pose matrix to the robotic arm base coordinate system, generate an obstacle avoidance operation path, and drive the robotic arm actuator to complete target grasping.

[0019] Furthermore, the specific steps of geometric manifold projection and scene occupancy modeling in step S4 are as follows:

[0020] S41: Perform scale-space mapping to project 3D points onto 2D grid coordinates: Set projection resolution parameters Traverse every three-dimensional point in the target point cloud Using mapping functions Transform it to the BEV image coordinate system The specific calculation formula is as follows:

[0021] ;

[0022] ;

[0023] in, This represents the minimum boundary value of the projection of the point cloud onto the horizontal plane within the current truncated cone region. This indicates a round-down operation;

[0024] S42: Implement feature domain assignment logic to generate a negative space binary mask image: Create a binary matrix with an initial value of 1 as the scene map base, if a certain pixel grid If a point cloud point falls into the memory, the pixel value is kept at 1 and it is considered a known background area; if a pixel grid is in the occlusion area of ​​an object and no point cloud data falls into it, it is assigned a value of 0, thereby constructing a binary mask of "negative space" that can reflect the top view outline of the object.

[0025] Furthermore, the specific steps of negative space feature parsing in step S5 are as follows:

[0026] S51: Perform morphological processing to complete the negative space connected component outline: Use rectangular structuring elements of preset size to perform a closing operation on the binarized scene occupancy map. First, fill the holes inside the negative space caused by reflection from the object surface or uneven lighting through a dilation operation. Then, restore the outline boundary through an erosion operation to obtain the complete object negative space outline map. ;

[0027] S52: Perform connected component feature parsing while locking the target projection region: Use the Two-Pass algorithm or Seed-Filling algorithm. Analysis was conducted, and the largest connected region in the negative space was identified as the geometric projection contour of the target to be grasped. ;

[0028] S53: Calculating the geometric distribution features of the target based on image moment theory: traversing the geometric projection contour For all pixels within the range, calculate the zeroth moment, first moment, and second central moment.

[0029] Furthermore, the specific steps for solving the pose in step S5 are as follows:

[0030] S54: Solve for the negative space centroid coordinates and determine the target's planar yaw angle: Based on the moment characteristics calculated in step S53, solve for the target's centroid coordinates and principal axis rotation yaw angle in the BEV plane. The specific calculation formula is as follows:

[0031] ;

[0032] ;

[0033] S55: Perform 3D spatial pose reconstruction to restore the target's physical center coordinates: using a projection mapping function. The inverse transformation of the two-dimensional centroid Mapping back to a 3D coordinate system Coordinates; simultaneously, extract the average height of the support plane fitted by the RANSAC algorithm in step S3 as the target object's coordinates. Axis reference value ;

[0034] S56: Synthesize the 6-DoF homogeneous pose transformation matrix of the target object and optimize the computational complexity: Combine the obtained centroid coordinates and rotation angles to construct the homogeneous pose matrix of the target object. The calculation process bypasses the complex point cloud registration calculation, and the overall computational complexity remains at a linear level with respect to the number of input points.

[0035] Furthermore, the configuration and application details of the YOLOv8 network in step S2 are as follows:

[0036] S21: Configure a lightweight detection network architecture to adapt to real-time inference on embedded processing units: Select a micro version of YOLOv8n as the basic network architecture, and replace standard convolutions with depthwise separable convolutions to reduce the number of parameters, enabling it to perform real-time inference on embedded CPUs.

[0037] S22: Implement search space constraint logic to achieve adaptive reduction of the original point cloud data volume: Set the confidence threshold of YOLOv8 prediction to 0.6 to filter out false targets; The width and height information of the 2D bounding box are transmitted to step S3 in real time to dynamically adjust the angle of the 3D truncated cone, thereby achieving adaptive filtering of the point cloud data volume and reducing the number of points to be processed to less than 5% of the original point cloud.

[0038] The beneficial effects of this invention include:

[0039] 1. Significantly Reduced Computational Complexity: This invention simplifies the 3D spatial pose estimation problem into 2D geometric contour analysis through geometric manifold projection technology, reducing the overall computational complexity to linear levels. Experimental data shows that the system's average response time under normal CPU conditions is only 210 ms, which is more than 2.6 times faster than traditional deep learning methods.

[0040] 2. Eliminates dependence on high-performance hardware: The algorithm design does not require high-dimensional convolution operations, relying only on basic geometric calculations. This allows the solution to be directly deployed on inexpensive industrial edge devices that lack GPU computing power, significantly reducing system deployment costs;

[0041] 3. Excellent adaptability to working conditions: It innovatively utilizes "negative space" for feature extraction. When faced with harsh visual conditions such as strong reflections on object surfaces, sparse point clouds, or incomplete outlines, the "vacuum area" formed by the object's position is more feature-stable than the object's own "fragmented point cloud," thus significantly improving the robustness of grasping.

[0042] 4. High integration and easy scalability: This system not only protects the method process, but also achieves a high degree of integration of visual perception, algorithm processing and motion control through modular design, which has great value for industrial application promotion. Attached Figure Description

[0043] Figure 1 This is an overall flowchart of a lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis, according to an embodiment of the present invention.

[0044] Figure 2 is a schematic diagram of the target detection network architecture and 3D truncated cone construction principle according to an embodiment of the present invention.

[0045] Figure 3 is a schematic diagram of geometric manifold projection and negative space binary mask generation according to an embodiment of the present invention.

[0046] Figure 4 is a hardware logic block diagram of the robotic arm vision grasping system provided in an embodiment of the present invention.

[0047] Figure 5 is a flowchart illustrating the actual grasping experiment of the robotic arm under 6 typical poses provided in the embodiments of the present invention. Detailed Implementation

[0048] Example 1: This example discloses the complete implementation process of a lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis.

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0050] As shown in Figure 1, this embodiment provides a robotic arm visual grasping method based on negative space analysis of point cloud bird's-eye view, which specifically includes the following steps:

[0051] S1: Initialize the system and complete the hand-eye calibration between the robotic arm and the camera. Specifically, step S1 includes:

[0052] S11: Install an industrial RGB-D depth camera on the end effector of the robotic arm, and place a standard calibration board (such as a checkerboard calibration board or a dot calibration board) on the worktable of the robotic arm. Preferably, the calibration board uses a 9×6 black and white checkerboard pattern, with each square having a fixed side length of 20 mm. This process must ensure that the calibration board is centered in the camera's field of view and that the lighting is uniform to avoid feature point extraction failure due to surface highlights.

[0053] S12: Controls the movement of the robotic arm within the workspace, moving the camera from... A different spatial perspective The calibration board is photographed, and the pose matrix of the robotic arm's end effector relative to the base coordinate system is recorded at each photograph. The stated field of view should cover the edges and core areas of the robotic arm's range of motion, and the number of fields of view should be... It is recommended to have more than 15 groups, and each group of poses should contain obvious differences in translation and rotation to increase the rank of the calibration equations, thereby improving the stability of solving the homogeneous transformation matrix.

[0054] S13: Use Zhang Zhengyou's calibration method to extract the feature point coordinates of the calibration board in each image, and calculate the pose matrix of the calibration board in the camera coordinate system. Specifically, the system calls the corner detection operator in the OpenCV library to achieve sub-pixel level positioning accuracy, and combines the physical size parameters of the calibration board to output the real-time homogeneous matrix of the calibration board relative to the optical center of the camera;

[0055] S14: Building Classics Mathematical model, in which The hand-eye transformation matrix from the camera coordinate system to the robot arm end effector coordinate system is to be solved. In this model, This represents the transformation of the robotic arm's end effector from pose 1 to pose 2. This represents the corresponding displacement of the calibration plate in the camera view. By comparing the motion increments of the two, the relative position between the camera optical center and the center of the robotic arm flange can be calculated.

[0056] S15: A nonlinear optimization algorithm is used to minimize the reprojection error of the transformation matrix. In this embodiment, the Levenberg-Marquardt algorithm is used iteratively to control the root mean square error within 0.5 mm. The resulting hand-eye transformation matrix... This will be used as a fixed transformation parameter to map all visual detection results to the robot arm's base coordinate system in real time, ensuring spatial consistency between "seeing" and "grasping".

[0057] S2: Acquire multimodal scene data and achieve rapid coarse localization of the target. Specifically, step S2 includes:

[0058] S21: Simultaneously acquire color images (RGB) and raw 3D point cloud data of the current industrial operation scene using an RGB-D camera. The camera operates at a frequency of 30 fps, and a hard-triggered synchronization signal ensures that the color pixels and depth pixels are strictly aligned on the time axis, thereby avoiding motion blur or ghosting caused by the movement of the robotic arm.

[0059] S22: The acquired color image is preprocessed, including size normalization and noise reduction, and then input into a pre-trained lightweight YOLOv8 object detection network. Specifically, the YOLOv8 structure used in this invention removes redundant connections, and employs Mosaic enhancement and mixed-precision training techniques during the training phase. Before entering the network, the image is padded into a square and scaled to 640×480 pixels to fit the receptive field of the convolutional kernel, improving the recognition accuracy of tiny industrial parts.

[0060] S23: The YOLOv8 network extracts multi-scale features through its backbone and outputs a two-dimensional bounding box of the target to be captured through the decoupled detection head. The bounding box information includes the pixel coordinates of the upper left corner. bottom right pixel coordinates And the target's category label and confidence score;

[0061] S24: Set a confidence threshold (preferably 0.6), retaining only detection results higher than this threshold as input for subsequent truncated cone construction, achieving rapid coarse target localization. This threshold is based on statistical analysis of industrial noise, effectively filtering interference from background impurities and preventing missed detections due to lighting fluctuations. Through this step, the system achieves preliminary "localization locking" of the target.

[0062] S3: Extract the point cloud region of interest and perform background stripping on the supporting plane. Specifically, step S3 includes:

[0063] S31: Obtain the intrinsic parameter matrix of the RGB-D camera Includes focal length parameters and the coordinates of the principal point of the optical center Based on the pinhole camera model, each pixel coordinate can be mapped to a ray in space. Connecting the four corner points of the bounding box to the optical center defines a pyramidal region in space. This step utilizes prior 2D detection information to perform spatial hashing filtering on millions of point cloud points across the entire field.

[0064] S32: Based on the four vertices of the two-dimensional bounding box in S2, the ray vectors in three-dimensional space are calculated using the principle of back projection, thereby constructing a 3D frustum region with the camera optical center as the vertex and extending into the object space.

[0065] S33: Traverse the original point cloud Determine whether the coordinates of each point cloud point fall inside the 3D truncated cone, retain the internal points and discard the external points to form the region of interest point cloud. This "2D-guided 3D" approach significantly reduces computational complexity. Experimental data shows that the number of processing points is reduced from that of the original scene. One reduced to Approximately one, directly eliminating interference points in non-working areas;

[0066] S34: Point cloud of region of interest The RANSAC (Random Sample Consensus) algorithm is invoked. First, three non-collinear points are randomly selected from the point cloud set, and the plane equation is constructed. ;

[0067] S35: Calculate the point cloud of the region of interest If the perpendicular distance from the other points in the plane is less than a preset distance threshold (e.g., 5 mm), then the point is determined to be an in-plane point.

[0068] S36: Through multiple iterations (e.g., 1000 times), select the planar model containing the most internal points as the optimal desktop support plane;

[0069] S37: Point cloud from region of interest Delete all point clouds belonging to this plane and extract a clean target point cloud containing only the target object to be grabbed. Even if the point cloud appears extremely sparse or only contains surface information due to occlusion, since we subsequently use negative spatial logic, we do not require the point cloud to have a complete 3D closed surface.

[0070] S4: Perform manifold projection transformation to establish the negative space geometric model of the object. Specifically, step S4 includes:

[0071] S41: Construct the geometric manifold projection model. Set the projection resolution parameters. (Preferably 5 mm), this parameter balances computational accuracy and processing speed. The choice of this resolution is crucial: if If it's too large, the edge details of the object will be lost; if... If the resolution is too small, the computational load will increase exponentially. The 5 mm resolution selected in this embodiment can meet the positioning accuracy requirements of most industrial parts (such as bottles and blocks);

[0072] S42: Traverse the 3D pure target point cloud Each point in Using the following mapping function Transform it to pixel grid coordinates in a two-dimensional bird's-eye view (BEV) plane. : , .in, Let be the minimum boundary value of the point cloud on the horizontal projection plane. This indicates a round-down operation;

[0073] S43: Create a binary matrix with an initial value of 1 as the scene occupancy map. During the projection process, if a certain grid... If there are point clouds falling into the area, that area is considered a visible background area and its pixel value is kept at 1.

[0074] S44: Define the area with no data feedback caused by camera ray occlusion due to the physical occupation of the target object as "Negative Space", and assign the pixel value of this area to 0;

[0075] S45: In the resulting binarized image, the black "negative space" connected components completely depict the geometric outline of the object viewed from above. This step is the core philosophy of this invention: we don't analyze the points of the object itself, but rather "the hole the object has dug in the scene." Through this reverse thinking, even if the point cloud reflecting off the object's surface is completely messed up, as long as the outline of that "hole" is still there, the localization is accurate.

[0076] S5: Analyze the negative space image moment features and solve for the 6-DoF capture pose. Specifically, step S5 includes:

[0077] S51: Binarization of the image Morphological closing operations are performed. Using a 3×3 all-1 structuring element, a dilation operation is first performed to fill small holes caused by sparse point clouds or sensor voids, followed by an erosion operation to restore the true boundaries of the object and obtain a smooth negative space contour. The specific operation is as follows: First, dilation is performed using a convolution kernel with all 1s, causing the boundaries of the connected regions in the negative space to expand outward, thereby blocking the false small holes caused by missing points in the point cloud; then, erosion of the same degree is performed. This process can greatly improve the connectivity of the negative space region and prevent the centroid shift caused by region breakage when calculating moment features in the subsequent process;

[0078] S52: Using Image Moments Theory to analyze contours Let's analyze it. First, calculate the zeroth moment. , which represents the area of ​​the connected region in the negative space; then the first moment is calculated. and ;

[0079] S53: Solve for the centroid coordinates of the negative space BEV plane using the following formula. : The total area is calculated using the zeroth moment, and the average coordinates are calculated using the sum of the first moments. The centroid represents the center of the projected plane of the object and is translationally invariant.

[0080] S54: Calculate the second-order central moments And calculate the principal axis direction of the object in the BEV plane, i.e., the plane yaw angle, according to the following formula. : The orientation information is constructed using the second-order moment, and the major axis angle of the optimal equivalent ellipse of the profile is solved using the arctangent function. This angle is the angle that the robotic arm gripper needs to rotate when approaching, ensuring that the gripper is parallel to the narrowest or widest side of the object;

[0081] S55: Pose Reconstruction. Utilizing mapping functions. The reverse process, the two-dimensional centroid Mapping back to coordinates in three-dimensional physical space By utilizing the inverse mapping relationship and combining it with the camera distortion correction parameters obtained from calibration, its physical coordinates can be accurately reconstructed;

[0082] S56: Extract the mean height of the desktop support plane obtained by the RANSAC algorithm fitting in step S3. .Will Use it as the target to capture the central point;

[0083] S57: Combined with yaw angle Synthesize the 6-DoF homogeneous transformation matrix of the target object relative to the camera coordinate system. .

[0084] S6: Generate automated path planning instructions and drive the robotic arm to perform grasping. Specifically, step S6 includes:

[0085] S61: Transforming the matrix via hand-eye connection The pose matrix under the camera system Transform to the robot arm base coordinate system The final physical grab coordinates are obtained below;

[0086] S62: Based on the known obstacle information in the scene and the environmental occupancy generated in step S4, call the path planning algorithm (such as A* or RRT*) to generate a collision-free, smooth approach path in 3D space;

[0087] S63: Send the path point sequence to the robotic arm controller, drive the servo motor to move along the planned trajectory, so that the end effector reaches the preset gripping point and controls the gripper to close to complete the gripping.

[0088] This embodiment discloses a complete implementation process of a lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis. To verify the effectiveness of this method, this embodiment built a highly integrated industrial vision grasping experimental platform, and its hardware selection logic is as follows.

[0089] The overall system adopts a closed-loop architecture of "sensing-computing-control". First, at the visual perception level, this embodiment uses an industrial-grade RGB-D depth camera. This camera has a hard-triggered synchronization function, which can ensure that the color image information and depth point cloud information acquired during the movement of the robotic arm achieve sub-millimeter-level alignment accuracy in terms of time axis and spatial pose. This selection effectively avoids "negative space" contour distortion caused by misalignment of image data and depth data, providing a reliable data source for subsequent geometric manifold projection.

[0090] At the computational processing level, to verify the lightweight advantage of the algorithm of this invention, this embodiment did not use an expensive graphics processing unit (GPU), but instead selected an industrial edge computing gateway equipped with a common low-voltage central processing unit (CPU, such as Intel i7 series). This selection aims to simulate resource-constrained industrial edge working conditions. By using only CPU computing power to run the YOLOv8 coarse localization operator, the RANSAC plane fitting operator, and the core image moment analysis algorithm, it is demonstrated that this invention still has a real-time response capability of 210 ms without relying on high-performance hardware.

[0091] At the execution and control level, this embodiment uses a 6-DOF (6-DoF) collaborative robotic arm, preferably the AUBO-i5 model. The core logic behind choosing this model lies in its extremely high industrial repeatability, enabling it to accurately translate the 6-DoF pose generated by the algorithm into grasping actions in physical space. Furthermore, a pneumatic soft gripper is added to the end effector of the robotic arm, utilizing its physical flexibility to compensate for any minor geometric residuals that may exist in the pose calculation, thereby further improving the overall grasping success rate of the system.

[0092] The specific technical parameters of the AUBO-i5 collaborative robotic arm used in this embodiment are shown in Table 1:

[0093] Table 1. Specific technical parameters of the robotic arm

[0094] Degrees of freedom 6 load 5 kg Working radius 886.5 mm Repeatability ±0.02 mm weight 40 kg Communication interface TCP / IP, Modbus

[0095] To verify the effectiveness and reliability of the above method, this embodiment conducted multiple sets of comparative experiments on the above hardware platform.

[0096] First, the accuracy of 6-DoF pose estimation was tested. The experiment set up 6 groups of grasping targets with known poses, and the pose analysis results are shown in Table 2:

[0097] Table 2 6D Pose Analysis Results

[0098] Group 1 Position / [mm] Attitude / [rad] [0, 0, 0][0.0, 0.0, 0.0] [0, 0, 0][0.0, 0.0, 0.0] Group 2 Position / [mm] Attitude / [rad] [-166, -70, -30][-0.002, 0.002, -0.002] [97.5, 0, 0][0.0, 0.0, 0.0] Group 3 Position / [mm] Attitude / [rad] [-89, -235, -21][-0.000, 0.005, -0.004] [0, 195, 0][0.0, 0.0, 0.0] Group 4 Position / [mm] Attitude / [rad] [-103, 113, -4][-0.000, 0.006, -0.005] [0, 0, 0][0.0, 0.873, 0.0] Group 5 Position / [mm] Attitude / [rad] [-46, 195, -32][0.000, 0.007, -0.006] [0, 0, 0][0.873, 0.0, 0.0] Group 6 Position / [mm] Attitude / [rad] [117, 41, -36][0.000, 0.007, -0.006] [195, 97.5, 0][0.0, 0.698, 0.0] Translation error 195.48 mm Rotational error 23.30 degrees

[0099] Secondly, the performance of the method of this invention was compared with that of traditional deep learning-based methods (such as the SSD+PointNetGPD combination), and the comparison results are shown in Table 3. Experimental results show that, under the same hardware environment, the average computation time for a single capture by the method of this invention is only about 210 ms, while the comparison method requires about 550 ms, which improves the system response speed by more than 2.6 times.

[0100] Table 3 Performance Comparison Table

[0101] Recognition accuracy 95% (57 / 60) 92% (55 / 60) Scraping success rate 73% (42 / 57) 70% (39 / 55) Overall success rate 70% (42 / 60) 65% (39 / 60) Average time per crawl ~210 ms ~550 ms

[0102] To more clearly illustrate the implementation details of each functional module of this invention, the following description is in conjunction with the appendix. Figure 2 To be continued Figure 5 To elaborate in depth.

[0103] As shown in Figure 2, the YOLOv8 network used in this invention consists of a backbone, a feature fusion network (Neck), and a decoupled prediction head. The system first uses 2D bounding boxes to locate the target boundary in the pixel plane, and then combines this with the camera's focal length. A 3D spatial inverse mapping is performed to generate a truncated pyramid with the camera's optical center as its vertex. This logic precisely narrows the point cloud processing scope from millions of discrete points across the entire field to the local "pipeline" where the target is located. This method of using 2D vision to guide 3D geometry is the core prerequisite for this system to bypass high-dimensional 3D convolution and achieve lightweight operation.

[0104] As shown in Figure 3, this invention projects the captured 3D target point cloud onto a 2D BEV plane and performs equal-spacing rasterization. The grid enclosed by solid lines in the figure represents the supporting area (background) where the point cloud falls, while the closed area lacking the point cloud in the middle is the "negative space" defined in this invention. Due to the physical volume of the object causing the laser probe light to be blocked, the outline of this "negative space" has a high degree of geometric overlap with the physical top-view outline of the object. (Figure...) The negative spatial geometric centroids calculated using image moments are labeled. This represents the principal axis yaw angle fitted based on the second-order central moment. This logic of using the "data vacuum" characteristic for reverse calculation greatly enhances the system's adaptability to complex materials such as reflective and translucent objects.

[0105] As shown in Figure 4, this system mainly consists of a visual perception unit, a data preprocessing module, a core algorithm processing center, and a control execution terminal. The visual perception unit uses an industrial-grade RGB-D depth camera with synchronous sampling capabilities, a selection designed to ensure spatial consistency between color images and depth information. The core algorithm processing center employs a general-purpose industrial CPU (such as an Intel i7 series low-voltage processor), which can run without GPU acceleration, reflecting the lightweight nature of this invention. The control execution terminal uses a 6-DOF collaborative robotic arm (such as the AUBO-i5 model) equipped with a pneumatic soft gripper, providing high-precision repeatability.

[0106] As shown in Figure 5, this embodiment demonstrates the end-to-end grasping experiment process for different types of targets (such as cylindrical bottles, cubic cardboard boxes, etc.) under six typical initial poses. The first row of the figure shows the fast coarse localization results of YOLOv8 on the target (detect stage); the second row shows the process of the system generating an obstacle avoidance path and controlling the robotic arm to approach (grasp stage); the third row shows the successful closure of the robotic arm's end gripper and the stabilization of the object (hold stage). Statistical analysis of the data from these six sets of experiments (Group 1 to Group 6) reveals that when the system handles poses with severe self-occlusion (such as the tilted pose in Group 2), although the point cloud contour is incomplete, the pose reconstruction based on the negative space centroid can still control the position error within a certain range. Within this period, the engineering application value of the present invention in a real production environment was verified.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight robotic arm grasping method based on point cloud bird's-eye view manifold projection and negative space analysis, characterized in that, Includes the following steps: S1: Initialize the system and complete the hand-eye calibration of the robotic arm and camera: Set up a calibration board at the target location in the industrial scene, acquire multiple sets of calibration images using an RGB-D camera, and solve for the homogeneous transformation matrix between the camera coordinate system and the robotic arm base coordinate system using the classic AX=XB calibration model. This achieves coordinate unification between the perception space and the execution space; S2: Acquire multimodal scene data and achieve rapid coarse target localization: Simultaneously acquire RGB color images and raw 3D point cloud data of the scene. The RGB image is input into the pre-trained YOLOv8 deep learning network for object detection, outputting the two-dimensional bounding box of the target object and extracting its category information and confidence score. S3: Extracting the region of interest (ROI) from the point cloud and performing background stripping on the supporting plane: Constructing a 3D frustum using the aforementioned 2D bounding box and camera intrinsic matrix projection, and extracting the ROI point cloud within the frustum from the original point cloud. Subsequently, the RANSAC random sampling consensus algorithm was used to... Plane fitting is performed to identify and remove point clouds from the desktop support plane, resulting in a clean object target point cloud. ; S4: Perform manifold projection transformation to establish the object's negative space geometry model: Construct a manifold projection transformation model to transform the 3D target point cloud. Linear dimensionality reduction is mapped to a two-dimensional bird's-eye view (BEV) plane, and the continuous space is discretized into a binary scene occupancy map through rasterization. And define the "data vacuum" area caused by object occupancy as the core analysis object—negative space; S5: Analyze negative space image moment features and solve 6-DoF grasping pose: Perform morphological optimization on the scene occupancy map, perform geometric analysis on the negative space connected components based on image moment theory, calculate the target center position and plane rotation angle, and reconstruct the object's 6-DoF grasping pose by combining plane height information; S6: Generate automated path planning instructions and drive the robotic arm to perform grasping: Apply the solved pose to the aforementioned homogeneous transformation matrix The coordinates are switched to the robotic arm coordinate system to generate obstacle avoidance path instructions, which drive the robotic arm end effector to accurately grasp the target object.

2. The robotic arm visual grasping method based on negative space analysis of point cloud bird's-eye view as described in claim 1, characterized in that, The specific steps for geometric manifold projection and scene occupancy modeling in step S4 are as follows: S41: Perform scale-space mapping to project 3D points onto 2D grid coordinates: Set projection resolution parameters Traverse every three-dimensional point in the target point cloud Using mapping functions Transform it to the BEV image coordinate system The calculation formula is as follows: ； ； in, This represents the minimum boundary value of the projection of the point cloud onto the horizontal plane within the current truncated cone region. This indicates a round-down operation; S42: Implement feature domain assignment logic to generate a negative space binary mask image: Create a binary matrix with an initial value of 1 as the scene map base, if a certain pixel grid If a point cloud point falls into the memory, the pixel value is kept at 1 and it is considered a known background area; if a pixel grid is in the occlusion area of ​​an object and no point cloud data falls into it, it is assigned a value of 0, thereby constructing a binary mask of "negative space" that can reflect the top view outline of the object.

3. The robotic arm visual grasping method based on negative space analysis of point cloud bird's-eye view as described in claim 1, characterized in that, The specific steps of negative space feature parsing in step S5 are as follows: S51: Perform morphological processing to complete the negative space connected component outline: Use rectangular structuring elements of preset size to perform a closing operation on the binarized scene occupancy map. First, fill the holes inside the negative space caused by reflection from the object surface or uneven lighting through a dilation operation. Then, restore the outline boundary through an erosion operation to obtain the complete object negative space outline map. ; S52: Perform connected component feature parsing while locking the target projection region: Use the Two-Pass algorithm or Seed-Filling algorithm to analyze the negative space contour map. Analysis was conducted, and the largest connected region in the negative space was identified as the geometric projection contour of the target to be grasped. ; S53: Calculating the geometric distribution features of the target based on image moment theory: traversing the geometric projection contour Calculate the zeroth moment for all pixels within the range. (Characteristic area), first moment (Characteristics of location distribution) and second-order central moments (Characterizing shape distribution); S54: Solve for the centroid coordinates in the negative space and determine the target's planar yaw angle: Based on the moment characteristics calculated in step S53, solve for the centroid coordinates of the target in the BEV plane. and spindle rotation yaw angle (Yaw) : ； ； S55: Perform 3D spatial pose reconstruction to restore the target's physical center coordinates: using a projection mapping function. The inverse transformation of the two-dimensional centroid Mapping back to a 3D coordinate system Coordinates; simultaneously, the average height of the support plane fitted by the RANSAC algorithm in step S3 is extracted as the target object's coordinates. Axis reference value ; S56: Synthesize the 6-DoF homogeneous pose transformation matrix of the target object and optimize the computational complexity: synthesize the obtained centroid coordinates. and rotation angle Construct the homogeneous pose matrix of the target object. The calculation process bypasses complex point cloud registration calculations, and the overall computational complexity remains constant with respect to the number of input points. linear level .

4. The robotic arm visual grasping method based on negative space analysis of point cloud bird's-eye view as described in claim 1, characterized in that, The configuration and application details of the YOLOv8 network in step S2 are as follows: S21: Configure a lightweight detection network architecture to adapt to real-time inference on embedded processing units: Select a micro version of YOLOv8n as the basic network architecture, and replace standard convolutions with depthwise separable convolutions to reduce the number of parameters, enabling it to perform real-time inference on embedded CPUs. S22: Implement search space constraint logic to achieve adaptive reduction of the original point cloud data volume: Set the confidence threshold of YOLOv8 prediction to 0.6 to filter out false targets; The width and height information of the 2D bounding box are transmitted to step S3 in real time to dynamically adjust the subtended angle of the 3D truncated cone, thereby achieving adaptive filtering of the point cloud data volume and reducing the number of points to be processed to less than 5% of the original point cloud.

Images