Special-shaped workpiece grabbing method and device, computer equipment, readable storage medium and program product

Abstract

The application relates to a special-shaped workpiece grabbing method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring workpiece point cloud information collected for a special-shaped workpiece; determining a target workpiece contour matched with the workpiece point cloud information from a plurality of candidate workpiece contours; calling a grabbing point determination mode corresponding to the target workpiece contour, analyzing the workpiece point cloud information, and determining a grabbing point of the special-shaped workpiece; and controlling a gripper to grab the special-shaped workpiece through gripper working parameters matched with the grabbing point. The method can improve flexibility.

Description

Technical Field

[0001] This application relates to the fields of industrial automation and robotics, and in particular to a method, apparatus, computer equipment, readable storage medium, and program product for gripping irregularly shaped workpieces. Background Technology

[0002] With the rapid development of industrial automation and robotics, intelligent gripping of irregularly shaped workpieces has become one of the key technologies in the field of intelligent manufacturing. Due to their complex shapes and diverse geometric features, traditional gripping methods based on regular geometry are difficult to apply directly to irregularly shaped workpieces.

[0003] In traditional technologies, the gripping position is usually determined by manually designing feature points. However, determining the gripping position by manually marking key points is difficult to adapt to changes in workpiece shape, resulting in insufficient generalization. Therefore, using traditional technologies to grip irregularly shaped workpieces has the problem of poor flexibility. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, readable storage medium, and program product for gripping irregularly shaped workpieces that can improve flexibility in addressing the aforementioned technical problems.

[0005] Firstly, this application provides a method for gripping irregularly shaped workpieces, including:

[0006] Acquire point cloud information of irregularly shaped workpieces;

[0007] From multiple candidate workpiece contours, determine the target workpiece contour that matches the workpiece point cloud information;

[0008] The gripping point determination method corresponding to the target workpiece contour is invoked to analyze the workpiece point cloud information and determine the gripping point of the irregular workpiece.

[0009] By adjusting the gripper's operating parameters to match the gripping point, the gripper is controlled to grip the irregularly shaped workpiece.

[0010] In one embodiment, the method further includes:

[0011] Obtain multiple candidate workpieces;

[0012] For each candidate workpiece, view capture is performed on the candidate workpiece to obtain point cloud information of the candidate workpiece from multiple perspectives;

[0013] For each point cloud information, point cloud projection is performed on the point cloud information to obtain the mask information corresponding to the point cloud information;

[0014] Contour extraction is performed on each of the mask information to obtain the candidate workpiece contour.

[0015] In one embodiment, the target workpiece contour comprises two intersecting planes with similar areas;

[0016] The step of calling the gripping point determination method corresponding to the target workpiece contour to analyze the workpiece point cloud information and determine the gripping points of the irregular workpiece includes:

[0017] From the workpiece point cloud information, a first planar point cloud and a second planar point cloud are determined; the first plane represented by the first planar point cloud intersects with the second plane represented by the second planar point cloud and their areas are similar;

[0018] Based on the first planar point cloud and the second planar point cloud, determine the intersection line of the first plane and the second plane;

[0019] The gripping point of the irregular workpiece is determined based on the intersection line; the gripping point is on the intersection line, or the distance between the gripping point and the intersection line satisfies the proximity condition of being less than or equal to a distance threshold.

[0020] In one embodiment, determining the gripping point of the irregularly shaped workpiece based on the intersection line includes:

[0021] From the workpiece point cloud information, target point clouds whose distance to the intersection line is less than or equal to a distance threshold are selected;

[0022] The centroid of the point cloud cluster formed by the target point cloud is used as the gripping point of the irregular workpiece.

[0023] In one embodiment, determining the target workpiece contour that matches the workpiece point cloud information from a plurality of candidate workpiece contours includes:

[0024] Contour recognition is performed on the point cloud information of the workpiece to obtain the contour features of the irregularly shaped workpiece;

[0025] From multiple candidate workpiece contours, a target workpiece contour that matches the workpiece contour features is determined.

[0026] In one embodiment, acquiring the workpiece point cloud information collected for the irregularly shaped workpiece includes:

[0027] A line scan is performed on the area where the irregularly shaped workpiece is located to obtain the regional point cloud information of the area;

[0028] Acquire illumination information for the area;

[0029] Based on the grayscale features of the illumination information, the point cloud information belonging to the regional background in the regional point cloud information is removed to obtain the workpiece point cloud information of the irregular workpiece.

[0030] Secondly, this application also provides an irregularly shaped workpiece gripping device, comprising:

[0031] The workpiece point cloud information acquisition module is used to acquire workpiece point cloud information collected for irregularly shaped workpieces.

[0032] The target workpiece contour determination module is used to determine the target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours.

[0033] The gripping point determination module is used to call the gripping point determination method corresponding to the contour of the target workpiece, analyze the point cloud information of the workpiece, and determine the gripping points of the irregular workpiece.

[0034] The gripping module is used to control the grippers to grip the irregular workpiece by means of gripper working parameters adapted to the gripping point.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0036] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described above.

[0037] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0038] The aforementioned method, apparatus, computer equipment, readable storage medium, and program product for grasping irregularly shaped workpieces can completely preserve their three-dimensional geometric features by acquiring point cloud information of the workpieces, avoiding matching errors caused by information loss in traditional methods. It automatically matches the target workpiece contour from multiple candidate contours, significantly improving the algorithm's adaptability to workpieces of different shapes. By calling the grasping point determination method corresponding to the target contour and combining it with the analysis of local geometric features of the workpiece to determine grasping stability, different grasping point determination methods can be determined for workpieces of different shapes, thus solving the problem of poor generalization of traditional manual annotation methods. Finally, by adapting the gripper's working parameters, it achieves precise grasping, effectively reducing workpiece slippage or damage caused by improper grasping point selection, while reducing reliance on manual intervention and greatly improving the flexibility of grasping irregularly shaped workpieces. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is an application environment diagram of an irregular workpiece gripping method in one embodiment;

[0041] Figure 2 This is a flowchart illustrating a method for gripping irregularly shaped workpieces in one embodiment;

[0042] Figure 3 This is a flowchart illustrating the method for gripping irregularly shaped workpieces in another embodiment;

[0043] Figure 4 This is a structural block diagram of an irregularly shaped workpiece gripping device in one embodiment;

[0044] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] The irregular workpiece gripping method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, server 102 communicates with grasping robot 104 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102, or it can be located in the cloud or on other network servers. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Grasping robot 104 is an industrial or service robot specifically designed for automatically identifying, locating, grasping, and moving objects. Its core function is to replace manual labor in performing repetitive, high-precision, or hazardous grasping tasks through the collaborative operation of a robotic arm and an end effector. The end effector can be, for example, a gripper. Specifically, during the process of grasping irregularly shaped workpieces, server 102 acquires workpiece point cloud information collected for the irregularly shaped workpieces; from multiple candidate workpiece contours, it determines the target workpiece contour that matches the workpiece point cloud information; it calls the grasping point determination method corresponding to the target workpiece contour to analyze the workpiece point cloud information and determine the grasping point of the irregularly shaped workpiece; and it controls the gripper to grasp the irregularly shaped workpiece through the gripper working parameters adapted to the grasping point.

[0047] In one exemplary embodiment, such as Figure 2 As shown, a method for gripping irregularly shaped workpieces is provided, which can be applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps S202 to S208. Wherein:

[0048] Step S202: Obtain the workpiece point cloud information collected for the irregularly shaped workpiece.

[0049] Among them, irregularly shaped workpieces are industrial parts or products with irregular shapes and non-standard geometries that cannot be directly described by simple mathematical models. The workpiece point cloud information is a set of three-dimensional coordinates of the object surface collected by scanning equipment such as lidar and structured light cameras. Each point contains (x, y, z) coordinates and possible reflection intensity or color data.

[0050] Specifically, in practice, the irregularly shaped workpiece must first be fixed on the scanning platform to ensure its stable position and avoid motion artifacts. The server can control the scanning device to emit lasers or projected stripes from multiple angles, and calculate the three-dimensional coordinates of surface points by receiving reflected light or deformed stripes. Since irregularly shaped workpieces may have complex curved surfaces or grooves, a multi-view scanning strategy is required, such as rotating the workpiece or moving the scanner to achieve 360° coverage. The acquired raw point cloud usually contains noise, which can be removed using statistical filtering. The final generated point cloud must retain the key geometric features of the workpiece, and data integrity must be checked using visualization tools to ensure there are no missing or deformed areas.

[0051] Step S204: Determine the target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours.

[0052] The candidate workpiece contour is a predefined set of various workpiece geometric models used for comparison with the actual acquired point cloud. For example, the candidate workpiece contour may include a CAD (Computer-Aided Design) model or a point cloud template.

[0053] Specifically, the server needs to perform a 3D matching comparison between the real-time acquired workpiece point cloud information and each candidate contour in the contour library. For example, this can be achieved through point cloud registration algorithms or feature matching techniques. Optionally, the server can perform contour recognition on the workpiece point cloud information to obtain the workpiece contour features of the irregularly shaped workpiece. Then, through spatial transformations such as rotation and translation, it can find the optimal alignment position, determine the target workpiece contour that matches the workpiece contour features from multiple candidate workpiece contours, calculate the geometric similarity or error distance between the workpiece contour features and each candidate contour, and finally select the candidate contour with the highest similarity and the smallest residual as the successfully matched target workpiece contour, thereby identifying the specific type and posture of the current workpiece. Optionally, the server can also directly determine the target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours based on the workpiece point cloud information. The matching process can preprocess the point cloud, such as denoising, downsampling, and normal estimation, and determine the target workpiece contour that matches the workpiece point cloud information based on the degree of matching between each point cloud in each workpiece point cloud information and the candidate workpiece contours.

[0054] Step S206: Call the gripping point determination method corresponding to the target workpiece contour, analyze the workpiece point cloud information, and determine the gripping points of the irregular workpiece.

[0055] The gripping point determination method is a pre-defined gripping strategy for workpieces of different shapes. The gripping point is the specific position where the robotic arm's gripper contacts the workpiece, and it must meet the requirements of gripping stability and task requirements.

[0056] Specifically, after successfully matching the target contour, the server can select the appropriate gripping point determination method based on the contour's preset strategy library. For example, for an irregularly shaped workpiece with a protruding handle, the gripping point determination method could be to find the midpoint of the thinnest part of the neck of the workpiece as the gripping point; for a flat irregularly shaped workpiece, its center of gravity can be calculated and used as the gripping point. As another example, for an irregularly shaped workpiece comprising two intersecting planes with similar areas, the gripping point determination method could be to determine the first and second plane point clouds from the workpiece point cloud information, determine the intersection line of the first and second planes based on the first and second plane point clouds, and determine the gripping point of the irregularly shaped workpiece based on the intersection line; the gripping point lies on the intersection line, or the distance between the gripping point and the intersection line satisfies the proximity condition of being less than or equal to a distance threshold, wherein the first plane represented by the first plane point cloud intersects with the second plane represented by the second plane point cloud and has similar areas.

[0057] Step S208: Control the gripper to grip the irregular workpiece by using gripper working parameters adapted to the gripping point.

[0058] Among them, the gripper working parameters are the specific control commands that the gripper executes when grasping. For example, the gripper working parameters may include gripper angle, opening and closing distance, force, movement speed, etc.

[0059] Specifically, the server can plan the robotic arm's movement path based on the determined gripping point information, enabling the gripper to safely move to a predetermined position above the gripping point and adjust the gripping angle. Before contacting the workpiece, the gripper prepares with a preset opening width, then closes at a specified speed and force, ensuring that the gripper tips accurately contact the gripping point and apply sufficient holding force to prevent the workpiece from slipping or falling off. Furthermore, throughout the gripping process, parameters can be fine-tuned using sensor data from force sensors or visual feedback to ensure smooth and reliable gripping action, ultimately achieving stable pickup of irregularly shaped workpieces and preparing for subsequent handling, assembly, or processing operations.

[0060] The aforementioned method for grasping irregularly shaped workpieces can completely preserve their three-dimensional geometric features by acquiring point cloud information of the workpieces, avoiding matching errors caused by information loss in traditional methods. It automatically matches the target workpiece contour from multiple candidate contours, significantly improving the algorithm's adaptability to workpieces of different shapes. By calling the grasping point determination method corresponding to the target contour and combining it with the analysis of local geometric features of the workpiece to determine grasping stability, different grasping point determination methods can be determined for workpieces of different shapes, thus solving the problem of poor generalization of traditional manual annotation methods. Finally, by adapting the gripper's working parameters, it achieves precise grasping, effectively reducing workpiece slippage or damage caused by improper grasping point selection, while reducing reliance on manual intervention and greatly improving the flexibility of grasping irregularly shaped workpieces.

[0061] In an exemplary embodiment, the method further includes: acquiring multiple candidate workpieces; for each candidate workpiece, performing view cropping on the candidate workpiece to obtain point cloud information of the candidate workpiece from multiple viewpoints; for each point cloud information, performing point cloud projection on the point cloud information to obtain mask information corresponding to the point cloud information; and extracting the contour of each mask information to obtain the candidate workpiece contour.

[0062] Candidate workpieces refer to workpieces of various known types and shapes that have appeared in historical production or identification tasks. View capture refers to obtaining surface information fragments of the workpiece from a virtual 3D model or an actual scanned point cloud by setting different virtual camera angles under different viewing directions. Point cloud projection projects point cloud data in 3D space onto a 2D plane to generate a 2D depth map or height map. Mask information refers to a binary image in which the workpiece area is marked with a specific value (such as 1 or 255), and the background area is marked with another value (such as 0) to distinguish the object from the background. Contour extraction extracts the boundary pixel sequence of the target area from the binary mask image using algorithms such as edge detection to form a 2D contour.

[0063] Specifically, first, all possible irregularly shaped workpiece types are selected as candidate workpieces. For each candidate workpiece, to generate its comprehensive contour description, the server simulates viewing the workpiece from multiple different perspectives. For example, in a virtual environment, a series of uniformly distributed or key-angle virtual cameras are set around the 3D model of the candidate workpiece, and then fragments of point cloud data of the workpiece surface "seen" from each perspective are acquired. Next, for each point cloud fragment from each perspective, the server projects it perpendicularly onto a 2D plane along the line of sight of that perspective. After projection, a binary image, i.e., a mask, is generated based on whether a point exists at a certain pixel position. White pixels (or values ​​of 1) represent workpiece point cloud projection at that position, while black pixels (or values ​​of 0) represent no point cloud (background). Finally, image processing algorithms such as edge detection are applied to this binary mask to extract the 2D outer contour line of the candidate workpiece from that perspective. In this way, each candidate workpiece obtains a set of 2D contour lines extracted from multiple perspectives. These contour sets constitute the "candidate workpiece contour" representation of the workpiece in the contour library, laying the foundation for subsequent rapid matching with real scanned workpieces.

[0064] In this embodiment, by extracting the two-dimensional contour of the standard workpiece from multiple perspectives, a rich and multi-angle contour template is established. This greatly improves the flexibility and accuracy of subsequent matching with the real scanned workpiece and can cope with various placement postures of the workpiece in space.

[0065] In an exemplary embodiment, the target workpiece contour includes two intersecting planes with similar areas. The gripping point determination method corresponding to the target workpiece contour is invoked to analyze the workpiece point cloud information and determine the gripping points of the irregularly shaped workpiece. This includes: determining a first plane point cloud and a second plane point cloud from the workpiece point cloud information; the first plane represented by the first plane point cloud intersects with the second plane represented by the second plane point cloud and has similar areas; determining the intersection line of the first plane and the second plane based on the first plane point cloud and the second plane point cloud; determining the gripping points of the irregularly shaped workpiece based on the intersection line; the gripping point is on the intersection line, or the distance between the gripping point and the intersection line satisfies the proximity condition of being less than or equal to a distance threshold.

[0066] Here, the first plane point cloud and the second plane point cloud refer to the subsets of point cloud data segmented from the entire workpiece point cloud, belonging to these two specific planes respectively. The intersection line is the straight line formed by the intersection of two planes in three-dimensional space. The proximity condition is a spatial distance constraint that requires the grab point to be located on the intersection line or very close to it.

[0067] Specifically, when the server matches the target contour feature of the current irregularly shaped workpiece as "two intersecting planes with similar areas," it will invoke a gripping point determination method specifically designed for this type of geometric feature. First, the server needs to identify and separate the two most important planar point cloud sets—the first and second planar point clouds—from the acquired complete workpiece point cloud using a planar segmentation algorithm. It then verifies whether these two planes actually intersect and whether the estimated planar areas of their point clouds are approximately equivalent. After confirmation, the server calculates their spatial intersection line based on the normal vectors and plane equations of these two planes. This intersection line is usually structurally robust and serves as an ideal reference for the gripping position. The principle for determining the gripping point is that the gripping point must be on this intersection line, or although not on the line, the vertical distance to the intersection line must be less than a pre-set threshold, ensuring that the gripping position is adjacent to this structurally reinforced area. The purpose of this is to ensure that the gripping force of the jaws acts on or near this robust part of the workpiece, avoiding applying force to thin-walled or suspended parts that could damage the workpiece or cause unstable gripping.

[0068] In this embodiment, a precise and physically reasonable method for calculating gripping points is provided for workpieces with specific geometric structures. By utilizing the inherent geometric features of the workpiece to determine the gripping position, the stability and reliability of the gripping can be ensured, effectively preventing the workpiece from sliding or tipping over during the gripping process. This method is suitable for handling irregularly shaped workpieces with angular structures.

[0069] In an exemplary embodiment, determining the gripping point of an irregularly shaped workpiece based on the intersection line includes: filtering target point clouds from the workpiece point cloud information whose distance from the intersection line is less than or equal to a distance threshold; and using the centroid of the point cloud cluster formed by the target point clouds as the gripping point of the irregularly shaped workpiece.

[0070] In this context, the target point cloud refers to the 3D points selected from the entire workpiece point cloud that are sufficiently close to the intersection line. A point cloud cluster is a group of target point clouds that are spatially aggregated. The centroid is the geometric center of all points in this point cloud cluster, and its coordinates are calculated from the arithmetic mean of the coordinates of all points.

[0071] Specifically, based on the already calculated intersection lines of the two planes, this embodiment further refines the specific calculation steps for the gripping point. The server doesn't just take a theoretical point on the intersection line, but considers the actual situation of the point cloud. Using the calculated spatial intersection line as a reference, it sets a small neighborhood distance threshold, then traverses the entire workpiece point cloud, filtering out all points whose spatial distance to this intersection line is less than or equal to the threshold; these points are called the target point cloud. Since the point cloud is discrete, these target point clouds will form a strip-shaped set of points along the intersection line in three-dimensional space, i.e., a "point cloud cluster." Next, the server calculates the average position of all points in this point cloud cluster, which is the centroid of the cluster. This centroid is ultimately determined as the gripping point for the irregularly shaped workpiece. The advantage of this method is that it determines a balanced gripping center position based on the actual point cloud data distribution, rather than relying on theoretical geometric calculations that may be sensitive to noise. Simultaneously, because it utilizes a region near the intersection line, the calculated centroid has better robustness to small noise or measurement errors in the point cloud.

[0072] In this embodiment, by calculating the centroid of the point cloud cluster near the intersection line as the grasping point, the influence of point cloud noise can be effectively smoothed, making the determined grasping point position more stable and reliable, and improving the adaptability of the automated grasping system in actual noisy environments.

[0073] In an exemplary embodiment, determining a target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours includes: performing contour recognition on the workpiece point cloud information to obtain the workpiece contour features of the irregular workpiece; and determining a target workpiece contour that matches the workpiece contour features from multiple candidate workpiece contours.

[0074] Contour recognition refers to the process of extracting the overall or significant shape boundary features from the input point cloud information of a workpiece using algorithms. Workpiece contour features refer to the quantitative or structured information obtained through the contour recognition process that describes the unique shape of the workpiece, such as the distribution of contour points, curvature, and key corner points.

[0075] Specifically, first, the server needs to process the real-time acquired workpiece point cloud information to extract workpiece contour features that represent its shape. For example, the 3D point cloud can be projected onto one or several main viewpoints on a 2D plane to form a 2D contour, or global contour features can be extracted directly from the 3D point cloud. Then, the server needs to compare these extracted workpiece contour features with each candidate workpiece contour in the contour library, calculating the similarity or distance between their feature vectors. Finally, the server selects the candidate workpiece contour that is most similar to the current workpiece contour features, i.e., the one with the highest matching degree, and determines it as the target workpiece contour for this operation.

[0076] In this embodiment, by comparing the contour features of the real-time workpiece with the pre-stored template library, the type of workpiece and the corresponding contour template can be quickly and accurately identified, providing a key basis for subsequently calling the correct grasping strategy.

[0077] In an exemplary embodiment, obtaining workpiece point cloud information collected for an irregularly shaped workpiece includes: performing a line scan on the area where the irregularly shaped workpiece is located to obtain the area point cloud information; obtaining the illumination information collected for the area; and removing point cloud information belonging to the background of the area from the area point cloud information based on the grayscale features of the illumination information to obtain the workpiece point cloud information of the irregularly shaped workpiece.

[0078] Line scanning here refers to scanning using a line laser scanner. The device emits a laser line projected onto the object's surface, and a camera captures the deformation of the laser line to quickly obtain the 3D coordinates of points along the line. By moving the scanner or object, a point cloud of the entire area is obtained. Area point cloud information refers to all the 3D point data obtained after scanning the entire target area. The entire target area includes the workpiece and its background. Illumination information typically refers to color image information acquired by a camera from the same viewpoint, including color and brightness data. Grayscale features refer to the brightness values ​​of pixels in the image. Objects of different materials and colors reflect light differently, resulting in different grayscale levels in the image. The area background refers to the parts of the scanned area that are not part of the target irregularly shaped workpiece, such as workbenches, conveyor belts, and fixtures.

[0079] Specifically, a line laser scanner is used to scan the entire area containing the irregularly shaped workpiece, quickly obtaining a dense point cloud of information, including both the workpiece's own point cloud and the point clouds of background objects. To separate the workpiece, the server simultaneously acquires the illumination information of the same area, i.e., a grayscale image. Since the workpiece and background typically have different materials and colors, their grayscale values ​​in the image will differ significantly. The server utilizes this grayscale feature, by setting a threshold or using a more advanced image segmentation algorithm, to distinguish the workpiece area and the background area in the image, generating a binary mask. Then, this image mask is mapped into 3D space and registered and associated with the regional point cloud information. Using the mask information, the server can filter and remove point cloud data in the regional point cloud that correspond to background pixels in the image. The final set of 3D points belonging only to the workpiece is the workpiece point cloud information. This method effectively solves the problem of directly extracting the 3D model of a workpiece from a cluttered background.

[0080] In this embodiment, by fusing the illumination information of 3D line scan point cloud and 2D image, the target workpiece can be effectively and accurately segmented from the complex background, obtaining high-quality, interference-free workpiece point cloud data. This provides a clean and reliable input for subsequent contour matching and grasping point calculation, improving the practicality of the entire system in real industrial environments.

[0081] In one specific embodiment, a method for grasping irregularly shaped workpieces is also provided. First, the CAD model is loaded offline. For each candidate workpiece, a 42-view point cloud is generated using the OpenGL view capture function. The point cloud is projected to generate a 2D mask image. The contour of the mask image is extracted to obtain the contour of the candidate workpiece. The contour is named with a workpiece ID agreed upon by the user. The contour of the candidate workpiece is saved offline as a matching template. Next, the center of the irregularly shaped workpiece is aligned with the center of the gripper. During calibration, the workpiece is first pointed to the center of the calibration plate in a fixed posture, and then rotated 180 degrees around the robot's Z-axis before pointing to the same center. This process is repeated to mark the required number of calibration points. The workpiece is then scanned by a 3D camera using a large field-of-view scanning device on the hand, obtaining illumination information and point cloud information for areas supporting neighborhood queries. The pixel positions in the illumination information correspond one-to-one with the point cloud coordinates to accelerate matching speed and accuracy. The background is segmented based on the grayscale values ​​in the illumination information, and a mask image is generated for each workpiece from the depth values ​​in the point cloud. After contour extraction and filtering of the mask image, the contour image of the workpiece to be grasped is obtained, i.e., the contour of the irregularly shaped workpiece. All imported candidate workpiece contours are read offline, and the irregularly shaped workpiece contour is matched with each candidate contour using the LineModel algorithm. The result with the highest matching score is selected, and the template ID is assigned to the current irregularly shaped workpiece to be grasped. Then, based on the workpiece ID calculated from the template, different gripping point determination methods are invoked. For irregularly shaped workpieces comprising two intersecting planes with similar areas, planar extraction is first used to obtain two planar point clouds. The intersection line of the two planar point clouds is then calculated. From the workpiece point cloud information, target point clouds with a distance less than or equal to a distance threshold from the intersection line are selected. The centroid of the point cloud cluster formed by the target point clouds is used as the gripping point of the irregularly shaped workpiece. Finally, rotation angle limitation processing is applied to the gripper and the irregularly shaped workpiece. Based on the relative pose of the workpiece and the gripper, it can be determined that the workpiece posture change only occurs around one of the X, Y, and Z axes of the gripper. The angular component of the workpiece direction vector and the remaining axis is the workpiece gripping angle.

[0082] In a specific embodiment, such as Figure 3 As shown, a method for gripping irregularly shaped workpieces is also provided, including:

[0083] Step S301: Obtain multiple candidate workpieces;

[0084] Step S302: For each candidate workpiece, perform view cropping on the candidate workpiece to obtain point cloud information of the candidate workpiece from multiple perspectives.

[0085] Step S303: For each point cloud information, perform point cloud projection on the point cloud information to obtain the mask information corresponding to the point cloud information.

[0086] Step S304: Extract the contour of each mask information to obtain the candidate workpiece contour.

[0087] Step S305: Perform a line scan on the area where the irregular workpiece is located to obtain the area point cloud information of the area;

[0088] Step S306: Obtain illumination information for the area;

[0089] Step S307: Based on the grayscale features of the illumination information, remove the point cloud information belonging to the background of the region from the regional point cloud information to obtain the workpiece point cloud information of the irregular workpiece.

[0090] Step S308: Perform contour recognition on the workpiece point cloud information to obtain the workpiece contour features of the irregular workpiece.

[0091] Step S309: Determine the target workpiece contour that matches the workpiece contour features from multiple candidate workpiece contours.

[0092] The target workpiece contour includes two intersecting planes with similar areas;

[0093] Step S310: Determine the first planar point cloud and the second planar point cloud from the workpiece point cloud information;

[0094] Among them, the first plane represented by the first plane point cloud intersects with the second plane represented by the second plane point cloud and has similar areas;

[0095] Step S311: Based on the first planar point cloud and the second planar point cloud, determine the intersection line of the first plane and the second plane;

[0096] Step S312: Filter the target point cloud from the workpiece point cloud information. The distance between the target point cloud and the intersection line is less than or equal to the distance threshold.

[0097] Step S313: Use the centroid of the point cloud cluster formed by the target point cloud as the gripping point of the irregular workpiece.

[0098] Among them, the grab point is on the intersection line, or the distance between the grab point and the intersection line satisfies the proximity condition of being less than or equal to the distance threshold;

[0099] Step S314: Control the gripper to grip the irregular workpiece by using gripper working parameters adapted to the gripping point.

[0100] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0101] Based on the same inventive concept, this application also provides an irregularly shaped workpiece gripping device for implementing the aforementioned irregularly shaped workpiece gripping method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the irregularly shaped workpiece gripping device provided below can be found in the limitations of the irregularly shaped workpiece gripping method described above, and will not be repeated here.

[0102] In one exemplary embodiment, such as Figure 4 As shown, an irregularly shaped workpiece gripping device 400 is provided, including: a workpiece point cloud information acquisition module 402, a target workpiece contour determination module 404, a gripping point determination module 406, and a gripping module 408, wherein:

[0103] The workpiece point cloud information acquisition module 402 is used to acquire workpiece point cloud information collected for irregularly shaped workpieces.

[0104] The target workpiece contour determination module 404 is used to determine the target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours.

[0105] The gripping point determination module 406 is used to call the gripping point determination method corresponding to the target workpiece contour, analyze the workpiece point cloud information, and determine the gripping points of the irregular workpiece.

[0106] The gripping module 408 is used to control the grippers to grip irregularly shaped workpieces by means of gripper working parameters adapted to the gripping point.

[0107] In one exemplary embodiment, the irregularly shaped workpiece gripping device 400 further includes a contour extraction module, specifically used for:

[0108] Obtain multiple candidate workpieces;

[0109] For each candidate workpiece, view capture is performed on the candidate workpiece to obtain point cloud information of the candidate workpiece from multiple perspectives;

[0110] For each point cloud information, point cloud projection is performed to obtain the mask information corresponding to the point cloud information;

[0111] Contour extraction is performed on each mask information to obtain the candidate workpiece contour.

[0112] In one exemplary embodiment, the target workpiece contour comprises two intersecting planes with similar areas. In this embodiment, the gripping point determination module 406 includes:

[0113] The planar point cloud determination unit is used to determine a first planar point cloud and a second planar point cloud from the workpiece point cloud information; the first plane represented by the first planar point cloud intersects with the second plane represented by the second planar point cloud and the areas are similar.

[0114] The intersection line determination unit is used to determine the intersection line of the first plane and the second plane based on the first plane point cloud and the second plane point cloud;

[0115] The gripping point determination unit is used to determine the gripping point of the irregular workpiece based on the intersection line; the gripping point is on the intersection line, or the distance between the gripping point and the intersection line satisfies the proximity condition of being less than or equal to the distance threshold.

[0116] In one exemplary embodiment, the grasp point determination unit is specifically used for:

[0117] From the workpiece point cloud information, filter out target point clouds whose distance to the intersection line is less than or equal to a distance threshold;

[0118] The centroid of the point cloud cluster formed by the target point cloud is used as the gripping point of the irregular workpiece.

[0119] In one exemplary embodiment, the target workpiece contour determination module 404 is specifically used for:

[0120] Contour recognition is performed on the point cloud information of the workpiece to obtain the contour features of the irregularly shaped workpiece.

[0121] From multiple candidate workpiece contours, determine the target workpiece contour that matches the workpiece contour features.

[0122] In an exemplary embodiment, the workpiece point cloud information acquisition module 402 is specifically used for:

[0123] Line scanning is performed on the area where the irregularly shaped workpiece is located to obtain the area point cloud information;

[0124] Acquire illumination information for the targeted area;

[0125] Based on the grayscale features of illumination information, point cloud information belonging to the regional background is removed from the regional point cloud information to obtain the workpiece point cloud information of the irregular workpiece.

[0126] Each module in the aforementioned irregular workpiece gripping device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0127] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for grasping irregularly shaped workpieces. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0128] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0131] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0135] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for gripping irregularly shaped workpieces, characterized in that, The method includes: Acquire point cloud information of irregularly shaped workpieces; From multiple candidate workpiece contours, a target workpiece contour that matches the workpiece point cloud information is determined; the target workpiece contour includes two intersecting planes with similar areas. From the workpiece point cloud information, a first planar point cloud and a second planar point cloud are determined; the first plane represented by the first planar point cloud intersects with the second plane represented by the second planar point cloud and their areas are similar; Based on the first planar point cloud and the second planar point cloud, determine the intersection line of the first plane and the second plane; From the workpiece point cloud information, target point clouds whose distance to the intersection line is less than or equal to a distance threshold are selected; The centroid of the point cloud cluster formed by the target point cloud is used as the gripping point of the irregular workpiece; the gripping point is on the intersection line, or the distance between the gripping point and the intersection line satisfies the proximity condition of being less than or equal to a distance threshold. By adjusting the gripper's operating parameters to match the gripping point, the gripper is controlled to grip the irregularly shaped workpiece.

2. The method according to claim 1, characterized in that, The method further includes: Obtain multiple candidate workpieces; For each candidate workpiece, view capture is performed on the candidate workpiece to obtain point cloud information of the candidate workpiece from multiple perspectives; For each point cloud information, point cloud projection is performed on the point cloud information to obtain the mask information corresponding to the point cloud information; Contour extraction is performed on each of the mask information to obtain the candidate workpiece contour.

3. The method according to claim 1, characterized in that, The step of determining the target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours includes: Contour recognition is performed on the point cloud information of the workpiece to obtain the contour features of the irregularly shaped workpiece; From multiple candidate workpiece contours, a target workpiece contour that matches the workpiece contour features is determined.

4. The method according to claim 3, characterized in that, The acquisition of workpiece point cloud information for irregularly shaped workpieces includes: A line scan is performed on the area where the irregularly shaped workpiece is located to obtain the regional point cloud information of the area; Acquire illumination information for the area; Based on the grayscale features of the illumination information, the point cloud information belonging to the regional background in the regional point cloud information is removed to obtain the workpiece point cloud information of the irregular workpiece.

5. A gripping device for irregularly shaped workpieces, characterized in that, The apparatus used in the irregular workpiece gripping method of claim 1 includes: The workpiece point cloud information acquisition module is used to acquire workpiece point cloud information collected for irregularly shaped workpieces; The target workpiece contour determination module is used to determine the target workpiece contour that matches the workpiece point cloud information from multiple candidate workpiece contours; the target workpiece contour includes two intersecting planes with similar areas. The capture point determination module includes: A planar point cloud determination unit is used to determine a first planar point cloud and a second planar point cloud from the workpiece point cloud information; the first plane represented by the first planar point cloud intersects with the second plane represented by the second planar point cloud and the areas are similar. An intersection line determination unit is used to determine the intersection line between the first plane and the second plane based on the first planar point cloud and the second planar point cloud; The gripping point determination unit is used to filter target point clouds from the workpiece point cloud information, where the distance between the target point cloud and the intersection line is less than or equal to a distance threshold; and to use the centroid of the point cloud cluster formed by the target point clouds as the gripping point of the irregular workpiece; the gripping point is on the intersection line, or the distance between the gripping point and the intersection line satisfies the proximity condition of being less than or equal to the distance threshold. The gripping module is used to control the grippers to grip the irregular workpiece by means of gripper working parameters adapted to the gripping point.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images