A method, system and device for pose estimation of randomly stacked workpieces in disorderly grasping

Abstract

The application discloses a kind of disordered grabbing in scattered workpiece pose estimation method, system and equipment, it is related to industrial automation field, comprising: using YOLACT model to detect and segment the two-dimensional image of scattered workpiece, obtain the category and mask of multiple to-be-grabbed workpieces;According to the mask of multiple to-be-grabbed workpieces, using structured light binocular stereo mechanism to build sparse point cloud;The point pair feature of the sparse point cloud of each to-be-grabbed workpiece is calculated;For each to-be-grabbed workpiece, according to the category of to-be-grabbed workpiece, select the template point pair feature corresponding to the workpiece of this category in template point pair feature library, carry out pose coarse registration based on voting matching with the point pair feature of the sparse point cloud of to-be-grabbed workpiece, then pose precision registration is carried out using iterative closest point algorithm, obtain the final pose of to-be-grabbed workpiece;Final pose is used for to-be-grabbed workpiece to be grabbed by mechanical arm after pose inspection.The application can improve the success rate of grabbing while reducing total system time consumption.

Description

Technical Field

[0001] This invention relates to the field of industrial automation technology, and in particular to a method, system and device for estimating the pose of scattered workpieces during disordered grasping. Background Technology

[0002] To enable robots to successfully complete tasks in a disordered and unknown environment, a perception system is essential to endow them with intelligence. Existing environmental perception systems mainly include 2D vision systems, 3D vision systems, and LiDAR. Among them, intelligent robots based on 3D vision systems can be applied in many fields, with mature applications including disordered grasping, robotic welding, and defect detection. In industrial scenarios, disordered grasping tasks typically involve workpieces without texture and in environments with relatively complex lighting conditions. This places relatively high demands on the 3D reconstruction and pose registration accuracy of the disordered grasping system.

[0003] Applying unordered grasping systems to real-world industrial environments currently faces two main challenges: accuracy and speed. Global scholars have conducted extensive research on these two aspects. Generally, methods with high accuracy are slower, while faster methods have lower accuracy. There is still room for improvement in the integration of accuracy and speed. Therefore, for unordered grasping tasks in industrial scenarios, how to improve the success rate of grasping while reducing the total system time has become a pressing problem for those skilled in the art. Summary of the Invention

[0004] To address the problems mentioned in the background art, the present invention provides a method, system, and device for estimating the pose of scattered workpieces in disordered grasping, so as to improve the grasping success rate while reducing the total system time and solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] On one hand, the present invention provides a method for estimating the pose of scattered workpieces during disordered grasping, the method comprising:

[0007] Workpiece detection data is generated and applied to the training of the YOLACT model to obtain a trained YOLACT model. The workpiece detection data includes multiple labeled two-dimensional images of scattered workpieces. Each two-dimensional image of scattered workpieces includes multiple workpieces. The two-dimensional images of scattered workpieces are labeled with the category and mask of each workpiece.

[0008] Establish a workpiece template point cloud library; the workpiece template point cloud library includes template point clouds corresponding to different categories of workpieces;

[0009] Calculate the point pair features of each template point cloud in the workpiece template point cloud library to establish a template point pair feature library; the template point pair feature library includes template point pair features corresponding to different categories of workpieces; the template point pair features are point pair features of template point clouds;

[0010] Obtain a two-dimensional image of the workpiece to be estimated;

[0011] The trained YOLACT model is used to detect and segment the two-dimensional image of the scattered workpiece to be estimated, and multiple categories and masks of the workpiece to be grasped are obtained.

[0012] Based on the masks of multiple workpieces to be grasped, a sparse point cloud of multiple workpieces to be grasped is constructed using a structured light binocular camera;

[0013] Calculate the point-pair features of the sparse point cloud of each of the workpieces to be grasped;

[0014] For each workpiece to be grasped, based on the category of the workpiece, template point pair features corresponding to the workpiece of the specified category are selected from the template point pair feature library. The point pair features of the sparse point cloud of the workpiece to be grasped are coarsely registered with the template point pair features corresponding to the workpiece of the specified category using a voting matching method to obtain the initial pose of the workpiece to be grasped. Based on the initial pose, an iterative nearest point algorithm is used for fine pose registration to obtain the final pose of the workpiece to be grasped. The final pose, after passing pose verification, is used by the robotic arm to grasp the workpiece.

[0015] Optionally, establishing the workpiece template point cloud library specifically includes:

[0016] Structured light binocular cameras are used to construct template point clouds for different types of workpieces;

[0017] Based on the template point clouds corresponding to different categories of workpieces, a workpiece template point cloud library is obtained.

[0018] Optionally, the step of constructing a sparse point cloud of multiple workpieces to be grasped using a structured light binocular camera based on multiple masks of the workpieces to be grasped specifically includes:

[0019] The Zhang Zhengyou camera calibration method was used to calibrate the structured light binocular camera;

[0020] Based on the masks of multiple workpieces to be grasped, a calibrated structured light binocular camera is used to perform stereo matching using a combination of Gray code and phase shift code encoding and decoding, and an initial sparse point cloud of multiple workpieces to be grasped is generated using a fixed interval skip sampling method.

[0021] The initial sparse point clouds of the multiple workpieces to be grasped are smoothed and denoised using moving least squares filtering to obtain the sparse point clouds of the multiple workpieces to be grasped.

[0022] Optionally, the step of calculating the point-pair features of the sparse point cloud of each of the workpieces to be grasped further includes:

[0023] The sparse point cloud of each of the workpieces to be grasped is preprocessed; the preprocessing includes downsampling and point cloud smoothing.

[0024] On the other hand, the present invention also provides a system for estimating the pose of scattered workpieces during disordered grasping, the system comprising:

[0025] The training module of the object detection network is used to generate workpiece detection data and apply the workpiece detection data to train the YOLACT model to obtain a trained YOLACT model. The workpiece detection data includes multiple labeled two-dimensional images of scattered workpieces. Each two-dimensional image of scattered workpieces includes multiple workpieces. The two-dimensional images of scattered workpieces are labeled with the category and mask of each workpiece.

[0026] The template point cloud generation module is used to establish a workpiece template point cloud library; the workpiece template point cloud library includes template point clouds corresponding to different categories of workpieces;

[0027] The template point-pair feature library management module is used to calculate the point-pair features of each template point cloud in the workpiece template point cloud library and establish the template point-pair feature library; the template point-pair feature library includes template point-pair features corresponding to different categories of workpieces; the template point-pair features are the point-pair features of the template point cloud;

[0028] The image acquisition module is used to acquire a two-dimensional image of the workpiece to be estimated.

[0029] The image detection and segmentation module is used to detect and segment the two-dimensional image of the scattered workpiece to be estimated using the trained YOLACT model, and obtain multiple categories and masks of the workpiece to be grasped.

[0030] A binocular structured light 3D point cloud local generation module is used to construct sparse point clouds of multiple workpieces to be grasped using a structured light binocular camera based on multiple masks of the workpieces to be grasped.

[0031] The sparse point cloud point pair feature calculation module is used to calculate the point pair features of the sparse point cloud of each of the workpieces to be grasped.

[0032] The pose registration and verification module is used to, for each workpiece to be grasped, select template point pair features corresponding to the workpiece of the specified category from the template point pair feature library according to the category of the workpiece to be grasped, perform coarse pose registration based on voting matching between the point pair features of the sparse point cloud of the workpiece to be grasped and the template point pair features corresponding to the workpiece of the specified category, to obtain the initial pose of the workpiece to be grasped, and perform fine pose registration based on the initial pose using an iterative nearest point algorithm to obtain the final pose of the workpiece to be grasped; the final pose, after passing pose verification, is used by the robotic arm to grasp the workpiece to be grasped.

[0033] Optionally, the template point cloud generation module specifically includes:

[0034] Template point cloud construction unit, used to construct template point clouds corresponding to different types of workpieces using a structured light binocular camera;

[0035] The workpiece template point cloud library creation unit is used to obtain the workpiece template point cloud library based on the template point clouds corresponding to different types of workpieces.

[0036] Optionally, the binocular structured light 3D point cloud local generation module specifically includes:

[0037] The camera calibration unit is used to calibrate the structured light binocular camera using Zhang Zhengyou's camera calibration method.

[0038] The initial sparse point cloud generation unit is used to generate the initial sparse point cloud of the multiple workpieces to be grasped based on the masks of the multiple workpieces to be grasped, using a calibrated structured light binocular camera, and employing a combination of Gray code and phase shift code encoding and decoding to perform stereo matching, and combining a fixed interval skip sampling method.

[0039] The smoothing and denoising unit is used to perform smoothing and denoising processing on the initial sparse point clouds of multiple workpieces to be grasped using moving least squares filtering, so as to obtain the sparse point clouds of multiple workpieces to be grasped.

[0040] Optionally, the system further includes:

[0041] The sparse point cloud preprocessing module is used to preprocess the sparse point clouds of each of the workpieces to be grasped; the preprocessing includes downsampling and point cloud smoothing.

[0042] In another aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for estimating the pose of scattered workpieces in disordered grasping.

[0043] Optionally, the memory is a non-transitory computer-readable storage medium.

[0044] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] This invention discloses a method, system, and device for estimating the pose of scattered workpieces in unordered grasping, primarily targeting unordered grasping tasks in industrial scenarios. It improves image detection and segmentation speed through YOLACT, generating point clouds of the workpieces to be grasped based on their mask information (not global point clouds), thus reducing point cloud reconstruction time to some extent and minimizing the influence of redundant point clouds from other non-object components in the environment, thereby improving the accuracy of the registration algorithm. Since multiple point clouds of the workpieces to be grasped are generated, image segmentation and workpiece point estimation are unnecessary after grasping the first workpiece. The cloud generation process allows for direct pose registration and verification (inspection) of the next workpiece, guiding the robotic arm to grasp it and thus improving the grasping speed of the entire system. Coarse pose matching based on voting is performed using point-to-feature features, followed by fine pose registration using an iterative nearest neighbor algorithm (iterative nearest point algorithm). Finally, pose verification further improves the grasping accuracy. This invention optimizes multiple aspects of the system process, not only improving registration accuracy and grasping success rate but also shortening the time for point cloud reconstruction and pose registration, thereby effectively reducing the total system time. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of a method for estimating the pose of scattered workpieces in unordered grasping according to the present invention;

[0048] Figure 2 This is a schematic diagram of the system flow of the present invention;

[0049] Figure 3 The camera calibration checkerboard pattern of this invention;

[0050] Figure 4 This is a schematic diagram of the polar line correction result of the present invention;

[0051] Figure 5 This is a flowchart of the binocular structured light 3D point cloud local generation module of the present invention;

[0052] Figure 6 This is a flowchart of the pose registration and verification module of the present invention;

[0053] Figure 7This is an illustration of the point-to-point feature calculation of the present invention;

[0054] Figure 8 This invention relates to a disordered grasping hardware platform;

[0055] Figure 9 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The purpose of this invention is to provide a method, system, and device for estimating the pose of scattered workpieces during disordered grasping, so as to improve the grasping success rate while reducing the total system time.

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Figure 1 This is a flowchart illustrating a method for estimating the pose of scattered workpieces during unordered grasping, as described in this invention. Figure 1 As shown, this invention provides a method for estimating the pose of scattered workpieces during unordered grasping, comprising the following steps:

[0060] Step 101: Generate workpiece detection data and apply the workpiece detection data to train the YOLACT model to obtain a trained YOLACT model; the workpiece detection data includes multiple labeled two-dimensional images of scattered workpieces; the two-dimensional images of scattered workpieces include multiple workpieces; the two-dimensional images of scattered workpieces are labeled with the category and mask of each workpiece.

[0061] Step 102: Establish a workpiece template point cloud library; the workpiece template point cloud library includes template point clouds corresponding to different categories of workpieces.

[0062] Step 102 specifically includes:

[0063] Structured light binocular cameras are used to construct template point clouds for different types of workpieces.

[0064] Based on the template point clouds corresponding to different categories of workpieces, a workpiece template point cloud library is obtained.

[0065] Step 103: Calculate the point pair features of each template point cloud in the workpiece template point cloud library and establish a template point pair feature library; the template point pair feature library includes template point pair features corresponding to different categories of workpieces; the template point pair features are the point pair features of the template point cloud.

[0066] Step 104: Obtain a two-dimensional image of the workpiece to be estimated.

[0067] Step 105: Use the trained YOLACT model to detect and segment the two-dimensional image of the scattered workpieces to be estimated, and obtain multiple categories and masks of the workpieces to be grasped.

[0068] Step 106: Based on the masks of multiple workpieces to be grasped, construct a sparse point cloud of multiple workpieces to be grasped using a structured light binocular camera.

[0069] Step 106 specifically includes:

[0070] The Zhang Zhengyou camera calibration method was used to calibrate the structured light binocular camera.

[0071] Based on the masks of multiple workpieces to be grasped, a calibrated structured light binocular camera is used to perform stereo matching using a combination of Gray code and phase shift code encoding and decoding. The initial sparse point cloud of multiple workpieces to be grasped is generated by combining a fixed-interval skip sampling method.

[0072] The initial sparse point cloud of multiple workpieces to be grasped is smoothed and denoised by using moving least squares filtering to obtain the sparse point cloud of multiple workpieces to be grasped.

[0073] Step 107: Calculate the point-pair features of the sparse point cloud for each workpiece to be grabbed.

[0074] The preceding steps, 107, also include:

[0075] The sparse point cloud of each workpiece to be grasped is preprocessed; the preprocessing includes downsampling and point cloud smoothing.

[0076] Step 108: For each workpiece to be grasped, according to the category of the workpiece, select the template point pair features corresponding to the workpiece of that category from the template point pair feature library. Perform coarse pose registration based on voting matching between the point pair features of the sparse point cloud of the workpiece to be grasped and the template point pair features corresponding to the workpiece of that category to obtain the initial pose of the workpiece to be grasped. Based on the initial pose, perform fine pose registration using the iterative nearest point algorithm to obtain the final pose of the workpiece to be grasped. After the final pose passes the pose verification, it is used by the robotic arm to grasp the workpiece.

[0077] Based on the method for estimating the pose of scattered workpieces in unordered grasping provided by this invention, this invention also provides a system for estimating the pose of scattered workpieces in unordered grasping, the system comprising the following modules:

[0078] The training module of the object detection network is used to generate workpiece detection data and apply the workpiece detection data to train the YOLACT model to obtain a trained YOLACT model. The workpiece detection data includes multiple labeled two-dimensional images of scattered workpieces. Each two-dimensional image of scattered workpieces contains multiple workpieces. The two-dimensional images of scattered workpieces are labeled with the category and mask of each workpiece.

[0079] The template point cloud generation module is used to establish a workpiece template point cloud library; the workpiece template point cloud library includes template point clouds corresponding to different categories of workpieces.

[0080] The module for generating the template point cloud specifically includes:

[0081] The template point cloud construction unit is used to construct template point clouds corresponding to different types of workpieces using a structured light binocular camera.

[0082] The workpiece template point cloud library creation unit is used to obtain the workpiece template point cloud library based on the template point clouds corresponding to different types of workpieces.

[0083] The template point pair feature library management module is used to calculate the point pair features of each template point cloud in the workpiece template point cloud library and establish the template point pair feature library; the template point pair feature library includes template point pair features corresponding to different categories of workpieces; the template point pair features are the point pair features of the template point cloud.

[0084] The image acquisition module is used to acquire two-dimensional images of the workpiece to be estimated.

[0085] The image detection and segmentation module is used to detect and segment two-dimensional images of the scattered workpieces to be estimated using a trained YOLACT model, and obtain multiple categories and masks of the workpieces to be grasped.

[0086] The binocular structured light 3D point cloud local generation module is used to construct sparse point clouds of multiple workpieces to be grasped using a structured light binocular camera based on the masks of multiple workpieces to be grasped.

[0087] The binocular structured light 3D point cloud local generation module specifically includes:

[0088] The camera calibration unit is used to calibrate the structured light binocular camera using the Zhang Zhengyou camera calibration method.

[0089] The initial sparse point cloud generation unit is used to generate initial sparse point clouds of multiple workpieces to be grasped based on the masks of multiple workpieces to be grasped, using a calibrated structured light binocular camera, and employing a combination of Gray code and phase shift code encoding and decoding methods for stereo matching, and combining a fixed interval skip sampling method.

[0090] The smoothing and denoising unit is used to smooth and denoise the initial sparse point cloud of multiple workpieces to be grasped by using moving least squares filtering, so as to obtain the sparse point cloud of multiple workpieces to be grasped.

[0091] The sparse point cloud point pair feature calculation module is used to calculate the point pair features of the sparse point cloud of each workpiece to be grasped.

[0092] The pose registration and verification module is used to select template point pair features corresponding to the workpiece category from the template point pair feature library for each workpiece to be grasped, based on the workpiece category. The point pair features of the sparse point cloud of the workpiece to be grasped are coarsely registered with the template point pair features corresponding to the workpiece category by voting matching to obtain the initial pose of the workpiece to be grasped. Based on the initial pose, the iterative nearest point algorithm is used to perform fine pose registration to obtain the final pose of the workpiece to be grasped. After the final pose passes the pose verification, it is used by the robotic arm to grasp the workpiece.

[0093] In addition, the system for estimating the pose of scattered workpieces during unordered grasping also includes:

[0094] The sparse point cloud preprocessing module is used to preprocess the sparse point clouds of each workpiece to be grasped; the preprocessing includes downsampling and point cloud smoothing.

[0095] The technical solution of the present invention is illustrated below with a specific embodiment:

[0096] This invention provides a fast pose estimation technique and system for scattered workpieces based on local generation of 3D point clouds and image classification guidance, such as... Figure 2 As shown, it includes the following steps:

[0097] The functional modules of this technical system are divided into two main categories: workpiece database management and fast pose estimation.

[0098] (1) Workpiece Database Management Class:

[0099] a. The training module of the object detection network (object detection network training module) is responsible for generating workpiece detection data and applying it to the training of the YOLACT model.

[0100] b. Template point cloud generation module (template point cloud generation module) is responsible for establishing the workpiece template point cloud library.

[0101] c. Template point pair feature library management module, which is responsible for calculating template point pair features (calculating template point cloud features) and establishing the template point pair feature library, which is used to match with the scene point cloud generated subsequently.

[0102] In step (1)-c, this invention uses point-pair features (PPF) to describe the point cloud features of the template. The PPF algorithm is a type of voting algorithm. The core prior of this type of algorithm is that each part of the data to be registered will affect the final pose of the object to be registered, that is, each primitive contributes to the final pose. Therefore, voting-based algorithms have strong robustness against the occlusion problem, which is difficult to solve in pose estimation.

[0103] (2) Fast pose estimation class:

[0104] The fast pose estimation class includes various functional modules for pose estimation of scattered workpieces, including an image detection and segmentation module, a binocular structured light 3D point cloud local generation module (binocular structured light 3D point cloud generation module), and a pose registration and verification module.

[0105] a. Image detection and segmentation module: The deep learning model YOLACT is used to detect and segment the two-dimensional image of the scattered workpieces, and to obtain the categories and masks of multiple workpieces to be grasped.

[0106] In step (2)-a, this invention uses the YOLACT deep learning model to detect and segment the two-dimensional image of the scattered workpieces. The YOLACT algorithm is a real-time instance segmentation algorithm, which is a "one-stage" algorithm. Compared with the "two-stage" algorithm, it is faster. The application of this algorithm can speed up the grasping speed of the entire system to a certain extent. In addition, in this step, only masks suitable for grasping are generated, creating conditions for the subsequent local point cloud generation, thereby reducing the influence of redundant point clouds from other non-objects in the environment and improving the accuracy of the registration algorithm.

[0107] b. Binocular structured light 3D point cloud local generation module: A combination of Gray code and phase-shift code encoding / decoding is used for stereo matching, and a fixed-interval skip sampling method is employed to generate sparse point clouds of multiple workpieces to be grasped. Moving least squares filtering is then used to smooth and denoise the point clouds, improving the accuracy and efficiency of registration in the next module.

[0108] In step (2)-b, this invention uses a structured light binocular camera to construct a local point cloud (template point cloud). Specifically, a combination of Gray code and phase-shift code is used, and the absolute phase of any point in the image is obtained by projecting Gray code fringes and a phase-shift grating. This method has high stereo matching stability and can bring higher 3D reconstruction accuracy. In addition, in this step, based on the workpiece mask information provided in the previous step, this invention uses a fixed-interval skip sampling method to generate a sparse point cloud of the workpiece to be grasped, so as to improve the reconstruction speed. The generation of the local workpiece point cloud to be grasped greatly reduces the complexity of point cloud matching compared with the global point cloud, thereby improving the accuracy and speed of registration in the next module. In particular, this invention generates multiple workpiece point clouds to be grasped at once in this step. This approach allows the robot arm to skip the image segmentation and workpiece point cloud generation steps after grasping, and directly perform pose registration and verification of the next workpiece to be grasped, thereby improving the system's grasping speed.

[0109] c. Pose Registration and Verification Module: Based on the workpiece category information obtained in (2)-a, the corresponding template point cloud is selected from the template feature library (template point-to-feature library). The template point cloud is registered with the scene point cloud to obtain the pose of the corresponding template in the environment point cloud (scene point cloud). Subsequently, a registration result verification algorithm is added to provide conditions for the robot's successful grasping.

[0110] In step (2)-c, firstly, coarse pose registration based on voting matching is performed using PPF point-pair features, then fine registration is performed using the Iterated Closest Points (ICP) algorithm, and finally, the pose verification algorithm is used to further improve the capture accuracy.

[0111] This invention provides a rapid pose estimation technology and system for scattered workpieces based on local generation of 3D point clouds and image classification guidance. It is suitable for grasping scattered workpieces. During batch authentication, this invention not only improves registration accuracy and grasping success rate but also shortens the time for point cloud reconstruction and pose registration, thereby effectively reducing the total system time. Furthermore, it exhibits stable performance across multiple authentication processes. The reason for this effect is as follows:

[0112] (1) Regarding the increase in speed

[0113] ①This invention uses the YOLACT network model for workpiece image detection and segmentation. YOLACT, as a "one-stage" algorithm, has a fast speed and can achieve real-time performance. The application of YOLACT improves the speed of step (2)-a.

[0114] ② In step (2)-b of this invention, a point cloud of the workpiece to be grasped is generated based on the mask information of the workpiece to be grasped output in the previous step. This is not a global point cloud, which reduces the point cloud reconstruction time to a certain extent.

[0115] ③ When generating point clouds, this invention uses a fixed-interval skip sampling method to generate sparse point clouds of the workpiece to be grasped, thereby reducing the amount of subsequent calculations and further improving the speed.

[0116] ④ In the robot grasping process, since point clouds of multiple workpieces to be grasped are generated in step (2)-b, after grasping the first workpiece, there is no need to perform steps (2)-a and (2)-b. The pose registration and verification of the next workpiece can be performed directly to guide the robotic arm to grasp, thereby improving the grasping speed of the entire system.

[0117] (2) Regarding the improvement of accuracy

[0118] ① In steps (1)-b and (2)-b, this invention uses a structured light binocular camera to construct the template point cloud and the scene point cloud. A combination of Gray code and phase-shift code is used, which has high stereo matching stability and can bring higher 3D reconstruction accuracy.

[0119] ② In steps (1)-c and (2)-c, the present invention uses point-to-point feature PPF to describe point cloud features. This algorithm has strong robustness in addressing the occlusion problem, which is difficult to solve in pose estimation.

[0120] ③ In step (2)-b, the present invention only generates the point cloud of the local workpiece to be grasped, rather than the global point cloud, thereby reducing the influence of redundant point clouds of other non-objects in the environment and improving the accuracy of the registration algorithm.

[0121] ④ In step (2)-c, a pose verification step is set up to output the correct pose, thereby effectively improving the accuracy of grasping.

[0122] The following description will focus on the fast pose estimation module. The specific details of the binocular structured light 3D point cloud local generation module, pose registration and verification module, and robot grasping process are as follows:

[0123] 1. Binocular structured light 3D point cloud local generation module

[0124] (1) Camera calibration

[0125] Binocular reconstruction calculates disparity based on corresponding pixels in the left and right camera patterns and obtains depth information for that point by combining intrinsic and extrinsic camera parameters. Reconstruction accuracy is primarily affected by the calibration accuracy of the reconstruction system and the performance of the image stereo matching algorithm; therefore, the binocular reconstruction system must be calibrated.

[0126] This invention employs Zhang Zhengyou's camera calibration method to calibrate the intrinsic and extrinsic parameters of the left and right cameras. Zhang Zhengyou's calibration method utilizes, for example... Figure 3 The checkerboard pattern shown completes the camera calibration, obtaining the camera's intrinsic and extrinsic parameters. This calibration method does not require knowledge of the calibration board's position and orientation in the world coordinate system, can calibrate multiple cameras, and does not require initial poses between multiple cameras. It has advantages such as high accuracy and robustness.

[0127] (2) Structured light encoding and decoding

[0128] In real-world industrial environments, the lighting conditions of the environment to be reconstructed are often unstable. Structured light methods, which encode the environment through active projection, are widely used in 3D reconstruction and matching in industrial scenarios due to their strong anti-interference capabilities. This invention employs a combination of Gray code and phase-shift code, using projected Gray code fringes and phase-shift gratings to obtain the absolute phase of any point in the image. This results in high stereo matching stability and higher 3D reconstruction accuracy.

[0129] ①Gray code encoding and decoding

[0130] In the reconstruction system used in this invention, each pixel of the environmental pattern is encoded by projecting multiple Gray code patterns, with each Gray code pattern corresponding to a specific bit in the Gray code. In this invention, a total of five Gray code patterns are projected, each Gray code being 5 bits long. All the Gray code patterns divide the entire image into 32 regions, where dark stripes correspond to Gray code digit 0, and bright stripes correspond to Gray code digit 1.

[0131] Gray code decoding involves first converting a series of environmental patterns captured by left and right cameras and projecting them into Gray code, then converting the binary code into decimal values. The Gray code to binary conversion formula is as follows:

[0132]

[0133] Among them B i G represents the value corresponding to the i-th bit in the binary code. i B represents the value corresponding to the i-th bit in Gray code. n G represents the number corresponding to the nth bit of the converted binary code. n This represents the digit corresponding to the nth position in Gray code, where n represents the number of digits.

[0134] ② Phase shift code encoding and decoding

[0135] To improve decoding performance, a phase-shift code is introduced as a supplementary encoding scheme for the minimum periodic stripes of the Gray code, ensuring the continuity of code values ​​in the image encoding and decoding regions and improving reconstruction accuracy. This invention employs a four-step phase shift, encoding the environmental pattern by projecting four phase-shifted patterns that are sequentially shifted by 1 / 4 of their respective periods.

[0136] The purpose of phase-shift code decoding is to calculate the principal phase value of each pixel from the captured scene image with phase-shift code. The projected phase-shift pattern is a grayscale stripe, and for a pixel P(x,y), its light intensity I(x,y) satisfies the following relationship:

[0137]

[0138] Where a(x,y) is the background light intensity, b(x,y) is the modulation intensity, and φ(x,y) represents the principal phase value. This represents the phase shift value. Taking the four-step phase shift method as an example, within a 2π period, the phase shifts by an amplitude of π / 2 each time. The light intensity distribution of each phase shift code pattern is shown in formula (3).

[0139]

[0140] Among them, I i (x, y) is the light intensity formula for the i-th image, where i = 1, 2, 3, 4. Solving the four equations in formula (3) simultaneously yields the principal phase value of pixel P, i.e.:

[0141]

[0142] Next, the absolute phase Φ(x,y) at point P can be obtained by superposition:

[0143]

[0144] In the formula, k is the Gray code decoding value at P, and φ(x,y) is the principal phase value at P.

[0145] (3) Local generation of 3D point cloud

[0146] ①Polar correction

[0147] The purpose of epipolar correction is to ensure that a pixel in the left (right) camera image has a matching point in the right (left) camera image that is in the same horizontal row as that pixel, such as... Figure 4 As shown, Figure 4 Part (a) in the image represents the right-eye camera image after epipolar correction, and part (b) represents the left-eye camera image after epipolar correction. Epipolar correction reduces the search range for pixel matching to a single row of pixels, minimizing the influence of redundant non-target point clouds in the environment and improving point cloud reconstruction speed.

[0148] ② Local point cloud reconstruction

[0149] After epipolar correction, the camera images can be stereo matched using structured light encoding and decoding. For a point (x, y) in the stereo matched image, x and y are the horizontal and vertical coordinates of that point, respectively. The lower coordinate of that point in the left camera image is (x, y). l y l ), the lower coordinates of the right camera image (x r y r ), satisfying the following relationship:

[0150]

[0151] Among them, C l C r Let x be the absolute phase encoding value of the left and right camera images. Then, the disparity between the left and right images is d = x. l -x r .

[0152] After obtaining the disparity, the depth information Z of point P can be calculated, as shown below:

[0153]

[0154] Where f is the focal length of the camera, B is the distance between the two cameras (i.e., the baseline), and d P This represents the parallax of the left and right camera images corresponding to point P.

[0155] Next, we can obtain the three-dimensional coordinates (X, Y, Z) of that point:

[0156]

[0157] Among them, (c x c y The principal point is the point where the camera lens optical axis perpendicularly intersects the imaging plane. Performing the same operation on each matching point within the mask area in the left and right images completes the 3D reconstruction and yields the environmental point cloud. The overall reconstruction process is as follows: Figure 5 As shown.

[0158] Specifically, when calculating 3D point information, a fixed-interval skip sampling method is used to calculate the 3D point information of a subset of points to generate a sparse point cloud (scene point cloud) of the workpiece to be grasped. Subsequently, moving least squares filtering is used to smooth and denoise the point cloud to improve the accuracy and efficiency of registration in the next module.

[0159] 2. Pose Registration and Verification

[0160] The pose registration and verification process of this invention is as follows: Figure 6As shown, the input to the pose registration algorithm is the workpiece category, template point cloud, template point pair features, and scene point cloud. The process first selects appropriate template point pair features based on the object category to perform coarse pose registration of the environment point cloud using voting matching. Then, after obtaining the initial pose, fine pose registration is performed, finally obtaining the final target pose. Simultaneously, after completing the fine pose registration, a pose registration result verification algorithm is introduced to improve the capture success rate.

[0161] (1) Coarse Pose Registration Based on Voting Matching

[0162] In the preparation stage of the registration algorithm, a template point-pair feature library needs to be established first. Specifically, the workpiece category of the point cloud of the workpiece model to be captured is used as the index value, and the point-pair features of the point cloud are used as the features of the corresponding workpiece. The point-pair features of objects of corresponding categories are stored in a hash table, with similar point-pair features placed in adjacent positions. Establishing the point-pair feature library only requires obtaining the category, point cloud, and point-pair features of a single workpiece, therefore it can be done offline. The calculation of PPF point-pair features is as follows:

[0163] For two surface points p1 and p2 in the template point cloud, the point pair feature PPF(p1,p2) is calculated as follows:

[0164] PPF(p1,p2)=[F1,F2,F3,F4] (9)

[0165] In this point cloud, p1 and p2 are two points, each with its corresponding point normal, denoted as n1 and n2 respectively. F1 is the Euclidean distance between p1 and p2, F2 and F3 are the angles between n1 and n2 and the lines connecting them, and F4 is the angle between n1 and n2. The PPF descriptor in the point cloud is explained as follows: Figure 7 As shown.

[0166] Before calculating the scene point cloud features, the scene point cloud needs to be preprocessed, which includes downsampling and point cloud smoothing. Since a sparse point cloud has already been generated during point cloud reconstruction, the computational load is greatly reduced. Next, point cloud smoothing is performed. Point cloud smoothing can effectively remove noise in the point cloud and reduce reconstruction errors. This invention uses the moving least squares method in the point cloud processing library PCL to perform point cloud smoothing.

[0167] After reconstructing and preprocessing the scene point cloud, the point-pair features of the obtained scene point cloud are calculated. The calculation process is the same as that for the point-pair features of the template point cloud. After obtaining the point-pair features of the scene point cloud, a voting-based method is used, combined with the object category information obtained from the object detection network, to search and match in the model point-pair feature (template point-pair feature) database. Let M be the model point cloud (template point cloud) and S be the environment point cloud. The two point clouds are represented as follows:

[0168]

[0169] M contains i points, and S contains j points. PPF Θ (p m1 ,p m2 ) represents the feature of the point pair formed by points m1 and m2 in the model point cloud, PPF Θ (p m(i-1) ,p mi ) represents the point pair features formed by points m(i-1) and mi in the model point cloud, PPF Ω (p s1 ,p s2 ) represents the point pair features formed by points s1 and s2 in the scene point cloud, PPF Ω (p s(j-1) ,p sj ) represents the point pair features formed by points s(j-1) and sj in the scene point cloud.

[0170] Next, a voting mechanism matching algorithm is used to estimate the workpiece pose. Since different poses may produce similar photoplethysms (PPFs), there are usually multiple candidate matches. Each matching point pair votes for its corresponding pose (rotation and translation) in the so-called "voting space." In the voting space, these votes are organized into different "clusters" using a clustering method. Each cluster represents a group of similar pose votes, and the score of each cluster is the sum of the pose votes in that cluster. Finally, the cluster with the highest score is selected, and the cluster center is calculated. The pose corresponding to the center point is the final coarse pose registration result.

[0171] (2) Pose fine registration based on iterative nearest neighbor

[0172] The PPF algorithm, as a voting-based registration algorithm, generally yields poses that are not refined enough. To obtain more refined poses, this invention employs the Iterative Closest Point (ICP) algorithm to refine the poses obtained from the coarse registration in the previous section. As an iterative optimization algorithm, ICP requires high initial poses between the point cloud to be matched and the target point cloud. Excessive pose differences can easily lead to the final iterative matching result getting trapped in local optima, resulting in registration failure. This invention uses the ICP algorithm as a pose refinement algorithm, embedding it after the coarse pose registration result, which effectively avoids getting trapped in local optima.

[0173] The core idea of ​​the ICP algorithm is to find the optimal transformation matrix through iterative optimization, minimizing the distance between corresponding points in one point cloud and another. This process typically includes the following steps:

[0174] ① Initialization: Select an initial transformation matrix, usually the identity matrix. This matrix describes the initial position and orientation differences;

[0175] ② Nearest Neighbor Search: For each point in the template point cloud, find the nearest point in the scene point cloud. This invention uses the kd-tree nearest neighbor search method to achieve this.

[0176] ③ Corresponding point matching: Based on the results of nearest neighbor search, establish the correspondence between template point cloud and scene point cloud;

[0177] ④ Calculate the error metric: Calculate the error metric between point pairs, which is usually the distance between point pairs;

[0178] ⑤ Parameter estimation: Based on the correspondence between point pairs and the error metric, estimate a new transformation matrix to reduce the error;

[0179] ⑥ Update the transformation matrix: Multiply the estimated new transformation matrix with the previous transformation matrix to obtain the new transformation matrix;

[0180] ⑦ Convergence Check: Check if the error is small enough or if the transformation matrix is ​​stable enough. If the convergence condition is met, the algorithm terminates. Otherwise, return to step ② and continue iterating.

[0181] The ICP algorithm repeats these steps until it converges to an optimal transformation matrix that minimizes the difference between the two point clouds.

[0182] (3) Pose inspection

[0183] In real-world industrial production environments, after acquiring the pose of an object, a disordered grasping system often uses a robot to perform subsequent grasping and placement actions. If pose registration fails without a pose verification module, and erroneous pose information is sent to the robot for grasping, it could potentially lead to damage to the workpiece and the machine, or even personal injury. Therefore, after implementing the aforementioned pose registration algorithm, a workpiece pose verification algorithm is introduced. This algorithm measures the result of specific pose matching by calculating the ratio of successfully registered points in the template point cloud to the total number of points in the scene point cloud, as shown below:

[0184]

[0185] Among them, S(P s ,P t P is the registered template point cloud. s With target point cloud (scene point cloud) P t The registration result score between N s m represents the total number of points in the template point cloud, and m represents the template point cloud P. s The target point cloud P can be matched. t The number of time points that satisfy the following formula, where r can be defined according to different specific use cases.

[0186] |p s -p t | <r (12)

[0187] In the above formula, |p s -p t |for point p s With p t The distance between them, where r is the distance judgment threshold.

[0188] The overall pose registration score calculation process is as follows:

[0189] ① First, based on the transformation matrix obtained in section (2), the template point cloud is transformed accordingly to obtain the template point cloud P. s Initialize m = 0;

[0190] ② Traverse the template point cloud P s Calculate the distance to each point in the scene point cloud. When the formula (12) is satisfied, delete the corresponding point pair between the template point cloud and the environment point cloud, and increment the value of m by 1.

[0191] ③ After the complete traversal is completed, the pose registration score S(P) of the current template point cloud is calculated according to formula (11). s ,P s );

[0192] ④ Set the score threshold S min When S>S min If the registration is successful, the final pose is output. Otherwise, return to the coarse pose registration stage and continue registration until S>S0. min Where S is the pose registration score obtained in step ③ above.

[0193] 3. Robotic grasping

[0194] (1) System hardware platform selection and setup

[0195] The main hardware components of this invention include: left and right cameras forming the structured light 3D reconstruction system, a projector, a computer for calculation, and a robot for performing grasping tasks, such as... Figure 8 As shown.

[0196] (2) Robot hand-eye calibration

[0197] In unordered grasping systems, pose registration during the pose matching stage is performed in the world coordinate system of the binocular reconstruction system. To enable the robot to correctly execute grasping actions, hand-eye calibration of the reconstruction system and the robot system is first required, unifying the robot's tool coordinate system with the reconstruction system's coordinate system.

[0198] (3) Grab point selection

[0199] Robotic gripping tools come in various forms, including clamps, nozzles, and electromagnets. Choosing the appropriate gripping tool depends on the robot's actual working environment and tasks. Considering the background of this invention—workpieces overlapping and being compressed in a three-dimensional environment—a nozzle is used as the robotic gripping tool. Using a nozzle effectively avoids obstruction and collisions; however, due to its working principle, the area to be gripped requires a smooth surface. Therefore, the gripping point is selected near the workpiece's center of gravity.

[0200] This invention optimizes several aspects of the system process, which not only reduces the time for point cloud reconstruction, but also reduces the total system time while improving the success rate of capture, and performs stably in multiple authentication processes.

[0201] This invention intersects multiple fields such as computer vision, 3D reconstruction, structured light, machine learning, and industrial automation. It utilizes a fast pose estimation technique for scattered workpieces guided by local 3D point cloud generation and image classification to achieve rapid workpiece grasping. This invention, based on a fast pose estimation technique and system for scattered workpieces guided by local 3D point cloud generation and image classification, such as... Figure 9 As shown, the functional modules of this technical system are divided into two main categories: workpiece database management and fast pose estimation. The workpiece database management module includes an object detection network training module, a template point cloud generation module, and a template point pair feature library management module. The fast pose estimation module includes functional modules for scattered workpiece pose estimation, including an image detection and segmentation module, a binocular structured light 3D point cloud local generation module, and a pose registration and verification module. In the image detection and segmentation module, the deep learning model YOLACT is used to detect and segment the 2D images of scattered workpieces, and to obtain the categories and masks of multiple workpieces to be grasped. In the binocular structured light 3D point cloud local generation module, a fixed-interval skip sampling method is used to generate sparse point clouds of multiple workpieces to be grasped, and then moving least squares filtering is used to smooth and denoise the point clouds. In the pose registration and verification module, based on the workpiece category information obtained by the image detection and segmentation module, the corresponding template point cloud is selected from the template point pair feature library, and the workpiece pose is obtained through coarse registration and fine registration. Subsequently, a pose verification algorithm was applied to further improve the grasping success rate. This technology was applied to a robot grasping system, and multiple comparative experiments demonstrated that the method proposed in this invention not only improved registration accuracy and grasping success rate, but also shortened the time for point cloud reconstruction and pose registration, thereby effectively reducing the total system time, and the results were stable.

[0202] Furthermore, the present invention also provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor can call a computer program stored in the memory to execute the described method for estimating the pose of scattered workpieces in unordered grasping.

[0203] Furthermore, when the computer program in the aforementioned memory is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-transitory computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0204] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0205] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for estimating the pose of scattered workpieces during unordered grasping, characterized in that, The method includes: Workpiece detection data is generated and applied to the training of the YOLACT model to obtain a trained YOLACT model. The workpiece detection data includes multiple labeled two-dimensional images of scattered workpieces. Each two-dimensional image of scattered workpieces includes multiple workpieces. The two-dimensional images of scattered workpieces are labeled with the category and mask of each workpiece. Establish a workpiece template point cloud library; the workpiece template point cloud library includes template point clouds corresponding to different categories of workpieces; Calculate the point pair features of each template point cloud in the workpiece template point cloud library to establish a template point pair feature library; the template point pair feature library includes template point pair features corresponding to different categories of workpieces; the template point pair features are point pair features of template point clouds; Obtain a two-dimensional image of the workpiece to be estimated; The trained YOLACT model is used to detect and segment the two-dimensional image of the scattered workpiece to be estimated, and multiple categories and masks of the workpiece to be grasped are obtained. Based on multiple masks of the workpieces to be grasped, a sparse point cloud of the multiple workpieces to be grasped is constructed using a structured light binocular camera, specifically including: The Zhang Zhengyou camera calibration method was used to calibrate the structured light binocular camera; Based on the masks of multiple workpieces to be grasped, a calibrated structured light binocular camera is used to perform stereo matching using a combination of Gray code and phase shift code encoding and decoding, and an initial sparse point cloud of multiple workpieces to be grasped is generated using a fixed interval skip sampling method. The initial sparse point clouds of the multiple workpieces to be grasped are smoothed and denoised using moving least squares filtering to obtain the sparse point clouds of the multiple workpieces to be grasped. Calculate the point-pair features of the sparse point cloud of each of the workpieces to be grasped; For each workpiece to be grasped, based on the category of the workpiece, template point pair features corresponding to the workpiece of the specified category are selected from the template point pair feature library. The point pair features of the sparse point cloud of the workpiece to be grasped are coarsely registered with the template point pair features corresponding to the workpiece of the specified category using a voting matching method to obtain the initial pose of the workpiece to be grasped. Based on the initial pose, an iterative nearest point algorithm is used for fine pose registration to obtain the final pose of the workpiece to be grasped. The final pose, after passing pose verification, is used by the robotic arm to grasp the workpiece.

2. The method for estimating the pose of scattered workpieces in unordered grasping according to claim 1, characterized in that, The establishment of the workpiece template point cloud library specifically includes: Structured light binocular cameras are used to construct template point clouds for different types of workpieces; Based on the template point clouds corresponding to different categories of workpieces, a workpiece template point cloud library is obtained.

3. The method for estimating the pose of scattered workpieces in unordered grasping according to claim 1, characterized in that, The calculation of point-pair features of the sparse point cloud of each of the workpieces to be grasped further includes, prior to: The sparse point cloud of each of the workpieces to be grasped is preprocessed; the preprocessing includes downsampling and point cloud smoothing.

4. A pose estimation system for scattered workpieces during unordered grasping, characterized in that, The system includes: The training module of the object detection network is used to generate workpiece detection data and apply the workpiece detection data to train the YOLACT model to obtain a trained YOLACT model. The workpiece detection data includes multiple labeled two-dimensional images of scattered workpieces. Each two-dimensional image of scattered workpieces includes multiple workpieces. The two-dimensional images of scattered workpieces are labeled with the category and mask of each workpiece. The template point cloud generation module is used to establish a workpiece template point cloud library; the workpiece template point cloud library includes template point clouds corresponding to different categories of workpieces; The template point-pair feature library management module is used to calculate the point-pair features of each template point cloud in the workpiece template point cloud library and establish the template point-pair feature library; the template point-pair feature library includes template point-pair features corresponding to different categories of workpieces; the template point-pair features are the point-pair features of the template point cloud; The image acquisition module is used to acquire a two-dimensional image of the workpiece to be estimated. The image detection and segmentation module is used to detect and segment the two-dimensional image of the scattered workpiece to be estimated using the trained YOLACT model, and obtain multiple categories and masks of the workpiece to be grasped. A binocular structured light 3D point cloud local generation module is used to construct sparse point clouds of multiple workpieces to be grasped using a structured light binocular camera based on multiple masks of the workpieces to be grasped. Specifically, it includes: The Zhang Zhengyou camera calibration method was used to calibrate the structured light binocular camera; Based on the masks of multiple workpieces to be grasped, a calibrated structured light binocular camera is used to perform stereo matching using a combination of Gray code and phase shift code encoding and decoding, and an initial sparse point cloud of multiple workpieces to be grasped is generated using a fixed interval skip sampling method. The initial sparse point clouds of the multiple workpieces to be grasped are smoothed and denoised using moving least squares filtering to obtain the sparse point clouds of the multiple workpieces to be grasped. The sparse point cloud point pair feature calculation module is used to calculate the point pair features of the sparse point cloud of each of the workpieces to be grasped. The pose registration and verification module is used to, for each workpiece to be grasped, select template point pair features corresponding to the workpiece of the specified category from the template point pair feature library according to the category of the workpiece to be grasped, perform coarse pose registration based on voting matching between the point pair features of the sparse point cloud of the workpiece to be grasped and the template point pair features corresponding to the workpiece of the specified category, to obtain the initial pose of the workpiece to be grasped, and perform fine pose registration based on the initial pose using an iterative nearest point algorithm to obtain the final pose of the workpiece to be grasped; the final pose, after passing pose verification, is used by the robotic arm to grasp the workpiece to be grasped.

5. The workpiece pose estimation system for disordered grasping according to claim 4, characterized in that, The template point cloud generation module specifically includes: Template point cloud construction unit, used to construct template point clouds corresponding to different types of workpieces using a structured light binocular camera; The workpiece template point cloud library creation unit is used to obtain the workpiece template point cloud library based on the template point clouds corresponding to different types of workpieces.

6. The workpiece pose estimation system for disordered grasping according to claim 4, characterized in that, The binocular structured light 3D point cloud local generation module specifically includes: The camera calibration unit is used to calibrate the structured light binocular camera using Zhang Zhengyou's camera calibration method. The initial sparse point cloud generation unit is used to generate the initial sparse point cloud of the multiple workpieces to be grasped based on the masks of the multiple workpieces to be grasped, using a calibrated structured light binocular camera, and employing a combination of Gray code and phase shift code encoding and decoding to perform stereo matching, and combining a fixed interval skip sampling method. The smoothing and denoising unit is used to perform smoothing and denoising processing on the initial sparse point clouds of multiple workpieces to be grasped using moving least squares filtering, so as to obtain the sparse point clouds of multiple workpieces to be grasped.

7. The workpiece pose estimation system for disordered grasping according to claim 4, characterized in that, The system also includes: The sparse point cloud preprocessing module is used to preprocess the sparse point clouds of each of the workpieces to be grasped; the preprocessing includes downsampling and point cloud smoothing.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for estimating the pose of scattered workpieces in unordered grasping as described in any one of claims 1-3.

9. The electronic device according to claim 8, characterized in that, The memory is a non-transitory computer-readable storage medium.

Images