Method and apparatus for obtaining information

Abstract

The application provides a method and device for obtaining information, and relates to the technical field of computers. The method comprises the following steps: analyzing a two-dimensional scene image, obtaining an occlusion rate corresponding to each first object to be grasped in the scene image, and taking the first object to be grasped with an occlusion rate less than a preset occlusion rate as a second object to be grasped; for each second object to be grasped, analyzing a local point cloud corresponding to the second object to be grasped to obtain grasping reference information of the second object to be grasped, wherein the grasping reference information comprises a center position of a target grasping plane and the target grasping plane; and generating grasping information according to the grasping reference information of each second object to be grasped, wherein the grasping information comprises a grasping pose of a target object to be grasped, the target object to be grasped is the second object to be grasped, and the grasping pose comprises the center position and a grasping attitude obtained from a normal vector of the target grasping plane. In this way, the success rate of grasping of a robot in a stacked occlusion scene can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method and apparatus for acquiring information. Background Technology

[0002] With the development of logistics automation and intelligent sorting technology, sorting equipment based on robot systems for automatic parcel grabbing has been widely used in warehousing, logistics transit, and e-commerce delivery. In practical applications, parcels are usually randomly stacked, with complex situations such as obstruction, tilting, and multiple exposed sides, which poses a significant challenge to the robot's stable grasping.

[0003] Currently, the following method is used to obtain grasping information and then grasp packages: For 2D images, the 2D contour region of the package is obtained through object detection or instance segmentation algorithms, and the identified package is used as the grasping object; then, the grasping pose of each grasping object is further analyzed as grasping information; finally, based on the above grasping information, a robot or robotic arm is controlled to grasp the package. In the above method, it is easy to select occluded objects as grasping objects, resulting in a high grasping failure rate. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and readable storage medium for obtaining grasping information. By filtering grasping objects based on the occlusion rate of the objects to be grasped, it can avoid using severely occluded objects as grasping objects, thereby improving the success rate of robot grasping in stacked occlusion scenarios.

[0005] The embodiments of this application can be implemented as follows: In a first aspect, embodiments of this application provide a method for obtaining crawled information, the method comprising: The two-dimensional scene image is analyzed to obtain the occlusion rate of each first object to be captured in the scene image, and the first object to be captured with an occlusion rate less than a preset occlusion rate is taken as the second object to be captured. For each second object to be captured, based on the local point cloud corresponding to the second object to be captured, the capture reference information of the second object to be captured is obtained by analysis, wherein the capture reference information includes the center position of the target capture plane and the target capture plane; Based on the grasping reference information of each second object to be grasped, grasping information is generated, wherein the grasping information includes the grasping pose of the target object to be grasped, the target object to be grasped is the second object to be grasped, and the grasping pose includes the center position and the grasping posture obtained by the normal vector of the target grasping plane.

[0006] Secondly, embodiments of this application provide an information acquisition device, the device comprising: The occlusion rate filtering module is used to analyze the two-dimensional scene image, obtain the occlusion rate of each first object to be captured in the scene image, and take the first object to be captured with an occlusion rate less than a preset occlusion rate as the second object to be captured. The reference information acquisition module is used to analyze and obtain the grasping reference information of each second object to be grasped based on the local point cloud corresponding to the second object to be grasped. The grasping reference information includes the center position of the target grasping plane and the target grasping plane. The information generation module is used to generate grasping information based on the grasping reference information of each second object to be grasped. The grasping information includes the grasping pose of the target object, which is the second object to be grasped. The grasping pose includes the center position and the grasping posture obtained by the normal vector of the target grasping plane.

[0007] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the information capture method described in any of the foregoing embodiments.

[0008] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the information capture and acquisition method as described in any of the foregoing embodiments.

[0009] The grasping information acquisition method, apparatus, electronic device, and readable storage medium provided in this application firstly analyze a two-dimensional scene image to obtain the occlusion rate of each first candidate grasping object in the scene image, and then selects the first candidate grasping objects with an occlusion rate less than a preset occlusion rate as second candidate grasping objects. Next, for each second candidate grasping object, grasping reference information is obtained by analyzing the local point cloud corresponding to the second candidate grasping object. This grasping reference information includes the center position of the target grasping plane and the target grasping plane itself. Finally, grasping information is generated based on the grasping reference information of each second candidate grasping object. This grasping information includes the grasping pose of each target grasping object, where the target grasping object is the second candidate grasping object, and the grasping pose includes the aforementioned center position and the grasping posture obtained from the normal vector of the target grasping plane. Thus, by selecting grasping objects based on their occlusion rates, severely occluded objects can be avoided as grasping objects, thereby improving the robot's grasping success rate in stacked occlusion scenarios. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application; Figure 2 A flowchart illustrating the information acquisition method provided in this application embodiment; Figure 3 for Figure 2 A flowchart illustrating the sub-steps included in step S110; Figure 4 for Figure 2 A flowchart illustrating the sub-steps included in step S120; Figure 5 for Figure 4 A flowchart illustrating the sub-steps included in the neutron step S122; Figure 6 for Figure 4 A flowchart illustrating the sub-steps included in the neutron step S124; Figure 7 for Figure 2 One of the flowcharts illustrating the sub-steps included in step S130; Figure 8 for Figure 2 The second flowchart of the sub-steps included in step S130; Figure 9 for Figure 2 The third flowchart of the sub-steps included in step S130; Figure 10 for Figure 2 The fourth flowchart of the sub-steps included in step S130; Figure 11 A schematic diagram of the image space and placement space provided in the embodiments of this application; Figure 12 This is a block diagram of the information capture and acquisition device provided in the embodiments of this application.

[0012] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication unit; 200 - Information capture and acquisition device; 210 - Occlusion rate filtering module; 220 - Reference information acquisition module; 230 - Information generation module. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0014] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0015] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0016] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0017] Please refer to Figure 1 , Figure 1 This is a block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 may be, but is not limited to, a computer, server, robot, etc. The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The memory 110, processor 120, and communication unit 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0018] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0019] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions. For example, the memory 110 stores a grasping information acquisition device 200, which includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running the software programs and modules stored in the memory 110, such as the grasping information acquisition device 200 in this embodiment, thereby implementing the grasping information acquisition method in this embodiment.

[0020] The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0021] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0022] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the information acquisition method provided in this application embodiment. The method can be applied to the aforementioned electronic device 100. The specific flow of the information acquisition method is described in detail below. In this embodiment, the method may include steps S110 to S130.

[0023] Step S110: Analyze the two-dimensional scene image to obtain the occlusion rate of each first object to be captured in the scene image, and take the first object to be captured with an occlusion rate less than a preset occlusion rate as the second object to be captured.

[0024] In this embodiment, a two-dimensional scene image can be obtained. This scene image can be an RGB image. Object recognition analysis can be performed on the scene image to determine each first object to be captured. The specific recognition analysis method can be determined based on actual needs and is not specifically limited here. For each first object to be captured, based on the object region where the first object to be captured is located in the scene image, the occlusion rate of the first object to be captured is analyzed; it is then determined whether the occlusion rate of the first object to be captured is less than a preset occlusion rate. If it is less, the first object to be captured is selected as a second candidate object to be captured; if it is not greater, the first object to be captured is not selected as a second candidate object to be captured. The preset occlusion rate can be determined based on actual needs.

[0025] Step S120: For each second object to be captured, the capture reference information of the second object to be captured is obtained by analyzing the local point cloud corresponding to the second object to be captured.

[0026] For each second object to be grasped, the point cloud of the second object to be grasped can be obtained in any way as the corresponding local point cloud. Then, by analyzing the local point cloud, the grasping reference information of the second object to be grasped can be obtained. The grasping reference information includes the center position of the target grasping plane and the target grasping plane, which is the plane on which the robot or robotic arm adheres to the object when grasping it.

[0027] Step S130: Generate crawling information based on the crawling reference information of each second object to be crawled.

[0028] Based on the obtained grasping reference information of each second object to be grasped, the second object to be grasped as the target object can be determined, and the grasping pose of each target object can be generated to obtain grasping information. The grasping information includes the grasping pose of the target object, which includes the center position and the grasping posture obtained from the normal vector of the target grasping plane. The robot or robotic arm can then use this grasping information to attract the corresponding object, thereby completing the grasping process.

[0029] In this way, by filtering objects based on the occlusion rate of the objects to be grasped, we can avoid using severely occluded objects as grasping objects, thereby improving the robot's grasping success rate in stacked occlusion scenarios.

[0030] Alternatively, as one possible implementation, it can be achieved through... Figure 3 The occlusion rate is shown below. Please refer to... Figure 3 , Figure 3 for Figure 2A flowchart illustrating the sub-steps included in step S110. In this embodiment, step S110 may include sub-steps S111 to S113.

[0031] Sub-step S111: Perform object recognition analysis on the scene image to obtain the object region corresponding to each first object to be captured.

[0032] Sub-step S112: For each first object to be grabbed, the minimum rotational bounding rectangle region of the first object to be grabbed is obtained based on the object region of the first object to be grabbed.

[0033] Sub-step S113: For each first object to be grabbed, the occlusion rate of the first object to be grabbed is calculated based on the distribution of other first objects to be grabbed in the minimum rotational bounding rectangle region of the first object to be grabbed.

[0034] Optionally, an object detection network can be used to detect objects in the scene image to obtain a set of object bounding boxes. Next, instance segmentation is performed based on the detection boxes to obtain the object regions of each first object to be captured within the scene object. The object detection network is used to detect the objects to be captured, and the object bounding box set includes the bounding boxes (also called detection boxes) of each first object to be captured in the scene image. The object region is the area of ​​the first object to be captured in the scene image. Optionally, the object region corresponding to each first object to be captured can be a pixel-level binary mask. This means that the foreground region in the mask is used to represent the object region. , where i represents the i-th first candidate object to be crawled.

[0035] For each first object to be grabbed, the minimum rotational bounding rectangle region of the first object to be grabbed can be calculated based on the object region of the first object to be grabbed. Minimum circumscribed rectangle region This represents the theoretical footprint of the first object to be captured in an ideal, unobstructed state. Then, the minimum rotational bounding rectangle region can be calculated. The area is taken as the first area. And, based on the object regions of other first objects to be grabbed, the minimum rotational bounding rectangle region of the other first objects to be grabbed within the first object to be grabbed is analyzed. The coverage area is used as the second area. Next, the quotient of the second area and the first area is calculated as the occlusion rate of the first object to be crawled. Then, the first object to be crawled with an occlusion rate less than the preset occlusion rate can be used as the second object to be crawled, thereby avoiding the problem of misjudging the crawled object based solely on the size of the object area of ​​the first object to be crawled.

[0036] Pixel-level binary masks can be used in the object area. In the case of this representation, the above analysis process can be expressed as follows. For each pixel-level binary mask... Calculate its minimum circumscribed rectangle region. : The minimum rotational circumscribed rectangle region This represents the theoretical footprint of a first object to be crawled in an ideal, unobstructed state. Minimum rotational bounding rectangle region. It can be represented as: ,in, Represents the smallest circumscribed rectangle region. The center point, Represents the smallest circumscribed rectangle region. width, Represents the smallest circumscribed rectangle region. Length, Represents the smallest circumscribed rectangle region. The rotation angle. The minimum rotation circumscribed rectangle region. The area is: Next, the minimum rotational bounding rectangle region of the other first objects to be grabbed within the first object to be grabbed is calculated. Coverage area within: Therefore, the occlusion rate of the first object to be crawled is: Then, the first object to be crawled with an occlusion rate less than a preset occlusion rate can be selected as the currently crawlable object. This method effectively avoids the problem of misjudging crawlable objects based solely on mask area.

[0037] In stacked parcel scenarios, objects often overlap or partially occlude each other. Simply relying on the visible area obtained from instance segmentation is insufficient to accurately reflect the true graspability of objects. This solution utilizes the occlusion rate quantification method based on a rotated bounding rectangle to assess the actual occlusion level of objects. This occlusion level is then used to filter grasping targets, avoiding the selection of severely occluded objects and thus improving the robot's grasping success rate in stacked occlusion scenarios.

[0038] The inventors of this application discovered through research that, in scenarios with multiple stacked objects, directly performing plane detection on the overall point cloud data is easily affected by interference from adjacent objects and the background, resulting in unstable plane fitting results and difficulty in accurately extracting the graspable plane of the target object. Based on the above findings, optionally, it can be achieved through... Figure 4 The method shown obtains the capture reference information for each second object to be captured based on the local point cloud of each object. Please refer to... Figure 4 , Figure 4 for Figure 2A flowchart illustrating the sub-steps included in step S120. In this embodiment, step S120 may include sub-steps S121 to S124.

[0039] Sub-step S121: Preprocess the local point cloud corresponding to the second object to be captured to obtain the target local point cloud.

[0040] Optionally, the object region of the second object to be captured can be back-projected using the depth map and camera intrinsic parameter matrix, and the calculation result can be used as the local point cloud of the second object to be captured. The depth map includes depth information corresponding to the scene image.

[0041] A pixel-level binary mask is used in the object region of the second object to be captured. In the case of a pixel-level binary mask, the above analysis process can be represented as follows. Combined with depth map and camera intrinsic parameter matrix Back-projection calculation is performed only on the foreground pixels of the mask area. Let the depth scaling system be s, then the formula for calculating the 3D coordinates of the surface point cloud of the second object to be grasped is:

[0042] Thus, the local point cloud of the first object to be captured is obtained, and the local point cloud can be represented by the following set: .

[0043] Optionally, a global scene point cloud corresponding to the scene image can be obtained, and cluster analysis can be performed on the global scene point cloud to obtain the local point cloud of each second object to be captured. It is understood that the local point cloud of each second object to be captured can also be obtained through other methods; the two methods described above are merely illustrative examples.

[0044] The local point cloud of the second object to be captured can be preprocessed to improve the accuracy of the subsequent processing results. Optionally, the preprocessing includes downsampling and / or outlier removal.

[0045] One possible implementation is to first perform voxel downsampling on the local point cloud of the second object to be captured, followed by outlier removal to obtain the target local point cloud. Specifically, voxel mesh filtering can be used to perform spatially uniform sampling of the local point cloud, thereby reducing point cloud density and subsequent processing load. Statistical consistency filtering can be applied to the downsampled point cloud to remove outliers, thus reducing noise interference and improving the stability of subsequent planar detection.

[0046] Sub-step S122: Perform planar detection on the local point cloud of the target to obtain the planar detection result.

[0047] In this embodiment, the target local point cloud can be fitted with planes in any way to obtain plane detection results. The plane detection results may include the detected planes. Optionally, the RANSAC (Random Sample Consensus) plane segmentation algorithm in PCL can be used to perform multi-plane detection on the local point cloud. The expression for a single plane model is: The plane normal vector is represented as: The RANSAC algorithm estimates plane parameters by randomly selecting the minimum sample point set and statistically analyzing the set of interior points that satisfy the distance constraints. ,in, The threshold is the distance from a point to a plane. Points that meet the condition are the set of interior points. The model with the largest number of interior points is selected as the current optimal plane.

[0048] Since the object surface may have multiple graspable planes, this solution employs an iterative "successive peeling" strategy for multi-plane detection. This method can be used as follows: Figure 5 As shown. Please refer to... Figure 5 , Figure 5 for Figure 4 A flowchart illustrating the sub-steps included in neutron step S122. In this embodiment, sub-step S122 may include sub-steps S1221 to S1224.

[0049] Sub-step S1221: Perform a planar detection on the current point cloud to be detected to obtain the current object plane and the current plane point cloud corresponding to the current object plane.

[0050] Sub-step S1221: Determine whether the first total number of points in the current planar point cloud meets the preset number requirement.

[0051] If the first total number of point cloud points in the current planar point cloud meets the preset number requirement, then execute sub-step S1223.

[0052] Sub-step S1223: Save the current object plane to the plane detection result corresponding to the second object to be captured, and delete the current plane point cloud from the current point cloud to be detected in order to update the current point cloud to be detected.

[0053] If the first total number of points in the current planar point cloud does not meet the preset number requirement, then sub-step S1224 is executed.

[0054] Sub-step S1224: Do not save the current object plane to the plane detection result corresponding to the second object to be captured, and stop detection.

[0055] In this embodiment, an arbitrary plane detection algorithm can be used to perform a plane detection on the current point cloud to be detected. The plane detected in this instance is taken as the current object plane, and the point cloud corresponding to the detected plane is taken as the current plane point cloud corresponding to the current object plane. The RANSAC algorithm can be used for this initial plane detection. Initially, the current point cloud to be detected is the target local point cloud. The current plane point cloud includes the point cloud points in the current point cloud to be detected that belong to the current object plane. The current object plane is the plane with the largest number of corresponding plane point clouds detected from the current point cloud to be detected.

[0056] Then, it is determined whether the first total number of points in the current planar point cloud meets a preset quantity requirement, in order to determine whether the scale of the current object plane meets the corresponding requirements. Specifically, when the first total number meets the preset quantity requirement, the first total number is greater than or equal to the quantity specified in the preset quantity requirement, meaning the scale of the current object plane meets the corresponding requirements. The specific preset quantity requirement can be determined based on actual needs and is not specifically limited here.

[0057] Alternatively, as a possible implementation, the determination can be made in the following way.

[0058] Optionally, the preset quantity requirement includes a first quantity. The first quantity is the product of a first preset ratio and a second total quantity of point cloud points in the target local point cloud. The first preset ratio can be set according to actual needs, for example, to 0.1. It is determined whether the first total quantity of point cloud points in the current planar point cloud is less than the first quantity. If the first total quantity is less than the first quantity, it is determined that the first total quantity does not meet the preset quantity requirement. If the first total quantity is greater than or equal to the first quantity, it is determined that the first total quantity meets the preset quantity requirement. Thus, based on the point proportion constraint in the plane, it is determined whether the current object plane has a sufficient set size, and then it is determined whether to continue planar detection.

[0059] Optionally, the preset quantity requirement includes a second quantity. The second quantity is the product of a second preset ratio and a third total quantity of point cloud points in the current point cloud to be detected. The second preset ratio can be set according to actual needs, for example, to 0.5. It is determined whether the first total quantity of point cloud points in the current planar point cloud is less than the second quantity. If the first total quantity is less than the second quantity, it is determined that the first total quantity does not meet the preset quantity requirement. If the first total quantity is greater than or equal to the second quantity, it is determined that the first total quantity meets the preset quantity requirement. Thus, based on the remaining point ratio constraint, it is determined whether the first total quantity meets the preset quantity requirement, and then whether to continue planar detection.

[0060] Optionally, the preset quantity requirement includes a first quantity and a second quantity. It is determined whether the first total number of point cloud points in the current planar point cloud is less than the first quantity and whether it is less than the second quantity. If the first total number is greater than or equal to both the first and second quantities, then the first total number is determined to meet the preset quantity requirement. If the first total number is less than either the first or the second quantity, then the first total number is determined not to meet the preset quantity requirement.

[0061] If the first total quantity is determined to meet the preset quantity requirement, the current object plane is saved to the plane detection result corresponding to the second object to be captured. Furthermore, the current plane point cloud is deleted from the current point cloud to be detected. The current point cloud to be detected after deleting the current plane point cloud becomes the updated current point cloud to be detected, and sub-step S1221 can be executed again based on the updated current point cloud to be detected.

[0062] If it is determined that the first total number does not meet the preset number requirement, the current object plane may not be saved to the plane detection result corresponding to the second object to be grabbed, and the detection may be stopped. The plane detection result corresponding to the second object to be grabbed may be used as the plane detection result for subsequent determination of the target grabbing plane.

[0063] Sub-step S123: Determine the target grasping plane from the plane detection results.

[0064] Optionally, in this embodiment, the plane normal vector can be extracted for each plane in the plane detection result: The angle between the plane's normal vector and the Z-axis of the world coordinate system is calculated. The plane corresponding to the smallest included angle among the obtained angles is taken as the target grasping plane. That is, the target grasping plane is the plane with the smallest angle between the plane normal vector and the Z-axis of the world coordinate system among the planes included in the plane detection results. In this way, the grasping surface can be given priority to face upward, improving grasping stability.

[0065] Alternatively, the plane detection result can be obtained by performing a single plane detection, where the number of planes in the detection result is no greater than 1, and the number of planes obtained from a single plane detection is no greater than 1. In this method, if a plane is obtained through a single plane detection, it can be directly used as the target grabbing plane.

[0066] Sub-step S124: Obtain the grasping reference information based on the target grasping plane.

[0067] Optionally, as a possible implementation, the average coordinates of the planar point cloud corresponding to the target grasping plane can be calculated directly, and this average coordinates can be used as the center position in the grasping reference information. Alternatively, the three-dimensional corner positions can be determined based on the planar point cloud corresponding to the target grasping plane, and the corner positions can be used as information in the grasping reference information.

[0068] Alternatively, as another possible implementation, it can be achieved through... Figure 6 The planar 2D model and geometric structure extraction are performed as shown to obtain the grasping reference information. Please refer to... Figure 6 , Figure 6 for Figure 4 A flowchart illustrating the sub-steps included in neutron step S124. In this embodiment, sub-step S124 may include sub-steps S1241 to S1245.

[0069] Sub-step S1241: Obtain transformation information to transform the target grabbing plane to the horizontal plane of the world coordinate system.

[0070] Sub-step S1242: Based on the transformation information, the target plane point cloud corresponding to the target grasping plane is transformed to the horizontal plane of the world coordinate system to obtain the target point set.

[0071] Sub-step S1243: Perform convex hull calculation on the target point set to obtain the outer plane point set.

[0072] Sub-step S1244: Obtain the minimum area bounding rectangle based on the set of outer points of the plane.

[0073] Sub-step S1245: Obtain rectangle information based on the minimum area bounding rectangle, and obtain the grasping reference information based on the rectangle information and the transformation information.

[0074] The center position is obtained based on the center of the rectangle in the rectangle information.

[0075] Since the detected target grasping plane is usually in an arbitrary spatial pose, directly calculating the planar envelope structure in three-dimensional space is not only computationally complex but also difficult to describe uniformly. Therefore, this scheme constructs a planar coordinate transformation relationship to map the target grasping plane to a two-dimensional plane, completing the geometric structure analysis in two-dimensional space.

[0076] In this embodiment, a planar two-dimensional transformation is first performed. The normalized normal vector of the target grasping plane can be assumed to be: , Let be the plane normal vector. Then, by calculating the rotation relationship between the plane normal vector and the Z-axis of the world coordinate system, a rotation matrix R is constructed to rotate the target grabbing plane to the standard XY plane (i.e., the horizontal plane of the world coordinate system): The Z-axis of the world coordinate system is represented as: The rotation matrix R can be constructed using the Rodrigues formula.

[0077] The rotation matrix R can be constructed as follows.

[0078] First, calculate the rotation axis. The cross product between the plane normal vector of the target grasping plane and the target's Z-axis is expressed as: ,in, Indicates the direction of the rotation axis, and its modulus length Less than the smallest threshold When the plane normal vector is parallel to the Z-axis, no rotation is needed, and the rotation matrix R can be the identity matrix.

[0079] If the plane normal vector of the target grabbing plane is not parallel to the Z-axis, then calculate the angle between them: ,in, The angle required to rotate the plane normal vector to the Z-axis; normalization is applied to the rotation axis: .

[0080] Next, based on the rotation axis and rotation angle, the rotation matrix is ​​constructed using Rodrigues' formula. First, the antisymmetric matrix of the rotation axis is constructed:

[0081] Therefore, the expression for the rotation matrix is: .

[0082] After rotation, although the target grasping plane is now parallel to the XY plane of the world coordinate system, its spatial position may still have a height offset. Therefore, in this scheme, the center point of the planar point cloud corresponding to the target grasping plane is further calculated: ,in, To capture points on a plane for the target, N is the number of points on the plane.

[0083] Rotate the center point: .

[0084] set up Then construct the translation vector: .

[0085] Finally, the planar coordinate transformation relationship is obtained (that is, the transformation information of transforming the target capturing plane to the horizontal plane of the world coordinate system): Based on this planar coordinate transformation relationship, the target planar point cloud can be transformed to the horizontal plane of the world coordinate system to obtain the target point set.

[0086] After completing the planar 2D transformation, geometric structure extraction can be performed. The original target point cloud of the arbitrary-pose target grasping plane is uniformly mapped to the XY plane of the world coordinate system, and the transformed target point set can be denoted as: .

[0087] Can target point set Perform convex hull calculation to obtain the set of points on the outer edge of the plane: Next, calculate the minimum area circumscribed rectangle based on the convex hull boundary: The set of parameters for the circumscribed rectangle with the smallest area is obtained as the rectangle information. This rectangle information may include the center of the rectangle.

[0088] Optionally, the rectangle information may include length, width, center of the rectangle, and four corner points:

[0089] in, All of these can be obtained from the minimum area circumscribed rectangle mentioned above. Let C be the side length of the rectangle, and let C be the center of the rectangle as the gripping position. Two-dimensional corner points can be used to determine the gripping direction.

[0090] Finally, for the robot to grasp, the calculated rectangle center can be restored to three-dimensional space based on the transformation information, and the transformed rectangle center can be used as the center position in the grasping reference information. The corner points can also be restored to three-dimensional space to obtain their positions.

[0091] Optionally, after obtaining the grasping reference information of each second object to be grasped, each second object to be grasped can be directly used as the target object to be grasped, and the grasping pose of each target object can be determined based on the grasping reference information of each target object.

[0092] Alternatively, the target object can be determined from the identified second objects to be crawled by combining the set filtering rules, and then the crawling information can be generated.

[0093] As one possible implementation method, it can be adopted Figure 7 The information is filtered as shown to obtain the crawled information. Please refer to... Figure 7 , Figure 7 for Figure 2 One of the flowcharts for the sub-steps included in step S130. In this embodiment, sub-step S130 may include sub-steps S1311 to S1313.

[0094] Sub-step S1311: For each second object to be grasped, determine whether the center position of the second object to be grasped is within the preset grasping area.

[0095] Sub-step S1312: If the center position of the second object to be grasped is not within the preset grasping area, then the second object to be grasped will not be used as the target object to be grasped.

[0096] Sub-step S1313: Based on the second object to be grabbed whose center position is within the preset grabbing area, the grabbing information is obtained.

[0097] In this embodiment, a three-dimensional preset grasping area can be determined based on the stacked area of ​​the objects to be grasped. After obtaining the grasping reference information of a second object to be grasped, it can be determined whether the target position in the grasping reference information of the second object to be grasped is within the preset grasping area. If the center position of the target grasping plane of the second object to be grasped is not within the preset grasping area, then the second object to be grasped is not considered as a target object. If the center position of the target grasping plane of the second object to be grasped is within the preset grasping area, the second object to be grasped can be directly considered as a target object, or further determined by combining other filtering rules. Then, the corresponding grasping pose can be generated based on the grasping reference information of each target object. In this way, the generation of grasping poses for objects that do not need to be grasped can be avoided.

[0098] As another possible implementation, one could adopt Figure 8 The information is filtered as shown to obtain the crawled information. Please refer to... Figure 8 , Figure 8 for Figure 2 The second flowchart illustrates the sub-steps included in step S130. In this embodiment, sub-step S130 may include sub-steps S1321 to S1323.

[0099] Sub-step S1321: For each second object to be grasped, determine whether the planar dimensions in the grasping reference information of the second object to be grasped are within the preset size range.

[0100] Wherein, the plane dimension is the dimension of the target grasping plane.

[0101] Sub-step S1322: If the planar size of the second object to be grasped is not within the preset size range, then the second object to be grasped will not be used as the target object to be grasped.

[0102] Sub-step S1323: Based on the second object to be grasped whose planar dimensions are within the preset size range, obtain the grasping information.

[0103] Optionally, for each second object to be grasped, an area can be calculated based on the length and width of the minimum area bounding rectangle corresponding to the second object to be grasped, as obtained above. This area can then be used as the planar dimension of the second object to be grasped and incorporated into the grasping reference information for that object. Alternatively, a preset size range can be determined in advance by combining the graspable planar dimensions.

[0104] For each second object to be grasped, it is determined whether the planar dimension in the grasping reference information of the second object to be grasped is within the preset size range. The planar dimension is the size of the target grasping plane. If the planar dimension of the second object to be grasped is not within the preset size range, then the second object to be grasped is not considered as a target grasping object. If the planar dimension of the second object to be grasped is within the preset size range, the second object to be grasped can be directly considered as a target grasping object, or further determined whether to consider the second object to be grasped as a target grasping object can be combined with other filtering rules. Then, based on the grasping reference information of each target grasping object, the corresponding grasping pose can be generated. In this way, the grasping pose of objects with target grasping planes that are too large or too small can be avoided.

[0105] Optionally, filtering can also be performed by combining a preset grasping area and a preset size range. In this method, a second object to be grasped whose center position is not within the preset grasping area or whose planar size is not within the preset size range may not be selected as the target grasping object. Alternatively, a second object to be grasped whose center position is within the preset grasping area and whose planar size is within the preset size range may be directly selected as the target grasping object, or other filtering rules may be combined to select the target grasping object.

[0106] After identifying the target object for crawling, you can... Figure 9 The grabbing information is generated as shown. In this embodiment, the grabbing reference information also includes the corner positions of the target grabbing plane, which are three-dimensional positions. Please refer to... Figure 9 , Figure 9 for Figure 2 The third flowchart illustrates the sub-steps included in step S130. In this embodiment, step S130 may include sub-steps S1331 to S1333.

[0107] Sub-step S1331: For each target object to be crawled, the reference vector corresponding to each corner position is calculated based on the corner position and center position of the target object to be crawled.

[0108] The corner points and the center point are located on the line containing the corresponding reference vector.

[0109] Sub-step S1332: The reference vector with the smallest angle to the X-axis of the robot end effector among the obtained reference vectors is taken as the X-axis direction of the grasping coordinate system.

[0110] Sub-step S1333: Based on the X-axis and Z-axis directions of the grasping coordinate system, construct the grasping posture using a right-handed coordinate system.

[0111] Wherein, the Z-axis direction of the grasping coordinate system is the direction indicated by the plane normal vector of the target grasping plane.

[0112] In this embodiment, the center position can be used as the grab position, and the direction vectors pointing from the center position to each corner position can be calculated as reference vectors: ,in, Indicates the center position. This represents the position of the i-th corner point. Next, the angles between each reference vector and the X-axis of the robot's end effector can be calculated, and the reference vector corresponding to the smallest angle is used as the X-axis direction of the grasping coordinate system. Then, using the reference vector corresponding to the smallest angle as the X-axis direction of the grasping coordinate system and the plane normal vector of the target grasping plane as the Z-axis direction, the grasping posture is constructed using a right-handed coordinate system. The grasping information may include the grasping posture of each target object and the center position of the grasping location. This allows for the construction of grasping postures that satisfy specific directional constraints and operational rules, ensuring consistency in the grasping posture. Consistency refers to: defining the X-axis direction of the suction cup coordinate system at the end of the robotic arm as coincident with the vector direction pointing from the center point of the package to the nearest corner point; that is, the suction cup X-axis is always aligned with the diagonal direction of the package; during the placement phase, controlling the suction cup X-axis to maintain a 90° perpendicular relationship with the conveyor belt's running direction, ensuring the package is placed on the conveyor belt in the same ionic state.

[0113] Optionally, as a possible implementation, after obtaining the grasping pose of each target grasping object, the target grasping objects can be sorted according to the occlusion rate of each target grasping object in ascending order of occlusion rate, and then the grasping pose of each target grasping object can be sorted based on the sorting result to obtain the grasping information.

[0114] Alternatively, as another possible implementation, it can be achieved through... Figure 10 The scraped information is obtained by sorting it as shown. Please refer to... Figure 10 , Figure 10 for Figure 2 The fourth flowchart illustrates the sub-steps included in step S130. In this embodiment, step S130 may further include sub-steps S1341 to S1342.

[0115] Sub-step S1341: Sort the target objects according to their position distribution and preset order in the scene image to obtain the sorting result.

[0116] Sub-step S1342: Sort the grabbing poses corresponding to each target grabbing object according to the sorting result to obtain the grabbing information.

[0117] In this embodiment, a preset order for object grasping can be determined by combining the image space corresponding to the scene image and the placement space of the grasped object to reduce collisions. Through the aforementioned recognition and analysis, the positional distribution of each target grasping object in the scene image can be determined. Then, based on the position of each target grasping object in the scene image and the preset order, the target grasping objects are sorted to obtain a sorting result. Subsequently, the grasping poses of each target grasping object are sorted based on this sorting result to obtain the grasping information. That is, the sorting of each target grasping object in the grasping information is the sorting result indicating the sorting order. When grasping target objects subsequently based on the grasping information, the robot or robotic arm can grasp them according to the sorting result.

[0118] For example, such as Figure 11 As shown, A is the image space, and B is the placement space. The placement space is on the left side of the image space. In this case, to avoid collisions, the preset order can be set as: left, right, top, bottom. When there are target objects C1, C2, and C3 in image space A (where C1 is located on the far left of the image space, C2 is located in the middle of the image space, and C3 is located on the right side of the image space), the sorting result is: C1, C2, C3. Then, during the grabbing process, target object C1 is grabbed first, then target object C2, and finally target object C3.

[0119] The grasping information acquisition method provided in this embodiment is a method for calculating the graspable position and grasping posture of an object (e.g., quickly) surface by combining occlusion rate screening, local point cloud multi-plane recursive detection, and planar two-dimensional geometric structure extraction. Based on the two-dimensional visual segmentation results, it introduces an occlusion rate quantification evaluation mechanism and a local three-dimensional plane analysis mechanism, combined with directional constraints and geometric structure extraction, to achieve stable grasping pose calculation in stacked scenes. In the above method, by constructing a minimum rotation bounding rectangle from the instance segmentation results and calculating the coverage ratio of other targets within this rectangle, an effective quantitative evaluation of the target occlusion degree is achieved, thereby filtering out graspable targets with low occlusion, which can improve the accuracy of target selection in complex stacked scenes. Furthermore, through the aforementioned multi-plane recursive detection method, the main planar structure of the object surface can be effectively extracted, and the dynamic proportional termination mechanism (i.e., the dual stopping conditions of the remaining point proportion constraint and the in-plane point proportion constraint mentioned above) suppresses noise plane interference, improving the robustness and stability of planar detection. Furthermore, by constructing a coordinate transformation matrix, the target grasping plane in three-dimensional space is mapped to a standard two-dimensional plane. Convex hull analysis and minimum bounding rectangle calculation are performed in two-dimensional space to stably obtain geometric information such as the center point, size, and boundary corner points of the grasping area. Based on the directional relationship between the center point and the corner points, a grasping posture matrix is ​​generated to improve the rationality of the robot's grasping direction and the grasping success rate.

[0120] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of the information acquisition device 200 is given below. Optionally, the information acquisition device 200 can adopt the above-described... Figure 1 The device structure of the electronic device 100 shown. Further, please refer to... Figure 12 , Figure 12 This is a block diagram of the crawling information acquisition device 200 provided in this embodiment. It should be noted that the basic principle and technical effects of the crawling information acquisition device 200 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. In this embodiment, the crawling information acquisition device 200 may include: an occlusion rate filtering module 210, a reference information acquisition module 220, and an information generation module 230.

[0121] The occlusion rate filtering module 210 is used to analyze the two-dimensional scene image, obtain the occlusion rate corresponding to each first object to be captured in the scene image, and take the first object to be captured with an occlusion rate less than a preset occlusion rate as the second object to be captured.

[0122] The reference information acquisition module 220 is used to analyze and obtain the grasping reference information of each second object to be grasped based on the local point cloud corresponding to the second object to be grasped. The grasping reference information includes the center position of the target grasping plane and the target grasping plane itself.

[0123] The information generation module 230 is used to generate grasping information based on the grasping reference information of each second object to be grasped. The grasping information includes the grasping pose of the target object, which is a second object to be grasped, and the grasping pose includes the center position and the grasping posture obtained from the normal vector of the target grasping plane.

[0124] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 executes the program. Meanwhile, the data and program code required to execute the above modules can be stored in the memory 110.

[0125] This application embodiment also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned information capture and acquisition method.

[0126] In summary, this application provides a method, apparatus, electronic device, and readable storage medium for obtaining grasping information. First, by analyzing a two-dimensional scene image, the occlusion rate of each first candidate grasping object in the scene image is obtained, and the first candidate grasping objects with occlusion rates less than a preset occlusion rate are designated as second candidate grasping objects. Then, for each second candidate grasping object, grasping reference information is obtained by analyzing the local point cloud corresponding to the second candidate grasping object. This grasping reference information includes the center position of the target grasping plane and the target grasping plane itself. Finally, based on the grasping reference information of each second candidate grasping object, grasping information is generated. This grasping information includes the grasping pose of each target grasping object, where the target grasping object is the second candidate grasping object, and the grasping pose includes the aforementioned center position and the grasping posture obtained from the normal vector of the target grasping plane. Thus, by selecting grasping objects based on their occlusion rates, severely occluded objects can be avoided as grasping objects, thereby improving the robot's grasping success rate in stacked occlusion scenarios.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0128] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0129] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] The above description is merely an optional embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for obtaining information by data capture, characterized in that, The method includes: The two-dimensional scene image is analyzed to obtain the occlusion rate of each first object to be captured in the scene image, and the first object to be captured with an occlusion rate less than a preset occlusion rate is taken as the second object to be captured. For each second object to be captured, based on the local point cloud corresponding to the second object to be captured, the capture reference information of the second object to be captured is obtained by analysis, wherein the capture reference information includes the center position of the target capture plane and the target capture plane; Based on the grasping reference information of each second object to be grasped, grasping information is generated, wherein the grasping information includes the grasping pose of the target object to be grasped, the target object to be grasped is the second object to be grasped, and the grasping pose includes the center position and the grasping posture obtained by the normal vector of the target grasping plane.

2. The method according to claim 1, characterized in that, The step of analyzing the two-dimensional scene image to obtain the occlusion rate corresponding to each first object to be captured in the scene image includes: The scene image is subjected to object recognition analysis to obtain the object region corresponding to each first object to be captured, wherein the object region is the region of the first object to be captured in the scene image; For each first object to be grabbed, the minimum rotational bounding rectangle region of the first object to be grabbed is obtained based on the object region of the first object to be grabbed. For each first object to be crawled, the occlusion rate of the first object to be crawled is calculated based on the distribution of other first objects to be crawled within the minimum rotational bounding rectangle region of the first object to be crawled.

3. The method according to claim 1, characterized in that, For each second object to be captured, based on the local point cloud corresponding to the second object, the capturing reference information of the second object to be captured is analyzed and obtained, including: The local point cloud corresponding to the second object to be captured is preprocessed to obtain the target local point cloud; Planar detection is performed on the local point cloud of the target to obtain the planar detection result; The target grabbing plane is determined from the plane detection results, wherein the target grabbing plane is the plane with the smallest angle between the plane normal vector and the Z-axis of the world coordinate system among the planes included in the plane detection results; The grasping reference information is obtained based on the target grasping plane.

4. The method according to claim 3, characterized in that, The step of performing planar detection on the target local point cloud to obtain planar detection results includes: Perform a planar detection on the current point cloud to be detected to obtain the current object plane and the current planar point cloud corresponding to the current object plane. The initial current point cloud to be detected is the target local point cloud. The current planar point cloud includes the point cloud points in the current point cloud to be detected that belong to the current object plane. The current object plane is the plane with the largest number of point cloud points detected from the current point cloud to be detected. Determine whether the first total number of points in the current planar point cloud meets a preset number requirement, wherein when the first total number meets the preset number requirement, the first total number is greater than or equal to the number in the preset number requirement; If the conditions are met, the current object plane is saved to the plane detection result corresponding to the second object to be captured, and the current plane point cloud is deleted from the current point cloud to be detected to update the current point cloud to be detected, and detection continues. If the conditions are not met, the current object plane will not be saved to the plane detection result corresponding to the second object to be captured, and the detection will stop.

5. The method according to claim 4, characterized in that, The preset quantity requirement includes a first quantity and / or a second quantity. Determining whether the first total number of point cloud points in the current planar point cloud meets the preset quantity requirement includes: Determine whether the first total number of point cloud points in the current planar point cloud is less than a first quantity, wherein the first quantity is the product of a first preset ratio and a second total number of point cloud points in the target local point cloud; and / or, Determine whether the first total number is less than the second number, wherein the second number is the product of the second preset ratio and the third total number of point cloud points in the current point cloud to be detected.

6. The method according to claim 3, characterized in that, The local point cloud is obtained by back-projection calculation based on the object region and corresponding depth information of the second object to be captured, or by clustering analysis of the overall scene point cloud; and / or, the preprocessing includes downsampling and / or removal of outliers; and / or, the plane detection result is obtained by performing a single plane detection, wherein the number of planes in the plane detection result is no greater than 1, and the number of planes obtained by a single plane detection is no greater than 1. And / or, obtaining the grasping reference information based on the target grasping plane includes: Obtain transformation information to transform the target grabbing plane to a horizontal plane in the world coordinate system; Based on the transformation information, the target plane point cloud corresponding to the target grasping plane is transformed to the horizontal plane of the world coordinate system to obtain the target point set; Perform convex hull calculation on the target point set to obtain the outer plane point set; Based on the set of points on the outer edge of the plane, obtain the circumscribed rectangle with the minimum area; The rectangle information is obtained based on the minimum area circumscribed rectangle, and the grasping reference information is obtained based on the rectangle information and the transformation information, wherein the center position is obtained based on the center of the rectangle in the rectangle information.

7. The method according to claim 1, characterized in that, The grasping reference information also includes the corner positions of the target grasping plane. The step of generating grasping information based on the grasping reference information of each second object to be grasped includes: For each target object to be crawled, a reference vector corresponding to each corner position is calculated based on the corner position and center position of the target object. The corner position and center position are located on the straight line where the corresponding reference vector is located. The reference vector with the smallest angle to the X-axis of the robot's end effector is taken as the X-axis direction of the grasping coordinate system. The grasping posture is constructed using a right-handed coordinate system based on the X-axis and Z-axis directions of the grasping coordinate system, wherein the Z-axis direction of the grasping coordinate system is the direction indicated by the plane normal vector of the target grasping plane.

8. The method according to any one of claims 1-7, characterized in that, The step of generating crawling information based on the crawling reference information of each second object to be crawled also includes: For each second object to be crawled, determine whether its center position is within a preset crawling area; if its center position is not within the preset crawling area, then do not include the second object as a target object; based on the second objects whose center position is within the preset crawling area, obtain the crawling information; and / or, For each second object to be grasped, it is determined whether the planar dimension in the grasping reference information of the second object to be grasped is within a preset size range, wherein the planar dimension is the dimension of the target grasping plane; if the planar dimension of the second object to be grasped is not within the preset size range, then the second object to be grasped is not regarded as a target grasping object; based on the second object to be grasped whose planar dimension is within the preset size range, the grasping information is obtained.

9. The method according to any one of claims 1-7, characterized in that, The step of generating crawling information based on the crawling reference information of each second object to be crawled also includes: Based on the position distribution and preset order of each target object in the scene image, the target objects are sorted to obtain a sorting result. The preset order is a sequence determined based on the image space corresponding to the scene image and the placement space of the object after it is grabbed, with the aim of reducing collisions. The grabbing poses corresponding to each target grabbing object are sorted according to the sorting result to obtain the grabbing information. When grabbing a target grabbing object based on the grabbing information, the grabbing is performed according to the sorting result.

10. An information acquisition device, characterized in that, The device includes: The occlusion rate filtering module is used to analyze the two-dimensional scene image, obtain the occlusion rate of each first object to be captured in the scene image, and take the first object to be captured with an occlusion rate less than a preset occlusion rate as the second object to be captured. The reference information acquisition module is used to analyze and obtain the grasping reference information of each second object to be grasped based on the local point cloud corresponding to the second object to be grasped. The grasping reference information includes the center position of the target grasping plane and the target grasping plane. The information generation module is used to generate grasping information based on the grasping reference information of each second object to be grasped. The grasping information includes the grasping pose of the target object, which is the second object to be grasped. The grasping pose includes the center position and the grasping posture obtained by the normal vector of the target grasping plane.

Images