Suction cup array control method, device, electronic device and storage medium

Abstract

The present application discloses a suction cup array control method, device, electronic device and storage medium. The suction cup array control method includes: obtaining point cloud information of the glass to be grasped and obstacles; generating a search state based on the point cloud information and the search boundary factor parameters of the suction cup array; for each search state, judging whether there is an obstacle under each suction cup, and if so, closing the suction cup in the search state; selecting an alternative search state according to the suction cup opening status of each search state; selecting the best search state from the alternative search states according to the number of suction cups opened and / or the distance between the suction cup and the center of the glass; and grasping the glass using the best search state. The present invention is specifically used for the industrial scenario of using a suction cup array for glass grasping, so that even if there are obstacles that cannot be grasped on the glass to be grasped, the glass can be grasped accurately by avoiding obstacles, thereby improving the accuracy and stability of glass grasping using a suction cup array.

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and more specifically, to a suction cup array control method, device, electronic device and storage medium. Background Art

[0002] At present, with the widespread popularity of intelligent program-controlled robots, more and more objects can be grasped and transported with the help of intelligent program-controlled robots. For example, logistics packaging can be grasped by intelligent program-controlled robots, which greatly improves the grasping efficiency. In order to improve the grasping efficiency and to be able to flexibly adapt to a variety of object objects, intelligent program-controlled robots are usually installed with a fixture group consisting of multiple fixtures, so that different fixtures in the fixture group can be flexibly called according to different object objects. Different fixtures can grasp different objects. For example, a suction cup array can absorb objects of glass-like materials, but once it encounters rubber or bumps, it will cause grasping failure. Objects or structures that cannot be grasped by such fixtures are called obstacles.

[0003] Conventional intelligent robots can only perform obstacle avoidance grasping for fixed-model grasping objects and simple obstacles. In this case, the size, shape and position of the grasping objects and obstacles are fixed. In the prior art, the clamp is set at the center of the grasping object and the non-obstacle position according to the model and the position of the obstacle, so as to avoid obstacles for grasping. However, this obstacle avoidance grasping method has the following defects: first, the grasping method can only be used for objects with obstacles in fixed positions and fixed models, that is, when the model of the object to be grasped is unknown or the model of the object to be grasped is known but the position of the obstacle is not fixed, it is impossible to accurately avoid obstacles for grasping; secondly, the grasping method does not determine whether multiple clamps can grasp objects stably. If the number of clamps used is insufficient or the arrangement of multiple clamps is inappropriate, for example, multiple clamps are arranged in a straight line, after grasping the object in this case, the center of gravity is unstable, and the object may swing during the grasping process, causing the object to fall or collide with other unexpected objects and be damaged. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to overcome the above problems or at least partially solve the above problems. Specifically, according to the above embodiments, firstly, the present invention can first obtain the possible grasping methods of the clamp, and select the best grasping method that will not grasp the obstacles to execute the grasping of the object, so that even if there are obstacles that cannot be grasped on the object, it is possible to accurately avoid obstacles and grasp the object; secondly, for obstacles such as protrusions / depressions that may cause the clamp to fail to grasp correctly, the present invention proposes a method for determining whether there is a non-planar structure in a specific area by grouping and determining the depth value difference of each pixel point in the specific area and the number of different depth values. Since this method uses a numerical statistical method rather than a method of determining the specific position of the non-planar structure to determine whether the non-planar structure exists, This process is highly efficient and practical, and can also be applied to industrial scenarios other than grasping that may require determining whether there are non-planar structures on the surface of an object; thirdly, the present invention can pre-verify whether the grasping method used can correctly grasp the object before the clamp group performs grasping, thereby improving the stability of grasping and avoiding problems such as unstable center of gravity that may occur during grasping, resulting in damage or loss of the grasped object; finally, based on the general obstacle avoidance grasping method and clamp verification method, the present invention has developed an obstacle avoidance grasping method and a suction cup verification method specifically for the industrial scenario of glass grasping using a suction cup array, which can improve the accuracy and stability of glass grasping using a suction cup array. It can be seen that the present invention solves all aspects of the problems that arise in the industrial scenario of grasping objects using a clamp.

[0005] All the solutions disclosed in the claims and the specification of this application have one or more of the above-mentioned innovations, and accordingly, can solve one or more of the above-mentioned technical problems. Specifically, this application provides a suction cup array control method, device, electronic device and storage medium.

[0006] The suction cup array control method of the embodiment of the present application includes:

[0007] Obtain point cloud information of the glass to be grasped and obstacles;

[0008] Generate a search state based on the point cloud information and the search boundary factor parameters of the suction cup array;

[0009] For each search state, determine whether there is an obstacle under each suction cup, and if so, close the suction cup in the search state;

[0010] Select an alternative search state according to the suction cup opening status of each search state;

[0011] Selecting the best search state from the alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass;

[0012] Grab the glass using the best search state.

[0013] In some embodiments, the obstacle determination includes at least one of the following: glass boundary obstacle determination, rubber pad obstacle determination, rubber strip obstacle determination and protrusion obstacle determination.

[0014] In some embodiments, the glass boundary obstacle determination includes determining whether there is a glass boundary obstacle based on a point cloud ratio in an area below the suction cup.

[0015] In some embodiments, the rubber pad obstacle determination includes: determining whether there is a rubber pad obstacle based on an obstacle point cloud area, wherein the obstacle point cloud area is pre-set according to the rubber pad area.

[0016] In some embodiments, the protrusion obstacle determination includes: determining whether a protrusion obstacle exists based on a preset depth difference threshold and a logarithmic threshold.

[0017] In some implementations, selecting the candidate search state according to the opening status of the suction cup in each search state includes: if there is at least one opened suction cup, selecting the search state as the candidate search state.

[0018] In some embodiments, selecting the best search state from alternative search states based on the number of suction cups opened and / or the distance between the suction cups and the center of the glass includes: selecting the search state with the largest number of suction cups opened; if there are multiple search states with the largest number of suction cups opened, further selecting the search state in which the suction cup is closest to the center of the glass.

[0019] In certain embodiments, a gripper calibration is performed for the optimal search state to determine whether the glass can be properly gripped using the search state.

[0020] The suction cup array control device of the embodiment of the present application includes:

[0021] Point cloud acquisition module, used to obtain point cloud information of the glass to be grasped and obstacles;

[0022] A search state generation module, used to generate a search state based on point cloud information and search boundary factor parameters of the suction cup array;

[0023] An obstacle determination module is used to determine whether there is an obstacle under each suction cup for each search state, and if so, close the suction cup in the search state;

[0024] An alternative search state selection module is used to select an alternative search state according to the suction cup opening status of each search state;

[0025] An optimal search state selection module, used for selecting an optimal search state from alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass;

[0026] The grasping module is used for grasping the glass based on the best search state.

[0027] In some embodiments, the obstacle determination module is used to perform at least one of the following obstacle determinations: glass boundary obstacle determination, rubber pad obstacle determination, rubber strip obstacle determination, and protrusion obstacle determination.

[0028] In some embodiments, when the obstacle determination module performs the glass boundary obstacle determination, it determines whether there is a glass boundary obstacle according to the point cloud ratio in the area below the suction cup.

[0029] In some embodiments, when the obstacle determination module performs rubber pad obstacle determination, it determines whether there is a rubber pad obstacle based on the area of ​​the obstacle point cloud, wherein the area of ​​the obstacle point cloud is pre-set according to the area of ​​the rubber pad.

[0030] In some embodiments, when the obstacle determination module performs the protruding obstacle determination, it determines whether there is a protruding obstacle based on a preset depth difference threshold and a logarithmic threshold.

[0031] In some implementations, the candidate search state selection module selects a search state as a candidate search state when there is at least one opened suction cup in a certain search state.

[0032] In some embodiments, the optimal search state selection module selects the search state with the largest number of open suction cups as the optimal search state. If there are multiple search states with the largest number of open suction cups, the search state in which the suction cup is closest to the center of the glass is further selected.

[0033] In certain embodiments, a gripper calibration is performed for the optimal search state to determine whether the glass can be properly gripped using the search state.

[0034] The electronic device of the embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the suction cup array control method of any of the above embodiments is implemented.

[0035] The computer-readable storage medium of the embodiment of the present application stores a computer program thereon, and when the computer program is executed by a processor, the suction cup array control method of any of the above embodiments is implemented.

[0036] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0038] Figure 1 is a flow chart of a gripper obstacle avoidance grasping method according to certain embodiments of the present application;

[0039] Figure 2 It is a schematic flow chart of a method for determining a non-planar structure on the surface of an object in certain embodiments of the present application;

[0040] Figure 3 is a schematic flow chart of a fixture calibration method according to certain embodiments of the present application;

[0041] Figure 4 is a schematic flow chart of a glass grabbing method using a suction cup according to certain embodiments of the present application;

[0042] Figure 5 is a schematic diagram of glass and obstacle point clouds and suction cup arrangements in certain embodiments of the present application;

[0043] Figure 6 It is a schematic diagram of a flow chart of obstacle judgment during glass grabbing in some preferred embodiments of the present application;

[0044] Figure 7 is a schematic diagram of a gluing process for a strip to be applied according to certain embodiments of the present application;

[0045] Figure 8 is a schematic flow chart of a suction cup array calibration method according to certain embodiments of the present application;

[0046] Fig. 9 is a schematic structural diagram of a fixture obstacle avoidance grasping device according to certain embodiments of the present application;

[0047] Fig.10 It is a schematic diagram of the structure of a device for determining a non-planar structure on the surface of an object according to some embodiments of the present application;

[0048] Fig.11 is a schematic diagram of the structure of a fixture calibration device according to certain embodiments of the present application;

[0049] Fig.12 is a schematic structural diagram of a glass gripping device using a suction cup in certain embodiments of the present application;

[0050] Fig.13 is a schematic diagram of the structure of a suction cup array calibration device according to certain embodiments of the present application;

[0051] Fig.14 It is a schematic diagram of the structure of an electronic device of certain embodiments of the present application. DETAILED DESCRIPTION

[0052] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0053] Figure 1 FIG. 1 shows a flow chart of a robot-based gripper obstacle avoidance grasping method according to an embodiment of the present invention, as shown in FIG. Figure 1 As shown, the method includes:

[0054] Step S100, obtaining point cloud information of the group of objects to be grasped.

[0055] The group of objects to be grasped may include one or more objects to be grasped, and may also include one or more obstacles. Obstacles may exist on the objects to be grasped, such as protrusions on the objects to be grasped, adhesive strips attached to the objects to be grasped, etc. Obstacles may also exist outside the objects to be grasped, such as the objects to be grasped may be stacked in layers, and there may be rubber pads between the layers, etc. Accordingly, the point cloud information of the group of objects to be grasped may include point cloud information of one or more objects to be grasped and / or obstacles.

[0056] As an example, point cloud information can be obtained through a 3D industrial camera. A 3D industrial camera is generally equipped with two lenses, which capture the group of objects to be grasped from different angles. After processing, the three-dimensional image of the object can be displayed. The group of objects to be grasped is placed under the visual sensor, and the two lenses shoot at the same time. According to the relative posture parameters of the two images obtained, a general binocular stereo vision algorithm is used to calculate the X, Y, and Z coordinate values ​​of each point of the glass to be coated and the coordinate orientation of each point, and then converted into point cloud data of the group of objects to be grasped. In specific implementation, laser detectors, visible light detectors such as LEDs, infrared detectors, and radar detectors can also be used to generate point clouds. The present invention does not limit the specific implementation method.

[0057] The point cloud data obtained in the above manner is three-dimensional data. In order to filter out the data corresponding to the dimension that has less impact on the grasping, reduce the amount of data processing and thus speed up the data processing speed and improve efficiency, the acquired three-dimensional point cloud data of the group of objects to be grasped can be orthographically projected onto a two-dimensional plane.

[0058] As an example, a depth map corresponding to the orthographic projection may also be generated. A two-dimensional color map corresponding to the three-dimensional object area and a depth map corresponding to the two-dimensional color map may be obtained along the depth direction perpendicular to the object. The two-dimensional color map corresponds to an image of a plane area perpendicular to a preset depth direction; each pixel in the depth map corresponding to the two-dimensional color map corresponds one-to-one to each pixel in the two-dimensional color map, and the value of each pixel is the depth value of the pixel.

[0059] Step S110: Generate a search state based on the point cloud information of the group of objects to be grasped and the search boundary factor parameters of the fixture.

[0060] The search boundary factor parameters of the fixture include the control boundary of the controllable parameters of the fixture, the control granularity and other boundary factor parameters. Depending on the actual situation, for example, the type of fixture used, the type and size of the object to be grasped, etc., the search boundary factor parameters and values ​​can be different.

[0061] As an example, the controllable parameters may include the moving distance in the X direction, the moving distance in the Y direction, and the rotation angle. The moving distance boundary in the X direction (i.e., the controllable boundary) may be set to -500mm to 500mm, and the control granularity is 100mm. Then the fixture can move from -500mm to 500mm in the X direction in units of 100mm, so there are 11 search states in the X direction; the moving distance boundary in the Y direction may be set to -100mm to 100mm, and the control granularity is 50mm. Then the fixture can move from -100mm to 100mm in the Y direction in units of 50mm, so there are 5 search states in the Y direction; the rotation angle may be set to -50 degrees to +50 degrees, and the control granularity is 10 degrees. Then the fixture can rotate from -50 degrees to +50 degrees in units of 10 degrees, so there are 11 search states in the rotation angle. Under the setting of the above search boundary factor parameters, there are a total of 11×5×11=605 search states. In addition, other boundary factor parameters can also be set. For example, the moving distance of the fixture can be set not to exceed 100 mm from a certain side. Then, when the fixture moves toward that side, it can only move to a position 100 mm away from that side.

[0062] In order to facilitate the control of the fixture, the shape, size and other information of the fixture can be configured, and the configuration information can be saved using a json file. Depending on the actual situation, such as the different fixtures used or the different objects to be grasped, the configuration information can be different. For example, when using a suction cup array for grasping, the configuration information may include the position of a single suction cup relative to the center of the entire suction cup array, the number of a single suction cup, the radius, etc.

[0063] The solution of the present invention can be used for various types of clamps, for example, including various types of general clamps. General clamps refer to clamps with standardized structures and a wide range of applications, such as three-jaw chucks and four-jaw chucks for lathes, flat-nose pliers and dividing heads for milling machines, etc. For another example, according to the clamping power source used by the clamp, the clamp can be divided into manual clamps, pneumatic clamps, hydraulic clamps, gas-liquid linkage clamps, electromagnetic clamps, vacuum clamps, etc. The present invention does not limit the specific type of the clamp, as long as it can realize the object grabbing operation.

[0064] Step S120: For each search state, perform an obstacle determination, and set the search state that passes the obstacle determination as an alternative search state.

[0065] Obstacle determination is used to determine the obstacles faced by the fixture in a certain search state, such as whether the obstacle exists, whether it affects the grasping, etc. If there is no obstacle, or the obstacle does not affect the grasping, the obstacle determination is passed, otherwise it fails. If multiple fixtures or a fixture array composed of multiple fixtures are used for grasping, obstacle determination can be performed on each fixture separately. One of the key points of the present invention is to traverse all search states and perform obstacle determination, so the obstacle determination method is not limited, and any obstacle determination method can be used in the present invention. The obstacle situation of the object to be grasped can be determined by only one obstacle determination method, or multiple obstacle determination methods can be combined for determination. If a combination of multiple determination methods is adopted, the multiple determination methods can be performed in a certain order or in parallel. When any of the multiple determination methods fails, it is considered that the obstacle determination of the search state fails.

[0066] In an optional implementation, the present invention exemplarily provides four obstacle determination methods, namely, boundary obstacle avoidance, fixed obstacle avoidance, convex / concave obstacle avoidance, and custom obstacle avoidance.

[0067] Boundary obstacle determination

[0068] Some clamps cannot grab items at the edge of the item. When using these clamps, the boundary of the item itself will constitute an obstacle to grabbing. Therefore, it is necessary to identify the boundary of the item to avoid grabbing the item to be grabbed at the boundary. In order to identify the boundary, the point cloud data of the group of items to be grabbed can be obtained. Specifically, the method of step 100 can be used to obtain the point cloud data, and the part of the point cloud data at the edge can be intercepted to obtain the contour point cloud of the group of items to be grabbed. Since the contour point cloud of the group of items to be grabbed is three-dimensional point cloud data, and the three-dimensional point cloud data will affect the determination of the edge contour of the item due to various external or internal factors. Therefore, in order to more accurately determine the contour of the edge of the item, the contour point cloud of the edge of the item can be projected, and the above-mentioned contour point cloud can be mapped to a two-dimensional plane to obtain the contour points of the edge of the group of items to be grabbed. Since the contour points of the edge of the group of items to be grabbed are two-dimensional data at this time, the contour points of the edge of the group of items to be grabbed can be more clearly determined based on the two-dimensional data.

[0069] In order to avoid the incomplete point cloud, the contour points at the four corners of the object to be grasped can be obtained from the obtained two-dimensional pattern of the group of objects to be grasped. According to the contour points at the four corners of the glass to be coated with glue, the minimum circumscribed rectangle of the glass to be coated with glue is obtained, and the minimum circumscribed rectangle is regarded as the contour quadrilateral of the group of objects to be grasped. Alternatively, the contour points at the four corners are connected clockwise from left to right and from top to bottom to form the edge line of the object to be grasped, and the quadrilateral surrounded by the edge line is regarded as the contour quadrilateral of the group of objects to be grasped.

[0070] For each search state, determine the relationship between the center of the clamp and the edge contour point of the object in the search state. If the center of the clamp is outside the obtained contour, it means that the clamp is located at the boundary of the object to be grasped. At this time, it is determined that there is a boundary obstacle and the obstacle determination fails; if the center of the clamp is within the contour, the obstacle determination passes.

[0071] Fixed obstacle determination

[0072] The object to be grasped may be placed together with some fixed obstacles to form a group of objects to be grasped. For example, when the object to be grasped is a steel plate or glass, multiple steel plates or glass may be stacked and placed, and each layer of steel plates or glass may be separated by rubber pads or foam plastics. When the upper steel plate or glass is grasped, the rubber pad or foam plastic will be left on the lower steel plate or glass, thus becoming an obstacle during grasping. In such cases, it is necessary to perform obstacle determination on these fixed obstacles that are fixed but have uncertain positions and affect the gripper's grasping of the object to be grasped.

[0073] In order to identify fixed obstacles, three-dimensional point cloud data of the group of objects to be grasped can be obtained and mapped onto a two-dimensional plane, wherein the method of step 100 can be used to obtain and map the point cloud data. Since fixed obstacles usually have uniform specifications, such as uniform size or shape, the specification parameters of the fixed obstacles can be preset. For each search state, it can be determined based on the two-dimensional point cloud data whether there are objects under the fixture that meet the preset specification parameters in the search state. If there are, it is determined that there are fixed obstacles and the obstacle determination fails; otherwise, the obstacle determination passes.

[0074] Determination of raised / recessed obstacles

[0075] Depending on the object to be grasped, the object itself may have convex or concave parts. These convexities / concave parts may be inherent in the object or caused by collisions. Some clamps may not be able to grasp objects correctly at convex or concave parts. For example, if a suction cup clamp is sucked on a convex edge, the suction cup cannot be completely attached to the object, resulting in air leakage, and then the object cannot be grasped at this position. Therefore, in some cases, it is necessary to treat the convexity / concave part as an obstacle for obstacle determination.

[0076] During the research and development process, the applicant discovered that protrusions / depressions, as part of the object to be grasped, are difficult to distinguish through contour point cloud data. Moreover, unlike fixed obstacles, protrusions / depressions usually do not have fixed specifications, and conventional methods cannot identify the protrusions / depressions of the object.

[0077] In order to solve this problem, the present invention proposes a method for determining the non-planar structure of an object surface, which is also one of the key points of the present invention. The method can be used in certain embodiments of the present invention to determine raised / recessed obstacles, and can also be used in other situations where it is necessary to determine whether the surface of an object has a non-planar structure, and is not limited to use in obstacle avoidance and grasping solutions. Figure 2 The schematic diagram of the flow chart of the method for determining the non-planar structure on the surface of an object according to the present invention is shown, and the non-planar structure includes convex, concave and other structures. Figure 2 As shown, the method includes:

[0078] Step 200: Obtain two-dimensional plane depth map information of the area to be determined on the surface of the object.

[0079] After the three-dimensional point cloud data of the object is orthographically projected onto a two-dimensional plane, a depth map corresponding to the orthographic projection is generated as the two-dimensional plane depth map information of the area to be determined. The depth map can be obtained in a manner similar to step 100. It is also possible to obtain a depth map of a partial area of ​​the object or the area to be determined instead of the entire depth map of the object.

[0080] Step 210: Obtain the depth value of each pixel in the area to be determined according to the two-dimensional plane depth map information.

[0081] Each pixel in the depth map corresponds to each pixel in the two-dimensional image of the orthographic projection, and the value of each pixel is the depth value of the pixel. In order to determine whether there is a non-planar structure such as a convex or concave in the area to be determined, the depth value of each pixel in the area to be determined can be obtained in this step.

[0082] Step 220: group the obtained depth values.

[0083] The depth values ​​of each pixel of a planar structure are the same or have a small difference, while the depth values ​​of each pixel of a non-planar structure such as a convex or concave structure may have a large difference. In this step, the depth values ​​of all the pixels obtained are grouped. If there are multiple identical depth values, they can be merged into one depth value and then grouped.

[0084] When grouping, all the obtained depth values ​​can be grouped into groups of two or more depth values. The grouping method can be random grouping or other arbitrary grouping methods, and the present invention is not limited to this. As a preferred embodiment, all the depth values ​​can be sorted first, and then the highest value and the lowest value are grouped into one group, the second highest value and the second lowest value are grouped into one group, and so on, all the depth values ​​are grouped.

[0085] Step 230, calculating the difference of the depth values ​​in each group and comparing it with a preset difference threshold; and / or calculating the number of groups and comparing it with a preset group number threshold.

[0086] The difference in depth values ​​can reflect the distance between pixels. The greater the difference in depth values ​​between two pixels, the greater the height difference between the two points. Therefore, the difference in depth values ​​can reflect the degree of unevenness of the area to be determined. In addition, the number of groups can reflect the number of pixels with different heights. The greater the number of groups in the area to be determined, the more pixels with ups and downs there are. Therefore, the number of groups can reflect the degree of unevenness in the area to be determined from another perspective. The difference in depth values ​​or the number of groups can be used alone to determine whether an object has a non-planar structure. As a preferred embodiment, the difference in depth values ​​and the number of groups can also be used to jointly determine whether an object has a non-planar structure. The applicant has found that this method can greatly increase the accuracy of non-planar structure determination compared to using it alone. In another embodiment, the difference threshold and / or the number of groups threshold can be set according to the needs of the actual situation. For example, when the method is applied to grasping and avoiding obstacles, for some fixtures, smaller protrusions do not affect grasping, so the threshold can be set to a relatively large value, so that smaller protrusions will not be determined as obstacles that cannot be grasped.

[0087] Step 240: determine the non-planar structure of the region according to the comparison result.

[0088] The non-planar structure situation may include whether the non-planar structure exists. When the difference in depth values ​​and / or the number of groups exceeds a threshold, the presence of the non-planar structure is determined. It may also include the degree of undulation of the non-planar structure, and multiple levels may be set for the degree of undulation. In other embodiments, a multi-level threshold may be set to determine whether the non-planar structure exists or the level of undulation by combining different thresholds. As a preferred embodiment, when the difference in depth values ​​and the number of groups are used to jointly determine the non-planar structure situation, it may be set that when the difference in depth values ​​exceeds a difference threshold and the number of groups exceeds a grouping number threshold, the presence of the non-planar structure is determined.

[0089] Custom obstacle determination

[0090] In industrial scenarios, there are some obstacles that cannot be identified by robot vision technology. For example, when the object to be grasped is glass, the glass may be coated with glue. The glue is usually transparent and has unclear features, so it cannot be correctly identified, and the fixture cannot grasp the glass at the glue coating position. In order to identify such obstacles, the size, shape, and position of the obstacle can be customized before grasping. In this way, for each search state, it is determined whether there is an obstacle under the fixture according to the predefined obstacle information. In one embodiment, the edge contour of the obstacle can be generated according to the customized obstacle information, and then for each search state, the relationship between the center of the fixture in the search state and the customized obstacle edge contour is determined. If the center of the fixture is outside the obtained contour, it is determined that there is an obstacle and the obstacle determination fails; if the center of the fixture is within the contour, the obstacle determination passes. In another embodiment, the retraction distance of the edge contour can also be set, that is, the original edge contour of the obstacle is retracted inward by a certain distance, and then the relative position of the fixture center and the edge contour is determined.

[0091] In step S120, the solution may be further expanded or improved as follows:

[0092] For a certain search state, if the fixture passes the obstacle judgment, the fixture is set to be turned on in the search state; if it does not pass, the fixture is set to be turned off in the search state;

[0093] If multiple fixtures or a fixture array consisting of multiple fixtures are used for grasping, when performing obstacle determination, obstacle determination may be performed for each of the multiple fixtures;

[0094] In a certain search state, if at least one fixture passes the obstacle determination, the search state can be set as an alternative search state;

[0095] When performing obstacle determination, it is also possible to determine that the obstacle determination is passed when an obstacle exists but does not affect the grasping, and to determine that it is not passed when the obstacle exists and affects the grasping.

[0096] Step S130: Select the best search state from the candidate search states.

[0097] Depending on the actual situation, such as the different objects to be grasped and the different clamps used, the method of selecting the best search state is also different. The present invention is not limited to this, and any selection method can be used in the scheme of the present invention. As a preferred embodiment, the best search state can be selected according to the distance between the clamp and the center of the object to be grasped in each search state. Generally speaking, the closer the gripping position of the clamp is to the center of the object, the more stable the grip, so the search state in which the clamp is closest to the center of the object can be set as the best search state. In order to calculate the distance between the clamp and the center of the object, the center position P1 of the clamp in a certain search state can be determined first, and then the circumscribed rectangle of the object can be obtained, and then the center position P2 of the object can be obtained based on the circumscribed rectangle of the object. The distance between P1 and P2 is the distance between the clamp and the center of the object.

[0098] If multiple clamps or a clamp array consisting of multiple clamps are used for grasping, the best search state can also be selected according to the number of clamps that have passed the obstacle determination, or the best search state can be selected by comprehensively considering the distance between the clamp and the center of the object and the number of clamps that have passed the obstacle determination. In a preferred embodiment, the selection can be based on the number first. If there are multiple search states with the largest number of passes, then from the multiple search states that have passed, the search state in which the clamp is closest to the center of the object is further selected as the best search state; in other embodiments, a quantity threshold can also be preset, and the search state in which the number of clamps opened exceeds the threshold is first selected, and then the search state closest to the center is selected from these search states as the best search state.

[0099] Step S140: grab the object to be grabbed based on the optimal search state.

[0100] The robot sets the position, angle and quantity of the fixture according to the configuration parameters of the fixture in the best search state, and then grabs the object and places it in the specified position. The specified position includes the ground, the object placement rack, etc. Some objects have position and state requirements when they are placed, such as vertical placement, and cannot be too high or too low. Therefore, the distance from the center of the fixture to the specified edge of the object in the final planning result can also be calculated, so that the object can be accurately placed in the specified position.

[0101] The inventors have discovered that when using multiple clamps or a clamp array composed of multiple clamps for grasping, if the number of clamps used in the optimal search state is insufficient or the arrangement of the multiple clamps is inappropriate, for example, multiple clamps are arranged in a straight line, it will cause the center of gravity to be unstable during grasping, so that the object may swing, fall, or collide with other unexpected objects and be damaged during the grasping process. To this end, the present invention also proposes a method for clamp grasping verification, which can determine whether the grasping method to be used can grasp the object steadily before grasping the object, thereby avoiding unnecessary losses during grasping. This method can be used in the obstacle avoidance grasping method of the present invention, and can also be used in other grasping scenarios. The present invention does not limit the specific usage scenario. As long as multiple clamps or a clamp array composed of multiple clamps are used in a grasping scenario, this method can be applied for verification. For the convenience of explanation, multiple clamps or a clamp array composed of multiple clamps are collectively referred to as a clamp group in the present invention.

[0102] Figure 3 FIG. 2 is a flow chart showing a method for checking a fixture assembly according to an embodiment of the present invention. Figure 3 As shown, the method includes:

[0103] Step S300: Obtaining status information of the fixture group.

[0104] The state of a fixture group refers to the combination of specific parameters of each fixture in the fixture group. The fixture group performs grabbing in a certain state means that the fixture is configured based on the parameters in the state and performs the grabbing action. For example, suppose a fixture has 3 fixtures, No. 1-3, No. 1 is at the left edge of the object to be grabbed, and the rotation angle is 30 degrees, No. 2 is at the right edge of the object, and the rotation angle is 0 degrees, No. 1 and No. 2 are both set to open, and No. 3 is outside the object and set to not open. Such a combination of specific parameters is a state of the fixture group. The state information may include the position information of the fixture group in this state, the position information of each fixture in the fixture group, the angle information, and whether these fixtures can be opened in the current state, which fixtures can be opened, and other information.

[0105] Step S310: Determine the quantity information of the openable clamps and / or the position information of the openable clamps based on the status information of the clamp group.

[0106] When determining the position information of the openable clamp, the position of the clamp relative to the object to be grasped can be determined based on the contour information of the object to be grasped and the clamp boundary parameters; the position of the clamp relative to other clamps or the position of the clamp relative to the clamp array can also be determined as the position of the clamp. In one embodiment, multiple clamps or clamp arrays can be divided into multiple areas, and the position information of the clamp can be represented by the area where the clamp is located.

[0107] Step S320: determining whether the number of the openable clamps meets a preset condition; and / or determining whether the position information of the openable clamps meets a preset condition.

[0108] Insufficient number of openable clamps or poor position of openable clamps may lead to unstable grasping. If the number of openable clamps cannot be adjusted or does not need to be adjusted, or the position of the clamp cannot be adjusted or does not need to be adjusted, only one of them can be judged. Those skilled in the art will understand that judging two conditions at the same time can significantly increase the probability of stable grasping of the clamp compared to judging only one condition. The conditions that need to be met can be pre-set according to the information of the object to be grasped, such as the weight of the object to be grasped, such as setting the conditions that the number of openable clamps should meet. For example, when grasping heavier objects, the number of clamps can be set to 5, so that the preset conditions are met only when the number of openable clamps exceeds 5; and for lighter objects, it can be set to 3. When the density of the objects to be grasped is uniform, the number of clamps to be used can also be set according to the area size of the objects. For example, for objects with larger areas, the preset conditions are 5 clamps, and for objects with smaller areas, it is 3. The area of ​​the object can be determined according to the contour information of the object. For the conditions that need to be met by the position information of the openable clamps, it is usually necessary to at least avoid multiple clamps being located in a straight line or approximately in a straight line. According to the needs of the actual situation, more specific position information can also be set. For example, it can be set that the connecting line of the positions of multiple openable clamps must form a stable triangle; the position information that needs to be met by the openable clamps can also be set according to the center of gravity of the object to be grasped to ensure a stable center of gravity during grasping. For example, when the center mass of the object to be grasped is large, the preset condition can be that there must be an openable clamp near the center of the object to be grasped or there must be a specific number of openable clamps.

[0109] Step S330: determining whether the clamp assembly can perform grasping in this state according to the judgment result.

[0110] In specific implementation, the grabbing can be performed only when the number of openable clamps meets the condition or the position information of the openable clamps meets the condition. The grabbing can also be performed when both conditions are met at the same time. When the position information conditions include multiple conditions, the grabbing can be performed only when multiple conditions are met.

[0111] In one embodiment, after determining to perform the grabbing of the item in a certain state and before performing the grabbing, the fixture verification can be performed. If used in conjunction with the obstacle avoidance grabbing solution, steps S300-S330 can be performed between steps S130 and S140 of the aforementioned embodiment, that is, after selecting the best search state, the best search state is used as the state of the fixture array to perform fixture verification to determine whether the best search state can be used for grabbing. In another embodiment, the fixture verification method can also be combined to determine to perform the grabbing of the item in a certain state. If used in conjunction with the obstacle avoidance grabbing solution, steps S300-S330 can be performed in steps S120 or S130 of the aforementioned embodiment, for example, an alternative search state can be selected by fixture verification or after selecting an alternative search state, the alternative search state is first used as the state of the fixture array, and the search state that cannot be grabbed is eliminated by the fixture verification method, and then the best search state is selected according to the number of fixtures and the distance from the center.

[0112] The obstacle avoidance grasping method and the clamp grasping verification method of the present invention are not limited to specific clamps and application scenarios. In order to be able to apply the above method in an industrial scenario of grasping glass using a suction cup array, the inventor has made arduous efforts to further refine the method to adapt to the scenario, which is also one of the key points of the present invention.

[0113] like Figure 4 A flow chart of a glass obstacle avoidance grasping method using a suction cup array according to a preferred embodiment of the present invention is shown, the method comprising:

[0114] Step S400: Obtaining point cloud information of the glass to be grasped and obstacles.

[0115] In this embodiment, the glass can be stacked in layers, and rubber pads are provided between each layer of glass to separate the glass. A method similar to step S100 can be used to obtain the rubber pad point cloud and the object point cloud, and the point cloud is orthographically projected onto a 2D image and a corresponding depth map is generated after orthographic projection. Figure 5 The point cloud image obtained in this way is shown, in which the black part is the transparent glass and the white part is the point cloud of the non-transparent part.

[0116] Step S410: Generate a search state based on the point cloud information and the search boundary factor parameters of the suction cup array.

[0117] The search state of the suction cup includes the different position states of the suction cup on the object and the rotation state of the suction cup itself. The specified search boundary factor parameters can include X direction, Y direction and rotation, and can also include the distance from the suction cup boundary to the specified edge. A variety of different search states are generated according to different search boundary parameters. Figure 5 As shown, the suction cup array used includes 10 suction cups, numbered 1-10 respectively. Figure 5 The positions of the 10 suction cups in a certain search state are shown. In other search states, the suction cups may be in other positions or have different rotation angles. As an example, the moving distance boundary (i.e., controllable boundary) in the X direction can be set to -500mm to 500mm, and the control granularity is 100mm. Then the fixture can move from -500mm to 500mm in the X direction in units of 100mm, so there are 11 search states in the X direction; the moving distance boundary in the Y direction can be set to -100mm to 100mm, and the control granularity is 50mm. Then the fixture can move from -100mm to 100mm in the Y direction in units of 50mm, so there are 5 search states in the Y direction; the rotation angle can be set to -50 degrees to +50 degrees, and the control granularity is 10 degrees. Then the fixture can rotate from -50 degrees to +50 degrees in units of 10 degrees, so there are 11 search states in the rotation angle. Under the above search boundary factor parameter settings, there are a total of 11×5×11=605 search states. It can also be set that the moving distance of the fixture cannot exceed 100mm of the upper boundary, so the fixture can only move to 100mm from the upper edge in the Y direction.

[0118] Step S420: for each search state, determine whether there is an obstacle under each suction cup, and if so, close the suction cup in the search state.

[0119] like Figure 6 As shown, judging whether there is an obstacle under the suction cup may include the following steps:

[0120] Step S421: Determine whether there is a glass boundary obstacle under the suction cup.

[0121] In the current search state, for each suction cup, check whether the center of this single suction cup is inside the object outline. You can determine whether the center of a single suction cup is inside the object outline by judging the point cloud ratio in the area below the suction cup. The point cloud ratio refers to the ratio of the point cloud area to the overall area in the area below the suction cup. When the suction cup is at the glass boundary, the point cloud will increase significantly. Figure 5 The suction cups 4-6 in the figure are located at the boundary of the glass. The white point cloud area below these suction cups accounts for a significantly larger proportion. Therefore, a point cloud proportion threshold can be preset. When the point cloud proportion in the area below the suction cup exceeds the threshold, it is determined that there is a glass boundary obstacle below the suction cup. The proportion threshold can be set according to the needs of the actual situation, such as the suction force of the suction cup and the situation of obstacles. The present invention does not limit the specific value. The inventor found that selecting a proportion threshold within 5%-40% can improve the accuracy of boundary obstacle judgment, among which 10% is considered to be the best.

[0122] Step S422: Determine whether there is a rubber pad obstacle under the suction cup.

[0123] In this embodiment, the glass is stacked layer by layer, and the layers are separated by rubber pads. Except for the top layer of glass, each glass in the lower layer will have a rubber pad for separating the glass layers. In this way, after grabbing the upper layer of glass, there will be remaining rubber pads on the lower layer of glass. Since the size of the rubber block is fixed and known, the rubber blocks can be filtered out according to the size. In one embodiment, the obstacle point cloud area can be pre-set. When the point cloud area in the area below the suction cup exceeds the obstacle point cloud area, it is determined that there is a rubber pad obstacle under the suction cup. The obstacle point cloud area can be set arbitrarily according to the size of the rubber pad used.

[0124] Step S423: Determine whether there is a rubber strip obstacle under the suction cup.

[0125] In industrial scenarios, glue may be applied near the border of the glass. The glue strip formed after application is usually transparent. The point cloud features of such glue strips are not obvious and difficult to identify, but the suction cup cannot grab the glass at the glue strips. In order to correctly identify the glue strip obstacle, the user can set the position of the glue strip by himself. The robot determines whether there is a glue strip obstacle under the suction cup array based on the glue strip obstacle information set by the user. Specifically, a circle of glue strips can be generated near the edge contour of the glass according to the user's settings. The user can set the position of the glue strip, the retraction distance and the width of the glue strip, or generate two glue strips on the same side. Figure 7 Four different gluing processes are shown. For process 1, there is no need to apply a strip, nor is there any need to preset the strip information. For process 2, there are two inner and outer tracks. The strip can be applied to the uncoated area between the inner and outer tracks, or the inner track can be not applied, and the inner track segment of the specified edge can be generated separately. For process 3, the opening segment, that is, the trackless segment on the right side of the figure, is applied with a strip. For process 4, the opening segment outside the inner track, that is, the segment in a similar position to the opening segment of process 3 in the figure, is applied with a strip. For the above processes 2-4, the strip obstacle information is set in advance at the position where the strip is required. In this way, after applying the strip, the robot can determine whether there is a strip obstacle in the area below the suction cup based on the strip information set by the user.

[0126] Step S424: Determine whether there is a protruding obstacle under the suction cup.

[0127] There may be protrusions on the glass surface. If the suction cup is sucked on the protruding edge, it cannot be completely attached to the glass surface, which will cause the suction cup to leak and the suction failure. The depth difference between each pixel point in the point cloud of the area below the suction cup and the number of pixels can be used to determine whether there is a protruding obstacle in the area. In one embodiment, the depth values ​​of all point clouds in the area below the suction cup can be obtained, all depth values ​​are sorted, and then the highest depth value is paired with the lowest depth value, the second highest with the second lowest, the third highest with the third lowest...all depth values ​​are paired in this way. The difference between the paired depth values ​​is calculated, the depth difference is compared with a preset depth difference threshold, and the number of pairs is compared with a preset logarithmic threshold. If the height difference exceeds the threshold and the number of pairs exceeds the threshold, it is determined that there is a protruding obstacle in the area below the suction cup. The depth difference threshold and the logarithmic threshold can be set according to the actual needs, such as the suction force of the suction cup and the obstacle situation. The present invention does not limit the specific value. The inventors found that selecting the logarithmic threshold and the depth difference threshold within the following range can improve the accuracy of protrusion obstacle judgment: the logarithmic threshold can be selected within 10 pairs-50 pairs, with 20 pairs being the best; the depth difference threshold can be selected within 0.500mm-0.005mm, with 0.015mm being the best.

[0128] As a preferred embodiment, the obstacle judgment can be performed strictly in the order of steps S421-S424. After the previous step judges that there is an obstacle and closes the suction cup, the subsequent steps will no longer be executed. For example, if it is judged in step S421 that there is a glass boundary obstacle under the suction cup, the suction cup will be closed and steps S422-S424 will no longer be executed. Since the above four steps are relatively independent, the obstacle situation of the glass can also be judged only by the obstacle judgment method of any step S421-S424, or any multiple steps can be combined for judgment, for example, using the combination of steps S421 and S422, but not using steps S423 and S424. If a combination of multiple steps is used, the multiple steps can be performed in a certain order or in parallel. It should be noted that although a single step or various combinations can achieve obstacle judgment, if the obstacle judgment method of the optimal embodiment of the present invention is used to sequentially execute steps S421-S424, compared with other judgment methods, it can not only improve the accuracy of obstacle judgment, but also improve the efficiency of obstacle judgment because it first judges the obstacles with a greater possibility of occurrence and uses a simpler algorithm first, thereby improving the efficiency of robot grasping, and thus has special advantages in industrial scenarios.

[0129] Step S430, selecting an alternative search state according to the suction cup opening status of each search state.

[0130] The selection conditions can be set by yourself, such as the number of open, whether a specific suction cup is open, etc. In one embodiment, if there is at least one open suction cup, the search state is selected as the candidate search state.

[0131] Step S440: Select the best search state from the candidate search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass.

[0132] The best search state can be selected according to the distance of each search state from the center of the glass to be grasped. Generally speaking, the closer to the center of the object, the more stable the grasping. Therefore, the search state closest to the center of the object can be set as the best search state. For each search state, determine the center position P1 of the clamp in the search state, and then find the center position P2 of the object through the circumscribed rectangle of the glass. The distance between P1 and P2 is the distance between the clamp and the center of the glass. It is also possible to select the search state with the largest number of passes as the best search state according to the number of suction cups determined by passing through obstacles. In a preferred embodiment, the best search state can be selected in combination with the number of suction cups opened and the distance between the suction cups and the center of the glass. For example, it can be selected based on the number. If there are multiple search states with the largest number of passes, then from the selected multiple search states, the search state closest to the center of the object is further selected as the best search state; in another embodiment, a quantity threshold can also be preset, first select all search states with the number of suction cups opened exceeding the threshold, and then select the search state closest to the center from these search states as the best search state.

[0133] Step S450, using the best search state to grab the glass.

[0134] In industrial scenarios, it is usually necessary to place the grasped glass on the ground or on an item placement rack. If the glass is placed on a shelf, it cannot exceed the edge of the shelf too much. Therefore, the distance from the center of the suction cup array to the specified edge of the glass in the final planning result can also be calculated. In this way, the robot can control the placement of the glass more accurately during placement, thereby placing the glass accurately in the specified position.

[0135] like Figure 8 FIG. 2 shows a flow chart of a glass grabbing and checking method using a suction cup array according to a preferred embodiment of the present invention. Figure 8 As shown, the method includes:

[0136] Step S500: grouping all suction cups in the array according to the shape of the suction cup array.

[0137] The suction cups can be grouped according to their distribution on the suction cup array. Figure 5In the illustrated embodiment, the suction cup array includes 12 suction cups, which are numbered 1-12. The suction cups can be divided into 4 groups, namely upper left, upper right, lower left, and lower right, according to their relative positions to the suction cup array. Specifically, suction cups 3 and 9 are the first group, suction cups 4 and 8 are the second group, suction cups 1, 2, and 10 are the third group, and suction cups 5, 6, and 7 are the fourth group.

[0138] Step S510: Obtaining status information of the suction cup array.

[0139] The state of the suction cup array refers to the combination of specific parameters of each suction cup in the suction cup array. The suction cup array performs grasping in a certain state means that the suction cup array is configured based on the parameters in the state and performs the grasping action. For example, suppose a suction cup array has 3 suction cups, No. 1-3, suction cup No. 1 is at the left edge of the object to be grasped, and the rotation angle is 30 degrees, suction cup No. 2 is at the right edge of the object, and the rotation angle is 0 degrees, suction cups No. 1 and No. 2 are both set to open, and suction cup No. 3 is outside the object and is set to not open. Such a combination of specific parameters is a state of the suction cup array. The state information of the suction cup array includes the position of the suction cup array, the position and angle of each suction cup in the suction cup array, and whether these suction cups can be opened in the current state, which suction cups can be opened, and other information.

[0140] Step S520: determining the number of openable suction cups and the grouping information of the openable suction cups based on the state information of the suction cup array.

[0141] In this step, for each suction cup state, the number of openable suction cups in this state and the group to which the openable suction cups belong are determined.

[0142] Step S530: Determine whether the number of openable suction cups meets a preset condition and / or whether the grouping information of the openable suction cups meets a preset condition.

[0143] An insufficient number of open suction cups, or a poor position of the open suction cups, may lead to unstable grasping. For the number of open suction cups, a single threshold can be set, for example, grasping is allowed only when more than 5 suction cups are present. The suction cup number threshold can also be preset according to the size of the glass to be grasped. For example, a minimum number threshold can be set to 3 suction cups, that is, no matter how large the glass area is, at least 3 suction cups can be opened; 4 suction cups can also be set for glass areas of 1 square meter to 2 square meters, and 5 suction cups for glass above 2 square meters. In this way, the area of ​​the glass and the number of suction cups opened are judged to determine whether grasping can be performed. In this way, when judging the number of suction cups that can be opened, the area of ​​the glass must also be judged. For example, it can be judged that the glass area is 2 square meters and the number of suction cups that can be opened is 4. When the corresponding relationship between the area and the number is met, it is considered that the quantity condition is met.

[0144] For the grouping information of the opened suction cups, it is possible to determine only which groups have opened suction cups. For example, in the above embodiment where the suction cups are divided into 4 groups, it is possible to determine which groups have opened suction cups in the first to fourth groups, respectively, based on the suction cup opening conditions. Suction cup distribution conditions that need to be met can be preset, for example, it can be set that when there are suction cups that can be opened in at least three groups (in this case, the suction cups can be arranged in a stable triangle or quadrilateral), the suction cup distribution conditions are met.

[0145] Step S540: determining whether the suction cup array can perform grasping in this state according to the judgment result.

[0146] In a specific implementation, the grabbing may be performed only when the number of openable suction cups meets a condition or the distribution of openable suction cups meets a condition. The grabbing may also be performed only when both conditions are met at the same time. When the preset conditions for suction cup grouping include multiple conditions, the grabbing may be performed only when multiple conditions are met.

[0147] Step S500 can be executed at any time before S510, as long as it is ensured that when steps S510-S540 are executed sequentially, there is confirmed grouping information. In one embodiment, a suction cup check can be performed after determining that the glass grabbing is to be performed in a certain state and before the grabbing is performed. If used in conjunction with an obstacle avoidance grabbing solution, steps S510-S540 can be performed between steps S430 and S440 of the aforementioned embodiment, that is, after selecting the best search state, the best search state is used as the state of the suction cup array, and a suction cup check is performed to determine whether the best search state can be used for grabbing. In another embodiment, the suction cup check method can also be combined to determine whether the grabbing of the object is to be performed in a certain state. If used in combination with an obstacle avoidance and grasping solution, steps S510-S540 may be executed in step S420 or step S430 of the aforementioned embodiment. For example, an alternative search state may be selected through a suction cup verification method or, after an alternative search state is selected, the alternative search state is first used as the state of the suction cup array, and search states that cannot be grasped are eliminated according to the suction cup verification method, and then the best search state is selected according to the number of suction cups and the distance from the center.

[0148] In addition, it should be noted that, although the present invention describes the general robot grasping method and the grasping method dedicated to glass grasping in multiple embodiments, and the technical details of the multiple embodiments are not the same, those skilled in the art can understand that the technical details not described in the dedicated method but described in the general method can actually be used in the dedicated method, and vice versa. In other words, although each embodiment of the present invention has a specific combination of features, further combinations and cross-combinations of these features between the embodiments are also feasible.

[0149] According to the above embodiments, firstly, the present invention can first obtain the possible grasping methods of the clamp, select the best grasping method that will not grasp obstacles, and perform the grasping of the object, so that even if there are obstacles on the object that cannot be grasped, the object can be accurately grasped by avoiding obstacles; secondly, for industrial scenarios that may need to determine whether there are non-planar structures on the surface of the object, the present invention proposes a solution that can identify the non-planar area on the surface of the object; thirdly, the present invention can pre-verify that the grasping method used can correctly grasp the object before the clamp group performs grasping, thereby improving the grasping stability and avoiding problems such as unstable center of gravity that may occur during the grasping process; fourthly, based on the general obstacle avoidance grasping method and clamp verification method, the present invention has developed an obstacle avoidance grasping method and a suction cup verification method specifically for the industrial scenario of using a suction cup array for glass grasping, which can improve the accuracy and stability of using a suction cup array for glass grasping. It can be seen that the present invention solves all aspects of the problems that arise in the industrial scenario of using a clamp to grasp objects.

[0150] Fig. 9 A clamp control device according to another embodiment of the present invention is shown, the device comprising:

[0151] The point cloud acquisition module 600 is used to acquire point cloud information of the group of objects to be grasped, that is, to implement step S100;

[0152] A search state generating module 610, used to generate a search state based on the point cloud information of the group of objects to be grasped and the search boundary factor parameters of the fixture, that is, to implement step S110;

[0153] The obstacle determination module 620 is used to perform obstacle determination for each search state and set the search state that passes the obstacle determination as the candidate search state, that is, to implement step S120;

[0154] The best search state determination module 630 is used to select the best search state from the candidate search states, that is, to implement step S130;

[0155] The grabbing module 640 is used to grab the object to be grabbed based on the optimal search state, that is, to implement step S140.

[0156] Fig.10 A non-planar structure determination device based on a robot according to another embodiment of the present invention is shown, the device comprising:

[0157] A depth map acquisition module 700 is used to acquire a two-dimensional plane depth map of the area to be determined on the surface of the object, that is, to implement step S200;

[0158] A depth value acquisition module 710 is used to acquire the depth value of each pixel in the area to be determined according to the two-dimensional plane depth map, that is, to implement step S210;

[0159] A grouping module 720, used for grouping the obtained depth values, i.e., for implementing step S220;

[0160] The comparison module 730 is used to calculate the difference of the depth values ​​in each group and compare it with a preset difference threshold; and / or calculate the number of groups and compare it with a preset group number threshold, that is, to implement step S230;

[0161] The determination module 740 is used to determine the non-planar structure of the region according to the comparison result, that is, to implement step S240.

[0162] Fig.11 A schematic diagram of the structure of a fixture group calibration device according to another embodiment of the present invention is shown, the device comprising:

[0163] A status information acquisition module 800 is used to acquire status information of the fixture array, that is, to execute step S300;

[0164] An information determination module 810, used to determine the quantity information of the openable clamps and / or the position information of the openable clamps based on the state information, that is, used to execute step S310;

[0165] The condition determination module 820 is used to determine whether the number of the openable clamps meets the preset condition and / or determine whether the position information of the openable clamps meets the preset condition, that is, to execute step S320;

[0166] The grasping determination module 830 is used to determine whether the clamp group can perform grasping in this state according to the judgment result, that is, to execute step S330.

[0167] Fig.12 A schematic structural diagram of a suction cup array control device according to another embodiment of the present invention is shown, the device comprising:

[0168] Point cloud acquisition module 900, used to acquire point cloud information of the glass to be grasped and the obstacle, that is, used to execute step S400;

[0169] A search state generating module 910, used to generate a search state based on the point cloud information and the search boundary factor parameters of the suction cup array, that is, to execute step S410;

[0170] The obstacle determination module 920 is used to determine whether there is an obstacle under each suction cup for each search state, and if there is an obstacle, close the suction cup in the search state, that is, to execute step S420;

[0171] The alternative search state selection module 930 is used to select an alternative search state according to the suction cup opening status of each search state, that is, to execute step S430;

[0172] The best search state selection module 940 is used to select the best search state from the candidate search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass, that is, to execute step S440;

[0173] The grabbing module 950 is used to grab the glass based on the optimal search state, that is, to execute step S450.

[0174] Fig.13 A schematic diagram of the structure of a suction cup array verification device according to another embodiment of the present invention is shown, the device comprising:

[0175] A grouping module 1000, used to group all the suction cups in the array according to the shape of the suction cup array, that is, to execute step S500;

[0176] The status information acquisition module 1010 is used to acquire the status information of the suction cup array, that is, to execute step S510;

[0177] An information determination module 1020, used to determine the number of openable suction cups and group information of the openable suction cups based on the state information of the suction cup array, i.e., used to execute step S520;

[0178] The condition determination module 1030 is used to determine whether the number of the openable suction cups meets a preset condition and / or whether the grouping information of the openable suction cups meets a preset condition, that is, to execute step S530;

[0179] The grasping determination module 1040 is used to determine whether the suction cup array can perform grasping in this state according to the judgment result, that is, to execute step S540.

[0180] Above Figure 9-13 In the device embodiment shown, only the main functions of the modules are described, and all the functions of each module correspond to the corresponding steps in the method embodiment. The working principle of each module can also refer to the description of the corresponding steps in the method embodiment, and will not be repeated here. In addition, although the correspondence between the functions of the functional modules and the methods is defined in the above embodiments, those skilled in the art will understand that the functions of the functional modules are not limited to the above correspondence, that is, a specific functional module can also implement other method steps or part of the method steps. For example, the above embodiments describe the method in which the grasping determination module 1040 is used to implement step S540, but according to the needs of the actual situation, the grasping determination module 1040 can also be used to implement the method or part of the method of steps S500, S510, S520 or S530.

[0181] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method of any of the above-mentioned embodiments is implemented. It should be pointed out that the computer program stored in the computer-readable storage medium of the embodiment of the present application can be executed by the processor of the electronic device. In addition, the computer-readable storage medium can be a storage medium built into the electronic device, or a storage medium that can be plugged into the electronic device. Therefore, the computer-readable storage medium of the embodiment of the present application has high flexibility and reliability.

[0182] Fig.14 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0183] like Fig.14 As shown, the electronic device may include: a processor (processor) 1102 , a communication interface (Communications Interface) 1104 , a memory (memory) 1106 , and a communication bus 1108 .

[0184] in:

[0185] The processor 1102 , the communication interface 1104 , and the memory 1106 communicate with each other via a communication bus 1108 .

[0186] The communication interface 1104 is used to communicate with other devices such as clients or other servers.

[0187] The processor 1102 is used to execute the program 1110, and specifically can execute the relevant steps in the above method embodiment.

[0188] Specifically, the program 1110 may include program codes, which include computer operation instructions.

[0189] The processor 1102 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0190] The memory 1106 is used to store the program 1110. The memory 1106 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0191] The program 1110 may be specifically used to enable the processor 1102 to perform various operations in the above method embodiment.

[0192] In summary, the invention content of the present invention includes:

[0193] A clamp control method, comprising:

[0194] Obtain point cloud information of the group of objects to be grasped;

[0195] Generate a search state based on the point cloud information of the group of objects to be grasped and the search boundary factor parameters of the fixture;

[0196] For each search state, an obstacle determination is performed, and the search state that passes the obstacle determination is set as an alternative search state;

[0197] Selecting a best search state from among the alternative search states;

[0198] The object to be grasped is grasped based on the optimal search state.

[0199] Optionally, the group of objects to be grasped includes one or more objects to be grasped and / or obstacles.

[0200] Optionally, use a json file to save the fixture configuration parameters.

[0201] Optionally, the obstacle determination includes at least one of the following: boundary obstacle determination, fixed obstacle determination, raised / recessed obstacle determination and custom obstacle determination.

[0202] Optionally, the boundary obstacle determination includes determining whether a boundary obstacle exists based on a relationship between a center of the clamp and an edge contour point of the object.

[0203] Optionally, the fixed obstacle determination includes determining whether a fixed obstacle exists based on the size or shape of the obstacle.

[0204] Optionally, the convex / concave obstacle determination includes determining whether a convex / concave obstacle exists based on two-dimensional plane depth map information.

[0205] Optionally, the custom obstacle determination includes generating an edge contour of the custom obstacle, and determining whether the custom obstacle exists based on the edge contour.

[0206] Optionally, the method further includes: performing a clamp check on the optimal search state to determine whether the object can be correctly grasped using the search state.

[0207] A clamp control device, comprising:

[0208] A point cloud acquisition module is used to obtain point cloud information of the group of objects to be grasped;

[0209] A search state generation module, used to generate a search state based on the point cloud information of the group of objects to be grasped and the search boundary factor parameters of the fixture;

[0210] An obstacle determination module is used to perform obstacle determination for each search state, and set the search state that passes the obstacle determination as an alternative search state;

[0211] An optimal search state determination module, used for selecting the optimal search state from the candidate search states;

[0212] The grabbing module is used to grab the object to be grabbed based on the optimal search state.

[0213] Optionally, the group of objects to be grasped includes one or more objects to be grasped and / or obstacles.

[0214] Optionally, it also includes: using a json file to save the fixture's configuration parameters.

[0215] Optionally, the obstacle determination module is used to perform at least one of the following obstacle determinations: boundary obstacle determination, fixed obstacle determination, raised / recessed obstacle determination and custom obstacle determination.

[0216] Optionally, when the obstacle determination module performs boundary obstacle determination, it determines whether there is a boundary obstacle based on the relationship between the center of the fixture and the edge contour point of the object.

[0217] Optionally, when performing fixed obstacle determination, the obstacle determination module determines whether a fixed obstacle exists based on the size or shape of the obstacle.

[0218] Optionally, when the obstacle determination module performs the convex / concave obstacle determination, it determines whether there is a convex / concave obstacle according to the two-dimensional plane depth map information.

[0219] Optionally, when executing the custom obstacle determination, the obstacle determination module determines whether a custom obstacle exists according to an edge contour of the custom obstacle.

[0220] Optionally, the method further includes: performing a clamp check on the optimal search state to determine whether the object can be correctly grasped using the search state.

[0221] A robot-based non-planar structure determination method, comprising:

[0222] Obtaining two-dimensional plane depth map information of the area to be determined on the surface of the object;

[0223] Obtain the depth value of each pixel in the area to be determined according to the two-dimensional plane depth map information;

[0224] Group the obtained depth values;

[0225] Calculate the difference of the depth values ​​in each group and compare it with a preset difference threshold; and / or calculate the number of groups and compare it with a preset group number threshold;

[0226] The non-planar structure of the region is determined based on the comparison results.

[0227] Optionally, after obtaining the depth value of each pixel, only one of multiple identical depth values ​​is retained.

[0228] Optionally, the grouping includes grouping two different depth values ​​into one group.

[0229] Optionally, the grouping includes grouping all the depth values ​​in the following manner: sorting all the acquired depth values ​​from high to low, and grouping the first depth value and the penultimate depth value into a group, the second depth value and the penultimate depth value into a group... the Nth depth value and the Nth-to-last depth value into a group, where N is a natural number greater than or equal to 1.

[0230] Optionally, the preset difference threshold and / or grouping number threshold includes a difference threshold and / or grouping number threshold preset based on the capability of the fixture.

[0231] Optionally, the non-planar structure condition includes the presence of a non-planar structure or the absence of a non-planar structure.

[0232] Optionally, the non-planar structure condition includes the degree of undulation of the non-planar structure.

[0233] A robot-based non-planar structure determination device, comprising:

[0234] A depth map acquisition module is used to acquire a two-dimensional plane depth map of the area to be determined on the surface of the object;

[0235] A depth value acquisition module, used to acquire the depth value of each pixel in the area to be determined according to the two-dimensional plane depth map;

[0236] A grouping module, used to group the obtained depth values;

[0237] A comparison module, used to calculate the difference of the depth values ​​in each group and compare it with a preset difference threshold; and / or, calculate the number of groups and compare it with a preset group number threshold;

[0238] The determination module is used to determine the non-planar structure of the region according to the comparison result.

[0239] Optionally, after the depth value acquisition module obtains the depth value of each pixel, only one of multiple identical depth values ​​is retained.

[0240] Optionally, the grouping module groups two different depth values ​​into one group.

[0241] Optionally, the grouping module groups all the depth values ​​in the following manner: sort all the acquired depth values ​​from high to low, and group the first depth value and the penultimate depth value into a group, the second depth value and the penultimate depth value into a group... the Nth depth value and the Nth-to-last depth value into a group, where N is a natural number greater than or equal to 1.

[0242] Optionally, the preset difference threshold and / or grouping number threshold includes a difference threshold and / or grouping number threshold preset based on the capability of the fixture.

[0243] Optionally, the determination module determines whether a non-planar structure exists or does not exist.

[0244] Optionally, the determination module determines the degree of undulation of the non-planar structure.

[0245] A fixture group calibration method, comprising:

[0246] Get the status information of the fixture group;

[0247] Based on the state information, determine the quantity information of the openable clamps and / or the position information of the openable clamps in the state;

[0248] Determining whether the number of the openable clamps meets a preset condition and / or determining whether the position information of the openable clamps meets a preset condition;

[0249] It is determined whether the clamp group can perform grasping in this state according to the judgment result.

[0250] Optionally, determining the position information of the openable clamp includes determining the position information of the openable clamp according to contour information of the object to be grasped.

[0251] Optionally, the conditions that need to be met by the number of openable clamps are preset based on the weight of the object to be grasped.

[0252] Optionally, the conditions that need to be met for the position of the openable clamp are preset based on the center of gravity of the object to be grasped.

[0253] Optionally, the conditions that the positions of the openable clamps need to meet include: the lines connecting the positions of multiple clamps cannot form a straight line.

[0254] Optionally, determining whether the clamp group can perform grasping in this state according to the judgment result includes determining that grasping can be performed when the number of clamps meets a preset condition and the position of the clamps meets a preset condition.

[0255] A fixture group calibration device, comprising:

[0256] A status information acquisition module, used to acquire status information of the fixture array;

[0257] An information determination module, used to determine the quantity information of the openable clamps and / or the position information of the openable clamps based on the state information;

[0258] A condition determination module, used to determine whether the number of the openable clamps meets a preset condition and / or determine whether the position information of the openable clamps meets a preset condition;

[0259] The grasping determination module is used to determine whether the clamp group can perform grasping in this state according to the judgment result.

[0260] Optionally, the information determination module determines the position information of the openable clamp according to the contour information of the object to be grasped.

[0261] Optionally, the conditions that need to be met by the number of openable clamps are preset based on the weight of the object to be grasped.

[0262] Optionally, the conditions that need to be met for the position of the openable clamp are preset based on the center of gravity of the object to be grasped.

[0263] Optionally, the conditions that the positions of the openable fixtures need to meet include: the lines connecting the positions of multiple fixtures cannot form a straight line.

[0264] Optionally, the grasping determination module determines that grasping can be performed when the number of clamps meets a preset condition and the positions of the clamps meet a preset condition.

[0265] A suction cup array control method, comprising:

[0266] Obtain point cloud information of the glass to be grasped and obstacles;

[0267] Generate a search state based on the point cloud information and the search boundary factor parameters of the suction cup array;

[0268] For each search state, determine whether there is an obstacle under each suction cup, and if so, close the suction cup in the search state;

[0269] Select an alternative search state according to the suction cup opening status of each search state;

[0270] Selecting the best search state from the alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass;

[0271] Grab the glass using the best search state.

[0272] Optionally, the obstacle determination includes at least one of the following: glass boundary obstacle determination, rubber pad obstacle determination, rubber strip obstacle determination and protrusion obstacle determination.

[0273] Optionally, the glass boundary obstacle determination includes determining whether there is a glass boundary obstacle based on a point cloud ratio in an area below the suction cup.

[0274] Optionally, the rubber pad obstacle determination includes: determining whether there is a rubber pad obstacle based on an obstacle point cloud area, wherein the obstacle point cloud area is pre-set according to the rubber pad area.

[0275] Optionally, the raised obstacle determination includes: determining whether a raised obstacle exists based on a preset depth difference threshold and a logarithmic threshold.

[0276] Optionally, selecting the alternative search state according to the opening status of the suction cup in each search state includes: if there is at least one opened suction cup, selecting the search state as the alternative search state.

[0277] Optionally, selecting the best search state from alternative search states based on the number of suction cups opened and / or the distance between the suction cups and the center of the glass includes: selecting the search state with the largest number of suction cups opened; if there are multiple search states with the largest number of suction cups opened, further selecting the search state in which the suction cup is closest to the center of the glass.

[0278] Optionally, the method further includes: performing a clamp calibration for an optimal search state to determine whether the glass can be correctly gripped using the search state.

[0279] A suction cup array control device, comprising:

[0280] Point cloud acquisition module, used to obtain point cloud information of the glass to be grasped and obstacles;

[0281] A search state generation module, used to generate a search state based on point cloud information and search boundary factor parameters of the suction cup array;

[0282] An obstacle determination module is used to determine whether there is an obstacle under each suction cup for each search state, and if so, close the suction cup in the search state;

[0283] An alternative search state selection module is used to select an alternative search state according to the suction cup opening status of each search state;

[0284] An optimal search state selection module, used for selecting an optimal search state from alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass;

[0285] The grasping module is used for grasping the glass based on the best search state.

[0286] Optionally, the obstacle determination module is used to perform at least one of the following obstacle determinations: glass boundary obstacle determination, rubber pad obstacle determination, rubber strip obstacle determination and protrusion obstacle determination.

[0287] Optionally, when the obstacle determination module executes glass boundary obstacle determination, it determines whether there is a glass boundary obstacle based on the proportion of the point cloud in the area below the suction cup.

[0288] Optionally, when the obstacle determination module performs rubber pad obstacle determination, it determines whether there is a rubber pad obstacle based on the area of ​​the obstacle point cloud, wherein the area of ​​the obstacle point cloud is pre-set according to the area of ​​the rubber pad.

[0289] Optionally, when the obstacle determination module performs protruding obstacle determination, it determines whether a protruding obstacle exists based on a preset depth difference threshold and a logarithmic threshold.

[0290] Optionally, when there is at least one opened suction cup in a certain search state, the alternative search state selection module selects the search state as the alternative search state.

[0291] Optionally, the optimal search state selection module selects the search state with the largest number of open suction cups as the optimal search state. If there are multiple search states with the largest number of open suction cups, the search state in which the suction cup is closest to the center of the glass is further selected.

[0292] Optionally, the method further includes: performing a clamp calibration for an optimal search state to determine whether the glass can be correctly gripped using the search state.

[0293] A suction cup array calibration method, comprising:

[0294] Grouping all the suckers in the array according to the shape of the sucker array;

[0295] Get the status information of the suction cup array;

[0296] Determine the number of openable suction cups and group information of the openable suction cups based on the state information of the suction cup array;

[0297] Determining whether the number of openable suction cups meets a preset condition and / or whether the grouping information of the openable suction cups meets a preset condition;

[0298] It is determined whether the suction cup array can perform grasping in this state according to the judgment result.

[0299] Optionally, grouping all suction cups in the array according to the shape of the suction cup array includes dividing the suction cup array into four areas: upper left, lower left, upper right, and lower right, and dividing all suction cups into four groups according to the areas where all suction cups are located.

[0300] Optionally, the conditions that need to be met for the number of openable suction cups are preset based on the weight of the object to be grasped.

[0301] Optionally, the conditions that need to be met for grouping information of the suction cups to be opened are preset based on the center of gravity of the object to be grasped.

[0302] Optionally, the grouping information of the openable suction cups needs to meet the condition that the openable suction cups are distributed in at least three groups.

[0303] Optionally, determining whether the suction cup array can perform grabbing in this state according to the judgment result includes: when the number of suction cups meets a preset condition and the suction cup grouping information meets a preset condition, determining that grabbing can be performed.

[0304] A suction cup array calibration device, comprising:

[0305] A grouping module, used for grouping all the suckers in the array according to the shape of the sucker array;

[0306] A status information acquisition module, used to acquire status information of the suction cup array;

[0307] An information determination module, used to determine the number information of the openable suction cups and the group information of the openable suction cups based on the state information of the suction cup array;

[0308] A condition determination module, used to determine whether the number of openable suction cups meets a preset condition and / or whether the grouping information of the openable suction cups meets a preset condition;

[0309] The grasping determination module is used to determine whether the suction cup array can perform grasping in this state according to the judgment result.

[0310] Optionally, the grouping module is specifically used to divide the suction cup array into four areas: upper left, lower left, upper right, and lower right, and divide all the suction cups into four groups according to the areas where all the suction cups are located.

[0311] Optionally, the conditions that need to be met for the number of openable suction cups are preset based on the weight of the object to be grasped.

[0312] Optionally, the conditions that need to be met for grouping information of the suction cups to be opened are preset based on the center of gravity of the object to be grasped.

[0313] Optionally, the grouping information of the openable suction cups needs to meet the condition that the openable suction cups are distributed in at least three groups.

[0314] Optionally, the grabbing determination module is specifically used to: when the number of suction cups meets a preset condition and the suction cup grouping information meets a preset condition, determine that grabbing can be performed.

[0315] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0316] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0317] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0318] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0319] It should be understood that the various parts of the embodiments of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0320] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0321] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0322] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0323] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above implementation methods within the scope of the present application.

Claims

1. A suction cup array control method, characterized in that: include: Obtain point cloud information of the glass to be grasped and obstacles; Generate a search state based on the point cloud information and the search boundary factor parameters of the suction cup array; Perform obstacle determination, for each search state, determine whether there is an obstacle under each suction cup, and if so, close the suction cup in this search state; Select an alternative search state according to the suction cup opening status of each search state; Selecting the best search state from the alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass; Grab the glass using the best search state.

2. The suction cup array control method according to claim 1, characterized in that: The obstacle determination includes at least one of the following: glass boundary obstacle determination, rubber pad obstacle determination, rubber strip obstacle determination and protrusion obstacle determination.

3. The suction cup array control method according to claim 2, characterized in that: The glass boundary obstacle determination includes determining whether there is a glass boundary obstacle according to the point cloud ratio in the area below the suction cup.

4. The suction cup array control method according to claim 2, characterized in that: The rubber pad obstacle determination includes: determining whether there is a rubber pad obstacle based on an obstacle point cloud area, wherein the obstacle point cloud area is pre-set according to the rubber pad area.

5. The suction cup array control method according to claim 4, characterized in that: The raised obstacle determination includes: determining whether there is a raised obstacle based on a preset pixel depth difference threshold and a logarithmic threshold, wherein the logarithm is the number of pairings obtained by pairing the depth values ​​of all point clouds in the area below the suction cup.

6. The suction cup array control method according to claim 1, characterized in that: The selecting of the candidate search state according to the opening status of the suction cup in each search state includes: if there is at least one opened suction cup, selecting the search state as the candidate search state.

7. The suction cup array control method according to claim 1, characterized in that: The selecting the best search state from the alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass includes: selecting the search state with the largest number of suction cups opened; if there are multiple search states with the largest number of suction cups opened, further selecting the search state in which the suction cup is closest to the center of the glass.

8. The suction cup array control method according to any one of claims 1 to 7, characterized in that: Also includes: Perform a gripper calibration for the best search state to determine if the glass can be properly gripped using that search state.

9. A suction cup array control device, characterized in that: include: Point cloud acquisition module, used to obtain point cloud information of the glass to be grasped and obstacles; A search state generation module, used to generate a search state based on point cloud information and search boundary factor parameters of the suction cup array; An obstacle determination module is used to determine whether there is an obstacle under each suction cup for each search state, and if so, close the suction cup in the search state; An alternative search state selection module is used to select an alternative search state according to the suction cup opening status of each search state; An optimal search state selection module, used for selecting an optimal search state from alternative search states according to the number of suction cups opened and / or the distance between the suction cups and the center of the glass; The grasping module is used for grasping the glass based on the best search state.

10. The suction cup array control device according to claim 9, characterized in that: The obstacle determination module is used to perform at least one of the following obstacle determinations: glass boundary obstacle determination, rubber pad obstacle determination, rubber strip obstacle determination and protrusion obstacle determination.

11. The suction cup array control device according to claim 10, characterized in that: When the obstacle determination module executes the glass boundary obstacle determination, it determines whether there is a glass boundary obstacle according to the point cloud ratio in the area below the suction cup.

12. The suction cup array control device according to claim 10, characterized in that: When the obstacle determination module performs rubber pad obstacle determination, it determines whether there is a rubber pad obstacle based on the obstacle point cloud area, wherein the obstacle point cloud area is pre-set according to the rubber pad area.

13. The suction cup array control device according to claim 10, characterized in that: When the obstacle determination module performs raised obstacle determination, it determines whether there is a raised obstacle based on a preset pixel depth difference threshold and a logarithmic threshold, where the logarithm is the number of pairings obtained by pairing the depth values ​​of all point clouds in the area below the suction cup.

14. The suction cup array control device according to claim 9, characterized in that: The candidate search state selection module selects a search state as a candidate search state when there is at least one opened suction cup in a certain search state.

15. The suction cup array control device according to claim 9, characterized in that: The optimal search state selection module selects the search state with the largest number of suction cups opened as the optimal search state. If there are multiple search states with the largest number of suction cups opened, the search state in which the suction cup is closest to the center of the glass is further selected.

16. The suction cup array control device according to any one of claims 9 to 15, characterized in that: Also includes: Perform a gripper calibration for the best search state to determine if the glass can be properly gripped using that search state.

17. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the suction cup array control method according to any one of claims 1 to 8 when executing the computer program.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the suction cup array control method according to any one of claims 1 to 8 is implemented.

Images