Item picking method and device, and computer-readable storage medium

Abstract

The present disclosure relates to an item picking method and apparatus, and a computer-storable medium, and relates to the field of warehousing and logistics. The item picking method includes: obtaining picking information for multiple items in a cargo box, wherein the picking information includes the picking surface and picking point of the items, and an image of the cargo box in which the items are located; determining the proximity relationship between the items based on the image of the cargo box in which the items are located; determining the constraint relationship between the items based on at least one of the tilt angle and height of the items, and the proximity relationship between the items, wherein the tilt angle of the items is determined based on the picking surface and picking point of the items, and the height of the items is determined based on the image of the cargo box in which the items are located; and determining the picking order of the multiple items based on the constraint relationship.

Description

Technical Field

[0001] The present disclosure relates to the field of warehousing and logistics, and in particular to an item picking method and device, and a computer-readable storage medium. Background Art

[0002] With the popularity of online shopping, goods distribution is trending towards just-in-time delivery and a wider variety of products. The variety and quantity of items to be sorted in warehousing and logistics centers has increased dramatically, and sorting operations are becoming an increasingly important part of the workload in logistics and warehousing. Manual sorting alone is no longer sufficient to meet these large-scale sorting requirements, and automated sorting has become the key to improving sorting efficiency.

[0003] Items may be stacked in a container, meaning one item may be placed on top of another. If the system doesn't understand the stacking relationship between items, it may generate incorrect picking orders. For example, picking the bottom item first may make the pressed item difficult to pick up, or it may cause the top item to fall, resulting in damage. Summary of the Invention

[0004] According to a first aspect of the present disclosure, a method for picking items is provided, comprising: an item picking method comprising: obtaining picking information of a plurality of items in a cargo box, wherein the picking information includes a picking surface and a picking point of the items, and an image of the cargo box where the items are located; determining a proximity relationship between the items based on the image of the cargo box where the items are located; determining a constraint relationship between the items based on at least one of an inclination angle and a height of the items, and the proximity relationship between the items, wherein the inclination angle of the item is determined based on the picking surface and the picking point of the item, and the height of the item is determined based on the image of the cargo box where the items are located; and determining a picking order of the plurality of items based on the constraint relationship.

[0005] In some embodiments, determining the proximity relationship between items based on the image of the container in which the items are located includes:

[0006] In the case where two objects are in contact with each other, the two objects are determined to be adjacent to each other;

[0007] When an object is in contact with two other objects, the other two objects are determined to be adjacent to each other.

[0008] In some embodiments, determining the constraint relationship between the items includes:

[0009] In the case where the height of the item is greater than the height of its neighboring item, it is determined that the item has a constraint on the neighboring item.

[0010] In some embodiments, determining the constraint relationship between the items includes:

[0011] Group multiple items according to their tilt angles;

[0012] When an object and its neighboring objects belong to the same group, the constraint relationship between the object and the neighboring objects is determined according to the tilt angle of the object.

[0013] In some embodiments, when the item and its adjacent items belong to the same group, determining the constraint relationship between the item and the adjacent items based on the tilt angle of the item includes:

[0014] determining a first picking direction based on the area of ​​the picking surface of the items in the group, wherein the first picking direction indicates a first picking order of the items in the group;

[0015] Determining a second picking direction of the item and the adjacent item based on the picking points of the item and the adjacent item, wherein the second picking direction indicates a second picking order of the item and the adjacent item;

[0016] A constraint relationship between the item and the adjacent item is determined according to the first picking direction and the second picking direction.

[0017] In some embodiments, determining the first picking direction based on the area of ​​the picking surface of the items in the group includes:

[0018] Among the picking surfaces of the same group of items, when the ratio of the picking surface with the largest area to the picking surface with the smallest area exceeds the ratio threshold, a first vector is calculated as the first picking direction, wherein the first vector is the direction from the picking point on the picking surface with the largest area to the picking point on the picking surface with the smallest area.

[0019] In some embodiments, determining the first picking direction based on the area of ​​the picking surface of the items in the group includes:

[0020] In the case where the ratio of the largest picking surface to the smallest picking surface in the same group of items does not exceed a ratio threshold, a second vector is calculated based on the tilt angles of the items in the group, wherein the second vector represents the overall tilt direction of the group of items;

[0021] When the angle between the first vector and the second vector does not exceed a first threshold, determining the first vector as a first picking direction;

[0022] When the angle between the first vector and the second vector exceeds a first threshold, the opposite direction of the first vector is determined as the first picking direction.

[0023] In some embodiments, the second picking direction is a direction from the picking point of the picking face of the item to the picking point of the picking face of the adjacent item.

[0024] In some embodiments, determining the constraint relationship between the item and the adjacent item based on the first picking direction and the second picking direction includes:

[0025] When the included angle between the first picking direction and the second picking direction does not exceed a second threshold, it is determined that the item has a constraint on the adjacent item.

[0026] In some embodiments, determining the constraint relationship between the items includes:

[0027] When the item and its neighboring items do not belong to the same group and the height of the item is greater than the height of the neighboring items, it is determined that the item has a constraint on the neighboring item.

[0028] In some embodiments, the items include multiple picking surfaces, and determining the proximity relationship between the items based on the image of the container in which the items are located includes:

[0029] In the case where two picking surfaces are in contact with each other, the two picking surfaces are determined to be adjacent picking surfaces to each other;

[0030] When one picking surface is in contact with the other two picking surfaces, the other two picking surfaces are determined to be adjacent picking surfaces to each other.

[0031] In some embodiments, determining the constraint relationship between the items includes:

[0032] In the case where the height of a picking face is greater than the height of its adjacent picking face, it is determined that the picking face has a constraint on the adjacent picking face.

[0033] In some embodiments, the tilt angle of the item includes the tilt angle of the picking surface of the item, and determining the constraint relationship between the items includes:

[0034] Group the picking surfaces of items according to their inclination angles;

[0035] When a picking surface and its adjacent picking surface belong to the same group, the constraint relationship between the picking surface and the adjacent picking surface is determined according to the inclination angle of the picking surface.

[0036] In some embodiments, when the picking face and its adjacent picking face belong to the same group, determining the constraint relationship between the picking face and the adjacent picking face according to the inclination angle of the picking face includes:

[0037] Determining a first picking direction according to the areas of the picking faces in the group, wherein the first picking direction indicates a first picking order of the picking faces in the group;

[0038] Determining a second picking direction of the picking face and the adjacent picking face based on the picking points of the picking face and the adjacent picking face, wherein the second picking direction indicates a second picking order of the picking face and the adjacent picking face;

[0039] According to the first picking direction and the second picking direction, a constraint relationship between the picking surface and the adjacent picking surface is determined.

[0040] In some embodiments, determining the first picking direction according to the area of ​​the picking surface within the group includes:

[0041] In the same group of picking surfaces, when the ratio of the picking surface with the largest area to the picking surface with the smallest area exceeds the ratio threshold, the first vector is calculated as the first picking direction, where the first vector is the direction from the picking point on the picking surface with the largest area to the picking point on the picking surface with the smallest area.

[0042] In some embodiments, determining the first picking direction according to the area of ​​the picking surface within the group includes:

[0043] In the same group of picking surfaces, when the ratio of the picking surface with the largest area to the picking surface with the smallest area does not exceed a ratio threshold, a second vector is calculated according to the inclination angle of the group of picking surfaces, wherein the second vector represents the overall inclination direction of the group of picking surfaces;

[0044] When the angle between the first vector and the second vector does not exceed a third threshold, determining the first vector as the first picking direction;

[0045] When the angle between the first vector and the second vector exceeds a third threshold, the opposite direction of the first vector is determined as the first picking direction.

[0046] In some embodiments, the second picking direction is a direction from a picking point of a picking face to a picking point of an adjacent picking face.

[0047] In some embodiments, determining the constraint relationship between the picking surface and the adjacent picking surface based on the first picking direction and the second picking direction includes:

[0048] When the angle between the first picking direction and the second picking direction does not exceed a fourth threshold, it is determined that the picking surface has a constraint on the adjacent picking surface.

[0049] In some embodiments, determining the constraint relationship between the items includes:

[0050] When a picking face and its adjacent picking face do not belong to the same group and the height of the picking face is greater than the height of its adjacent picking face, it is determined that the picking face has a constraint on the adjacent picking face.

[0051] In some embodiments, determining the picking order of the plurality of items according to the constraint relationship includes:

[0052] Generate a sorting result based on the picking information, wherein the sorting result indicates the difficulty level of item picking;

[0053] Determine the picking order of multiple items based on the sorting results and the constraints between items.

[0054] According to a second aspect of the present disclosure, there is provided an item picking method, comprising:

[0055] an acquisition module configured to acquire picking information of a plurality of items in a cargo box, wherein the picking information includes a picking surface and a picking point of the items, and an image of the cargo box in which the items are located;

[0056] a proximity relationship determination module configured to determine proximity relationships between items based on images of the cargo boxes in which the items are located;

[0057] a constraint relationship determination module configured to determine the constraint relationship between the items based on at least one of an inclination angle and a height of the items and a proximity relationship between the items, wherein the inclination angle of the items is determined based on the picking surface and the picking point of the items, and the height of the items is determined based on an image of the cargo box in which the items are located;

[0058] The picking sequence determination module is configured to determine the picking sequence of multiple items according to the constraint relationship.

[0059] According to a third aspect of the present disclosure, there is provided an article picking device, comprising:

[0060] Memory; and

[0061] A processor coupled to the memory, wherein the processor is configured to execute the item picking method according to any embodiment of the present disclosure based on instructions stored in the memory.

[0062] According to a fourth aspect of the present disclosure, a computer storable medium stores computer program instructions thereon, which, when executed by a processor, implements the item picking method according to any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0064] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0065] Figure 1 A flowchart showing an item picking method according to some embodiments of the present disclosure is shown;

[0066] Figure 2 A schematic diagram showing stacking of items according to some embodiments of the present disclosure is shown;

[0067] Figure 3 Schematic diagrams showing stacking of articles according to other embodiments of the present disclosure;

[0068] FIG4( a ) shows a schematic diagram of determining a first picking direction according to some embodiments of the present disclosure;

[0069] FIG4( b ) shows a schematic diagram of determining a first picking direction according to some other embodiments of the present disclosure;

[0070] FIG4( c ) shows a schematic diagram of determining a second picking direction according to some embodiments of the present disclosure;

[0071] FIG4( d ) shows a schematic diagram of determining a constraint relationship between adjacent items according to some embodiments of the present disclosure;

[0072] Figure 5 A flowchart of generating a constraint relationship matrix according to some embodiments of the present disclosure is shown;

[0073] Figure 6 Schematic diagram showing stacking of items according to some further embodiments of the present disclosure;

[0074] Figure 7 Schematic diagrams showing stacking of items according to yet other embodiments of the present disclosure;

[0075] Figure 8 A schematic diagram illustrating determining constraint relationships between adjacent picking surfaces according to some embodiments of the present disclosure is shown;

[0076] Figure 9 A block diagram showing an article picking device according to some embodiments of the present disclosure;

[0077] Figure 10 A block diagram showing an article picking device according to some other embodiments of the present disclosure;

[0078] Figure 11 A block diagram of a computer system for implementing some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0079] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0080] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0081] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0082] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0083] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0084] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0085] In a cargo box, items may be stacked, with one item potentially pressing against another. This stacking of items can affect picking success rates, as pressed items are difficult to securely grasp. Improper picking order—for example, picking the lower item first—can cause the upper items to fall, damaging them. Furthermore, besides stacking items, tilting them can cause the left-hand item to partially press against the right-hand item, making it difficult to grasp. Furthermore, removing the right-hand item first can damage the left-hand item due to lack of support.

[0086] In the related art, it is difficult to determine the mutual constraint relationship between the stacked items, so the items can only be picked in the order in which they are identified, which may cause damage to the items.

[0087] Figure 1 A flowchart of an item picking method according to some embodiments of the present disclosure is shown.

[0088] like Figure 1 As shown, the item picking method includes steps S1 to S4.

[0089] In step S1, picking information of multiple items in a cargo box is obtained, wherein the picking information includes the picking surface and picking points of the items, and an image of the cargo box where the items are located.

[0090] For example, a camera is used to capture images of items stacked in a cargo box from above, that is, to capture a top-down view of the items, and the picking points and picking surfaces of the items are calculated. The picking point is the point on the item that the robotic arm contacts when grasping or sucking the item, and the picking surface is the surface where the picking point is located.

[0091] In step S2, the proximity relationship between the items is determined based on the image of the container where the items are located.

[0092] In some embodiments, when two objects are in contact with each other, the two objects are determined to be adjacent objects to each other; when one object is in contact with two other objects, the other two objects are determined to be adjacent objects to each other.

[0093] For example, adjacent items include adjacent items and sub-adjacent items. Adjacent items are two items that are in direct contact, and sub-adjacent items are adjacent items of adjacent items.

[0094] Figure 2 A schematic diagram showing stacking of items according to some embodiments of the present disclosure is shown.

[0095] Figure 2 It is a two-dimensional side view of items a, b, and c stacked vertically in the cargo box. Figure 2 As shown in the figure, item a and item b are in contact, so items a and b are defined as neighboring items. Item b is in contact with both items a and c, and there may be a constraint relationship between items a and c. Therefore, items a and c are also defined as neighboring items.

[0096] The present disclosure not only considers the constraint relationship between two items in direct contact, but also considers the possible constraint relationship between two items in indirect contact, comprehensively considers various possible situations, reduces omissions, and thus improves the success rate of picking according to the constraint relationship.

[0097] In some embodiments, a machine learning model is used to identify images of cargo boxes containing items and determine whether the items are in contact with each other.

[0098] For example, when an edge or a face of two objects partially or completely overlaps, the two objects are considered to be in contact with each other.

[0099] In step S3, the constraint relationship between the items is determined based on at least one of the inclination angle and height of the items and the proximity relationship between the items, wherein the inclination angle of the items is determined based on the picking surface and picking point of the items, and the height of the items is determined based on the image of the cargo box where the items are located.

[0100] In some embodiments, the constraint relationship between the items represents a stacking relationship between the items.

[0101] In some embodiments, determining the constraint relationship between the items includes: if a height of the item is greater than a height of a neighboring item, determining that the item has a constraint on the neighboring item.

[0102] exist Figure 2In the example, if item a is taller than item b and they are adjacent, then item a constrains item b. Similarly, if item a and item c are adjacent and item a is taller than item c, then item a also constrains item c. The item height can be the height of a surface, the height of the geometric center, the height of the center of gravity, the height of the picking point, or a combination of these.

[0103] In some embodiments, determining the constraint relationship between items includes: grouping multiple items according to the tilt angles of the items; if an item and its adjacent items belong to the same group, determining the constraint relationship between the item and the adjacent item according to the tilt angles of the item.

[0104] For example, items with the same tilting trend may overlap. Therefore, items with the same tilting trend are grouped together, and the stacking relationship of items within the group is determined based on the tilt angle. Furthermore, grouping can provide prior guidance for the picking order of items with the same tilting trend. Grouping by tilting trend provides a certain degree of error tolerance when determining the picking order of group members. Even if the angle of one member deviates to a certain extent from the main direction, the unified picking order of the entire group will not be affected.

[0105] In some embodiments, grouping a plurality of items according to their tilt angles includes grouping items having tilt angles within a preset range into one group.

[0106] Figure 3 Schematic diagrams showing stacking of items according to other embodiments of the present disclosure are shown.

[0107] Figure 3 It is a two-dimensional side view of items a, b, and c stacked horizontally in the cargo box. Figure 3 shown

[0108] If the inclination angles of items a, b, and c are similar, then items a, b, and c are grouped together.

[0109] In some embodiments, when an item and its neighboring item belong to the same group, determining the constraint relationship between the item and the neighboring item based on the tilt angle of the item includes: when the direction of the tilt angle of the group to which the item belongs and the direction from the item to its neighboring item are less than a preset threshold, determining that the item has a constraint on the neighboring item.

[0110] For example, Figure 3As shown, item a is partially pressing against item b. When calculating the constraint relationship, items a, b, and c are all tilted to the left. However, item a is to the left of its neighboring item b, meaning it points from right to left, at the opposite angle of tilt. Therefore, item a has no constraint on item b. Conversely, item b points from left to right, at a similar angle to the tilt, so item b has a constraint on item a.

[0111] In some embodiments, determining the constraint relationship between an item and a neighboring item based on the inclination angle of the item includes: determining a first picking direction based on the area of ​​the picking surface of the item in the group, wherein the first picking direction indicates a first picking order of the items in the group; when the item and the neighboring item belong to the same group, determining a second picking direction between the item and the neighboring item based on the picking points of the item and the neighboring item, wherein the second picking direction indicates a second picking order between the item and the neighboring item; determining the constraint relationship between the item and the neighboring item based on the first picking direction and the second picking direction.

[0112] In some embodiments, the tilt angle of the item is the angle between the normal vector of the picking surface of the item and the bottom surface of the cargo box.

[0113] The first picking direction can be determined according to the following method to represent the stacking relationship of items in the group.

[0114] In some embodiments, when the item and the adjacent item belong to the same group, the first picking direction is determined based on the area of ​​the picking surface of the items in the group, including: among the picking surfaces of the same group of items, when the ratio of the picking surface with the largest area to the picking surface with the smallest area exceeds a ratio threshold, calculating a first vector as the first picking direction, wherein the first vector is the direction from the picking point on the picking surface with the largest area to the picking point on the picking surface with the smallest area.

[0115] FIG4( a ) shows a schematic diagram of determining a first picking direction according to some embodiments of the present disclosure.

[0116] Figure 4(a) shows a two-dimensional side view of objects a, b, and c. As shown in Figure 4, objects a, b, and c have similar tilt angles and are grouped together.

[0117] Assuming that only one picking surface is considered for each item, the picking surfaces of items a, b, and c are sorted by area. In Figure 4(a), the picking surface with the smallest area is assumed to be Picking Surface 1, and the picking surface with the largest area is Picking Surface 2. The black dots are picking points, and the direction from the picking surface with the largest area to the picking surface with the smallest area is shown by the solid arrow in the figure. Assuming that the ratio of Picking Surface 2 to Picking Surface 1 exceeds the ratio threshold, it is considered that there is a significant difference in the area of ​​the picking surfaces of the items in the group, and the first picking direction is the direction from the picking surface with the largest area to the picking surface with the smallest area.

[0118] If an item has multiple picking surfaces, then from the multiple picking surfaces, the picking surface that is easiest to pick is selected based on the difficulty of grasping, or the angle between the inclination angle of the picking surface of the item and the inclination angle of the item is calculated, and the picking surface with an angle less than a preset threshold is selected, that is, the picking surface that is consistent with the inclination trend of the item is selected.

[0119] Typically, the topmost item has a larger exposed picking surface, while the pressed items have a smaller exposed picking surface. The area of ​​the picking surface can reflect the stacking relationship of the items. Therefore, the first picking direction for items in the same group is determined based on the size of the picking surface.

[0120] In some embodiments, the first picking direction of the items in the same group is determined based on the area of ​​the picking surfaces of the items in the group, including: among the picking surfaces of the items in the same group, when the ratio of the picking surface with the largest area to the picking surface with the smallest area does not exceed a ratio threshold, calculating a second vector based on the inclination angle of the items in the group, wherein the second vector represents the overall inclination direction of the items in the group; when the angle between the first vector and the second vector does not exceed a first threshold, determining the first vector as the first picking direction; when the angle between the first vector and the second vector exceeds the first threshold, determining the opposite direction of the first vector as the first picking direction.

[0121] For example, if the picking surfaces of the items within a group do not differ significantly in area, the system then determines whether the direction of the vector connecting the largest picking point to the smallest picking point is similar to the overall tilt of the group. If so, the group members' picking order follows the direction of the vector connecting the largest picking point to the smallest picking point. If not, the group members' picking order follows the direction opposite to the vector connecting the largest picking point to the smallest picking point.

[0122] In addition, the tilt angle of an item in the group can be selected as the overall tilt direction of the group, or the average or median value of the tilt angles of the group can be calculated as the overall tilt direction of the group, which is not limited here.

[0123] FIG4( b ) shows a schematic diagram of determining the first picking direction according to some other embodiments of the present disclosure.

[0124] As shown in Figure 4(b), assuming that the ratio of picking surface 2 to picking surface 1 does not exceed the ratio threshold, it is also necessary to compare the first vector and the second vector (i.e., the overall tilt direction of the group of items). Assuming that the first threshold is 90 degrees, it is obvious that the angle between the first vector and the second vector in Figure 4(b) does not exceed 90 degrees, then the first picking direction is the direction of the first vector. On the contrary, if the angle between the first vector and the second vector exceeds 90 degrees, then the first picking direction is the opposite direction of the first vector. In other words, the first picking direction finally selected is a direction that is closer to the overall tilt trend of the group of items, so that the first picking direction can more accurately represent the stacking situation of the group.

[0125] In some embodiments, the second picking direction is a direction from the picking point of the picking face of an item to the picking point of the picking face of an adjacent item.

[0126] FIG4( c ) shows a schematic diagram of determining a second picking direction according to some embodiments of the present disclosure.

[0127] As shown in Figure 4(c), assuming that the current item is b, it is necessary to determine the constraint relationship between item b and its adjacent item c. Then the second picking direction is defined as pointing from the picking point of item b to the picking point of item c.

[0128] In some embodiments, the constraint relationship between the item and the adjacent item is determined based on the first picking direction and the second picking direction, including: when the angle between the first picking direction and the second picking direction does not exceed a second threshold, determining that the item has a constraint on the adjacent item.

[0129] FIG4( d ) shows a schematic diagram of determining a constraint relationship between adjacent items according to some embodiments of the present disclosure.

[0130] The first and second picking directions are shown in Figure 4(d). Assuming the second threshold is 90 degrees, the angle between the first and second picking directions in Figure 4(d) is clearly greater than the second threshold, so item b has no constraint on item c. Conversely, if the angle between the direction from item c's picking point to item b's picking point and the first picking direction is less than the threshold, then item c has a constraint on item b.

[0131] In some embodiments, determining the constraint relationship between the items includes: if the item and its neighboring item do not belong to the same group and the height of the item is greater than that of the neighboring item, determining that the item has a constraint on the neighboring item.

[0132] For example, if the tilt trends of two objects are not similar, there is no need to determine the left-right stacking constraint, only the longitudinal stacking constraint needs to be determined. Therefore, the constraint relationship is determined based on the height.

[0133] In some embodiments, determining the constraint relationship between the items includes generating a constraint relationship matrix based on at least one of a height and an inclination angle.

[0134] The values ​​of the elements in the constraint relationship matrix adjacent_matrix are defined as follows.

[0135] adjacent_matrix[i,j]=0: Item i and item j are not adjacent items and have no constraint relationship;

[0136] adjacent_matrix[i,j]=1: Item i and item j are adjacent items, and item i has a constraint relationship with item j;

[0137] adjacent_matrix[i,j]=-1: Item i and item j are adjacent items, and item j has a constraint relationship with item i;

[0138] adjacent_matrix[i,j]=2: Item i and item j are adjacent items, but there is no constraint relationship.

[0139] Wherein, if adjacent_matrix[i,j]=1, then the corresponding adjacent_matrix[j,i]=-1.

[0140] Figure 5 A flowchart of generating a constraint relationship matrix according to some embodiments of the present disclosure is shown.

[0141] like Figure 5 As shown, generating the constraint relationship matrix includes steps (1)-(15).

[0142] In step (1), based on the angle between the normal vector of the picking point on the picking surface and the bottom of the cargo box, it is determined whether there is a common tilt trend between two items. Items with the same tilt trend are grouped together, and the first picking direction of each group is calculated. The first picking direction is the sorting direction of the picking sequence, which is a directional spatial vector used to indicate the picking order of items and the order in which the items are stacked.

[0143] In step (2), each current item is taken as a candidate and all items are traversed;

[0144] In step (3), determine whether the constraint relationships between all candidate items and their adjacent items have been calculated, where adjacent items refer to two items that are in contact with each other, and the corresponding relationships of each set of adjacent items are saved in the adjacent item set N. If yes, go to step (10), otherwise, go to step (4);

[0145] In step (4), traverse other items besides the current candidate;

[0146] In step (5), determine whether candidate item i and other item j are adjacent items. If yes, go to step (6). If not, set the position adjacent_matrix[i,j] of candidate item i to j in the constraint matrix to 0.

[0147] In step (6), determine whether candidate item i and adjacent item j belong to the same tilt trend group. If yes, go to step (7); if not, go to step (8);

[0148] In step (7), determine whether the first picking direction of the group of candidate item i and adjacent item j is consistent with the second picking direction from candidate item i to adjacent item j. If so, set the position of candidate item i to j in the constraint matrix to 1; if not, set the position of candidate item i to j to -1;

[0149] In step (8), determine whether the height of the candidate item and the adjacent item are equal. If so, set the position of candidate item i to j in the constraint matrix to 2. Otherwise, go to step (9);

[0150] In step (9), determine whether the height of the candidate item is lower than the height of the adjacent item. If so, set the position of candidate item i to j in the constraint matrix to -1. If not, set the position of candidate item i to j to 1.

[0151] In step (10), determine whether the constraint relationships between all candidate items and the next adjacent items have been calculated. If yes, end the calculation; if not, go to step (11);

[0152] In step (11), the adjacent items of the adjacent items of the current candidate are taken as the secondary adjacent items of the current candidate, the corresponding relationship between the items and their secondary adjacent items is stored in the secondary adjacent item set M, and the secondary adjacent items of the current candidate in the set M are traversed;

[0153] In step (12), determine whether candidate item i belongs to the group with the same tilt trend as the adjacent item j. If yes, go to step (13); if not, go to step (14);

[0154] In step (13), determine whether the first picking direction of the group of candidate item i and the next adjacent item j is consistent with the second picking direction from candidate item i to the next adjacent item j. If yes, set the position of candidate item i to j in the constraint matrix to 1; if not, set the position of candidate item i to j to -1;

[0155] In step (14), determine whether the height of candidate item i and the adjacent item j are equal. If so, set the position of candidate item i to j in the constraint matrix to 2. Otherwise, go to step (15);

[0156] In step (15), determine whether the height of candidate item i is lower than the height of the next adjacent item j. If so, set the position of candidate item i to j in the constraint matrix to -1. If not, set the position of candidate item i to j to 1. In addition, if in step (5), the corresponding position of i to j is calculated to be 0, but in step (15), the corresponding position of i to j is recalculated to be 1 or -1, then the current calculation result will overwrite the original calculation result.

[0157] The present disclosure first finds out the adjacent items that may have a constraint relationship with each other by determining the proximity relationship between the items, and then determines the mutual stacking constraint relationship between the items based on at least one of the height and tilt angle of the items and the proximity relationship, thereby realizing the identification of the horizontal stacking relationship and / or the vertical stacking relationship of the items, avoiding the risk of arranging the pressed items for picking before other items that press the items, solving the problem of the stacking of items affecting the picking order, and improving the success rate of the robot arm in picking items.

[0158] In addition to directly determining the constraint relationships between items, you can also determine the constraint relationships between the picking faces of items, thereby reflecting the constraint relationships between items. The following describes a method for determining the constraint relationships between the picking faces of items.

[0159] Determining the constraints between item picking faces is similar to determining constraints between items directly. In the aforementioned direct constraint determination method, each item is considered a candidate, and the proximity and constraint relationships between items are determined. However, when determining the constraints between item picking faces, each item's picking face is considered a candidate, and the proximity and constraint relationships between picking faces are determined.

[0160] In some embodiments, the items include multiple picking faces, and determining the proximity relationship between the items based on the image of the container in which the items are located includes: if there is contact between two picking faces, determining that the two picking faces are adjacent picking faces to each other;

[0161] When one picking surface is in contact with the other two picking surfaces, the other two picking surfaces are determined to be adjacent picking surfaces to each other.

[0162] Figure 6 Schematic diagram showing stacking of items according to further embodiments of the present disclosure.

[0163] Figure 6 is a three-dimensional side view of objects a and b stacked vertically. Figure 6 As shown in the figure, if side A1 of item a and side B1 of item b are adjacent picking surfaces, and side A1 of item a and side A2 of item a are adjacent picking surfaces, then side A2 of item a and sides B1 and B2 of item b are also adjacent picking surfaces. In other words, not only can two picking surfaces of different items be adjacent picking surfaces, but two picking surfaces of the same item can also be adjacent picking surfaces.

[0164] In some embodiments, a machine learning model is used to identify images of the cargo box where the items are located to determine whether the picking surfaces are in contact.

[0165] For example, when an edge or a face of two picking surfaces partially or completely overlaps, the two picking surfaces are considered to be in contact with each other.

[0166] like Figure 6 As shown, the A1 surface and the B1 surface have an edge that overlaps, that is, the A1 surface of the item a is in contact with the B1 surface of the item b, and the A1 surface of the item a and the B1 surface of the item b are determined to be adjacent picking surfaces.

[0167] The method for determining the constraint relationship between a picking surface and adjacent picking surfaces is similar to the method for determining the constraint relationship between an item and adjacent items. It will be briefly introduced below. For the same content, please refer to the aforementioned method for determining the constraint relationship between an item and adjacent items. It will not be repeated here.

[0168] In some embodiments, determining the constraint relationship between items includes: when the height of one picking surface is greater than the height of its adjacent picking surface, determining that the picking surface has a constraint on the adjacent picking surface.

[0169] For example, in Figure 6 In the example, surface A1 is the adjacent picking surface to surfaces B1 and B2, and its height is greater than that of surfaces B1 and B2. Therefore, surface A1 has a constraint on surfaces B1 and B2, requiring grasping from surface A1 before grasping from surfaces B1 or B2. The height of the picking surface can be the height of one edge of the picking surface, the height of the geometric center of the picking surface, the height of the picking point on the picking surface, or a combination of these heights based on actual needs.

[0170] In some embodiments, the tilt angle of an item includes the tilt angle of a picking surface of the item, and determining the constraint relationship between items includes: grouping the picking surfaces of the item according to the tilt angle of the picking surface; and, if a picking surface and its adjacent picking surface belong to the same group, determining the constraint relationship between the picking surface and the adjacent picking surface based on the tilt angle of the picking surface. The tilt angle is the tilt angle of the normal vector of the picking point on the picking surface relative to the bottom plane of the cargo box.

[0171] For example, picking surfaces with the same inclination trend may overlap with each other. Therefore, the picking surfaces with the same inclination trend are grouped together, and the overlapping relationship of the picking surfaces in the same group is determined based on the inclination angle.

[0172] In some embodiments, multiple picking surfaces are grouped according to the inclination angles of the picking surfaces, including: grouping the picking surfaces whose inclination angles are within a preset range into one group.

[0173] Figure 7 Schematic diagrams showing stacking of items according to further embodiments of the present disclosure are shown.

[0174] exist Figure 7 For the picking surfaces A1, B1, C1, A2, B2, and C2, the picking surfaces A1, B1, and C1 can be divided into one group, and A2, B2, and C2 can be divided into another group according to the inclination angle of the picking surfaces.

[0175] In some embodiments, determining the constraint relationship between a picking face and the adjacent picking face includes: determining a first picking direction based on the area of ​​the picking face in the group, wherein the first picking direction indicates a first picking order of the picking faces in the group; when the picking face and the adjacent picking face belong to the same group, determining a second picking direction between the picking face and the adjacent picking face based on the picking points of the picking face and the adjacent picking face, wherein the second picking direction indicates a second picking order between the picking face and the adjacent picking face; determining the constraint relationship between the picking face and the adjacent picking face based on the first picking direction and the second picking direction. The second picking direction is a direction from the picking point of the picking face to the picking point of its adjacent picking face.

[0176] The first picking direction can be determined according to the following method to represent the overlapping relationship of the picking surfaces within the group.

[0177] (1) In the same group of picking surfaces, when the ratio of the picking surface with the largest area to the picking surface with the smallest area exceeds a ratio threshold, a first vector is calculated as the first picking direction, wherein the first vector is the direction from the picking point on the picking surface with the largest area to the picking point on the picking surface with the smallest area.

[0178] (2) In the same group of picking surfaces, when the ratio of the picking surface with the largest area to the picking surface with the smallest area does not exceed the ratio threshold, a second vector is calculated based on the inclination angle of the group of picking surfaces, wherein the second vector represents the overall inclination direction of the group of picking surfaces; when the angle between the first vector and the second vector does not exceed the third threshold, the first vector is determined as the first picking direction; when the angle between the first vector and the second vector exceeds the third threshold, the opposite direction of the first vector is determined as the first picking direction.

[0179] In some embodiments, the constraint relationship between the picking surface and the adjacent picking surface is determined based on the first picking direction and the second picking direction, including: when the angle between the first picking direction and the second picking direction does not exceed a fourth threshold, determining that the picking surface is constrained to the adjacent picking surface.

[0180] Figure 8 A schematic diagram of determining constraint relationships between adjacent picking surfaces according to some embodiments of the present disclosure is shown.

[0181] like Figure 8 As shown in Figure 4(d), the first picking direction for the set of picking faces A2, B2, and C2, as well as the second picking direction from B2 to C2, are similar to those in Figure 4(d). Assuming the second threshold is 90 degrees, the angle between the first picking direction and the second picking direction is greater than the second threshold. Therefore, picking face A2 has no constraint on picking face C2. Conversely, picking face C2 has a constraint on picking face A2.

[0182] Image recognition solutions in picking systems such as robotic arms (for example, identifying images to determine height, contact, etc.) use supervised learning, which requires a large amount of labeled data. For two-dimensional picking surface-level recognition solutions, labeling technology is mature, labeling costs are low, and manual labeling is also easier to operate. According to some embodiments of the present disclosure, the constraint relationship between each picking surface is calculated based on the picking surface, thereby determining the picking order and reducing costs.

[0183] In some embodiments, determining the picking order of multiple items based on the constraint relationship includes: generating a sorting result based on the picking information, wherein the sorting result indicates the difficulty of picking the items; and determining the picking order of multiple items based on the sorting result and the constraint relationship between the items.

[0184] For example, based on the influencing factors that indicate the difficulty of item picking, a ranking result indicating the ease of item picking is generated. For example, a ranking result is generated based on the weighted sum of the influencing factors, or the influencing factors are used as input to a machine learning model to obtain a ranking result. The influencing factors include: the area of ​​the picking surface, the area of ​​the largest inscribed circle of the picking surface with the picking point as the center, the angle between the normal vector of the picking point and the normal vector of the bottom surface of the cargo box, the distance from the picking point to each surface of the cargo box, the distance from the picking point to the center of the cargo box, and the probability that the item identified by the machine learning model is the item to be picked.

[0185] In step S4, the picking order of the plurality of items is determined according to the constraint relationship.

[0186] For example, the picking order of an item is placed before the items it constrains. For an item i, if it is constrained by item j, the picking order of j is placed before i, thus obtaining the picking order of multiple items.

[0187] In some embodiments, determining the picking order of multiple items based on the constraint relationship includes: generating a sorting result based on the picking information, wherein the sorting result indicates the difficulty of picking the items; and determining the picking order of multiple items based on the sorting result and the constraint relationship between the items.

[0188] For example, the picking order of multiple items needs to meet the following conditions:

[0189] (1) Satisfy the constraints between items, that is, put the picking order of items before the items constrained by them.

[0190] (2) Under the premise of satisfying (1), the items whose sorting position cannot be determined are sorted according to the sorting result indicating the difficulty of picking the items.

[0191] When (1) and (2) conflict, it is necessary to ensure that the final item picking order satisfies (1).

[0192] If the constraint relationship between items is directly obtained in step S3, then multiple items are also sorted when calculating the sorting results and the final picking order. If the constraint relationship between picking surfaces is calculated in step S3, then multiple picking surfaces are also sorted when calculating the sorting results and the final picking order.

[0193] This method initially sorts items based on their difficulty level to produce a sorting result. Then, based on constraints, it rearranges these initial sorting results to produce a final picking order. While satisfying the constraints between items and the picking order requirements, it also considers the difficulty level of the items. This solves the problem of overlapping items impacting the grasping process and causing damage, while also ensuring that easier-to-pick items are picked first, improving both the success rate and efficiency of picking.

[0194] Figure 9 A block diagram of an item picking device according to some embodiments of the present disclosure is shown.

[0195] like Figure 9 As shown, the item picking device 9 includes an acquisition module 91 , a proximity relationship determination module 92 , a constraint relationship determination module 93 , and a picking order determination module 94 .

[0196] The acquisition module 91 is configured to acquire the picking information of multiple items in the container, wherein the picking information includes the picking surface and picking point of the items, and the image of the container where the items are located. For example, Figure 1 Step S1 shown;

[0197] The proximity relationship determination module 92 is configured to determine the proximity relationship between the items based on the image of the container where the items are located, for example, by performing the following steps: Figure 1 Step S2 shown;

[0198] The constraint relationship determination module 93 is configured to determine the constraint relationship between the items based on at least one of the tilt angle and height of the items and the proximity relationship between the items, wherein the tilt angle of the items is determined based on the picking surface and picking point of the items, and the height of the items is determined based on the image of the cargo box where the items are located. For example, Figure 1 Step S3 shown;

[0199] The picking order determination module 94 is configured to determine the picking order of multiple items according to the constraint relationship, for example, Figure 1 Step S4 is shown.

[0200] Figure 10 A block diagram illustrating an item picking device according to some other embodiments of the present disclosure is shown.

[0201] like Figure 10 As shown, the item picking device 10 includes a memory 101 and a processor 102 coupled to the memory 101. The memory 101 is used to store and execute the item picking method. The processor 102 is configured to execute the item picking method in any of the embodiments of the present disclosure based on the instructions stored in the memory 101.

[0202] Figure 11A block diagram of a computer system for implementing some embodiments of the present disclosure is shown.

[0203] like Figure 11 As shown, the computer system 110 may be implemented as a general-purpose computing device and includes a memory 1110, a processor 1120, and a bus 1100 that connects various system components.

[0204] Memory 1110 may include, for example, system memory, non-volatile storage media, and the like. System memory may store, for example, an operating system, application programs, a boot loader, and other programs. System memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. Non-volatile storage media may store, for example, instructions for executing the item picking method described in any of the embodiments of the present disclosure. Non-volatile storage media include, but are not limited to, disk storage, optical storage, and flash memory.

[0205] The processor 1120 can be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, or as discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the judgment module and the determination module, can be implemented by a central processing unit (CPU) executing instructions in a memory that execute corresponding steps, or by dedicated circuits that execute corresponding steps.

[0206] The bus 1100 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0207] Computer system 110 may also include input / output interfaces 1130, a network interface 1140, a storage interface 1150, and the like. These interfaces 1130, 1140, and 1150, as well as memory 1110 and processor 1120, may be connected via bus 1100. Input / output interfaces 1130 provide connection interfaces for input / output devices such as a display, mouse, and keyboard. Network interface 1140 provides connection interfaces for various networked devices. Storage interface 1150 provides connection interfaces for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0208] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0209] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0210] These computer-readable program instructions can also be readable and stored in a computer-readable memory. These instructions cause the computer to work in a specific manner, thereby producing an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0211] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0212] The article picking method and apparatus, and the computer-readable storage medium in the above-mentioned embodiments improve the success rate of article picking.

[0213] The article picking method and apparatus, as well as the computer-readable storage medium, according to the present disclosure have been described in detail. To avoid obscuring the present disclosure, some details known in the art have been omitted. Based on the above description, those skilled in the art will readily understand how to implement the disclosed technical solutions.

Claims

1. An item picking method, comprising: Obtaining picking information of multiple items in a cargo box, wherein the picking information includes the picking surface and picking point of the items, and an image of the cargo box where the items are located; Determine the proximity between items based on the image of the container where the items are located; Determining the constraint relationship between the objects based on at least one of the tilt angle and height of the objects and the proximity relationship between the objects includes: Grouping items according to their tilt angles, and determining the constraint relationship between an item and its neighboring items based on their tilt angles when the item and its neighboring items belong to the same group, including: determining a first picking direction based on the area of ​​a picking surface of the items in the group, the first picking direction indicating a first picking order for the items in the group; determining a second picking direction based on the picking points of the item and its neighboring items, the second picking direction indicating a second picking order for the item and its neighboring items; and determining the constraint relationship between the item and the neighboring items based on the first picking direction and the second picking direction; or The picking surfaces of the items are grouped according to their inclination angles. When a picking surface and its adjacent picking surfaces belong to the same group, the constraint relationship between the picking surface and its adjacent picking surfaces is determined according to the inclination angles of the picking surfaces, including: determining a first picking direction according to the area of ​​the picking surfaces in the group, the first picking direction indicating a first picking order of the picking surfaces in the group; determining a second picking direction according to the picking points of the picking surface and its adjacent picking surfaces, the second picking direction indicating a second picking order of the picking surface and its adjacent picking surfaces; and determining the constraint relationship between the picking surface and the adjacent picking surface according to the first picking direction and the second picking direction; The tilt angle of the item is determined by the picking surface and picking point of the item, and the height of the item is determined by the image of the container where the item is located. Determine the picking order of multiple items based on constraints.

2. The article picking method according to claim 1, wherein: Determining the proximity relationship between items based on the image of the container where the items are located includes: In the case where two objects are in contact with each other, the two objects are determined to be adjacent to each other; When an object is in contact with two other objects, the other two objects are determined to be adjacent to each other.

3. The article picking method according to claim 2, wherein: Determining the constraint relationship between items includes: In the case where the height of the item is greater than the height of its neighboring item, it is determined that the item has a constraint on the neighboring item.

4. The article picking method according to claim 1, wherein: Determining the first picking direction according to the area of ​​the picking surface of the items in the group includes: Among the picking surfaces of the same group of items, when the ratio of the picking surface with the largest area to the picking surface with the smallest area exceeds the ratio threshold, a first vector is calculated as the first picking direction, wherein the first vector is the direction from the picking point on the picking surface with the largest area to the picking point on the picking surface with the smallest area.

5. The article picking method according to claim 4, wherein: Determining the first picking direction according to the area of ​​the picking surface of the items in the group includes: In the case where the ratio of the largest picking surface to the smallest picking surface in the same group of items does not exceed a ratio threshold, a second vector is calculated based on the tilt angles of the items in the group, wherein the second vector represents the overall tilt direction of the group of items; When the angle between the first vector and the second vector does not exceed a first threshold, determining the first vector as a first picking direction; When the angle between the first vector and the second vector exceeds a first threshold, the opposite direction of the first vector is determined as the first picking direction.

6. The article picking method according to claim 1, wherein: The second picking direction is a direction from the picking point of the picking surface of the item to the picking point of the picking surface of the adjacent item.

7. The article picking method according to claim 1, wherein: Determining the constraint relationship between the item and the adjacent item according to the first picking direction and the second picking direction includes: When the included angle between the first picking direction and the second picking direction does not exceed a second threshold, it is determined that the item has a constraint on the adjacent item.

8. The article picking method according to claim 1, wherein: Determining the constraint relationship between items includes: When the item and its neighboring items do not belong to the same group and the height of the item is greater than the height of the neighboring items, it is determined that the item has a constraint on the neighboring item.

9. The article picking method according to claim 1, wherein: The item includes multiple picking surfaces, and determining the proximity relationship between the items based on the image of the container where the items are located includes: In the case where two picking surfaces are in contact with each other, the two picking surfaces are determined to be adjacent picking surfaces to each other; When one picking surface is in contact with the other two picking surfaces, the other two picking surfaces are determined to be adjacent picking surfaces to each other.

10. The article picking method according to claim 9, wherein: Determining the constraint relationship between items includes: In the case where the height of a picking face is greater than the height of its adjacent picking face, it is determined that the picking face has a constraint on the adjacent picking face.

11. The article picking method according to claim 1, wherein: The determining of the first picking direction according to the area of ​​the picking surface within the group includes: In the same group of picking surfaces, when the ratio of the picking surface with the largest area to the picking surface with the smallest area exceeds the ratio threshold, the first vector is calculated as the first picking direction, where the first vector is the direction from the picking point on the picking surface with the largest area to the picking point on the picking surface with the smallest area.

12. The article picking method according to claim 11, wherein: The determining of the first picking direction according to the area of ​​the picking surface within the group includes: In the same group of picking surfaces, when the ratio of the picking surface with the largest area to the picking surface with the smallest area does not exceed a ratio threshold, a second vector is calculated according to the inclination angle of the group of picking surfaces, wherein the second vector represents the overall inclination direction of the group of picking surfaces; When the angle between the first vector and the second vector does not exceed a third threshold, determining the first vector as the first picking direction; When the angle between the first vector and the second vector exceeds a third threshold, the opposite direction of the first vector is determined as the first picking direction.

13. The article picking method according to claim 1, wherein: The second picking direction is a direction from a picking point of a picking surface to a picking point of an adjacent picking surface.

14. The article picking method according to claim 1, wherein: The determining of the constraint relationship between the picking surface and the adjacent picking surface according to the first picking direction and the second picking direction includes: When the angle between the first picking direction and the second picking direction does not exceed a fourth threshold, it is determined that the picking surface has a constraint on the adjacent picking surface.

15. The article picking method according to claim 1, wherein: Determining the constraint relationship between items includes: When a picking face and its adjacent picking face do not belong to the same group and the height of the picking face is greater than the height of its adjacent picking face, it is determined that the picking face has a constraint on the adjacent picking face.

16. The article picking method according to claim 1, wherein: Determining the picking order of multiple items based on constraints includes: Generate a sorting result based on the picking information, wherein the sorting result indicates the difficulty level of item picking; Determine the picking order of multiple items based on the sorting results and the constraints between items.

17. A method for picking items, comprising: an acquisition module configured to acquire picking information of a plurality of items in a cargo box, wherein the picking information includes a picking surface and a picking point of the items, and an image of the cargo box in which the items are located; a proximity relationship determination module configured to determine proximity relationships between items based on images of the cargo boxes in which the items are located; The constraint relationship determination module is configured to determine the constraint relationship between the objects based on at least one of the tilt angle and height of the objects and the proximity relationship between the objects, including: Grouping items according to their tilt angles, and determining the constraint relationship between an item and its neighboring items based on their tilt angles when the item and its neighboring items belong to the same group, including: determining a first picking direction based on the area of ​​a picking surface of the items in the group, the first picking direction indicating a first picking order for the items in the group; determining a second picking direction based on the picking points of the item and its neighboring items, the second picking direction indicating a second picking order for the item and its neighboring items; and determining the constraint relationship between the item and the neighboring items based on the first picking direction and the second picking direction; or According to the inclination angle of the picking surface of the item, the picking surfaces are grouped. When the picking surface and its adjacent picking surfaces belong to the same group, the constraint relationship between the picking surface and its adjacent picking surfaces is determined according to the inclination angle of the picking surface, including: determining a first picking direction according to the area of ​​the picking surface in the group, the first picking direction indicates the first picking order of the picking surfaces in the group; determining a second picking direction according to the picking points of the picking surface and its adjacent picking surfaces, the second picking direction indicates the second picking order of the picking surface and its adjacent picking surfaces; determining the constraint relationship between the picking surface and the adjacent picking surface according to the first picking direction and the second picking direction, The tilt angle of the item is determined based on the picking surface and picking point of the item, and the height of the item is determined based on the image of the container where the item is located; The picking sequence determination module is configured to determine the picking sequence of multiple items according to the constraint relationship.

18. An article picking device, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the item picking method according to any one of claims 1 to 16 based on instructions stored in the memory.

19. A computer storable medium having computer program instructions stored thereon, wherein when the instructions are executed by a processor, the method for picking items according to any one of claims 1 to 16 is implemented.

Images