Item picking method and device, and computer-readable storage medium

Abstract

The present disclosure relates to an item picking method and device, and a computer-storable medium, and relates to the field of warehousing and logistics. The item picking method includes: obtaining picking information for multiple items in a cargo box; generating a first sorting result based on the picking information, wherein the first sorting result indicates the difficulty of picking the items; calculating the constraint relationship between the items based on the picking information; generating a second sorting result based on the first sorting result and the constraint relationship between the items, wherein the second sorting result indicates the picking priority of the items; and determining the picking order of the items to be picked according to the order of the second sorting results. According to the present disclosure, the efficiency of item picking is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of warehousing and logistics, and in particular to an item picking method and device, and a computer-readable storage medium. Background Art

[0002] With the popularity of online shopping, goods distribution is trending towards just-in-time delivery and a wider variety of products. The variety and quantity of items to be sorted in warehousing and logistics centers has increased dramatically, and sorting operations are becoming an increasingly important part of the workload in logistics and warehousing. Manual sorting alone is no longer sufficient to meet these large-scale sorting requirements, and automated sorting has become the key to improving sorting efficiency.

[0003] In automated sorting, when instructing a robotic arm to pick multiple items from a container, the robotic arm's operating system needs to be provided with the order in which the items are to be picked. In related art, items are picked in the order in which they are identified by the picking system. Summary of the Invention

[0004] According to a first aspect of the present disclosure, there is provided an item picking method, comprising:

[0005] Get the picking information of multiple items in a container;

[0006] Generating a first sorting result according to the picking information, wherein the first sorting result indicates the difficulty level of item picking;

[0007] Calculate the constraint relationship between items based on the picking information;

[0008] Generate a second sorting result based on the first sorting result and the constraint relationship between the items, wherein the second sorting result represents the picking priority of the items;

[0009] The picking order of the items to be picked is determined according to the order of the second sorting results.

[0010] In some embodiments, generating the second sorting result based on the first sorting result and the constraint relationship between the items includes:

[0011] Generate a structure tree based on the first sorting result and the constraint relationship between the items, wherein each node in the structure tree corresponds to an item, the priority sorting of all items in the same layer conforms to the first sorting result, and there is a constraint relationship between the item and at least one item in the layer above it;

[0012] Under the premise of retaining the sorting of items in the same layer, according to the constraint relationship between items and the preset merging rules, multiple layers of the structure tree are merged into one layer to obtain a second sorting result.

[0013] In some embodiments, generating a structure tree based on the first sorting result and the constraint relationship between items includes:

[0014] Set the items without parent nodes as the first level of the structure tree. Starting from the first level, sort the items in the current level according to the first sorting result, where the parent node is the item that constrains the item.

[0015] Determine the child nodes of the item in the current layer as the items in the layer below the current layer, where the child nodes are the items constrained by the item;

[0016] The steps of sorting the items in the current layer according to the first sorting result and determining the items in the next layer are repeated until the items in the current layer have no child nodes.

[0017] In some embodiments, under the premise of retaining the order of items in the same layer, merging multiple layers of the structure tree into one layer according to the constraint relationship between the items and the preset merging rules to obtain the second sorting result includes:

[0018] Starting from the items at the last level of the structure tree, under the premise of keeping the order of the items in the previous level unchanged and the priority of the items in the current level lower than the priority of its parent node, determine the position of the items in the current level after merging them into the previous level according to the first sorting result;

[0019] The steps of determining the location of items are repeated until the structure tree is merged into one layer.

[0020] In some embodiments, while retaining the order of items within the same layer, merging multiple layers of the structure tree into one layer according to the constraints between the items and a preset merging rule to obtain a second sorting result includes:

[0021] Starting from the items on the first level of the structure tree, under the premise of keeping the order of the items on the current level unchanged and the priority of the items on the current level being greater than the priority of its child nodes, determine the position of the items on the next level after merging them into the current level based on the first sorting result;

[0022] The steps of determining the location of items are repeated until the structure tree is merged into one layer.

[0023] In some embodiments, the picking information includes:

[0024] An image of the bin the item is in, the item's pick point, and the pick face.

[0025] In some embodiments, generating a first sorting result according to the picking information includes:

[0026] Based on the picking information, calculate the influencing factors reflecting the difficulty of picking items;

[0027] A first ranking result is generated according to the impact factor.

[0028] In some embodiments, generating a first ranking result according to the impact factor includes:

[0029] A first ranking result is generated based on the top N items ranked by impact factor, where N is a positive integer.

[0030] In some embodiments, the influencing factors include at least one of the area of ​​the picking surface, the area of ​​the largest inscribed circle of the picking surface with the picking point as the center, the angle between the normal vector of the picking point and the normal vector of the bottom surface of the cargo box, the distance from the picking point to each surface of the cargo box, and the distance from the picking point to the center of the cargo box.

[0031] In some embodiments, the influencing factor includes a probability that the item is an item to be picked, and calculating the influencing factor for item picking based on the picking information includes:

[0032] Based on the image of the container where the item is located, a machine learning model is used to calculate the probability that the item is an item to be picked.

[0033] In some embodiments, the item picking method further includes:

[0034] Before generating the first sorting result, items to be sorted are screened out based on the picking information and preset filtering conditions.

[0035] In some embodiments, the constraint relationship between the items reflects the stacking relationship between the items.

[0036] According to a second aspect of the present disclosure, there is provided an article picking device, comprising:

[0037] an acquisition module configured to acquire picking information of a plurality of items in a cargo box;

[0038] A first generating module is configured to generate a first sorting result based on the picking information, wherein the first sorting result indicates the difficulty level of item picking;

[0039] a calculation module configured to calculate constraint relationships between items based on the picking information;

[0040] a second generating module configured to generate a second sorting result according to the first sorting result and the constraint relationship between the items, wherein the second sorting result represents the picking priority of the items;

[0041] The determination module is configured to determine the order of picking the items to be picked according to the order of the second sorting result.

[0042] According to a third aspect of the present disclosure, there is provided an article picking device, comprising:

[0043] Memory; and

[0044] A processor coupled to the memory, wherein the processor is configured to execute the item picking method according to any embodiment of the present disclosure based on instructions stored in the memory.

[0045] According to a fourth aspect of the present disclosure, a computer storable medium is provided, on which computer program instructions are stored. When the instructions are executed by a processor, the item picking method according to any embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0048] Figure 1 A flowchart of an item picking method according to some embodiments of the present disclosure is shown;

[0049] FIG2( a ) shows a schematic diagram of constraint relationships between items according to some embodiments of the present disclosure;

[0050] FIG2( b ) shows a schematic diagram of constraint relationships between items according to other embodiments of the present disclosure;

[0051] Figure 3 A schematic diagram showing a structure tree according to some embodiments of the present disclosure is shown;

[0052] Figure 4 A flowchart of determining a parent node according to some embodiments of the present disclosure is shown;

[0053] Figure 5 A flowchart of generating a structure tree according to some embodiments of the present disclosure is shown;

[0054] Figure 6 A flowchart of structure tree merging according to some embodiments of the present disclosure is shown;

[0055] Figure 7 A block diagram of an article picking device according to some embodiments of the present disclosure is shown;

[0056] Figure 8 A block diagram showing an article picking device according to other embodiments of the present disclosure

[0057] Figure 9 A block diagram of a computer system for implementing some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0058] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0059] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0060] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0061] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0062] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0063] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0064] In related technologies, items are picked in the order they are identified. For example, an artificial intelligence model is used to identify images of items in a container, and each item is picked upon identification. However, this picking method has the following problems.

[0065] (1) Due to factors such as posture and placement, different items are difficult to pick up at different levels. If difficult items are picked up first, the efficiency of the entire picking process may be affected.

[0066] (2) The mutual constraints between items are not taken into consideration. For example, if one item is pressing down on another item, the item on top should be picked up first, rather than the item underneath.

[0067] In addition, if a single indicator is used as the evaluation standard for sorting picking points, it is difficult to take into account a more comprehensive situation. For example, if only the distance from the item to the robot arm is used as the basis for sorting, some items that are closer to the robot arm but have a smaller picking surface and a larger picking surface inclination angle are still difficult to pick and will be picked first; while some items that are slightly farther away from the robot arm but have a larger flat picking surface and a smaller picking surface inclination angle are more conducive to picking, but will not be picked first. It is also possible that an item may be easier to pick but is pressed underneath other items. If sorting is only based on the difficulty of picking, picking failure may occur.

[0068] Figure 1 A flowchart of an item picking method according to some embodiments of the present disclosure is shown.

[0069] like Figure 1 As shown, the item picking method includes steps S1 to S5.

[0070] In step S1, picking information of multiple items in a container is obtained.

[0071] In some embodiments, the picking information includes: an image of the container in which the item is located, the picking point and the picking surface of the item.

[0072] For example, a camera is used to capture images of items stacked in a cargo box from above, that is, to capture a top-down view of the items, and the picking points and picking surfaces of the items are calculated. The picking point is the point on the item that the robotic arm contacts when grasping or sucking the item, and the picking surface is the surface where the picking point is located.

[0073] In some embodiments, the picking information also includes camera parameters.

[0074] The picking information can be further processed to determine the coordinates of the cargo box based on the image and camera parameters in the picking information.

[0075] In step S2, a first sorting result is generated according to the picking information, wherein the first sorting result indicates the difficulty level of item picking.

[0076] For example, items are initially sorted according to the difficulty of picking them, generating a first sorting result.

[0077] In some embodiments, generating the first sorting result based on the picking information includes: calculating an influencing factor reflecting the difficulty of picking the item based on the picking information; and generating the first sorting result based on the influencing factor.

[0078] In some embodiments, the influencing factors include at least one of the area of ​​the picking surface, the area of ​​the largest inscribed circle of the picking surface with the picking point as the center, the angle between the normal vector of the picking point and the normal vector of the bottom surface of the cargo box, the distance from the picking point to each surface of the cargo box, and the distance from the picking point to the center of the cargo box.

[0079] Among them, the angle between the normal vector of the picking point and the normal vector of the bottom surface of the cargo box, the distance from the picking point to each surface of the cargo box, and the distance from the picking point to the center of the cargo box are determined according to the coordinates of the cargo box, the picking point, and the picking surface.

[0080] In some embodiments, the influencing factor includes the probability that the item is an item to be picked, and calculating the influencing factor for item picking based on the picking information includes: calculating the probability that the item is an item to be picked based on an image of the cargo box where the item is located using a machine learning model.

[0081] For example, not all items in a container need to be picked, so the probability of an item being a candidate for picking must be determined. Furthermore, during the system's recognition process, items that are not merchandise may sometimes be considered merchandise and included in the picking information. Therefore, further filtering of these items that should not be picked is necessary to ultimately determine the items that should be included in the first and second sorting results.

[0082] The relationship between the above-mentioned influencing factors and the difficulty of item picking is as follows.

[0083] 1) Picking surface area: The larger the picking surface area, the easier it is to pick. The picking surface area is calculated in the pixel coordinate system.

[0084] 2) The maximum inscribed circle area of ​​the picking surface with the picking point as the center: the larger the inscribed circle area, the easier it is to pick. The inscribed circle area of ​​the picking surface is calculated in the pixel coordinate system;

[0085] 3) The angle between the normal vector of the picking point and the normal vector of the cargo box bottom: The smaller the angle, the smaller the tilt, and the easier it is to pick. The angle between the normal vector of the picking point and the normal vector of the cargo box bottom is calculated in the world coordinate system;

[0086] 4) Distance from the picking point to the bottom of the container: The larger this distance, the closer the picking point is to the robotic arm, making it less likely that items will be crushed by other items and easier to pick. The distance from the picking point to the bottom of the container is calculated in the world coordinate system.

[0087] 5) Minimum distance from the picking point to the four walls of the container: The smaller this minimum distance, the more likely the robot arm will collide with the four walls during the picking process, making it less likely to pick items. The minimum distance from the picking point to the four walls of the container is calculated in the pixel coordinate system.

[0088] 6) Distance from the picking point to the center of the container: The smaller this distance, the closer the items on the surface are to the center of the container, the less likely the robot arm will collide with the container, and the easier it is to pick them. The distance from the picking point to the center of the container is calculated in the pixel coordinate system.

[0089] 7) Probability of the item being picked (i.e., the recognition model score corresponding to the item being picked): This score indicates how well the machine learning model recognizes the item. A higher score indicates a higher probability that the object is a commodity that needs to be picked, and a higher probability that it should be picked.

[0090] In order to maintain the uniformity of the dimensions of each influencing factor, the scores of the above influencing factors need to be normalized before calculating the initial ranking score of each item. For example, the softmax method is used for normalization.

[0091] In some embodiments, generating the first ranking result according to the impact factors includes: generating the first ranking result according to a weighted sum of the impact factors.

[0092] For example, multiple influencing factors are calculated, and each factor is assigned a weight w. Assuming an influencing factor has a value of v, then w*v is the weighted score of that factor. Summing the weighted scores of all influencing factors yields the overall ranking score, which in turn yields the first ranking result. By comprehensively considering multiple influencing factors to determine the difficulty of item picking, we can account for more situations and improve the success rate of picking.

[0093] A machine learning model can also be used to generate the first ranking result.

[0094] For example, machine learning models such as Xgboost, Lightgbm, RankNe LambdaMART, ES-Rank, DirectRanker, and PiRank can be used. During the training phase, impact factor data and some manually labeled ranking data are used as input for training to generate a ranking model. During the model inference phase, the weights of the trained ranking model are used to rank new data to obtain a preliminary ranking result. Using machine learning models can produce more accurate first-order ranking results.

[0095] In some embodiments, generating a first ranking result based on the impact factor includes: generating a first ranking result based on N items ranked first by the impact factor, where N is a positive integer.

[0096] For example, the robotic arm only needs to pick up 10 items this time, so it only cares about the N results with the highest influencing factors. Then, the items ranked after 10 will not be included in the first sorting result as items to be picked, and will not participate in subsequent steps, so that it can flexibly adapt to picking needs.

[0097] In some embodiments, the item picking method further includes, before generating the first sorting result, screening out items to be sorted based on the picking information and preset filtering conditions.

[0098] For example, some filtering conditions are set in advance to determine whether the item can be picked and whether the item identification is effective. The filtering conditions can be the inclination angle, the detection score of the machine learning model, etc. If these conditions are not met, the item will not participate in the initial sorting, and will not be included in the first sorting result as an item to be picked. In this way, items that are not suitable for picking can be excluded first, avoiding these items affecting the efficiency of the robot arm's picking, and improving the calculation efficiency of the first sorting result and the second sorting result.

[0099] In step S3, the constraint relationships between items are calculated based on the picking information.

[0100] In some embodiments, the constraint relationship between the items reflects the stacking relationship between the items.

[0101] FIG2( a ) shows a schematic diagram of constraint relationships between items according to some embodiments of the present disclosure.

[0102] Figure 2(a) shows a side view of stacked items a and b. As shown in Figure 2(a), if you want to pick up item b, you should pick up item a first. Otherwise, if you pick up item b first, the robot arm might pick up item b while also picking up item a, causing it to fall and damage. Therefore, item a is set to have a constraint on item b.

[0103] In some embodiments, calculating the constraint relationship between the items based on the picking information includes: if the height of the item is greater than the height of its adjacent item, determining that the item has a constraint on the adjacent item, wherein the adjacent item is an item that is in contact with the item.

[0104] For example, in Figure 2(a), the height of item a is greater than that of item b, and items a and b are adjacent, so item a has a constraint on item b.

[0105] In some embodiments, the constraint relationship between items is calculated based on the picking information, including: when the angle between the inclination angle of the item and the inclination angle of its adjacent item is less than a constraint threshold, the constraint relationship between the item and the adjacent item is determined based on the inclination angle of the item.

[0106] For example, when the direction of the tilt angle between the object and its adjacent object and the direction from the object to its adjacent object are smaller than a preset threshold, it is determined that there is a constraint between the object and the adjacent object.

[0107] FIG2( b ) shows a schematic diagram of the constraint relationship between items according to other embodiments of the present disclosure.

[0108] Figure 2(b) shows a side view of objects a and b stacked horizontally. As shown in Figure 2(b), object a partially presses against object b. When calculating the constraint relationship, object a and its adjacent object b have the same tilt trend, both tilting to the right. Object A is to the left of object B, meaning that object A points from left to right towards object B, similar to the tilt angle. Therefore, object A constrains object B.

[0109] In step S4, a second sorting result is generated according to the first sorting result and the constraint relationship between the items, wherein the second sorting result represents the picking priority of the items.

[0110] For example, under the premise of ensuring that the constrained items are picked later than the constraining items, the picking order of the items is adjusted according to the difficulty of picking, thereby determining the final picking order. By calculating the constraint relationship between items and calculating the second sorting result, the risk of stacked items at the bottom being picked before the top items in the final sorting result can be avoided.

[0111] In some embodiments, a structure tree is generated based on the first sorting result and the constraint relationship between items, wherein each node in the structure tree corresponds to an item, the priority sorting of all items in the same layer conforms to the first sorting result, and there is a constraint relationship between the item and at least one item in the layer above it; while retaining the sorting of items in the same layer, according to the constraint relationship between the items and the preset merging rules, the multiple layers of the structure tree are merged into one layer to obtain a second sorting result.

[0112] Figure 3 A schematic diagram of a structure tree according to some embodiments of the present disclosure is shown.

[0113] Assume that item 1 has constraints on items 4, 3, and 2, and item 4 has constraints on item 6. In the first sorting result, the order of difficulty of picking items from easy to difficult is: item 6 > item 5 > item 4 > item 3 > item 2 > item 1. Figure 3 As shown, items 1 and 5 are not constrained by other items and are placed on the first layer, sorted according to the first sorting result. Items 4, 3, and 2, which are constrained by item 1, are placed on the second layer, and the nodes on the second layer are sorted according to the first sorting result, for example, item 4, item 3, and item 2.

[0114] Within the structure tree, items at the top have higher priority than their children at the bottom. This aligns with the actual stacking order of items. For example, items at the top of a crate are more likely to be at the top of the structure tree. Within each level of the tree, the order is also based on the highest first-sort result. The first level of the tree structure consists of candidates from the first-sort result that are not suppressed by constraints, i.e., the top-most item in the crate.

[0115] Constructing a structure tree based on the constraint relationship can simulate the actual stacking of items in the cargo box, while also taking into account the difficulty of picking items, thereby optimizing the final second sorting result.

[0116] In some embodiments, generating a structure tree based on the first sorting result and the constraint relationship between the items includes:

[0117] Items without parent nodes are regarded as the first layer of the structure tree. Starting from the first layer, the items in the current layer are sorted according to the first sorting result, where the parent node is the item that constrains the item; the child nodes of the items in the current layer are determined to be items in the layer below the current layer, where the child nodes are items constrained by the item; the steps of sorting the items in the current layer according to the first sorting result and determining the items in the next layer are repeated until the items in the current layer have no child nodes.

[0118] In some embodiments, when there are multiple items that constrain item i, among the multiple items that constrain item i (e.g., k, h, j), item j that is sorted last in the first sorting result (i.e., item j that is the most difficult to pick) is selected as the parent node of item i.

[0119] Figure 4 A schematic diagram of determining a parent node according to some embodiments of the present disclosure is shown.

[0120] (1) Take the items in the first sorting result as candidate items. Traverse each candidate item i and determine whether all candidate items have been traversed. If not, go to step (2). Otherwise, end the algorithm.

[0121] (2) Determine whether there is an item with a constraint relationship with the candidate item i. If so, go to step (3); otherwise, go to step (5);

[0122] (3) Determine whether the item that has a constraint relationship with the candidate item i is in the first sorting result. If so, go to step (4); otherwise, go to step (5);

[0123] (4) Among the items that have a constraint relationship with candidate item i and are included in the first sorting result, select item j that is ranked lowest in the first sorting result as the parent node of candidate item i, and candidate item i as the child node of item j;

[0124] (5) Keep the original information of the candidate item, do not establish the parent-child node relationship of the candidate item, and proceed to the next candidate item.

[0125] In some embodiments, a structure tree is generated based on the first sorting result and the constraint relationship between the items, including: taking items without parent nodes as the first layer of the structure tree, starting from the first layer, sorting the items in the current layer according to the first sorting result, wherein the parent node is the item that constrains the item; determining the child nodes of the items in the current layer as items in the layer below the current layer, wherein the child nodes are items constrained by the item; and looping the steps of sorting the items in the current layer according to the first sorting result and determining the items in the next layer until the items in the current layer have no child nodes.

[0126] like Figure 3 As shown, a structure tree is built upwards from the first level. Assume that item 1 is the parent node of items 4, 3, and 2, and item 4 has a constraint on item 6. The first sorting result is item 6 > item 5 > item 4 > item 3 > item 2 > item 1. Items 1 and 5, which have no parent nodes, are placed on the first level. Items 4, 3, and 2, the children of item 1, are then placed on the second level. The nodes on the second level are sorted according to the order of the first sorting result, and item 6, the child of item 4, is placed on the third level.

[0127] Figure 5 A flowchart of generating a structure tree according to some embodiments of the present disclosure is shown.

[0128] like Figure 5 As shown, starting from the first layer of the tree, according to the "father-child" relationship obtained previously, the tree structure is constructed layer by layer to obtain a complete tree structure.

[0129] First, traverse the candidate items in the first sorting result to determine whether the candidate item has a parent node. If not, add the candidate item to the first-level node of the structure tree. If so, do not perform additional processing on the current candidate item and proceed to the next candidate item.

[0130] After finding all the first-level nodes, start from the first level of the structure tree and traverse each level from top to bottom to determine whether all items in the current level of the structure tree have child nodes. If so, add the child nodes of the current level item to the next level of the structure tree, maintaining the relative position relationship of the child nodes, until all the child nodes of the current level are found. If not, it means that the leaf nodes of the structure tree have been created, and the structure tree construction ends.

[0131] In some embodiments, the order of items within the same layer is retained, and multiple layers of the structure tree are merged based on the constraint relationship between the items to obtain a second sorting result, including: starting from the items in the last layer of the structure tree, while retaining the priority order of the items in the previous layer unchanged and the priority of the items in the current layer is less than the priority of its parent node, determining the position of the items in the current layer after merging them into the previous layer based on the first sorting result; and looping the step of determining the position of the items until the structure tree is merged into one layer.

[0132] For example, merge the structure tree starting from the last level upwards. Figure 3 First, merge item 6 into the second layer. This ensures that item 6 comes after item 4. Then, sort items 6, 3, and 2 according to the first sorting result. The merged second layer's order is 4, 6, 3, and 2. Then, merge the second layer into the first layer. Because item 1 is the last item in the first layer, to ensure that items 4, 6, 3, and 2 come after item 1, simply place them after item 1, leaving the other items in the first layer unchanged. Finally, the second sorting result, or the merged structure tree, is 5, 1, 4, 6, 3, and 2.

[0133] Figure 6 A flowchart of structure tree merging according to some embodiments of the present disclosure is shown.

[0134] like Figure 6 As shown, starting from the bottom layer of the structure tree, each layer is merged from bottom to top.

[0135] First, determine whether the top level of the structure tree has been reached. If so, end the merge. If not, determine whether all nodes in the current level have been merged into the previous level, that is, the items in the current level have been processed.

[0136] When determining whether all nodes in the current layer have been processed, if the result is yes, then move to the previous layer as the current layer. If the result is no, then for each candidate item in the layer, search for its parent node in the previous layer nodes and determine whether the parent node is at the end of the queue in the upper layer. If the parent node of the lower layer node is at the end of the upper layer, then insert it after the end; if the parent node of the lower layer node is in the middle of the upper layer, then the relative position of the parent node in the upper layer and other nodes remains unchanged, and the inserted child node is arranged in the appropriate position according to the first sorting result.

[0137] In some embodiments, the order of items within the same layer is retained, and multiple layers of the structure tree are merged based on the constraint relationship between the items to obtain a second sorting result, including: starting from the items in the first layer of the structure tree, while retaining the priority order of the items in the current layer unchanged and the priority of the items in the current layer is greater than the priority of its child nodes, based on the first sorting result, determining the position of the items in the next layer after merging them into the current layer; and looping the step of determining the position of the items until the structure tree is merged into one layer.

[0138] The top-down merging method is similar to the bottom-up merging method. For example, start merging the structure tree from the first level downwards. While ensuring that the order of item 5 and item 1 remains unchanged, merge items 4, 3, and 2 into item 1 on the first level. The order of the first level becomes items 5, 1, 4, 3, and 2. Then merge item 4's child node (i.e., item 6) into the first level. Item 6 should be after item 4, and the order of items 6, 3, and 2 should be consistent with the first ordering result. Ultimately, the second ordering result, that is, the result of the merged structure tree, is still items 5, 1, 4, 6, 3, and 2, just like merging the structure tree from the bottom up.

[0139] In step S5, the order of picking the items to be picked is determined according to the order of the second sorting result.

[0140] For example, the second sorting result is used as the final picking order and sent to the robot arm control system, so that the robot arm picks according to the second sorting result.

[0141] This method initially sorts items based on their difficulty level, producing a first sorting result. It then constructs a structure tree and rearranges the initial sorting result based on the constraints between items, producing a second sorting result. While satisfying the constraints between items on the picking order, it also considers the difficulty level of item picking. This method not only addresses the issue of overlapping items impacting picking and causing damage, but also ensures that easier-to-pick items are picked first, improving picking efficiency and success rate, and reducing item loss.

[0142] Figure 7 A block diagram of an item picking device according to some embodiments of the present disclosure is shown.

[0143] like Figure 7 As shown, the item picking device 7 includes an acquisition module 71 , a first generation module 72 , a calculation module 73 , a second generation module 74 , and a determination module 75 .

[0144] The acquisition module 71 is configured to acquire the picking information of multiple items in the container, for example, Figure 1 Step S1 shown;

[0145] The first generating module 72 is configured to generate a first sorting result according to the picking information, wherein the first sorting result indicates the difficulty of picking the items, for example, Figure 1 Step S2 shown;

[0146] The calculation module 73 is configured to calculate the constraint relationship between items based on the picking information, for example, Figure 1 Step S3 shown;

[0147] The second generating module 74 is configured to generate a second sorting result based on the first sorting result and the constraint relationship between the items, wherein the second sorting result represents the picking priority of the items, for example, Figure 1 Step S4 shown;

[0148] The determination module 75 is configured to determine the order of picking the items to be picked according to the order of the second sorting result, for example, Figure 1 Step S5 is shown.

[0149] Figure 8 A block diagram illustrating an item picking device according to some other embodiments of the present disclosure is shown.

[0150] like Figure 8 As shown, the item picking device 8 includes a memory 81 and a processor 82 coupled to the memory 81. The memory 81 is used to store instructions for executing the item picking method. The processor 82 is configured to execute the item picking method of any of the embodiments of the present disclosure based on the instructions stored in the memory 81.

[0151] Figure 9 A block diagram of a computer system for implementing some embodiments of the present disclosure is shown.

[0152] like Figure 9 As shown, computer system 90 may be embodied as a general-purpose computing device. Computer system 90 includes memory 910, processor 920, and bus 900 that connects various system components.

[0153] The memory 910 may include, for example, system memory, non-volatile storage media, and the like. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions for executing the item picking method described in any of the embodiments of the present disclosure. Non-volatile storage media include, but are not limited to, disk storage, optical storage, and flash memory.

[0154] The processor 920 can be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, or as discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the judgment module and the determination module, can be implemented by a central processing unit (CPU) executing instructions in a memory that execute corresponding steps, or by dedicated circuits that execute corresponding steps.

[0155] The bus 900 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0156] The computer system 90 may also include an input / output interface 930, a network interface 940, a storage interface 950, and the like. These interfaces 930, 940, and 950, as well as the memory 910 and the processor 920, may be connected via a bus 900. The input / output interface 930 provides a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 940 provides a connection interface for various networked devices. The storage interface 950 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0157] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0158] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0159] These computer-readable program instructions can also be readable and stored in a computer-readable memory. These instructions cause the computer to work in a specific manner, thereby producing an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowcharts and / or block diagrams.

[0160] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0161] The item picking method and device, and the computer-readable storage medium in the above-mentioned embodiments improve the efficiency of item picking.

[0162] The article picking method and apparatus, as well as the computer-readable storage medium, according to the present disclosure have been described in detail. To avoid obscuring the present disclosure, some details known in the art have been omitted. Based on the above description, those skilled in the art will readily understand how to implement the disclosed technical solutions.

Claims

1. An item picking method, comprising: Get the picking information of multiple items in a container; Generating a first sorting result according to the picking information, wherein the first sorting result indicates the difficulty level of item picking; Calculate the constraint relationship between items based on the picking information; Generating a second sorting result based on the first sorting result and the constraint relationship between the items, including: generating a structure tree based on the first sorting result and the constraint relationship between the items, wherein each node in the structure tree corresponds to an item, the priority sorting of all items in the same layer conforms to the first sorting result, and a constraint relationship exists between the item and at least one item in the layer above it; while retaining the sorting of items in the same layer, merging multiple layers of the structure tree into one layer based on the constraint relationship between the items and a preset merging rule to obtain a second sorting result, wherein the second sorting result represents the picking priority of the items; The picking order of the items to be picked is determined according to the order of the second sorting results.

2. The article picking method according to claim 1, wherein: Generating a structure tree based on the first sorting result and the constraint relationship between the items includes: Set the items without parent nodes as the first level of the structure tree. Starting from the first level, sort the items in the current level according to the first sorting result, where the parent node is the item that constrains the item. Determine the child nodes of the item in the current layer as the items in the layer below the current layer, where the child nodes are the items constrained by the item; The steps of sorting the items in the current layer according to the first sorting result and determining the items in the next layer are repeated until the items in the current layer have no child nodes.

3. The article picking method according to claim 1, wherein: The step of merging multiple layers of the structure tree into one layer based on the constraints between the items and a preset merging rule while preserving the order of the items in the same layer to obtain a second sorting result includes: starting from the items in the last layer of the structure tree, determining the position of the items in the current layer after merging them into the previous layer based on the first sorting result while preserving the order of the items in the previous layer and providing that the priority of the items in the current layer is lower than the priority of its parent node; The steps of determining the location of items are repeated until the structure tree is merged into one layer.

4. The article picking method according to claim 1, wherein: The step of merging multiple layers of the structure tree into one layer based on the constraints between the items and a preset merging rule while preserving the order of the items in the same layer to obtain a second sorting result includes: starting from the items in the first layer of the structure tree, determining the position of the items in the next layer after merging them into the current layer based on the first sorting result while preserving the order of the items in the current layer and providing that the priority of the items in the current layer is greater than the priority of its child nodes; The steps of determining the location of items are repeated until the structure tree is merged into one layer.

5. The article picking method according to claim 1, wherein: The picking information includes: An image of the bin the item is in, the item's pick point, and the pick face.

6. The article picking method according to claim 5, wherein: Generating a first sorting result according to the picking information includes: Based on the picking information, calculate the influencing factors reflecting the difficulty of picking items; A first ranking result is generated according to the impact factor.

7. The article picking method according to claim 6, wherein: Generating a first ranking result according to the impact factor includes: A first ranking result is generated based on the top N items ranked by impact factor, where N is a positive integer.

8. The article picking method according to claim 6, wherein: The influencing factors include at least one of the area of ​​the picking surface, the area of ​​the largest inscribed circle of the picking surface with the picking point as the center, the angle between the normal vector of the picking point and the normal vector of the bottom surface of the cargo box, the distance from the picking point to each surface of the cargo box, and the distance from the picking point to the center of the cargo box.

9. The article picking method according to claim 6, wherein: The influencing factor includes the probability that the item is an item to be picked. The calculation of the influencing factor for item picking based on the picking information includes: Based on the image of the container where the item is located, a machine learning model is used to calculate the probability that the item is an item to be picked.

10. The article picking method according to claim 1, further comprising: Before generating the first sorting result, items to be sorted are screened out based on the picking information and preset filtering conditions.

11. The article picking method according to claim 1, wherein: The constraint relationship between the items reflects the stacking relationship between the items.

12. An article picking device, comprising: an acquisition module configured to acquire picking information of a plurality of items in a cargo box; A first generating module is configured to generate a first sorting result based on the picking information, wherein the first sorting result indicates the difficulty level of item picking; a calculation module configured to calculate constraint relationships between items based on the picking information; The second generation module is configured to generate a second sorting result based on the first sorting result and the constraint relationship between the items, including: generating a structure tree based on the first sorting result and the constraint relationship between the items, wherein each node in the structure tree corresponds to an item, the priority ranking of all items in the same layer conforms to the first sorting result, and there is a constraint relationship between the item and at least one item in the layer above it; while retaining the order of the items in the same layer, merging multiple layers of the structure tree into one layer based on the constraint relationship between the items and a preset merging rule to obtain a second sorting result, wherein the second sorting result represents the picking priority of the items; The determination module is configured to determine the order of picking the items to be picked according to the order of the second sorting result.

13. An article picking device comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the item picking method according to any one of claims 1 to 11 based on instructions stored in the memory.

14. A computer storable medium having computer program instructions stored thereon, wherein when the instructions are executed by a processor, the method for picking items according to any one of claims 1 to 11 is implemented.

Images