Removal sequence decision-making method for sheltering objects in openable container scene

Abstract

The invention belongs to the technical field of robot operation, and discloses a removal sequence decision-making method for shelters in an openable container scene, which comprises the following steps of: firstly, acquiring position information, size, accessibility and sheltering relationship of each object in the scene, then, representing the objects as nodes in a graph, and connecting the sheltering relationships among the objects through edges; defining the utility of each object according to the accessibility and visibility of the object, and optimizing the utility of the object by evaluating the shielding volume of each object under the condition of the shielding area; and finally, optimizing the removal sequence of the articles by using an improved greedy algorithm, and finally, ensuring that the operation time of the removal sequence of the articles is minimized through global optimization. According to the method, the problem of non-optimal solution under the shielding condition is solved, and objects with constraint relations in a scene form a sub-graph to realize global optimization of a removal sequence by constructing a spatial relation graph between the objects, calculating the shielding volume, introducing an improved greedy strategy and defining group utility.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot operation, and particularly relates to a method for making a decision on the removal order of obstacles in an openable container scenario. Background Art

[0002] In robot operation and intelligent warehousing systems, optimizing the item removal order is a key issue. The current mainstream methods mainly include:

[0003] 1. Rule-based methods: Remove items according to fixed preset rules (such as in the order of item size, weight, or position), but lack flexibility.

[0004] 2. Removal strategies based on greedy algorithms: Greedy algorithms are often used to quickly make decisions on the item removal order. However, since they only focus on local optimality, they may lead to an increase in the overall operation complexity.

[0005] 3. Decision-making methods based on reinforcement learning: Use deep reinforcement learning to train the robot to autonomously make decisions on the removal order, which can achieve good results in complex environments, but have high training costs and limited generalization ability.

[0006] Currently, the technical solution closest to the present invention is an item removal decision method based on the standard greedy algorithm, and its typical steps are as follows:

[0007] 1. Obtain item information: Obtain information such as the size, position, and occlusion relationship of items in the scene through an RGB-D camera or sensor.

[0008] 2. Greedily select the item to be removed: Each time, select the item that is easiest to remove in the current state (such as the topmost item or the item with the smallest volume).

[0009] 3. Update the scene state: After removing the item, recalculate the removability of the remaining items and repeat the greedy selection.

[0010] This solution has the following defects:

[0011] Leading to a non-optimal solution in the case of occlusion: When the grasping path of some objects in the scene is blocked by other objects, or there is a spatial area that is blocked by multiple objects at the same time, the greedy algorithm will result in a sub-optimal grasping order. Summary of the Invention

[0012] The purpose of the present invention is to provide a method for making a decision on the removal order of obstacles in an openable container scenario to solve the above technical problems.

[0013] To solve the above technical problems, the specific technical solution of a method for making a decision on the removal order of obstacles in an openable container scenario of the present invention is as follows:

[0014] A method for determining the removal order of occluders in an openable container scenario, comprising the following steps:

[0015] Step 1: First, obtain the position information, size, accessibility, and occlusion relationships of each object in the scenario. Then, represent these objects as nodes in a graph, and connect the occlusion relationships between objects with edges;

[0016] Step 2: Define the utility of each object according to the accessibility and visibility of the object. In the case of an occlusion area, optimize the utility of the object by evaluating the occlusion volume of each object;

[0017] Step 3: Use an improved greedy algorithm to optimize the removal order of items. First, identify the subgraphs between objects, that is, the set of objects with constraint relationships, and then use the greedy algorithm to determine the removal order within each subgraph; finally, through global optimization, ensure that the removal order of items minimizes the operation time.

[0018] Further, the Step 1 includes the following steps:

[0019] The occlusion space volume is defined as: from the perspective of the camera, the volume of the scene space occluded by an object in the scenario. Based on this definition, the probability that the target object appears after removing object A from the scene is expressed as:

[0020]

[0021] where V A is the occlusion space volume of object A, and V Oseen is the sum of the occlusion space volumes of all visible objects in the scene; the robot operates on the visible objects in the scene one by one and calculates the current optimal grasping object, so as to dynamically calculate the optimal removal object during the dynamic operation process.

[0022] Further, the Step 2 includes the following steps:

[0023] Define accessibility as: if the end effector must remove object A to access object B, then object A constrains the accessibility of object B, and object B is inaccessible;

[0024] Any arrangement of objects in the scene must satisfy the accessibility constraints of the objects;

[0025] After obtaining the visible objects and their poses, use a motion planning tool to identify the accessibility constraints in the scene, that is, the volume of the space covered by the execution trajectory of the object. If the execution volume of an object penetrates the execution volume of other objects, then the object is inaccessible; if the execution volume of an object penetrates the occlusion space of other objects, the object is also inaccessible; the time required to grasp a specific object is obtained by estimating the time required for the robot to execute the grasping trajectory.

[0026] Furthermore, step 2 includes the following steps:

[0027] Based on the greedy idea, define the utility of object A as:

[0028]

[0029] Sort according to the utility of the accessible objects in the scene and remove the object with the highest utility, which will generate a new scene. Iteratively update until all objects are removed;

[0030] When all objects in the scene are directly reachable and there is no space blocked by multiple visible objects at the same time, the grasping order planning method based on the greedy idea can find the target object in the shortest time. Furthermore, when the grasping paths of some objects in the scene are blocked by other objects, or there is a space area blocked by multiple objects at the same time, let the volume of the jointly blocked space be V joint , all objects are directly graspable, V C < V A + V joint < V B + V joint , V A , V B , V C are the volumes of the spaces blocked by objects A, B, and C. The time for the robot to move objects A, B, and C is similar. Since objects A and B jointly block V joint space, resulting in 2 * V C > V A ∪ V B , since the space jointly blocked by objects A and B needs to be processed twice, it is a better removal plan to remove object C first and then objects A and B.

[0031] Furthermore, step 3 represents the visibility constraint and reachability constraint of the objects in the scene as a graph. Each graph node corresponds to an object in the scene. When the following situations occur, there is an edge between node A and node B:

[0032] (1) The removal path of object B is blocked by object A, and vice versa;

[0033] (2) Objects A and B jointly block a non-empty space;

[0034] The objects with constraint relationships in the scene form a subgraph. The objects in a certain subgraph do not affect the utility of the objects in other subgraphs. Independently determine the removal order of the objects in each subgraph, and then merge these removal orders to obtain the complete removal order;

[0035] Consider the group utility of subgraphs to determine the order of subgraph removal. Define the group utility: The utility of a set of objects o in a subgraph is defined as

[0036] where T o is the sum of the removal times of the objects in the subgraph; V o is the spatial volume jointly occluded by this set of objects;

[0037] Use the greedy algorithm to determine the removal order of each subgraph. The removal order of the objects in the subgraph is determined according to the reachability constraint and the visibility constraint.

[0038] The method for determining the removal order of occluders in an openable container scenario of the present invention has the following advantages:

[0039] Optimal removal order in case of occlusion: The improved greedy algorithm can consider the global impact in the whole operation process when removing items each time, avoiding the local optimal problem.

[0040] Improve operation efficiency: By optimizing the item removal order, redundant steps of the operation are reduced, and the overall removal efficiency is improved.

[0041] The present invention solves the problem of non-optimal solutions in case of occlusion: By constructing a spatial relationship graph between items, calculating the occlusion volume, introducing an improved greedy strategy, defining the group utility, and forming subgraphs for the objects with constraint relationships in the scenario to achieve global optimization of the removal order. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram for exploring the occluded space in the scenario;

[0043] Figure 2 It is a schematic diagram of the optimal removal order under reachability constraints;

[0044] Figure 3 It is a schematic diagram of the optimal removal order under reachability constraints;

[0045] Figure 4 It is a schematic diagram of a composite scenario with multiple occlusions;

[0046] Figure 5 It is a schematic diagram of scenario construction;

[0047] Figure 6 It is a schematic diagram of a search scenario with 6, 8, 10, 12 visible items;

[0048] Figure 7 It is a schematic diagram of a target item search scenario;

[0049] Figure 8 It is a schematic diagram of the removal order of the greedy search method;

[0050] Figure 9 Schematic diagram of the removal order for the comprehensive search method;

[0051] Figure 10 Schematic diagram of the performance of various methods when CubiodT1 is the search object;

[0052] Figure 11 Schematic diagram of the performance of various methods when CubiodT2 is the search object;

[0053] Figure 12 Schematic diagram comparing a group of comprehensive search methods and the greedy search method. Detailed implementation manner

[0054] To better understand the purpose, structure, and function of the present invention, the following further describes in detail a method for making a decision on the removal order of occluders in an openable container scenario of the present invention with reference to the accompanying drawings.

[0055] The target object search scenario consists of the visible object O seen and the space occluded by the visible object. The robot searches for the target object by removing the visible object O seen in the scenario until the target object is visible or all objects in the scenario are removed. The target object search problem can be defined as: Given the geometric information and pose information of the visible objects, find an optimal removal order to make the target object in the scenario visible as quickly as possible.

[0056] Assume that the target object to be searched is unique. When the target object is not unique, the method of the present invention can still be used to solve the problem, and the only difference lies in the number of items to be searched. Define the removal order of the visible objects as a permutation A o : {1,..., |o|} → o, where A o (i) is the i-th object to be removed. Given the permutation A o for searching the target object, the execution cost of searching the target object is expressed as:

[0057]

[0058] where represents the probability that the target object appears in the field of view after removing object i; is the time taken to remove object i.

[0059] The optimization goal is to find a removal order that minimizes , that is, to expose the target object in the shortest time. To this end, it is necessary to consider the increase in the visible space in the scenario after removing the visible object, and whether removing a certain visible object will affect other objects.

[0060] A method for determining the removal order of occluders in an openable container scenario of the present invention includes the following steps:

[0061] Step 1: First, obtain the position information, size, accessibility, and occlusion relationship of each object in the scenario. Then, represent these objects as nodes in a graph, and connect the occlusion relationships between objects with edges.

[0062] After removing a visible object in the scenario, the space it occludes is exposed. Therefore, the volume of the occluded space can be used as a metric to measure the occlusion ability of an object on the scenario. When the occlusion spaces of two objects overlap, the influence of the overlapping area needs to be considered. As Figure 1 shown in (a) of, objects A and B jointly occlude a part of the space. When only object A or object B is removed, the jointly occluded space remains occluded.

[0063] After removing a certain object, the visible information of other objects in the space will also change. When the shadow heights of object A and object B are inconsistent in the jointly occluded area, the jointly occluded space will be assigned to the object with the higher shadow height. As Figure 1 shown in (a) of, if the shadow volume of object B is higher, then the volume V of the space it occludes B includes the jointly occluded space. After removing object B, the space occluded by object B is released, and the occlusion volume V of object A A includes the jointly occluded space, as Figure 1 shown in (b) of. Conversely, more complete structural information of object A can be obtained, and the jointly occluded space is the occlusion space of object A.

[0064] The mutual occlusion between objects is related to the shape of the objects, the position of the viewpoint, the viewing angle, etc. The occlusion space volume is defined as: the volume of the scene space occluded by an object in the scene as observed from the camera viewpoint. Based on this definition, the probability of the target object appearing after removing object A in the scene can be expressed as:

[0065]

[0066] where V A is the occlusion space volume of object A, and V Oseen is the sum of the occlusion space volumes of all visible objects in the scene.

[0067] The robot operates on the visible objects in the scene one by one and calculates the current optimal grasping object, so as to dynamically calculate the optimal removal object during the dynamic operation process.

[0068] Step 2: Define the utility of each object according to the accessibility and visibility of the object. In the case of the occlusion area, optimize the utility of the object by evaluating the occlusion volume of each object.

[0069] When a robot uses an end effector to remove a visible object in a scene, there should be no other objects on the removal path; otherwise, it will collide with other objects. Define reachability as follows: If the end effector must remove object A to access object B, then object A constrains the reachability of object B, and object B is unreachable.

[0070] Any arrangement of objects in the scene must satisfy the reachability constraints of the objects. For example, Figure 2 As shown, objects A, B, C, and D are all within the visible range of the robot, but the removal path of object B is blocked by object A. Therefore, object B cannot be a direct operation object.

[0071] After obtaining the visible objects and their poses, motion planning tools can be used to identify the reachability constraints in the scene, that is, the spatial volume covered by the execution trajectories of the objects. If the execution volume of an object penetrates the execution volume of other objects, then the object is unreachable. If the execution volume of an object penetrates the occlusion space of other objects, the object is also unreachable.

[0072] The time required to grasp a specific object can be obtained by estimating the time required for the robot to execute the grasping trajectory. Since the action of grasping object A is constant and also constant in a given scene, it does not depend on its removal order.

[0073] The faster the robot shows a large occlusion volume, the faster it can find the target. Therefore, based on the greedy idea, define the utility of object A as:

[0074]

[0075] Sort according to the utility of the accessible objects in the scene and remove the object with the highest utility. This will generate a new scene, which is iteratively updated until all objects are removed.

[0076] Step 3: Optimize the removal order of the items using the improved greedy algorithm. First, identify the subgraphs between objects (i.e., the set of objects with constraint relationships), and then use the greedy algorithm to determine the removal order within each subgraph.

[0077] When all objects in the scene are directly reachable and there is no space occluded by multiple visible objects at the same time, the grasping order planning method based on the greedy idea can find the target object in the shortest time.

[0078] However, when the grasping paths of some objects in the scene are blocked by other objects, or there are spatial regions occluded by multiple objects at the same time, the greedy algorithm will result in a suboptimal grasping order. For example, Figure 2As shown, since the movement path of object B is blocked by object A, object B cannot be the direct operation object. The operable objects in the scene are object A, object C, and object D. The time for the robot to move object A and object D is close. Since V D > V A , U D > U A , the greedy algorithm first moves object D, and the final movement order is D → A → B → C. However, moving object A first is the best because it will more quickly display the spatial volume blocked by object B, and the best movement order is A → B → D → C. When adding other objects similar to object A to the scene, the greedy algorithm will produce more suboptimal results.

[0079] Multiple objects occluding the same space lead to suboptimal results. As shown in the scene of (a) in Figure 3 , let the spatial volume of the combined occlusion be V joint , all objects are directly graspable, V C < V A + V joint < V B + V joint , V A , V B , V C are the spatial volumes occluded by objects A, B, and C. The time for the robot to move objects A, B, and C is close. Since objects A and B jointly occlude the V joint space, resulting in 2 * V C > V A ∪ V B , the greedy algorithm selects to move objects A and B first. However, since the space jointly occluded by objects A and B needs to be processed twice, it is a more optimal removal plan to remove object C first and then objects A and B. Figure 3 The scene after removing object A is shown in (b) of

[0080] In the actual object search scene, it is common for any number of objects to jointly occlude a certain spatial volume. The present invention represents the visibility constraint and reachability constraint of the objects in the scene as a graph. Each graph node corresponds to an object in the scene, and there is an edge between node A and node B when the following situations occur:

[0081] (1) The removal path of object B is blocked by object A, and vice versa;

[0082] (2) Objects A and B jointly occlude a non-empty space;

[0083] The objects in the scene with constraint relationships form a subgraph, as shown in Figure 4As shown, there are three subgraphs of constraint relationships in the scene: {A, B, C}, {D}, and {E, F}.

[0084] The objects in a certain subgraph do not affect the utility of the objects in other subgraphs. Therefore, the removal order of the objects in each subgraph can be determined independently, and then these removal orders are combined to obtain the complete removal order.

[0085] Similarly, considering the group utility of the subgraphs to determine the order of removal of the subgraphs. Define the group utility: The utility of a set of objects o in a subgraph is defined as

[0086] where T o is the sum of the removal times of the objects in the subgraph; V o is the spatial volume jointly occluded by this set of objects.

[0087] Use the greedy algorithm to determine the removal order of each subgraph, and the removal order of the objects in the subgraph is determined according to the reachability constraint and the visibility constraint. For example, Figure 4 in the subgraph {A, B, C}, the removal order of the objects is C → B → A.

[0088] It can be verified that for Figure 2 and Figure 3 the two occlusion scenarios shown, the method can expose a larger occluded space in a shorter time.

[0089] Finally, through global optimization, ensure that the removal order of the items minimizes the operation time and improves the removal efficiency.

[0090] Based on the RGB-D information collected by the robot, the target object search process is as follows:

[0091] (1) Input the RGB-D information, extract the three-dimensional point cloud of each visible object and fit the bounding box;

[0092] (2) Implement the accommodability analysis, judge the possible positions of the target object in the scene, and exclude the visible objects that cannot accommodate the target object;

[0093] (3) Use the reachability constraint conditions and the visibility constraint conditions to construct subgraphs, calculate the group utility of each subgraph, and use the greedy idea to determine the removal order of the nodes of each subgraph;

[0094] (4) Use the reachability constraint conditions and the greedy search method for a single object to determine the execution order of the objects in the subgraph;

[0095] (5) Combine the execution orders of all subgraphs to obtain the final execution order.

[0096] Effectiveness evaluation experiment of the method:

[0097] 1. Experimental environment setup

[0098] The algorithm libraries used in the experiment include Open3D, OpenCV, Shapely, Scipy, OpenMesh, etc. Based on the CoppeliaSim (formerly V-REP) simulator, a virtual environment for robotic arm grasping is built, and a combination of a six-degree-of-freedom robotic arm UR5 and a two-finger gripper RG2 is used as the grasping actuator. To ensure that the grasping effect in the simulation environment is close to that in the real environment, a virtual experimental scene is constructed using the three-dimensional dimensions of several common items and a certain actual openable container, as Figure 5 shown.

[0099] The three-dimensional models of the items to be grasped are approximately constructed using geometric bodies. It includes 3 visible items approximated by cuboids with dimensions (length, width, height) of Cubiod1 (4.4 cm, 5 cm, 10.2 cm), Cubiod2 (6.2 cm, 4 cm, 10.5 cm), and Cubiod3 (6.4 cm, 6.4 cm, 18 cm) respectively. 3 items approximated by cylinders with dimensions (radius, height) of Cylinder1 (3.2 cm, 14.6 cm), Cylinder2 (3.3 cm, 16.5 cm), and Cylinder3 (2.8 cm, 8.5 cm) respectively. Two cuboids representing the target items with dimensions of CubiodT1 (3 cm, 2 cm, 8 cm) and CubiodT2 (4.4 cm, 5 cm, 10.2 cm).

[0100] The dimensions of the used openable container model are l w = 76 cm, l d = 36 cm, l h = 37 cm. The reachable space of the robotic arm is determined by sampling in its joint space, and the relative pose is adjusted to ensure that the placement positions of the items inside the openable container are all within the reachable range of the robotic arm. After constructing the virtual scene, the RRT-Connect algorithm is used to plan the grasping path of the object.

[0101] 2. Experimental results and analysis

[0102] To illustrate the effectiveness of the constructed method, two baseline methods are constructed. The three methods of the experiment are as follows:

[0103] (1) Random removal method: Remove the movable objects in the scene in a random order.

[0104] (2) Greedy search method: A method that preferentially removes the movable object with the maximum utility.

[0105] (3) Comprehensive search method: Use the acceptability analysis method to determine the potential positions of the target objects, and then use the improved greedy method to determine the removal order of the objects to be removed.

[0106] To evaluate the impact of the size of the item to be searched on the search efficiency, CubiodT1 and CubiodT2 are used as the target items to be searched respectively. CubiodT1 can be completely blocked by visible items of other six sizes, and CubiodT2 can be completely blocked by Cubiod3, Cylinder1, and Cylinder2, and is partially blocked by the other three visible items.

[0107] To evaluate the effectiveness of the method in the presence of different numbers of visible objects, 4 groups of experimental scenarios with 6, 8, 10, and 12 visible objects are set up. Each group of experimental scenarios contains 30 specified numbers of visible object placement situations. Except for containing one of the target items, the types and numbers of the remaining items in each group of scenarios are randomly selected from the aforementioned item models and randomly placed in the scenario in a non-overlapping form in the vertical direction.

[0108] Different numbers of visible objects mean that the degree and type of mutual occlusion between items are significantly different. In the scenario constructed with 6 items, there are no mutually occluding items or only two items are mutually occluding in 90% of the scenarios. In the scenario constructed with 12 items, there are multiple items mutually occluding in each scenario. Therefore, different numbers of visible items are used to evaluate the practicality of the method in scenarios with different occlusion degrees. Figure 6 Show one example of each of the four scenarios, and the target item to be searched is within the red circle.

[0109] Figure 7 For a search target item scenario, the target item is located behind item D with the maximum utility (Utility, U), but the removal path of item D is blocked by item E and cannot be directly removed.

[0110] The greedy search method compares the utilities of the items that can be directly removed to determine the current removal object. After item E is removed, item D is added to the sequence of removable objects and is selected in subsequent steps. The removal process based on the greedy search method is as Figure 8 shown, and the item removed in the current stage is within the red frame.

[0111] Figure 9 Shows the removal order of the comprehensive method of the present invention. All items here may block the target item. Item E blocks item D, and the group utility is used to determine the removal order of these two items. The utility of item E is small, but the utility of item D is large, and their collective utility is also greater than the utility of other items. Therefore, they are processed first.

[0112] The effectiveness of the evaluation method is jointly evaluated using the number of grasps and the total grasping time. The pinhole camera model is used to simulate the visibility of visible objects in the scene. When the target object in the scene can be obtained through a single grasping operation, the target object is considered visible.

[0113] The experimental results are as Figure 10 and Figure 11 shown, from which the following conclusions can be drawn:

[0114] (1) As the number of visible items increases, the performance of the greedy search method gradually decreases. In the search scenario of CubiodT1, when there are only 6 visible items, the number of searches of the greedy search method is close to that of the comprehensive search method; when there are 12 items, the number of times required by the greedy search method is 0.44 times more than that of the comprehensive search method. The data in the search scenario of CubiodT2 also supports this conclusion. Through the analysis of relevant scenarios, it is found that the greedy search method first removes the movable objects with a large occlusion space volume near the camera in the workspace. In many scenarios, it is superior to the random search method. However, when the object with a large occlusion volume near the camera is occluded and immovable, the greedy search method does not first consider removing the relevant object, resulting in poor performance. The method can balance between the occlusion space volume and the number of visible objects with a large occlusion space volume, and remove the visible objects with a large occlusion volume through multiple operations. In the scenario with 12 visible items, the improvement effect of the comprehensive search method is the most obvious, with 1.51 fewer searches than the random search method.

[0115] (2) When the object to be searched is larger, the comprehensive search method has better results. From the data in Figure 10 and Figure 11 , it can be seen that when the object to be searched is larger, the number of searches required in different occlusion degree scenarios is less than that in the scenario where the object to be searched is smaller. The reason for the excellent performance is that when the object to be searched is larger, the visibility constraint method is more effective in excluding the visible objects that cannot occlude the target object.

[0116] Figure 12 shows a set of scenarios where the comprehensive search method is superior to the greedy search method. Item T is the target item to be searched, and the rest are visible items. The table provides images of the camera view and the top-down view of the scene, and shows the search order of the greedy search strategy and the comprehensive search strategy.

[0117] The experiment proves that the comprehensive search method of the present invention can reduce the number of searches. In the scenario with the best performance, the comprehensive search method can search 1.51 times less. When the target object to be searched is larger and the occlusion between objects in the scene is more serious, the comprehensive search method has better search results.

[0118] It will be understood that the present invention is described by way of some embodiments, and those skilled in the art will know that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Additionally, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A method for making a decision on the removal order of an occluder in an openable container scenario, characterized in that, It includes the following steps: Step 1: First, obtain the position information, size, accessibility, and occlusion relationship of each object in the scene. Then, represent these objects as nodes in a graph, and connect the occlusion relationships between objects with edges; Step 2: Define the utility of each object according to the accessibility and visibility of the object. In the case of an occlusion area, optimize the utility of the object by evaluating the occlusion volume of each object; Step 3: Use the improved greedy algorithm to optimize the removal order of the items. First, identify the subgraphs between objects, that is, the set of objects with constraint relationships between them. Then, use the greedy algorithm inside each subgraph to determine the removal order; finally, through global optimization, ensure that the removal order of the items minimizes the operation time.

2. The method for making a decision on the removal order of the occluder in the openable container scenario according to claim 1, wherein, The said Step 1 includes the following steps: The occlusion space volume is defined as: the volume of the scene space occluded by an object in the scene when viewed from the camera's perspective. Based on this definition, the probability that the target object appears after removing object A from the scene is expressed as: where V A is the occlusion space volume of object A, and V Oseen is the sum of the occlusion space volumes of all visible objects in the scene; the robot operates on the visible objects in the scene one by one and calculates the current optimal grasping object, so as to dynamically calculate the optimal removal object during the dynamic operation process.

3. The method for making a decision on the removal order of the occluder in the openable container scenario according to claim 1, wherein, The said Step 2 includes the following steps: Define accessibility as: If the end effector must remove object A to access object B, then object A constrains the accessibility of object B, and object B is inaccessible; Any arrangement of objects in the scene must satisfy the accessibility constraints of the objects; After obtaining the visible objects and their poses, use the motion planning tool to identify the accessibility constraints in the scene, that is, the volume of space covered by the execution trajectory of the object. If the execution volume of an object penetrates the execution volume of other objects, then this object is inaccessible; If the execution volume of an object penetrates the occlusion space of other objects, this object is also inaccessible; the time required to grasp a specific object is obtained by estimating the time required for the robot to execute the grasping trajectory.

4. The method for making a decision on the removal order of the occluder in the openable container scenario according to claim 1, wherein The said Step 2 includes the following steps: Based on the greedy idea, define the utility of object A as: Sort according to the utility of the accessible objects in the scene, and remove the object with the highest utility, which will generate a new scene. Iteratively update until all objects are removed; When all objects in the scene are directly accessible and there is no space simultaneously occluded by multiple visible objects, the grasping order planning method based on the greedy idea can find the target object in the shortest time.

5. The method for making a decision on the removal order of an occluder in an openable container scenario according to claim 1, wherein In step 3, when the grasping paths of some objects in the scene are blocked by other objects, or there is a spatial area blocked by multiple objects simultaneously, let the spatial volume of the combined occlusion be V joint , all objects are directly graspable, V C < V A + V joint < V B + V joint , V A , V B , V C are the spatial volumes occluded by objects A, B, and C. The robot moves objects A, B, and C in close proximity. Since objects A and B jointly occlude V joint space, resulting in 2 * V C > V A ∪ V B . Since the combined occlusion of objects A and B needs to be processed twice, it is a better removal plan to first remove object C and then remove objects A and B 6. The method for making a decision on the removal order of the occluder in the openable container scenario according to claim 1, wherein The said Step 3 represents the visibility constraints and accessibility constraints of the objects in the scene as a graph. Each graph node corresponds to an object in the scene. When the following situations occur, there is an edge between node A and node B: (1) The removal path of object B is blocked by object A, and vice versa; (2) Object A and object B jointly occlude a non-empty space; The objects in the scene with constraint relationships form subgraphs. The objects in a certain subgraph do not affect the utility of the objects in other subgraphs. Independently determine the removal order of the objects in each subgraph, and then merge these removal orders to obtain the complete removal order; Consider the group utility of subgraphs to determine the order of subgraph removal. Define the group utility: The utility of a set of objects \(o\) in a subgraph is defined as where \(T\) o is the sum of the removal times of the objects in the subgraph; \(V\) o is the spatial volume jointly occluded by this set of objects; Use the greedy algorithm to determine the removal order of each subgraph. The removal order of the objects in the subgraph is determined according to the accessibility constraints and visibility constraints.

Images