Robot grabbing control method based on stability risk assessment

Abstract

The invention relates to a robot grabbing control method based on stability risk assessment. The robot grabbing control method is suitable for object grabbing in a disordered stacking scene. The method comprises the following steps: acquiring scene point cloud in real time through an RGB-D camera, and preprocessing and segmenting the scene point cloud to obtain a point cloud cluster of each object; for each object, a longitudinal and transverse support relation graph is constructed, the longitudinal risk and the transverse risk of the object are calculated respectively, and grabbing stability risk parameters are obtained through coupling; the object with the lowest risk is selected as a grabbing target, and grabbing actions are planned and executed; and scene point cloud is updated after capturing and re-evaluation is carried out. By quantizing the influence of grabbing on the stacking stability, the stacking collapse risk in the grabbing process is effectively reduced, and the method is suitable for unstable environments such as a dynamic conveyor belt and can be cooperatively used with other grabbing evaluation parameters, so that the safety and reliability of a grabbing system are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of motion control based on image recognition, and particularly relates to a robot grasping control method based on stability risk assessment. BACKGROUND

[0002] In the automatic sorting scene of logistics, warehousing, etc., robots often face the task of grasping disordered stacked grasping objects from a conveyor belt or a shelf. The existing robot grasping decision method mainly focuses on the properties of the objects themselves, such as selecting the easiest-to-grasp object based on visibility or selecting the most stable grasping point based on grasping quality score. However, in the complex scene where objects are in contact, stacked or dependent on each other, the above methods have obvious limitations: they fail to fully consider the impact of grasping a certain object on the stability of its surrounding objects.

[0003] Directly grasping an object that plays a key supporting or constraining role in the stacked structure is extremely likely to cause the objects above it that depend on it to fall, or the objects around it that lose lateral support to slip or topple, triggering a "domino effect" of cascading collapse. This not only destroys the work scene and causes the sorting task to fail, but also may damage the grasping objects and endanger the safety of the equipment. Therefore, the existing technology lacks a grasping decision mechanism that can effectively predict and avoid such risks, making it difficult to achieve safe and reliable continuous automated operation in a dynamic and unstable stacked environment. SUMMARY

[0004] The application discloses a robot grasping control method based on stability risk assessment, which comprises the following steps: S1, dynamic point cloud acquisition and preprocessing: acquiring color images and depth images of a scene through an RGB-D camera to generate point cloud data; performing filtering and segmentation processing on the point cloud data to obtain a point cloud cluster set representing a plurality of grasping objects; S2, grasping stability risk calculation: for each grasping object in the set, calculating a grasping stability risk parameter L i , which comprises: a. constructing a longitudinal support directed graph to calculate a longitudinal risk L i vert ; b. constructing a lateral support relationship graph to calculate a lateral risk L i horz ; c. coupling the longitudinal risk L i vert and the lateral risk L i horz to obtain the grasping stability risk parameter L i ; S3, grasping decision and execution: based on the grasping stability risk parameter Li selecting a target grasping object from the set; calculating a grasping pose of the target grasping object, and controlling the robot to perform a grasping action; S4, scene update: removing the point cloud of the grasped object from the scene point cloud, and returning to step S1 to re-perform perception and calculation.

[0005] Specifically, in step S2a, the method of constructing the longitudinal support directed graph is: taking each grasping object as a node, and establishing a directed edge e ij v from node i to node j when and only when the bottom of grasping object j has vertical projection overlap with the top of i, and assigning a weight w ij v to the edge.

[0006] Specifically, the calculation formula of the longitudinal edge weight w is: ij contact where A k is the contact area of i and j, ∑ kj A contact is the sum of all support contact areas of j, C ij center is an adjustment factor based on the relative position of the contact center to the center of gravity of j.

[0007] Specifically, the calculation formula of the longitudinal risk in step S2a is: where S i is the set of all grasping objects reachable from node i in the longitudinal support directed graph, is the product of the weights of all longitudinal edges on the path from node i to node j, m j is the mass of grasping object j, m max is the maximum mass among all grasping objects, ΔR j is the loss of support redundancy of grasping object j after removing grasping object i, R j current is the current support redundancy of grasping object j.

[0008] Specifically, in step S2b, the method of constructing the lateral support relationship graph is: taking each grasping object as a node, and establishing an undirected edge e ij h between node i and node j when and only when the horizontal interval of grasping objects i and j is less than a threshold or there is side contact between their point clouds, and assigning a weight w ij h to the edge.

[0009] Specifically, the lateral edge weight w ij hThe calculation formula of w ij h = μ·F ij ·cos(θ ij ), wherein μ is an effective friction coefficient, F ij is the normal contact pressure between the grasped objects i and j due to the upper stacking load, θ ij is the angle between the contact surface normal vector and the horizontal plane.

[0010] Specifically, the calculation formula of the lateral risk L i horz in step S2b is: wherein N i is the set of grasped objects directly connected to node i in the lateral support relationship graph, I j critical is the criticality indication function of grasped object j, and ΔP j is the lateral constraint potential energy loss of grasped object j after removing grasped object i.

[0011] Specifically, the formula of the coupling in step S2c is: wherein α and β are weight coefficients, and γ is a coupling coefficient.

[0012] Specifically, in step S3, the grasped object with the lowest grasping stability risk parameter is selected as the target grasped object; the grasping direction of the grasping pose is determined based on the average normal vector of the top surface of the target grasped object, and the grasping position is determined based on the center point or the main direction of the point cloud of the top surface of the target grasped object.

[0013] Beneficial technical effects: By quantitatively calculating the longitudinal (loss of support) and lateral (loss of constraint) stability risks caused by grasping each object, the system can actively select the target with the smallest impact on the overall structural stability for grasping, thereby fundamentally avoiding the cascading failure caused by blind grasping, greatly improving the safety and reliability of the operation; the present application innovatively constructs a longitudinal support directed graph and a lateral support relationship graph, and converts the complex support and constraint relationship in the physical world into a calculable model. The method of the present application can accurately and quantitatively evaluate the "criticality" of each object in the stacking structure, providing a scientific and objective basis for grasping decision-making; the stability risk evaluation module of the present application can not only serve as an independent grasping strategy core, but also can be seamlessly combined with existing evaluation parameters based on visibility, grasping quality, etc. to form a more comprehensive and integrated grasping suitability evaluation system, which can be flexibly configured according to the emphasis of different application scenarios, and has wide applicability. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings illustrate the present application. Figure 1 is a schematic diagram of a robot grasping operation scene according to the present application.

[0015] Appendix Figure 2 This is a flowchart of a robot grasping control method based on stability risk assessment according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, the robot system of this invention is equipped with an RGB-D camera system, which can acquire scene point cloud data in real time. The robot has a 6-DOF robotic arm and a suitable gripper, such as a vacuum suction cup or a two-finger gripper. The object to be grasped can be placed in a box / cabinet, which can be placed on a conveyor belt, or the object to be grasped can be placed directly on the conveyor belt. The speed of the conveyor belt is known and allows for decision-making during movement.

[0018] like Figure 2 As shown, this invention discloses a robot grasping control method based on stability risk assessment, which includes: 1. Dynamic point cloud acquisition and preprocessing Color and depth images of the scene, including disordered stacked grasped objects, are acquired using an RGB-D camera to generate point cloud data. The point cloud data is then filtered to reduce noise, using techniques such as voxel mesh downsampling and statistical outlier removal. Geometric feature-based segmentation methods, such as region growing and Euclidean clustering, are used to segment the point cloud into different clusters, with each cluster representing a grasped object. The segmentation algorithm separates the point cloud of each grasped object, forming a set of grasped objects O = {o1, o2, ..., o...}. n}

[0019] 2. Calculation of crawling stability risk for crawled objects For each crawled object in set O, calculate its crawling stability risk L. i First, calculate the longitudinal risk L. i vert and horizontal risk L i horz Then, the stability risk parameter L is obtained through coupling. i The specific process is as follows.

[0020] (1) Construct a vertically supported directed graph, where the nodes are the crawling objects. A directed edge e from crawling object i to crawling object j is defined if and only if the bottom of crawling object j and the top of crawling object i have vertical projection overlap.ij v exists, edge weight w ij v represents the proportion of load transferred through the contact surface, and the calculation formula is: where A ij contact is the contact area of i and j, and ∑ k A kj contact is the sum of all support contact areas of j, C ij center is an adjustment factor based on the relative position of the contact center to the center of gravity of j, which can take a value of 0.8-1.2. The closer the contact center is to the center of gravity of j, the closer C ij center is to 1.2, indicating that the path bears more load.

[0021] (2) Build a lateral support relationship graph, where the nodes are each grasped object. There is an undirected edge e ij h between grasped objects i and j if and only if the horizontal gap between them is less than a threshold or there is actual side contact between their point clouds. Whether there is actual side contact can be identified through normal vector analysis. The edge weight w ij h quantifies the strength of lateral constraint, and the calculation formula is: w ij h = μ·F ij · cos(θ ij ). Where μ is the effective friction coefficient, which can be estimated according to material properties, F ij is the normal contact pressure between i and j due to the load of the stack above, which can be recursively estimated through the longitudinal force flow graph, θ ij is the angle between the normal vector of the contact surface and the horizontal plane, and the smaller θ ij is, the more perpendicular the contact surface is, and the stronger the lateral constraint it provides.

[0022] (3) Calculate the longitudinal risk L i vert : L i vert evaluates whether the grasped objects above i will fall due to the loss of support after i is removed, which is specifically where is the product of the weights of all longitudinal edges on the path from i to j, reflecting the support contribution of i to j, m j is the mass of grasped object j, and m max is the maximum mass among all grasped objects, S iis the dependent subtree, i.e. the set of graspable objects reachable from i in the vertical support digraph, i.e. the set of graspable objects directly or indirectly supported by i, AR j is the support redundancy loss, i.e. the support redundancy loss of graspable object j after removing graspable object i, used to evaluate S i for each graspable object j in S, the extent to which j's original support structure is disrupted after removing i. AR j The detailed calculation process is as follows: Calculate the current support redundancy of j: where K j is the set of all supporters of j, A j base is the base area of j, R j current The larger the value of R Calculate the support redundancy after removing i: ; Calculate the stability loss of j: .

[0023] (4) Calculate the lateral risk L i horz : L i horz Evaluate whether the surrounding graspable objects will slip or topple due to the loss of lateral constraints after removing i. Specifically: where N i is the set of lateral neighbors, i.e. the set of all graspable objects directly connected to i in the lateral support graph, I j critical is the criticality indicator function. If j itself is a "critical node" that is tall, high in center of gravity, or supports a large number of other graspable objects, then I j critical > 1, so that the risk of losing constraints on critical graspable objects can be fully displayed; ΔP j is the lateral constraint potential loss, i.e. the lateral constraint potential loss of graspable object j after removing graspable object i, which is calculated as follows: Calculate the current lateral constraint potential: i.e. the sum of the weights of all lateral constraints of j;

[0024] Calculate the lateral constraint potential after removing i: ; Calculate the lateral stability loss of j: .

[0025] (5) Coupling risk: where a and b are weight coefficients to adjust the importance of the two risks, and g is a coupling coefficient to capture the synergistic amplification effect when both risks exist simultaneously, i.e., a grasping object is both a load-bearing key point and a lateral constraint key point. The geometric mean ensures that the coupling term is significant only when both risks are high.

[0026] 3. Grasping decision and execution The grasping object with the lowest grasping stability risk is selected as the grasping target, and its grasping direction and grasping position are calculated. For example, the grasping direction can take the average normal vector of the top surface of the grasping object as the grasping direction, which can be perpendicular to the surface for a suction cup gripper. The grasping position can select the center point of the top surface, or the long axis direction calculated by the point cloud principal direction, adjusted according to the gripper. The robot motion planner plans a collision-free path for the gripper to move from the current position to the grasping pose, after which the robot performs the grasping action and removes the grasping object.

[0027] 4. Scene update The point cloud of the grasped object is removed from the scene point cloud. Since grasping can cause the stacked grasping objects to move or collapse, it is necessary to re-perceive the scene, i.e., return to step 1 and re-perform point cloud segmentation and parameter calculation. If the scene changes are not significant, only the affected area can be updated locally, but for safety, global update is usually recommended.

[0028] It should be noted that in addition to the above grasping stability risk, other parameters can be introduced to determine the grasping object, including but not limited to visibility and grasping quality related parameters. Stability risk assessment can be used independently or in conjunction with visibility, grasping quality and other parameters to form a comprehensive grasping suitability assessment system. For example, in a stability-sensitive scenario, the stability parameter is used first, in a simple stack, the visibility parameter is used quickly, or in a complex environment, a comprehensive assessment is used. The grasping stability risk of the present application can be used alone or in combination with other parameters to determine the most suitable grasping object.

[0029] In the several embodiments of the present application, it should be understood that the disclosed method can be implemented in other ways. For example, the embodiments of the application described above are merely illustrative. For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the essential characteristics of the present application. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application.

Claims

1. A robot grasping control method based on stability risk assessment, characterized by, The method comprises the following steps: S1, dynamic point cloud acquisition and preprocessing: acquiring color images and depth images of a scene, generating point cloud data; filtering and segmenting the point cloud data to obtain a point cloud cluster set representing a plurality of grasping objects; S2, calculate the grasping stability risk: for each grasping object in the set, calculate its grasping stability risk parameter L i , the calculation comprising: a. Constructing a longitudinal support digraph, computing longitudinal risk L i vert ; b. Construct a lateral support relationship graph, calculate the lateral risk L i horz ; c. coupling said longitudinal risk L i vert and lateral risk L i horz , obtaining said grasping stability risk parameter L i ; S3, grasping decision and execution: based on the grasping stability risk parameter L i selecting a target grasping object from the set; computing a grasping pose for the target grasping object and controlling the robot to perform a grasping action; S4, scene updating: removing the point cloud of the grasped object from the scene point cloud, and returning to step S1 to re-perceive and calculate.

2. The method of claim 1, wherein, In step S2a, the method of constructing the vertical support directed graph is: taking each grasped object as a node, and establishing a directed edge e from node i to node j if and only if the bottom of grasped object j and the top of i have vertical projection overlap ij v , and assigning a weight w to the edge ij v .

3. The method of claim 2, wherein, The calculation formula of the longitudinal edge weight is: where A ij contact is the contact area of i and j,∑ k A kj contact is the sum of all support contact areas of j,C ij center is an adjustment factor based on the relative position of the contact center to the j gravity center.

4. The method of claim 2, wherein, The formula for calculating the longitudinal risk in step S2a is: wherein S i is the set of grasped objects reachable from node i in the longitudinal support directed graph, is the product of the weights of all longitudinal edges on the path from node i to node j, m j is the mass of grasped object j, m max is the maximum mass among all grasped objects, ΔR j is the loss of support redundancy of grasped object j after removing grasped object i, R j current is the current support redundancy of grasped object j.

5. The method of claim 1, wherein, In step S2b, the method of constructing the lateral support relationship graph is: taking each grasped object as a node, and establishing an undirected edge e between node i and node j if and only if the horizontal interval of grasped objects i and j is less than a threshold or there is side contact between their point clouds ij h , and assigning a weight w to the edge ij h .

6. The method of claim 5, wherein, The lateral side weight w ij h The calculation formula is: w ij h = μ·F ij · cos(θ ij ), wherein μ is the effective friction coefficient, F ij is the normal contact pressure between the grasped objects i and j due to the upper stacking load, and θ ij is the angle between the contact surface normal and the horizontal plane.

7. The method of claim 5, wherein, The lateral risk L in step S2b i horz The calculation formula is: Where N i is the set of grasped objects directly connected to node i in the lateral support relationship graph, I j critical is the criticality indication function of grasped object j, ΔP j is the lateral constraint potential energy loss of grasped object j after removing grasped object i.

8. The method of claim 1, wherein, The formula of the coupling in step S2c is: wherein a and β are weight coefficients, and γ is a coupling coefficient.

9. The method of claim 1, wherein, In step S3, the grasping object with the lowest grasping stability risk parameter is selected as the target grasping object.

Images