Cooperative grasping control method and system of multi-nozzle mechanical arm

Abstract

This invention discloses a collaborative grasping control method and system for a multi-nozzle robotic arm, addressing the problems of lack of dynamic feedback and poor stability in existing grasping processes. The method includes: acquiring a grasping plan; performing static path pre-planning based on obstacle information and calculating center-of-gravity compensation and dynamic anti-sway control parameters; controlling the robotic arm to move to the grasping point to perform adsorption, monitoring in real time via pressure sensors, and implementing a tiered retry mechanism if the target is not met; after successful adsorption, performing dynamic anti-sway control and dynamic interference detection during lifting and movement; and performing integrity verification and recording data for closed-loop optimization after placement. This invention significantly improves the dynamic safety, reliability, and adaptability of multi-box combined grasping through collaborative control of static pre-planning, dynamic center-of-gravity compensation, and PID anti-sway, combined with retry and verification mechanisms.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics and automated handling technology, and more particularly to a collaborative gripping control method and system for a multi-nozzle robotic arm. This invention also relates to electronic devices and storage media for implementing the above method. Background Technology

[0002] Currently, automated gripping technology for stacked cardboard boxes in warehousing and port settings mainly relies on robotic arms working in conjunction with suction nozzles to perform the gripping operation. While existing technologies have made some progress, significant bottlenecks in control capabilities still exist at the execution level.

[0003] The following existing patented technologies are cited:

[0004] 1. CN110103231B (A precise grasping method and system for a robotic arm): Discloses a path planning method in a static obstacle environment, but does not consider the dynamic characteristics of the grasped object itself, nor does it consider the impact of swinging after grasping and changes in the combined center of gravity on path safety.

[0005] 2. CN118123802B (Robotic arm gripping method, system, device, electronic equipment and storage medium): It uses a single suction nozzle in conjunction with 3D vision to grip a single box at a time, but does not cover center of gravity compensation and anti-sway control when gripping multiple boxes in combination.

[0006] Summary of core defects in existing technologies:

[0007] 1. Static Path Planning: Existing path planning algorithms (such as RRT algorithm and A* algorithm) mainly consider static obstacles in the environment, ignoring the risk of dynamic secondary collisions caused by the swing of the cardboard box (swing amplitude > 5cm) during the grasping and lifting process, as well as the impact of the overall center of gravity shift (> 1.5cm) after multiple boxes are combined on the stability of the robotic arm's movement, which poses safety hazards.

[0008] 2. Lack of closed-loop feedback mechanism: Most existing systems are open-loop control systems, i.e., "visual positioning-planning-execution," lacking real-time feedback and response mechanisms for key states such as whether the adsorption process is successful and whether the cardboard box is intact after gripping. Once adsorption fails or the cardboard box falls off midway, the system cannot detect and correct it, making it difficult to break through the 90% bottleneck in overall gripping success rate.

[0009] 3. Lack of dynamic control capability: Existing technology lacks the ability to dynamically suppress the swaying of the carton during the gripping process. It cannot adjust the motion parameters of the robotic arm according to the real-time swaying state, resulting in unstable gripping and easy slippage or damage.

[0010] Therefore, this invention aims to provide a collaborative grasping control method for a multi-nozzle robotic arm, which achieves precise, stable and reliable control of the grasping process of the multi-nozzle robotic arm through static path pre-planning, dynamic center of gravity compensation, PID anti-sway control, three-level retry mechanism and closed-loop optimization. Summary of the Invention

[0011] To address the technical problem of the inability to perform dynamic, closed-loop, and stable control of the grasping process of multi-nozzle robotic arms in the prior art, this invention provides a collaborative grasping control method for multi-nozzle robotic arms.

[0012] Therefore, the first objective of this invention is to provide a collaborative grasping control method for a multi-nozzle robotic arm, which includes the following steps: Obtain a grasping scheme, which includes the grasping point position, the nozzle layout, the suction pressure setting, and the grasping posture of the robotic arm end effector; Based on the obstacle information of the current scene and the grasping points in the grasping scheme, static path pre-planning is performed, and dynamic control parameters during the grasping process are calculated to form the planned path; the dynamic control parameters include center of gravity compensation parameters and dynamic anti-sway control parameters. The robotic arm is controlled to move to the gripping point along the planned path, perform the adsorption operation, and monitor the adsorption status in real time through a pressure sensor. If the adsorption status does not meet the standard, a preset retry mechanism is executed. After successful adsorption, the robotic arm is controlled to perform lifting, moving, lowering and releasing the grasped object. During this process, dynamic anti-sway control and dynamic interference detection are performed based on the dynamic control parameters. After the object is placed, perform post-grab integrity verification and record the entire grabbing process data for subsequent closed-loop optimization of strategies.

[0013] Preferably, the static path pre-planning includes: The path planning algorithm is selected based on the occlusion situation: in scenarios with no occlusion or low occlusion, the A* algorithm is used to perform fast path search in joint space; in scenarios with occlusion ahead or multiple occlusions, the RRT* algorithm and D*Lite algorithm are combined to generate a bypass path, ensuring that the bypass distance is at least the preset safe distance. If path planning fails, the failure handling process is executed: record the failure information, relax the parameters and retry, and terminate the current crawling if there are no more candidate solutions.

[0014] Preferably, the method for calculating the center of gravity compensation parameter includes: Estimate the weight of each carton based on its dimensions and material, and calculate the coordinates of the combined center of gravity. Calculate the offset of the combined center of gravity relative to the gripping point at the end of the robotic arm; The lifting acceleration of the robotic arm during the lifting phase and the moving acceleration during the moving phase are adjusted according to the offset: when the offset is less than or equal to the preset offset threshold, the acceleration is calculated according to the linear decreasing rule; when the offset exceeds the preset offset threshold, the minimum safe acceleration is used; when there is no offset, the default acceleration is used.

[0015] Preferably, the dynamic anti-sway control parameters include the sway amplitude threshold and the proportional, integral, and derivative coefficients of the PID controller; The swaying amplitude of the carton is monitored in real time by a posture gyroscope. When the swaying exceeds the preset swaying threshold, the PID controller calculates the output based on the error and adjusts the movement speed and end-effector posture of the robotic arm to suppress the swaying.

[0016] Preferably, the adsorption operation and retry mechanism specifically include: The robotic arm approaches at low speed to bring the suction nozzle into contact with the surface of the carton, opens the vacuum valve to establish suction pressure, and holds the pressure for a preset time. The adsorption pressure is detected by a pressure sensor integrated into the nozzle end, and the pressure value is required to fluctuate within the set target pressure ±0.003MPa and not be lower than 90% of the set value; If the adsorption pressure is not up to standard, perform a preset number of retries: the first retries fine-tune the nozzle stroke and the robotic arm contact position; the second retries add an auxiliary nozzle; the final retries abandon the combined gripping, split it into single-box gripping, regenerate the single-box gripping plan, and execute it sequentially.

[0017] Preferably, the dynamic anti-sway control and dynamic interference detection include: During the vertical lifting process, the swing amplitude is monitored in real time, and PID anti-sway control is triggered when the swing exceeds the preset swing threshold. During lifting and movement, the scene point cloud is continuously collected by the 3D main camera and fused with the current real-time posture of the cardboard box to detect new obstacles or sudden posture changes. If a potential interference risk is detected during the elevation process, the elevation is paused, and the D*Lite algorithm is triggered for local path replanning; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered. If a potential interference risk is detected during horizontal movement, the movement is paused, the D*Lite algorithm is triggered to perform local path replanning, and a new obstacle avoidance path is generated before execution continues; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered.

[0018] Preferably, the post-crawl integrity verification includes: The image of the cardboard box is captured by a secondary vision inspection camera and compared with the image before the grasping process to identify newly added damaged areas. If the newly added damaged area exceeds the preset area threshold or the cardboard box is detected to have fallen off, the grasping process is deemed to have failed. Simultaneously monitor the pressure fluctuation of the suction nozzle and the swing amplitude of the carton. If the pressure fluctuation exceeds the preset pressure fluctuation threshold or the swing amplitude exceeds the preset emergency swing threshold, the gripping is also determined to be in failure. If the capture fails, perform emergency placement and record the failure information; if the capture is successful, record the entire process data for closed-loop optimization.

[0019] The second objective of this invention is to provide a collaborative gripping control system for a multi-nozzle robotic arm, comprising: The path planning and anti-sway module is used to perform static path pre-planning and calculate dynamic control parameters during the grasping process; The instruction generation module is used to generate control instructions to drive the hardware to perform the grasping operation and to process feedback signals in real time. The feedback detection module is used for real-time monitoring by calling the feedback data from various hardware sensors; The data storage and optimization module is used to record data throughout the entire process and to achieve iterative optimization of the system.

[0020] A third objective of this invention is to provide an electronic device comprising: The 3D vision perception system consists of an ultra-high dynamic range (HDR) structured light 3D main camera and a secondary detection camera, used to acquire raw point cloud data and two-dimensional RGB images of stacked scenes; The reconfigurable multi-nozzle end effector is a 4×4 matrix-type independently controlled nozzle array, integrating pressure sensors and stroke adjustment mechanisms; A load-redundant six-axis high-speed robotic arm that integrates force control sensors and attitude gyroscopes; Industrial-grade PLC controller; The central control system is connected via a bus to the 3D vision perception system, the reconfigurable multi-nozzle end effector, the robotic arm, and the PLC controller. The central control system includes: Memory, used to store computer programs; A processor is configured to execute a computer program stored in the memory to implement the steps of any of the above-described collaborative grasping control methods for a multi-nozzle robotic arm.

[0021] The fourth objective of this invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described collaborative grasping control methods for a multi-nozzle robotic arm.

[0022] The beneficial effects of this invention are as follows:

[0023] 1. Enhanced Dynamic Safety: Through a collaborative path planning mechanism of "static pre-planning + dynamic center of gravity compensation + PID anti-sway," the physical attributes (center of gravity offset, sway amplitude) of multiple boxes are incorporated into real-time control. The collision failure rate is ≤0.3%, the secondary collision rate is ≤0.2%, the carton breakage rate is ≤0.8%, and the multi-box grasping center of gravity instability rate is ≤0.1%, achieving orders-of-magnitude optimization compared to existing technologies.

[0024] 2. High reliability: Establish a closed-loop feedback control system for the entire process of "adsorption detection - three-level retry mechanism - integrity verification" to achieve intelligent retry after adsorption failure and status confirmation after capture, thereby increasing the capture success rate to over 99% and the adsorption failure retry success rate to ≥95%.

[0025] 3. Closed-loop optimization capability: By recording data throughout the entire process and using it for strategy optimization, the system has the ability to iterate itself and improve with use, and can quickly adapt to warehousing and port operation scenarios of different scales. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0027] Figure 1 This is a flowchart illustrating a collaborative grasping control method for a multi-nozzle robotic arm according to an embodiment of the present invention.

[0028] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art to which this invention pertains.

[0030] Terminology Explanation:

[0031] A* algorithm: A classic heuristic pathfinding algorithm widely used to find the shortest path from a starting point to a target point in a graph. It guides the search by considering two factors: the actual cost incurred from the starting point to the current position, and the estimated cost from the current position to the target point. This "actual cost plus estimated cost" evaluation method allows A* algorithm to both guarantee finding the optimal path like Dijkstra's algorithm and achieve high search efficiency like a greedy algorithm.

[0032] The core of the algorithm lies in the design of the heuristic function. For each node, a total cost is calculated, where the actual cost reflects the length of the path already traversed, and the heuristically estimated cost predicts how far away from the target. As long as the value estimated by the heuristic function does not exceed the actual shortest distance, the algorithm guarantees finding the optimal solution. During the search process, the algorithm maintains a priority queue, and each time it takes the node with the smallest total cost from the queue to expand, until the target point is reached.

[0033] In the field of robot path planning, the A* algorithm is commonly used for navigation on two-dimensional planes or mesh maps. In this invention, the A* algorithm is applied to joint space planning of a robotic arm to handle simple grasping scenarios with no or low occlusion. Here, each node represents a set of joint angles of the robotic arm, with the actual cost being the cumulative change in these joint angles. The heuristic function uses the Euclidean distance from the current joint angle to the target joint angle. Collision detection is performed while expanding the nodes to ensure that the generated path does not interfere with known walls of cardboard boxes or containers in the environment. In this way, the A* algorithm can quickly generate a shortest path that satisfies joint constraints and kinematic constraints, providing a motion basis for subsequent grasping actions.

[0034] The RRT* algorithm is a path planning method based on random sampling. It continuously generates random sampling points in the robot's configuration space (i.e., the space consisting of all possible poses), progressively building a search tree expanding outwards from the starting point until the destination is reached. Compared to the original RRT algorithm, the RRT* algorithm introduces a "reconnection" mechanism: whenever a new node is added to the tree, the algorithm checks its neighboring nodes. If the path to these nodes via the new node is shorter, the connection relationship is updated, allowing the path to gradually approach the optimal solution with increasing iterations. This algorithm can effectively handle planning problems in high-dimensional spaces (such as the joint angle space of a robotic arm) and does not require precise environmental modeling; theoretically, a feasible path can be found as long as it exists. However, the paths generated by the RRT* algorithm are usually not smooth enough, and need to be replanned when the environment changes dynamically, limiting its adaptability.

[0035] The D*Lite algorithm is a path planning method for dynamic environments. Evolved from the A* algorithm, it efficiently adjusts the original path based on environmental changes (such as the appearance of new obstacles). D*Lite maintains the current optimal path from the starting point to the destination and stores the cost estimate for each node. When an environmental change is detected, the algorithm only updates the costs of the affected local nodes, prioritizing the most significantly changed parts through a priority queue, thus achieving rapid incremental replanning and avoiding recalculation from scratch. This makes the D*Lite algorithm highly suitable for scenarios where robots need to avoid obstacles in real time during movement, enabling rapid response to sudden obstacles and ensuring the continued feasibility of the path.

[0036] In this invention, the RRT* algorithm and the D*Lite algorithm work together: during static planning, the RRT* algorithm is first used to quickly generate an initial path that bypasses all known cardboard boxes; then the D*Lite algorithm optimizes the path to ensure that the detour distance meets safety requirements; during the movement of the robotic arm, if the 3D vision detects a new obstacle, the D*Lite algorithm is immediately triggered to perform local replanning, adjusting only the affected path segments, thus taking into account both global exploration capability and dynamic response speed, and improving the overall planning efficiency by 5% to 8% compared to a single algorithm.

[0037] Example 1

[0038] This embodiment provides a collaborative grasping control method for a multi-nozzle robotic arm. This embodiment uses a "grasping scheme" as input and "path planning - dynamic control - retry mechanism - closed-loop optimization" as its core to construct a complete control chain from instruction to execution.

[0039] First, the hardware system is deployed. The system includes: a 3D vision perception system, consisting of an ultra-high dynamic range (HDR) structured light 3D main camera and a secondary inspection camera; a reconfigurable multi-nozzle end effector, a 4×4 matrix of independently controlled nozzles; a load-redundant six-axis high-speed robotic arm, integrating force control sensors and attitude gyroscopes; an industrial-grade PLC controller; and a feedback detection module, consisting of multi-dimensional pressure sensors at the nozzle end, a cardboard box attitude gyroscope, and a secondary vision inspection camera. All hardware is connected to the central control system via a bus.

[0040] Before the operation begins, the size range of the cardboard boxes to be picked up is imported into the control system (in this embodiment, a length and width of 30-80cm and a thickness of 5-20cm are used as an example).

[0041] In this embodiment, the scenario includes three layers of stacked cardboard boxes, totaling seven boxes. The characteristic data of each box are as follows:

[0042] Cardboard box A: Basic geometric features: Dimensions 0.5m × 0.4m × 0.12m, center coordinates (1.25m, 0.80m, 0.38m), attitude angles Φx=0°, Φy=18°, Φz=0°; Damage characteristics: No damage; Occlusion: No occlusion (occlusion rate 104.5%). Suspended / overlapping characteristics: Number of support points Nsupport=2, overlap length Loverlap=0.36m; Tilt characteristics: Attitude angle 18°, tilt direction along the positive X-axis, support area ratio 90%; Multi-layer staggered features: layer number Li=3, overhang area ratio αoverhang=0, overlapping carton number {C}, interlayer spacing dz=0.025m.

[0043] Cardboard box B: Basic geometric features: Dimensions 0.5m × 0.4m × 0.12m, center coordinates (1.25m, 1.22m, 0.41m), attitude angles Φx = 0°, Φy = 15.7°, Φz = 0°; Damage characteristics: No damage; Occlusion: No occlusion (occlusion ratio 102%) Suspended / overlapping characteristics: Number of support points Nsupport=2, overlap length Loverlap=0.38m; Tilt characteristics: attitude angle 15.7°, tilt direction along the positive X-axis, support area ratio 95%; Multi-layer staggered features: layer number Li=3, overhang ratio αoverhang=0, overlapping carton number {D}, interlayer spacing d_z=0.055m.

[0044] Cardboard box C: Basic geometric features: Dimensions 0.603m × 0.402m × 0.151m, center coordinates (1.20m, 0.80m, 0.22m), attitude angle 0°; Damage characteristics: No damage; Occlusion status: No occlusion (occlusion rate 97.7%). Overhang / overlap feature: Not applicable (middle layer carton); Tilt characteristics: Attitude angle 0°; Multi-layer staggered features: layer number Li=2, overhang area ratio αoverhang=0, overlapping carton numbers {E, F}, interlayer spacing d_z=0.045m.

[0045] Cardboard box D: Basic geometric features: Dimensions 0.601m × 0.398m × 0.150m, center coordinates (1.20m, 1.50m, 0.22m), attitude angle 0°; Damage characteristics: No damage; Occlusion status: No occlusion (occlusion rate 98.4%). Overhang / overlap feature: Not applicable; Tilt characteristics: Attitude angle 0°; Multi-layer staggered features: layer number Li=2, overhang area ratio αoverhang=0, overlapping carton number {G}, interlayer spacing d_z=0.045m.

[0046] Cardboard box E: Basic geometric features: Dimensions 0.552m × 0.351m × 0.101m, center coordinates (0.50m, 0.60m, 0.05m), attitude angle 0°; Damage characteristics: No damage; Occlusion status: No occlusion (occlusion rate 98.7%). Overhang / overlap feature: Not applicable; Tilt characteristics: Attitude angle 0°; Multi-layer staggered feature: layer number Li=1, output is empty.

[0047] Cardboard box F: Basic geometric features: Dimensions 0.551m × 0.350m × 0.100m, center coordinates (0.50m, 1.20m, 0.05m), attitude angle 0°; Damage characteristics: No damage; Occlusion status: No occlusion (occlusion rate 98.7%). Overhang / overlap feature: Not applicable; Tilt characteristics: Attitude angle 0°; Multi-layer staggered feature: layer number Li=1, output is empty.

[0048] Cardboard box G: Basic geometric features: Dimensions 0.550m × 0.352m × 0.100m, center coordinates (0.50m, 1.80m, 0.05m), attitude angle 0°; Damage characteristics: No damage; Occlusion status: No occlusion (occlusion rate 98.7%). Overhang / overlap feature: Not applicable; Tilt characteristics: Attitude angle 0°; Multi-layer staggered feature: layer number Li=1, output is empty.

[0049] An embodiment of the present invention provides a collaborative grasping control method for a multi-nozzle robotic arm, such as... Figure 1 As shown, it includes the following steps:

[0050] S1, Obtain the grasping scheme, which includes the grasping point position, the nozzle layout, the adsorption pressure setting, and the grasping posture of the robotic arm end effector.

[0051] The grasping scheme is generated in real time by the central control system based on the stacked scene point cloud and RGB images collected by the 3D vision perception system. Specifically, it includes the grasping point position, nozzle layout, suction pressure setting, and the grasping posture of the robotic arm end effector. The generation method is as follows: First, the point cloud is clustered and segmented to identify the size, center coordinates, and posture angle of each cardboard box. Then, the number of nozzles is allocated according to the size of the cardboard box (2 for small boxes, 3 for medium boxes, and 4 for large boxes), and the basic suction pressure is set according to the cardboard material (0.06-0.08 MPa for corrugated paper and 0.05-0.07 MPa for kraft paper). Next, the grasping point position is set at a preset height directly above the combined center of gravity of all grasped cardboard boxes. Finally, the robotic arm end effector's pre-adjusted posture is calculated based on the tilt angle of the cardboard box, making the nozzle plane nearly parallel to the cardboard box surface.

[0052] In this embodiment, the scene contains seven cardboard boxes stacked in three layers. The top two boxes, A and B, are the targets to be grasped; both are 0.5m × 0.4m × 0.12m in size and weigh 6kg. Their center coordinates are (1.25m, 0.80m, 0.38m) and (1.25m, 1.22m, 0.41m), respectively, with attitude angles of 18° and 15.7°. The middle boxes C and D, and the bottom boxes E, F, and G are regularly stacked static obstacles; their dimensions, center coordinates, and other characteristic data are omitted.

[0053] The combined gripping method is used to grasp the upper inclined cartons A and B. The specific parameters of the gripping scheme are as follows:

[0054] Grab point location: 5cm directly above the center of gravity of the assembly, i.e. (1.25, 1.01, 0.445).

[0055] Nozzle layout: A 4×4 nozzle array is used, with 3 nozzles allocated to each carton: 2 on the lower side and 1 on the higher side. Carton A has lower nozzle positions at (1.15, 0.95, 0.38) and (1.35, 0.95, 0.38), and a higher nozzle position at (1.25, 0.65, 0.38). Carton B has lower nozzle positions at (1.15, 1.37, 0.41) and (1.35, 1.37, 0.41), and a higher nozzle position at (1.25, 1.07, 0.41).

[0056] Adsorption pressure setting: All nozzles are uniformly set to 0.06MPa (kraft paper material).

[0057] Robotic arm end-effector grasping posture: Pre-adjust reverse posture 10° (around X-axis -10°).

[0058] S2, based on the obstacle information of the current scene and the grasping points in the grasping scheme, perform static path pre-planning and calculate the dynamic control parameters during the grasping process to form the planned path; the dynamic control parameters include center of gravity compensation parameters and dynamic anti-sway control parameters.

[0059] Based on the grasping scheme output in step S1 (including the target cardboard box group, the size, weight, spatial coordinates, posture, and nozzle layout of each box), this step performs static path planning and dynamic control parameter calculation to form a planned path, providing a foundation for subsequent adsorption execution and motion control. The dynamic control parameters include center of gravity compensation parameters and dynamic anti-sway control parameters, and the specific implementation process is as follows:

[0060] S21, Select the path planning algorithm based on the occlusion situation.

[0061] Based on the obstacle information of the current scene (point cloud data collected by the 3D vision perception system and the identified cardboard box positions) and the gripping point positions in the gripping scheme, a collision-free path is generated from the current position of the robotic arm to the gripping point. Path planning needs to consider the joint limits of the robotic arm, speed constraints, and obstacle avoidance with the surrounding environment, and different algorithms are selected according to the occlusion situation.

[0062] 1. In unobstructed or low-obstruction scenarios, the A* algorithm is used for fast path search in joint space, generating the shortest path at the cost of minimizing joint angle changes. After planning, it is necessary to verify whether the path meets the joint limits and velocity / acceleration constraints of the robotic arm. If not, the path is replanned.

[0063] 2. In scenarios with obstructions or multiple layers of obstruction, a hybrid RRT* algorithm and D*Lite algorithm are used to generate an avoidance path, ensuring that the detour distance is at least a preset safe distance. Preferably, the detour distance is at least the thickness of the cardboard box plus 8cm, with allowance for dynamic adjustment redundancy. The specific implementation steps are as follows:

[0064] First, a configuration space (C-space) is constructed, incorporating the carton size, the robotic arm's range of motion, and a safety distance (typically 5cm), establishing an obstacle map. Then, the RRT* algorithm is used to generate an initial feasible path: randomly sampling points within the C-space, ensuring these points are within safe zones; growing new nodes from the current tree node towards the sampling points and performing collision detection; optimizing the path through reconnection operations to make it shorter and smoother. The RRT* algorithm parameters are set to a maximum of 1000 iterations, a step size of 0.1m, and a collision detection accuracy of ±0.01m.

[0065] After generating the initial path, B-spline curves are used for smoothing to eliminate unnecessary inflection points and make the path continuous and smooth.

[0066] Subsequently, based on the initial path generated by the RRT* algorithm, the D*Lite algorithm is applied for dynamic optimization. The specific process is as follows: initialize the priority queue and set the initial cost from the start point to the end point; the cost calculation needs to consider the carton thickness, safety distance, and robotic arm motion parameters (such as joint speed and acceleration limits).

[0067] Preferably, the cost weight coefficients α and β are set to 0.7 and 0.3 respectively; when a new occlusion or obstacle is detected, the cost function is updated and only the affected path segment is recalculated, rather than the entire path, to improve the replanning efficiency; the update frequency of D*Lite is set to 50ms.

[0068] Finally, during the crawling process, if the environment changes (such as the addition of occlusion), the D*Lite algorithm is triggered for local replanning to ensure the path remains safe. The detour distance of the output path should meet the requirement of not less than the preset safe distance.

[0069] In this embodiment, the grasping targets are the upper-layer tilted cardboard boxes A and B, and the grasping point is located 5cm directly above the combined center of gravity, i.e., (1.25, 1.01, 0.445). The robotic arm is currently in a standby pose, and there are no other obstacles around the grasping point, constituting an unobstructed scene. Therefore, the A* algorithm is used for path planning. After converting the grasping point into joint angles using inverse kinematics, the A* algorithm generates a path at the cost of minimizing the change in joint angles. This path includes 10 intermediate points, has a total motion time of 1.2s, and both joint velocity and acceleration are within the specified limits, thus meeting the requirements.

[0070] S22, if path planning fails, execute the failure handling process: record the failure information, relax the parameters and retry, and terminate this round of crawling if there are no more candidate solutions.

[0071] If the aforementioned static path pre-planning algorithm (A* algorithm or a hybrid algorithm of RRT* and D*Lite) cannot generate a collision-free path from the current position of the robotic arm to the grasping point under given constraints, the current grasping scheme is deemed infeasible. In this case, the system executes the following processing flow:

[0072] 1. Failure Log: Records information about the planning failure, including the reason for failure (such as path not existing, collision unavoidable, joint limit, search timeout, etc.), grabbing scheme identifier, target carton group ID, current timestamp, etc., for subsequent closed-loop optimization.

[0073] 2. Parameter Relaxation and Retry: If the planning failure is caused by adjustable conservative parameters (such as excessively large safety distance or excessively high detour redundancy requirements), the system automatically relaxes the constraints, re-executes path planning, and retries the path planning for a preset maximum number of times (in this embodiment, the maximum number of retries is set to 2). The relaxation strategy is as follows:

[0074] Safe distance: The safe distance is gradually reduced from the default safe distance to a smaller safe distance (in this embodiment, it is gradually reduced from 5cm to 3cm and 2cm).

[0075] Detour distance requirement: It decreases successively from the thickness of the carton plus a preset redundancy distance (in this embodiment, it decreases successively from "carton thickness + 8cm" to "carton thickness + 5cm" and "carton thickness + 3cm").

[0076] If the retry is successful, continue with the subsequent steps.

[0077] If retrying with relaxed parameters still fails, the current crawling plan is deemed unexecutable. The system records this failure and terminates the current round of crawling, waiting for the next round of instructions or rescanning the scene. Optionally, an alarm notification can be issued for manual intervention.

[0078] S23, center of gravity compensation parameter, calculates the coordinates of the combined center of gravity and the offset relative to the end gripping point of the robotic arm based on the estimated weight and spatial coordinates of each carton, and dynamically adjusts the acceleration of the robotic arm during the lifting and moving phases according to the offset.

[0079] To overcome the impact of center of gravity shift on the stability of the robotic arm when grasping multiple boxes in combination, center of gravity compensation is required.

[0080] S231, estimate the weight of each carton based on its size and material, and calculate the coordinates of the combined center of gravity.

[0081] Estimate the weight of each carton based on its dimensions and material. Assume the carton is a rectangular shape with uniform density and its center of gravity at its geometric center. Corrugated cardboard has a density of approximately 0.6 g / cm³, and kraft paper approximately 0.8 g / cm³. The weight calculation formula is as follows: mi = ρ × Li × Wi × Hi Using the spatial coordinates (geometric center coordinates (xi, yi, zi)) of each carton obtained in step S2, calculate the coordinates of the combined centroid: xc = (Σmi × xi) / Σmi yc = (Σmi × yi) / Σmi zc = (Σmi × zi) / Σmi

[0082] S232, calculate the offset of the combined center of gravity relative to the gripping point at the end of the robotic arm.

[0083] Let the coordinates of the center of the suction cup array (i.e., the reference point when the robotic arm grasps) be (x0, y0, z0), calculate: Centroid offset: Δx = |xc - x0|, Δy = |yc - y0|, Δz = |zc - z0| Horizontal offset: Δxy = √(Δx² + Δy²)

[0084] S233, adjust the lifting acceleration of the robotic arm during the lifting phase and the moving acceleration during the moving phase according to the offset.

[0085] The robotic arm's motion parameters are adjusted based on the offset to reduce the risk of instability. The specific rules are as follows:

[0086] 1. When the offset is less than or equal to a preset offset threshold, the acceleration is calculated according to a linear decreasing rule; the preset offset threshold is 1.5cm. The lifting acceleration and the moving acceleration are calculated using the following formula: Increased acceleration: az = 0.5 × (1 - |Δz| / 0.015) m / s² Acceleration during movement: axy = 0.8 × (1 - |Δxy| / 0.015) m / s²

[0087] 2. When the offset exceeds the preset offset threshold, the minimum safe acceleration is adopted: in this embodiment, the lifting acceleration is 0.1 m / s² and the moving acceleration is 0.2 m / s².

[0088] 3. Default acceleration is used when there is no offset: The default acceleration is used. In this embodiment, the lifting acceleration is 0.5 m / s² and the moving acceleration is 0.8 m / s².

[0089] In this embodiment, both boxes A and B are made of kraft paper with a density of 250 kg / m³ (0.25 g / cm³), dimensions of 0.5 m × 0.4 m × 0.12 m, volume of 0.024 m³, and mass of m = 6 kg. Their center of gravity coordinates are (1.25, 0.8, 0.38) and (1.25, 1.22, 0.41), respectively. Combined center of gravity: xc = (6×1.25+6×1.25) / 12 = 1.25 yc = (6×0.8+6×1.22) / 12 = 1.01 zc = (6×0.38+6×0.41) / 12 = 0.395 The center of the suction cup array is designed to align with the combined center of gravity on the horizontal projection, therefore Δx=0, Δy=0. In the vertical direction, since the suction cup plane is above the cardboard surface during gripping, the actual vertical distance between the center of gravity and the suction cup center is half the cardboard thickness (approximately 0.06m). However, here Δz is defined as the height difference between the center of gravity and the suction cup plane. Since the suction cup plane is in contact with the cardboard surface, the vertical offset of the center of gravity can be ignored. To demonstrate dynamic adjustment, a small offset is assumed to exist: Δxy=0.01m, Δz=0.005m. Calculate the acceleration: The acceleration az = 0.5 × (1 - 0.005 / 0.015) = 0.5 × 0.667 = 0.3335 m / s²; The acceleration axy = 0.8 × (1 - 0.01 / 0.015) = 0.8 × 0.333 = 0.2664 m / s².

[0090] S24, dynamic anti-sway control parameters, including preset swing amplitude threshold and initial proportional, integral, and derivative coefficients of the PID controller used to suppress swing.

[0091] To suppress the swaying of the carton during the gripping and lifting process, a PID algorithm is used for dynamic anti-sway control. Specifically, the swaying amplitude of the carton is monitored in real time by an attitude gyroscope. When the swaying exceeds a preset sway threshold (the preset sway threshold is 5cm in this embodiment), the PID controller calculates the error and outputs an adjustment to the movement speed and end effector posture of the robotic arm to suppress the swaying.

[0092] The implementation steps are as follows:

[0093] First, the angular velocity and acceleration data of the carton are acquired in real time using an attitude gyroscope. The angular velocity is integrated to obtain the angle change, and then the swing amplitude M of the current frame relative to the average attitude is calculated (taking the maximum linear displacement of the carton edge). At the same time, the main direction of the swing (X, Y or Z axis) is identified.

[0094] Let the swing threshold Mth = 5cm, and calculate the error e = M - Mth. PID control is triggered when e > 0.

[0095] The output u of a PID controller consists of a proportional term, an integral term, and a derivative term: u(t) = Kp × e(t) + Ki × ∫e(t)dt + Kd × de(t) / dt The proportional coefficient Kp, integral coefficient Ki, and derivative coefficient Kd are set according to the number of cartons grasped, their total weight, and the estimated swing characteristics (these can be calibrated through simulation or experiment). The default PID parameters are Kp=0.5, Ki=0.1, and Kd=0.05. The control cycle is synchronized with the robotic arm servo cycle (usually 4ms). It should be noted that the PID parameters can be adaptively adjusted according to the total weight of the cartons grasped and the swing characteristics.

[0096] Adjust the robotic arm's motion parameters based on the PID output u: Speed ​​adjustment: vnew = vcurrent × (1 - u / 100) Attitude adjustment: θnew = θcurrent + u / 5 × ,in is the unit vector representing the direction of the swing.

[0097] The correction amount is limited to a reasonable range to avoid sudden changes in the robotic arm.

[0098] In this embodiment, the default PID parameters Kp=0.5, Ki=0.1, Kd=0.05, oscillation threshold of 5cm, and control cycle of 4ms are used. The attitude gyroscope samples at a frequency of 100Hz to calculate the oscillation amplitude in real time. In the initial lifting stage, due to the stable suction pressure, the carton does not oscillate significantly. The angle change obtained by integrating the angular velocity measured by the gyroscope is less than 1°, corresponding to an oscillation amplitude of less than 1.7cm, which is below the threshold. Therefore, the PID controller does not trigger correction.

[0099] S3, control the robotic arm to move to the gripping point according to the planned path, perform the adsorption operation, and monitor the adsorption status in real time through the pressure sensor. When the adsorption status does not meet the standard, execute the preset retry mechanism.

[0100] Based on the path planned in step S2 and the nozzle layout and suction pressure settings obtained in step S1, the robotic arm moves to above the gripping point along the planned path, performs the suction operation, and monitors the suction status in real time through a pressure sensor. The decision to trigger a retry logic is based on the detection results. The specific implementation process is as follows:

[0101] S31, the robotic arm approaches at low speed to make the suction nozzle contact the surface of the carton, opens the vacuum valve to establish adsorption pressure, and holds the pressure for a preset time.

[0102] The robotic arm moves along the path planned in step S2 to a preset height (2cm in this embodiment) directly above the gripping point, and then descends vertically at a preset low speed (0.1-0.15m / s in this embodiment) until the suction nozzle contacts the upper surface of the carton. Contact is determined by a force control sensor detecting a sudden change in force in the Z-direction exceeding a preset threshold (5N in this embodiment) or compression of the suction nozzle stroke. Upon contact, the control system immediately opens the vacuum valve of the corresponding suction nozzle, evacuates the inner cavity of the nozzle, establishes adsorption pressure, and maintains the pressure for a preset time (0.3-0.5s in this embodiment) to ensure stable adsorption between the suction nozzle and the carton surface.

[0103] S32 detects the adsorption pressure through a pressure sensor integrated at the nozzle end, requiring the pressure value to fluctuate within the set target pressure ±0.003MPa range and not be lower than 90% of the set value.

[0104] After the pressure holding period ends, the pressure sensor integrated into the nozzle reads the vacuum pressure value of each nozzle's adsorption pressure. For each nozzle participating in adsorption, its pressure value is required to fluctuate within the set target pressure ±0.003 MPa, and the pressure of all nozzles must not be lower than the minimum allowable pressure (usually 90% of the set value). If the pressure of all nozzles meets the standard, adsorption is considered successful, and the system enters the boosting stage (step S4); if the adsorption pressure of one nozzle does not meet the standard, adsorption is considered to have failed, and a preset retry mechanism is executed.

[0105] S33, if the adsorption pressure does not meet the standard, perform a preset number of retry.

[0106] If the adsorption pressure is not up to standard, adsorption will fail, and the following operations will be performed sequentially based on the number of failures (in this embodiment, the maximum number of failures is set to 3):

[0107] 1. On the first attempt, fine-tune the nozzle stroke and the robotic arm's contact position.

[0108] Fine-tune the nozzle stroke and the robotic arm contact position, and repeat the adsorption process. Based on the pressure detection results, analyze the position of the nozzle with low pressure, fine-tune the stroke of the corresponding nozzle (in this embodiment, extend the low-position nozzle by 1mm and shorten the high-position nozzle by 1mm) and adjust the robotic arm contact position (in this embodiment, fine-tune the vertical descent depth by ±1mm), and then repeat the adsorption operation (i.e., repeat steps S31-S32).

[0109] 2. Add an auxiliary suction nozzle for the second retry.

[0110] If the first retry still fails, an additional number of auxiliary nozzles (1-2 in this embodiment) are added to the spare nozzles that have never participated in adsorption, based on the original nozzles. These auxiliary nozzles are placed near the center of gravity of the assembly or in areas with low pressure, and then the adsorption operation is performed again.

[0111] 3. On the final retry, abandon the combined crawling, split it into single-box crawling, regenerate the single-box crawling plan, and execute it sequentially.

[0112] If the second retry also fails, the current combined grabbing is abandoned, and the grabbing plan is split into single-box grabbing. That is, each carton in the combination is marked as a single-box grabbing target, and the system regenerates the single-box grabbing plan and executes it sequentially, recording the failure data for closed-loop optimization.

[0113] If the adsorption is successful in any of the above retry attempts, the subsequent steps continue; if it still fails after reaching the maximum number of retry attempts, the system records this grab as "complete failure" and updates the carton status to "ungrabable" (in this embodiment, manual intervention or rescanning is required).

[0114] In this embodiment, the robotic arm moves along the path planned in step S2 to a position 2cm directly above the gripping point (coordinates (1.25, 1.01, 0.465)), and then descends vertically at a low speed of 0.08m / s. After the suction nozzles contact the surfaces of cartons A and B, the force control sensor detects a sudden change in force of approximately 8N in the Z direction. The system immediately opens the vacuum valve and holds the pressure for 0.5s. After the pressure holding period, the pressure values ​​of the six suction nozzles are read: for carton A, the low-position suction nozzle 1 is -0.058MPa, the low-position suction nozzle 2 is -0.059MPa, and the high-position suction nozzle is -0.057MPa; for carton B, the low-position suction nozzle 1 is -0.060MPa, the low-position suction nozzle 2 is -0.058MPa, and the high-position suction nozzle is -0.059MPa. All pressure values ​​are within the set value of 0.06MPa ± 0.003MPa, and the fluctuation meets the requirements. Therefore, the adsorption is considered successful, and there is no need to trigger the retry logic.

[0115] S4. After successful adsorption, the robotic arm is controlled to perform lifting, moving, lowering and releasing the grasped object. During this process, dynamic anti-sway control and dynamic interference detection are performed based on the dynamic control parameters.

[0116] After successful adsorption, the robotic arm performs lifting and movement operations according to the path and dynamic parameters planned in step S2, and performs dynamic anti-sway and interference detection through attitude gyroscope and 3D vision during the process. The specific implementation process is as follows:

[0117] S41 monitors the swing amplitude in real time during vertical lifting, and triggers PID anti-sway control when the swing exceeds the preset swing threshold.

[0118] The robotic arm begins vertical lifting, with the initial acceleration being the lifting acceleration az calculated in step S23, and the target speed set to a preset low speed (0.1 m / s in this embodiment). During the lifting process, the attitude gyroscope collects the angular velocity and acceleration data of the carton in real time, and calculates the horizontal swing amplitude A of the carton (taking the maximum linear displacement of the carton's edge) through integration. When A exceeds the preset swing threshold (5 cm in this embodiment), PID anti-sway control is triggered (PID parameters are set in step S24). The PID controller calculates the control quantity according to the deviation between the actual swing amplitude and the threshold, using proportional, integral, and derivative rules, and adjusts the lifting speed and end-effector attitude of the robotic arm accordingly (fine-tuning around the swing axis in the opposite direction) to suppress the swing. When the swing amplitude falls back below the threshold, the original motion parameters are restored.

[0119] In this embodiment, the robotic arm lifts the carton vertically with an acceleration of 0.33 m / s² and a speed of 0.1 m / s. Initially, the carton does not sway significantly; the angle change obtained by integrating the angular velocity measured by the gyroscope is less than 1°, corresponding to a sway amplitude of less than 1.7 cm, which is below the 5 cm threshold, and the PID controller is not triggered.

[0120] S42. If a potential interference risk is detected during the lifting process, the lifting is paused and the D*Lite algorithm is triggered for local path replanning; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered.

[0121] During the robotic arm's movement, the 3D main camera continuously acquires scene point clouds at a frequency no less than a preset frequency (10 fps in this embodiment), and fuses them with the real-time posture of the current cardboard box (provided by the gyroscope). By comparing the real-time point cloud with the point cloud before grasping, the system detects whether there are any new obstacles (such as other cardboard boxes moving or personnel entering) or interference that may be caused by sudden changes in the cardboard box's posture. If a potential interference risk is detected, the system immediately pauses the lifting process, triggers the D*Lite algorithm to perform local path replanning, generates a new lifting path, and then continues execution; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered.

[0122] In this embodiment, the dynamic interference detection module is initialized, and the point cloud update frequency is set to 10 fps. During the lifting and movement of the robotic arm, the 3D main camera continuously collects point clouds and compares them with the point clouds before grasping. No new obstacles are detected. At the same time, the attitude gyroscope data of cartons A and B are stable and without sudden changes, so there is no risk of interference.

[0123] S43. If a potential interference risk is detected during horizontal movement, the movement is paused, the D*Lite algorithm is triggered to perform local path replanning, and a new obstacle avoidance path is generated before execution continues; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered.

[0124] After the robotic arm is vertically raised to a preset safe height (at least 30 cm from its original position in this embodiment), it begins to move horizontally. The acceleration is the moving acceleration axy calculated in step S23, and the target velocity is set to a preset value (0.2 m / s in this embodiment). During the movement, the 3D main camera continues to collect point cloud data and fuse attitude data at a frequency of no less than 10 fps, detecting new obstacles or sudden attitude changes. If a potential interference risk is detected, the system immediately pauses the movement, triggers the D*Lite algorithm to perform local path replanning, generates a new obstacle avoidance path, and continues execution; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered.

[0125] In this embodiment, after being raised to a safe height of 30 cm, the robotic arm moves horizontally to the placement point (above the conveyor belt) with an acceleration of 0.27 m / s² and a speed of 0.2 m / s. During the movement, the 3D main camera captures a point cloud at 10 fps and compares it with the point cloud before grasping; no new obstacles are detected. At the same time, the secondary detection camera captures real-time images of the cardboard box at 25 fps. The images show that the cardboard box is in a stable posture and there is no risk of it falling off or breaking.

[0126] S44, descend and release the grasped object.

[0127] After moving to the placement point, the robotic arm begins to descend vertically and place the cardboard box in the predetermined position. Once placement is complete, the control system closes the vacuum valve of the corresponding suction nozzle. After the pressure inside the suction nozzle returns to zero, the robotic arm rises and leaves, completing this gripping cycle.

[0128] In this embodiment, after moving to the placement point, the robotic arm descends vertically to place the cardboard box, first closing the vacuum valve, and then rising and leaving after the pressure returns to zero.

[0129] S5 performs post-grab integrity verification after the object is placed and records the entire grabbing process data for subsequent closed-loop optimization of strategies.

[0130] After the cardboard boxes are placed, the system performs an integrity verification after capture and records all data from this capture for subsequent strategy optimization. The specific implementation process is as follows:

[0131] S51 captures images of the cardboard box using a secondary vision detection camera, compares them with the image before grasping, and identifies newly added damaged areas; if the newly added damaged area exceeds a preset area threshold or the cardboard box is detected to have fallen off, the grasping is deemed to have failed.

[0132] The secondary detection camera continuously captures images of the cardboard box during movement (or takes images specifically after placement), compares these images with the images taken before grasping, and uses image difference algorithms or deep learning semantic segmentation models to identify whether there are any newly added damaged areas on the surface of the cardboard box. In this embodiment, the preset area threshold is 500 mm². If the newly added damaged area exceeds this value or if the cardboard box is detected to have detached, the grasping is deemed to have failed.

[0133] S52 simultaneously monitors nozzle pressure fluctuations and carton swing amplitude. If the pressure fluctuation exceeds the preset pressure fluctuation threshold or the swing amplitude exceeds the preset emergency swing threshold, the gripping is also deemed to have failed.

[0134] Throughout the gripping process, the system continuously monitors nozzle pressure fluctuations and carton sway amplitude. In this embodiment, the preset pressure fluctuation threshold is ±0.005 MPa, and the preset emergency sway threshold is 8 cm. If any nozzle pressure fluctuation exceeds this threshold or the sway amplitude exceeds this threshold, the gripping is deemed to have failed.

[0135] S53: If the capture fails, perform emergency placement and record the failure information; if the capture is successful, record the entire process data for closed-loop optimization.

[0136] If the grasping failure is determined, the system immediately performs emergency placement: the robotic arm moves the cardboard box to a preset safe area (such as an open space on the ground) at a preset low speed (≤0.1m / s in this embodiment), then releases the cardboard box and records the failure information; if the cardboard box has fallen off, it is recorded directly.

[0137] If the grab is successful, the system records complete data for this grab, including: carton ID, grab time, number of combinations, adsorption success rate (pressure values ​​of each nozzle), maximum swing amplitude, number of retries, and whether there is any damage. This data is stored in the database for subsequent reinforcement learning iterations and optimizations. The system dynamically adjusts the grab strategy parameters based on historical data. For example, if retries are frequent in a certain scenario, the number of combinations is automatically reduced or the nozzle pressure is adjusted; if the damage rate in a certain area is high, the avoidance distance in that area is increased, etc.

[0138] After each grasping action is completed, the system re-executes steps S1 to S5 based on the updated scene data to perform a new round of path planning and dynamic control, thereby achieving continuous grasping. This dynamic update mechanism is the foundation of closed-loop optimization control.

[0139] In this embodiment, the image captured by the secondary detection camera during the movement is compared with the image before grasping. No new damage is detected (the area of ​​new damage is 0, less than 500 mm²). Throughout the process, the maximum fluctuation of the suction nozzle pressure is ±0.002 MPa (less than ±0.005 MPa), and the maximum swing amplitude is 3 cm (less than 8 cm), indicating successful grasping. The system records the grasping data: boxes A and B successfully grasped, combination quantity 2, adsorption success rate 100%, maximum swing 3 cm, 0 retries, no damage. The data is stored in the database.

[0140] This method has demonstrated excellent performance in practical tests:

[0141] Efficiency: The total time from visual acquisition to placement of two cardboard boxes in a single grab is 3.2 seconds (0.15s for visual processing, 0.2s for planning, 2.8s for motion, and 0.05s for other processes), with a grabbing cycle of 1.6 seconds per box.

[0142] Accuracy: The two cartons were placed with a positional deviation of 2mm and 3mm respectively after being gripped, which meets the positioning requirement of ±5mm; there was no slippage or damage during the gripping process.

[0143] Stability: The maximum swing amplitude during the lifting process is 3cm, which is below the 5cm threshold; when the combined center of gravity shifts by 1cm, the robotic arm operates smoothly without any instability.

[0144] Robustness: The nozzle offset layout and differentiated pressure settings ensure 100% adsorption success rate even with complex carton tilts of 18° and 15.7°.

[0145] Example 2

[0146] This embodiment provides a collaborative grasping control system for a multi-nozzle robotic arm. This system is deployed on a central control system and uses software modules to collaboratively control all hardware devices, thereby realizing a collaborative grasping control method for a multi-nozzle robotic arm. The system includes the following software modules deployed on the central control system:

[0147] Path planning and anti-sway module: Used for static path pre-planning and calculating dynamic control parameters during the grasping process. This module incorporates the A* algorithm, a hybrid algorithm including the RRT* and D*Lite algorithms, and supports path planning in both unobstructed and obstructed scenarios; it also incorporates a center of gravity compensation algorithm and a PID anti-sway control algorithm, which can dynamically adjust motion parameters based on the weight distribution and real-time swaying status of the combined carton.

[0148] The instruction generation module generates control instructions to drive the hardware to perform the grasping operation and processes feedback signals in real time. This module includes a robotic arm motion control submodule, a nozzle control submodule, a stroke adjustment submodule, a retry control submodule, a dynamic response control submodule, and an integrity verification control submodule.

[0149] Feedback detection module: Used for real-time monitoring by calling feedback data from various hardware sensors, including secondary visual integrity monitoring, carton posture sway monitoring, and nozzle pressure monitoring.

[0150] Data storage and optimization module: Used to record data throughout the entire process and to achieve iterative optimization of the system, including a scenario-strategy-effect database and a reinforcement learning optimization module.

[0151] Example 3

[0152] This embodiment provides an electronic device, which is the central control system described in Embodiments 1 and 2, and is an industrial-grade PLC controller. The electronic device includes: A memory for storing computer programs, the memory including non-volatile memory and random access memory, the non-volatile memory for storing system programs and the random access memory for caching runtime data.

[0153] The processor is used to execute the program stored in the memory to implement the steps of the above-mentioned collaborative grasping control method for a multi-nozzle robotic arm.

[0154] For details on the specific implementation of each step and related explanations, please refer to the aforementioned embodiment of a collaborative grasping control method for a multi-nozzle robotic arm, which will not be repeated here.

[0155] The electronic device serves as a central control system, communicating with the following hardware devices via an industrial bus to collaboratively complete the grasping operation:

[0156] 1) 3D vision sensing hardware, including: The 3D structured light main camera is deployed on the end flange of the robotic arm to acquire raw point cloud data and two-dimensional RGB images of the stacked scene; A secondary inspection camera is used to capture real-time images of the cardboard box during or after the grasping process for integrity verification.

[0157] 2) Reconfigurable multi-nozzle end effector hardware, including: A 4×4 matrix-style independently controlled nozzle array, with each nozzle as an independent hardware unit, possessing independent start / stop control and vacuum adsorption functions; A pressure sensor integrated into each nozzle end is used to monitor the adsorption pressure in real time; The stroke adjustment mechanism, controlled by the electronic device, enables precise adjustment of the nozzle stroke.

[0158] 3) The redundant robotic arm hardware is a six-axis high-speed robotic arm, whose end flange is fixedly connected to the reconfigurable multi-nozzle end effector; the robotic arm integrates a force control sensor and an attitude gyroscope. The force control sensor is used to detect sudden changes in force when the nozzle contacts the surface of the carton, and the attitude gyroscope is used to monitor the swing amplitude of the carton in real time during the gripping process.

[0159] The electronic device generates control instructions by running a computer program in its memory, driving the aforementioned hardware devices to work together to achieve a complete process from visual acquisition to grasping. The processor can be a general-purpose processor, including a central processing unit, a network processor, etc.; it can also be a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices.

[0160] Example 4

[0161] This embodiment provides a computer-readable storage medium, which is a non-volatile memory embedded in or accessible to the central control system described in Embodiment 3. A computer program is permanently stored on this medium, burned into it in firmware form, and deeply coupled with the hardware as part of the central control system. When the computer program is executed by a processor, it implements the steps of a collaborative grasping control method for a multi-nozzle robotic arm. For the specific implementation and explanation of each step of this method, please refer to the aforementioned embodiment of a collaborative grasping control method for a multi-nozzle robotic arm, which will not be repeated here.

[0162] Specifically, the computer-readable storage medium is the flash memory, EEPROM, or solid-state storage chip built into the PLC. The computer program is stored therein in firmware form, loaded into the random access memory after the system is powered on, and executed by the processor. When the computer program is executed by the processor, the processor acts as a central control system, communicating with the 3D vision perception hardware, the reconfigurable multi-nozzle end effector hardware, and the load-redundant robotic arm hardware via a bus. It generates control commands to drive the various hardware devices to work collaboratively, realizing the complete process from vision acquisition to grasping completion.

[0163] The program modules embedded in the computer-readable storage medium include the path planning and anti-sway module, execution instruction generation module, feedback detection module, and data storage and optimization module described in Embodiment 2. Each module is stored in the form of executable code and runs collaboratively.

[0164] The integrated design of the computer-readable storage medium and central control system described in this embodiment ensures high reliability and real-time response performance of the grasping system in industrial environments. The computer-readable storage medium can also be other forms of non-volatile memory, such as SD cards, solid-state drives, read-only memory, etc., as long as the computer program stored within it can be executed by a processor to implement the method of this invention.

[0165] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for cooperative grasping control of a multi-nozzle robot, characterized by, include: Obtain a grasping scheme, which includes the grasping point position, the nozzle layout, the suction pressure setting, and the grasping posture of the robotic arm end effector; Based on the obstacle information of the current scene and the grasping points in the grasping scheme, static path pre-planning is performed, and dynamic control parameters during the grasping process are calculated to form the planned path; the dynamic control parameters include center of gravity compensation parameters and dynamic anti-sway control parameters. The robotic arm is controlled to move to the gripping point along the planned path, perform the adsorption operation, and monitor the adsorption status in real time through a pressure sensor. If the adsorption status does not meet the standard, a preset retry mechanism is executed. After successful adsorption, the robotic arm is controlled to perform lifting, moving, lowering and releasing the grasped object. During this process, dynamic anti-sway control and dynamic interference detection are performed based on the dynamic control parameters. After the object is placed, perform post-grab integrity verification and record the entire grabbing process data for subsequent closed-loop optimization of strategies.

2. The method according to claim 1, characterized in that, The static path pre-planning includes: The path planning algorithm is selected based on the occlusion situation: in scenarios with no occlusion or low occlusion, the A* algorithm is used to perform fast path search in joint space; in scenarios with occlusion ahead or multiple occlusions, the RRT* algorithm and D*Lite algorithm are combined to generate a bypass path, ensuring that the bypass distance is at least the preset safe distance. If path planning fails, the failure handling process is executed: record the failure information, relax the parameters and retry, and terminate the current crawling if there are no more candidate solutions.

3. The method according to claim 1, characterized in that, The calculation method for the center of gravity compensation parameter includes: Estimate the weight of each carton based on its dimensions and material, and calculate the coordinates of the combined center of gravity. Calculate the offset of the combined center of gravity relative to the gripping point at the end of the robotic arm; The lifting acceleration of the robotic arm during the lifting phase and the moving acceleration during the moving phase are adjusted according to the offset: when the offset is less than or equal to the preset offset threshold, the acceleration is calculated according to the linear decreasing rule; when the offset exceeds the preset offset threshold, the minimum safe acceleration is used; when there is no offset, the default acceleration is used.

4. The method according to claim 1, characterized in that, The dynamic anti-sway control parameters include the swing amplitude threshold and the proportional, integral, and derivative coefficients of the PID controller; The swaying amplitude of the carton is monitored in real time by a posture gyroscope. When the swaying exceeds the preset swaying threshold, the PID controller calculates the output based on the error and adjusts the movement speed and end-effector posture of the robotic arm to suppress the swaying.

5. The method according to claim 1, characterized in that, The adsorption operation and retry mechanism specifically include: The robotic arm approaches at low speed to bring the suction nozzle into contact with the surface of the carton, opens the vacuum valve to establish suction pressure, and holds the pressure for a preset time. The adsorption pressure is detected by a pressure sensor integrated into the nozzle end, and the pressure value is required to fluctuate within the set target pressure ±0.003MPa and not be lower than 90% of the set value; If the adsorption pressure is not up to standard, perform a preset number of retries: the first retries fine-tune the nozzle stroke and the robotic arm contact position; the second retries add an auxiliary nozzle; the final retries abandon the combined gripping, split it into single-box gripping, regenerate the single-box gripping plan, and execute it sequentially.

6. The method according to claim 1, characterized in that, The dynamic anti-sway control and dynamic interference detection include: During the vertical lifting process, the swing amplitude is monitored in real time, and PID anti-sway control is triggered when the swing exceeds the preset swing threshold. During the lifting and movement process, the scene point cloud is continuously collected by the 3D main camera and fused with the current real-time posture of the cardboard box to detect new obstacles or sudden changes in posture. If a potential interference risk is detected during the elevation process, the elevation is paused, and the D*Lite algorithm is triggered for local path replanning; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered. If a potential interference risk is detected during horizontal movement, the movement is paused, the D*Lite algorithm is triggered to perform local path replanning, and a new obstacle avoidance path is generated before execution continues; if the risk cannot be avoided, an emergency stop is triggered and an alarm is triggered.

7. The method according to claim 1, characterized in that, The post-fetch integrity verification includes: The image of the cardboard box is captured by a secondary vision inspection camera and compared with the image before the grasping process to identify newly added damaged areas. If the newly added damaged area exceeds the preset area threshold or the cardboard box is detected to have fallen off, the grasping process is deemed to have failed. Simultaneously monitor the pressure fluctuation of the suction nozzle and the swing amplitude of the carton. If the pressure fluctuation exceeds the preset pressure fluctuation threshold or the swing amplitude exceeds the preset emergency swing threshold, the gripping is also determined to be in failure. If the capture fails, perform emergency placement and record the failure information; if the capture is successful, record the entire process data for closed-loop optimization.

8. A collaborative gripping control system for a multi-nozzle robotic arm, characterized in that, include: The path planning and anti-sway module is used to perform static path pre-planning and calculate dynamic control parameters during the grasping process; The instruction generation module is used to generate control instructions to drive the hardware to perform the grasping operation and to process feedback signals in real time. The feedback detection module is used for real-time monitoring by calling the feedback data from various hardware sensors; The data storage and optimization module is used to record data throughout the entire process and to achieve iterative optimization of the system.

9. A collaborative gripping control device for a multi-nozzle robotic arm, characterized in that, include: The 3D vision perception system consists of an ultra-high dynamic range (HDR) structured light 3D main camera and a secondary detection camera, used to acquire raw point cloud data and two-dimensional RGB images of stacked scenes; The reconfigurable multi-nozzle end effector is a 4×4 matrix-type independently controlled nozzle array, integrating pressure sensors and stroke adjustment mechanisms; A load-redundant six-axis high-speed robotic arm that integrates force control sensors and attitude gyroscopes; Industrial-grade PLC controller; The central control system is connected via a bus to the 3D vision perception system, the reconfigurable multi-nozzle end effector, the robotic arm, and the PLC controller. The central control system includes: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images