Multi-arm collaborative control method and system based on virtual decomposition and hierarchical perception

Abstract

The application discloses a multi-arm cooperative control method and system based on virtual decomposition and hierarchical perception, relates to the field of intelligent agricultural equipment and robot bottom control technology, and comprises the following steps: firstly, a multi-objective genetic algorithm is used to optimize geometric parameters, so that the multi-arm workspace overlap rate is reduced from the source of structure; secondly, a global-local three-layer perception architecture is constructed, a weighted Voronoi diagram which fuses local Jacobian determinants is used for task division, and offline decoupling is realized; when the mechanical arm approaches a safety threshold in the free space, a virtual cut point without physical contact is constructed between the two arms, virtual power flow is calculated and converted into an additional control torque of the bottom servo motor, and the mechanical arm is forced to generate a repulsive movement away from the global stability constraint; finally, dynamic arbitration of the bottom dynamics closed loop and end impedance control are combined. The application breaks through the limitation of traditional anti-collision theory, and realizes high-frequency, stable, deadlock-free and flexible cooperative operation of multiple independent mechanical arms in a limited space.

Description

Technical Field

[0001] This invention relates to the field of agricultural intelligent equipment and robot underlying control technology, specifically to a multi-arm collaborative control method and system based on virtual decomposition and hierarchical perception. It is particularly suitable for control scenarios such as orchard multi-arm harvesting robots that need to achieve safe collaborative operation and spatiotemporal decoupling of multiple independent robotic arms in complex environments that are unstructured, highly occluded and spatially limited. Background Technology

[0002] With the development of intelligent modern agriculture, multi-robotic arm collaborative operation systems have shown great application potential in agricultural scenarios such as orchard harvesting and phenotypic detection due to their high concurrency processing capabilities. However, the real agricultural canopy environment is a typical unstructured environment, characterized by high shading, dense distribution of targets, wind load disturbance, and extremely limited operating space. When multiple high-degree-of-freedom robotic arms operate concurrently at high frequency in the same confined space, kinematic interference and dynamic collisions are highly likely to occur.

[0003] Existing multi-robotic arm collaborative control and obstacle avoidance technologies generally face the following insurmountable technical bottlenecks and shortcomings when dealing with such complex scenarios:

[0004] First, the control framework lacks the ability to decouple the underlying physical space of multiple independent robotic arms, making traditional obstacle avoidance algorithms prone to system instability. Traditional centralized control architectures face the curse of computational dimensionality when dealing with the inverse dynamics of multiple arms; while existing distributed control schemes, such as obstacle avoidance control based on traditional artificial potential fields, often generate geometric repulsive forces independent of the underlying servo control loop, easily leading to high-frequency resonance with the underlying dynamics, causing the robotic arm to produce local minimum oscillations or directly become unstable and stop in a narrow space. In recent years, although virtual decomposition control theory has been introduced into the robotics field, such as patent publication number CN118123833A, a virtual decomposition control method for a flexible hydraulic robotic arm considering nonlinear deformation, its application is limited to decoupling between joints within a single arm, or for the distribution of internal forces when two arms are jointly clamping the same rigid load. Existing VDC technologies all rely on real physical contact points to transfer energy, and there is no effective solution in the industry to extend VDC theory to the collaborative decoupling of multiple independent robotic arms in free space without physical connections, thus failing to eradicate spatial conflicts of multiple arms from the underlying dynamics level.

[0005] Secondly, the disconnect between the geometric topology design of the mechanical system and the underlying control algorithm leads to an overload on the algorithm's computational power. Existing technologies typically treat the structural design of the robotic arm and the development of the control algorithm as two independent processes. During the multi-arm layout phase, the base spacing and link ratio are often set based on experience, resulting in an abnormally high overlap rate in the workspace of the multi-arm, typically exceeding 40%. This inherent physical defect of excessive overlap transfers all the collision avoidance pressure to the backend control algorithm. In densely populated work areas, the algorithm needs to frequently call complex interference detection and trajectory replanning modules, greatly consuming the industrial control computer's computational power and significantly increasing the probability of unsolvable trajectories. Currently, there is an urgent need for a method that integrates structural parameters with prior constraints and the posterior closed-loop control system.

[0006] Third, the environmental perception system is limited in scope, and the coupling between the perceived data and the underlying control is insufficient. Existing harvesting robots mostly employ a single global vision system, such as a binocular camera or a single-layer depth camera. In dense canopies, a single perspective is easily affected by sudden changes in lighting and foliage obstruction, creating numerous blind spots. Furthermore, the global data acquired by existing perception systems is often only used for offline target recognition and static path planning at the front end. When the robotic arm's end effector cuts into the canopy to perform the grasping action, the system lacks close-range dynamic visual compensation and force feedback, leading to localized blind spots and failing to provide complete, real-time obstacle avoidance decision-making basis.

[0007] Fourth, the macro-level task conflict scheduling and the micro-level low-level motion control are in an open-loop state, which easily leads to dynamic deadlock. Existing multi-task conflict handling mechanisms, such as the robot task conflict handling method disclosed in patent publication number CN120347748A, mostly remain at the macro-level software scheduling level. Their logic is similar to route deduplication for multiple AGVs, with a long response cycle, usually in the hundreds of milliseconds to seconds. When multiple picking robotic arms simultaneously extend into the same dense fruit cluster area, this high-level macro-scheduling cannot form a closed-loop match with the low-level millisecond-level servo control cycle, such as the VDC dynamic control loop. This results in the scheduling command often being issued late at the moment of impending collision. The robotic arms get stuck in a physical deadlock state in a confined space due to mutual avoidance, which in turn triggers the equipment overload protection, seriously reducing the overall work cycle and safety. Summary of the Invention

[0008] The problem this invention aims to solve is that existing multi-arm collaborative operation in complex unstructured environments suffers from inherent defects such as spatial obstacle avoidance algorithms that are prone to dynamic instability, fragmented allocation of hardware and software anti-collision computing power, blind spots in global perception, and lag in dynamic scheduling response that can easily lead to deadlock. This invention provides a multi-arm collaborative control method and system based on virtual decomposition and hierarchical perception to achieve high-frequency, stable, collision-free, and compliant collaborative operation of multiple independent robotic arms in confined spaces.

[0009] To address the above problems, this invention provides a multi-arm cooperative control method based on virtual decomposition and hierarchical sensing, comprising the following steps:

[0010] S1. System Modeling and Parameter Optimization Steps: Based on virtual decomposition control theory, the multi-arm system is virtually decomposed into multiple independent subsystems. The base position, link length ratio and joint limit angle of each arm are used as optimization variables. Multi-objective optimization is carried out with minimizing the overlap rate of the workspace and maximizing the utilization rate of the reachable space as optimization objectives to obtain the optimal geometric parameter configuration of each arm.

[0011] S2, Layered sensor layout and spatiotemporal alignment steps: Construct a global-local three-layer perception architecture. The global perception layer is formed by a depth camera at a fixed position on the robot body, and the local perception layer is formed by a near-field vision sensor and a force sensor at the end of each robotic arm. Spatiotemporal alignment and fusion of multi-source heterogeneous sensor data are then performed.

[0012] S3. Task allocation steps based on kinematic weights: Obtain 3D point cloud data of the work scene and the fruit target position through the global perception layer. Based on the optimal geometric parameter configuration, use a weighted Voronoi diagram to divide the workspace of the multi-robotic arm into task areas and assign a target picking point sequence to each robotic arm.

[0013] S4. Robotic arm motion planning and real-time status monitoring steps: Based on the assigned picking point sequence, plan the desired trajectory for each robotic arm, and obtain the pose information and environmental dynamic information of each robotic arm in real time through the three-layer perception architecture.

[0014] S4. Steps for robotic arm motion planning and real-time status monitoring: Plan the desired trajectory of the end effector for each robotic arm, and obtain the pose information and working environment information of each robotic arm in real time through the global perception layer and the local perception layer.

[0015] S5. Free Space Virtual Tangent Point Construction and Cooperative Decoupling Steps: Based on the real-time pose prediction, the shortest distance between each robotic arm is predicted. When the shortest distance is less than or equal to a preset safety distance threshold, a virtual tangent point without physical contact is constructed between the points where the shortest distances of the conflicting robotic arms are located. Based on the virtual tangent point, the virtual power flow transmitted along the virtual tangent point is calculated using a virtual decomposition control framework. By adjusting the virtual stiffness parameters and virtual damping parameters of the virtual power flow, a trajectory correction amount is generated, and the virtual power flow is converted into an additional control torque for the underlying servo motor. This forces the robotic arm to generate a divergent motion tendency in Cartesian space to achieve spatial decoupling. The divergent motion is constrained by the Lyapunov global asymptotic stability constraint of the virtual decomposition control framework.

[0016] S6. Dynamic task scheduling steps based on the underlying dynamic closed loop: When spatial repulsion causes multiple robotic arms to generate dynamic conflict deadlock in the same working area at the same time, the task timing planner performs multi-dimensional weight arbitration based on the collision risk level, current task priority and remaining workload, and dynamically adjusts the task execution sequence of the target robotic arm.

[0017] S7. Compliant operation execution steps: Each robotic arm performs picking according to the corrected trajectory and adjusted timing. During the fruit contact stage, the force sensor monitors the end contact force in real time. When the contact force exceeds the preset threshold, it switches to compliant force control mode based on impedance control strategy until the picking task is completed.

[0018] Preferably, the multi-objective optimization in step S1 specifically includes: employing the NSGA-II multi-objective genetic algorithm, using the workspace overlap rate of each robotic arm as the first minimization objective function, and the reachable space utilization rate of each robotic arm as the second maximization objective function, using the base position coordinates, the ratio of proximal to distal link lengths, and joint limit angles as chromosome encoding variables, performing non-dominated sorting and crowding distance calculation, obtaining the Pareto optimal solution set, and outputting the geometric parameter configuration. Using the NSGA-II algorithm for multi-objective optimization, by introducing a non-dominated sorting mechanism and crowding distance calculation, can maintain population diversity without relying on manually set absolute weights. This avoids the blindness of traditional experience-based settings and achieves a true Pareto optimal balance between workspace overlap rate and robotic arm reachable space utilization rate at the physical structure level.

[0019] Preferably, the optimal geometric parameter configuration satisfies the following physical boundary constraints: the distance between the base positions of the multiple robotic arms is not less than 1.2 times the maximum reach of the robotic arm; the length ratio of the proximal link to the distal link is limited to the range of 1.5:1 to 2.0:1; the joint limiting angle retains an internal safety margin of ±15° based on the standard limiting angle; ensuring that the workspace overlap rate of the multi-robotic arm system is less than 10%, and the reachable space utilization rate of the core working area is not less than 90%. By limiting the specific ratio of base spacing and proximal / distal link ratio, singular shapes are pushed away from the core working area to the maximum extent from the physical source, reducing the sweeping volume of the elbow joint when passing through branches and leaves. This rigid physical envelope constraint forces the workspace overlap rate of the multi-arm system to be compressed to below 10%, reducing the probability of triggering the anti-collision algorithm that the back-end control system needs to call frequently by more than 90%, achieving a breakthrough in computing power release.

[0020] Preferably, the global-local three-layer perception architecture described in step S2 is as follows: the global layer consists of at least four fixed-depth cameras, each with a field of view of not less than 120° and a detection distance of not less than 2m, which, when combined, form a static environmental map covering the work area; the intermediate layer consists of near-field vision sensors installed at the ends of each robotic arm, used to implement eye-to-hand visual servo compensation within a preset near-field working range when the global line of sight is obstructed; the local layer consists of force sensors installed at the ends of each robotic arm with a force resolution of not less than 0.1N, used to trigger the underlying force control feedback when the actuator at the end of the robotic arm makes physical contact. The global-local three-layer perception architecture achieves a smooth transition and complementarity in spatiotemporal resolution from macroscopic scene reconstruction to microscopic force feedback. The global camera provides macroscopic prior terrain, the near-field vision sensor implements servo compensation through eye-to-hand, overcoming the system error caused by severe canopy occlusion, while the force sensor compensates for the resolution limit of the optical sensor at the moment of contact. The three-layer fusion completely solves the problem of perception islands in traditional distributed control, where local visibility is achieved but the overall perception is blind.

[0021] Preferably, step S3 involves using a weighted Voronoi diagram to divide the workspace of the multi-arm robot into task regions. Specifically, this includes: integrating the reachability and dexterity indices of each robot in different orientations as kinematic weights for generating Voronoi cell boundaries; if the target fruit is located within the physically overlapping area of ​​the multi-arm robot, calculating the local Jacobian determinant of the picking posture required for each robot to reach the fruit, and assigning the target fruit to the Voronoi cell of the robot with the optimal Jacobian determinant, i.e., the best energy consumption and singularity margin. This preferred solution overcomes the drawbacks of traditional task allocation based solely on Euclidean distance. The weighted Voronoi partitioning based on local Jacobian determinants not only considers the target distance but also evaluates the joint energy consumption and singularity margin required for the robot to reach the fruit. This causes the overall state space of the multi-arm system to degenerate into the sum of multiple nearly linear subspaces during the offline planning stage, achieving initial interference decoupling from the task source.

[0022] Preferably, step S5 involves calculating the virtual power flow transmitted along the virtual tangent point and converting it into additional control torque for the underlying servo motor. The virtual power flow is defined as the inner product of the linear / angular velocity error vector and the force / torque error vector at the virtual tangent point. As the shortest distance between the two robotic arms approaches the danger distance threshold, the virtual stiffness and virtual damping parameters are nonlinearly increased, causing a sharp increase in the virtual power flow. Based on the Newton-Euler equations, this surge in virtual power flow is inversely mapped to additional torque commands for each robotic arm joint space via the transpose of the Jacobian matrix, and then superimposed onto the current loop of the servo driver. This preferred solution creatively transforms the abstract geometric anti-interference repulsion force into a physical torque executable by the underlying Newton-Euler equations. By nonlinearly amplifying the virtual power flow at the virtual tangent point and inversely mapping it to the joint space via the transpose of the Jacobian matrix, the obstacle avoidance strategy is directly embedded in the underlying dynamic energy flow equations. This not only ensures that the anti-collision repulsion force does not exceed the physical tolerance limit of the motor, but also avoids high-frequency resonance between the upper-level geometric design and the lower-level servo closed loop.

[0023] Preferably, the rules and parameter restrictions for the multi-dimensional weighted arbitration in step S6 are as follows: arbitration intervention is carried out in descending order of weight, prioritizing collision risk level, task priority, and minimum remaining workload; the collision risk level is divided into high, medium, and low levels based on the ratio of the shortest distance between arms to the safety distance threshold; when the collision risk level is high, the system unconditionally blocks the trajectory of the suboptimal arm; the arbitration response and scheduling instruction issuance delay of the task timing planner are strictly controlled to be less than 50ms to match the underlying servo control cycle and prevent physical collisions. This multi-dimensional descending arbitration mechanism, prioritizing collision risk and considering task margin, and strictly controlling the arbitration instruction issuance delay to within 50ms at the hardware level, allows the macro-level task timing scheduling to perfectly align with the micro-level VDC high-frequency servo cycle, effectively balancing the system's work rhythm and completely eliminating physical deadlock caused by delayed instruction issuance.

[0024] Preferably, the impedance control strategy for entering the compliant force control mode in step S7 specifically includes: adjusting the apparent mass, spring stiffness, and damping characteristics of the robotic arm's end effector in the physical contact direction to absorb excess contact energy and generate a compliant yielding displacement, thus limiting the contact force to a preset safe range that does not damage the fruit tree or the reducer. The end effector impedance control strategy can instantly adjust the apparent mass and stiffness of the robotic arm in the event of unexpected physical contact, transforming a rigid collision into a compliant yielding action like a spring. This not only effectively protects the fragile precision reducer inside the robotic arm from transient overload impacts but also greatly reduces mechanical tearing damage to the fruit tree during unstructured harvesting.

[0025] Preferably, a multi-robotic arm collaborative control system based on virtual decomposition control and hierarchical sensor fusion is applied to multi-arm collaborative harvesting in unstructured agricultural environments, comprising: at least two independent robotic arm bodies; a global-local three-layer perception module deployed at the global, end-effector, and gripper positions, including a depth camera, a near-field vision sensor, and a force sensor; and a control and data processing module, which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the operation steps of the multi-robotic arm collaborative control method described above.

[0026] Preferably, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the multi-arm cooperative control method based on virtual decomposition and hierarchical perception described above.

[0027] Compared with the prior art, the present invention achieves the following beneficial technical effects:

[0028] This invention achieves virtual decomposition control, decoupling the underlying dynamics from physical contact to free space, thus ensuring the global asymptotic stability of the obstacle avoidance process. Breaking away from the traditional VDC theory's limitation to only single-arm internal joints or physically rigidly connected dual arms, this invention proposes for the first time to construct a virtual tangent point without physical contact between robotic arms about to collide in free space. By adjusting the virtual power flow transmitted along this virtual tangent point and converting it into additional torque for the underlying servo motor, physical repulsion is generated between the robotic arms. This obstacle avoidance method fully incorporates geometric repulsion force into the VDC's dynamic equations, completely eliminating the high-frequency servo resonance problem easily caused by traditional artificial potential field-based external obstacle avoidance methods. Mathematically, it satisfies global asymptotic stability constraints, fundamentally achieving high-frequency, stable collaboration of multiple independent robotic arms in confined spaces.

[0029] This invention achieves joint dimensionality reduction from prior structural constraints to posterior control loops, significantly reducing the computational load on collision avoidance algorithms. It overcomes the traditional disconnect between mechanical structure design and control algorithms. By introducing the NSGA-II multi-objective genetic algorithm, core physical boundary constraints such as base position spacing and link length ratios are strictly limited. This design, which optimizes kinematic performance upfront, constructs an anti-interference firewall at the physical source, forcibly compressing the overlap rate of the multi-arm system's workspace while maintaining over 90% utilization of the core area's reachable space. This effect exponentially reduces the interference state space that the backend control system needs to process, greatly freeing up the industrial control computer's collision avoidance computational power.

[0030] This invention constructs an all-weather, blind-spot-free, cross-scale perception closed loop, eliminating spatial conflicts between arms at the task source. Addressing the pain point of severe canopy shading in agriculture, this invention constructs a three-layer heterogeneous perception architecture: global depth, local vision, and end-effector force sensing, achieving seamless coverage of both macroscopic environmental contours and microscopic physical contacts. More importantly, this solution does not employ simple distance polling in the task allocation phase, but innovatively uses a weighted Voronoi diagram that integrates accessibility and dexterity, along with a local Jacobian determinant, for task partitioning. This spatial partitioning, deeply integrating kinematic performance, logically isolates multi-arm operation trajectories during offline planning, minimizing the probability of overlapping and boundary-crossing operations.

[0031] This invention achieves a spatiotemporal closed loop of microsecond-level underlying dynamics and macroscopic task scheduling, possessing extremely high anti-deadlock and compliant protection capabilities. Addressing the deficiency of existing macroscopic scheduling response lag leading to equipment deadlock, this invention deeply binds the task timing planner with the underlying VDC servo control cycle, strictly controlling the arbitration intervention delay to within 50ms. When a high-risk collision is detected, not only is virtual repulsion generated in the spatial domain, but also the suboptimal arm trajectory is promptly truncated in the temporal domain, achieving true spatiotemporal decoupling. Furthermore, this invention introduces an impedance control strategy based on force feedback at the end-effector contact stage, endowing the robotic arm with physical passive compliance, effectively avoiding impact damage to the precision reducer and mechanical damage to the fruit tree itself when accidentally grasping thick branches, greatly improving operational safety and robustness in unstructured agricultural environments. Attached Figure Description

[0032] Figure 1 This is the overall architecture and hierarchical perception topology diagram of the multi-robotic arm collaborative control system of the present invention;

[0033] Figure 2 This is a flowchart illustrating the multi-objective optimization of system structure parameters based on the NSGA-II algorithm of this invention.

[0034] Figure 3 This is a schematic diagram of the weighted Voronoi space partitioning based on the local Jacobian determinant of the present invention;

[0035] Figure 4 This is a schematic diagram illustrating the dynamic principle of constructing a virtual tangent point and transferring virtual power flow (VPF) in free space according to the present invention. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be emphasized that the following description is merely exemplary and is not intended to limit the scope and application of the present invention.

[0037] This embodiment provides a multi-arm cooperative control method based on virtual decomposition and hierarchical perception, applied to an orchard harvesting robot system containing two or more independent robotic arms. This method covers the entire collaborative process from hardware physical dimensionality reduction to software-level dynamic decoupling. The specific execution steps are as follows:

[0038] S1. System modeling and parameter optimization based on NSGA-II algorithm, and physical anti-interference dimensionality reduction.

[0039] The kinematic coupling complexity of multi-arm robotic systems increases exponentially with spatial overlap. This step introduces prior constraints during the physical design phase and employs the NSGA-II fast non-dominated sorting genetic algorithm with an elite retention strategy for geometric optimization.

[0040] Let the chromosome-encoded variables of the robotic arm system parameters be:

[0041] ,

[0042] Where pi is the three-dimensional coordinate of the base of the i-th robotic arm, λi is the ratio of the proximal and distal link lengths constrained between 1.5 and 2.0, and Θlimit is the physical limit angle of the joint.

[0043] The first optimization objective, namely minimizing the workspace overlap rate, is defined as:

[0044]

[0045] Where Vi(X) is the volume of the reachable space of a single arm calculated by random sampling of Monte Carlo forward kinematics, and Voverlap(X) is the Boolean intersection volume of the reachable spaces of multiple robotic arms.

[0046] Define the second optimization objective as maximizing achievable space utilization, i.e., achieving effective canopy coverage for the target area:

[0047]

[0048] Where Vtarget is the standard bounding box volume of the fruit tree canopy given by agronomic prior data.

[0049] The NSGA-II algorithm generates offspring by simulating binary crossover and polynomial mutation, performs non-dominated sorting based on the two mutually exclusive objectives mentioned above, and finally outputs the Pareto optimal solution set. In this embodiment, the solution with a base spacing of not less than 1.2 times the maximum arm span is selected, and f1 is forcibly compressed to below 10%, thereby filtering out more than 90% of potential spatial conflicts at the physical source.

[0050] S2, Layered Sensor Layout and Spatiotemporal Alignment

[0051] Obtain macroscopic three-dimensional point cloud data through a global depth camera. Install a force sensor with a resolution of not less than 0.1 N and a near-distance vision sensor at the end of the robotic arm, and achieve the alignment of multi-source heterogeneous data in a unified coordinate system through extended Kalman filtering in the industrial control computer.

[0052] S3. Weighted Voronoi task area division based on kinematic weights and logical interference decoupling

[0053] To avoid frequent cross-region operations of multiple arms, the present invention modifies the traditional Voronoi diagram. Suppose there are M fruit target points in the operation space , and the equivalent virtual centers of the bases of each robotic arm are Si.

[0054] Define the kinematic comprehensive metric weight wi(Tj), which not only considers the distance but also introduces the dexterity metric at the target point:

[0055]

[0056] where Ji(θ) is the local Jacobian matrix of the i-th robotic arm when reaching the point Tj, and α and β are normalized weight coefficients.

[0057] The division rule of the weighted Voronoi cell is: if for all k≠i, wi(Tj) < wk(Tj) is satisfied, then the fruit Tj is assigned to the robotic arm i.

[0058] The physical meaning of this formula is that even if the fruit A is closer to the robotic arm 2, but if the robotic arm 2 is close to the kinematic singularity at this position, the Jacobian determinant approaches 0, consuming a large amount of joint energy consumption, the algorithm will still assign it to the robotic arm 1 with a better posture, achieving trajectory anti-winding from the source of the task.

[0059] S4 - S5. Construction of free space virtual tangent points and collaborative decoupling of VDC

[0060] This is the key to the present invention's breakthrough in the theoretical limitations of the traditional virtual decomposition control VDC. The traditional VDC decomposes the robotic arm into a rigid body and a joint subsystem, and only processes the connection points with physical contact. The present invention extends it to the free space.

[0061] Suppose when the robotic arm A and the robotic arm B are performing tasks, the shortest distance between them in the Cartesian space is calculated in real time through the global / local perception layer , where PA and PB are the closest points on the two arm bodies.

[0062] When , dsafe is the safety distance threshold, preferably 15 cm, the system constructs an invisible mechanical connection link between PA and PB, and defines the PA as the free space virtual tangent point.

[0063] Definition and Nonlinear Regulation of Virtual Power Flow: In VDC theory, the virtual power flow PA is defined as the inner product of the linear / angular velocity error vector and the force / torque error vector. In the free space of this invention, the system virtualizes a nonlinear repulsive spinor. (Includes three-dimensional force and three-dimensional torque).

[0064] The magnitude of the repulsive force is related to the distance d and the approximate relative velocity ḋ:

[0065]

[0066] in, Let PB be the unit direction vector pointing from PA; Kv is the virtual stiffness parameter, and Dv is the virtual damping parameter. The closer the distance, the more nonlinearly the repulsive force increases.

[0067] At the virtual tangent point PA, the virtual power flow pvirtual generated by this repulsive force is:

[0068]

[0069] Where ΔVA is the difference between the expected speed and the actual speed of robotic arm A at the tangent point PA.

[0070] Torque mapping to the underlying servo closed loop:

[0071] This virtual repulsive force is no longer a simple geometric guiding vector, but is directly substituted into the inverse Newton-Euler dynamics equations. For robotic arm A, the additional compensation torque τcomp of its underlying servo controller is calculated as follows:

[0072]

[0073] Here, JVCP(θ) is the local Jacobian matrix from the robot arm base to the virtual tangent point PA. This compensating torque is directly superimposed on the current loop of the joint motor, forcing the robot arm to produce a dynamic divergent motion.

[0074] Stability proof: This invention is based on the VDC framework. The overall Lyapunov candidate function Lsys is defined as the sum of the kinetic energy errors of each rigid body / joint subsystem, plus the energy exchanged through virtual tangent points.

[0075]

[0076] Where si is the integrated velocity error operator of the subsystem, and Mi is the inertial matrix.

[0077] Differentiating Lsys, due to the damping term in the virtual repulsive force Frep Since it always does negative work (consuming the system's approximate kinetic energy), it can be proven that the energy derivative of the entire system satisfies:

[0078]

[0079] As can be seen from the above, no matter how large the virtual repulsive force Frep is, that is, no matter how close the robotic arm is, the dynamic state of the system always converges because it completely follows the passive principle. This ensures, from a strict mathematical perspective, that the multi-arm obstacle avoidance process has L2 and L∞ stability, and completely eliminates the system resonance and divergence loss of control problems that are easily caused by the traditional artificial potential field method.

[0080] S6. Dynamic task temporal decoupling based on multi-dimensional weight arbitration

[0081] When d approaches 0 and becomes trapped in a local minimum dynamic deadlock, the underlying scheduler intervenes. The arbitration function is defined as:

[0082]

[0083] Where R(d) is the collision risk index, Ptask is the current grasping action priority, and Wremain is the number of remaining fruits. The system calculates Ci using a high-frequency cycle of less than 50ms, matching the VDC underlying control frequency, and forcibly intervenes and blocks the motion permission signal of the robotic arm with a smaller Ci, i.e., sets its desired speed. This enables deadlock prevention in the time domain.

[0084] S7, Impedance-controlled end-contact compliant execution

[0085] When the force sensor in the local sensing layer detects that the end contact force Fext exceeds the limit, the impedance control law is applied:

[0086]

[0087] By adjusting the target inertia matrix Mm, damping Bm, and stiffness matrix Km in real time online, the robotic arm's end effector exhibits spring-like compliant physical characteristics. Upon contact with tree branches, it actively yields a displacement Δx, absorbing impact energy and achieving dual protection for both the precision reducer and the fruit tree itself.

[0088] The above embodiments of the present invention illustrate how to comprehensively solve the technical bottlenecks of collaborative operation of multiple independent robotic arms through structural optimization, mathematically weighted space partitioning, and extremely rigorous underlying VDC dynamic control. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-arm cooperative control method based on virtual decomposition and hierarchical sensing, characterized in that, Includes the following steps: S1. System Modeling and Parameter Optimization Steps: Based on virtual decomposition control theory, the multi-arm system is virtually decomposed into multiple independent subsystems. The base position, link length ratio and joint limit angle of each arm are used as optimization variables. Multi-objective optimization is carried out with minimizing the overlap rate of the workspace and maximizing the utilization rate of the reachable space as optimization objectives to obtain the optimal geometric parameter configuration of each arm. S2, Layered sensor layout and spatiotemporal alignment steps: Construct a global-local three-layer perception architecture. The global perception layer is formed by a depth camera at a fixed position on the robot body, and the local perception layer is formed by a near-field vision sensor and a force sensor at the end of each robotic arm. Spatiotemporal alignment and fusion of multi-source heterogeneous sensor data are then performed. S3. Task allocation steps based on kinematic weights: Obtain 3D point cloud data of the work scene and the fruit target position through the global perception layer. Based on the optimal geometric parameter configuration, use a weighted Voronoi diagram to divide the workspace of the multi-robotic arm into task areas and assign a target picking point sequence to each robotic arm. S4. Robotic arm motion planning and real-time status monitoring steps: Based on the assigned picking point sequence, plan the desired trajectory of the end effector for each robotic arm, and obtain the pose information of each robotic arm and the dynamic information of the working environment in real time through the global-local three-layer perception architecture. S5. Free Space Virtual Tangent Point Construction and Cooperative Decoupling Steps: Based on the real-time pose prediction, the shortest distance between each robotic arm is predicted. When the shortest distance is less than or equal to a preset safety distance threshold, a virtual tangent point without physical contact is constructed between the points where the shortest distances of the conflicting robotic arms are located. Based on the virtual tangent point, the virtual power flow transmitted along the virtual tangent point is calculated using a virtual decomposition control framework. By adjusting the virtual stiffness parameters and virtual damping parameters of the virtual power flow, a trajectory correction amount is generated, and the virtual power flow is converted into an additional control torque for the underlying servo motor. This forces the robotic arm to generate a divergent motion tendency in Cartesian space to achieve spatial decoupling. The divergent motion is constrained by the global asymptotic stability constraints of the virtual decomposition control framework. S6. Dynamic task scheduling steps based on the underlying dynamic closed loop: When spatial repulsion causes multiple robotic arms to generate dynamic conflict deadlock in the same working area at the same time, the task timing planner performs multi-dimensional weight arbitration based on the collision risk level, current task priority and remaining workload, and dynamically adjusts the task execution sequence of the target robotic arm. S7. Compliant operation execution steps: Each robotic arm performs picking according to the corrected trajectory and adjusted timing. During the fruit contact stage, the force sensor monitors the end contact force in real time. When the contact force exceeds the preset threshold, it switches to compliant force control mode based on impedance control strategy until the picking task is completed.

2. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 1, characterized in that, The multi-objective optimization described in step S1 specifically includes: using the NSGA-II multi-objective genetic algorithm, taking the workspace overlap rate of each robotic arm as the first minimization objective function, taking the reachable space utilization rate of each robotic arm as the second maximization objective function, and using the base position coordinates, the ratio of the proximal to distal link lengths, and the joint limit angle as chromosome encoding variables, performing non-dominated sorting and crowding distance calculation, obtaining the Pareto optimal solution set, and outputting the geometric parameter configuration.

3. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 2, characterized in that, The optimal geometric parameter configuration satisfies the following physical boundary constraints: the distance between the base positions of the multiple robotic arms is not less than 1.2 times the maximum reach of the robotic arm; the length ratio of the proximal link to the distal link is limited to the range of 1.5:1 to 2.0:1; the joint limiting angle retains an internal safety margin of ±15° based on the standard limiting angle; so that the workspace overlap rate of the multi-robotic arm system is less than 10%, and the reachable space utilization rate of the core working area is not less than 90%.

4. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 1, characterized in that, The global-local three-layer perception architecture described in step S2 is as follows: The global layer consists of at least four fixed depth cameras, each with a field of view of not less than 120° and a detection distance of not less than 2m, which together form a static environmental map covering the work area; the intermediate layer consists of near-field vision sensors installed at the end of each robotic arm, used to implement eye-to-hand visual servo compensation within a preset near-field working range when the global line of sight is obstructed; the local layer consists of force sensors installed at the end of each robotic arm with a force resolution of not less than 0.1N, used to trigger the underlying force control feedback when the actuator at the end of the robotic arm makes physical contact.

5. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 1, characterized in that, The step S3, which involves using a weighted Voronoi diagram to divide the workspace of the multi-robotic arm into task regions, specifically includes: integrating the accessibility and dexterity indices of each robotic arm in different orientations as kinematic weights for generating Voronoi cell boundaries; if the target fruit is located within the physical overlap area of ​​the multi-robotic arm, calculating the local Jacobian determinant of the picking posture required for each robotic arm to reach the fruit, and assigning the target fruit to the Voronoi cell of the robotic arm with the optimal Jacobian determinant, i.e., the best energy consumption and singularity margin.

6. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 5, characterized in that, In step S5, the virtual power flow transmitted along the virtual tangent point is calculated and converted into additional control torque of the underlying servo motor. The virtual power flow is defined as the inner product of the linear / angular velocity error vector and the force / torque error vector at the virtual tangent point. When the shortest distance between the two robotic arms is closer to the danger distance threshold, the virtual stiffness parameter and virtual damping parameter are nonlinearly increased, causing the virtual power flow to increase sharply. Based on the Newton-Euler equations, the surge in virtual power flow is inversely mapped to additional torque commands in the joint space of each robotic arm through the transpose of the Jacobian matrix and superimposed on the current loop of the servo driver.

7. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 1, characterized in that, The rules and parameter restrictions for the multi-dimensional weighted arbitration described in step S6 are as follows: arbitration intervention is carried out in descending order of weight, prioritizing collision risk level, task priority, and less remaining workload. The collision risk level is divided into three levels: high, medium, and low, based on the ratio of the shortest arm-to-arm distance to the safe distance threshold. When the collision risk level is high, the system unconditionally blocks the trajectory of the suboptimal arm; the arbitration response and scheduling instruction issuance delay of the task timing planner are strictly controlled to be less than 50ms to match the underlying servo control cycle and prevent physical collisions.

8. The multi-arm cooperative control method based on virtual decomposition and hierarchical sensing according to claim 1, characterized in that, The step S7, which involves switching to the compliant force control mode based on impedance control strategy, specifically includes: adjusting the apparent mass, spring stiffness, and damping characteristics of the end effector of the robotic arm in the physical contact direction to absorb excess contact energy and generate compliant yielding displacement, thereby limiting the contact force to a preset safe range that does not damage the fruit tree or the reducer.

9. A multi-robotic arm collaborative control system based on virtual decomposition control and hierarchical sensor fusion, applied to multi-robotic arm collaborative harvesting in unstructured agricultural environments, characterized in that, include: At least two independent robotic arm bodies; A global-local three-layer perception module deployed at the global, end-effector, and gripper positions includes a depth camera, a near-field vision sensor, and a force sensor; a control and data processing module, the control and data processing module including a processor and a memory, the memory storing a computer program, characterized in that, when the processor executes the computer program, it implements the operation steps of the multi-robotic arm collaborative control method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the multi-arm cooperative control method based on virtual decomposition and hierarchical perception as described in any one of claims 1 to 8.

Images