Control method and system for aircraft belly cargo / logistics dual-arm heavy load handling robot

Abstract

The application provides an aircraft belly compartment / logistics dual-arm large load carrying robot control method and system, relates to the technical field of robot control, and the method acquires a task instruction and multi-modal perception data, constructs and updates a dynamic semantic map, parses the instruction into task graph data, encodes the map, the task graph and a historical contact state into three-dimensional observation features, inputs a pre-trained conditional diffusion strategy model, generates a candidate dual-arm action sequence, filters and adjusts the sequence under torque, speed, collision, stability, contact force and safety distance constraints through a whole-body optimization control algorithm, obtains an optimal action sequence, calls a lightweight digital twin model for pre-execution simulation checking, and when the risk is lower than a safety threshold, the action sequence is sent to a bottom layer impedance controller for execution. The application reduces the ergonomic risk of manual bending, kneeling and heavy lifting, and improves the safety and efficiency of luggage and mail carrying.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and more specifically, to a control method and system for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics. Background Technology

[0002] Cargo and mail handling in the belly of aircraft is a typical heavy-load operation scenario in aviation ground support. The belly of narrow-body passenger aircraft is generally less than 1.2 meters high, and contains bulkheads, door frames, roller rails, and temporary stacked items. Personnel and equipment share the space, which is cramped, has variable lighting, and suffers from severe obstruction. Currently, handling mainly relies on manual labor or simple conveyor belts / rollers. Workers are required to bend over, kneel, turn around, and lift heavy loads of luggage (often 30-40 kg per piece), which easily leads to occupational injuries to the back and knees. Furthermore, handling efficiency is limited by physical strength.

[0003] In terms of robot control technology, existing methods mainly have the following shortcomings: First, traditional mobile operation robots or single-arm grasping systems usually rely on precise geometric models and preset trajectories. They have weak generalization ability for irregularly shaped packages such as soft bags, hard boxes, and cardboard boxes with unknown centers of gravity. In addition, the single-arm load capacity is limited, and slippage or overturning is likely to occur when grasping large, eccentric luggage.

[0004] Second, although end-to-end learning-based control strategies (such as imitation learning and reinforcement learning) can generate actions, they lack explicit modeling and verification of constraints such as collisions, contact forces, and safe distances of aircraft structures. They are difficult to apply directly to high-safety-level aviation operation environments and pose unexplainable and unpredictable execution risks.

[0005] Third, existing methods generally lack collaborative processing mechanisms for complex constraints such as low ceilings in the cargo compartment, roller guides, and mixed personnel movement, making it difficult to unify perception data, task semantics, and dual-arm motion planning within a computable and verifiable framework.

[0006] Therefore, there is an urgent need for a control method for heavy-duty handling robots in aircraft belly cargo compartments that can integrate dynamic environment perception, intelligent motion generation, multi-constraint optimization control, and online safety verification, so as to reduce the intensity of manual labor and improve handling efficiency while ensuring operational safety. Summary of the Invention

[0007] The problem solved by this invention is one or more of the aforementioned related technical problems.

[0008] To address the aforementioned problems, this invention provides a control method and system for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics.

[0009] In a first aspect, the present invention provides a control method for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, comprising: Acquire task instructions and multimodal perception data, construct and update a dynamic semantic map based on the multimodal perception data and robot state data, and parse the task instructions into executable task graph data; The dynamic semantic map, the task graph data, and the historical contact state at the current moment are encoded to obtain three-dimensional observation features. The three-dimensional observation features are then input into a pre-trained conditional diffusion strategy model to generate one or more candidate bi-arm action sequences. The optimal action sequence is obtained by screening and locally adjusting the candidate dual-arm action sequences through a whole-body optimization control algorithm under preset comprehensive constraints. Based on the optimal action sequence, a lightweight digital twin model is invoked for pre-execution simulation verification; When all risks verified by the simulation are lower than the preset safety threshold, the optimal action sequence is sent to the robot's underlying impedance controller for execution. The control method for the aircraft belly cargo / logistics dual-arm heavy-load handling robot also includes suction-grip composite gripping control, the process of which includes: Before grasping, the vacuum adsorption unit on the end effector of the robot is brought into contact with the surface of the cargo package and a negative pressure is established. The compliance of the cargo package surface is evaluated based on the negative pressure establishment speed and stability value. When the vacuum adsorption is determined to be unreliable, the robot, without retracting the robotic arm, controls the flexible gripper fingers to slightly open to release the initial limit on the cargo package, and then coordinates the adjustment of the chassis position and lifting and pitching posture so that the lifting palm surface is inserted into the bottom of the cargo package. Finally, the robot controls the flexible gripper fingers to close again, expanding the contact area with the side wall of the cargo package, and switches to pure lifting mode to complete the grasping.

[0010] Optionally, the step of constructing and updating the dynamic semantic map based on the multimodal perception data and robot state data includes: The RGB-D image, laser point cloud, and robot pose in the multimodal perception data are spatiotemporally aligned to generate a three-dimensional occupied grid map, which serves as the geometric layer data. The RGB-D image is semantically segmented to identify cargo packages, personnel, and aircraft structure. The identification results are then superimposed onto the geometric layer data to form semantic layer data. Based on the preset dynamic Bayesian estimation algorithm, the dynamic semantic map of the previous moment, the semantic layer data and geometric layer data of the current moment, the current pose in the robot state data, and the action executed in the previous moment are taken as inputs to perform recursive state updates and output the dynamic semantic map of the current moment. In the dynamic semantic map at the current moment, each aircraft structure semantic node is expanded outward by a preset distance to generate an aircraft structure safety distance field. This safety distance field is then superimposed onto the dynamic semantic map at the current moment to obtain the updated dynamic semantic map.

[0011] Optionally, when parsing the task instructions into executable task graph data, the method further includes: Extract the package weight, handling frequency, and target location from the task instructions; An efficiency risk score is obtained based on the cargo package quality, the handling frequency, and the target location. Subtasks whose work efficiency risk scores exceed a preset threshold are assigned a higher execution priority in the task graph data.

[0012] Optionally, the step of inputting the three-dimensional observation features into a pre-trained conditional diffusion strategy model to generate one or more candidate bi-arm movement sequences includes: The three-dimensional observation features are used as conditions and input into a pre-trained conditional diffusion strategy model. Through an iterative denoising process, one or more candidate double-arm action sequences are gradually recovered from random noise. Each candidate double-arm action sequence includes the left and right arm grasping posture, end pose, relative configuration of the arms and motion trajectory. The relative configuration of the arms includes the switching trajectory between narrow-space passage state and wide-space transport state. The pre-trained conditional diffusion strategy model is as follows: ; in, Indicates parameterization Conditional diffusion strategy model; This represents the candidate sequence of arm movements from the current time t to H future steps; This represents the multimodal observation data at the current moment; The task conditions include the target package identity, stacking location, and handling sequence constraints in the task graph data. Denoising network for conditional diffusion model; This represents the multimodal observation data at the current moment. The three-dimensional observation features obtained after three-dimensional encoding.

[0013] Optionally, the step of using a whole-body optimization control algorithm to screen and locally adjust the candidate dual-arm movement sequences under preset comprehensive constraints to obtain the optimal movement sequence includes: Calculate the comprehensive cost for each candidate dual-arm motion sequence, and select the candidate dual-arm motion sequence with the lowest comprehensive cost as the initial motion sequence, provided that the preset joint speed upper limit constraint, joint torque upper limit constraint, chassis stability margin constraint and aircraft structural safety distance constraint are met in the comprehensive constraints. On the execution trajectory of the initial action sequence, the optimal load distribution coefficient is solved online, where the solution formula is: ; in: This represents the optimal load-sharing factor for the left arm; The left arm load-sharing coefficient is the independent variable, and the right arm load-sharing coefficient is... ; This indicates that the left arm has a sharing coefficient of 1. The joint torque vector at that time; This indicates that the right arm has a sharing coefficient of 1. The joint torque vector at that time; It represents the norm weighted by the weight matrix Q, where Q is the weight matrix of the joint torques; Preset weighting coefficients for the slippage risk item; This indicates the slippage risk item, calculated based on the friction cone safety margin at the current gripping point and the tactile slippage detection results. Based on the optimal load distribution coefficient obtained by the solution, the lifting force of the two arms, the chassis position, and the lifting and pitching attitude are locally fine-tuned to generate the optimal action sequence.

[0014] Optionally, the lightweight digital twin model is generated jointly from a preset model parameter template and real-time scanning data. Using the optimal action sequence as input, the lightweight digital twin model is invoked for simulation verification. The simulation verification includes: Calculate the minimum safe distance from the outline of the robot and cargo to the restricted area of ​​the aircraft structure, as well as the risks of collision, overturning, slippage, excessive contact force, and close proximity to personnel; When any risk exceeds the corresponding preset threshold, the system will enter a slowdown, withdrawal, hold, or request manual takeover mode. Specifically, the manual takeover mode includes: receiving action instructions from staff via a handheld terminal or remote control device, or receiving teaching actions from staff within a safe area; after the takeover is completed, the robot's pose is recalibrated using the multimodal perception data, and the dynamic semantic map is automatically refreshed based on a preset dynamic Bayesian estimation algorithm, without the need for manual repair.

[0015] Optionally, the underlying impedance controller outputs a command force according to a preset control law; the command force is sent to the robot's servo driver and converted into joint torque or end effector force to drive the robot's chassis, lifting mechanism and arms to execute the optimal motion sequence; During execution, the contact force, slip state, and aircraft structural safety distance field are monitored in real time, and rolling temporal replanning is performed based on the real-time updates of the dynamic semantic map. When force exceeding limits, slippage, or personnel entering a dangerous area are detected, local replanning is immediately triggered to generate an avoidance path.

[0016] Optionally, the control method for the aircraft belly cargo / logistics dual-arm heavy-load handling robot further includes a data flywheel strategy, which includes: After each operation is completed, the operation data, including the success rate of grasping, force and tactile waveform, reasons for failure, placement photos and manual intervention records, will be uploaded to the edge-cloud collaborative system. When the cumulative number of failed samples in the same scenario or the same type of cargo reaches a preset threshold, the conditional diffusion strategy model fine-tuning training in the offline environment is triggered.

[0017] In a second aspect, the present invention provides a control system for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, used to implement the control method for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics as described in the first aspect, the system comprising: The acquisition unit is used to acquire task instructions and multimodal perception data, construct and update a dynamic semantic map based on the multimodal perception data and robot state data, and parse the task instructions into executable task graph data. The processing unit is used to encode the current dynamic semantic map, the task graph data, and the historical contact state to obtain three-dimensional observation features, input the three-dimensional observation features into a pre-trained conditional diffusion strategy model, generate one or more candidate double-arm action sequences, and filter and locally adjust the candidate double-arm action sequences through a whole-body optimization control algorithm under preset comprehensive constraints to obtain the optimal action sequence. The simulation unit is used to call the lightweight digital twin model to perform simulation verification before execution based on the optimal action sequence; The execution unit is used to send the optimal action sequence to the robot's underlying impedance controller for execution when all risks verified by the simulation are lower than the preset safety permission threshold.

[0018] Thirdly, the present invention provides a control device for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the control method for the dual-arm heavy-load handling robot for aircraft cargo hold / logistics as described in the first aspect.

[0019] The beneficial effects of the control method and system for the aircraft belly cargo / logistics dual-arm heavy-load handling robot of the present invention are: By integrating dynamic semantic map construction, conditional diffusion strategy candidate action generation, whole-body optimization control screening and adjustment, lightweight digital twin simulation verification, and low-level impedance safety execution, a closed-loop process from environmental perception to motion control is formed, which can reliably adapt to complex constraint scenarios such as low-ceilinged aircraft cargo holds, narrow passages, roller rails, and mixed personnel traffic. Multiple candidate dual-arm action sequences are generated using a diffusion strategy driven by 3D observation features, and then screened and locally adjusted under multiple constraints such as torque, speed, collision, stability, contact force, and safety distance through whole-body optimization control. This retains the generalized grasping ability of the learning strategy for unstructured objects, while ensuring the physical feasibility and engineering safety of the trajectory through optimized control. The digital twin pre-verification mechanism quantitatively assesses risks such as collision, overturning, slippage, and close proximity to personnel before execution. Commands are only issued when all risks are below the safety permission threshold, significantly reducing the safety hazards caused by direct execution of black-box actions. The low-level impedance controller provides a smooth response in real time during contact, avoiding impact on the aircraft structure, luggage, and personnel during heavy-load handling. Therefore, this invention effectively reduces the labor efficiency risks of repeated bending, kneeling, twisting and heavy handling in low-ceilinged cargo compartments, improves the safety and efficiency of autonomous handling of large-load (e.g., 30-40kg) baggage and cargo packages, and provides a traceable and verifiable technical foundation for the large-scale deployment of aviation ground support robots.

[0020] Furthermore, this invention achieves intelligent adaptive gripping of packages with different surface characteristics through a composite control of vacuum adsorption and flexible gripping fingers and palm lifting. Real-time assessment of the negative pressure establishment speed and stability provides a quantitative basis for judging adsorption reliability, avoiding gripping failures caused by blindly relying on adsorption. When adsorption is unreliable, the robot can smoothly switch to pure lifting mode by maintaining the robotic arm close to the package through a continuous sequence of actions: "slightly opening the gripping fingers → coordinated chassis and lifting → palm lifting and insertion → re-closing the gripping fingers." This avoids interrupting the workflow and prevents cycle delays and cargo damage caused by forced adsorption or repeated withdrawals. This mechanism significantly improves the robot's success rate in gripping difficult-to-adsorb packages such as soft bags, woven bags, and breathable cardboard boxes, while reducing the risk of slippage and falls during heavy-duty handling. It is a key control strategy for ensuring the safe handling of heavy luggage and packages. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a control method for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a dual-arm heavy-load handling robot control system for aircraft belly cargo / logistics, according to an embodiment of the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0023] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0024] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0025] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0026] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0027] Existing digital twins or simulation verifications are mostly used in offline design and debugging stages, and do not form a closed loop with real-time perception, decision-making, and control. They cannot perform online verification of risks such as collisions, overturning, slippage, and close proximity of personnel before the action is issued.

[0028] To solve the above-mentioned problems, such as Figure 1 As shown in the figure, an embodiment of the present invention provides a control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics, comprising: Step S100: Obtain task instructions and multimodal perception data; construct and update a dynamic semantic map based on the multimodal perception data and robot state data; and parse the task instructions into executable task graph data.

[0029] Specifically, the system first acquires two types of input information: one is the task instructions from the flight scheduling system, handheld terminal, or human voice, which specify the target cargo package to be moved (e.g., "Place the baggage labeled BA123 on the conveyor belt into the left side of the front section of the belly compartment"), the weight class of the cargo package, and the target stacking position; the other is multimodal perception data collected in real time by various sensors on the robot, including but not limited to visual images, 3D point clouds, force and tactile signals.

[0030] At the same time, it reads the robot's own state data, such as its current position, posture, joint angles, and movement speed.

[0031] Based on the aforementioned multimodal perception data and robot state data, a dynamic semantic map is constructed and maintained. This map not only describes the three-dimensional geometry of obstacles in the environment (such as bulkheads, door frames, roller rails, and stacked cargo), but also labels semantic information (such as where personnel are prohibited, where cargo packages are, and which cargo package is the current target). Furthermore, it can be updated in real time as the robot moves and the environment changes (such as personnel moving or cargo being removed).

[0032] Simultaneously, task instructions in natural language or structured formats are converted into executable task graph data. Task graph data is a hierarchical, dependency-based task description that decomposes the abstract instruction of "moving a piece of luggage" into a series of ordered subtasks that can be directly processed by the robot's control program, such as "go to the cargo hold door" → "identify the target luggage" → "plan the grasping posture" → "lift and transfer" → "place in the designated location" → "return to the standby position". Each subtask includes necessary spatial coordinates, safety constraints, and success criteria.

[0033] By simultaneously acquiring task instructions and multimodal environmental perception data, and dynamically constructing a semantic map, this invention enables the robot to understand the complex geometric and semantic environment within the cargo compartment in real time. It also automatically breaks down abstract handling instructions into executable task steps, providing clear and computable input for subsequent motion planning and safety control. The dynamic update mechanism ensures the map's timeliness in dynamic scenarios such as personnel entry and exit, and cargo changes, avoiding collisions or misoperations caused by environmental changes. The introduction of task map data enables automatic conversion from natural language or scheduling instructions to the robot's underlying actions, reducing reliance on manual programming and on-site teaching. This significantly improves adaptability to different flight tasks and loading rules, laying the foundation for an automated closed-loop handling operation.

[0034] Step S200: Encode the current dynamic semantic map, the task graph data, and the historical contact state to obtain three-dimensional observation features, and input the three-dimensional observation features into a pre-trained conditional diffusion strategy model to generate one or more candidate dual-arm action sequences.

[0035] Specifically, after obtaining the dynamic semantic map, task graph data and historical contact status at the current moment in step S100, this step first fuses and encodes this information into a compact three-dimensional observation feature vector.

[0036] Dynamic semantic map: Contains the 3D position, geometry, and semantic category (such as cargo packages, bulkheads, personnel, passable areas, etc.) of all objects in the current environment, as well as a real-time updated aircraft structural safety distance field. Local spatial information centered on the target cargo package is extracted from it.

[0037] Task graph data: describes the specific sub-tasks that need to be performed, such as "grabbing a specific piece of luggage from the conveyor belt and stacking it at specified coordinates in the cargo hold". This data provides constraints such as the identity of the target cargo, its target location, and the handling sequence.

[0038] Historical contact status: Records recent changes in the contact force / torque between the robot and the package or environment, the pressure distribution of the tactile array, and whether events such as slippage or force exceeding limits have occurred. This information reflects the effectiveness of previous grasping attempts.

[0039] A pre-defined 3D encoder (such as a point cloud-based neural network or an occupies a grid convolutional network) maps the above three types of information into fixed-dimensional feature vectors, i.e., the 3D observation features. These features uniformly express "what the current environment is", "what task needs to be completed", and "what the previous interaction results were".

[0040] Subsequently, the 3D observation features are used as input to a pre-trained conditional diffusion strategy model. This model is a generative model that, based on the 3D observation features, outputs one or more candidate sequences of robot arm movements. Each candidate sequence describes how the robot's arms (including the end effector), chassis, lifting mechanism, and other degrees of freedom should move over a future period to grasp, lift, or transfer the target package. Because the conditional diffusion strategy model can capture a multimodal solution space (e.g., there may be multiple different grasping methods for the same package), it can generate multiple different, theoretically feasible candidate sequence sequences at once for subsequent screening and optimization.

[0041] By encoding dynamic semantic maps, task graph data, and historical contact states into unified 3D observation features, and using these features to drive a pre-trained diffusion strategy model, a direct mapping from high-dimensional perceptual input to multimodal action candidates is achieved. The 3D observation features retain rich information about the environmental geometry and semantics while embedding task objectives and historical experience. This enables the diffusion strategy model to quickly generate multiple physically reasonable dual-arm action candidates in complex cargo handling scenarios, avoiding the poor adaptability and weak generalization capabilities of traditional methods that rely on single trajectory planning or manual teaching. Simultaneously, generating multiple candidate action sequences provides sufficient decision redundancy for subsequent optimization, screening, and safety verification, helping to select the optimal execution plan while ensuring safety. This improves the robot's autonomous handling capability for irregular, heavy (e.g., 30-40kg) luggage with unknown weight distribution.

[0042] Step S300: Using a whole-body optimization control algorithm, under preset comprehensive constraints, the candidate double-arm movement sequences are screened and locally adjusted to obtain the optimal movement sequence.

[0043] Specifically, after generating one or more candidate dual-arm motion sequences in step S200, a whole-body optimization control algorithm is used to evaluate and select the best candidate scheme. This algorithm treats the robot as a whole kinematic chain, taking into account the dynamic coupling between the low-profile mobile chassis, the lifting and pitch compensation mechanism, and the dual arms.

[0044] First, the algorithm loads a pre-set set of comprehensive constraints, including: the upper limit of torque for each joint, the speed limit of the end effector and joints, the minimum safe distance between the robot body and cargo outline and the aircraft structure (bullets, door frames, roller rails, etc.), the stability margin of cargo during grasping and transfer (to prevent tipping or slipping), the upper limit of contact force between the end effector and the cargo package (to prevent damage to luggage), and the threshold for close proximity risk to personnel.

[0045] For each candidate bi-arm movement sequence, the algorithm calculates its satisfaction level under the aforementioned constraints and evaluates a comprehensive performance index (e.g., weighted summation). Candidate sequences that violate any hard constraints (such as insufficient safety distance or excessive torque) are directly eliminated; for feasible sequences, the algorithm selects the one with the best comprehensive performance as the initial movement sequence.

[0046] Subsequently, the algorithm performs local adjustments to the initial sequence. The purpose of these adjustments is to further optimize the execution performance without violating constraints. For example, this includes fine-tuning the lifting force distribution of the arms to balance the load on the left and right joints, slightly adjusting the chassis position to improve the projection of the cargo's center of gravity within the support polygon, and moderately altering the attitude of the lifting and pitching mechanism to prevent the cargo corners from approaching the bulkhead. The adjustment process typically employs gradient optimization or local search methods to generate a corrected trajectory near the original trajectory that satisfies all constraints and has a lower overall cost, which is then output as the final optimal action sequence.

[0047] Suppose candidate sequence A requires the left arm to bear 70% of the load, but calculations show that the left shoulder joint torque is close to its upper limit, while the right arm still has a margin. The whole-body optimization control algorithm, while maintaining gripping stability, appropriately reduces the lifting force of the left arm and increases it accordingly for the right arm. Simultaneously, it fine-tunes the chassis by moving it forward by 5 centimeters, bringing the center of gravity of the load closer to the robot's center, thereby reducing the left arm torque and meeting safety constraints. The final optimal motion sequence achieves a more balanced load distribution while ensuring no collisions, no tipping, and no exceeding limits.

[0048] This invention organically combines the generalization ability of a learning strategy with model-based multi-constraint optimization control by using a whole-body optimization control algorithm to screen and locally refine multiple candidate action sequences generated by a diffusion strategy. The screening process ensures that only actions meeting hard engineering constraints such as torque, speed, safety distance, stability, and contact force are retained, effectively preventing the learning strategy from directly outputting infeasible or high-risk actions. The local refinement process, without disrupting the original trajectory structure, further improves the smoothness, stability, and safety of the actions by fine-tuning load distribution, chassis position, and lifting attitude. This hierarchical architecture of "candidate generation + constraint screening + local optimization" retains multimodal adaptability to irregular cargo packages while ensuring that the final executed actions meet the stringent requirements of aviation operations for collision avoidance, over-limit prevention, and rollover prevention, significantly improving the safety redundancy and practical deployment feasibility of heavy-load handling.

[0049] Step S400: Based on the optimal action sequence, call the lightweight digital twin model to perform simulation verification before execution; Step S500: When all the risks verified by the simulation are lower than the preset safety permission threshold, the optimal action sequence is sent to the robot's underlying impedance controller for execution.

[0050] Specifically, after the whole-body optimization control algorithm outputs the optimal action sequence, this sequence is fed into a lightweight digital twin model for pre-execution simulation verification. The lightweight digital twin model refers to a pre-built or real-time generated simplified three-dimensional geometric and physical model based on the current operational scenario (aircraft belly hold, cargo door, conveyor belt area, etc.), which includes the following key data: Robot geometric model: including the simplified outer contour envelope and kinematic relationships of the chassis, lifting mechanism, dual arms, and end effector; Cargo outline model: The three-dimensional shape of the target cargo package reconstructed in real time from multimodal perception data (such as the point cloud outline of a hard box or the approximate envelope of a soft package). Aircraft structural model: The safety envelope or occupancy grid of key fixed structures (bulls, door frames, roller rails, seat rails, etc.) in the belly compartment is usually loaded from a preset aircraft parameter template and locally modified using on-site scanning data; Dynamic obstacle model: Simplified position and velocity information of people and temporary stacks.

[0051] Simulation verification typically includes: simulating each control cycle of the optimal action sequence one by one, calculating the spatial pose of each link of the robot, the end effector, and the cargo outline during the motion, and performing collision detection with the aircraft structural model and dynamic obstacle model; simultaneously assessing the cargo slippage risk (e.g., whether the lifting surface friction is sufficient), the tipping risk (e.g., whether the cargo's center of gravity projection is within the chassis support polygon), and whether the contact force exceeds the tolerance limit of the baggage or aircraft structure. Because digital twin models employ lightweight designs (e.g., using convex hull approximation, occupying grids, or combinations of basic geometries), a single simulation calculation can be completed in milliseconds without affecting overall real-time performance.

[0052] The simulation verification in step S400 will output specific values ​​for each risk (such as minimum safe distance, maximum contact force, slip margin, etc.). These risk values ​​are compared one by one with the pre-set safety allowance thresholds. Only when all risk indicators are lower than the corresponding thresholds (such as minimum safe distance ≥ 3cm, maximum contact force ≤ 80% of the rated value, overturning stability margin ≥ 1.2, etc.) is the optimal action sequence considered safe and can be issued for execution.

[0053] At this point, the optimal motion sequence (including the trajectory of each joint angle, end effector force / torque commands, chassis velocity curve, etc.) is sent to the robot's lowest-level impedance controller. The lowest-level impedance controller is the lowest-level execution unit of the robot's control system. It receives position, velocity, or force commands and drives the robot's various motors through the servo drive system. The impedance control law enables the robot to maintain high-precision trajectory tracking in free space, while exhibiting compliant characteristics when in contact with cargo or the environment, avoiding rigid impacts.

[0054] If any risk in the simulation verification exceeds the threshold, no action will be issued; instead, pre-defined exception handling will be executed, such as re-invoking the diffusion strategy to generate new candidate sequences, requesting manual intervention, or performing a safe withdrawal.

[0055] By rapidly simulating and verifying the optimal action sequence using a lightweight digital twin model before execution, this invention adds a safety permission gate to the trajectory generated by the learning strategy, effectively avoiding actual collisions, slippage, or overturning accidents caused by model errors, perceptual noise, or unmodeled dynamics. The lightweight design ensures the efficiency of simulation calculations without slowing down the real-time control loop. Execution is only permitted when all risk indicators are below preset thresholds, ensuring the stringent requirements of aviation operations for aircraft structural damage prevention, personnel safety, and baggage integrity. After receiving the optimal action sequence that has undergone dual verification (optimized control screening + digital twin verification), the underlying impedance controller drives execution in a compliant manner, further reducing contact impacts and unexpected disturbances. Thus, the "generation-screening-verification-execution" closed loop of this invention significantly improves the safety and reliability of heavy-load handling in aircraft belly cargo holds, providing crucial assurance for the stable deployment of robots in real aviation ground support environments.

[0056] This embodiment integrates dynamic semantic map construction, candidate action generation using conditional diffusion strategies, whole-body optimization control screening and adjustment, lightweight digital twin simulation verification, and low-level impedance safety execution to form a closed-loop process from environmental perception to motion control. This allows for reliable adaptation to complex constraint scenarios such as low-ceilinged aircraft cargo holds, narrow passages, roller rails, and mixed personnel traffic. Multiple candidate dual-arm action sequences are generated using a diffusion strategy driven by 3D observation features. These sequences are then screened and locally adjusted under multiple constraints such as torque, speed, collision, stability, contact force, and safety distance through whole-body optimization control. This retains the generalized grasping ability of the learning strategy for unstructured objects while ensuring the physical feasibility and engineering safety of the trajectory through optimized control. The digital twin pre-verification mechanism quantitatively assesses risks such as collision, overturning, slippage, and close proximity to personnel before execution. Commands are only issued when all risks are below the safety threshold, significantly reducing the safety hazards caused by direct execution of black-box actions. The low-level impedance controller provides a smooth, real-time response during contact, avoiding impacts on the aircraft structure, luggage, and personnel during heavy-load handling. Therefore, this invention effectively reduces the labor efficiency risks of repeated bending, kneeling, twisting and heavy handling in low-ceilinged cargo compartments, improves the safety and efficiency of autonomous handling of large-load (e.g., 30-40kg) baggage and cargo packages, and provides a traceable and verifiable technical foundation for the large-scale deployment of aviation ground support robots.

[0057] Optionally, the step of constructing and updating the dynamic semantic map based on the multimodal perception data and robot state data includes: The RGB-D image, laser point cloud, and robot pose in the multimodal perception data are spatiotemporally aligned to generate a three-dimensional occupied grid map, which serves as the geometric layer data. The RGB-D image is semantically segmented to identify cargo packages, personnel, and aircraft structure. The identification results are then superimposed onto the geometric layer data to form semantic layer data. Based on the preset dynamic Bayesian estimation algorithm, the dynamic semantic map of the previous moment, the semantic layer data and geometric layer data of the current moment, the current pose in the robot state data, and the action executed in the previous moment are taken as inputs to perform recursive state updates and output the dynamic semantic map of the current moment. In the dynamic semantic map at the current moment, each aircraft structure semantic node is expanded outward by a preset distance to generate an aircraft structure safety distance field. This safety distance field is then superimposed onto the dynamic semantic map at the current moment to obtain the updated dynamic semantic map.

[0058] Specifically, the RGB-D images and laser point clouds in the multimodal perception data are spatiotemporally aligned with the robot's current pose (obtained by fusion of IMU, wheel speed odometry, and visual / laser odometry) to generate a three-dimensional occupancy grid map. This map records whether each voxel in the work space is occupied, forming geometric layer data.

[0059] Subsequently, a pre-trained semantic segmentation network (such as a lightweight general semantic segmentation model) is used to process the RGB-D images to identify cargo packages, personnel, and aircraft structures (such as bulkheads, door frames, and roller rails). The recognition results are then superimposed onto the geometric layer data in the form of semantic labels to form semantic layer data. The semantic layer data assigns category attributes to each object or region (e.g., "grabable cargo package", "restricted aircraft structure", "moving personnel").

[0060] Based on the aforementioned geometric and semantic layer data, a pre-defined dynamic Bayesian estimation algorithm is used to recursively update the dynamic semantic map. This algorithm takes the complete dynamic semantic map from the previous time step, the semantic and geometric layer data from the current time step, the robot's current pose, and the action performed in the previous time step as input, and outputs the dynamic semantic map from the current time step. The recursive update process can be expressed by the following formula: ; in: This represents the dynamic semantic map output at time t; A dynamic semantic map representing the previous moment; This represents the multimodal observation features at the current moment (integrating RGB-D, laser point cloud, force / tactile data, etc.). This indicates the robot's current full-body pose and joint status; This indicates the action performed by the robot at the previous moment. This typically represents the raw multimodal observation data at the current moment, containing all the multimodal perception information collected by the robot at time t.

[0061] This recursive update can integrate historical information with current observations, improving the tracking robustness for dynamic environments such as people moving or goods being moved.

[0062] Finally, in the dynamic semantic map at the current moment, each aircraft structural semantic node (such as the corresponding voxel or bounding box of bulkhead, door frame, or guide rail) is expanded outward by a preset safety distance (e.g., 5 cm) to generate an aircraft structural safety distance field. This safety distance field is superimposed as an additional layer onto the dynamic semantic map to obtain the final updated dynamic semantic map used in subsequent steps.

[0063] It should be noted that when the dynamic semantic map of the previous moment does not exist, an initial dynamic semantic map is constructed based on the semantic layer data and geometric layer data of the current moment using a preset initialization algorithm.

[0064] By constructing a 3D dynamic semantic map combining geometric and semantic layers and recursively updating it using a dynamic Bayesian estimation algorithm, high-fidelity real-time modeling of the complex environment of an aircraft belly (low ceiling, roller rails, mixed personnel traffic, stacked cargo) was achieved. This enables the robot to accurately distinguish between passable areas, graspable cargo packages, and prohibited aircraft structures. A safety distance field for the aircraft structure is overlaid on this map, providing explicit collision avoidance constraints for subsequent path planning and motion generation, fundamentally avoiding the risk of robot contact with the aircraft structure. The recursive update mechanism ensures the timeliness and accuracy of the map in dynamic scenarios such as personnel entry and exit and cargo changes, significantly improving the robot's adaptability to the uncertain environment within the belly. The entire mapping process integrates multimodal perception and historical motion feedback, laying a reliable spatial semantic foundation for subsequent high-safety dual-arm heavy-duty handling operations.

[0065] Optionally, when parsing the task instructions into executable task graph data, the method further includes: Extract the package weight, handling frequency, and target location from the task instructions; An efficiency risk score is obtained based on the cargo package quality, the handling frequency, and the target location. Subtasks whose work efficiency risk scores exceed a preset threshold are assigned a higher execution priority in the task graph data.

[0066] Specifically, during the process of parsing task instructions into executable task graph data, an additional ergonomic risk assessment step is performed to identify which sub-tasks pose a high labor intensity or health risk to manual operation. This allows for an appropriate increase in priority in the task graph data, facilitating the allocation of high-risk tasks to robots during subsequent task scheduling.

[0067] Specifically, the following key parameters are extracted from the task instructions: Packaging quality : The weight of the target baggage or cargo (in kg), for example, obtained from barcode scanning or weight sensors; Frequency of handling The number of times the package needs to be handled per unit of time, or the frequency of repeated occurrences of the same type of package; Target location: The location where the cargo packages need to be stacked (such as the front left side of the cargo compartment, a deep corner, etc.). This location is mapped to a personnel posture risk coefficient. For example, working deep inside the cargo compartment requires manual bending or kneeling, which carries a higher risk factor than working near the hatch. Number of repetitions The number of times the same type of cargo package or the same location is repeatedly handled (which can be obtained from historical task statistics or task instructions).

[0068] The following linear weighted formula is used to calculate the efficiency risk score: ; in, This represents the work efficiency risk score; the higher the value, the greater the potential harm or fatigue risk to the human body from the task. The preset weighting factor (e.g., 0.5) for the quality item of the cargo package; The weight of the package (kg); The preset weighting coefficient for the frequency of handling (e.g., 0.3); This refers to the frequency of handling (times / hour). The preset weighting coefficient for the attitude risk term (e.g., 0.8); The personnel posture risk coefficient typically ranges from 0 to 1, depending on the target location (e.g., 0.2 near the hatch and 0.9 deep in the belly compartment). A preset weighting coefficient (e.g., 0.2) for the repetition count item; This represents the number of times the goods are repeatedly moved.

[0069] Calculate each subtask according to the above formula. .when When the threshold is exceeded (e.g., 10.0), the subtask is given a higher execution priority in the task graph data, so that the task scheduling module will prioritize assigning it to the robot for execution rather than to a human.

[0070] By extracting package quality, handling frequency, target location, and repetition count from task instructions, and calculating a weighted formula for ergonomic risk scores, ergonomic risks are quantified into calculable indicators. This allows the task scheduling module to proactively identify high-risk sub-tasks (such as heavy-load, high-frequency, and deep-cabin operations) that require significant manual labor and are prone to occupational injuries, and automatically assign them higher execution priority, prioritizing their allocation to robots. This mechanism effectively reduces the frequency of ground staff repeatedly bending, kneeling, twisting, and lifting heavy loads in low-ceilinged cargo compartments, significantly reducing occupational health risks. Simultaneously, it achieves the human-robot collaborative scheduling goal of "robots handling high-risk, heavy tasks, while humans handle light-load sorting and anomaly handling," improving overall operational safety and efficiency.

[0071] Optionally, the step of inputting the three-dimensional observation features into a pre-trained conditional diffusion strategy model to generate one or more candidate bi-arm movement sequences includes: The three-dimensional observation features are used as conditions and input into a pre-trained conditional diffusion strategy model. Through an iterative denoising process, one or more candidate double-arm action sequences are gradually recovered from random noise. Each candidate double-arm action sequence includes the left and right arm grasping posture, end pose, relative configuration of the arms and motion trajectory. The relative configuration of the arms includes the switching trajectory between narrow-space passage state and wide-space transport state. The pre-trained conditional diffusion strategy model is as follows: ; in, Indicates parameterization Conditional diffusion strategy model; This represents the candidate sequence of arm movements from the current time t to H future steps; This represents the multimodal observation data at the current moment; The task conditions include the target package identity, stacking location, and handling sequence constraints in the task graph data. Denoising network for conditional diffusion model; This represents the multimodal observation data at the current moment. The three-dimensional observation features obtained after three-dimensional encoding.

[0072] Specifically, after encoding the 3D observation features, these features are used as conditional inputs into a pre-trained conditional diffusion strategy model. This model is a generative model based on the diffusion process, whose core idea is to start with pure random noise and gradually recover the action sequence that conforms to the conditional distribution through multiple iterations of denoising.

[0073] Specifically, the model receives two key inputs: one is three-dimensional observation features. It encodes the geometric and semantic information in the current dynamic semantic map, the task objectives in the task graph data, and the historical contact status; secondly, the task conditions. This includes the identity of the target cargo package, the desired stacking location, and the handling sequence constraints (e.g., "first move baggage A to the front left side of the belly compartment, then move baggage B to the middle right side").

[0074] The conditional diffusion strategy model uses a parameterized denoising network. Starting with a Gaussian noise vector, a back-diffusion step is applied progressively. At each step, the noise residual is predicted based on the current noise level, 3D observation features, and task conditions, thereby updating the action sequence. After a predetermined number of iterations (e.g., H-step diffusion steps), the model ultimately outputs one or more complete candidate dual-arm action sequences. .

[0075] Each candidate action sequence describes a continuous motion trajectory from the current time t to H future time steps, specifically including the following: Left and right arm grasping posture: The change of the pose (position and orientation) of each robotic arm end effector in space over time; End position: The contact point and contact angle of the end effector relative to the cargo package; Relative configuration of the two arms: the spatial relationship between the left and right arms. This configuration can be dynamically switched according to the operation stage, including narrow passage mode (both arms are folded to facilitate passage through narrow passages or hatches in the belly compartment) and wide transport mode (both arms are extended to form a stable lifting surface), as well as the smooth switching trajectory between the two. Motion trajectory: Time sequence of angular velocity, angular acceleration and end force / torque commands of each joint.

[0076] By utilizing a conditional diffusion strategy model, multiple physically reasonable candidate dual-arm action sequences that conform to task constraints can be generated simultaneously in a high-dimensional action space. This overcomes the limitations of traditional single-path planning methods, which struggle to obtain feasible solutions or have a limited solution space when dealing with irregular packages (soft packages, eccentric luggage, and packages with unknown shapes). The iterative denoising mechanism of the diffusion model naturally supports multimodal output, allowing the system to select the optimal solution based on safety and efficiency in subsequent steps, significantly improving the robot's adaptability and robustness to complex cargo handling tasks. Furthermore, the action sequences explicitly include switching trajectories between narrow and wide-space states, enabling the robot to automatically adjust its dual-arm configuration according to environmental conditions. This satisfies the stringent requirements for maneuverability in low-ceilinged and narrow-passage cargo compartments while ensuring the stability of dual-arm collaboration during heavy-load handling. This step, as the first layer of a three-layer architecture of "learning-based generation + optimization screening + digital twin verification," provides a rich candidate foundation for subsequent multi-constraint optimization and safety verification, and is a key technical support for achieving highly generalized and highly safe dual-arm handling.

[0077] Optionally, the step of using a whole-body optimization control algorithm to screen and locally adjust the candidate dual-arm movement sequences under preset comprehensive constraints to obtain the optimal movement sequence includes: Calculate the comprehensive cost for each candidate dual-arm motion sequence, and select the candidate dual-arm motion sequence with the lowest comprehensive cost as the initial motion sequence, provided that the preset joint speed upper limit constraint, joint torque upper limit constraint, chassis stability margin constraint and aircraft structural safety distance constraint are met in the comprehensive constraints. On the execution trajectory of the initial action sequence, the optimal load distribution coefficient is solved online, where the solution formula is: ; in: This represents the optimal load-sharing factor for the left arm; The left arm load-sharing coefficient is the independent variable, and the right arm load-sharing coefficient is... ; This indicates that the left arm has a sharing coefficient of 1. The joint torque vector at that time; This indicates that the right arm has a sharing coefficient of 1. The joint torque vector at that time; It represents the norm weighted by the weight matrix Q, where Q is the weight matrix of the joint torques; Preset weighting coefficients for the slippage risk item; This indicates the slippage risk item, calculated based on the friction cone safety margin at the current gripping point and the tactile slippage detection results. Based on the optimal load distribution coefficient obtained by the solution, the lifting force of the two arms, the chassis position, and the lifting and pitching attitude are locally fine-tuned to generate the optimal action sequence.

[0078] Specifically, after the diffusion strategy model generates one or more candidate dual-arm motion sequences, the whole-body optimization control algorithm evaluates the feasibility of each candidate sequence and selects the best one. This algorithm treats the robot (chassis, lifting mechanism, dual arms) as a whole kinematic chain, while taking into account multiple engineering constraints.

[0079] First, the algorithm calculates the comprehensive cost for each candidate two-arm motion sequence, using the following formula: ; in, This represents the total cost of the candidate action sequence; Preset weighting coefficients for energy consumption costs; This represents the energy cost of executing this sequence of actions; Preset weighting coefficients for collision costs; The collision cost is calculated based on the no-entry zones of the aircraft structure and the outline of the cargo in the dynamic semantic map. Preset weighting coefficients for the risk of exceeding contact force limits; This represents the risk cost of exceeding the contact force limit, calculated based on the estimated contact force and preset force threshold of the end effector. Preset weighting coefficients for cycle time cost; This represents the time cost of completing the sequence of actions. Preset weighting coefficients for the cost of slippage risk; The cost of slippage risk is represented by calculations based on the friction cone margin at the gripping point and tactile slippage detection. Preset weighting coefficients for the risk cost of close proximity to personnel; The risk cost of proximity to personnel is calculated based on the distance between the personnel's position and the robot's motion envelope in the dynamic semantic map.

[0080] Secondly, the algorithm checks whether the sequence meets the preset hard synthesis constraints, including: Joint velocity limit constraint: The angular velocity of each joint shall not exceed the maximum value allowed by the robotic arm; Joint torque limit constraint: The driving torque of each joint shall not exceed the rated limit of the motor and reducer; Chassis stability margin constraint: The projection of the cargo's center of gravity must always be inside the chassis support polygon, and the minimum stability margin must be greater than the safety threshold. Aircraft structural safety distance constraints: The minimum distance between each link of the robot, the end effector, and the cargo outline and the aircraft structure such as the bulkhead, door frame, and roller rails shall not be less than the preset safety value.

[0081] Only candidate sequences that simultaneously satisfy all the above hard constraints proceed to the next step. Then, the algorithm selects the comprehensive cost. The lowest feasible candidate sequence is used as the initial action sequence.

[0082] To further optimize the load-bearing efficiency and gripping stability of the dual arms, the algorithm performs online optimization based on the following dual-arm load-sharing model on the execution trajectory of the initial action sequence: ; in, The equivalent force (including lifting force, clamping force, etc.) borne by the end of the i-th arm. Load sharing factor (satisfying) ), For the quality of the goods packaging, It is the acceleration due to gravity. This refers to the additional force (inertial force or compensating force) generated due to acceleration, deceleration, or attitude adjustment. This model expresses the fundamental force balance relationship in which the two arms share the heavy load during dynamic processes.

[0083] Building upon this basic model, the algorithm further solves for the optimal load distribution coefficient, maximizing the balance of joint torques between the left and right arms while minimizing the risk of slippage. This optimization is achieved by adjusting the load distribution coefficient through formula calculation. This minimizes the combined weighted joint torque and slippage risk of both arms. Because There is a nonlinear mapping between the torques of each joint (dependent on the robot's kinematics, dynamics, and current configuration), which is usually solved in the [0,1] interval using numerical optimization methods (such as golden section search or gradient descent).

[0084] Based on the optimal load distribution coefficient obtained from the solution The algorithm performs local fine-tuning on the initial motion sequence, specifically including: adjusting the lifting force distribution of the arms; fine-tuning the position of the chassis; and fine-tuning the attitude of the lifting and pitching mechanism. After the above fine-tuning, the final optimal motion sequence is obtained.

[0085] A comprehensive cost evaluation and hard constraint screening of candidate action sequences are performed using a whole-body optimization control algorithm, eliminating any unworkable actions that violate torque, speed, stability, or safe distance requirements, thus ensuring the engineering feasibility of the final execution sequence. The weights of each item in the comprehensive cost formula can be flexibly adjusted according to task requirements, achieving a multi-objective balance between energy consumption, collision, slippage, cycle time, contact force exceeding limits, and personnel proximity risks. Based on this, the optimal load distribution coefficient is solved online, achieving active balancing of left and right arm torques and minimizing slippage risks, avoiding single-arm overload or cargo slippage, and significantly improving safety for heavy-duty handling such as 30-40kg loads. Local fine-tuning of the chassis position and lifting posture further optimizes the overall stability and force transmission efficiency. This step constitutes the core optimization link in the "intelligent generation → optimized screening → precise execution" technical closed loop, effectively solving core problems such as uneven load distribution and insufficient stability margin in heavy-duty handling.

[0086] Optionally, the lightweight digital twin model is generated jointly from a preset model parameter template and real-time scanning data. Using the optimal action sequence as input, the lightweight digital twin model is invoked for simulation verification. The simulation verification includes: Calculate the minimum safe distance from the outline of the robot and cargo to the restricted area of ​​the aircraft structure, as well as the risks of collision, overturning, slippage, excessive contact force, and close proximity to personnel; When any risk exceeds the corresponding preset threshold, the system will enter a slowdown, withdrawal, hold, or request manual takeover mode. Specifically, the manual takeover mode includes: receiving action instructions from staff via a handheld terminal or remote control device, or receiving teaching actions from staff within a safe area; after the takeover is completed, the robot's pose is recalibrated using the multimodal perception data, and the dynamic semantic map is automatically refreshed based on a preset dynamic Bayesian estimation algorithm, without the need for manual repair.

[0087] Specifically, after the whole-body optimization control algorithm generates the optimal action sequence, the sequence is input into the lightweight digital twin model for pre-execution simulation verification. This lightweight digital twin model is generated by two parts of data: one is a preset aircraft parameter template (such as the geometric dimensions and safety envelope of the bulkheads, door frames, and roller guides of the A320 or B737 belly compartment), and the other is the current operation scenario data (such as cargo stacking status, personnel positions, and temporary obstacles) obtained in real time through multimodal sensing components.

[0088] The core content of simulation verification includes: Minimum safe distance calculation: Calculate the minimum distance from each link of the robot, the end effector, and the outline of the cargo to the restricted areas of the aircraft structure (bullets, door frames, guide rails, etc.). : ; in: This represents the sampling point on the j-th link or end effector of the robot. Indicates sampling points on the outer contour of the transported goods; This represents the union of the robot's body and the cargo's outline; A set of points representing restricted areas of the aircraft structure (bullets, door frames, guide rails, etc.). This represents the minimum Euclidean distance from a point to a set of points.

[0089] Among them, collision risk: detect whether the robot or cargo interferes with the aircraft structure, personnel, or temporary obstacles; overturning risk: assess whether the cargo's center of gravity projection exceeds the boundary of the chassis support polygon, and whether the stability margin is sufficient; slippage risk: determine whether the cargo may slip based on the friction cone margin of the gripping point and tactile history; contact force exceeding limit risk: predict whether the contact force between the end effector and the cargo or aircraft structure exceeds the baggage protection or structural safety threshold; personnel proximity risk: determine whether there is a collision hazard based on the distance between the personnel position in the dynamic semantic map and the robot's motion envelope.

[0090] When any of the above risks exceeds the corresponding preset security permission threshold (e.g., minimum security distance) If the robot's tipping margin is less than 1.2, it will not immediately execute the action sequence, but will instead enter a deceleration, withdrawal, hold, or manual takeover mode. The manual takeover mode involves the operator sending action commands (such as "move 10 cm to the left" or "lift the end effector") to the robot via a handheld terminal or teleoperation device, or directly teaching the robot actions within the safe zone (pushing or pulling the robotic arm or recording the trajectory via a teach pendant). During takeover, the robot's underlying safety controller maintains force / position limitation to ensure the safety of both personnel and equipment.

[0091] After the takeover is completed, the robot recalibrates its pose using multimodal perception data (RGB-D, laser point cloud, IMU, etc.) and automatically refreshes the dynamic semantic map based on a preset dynamic Bayesian estimation algorithm, without the need for manual map patching. Subsequently, it can re-invoke the diffusion strategy to generate new candidate actions or continue executing unfinished tasks.

[0092] In some embodiments, after the digital twin simulation verification is completed before execution, the decision on whether to issue the action sequence is made through a security permission gate function based on the comparison results of various risk indicators with preset thresholds: ; in: This is a security clearance indicator; a value of 1 indicates that issuance is permitted. This is an indicator function that takes the value 1 if the condition is true, and 0 otherwise. The minimum safe distance between the robot and cargo outline and the aircraft structure. To preset a safe distance threshold; The contact force between the end effector and the cargo package or aircraft structure. The preset force limit threshold; For the robot's movement speed, The preset speed limit; As an indicator of close proximity risk to personnel, This sets a preset upper limit for close-range personnel risk.

[0093] Only when all indicator functions are 1 Only when the optimal action sequence is sent to the underlying impedance controller for execution will it be executed; otherwise, it will enter the slowdown, withdrawal, hold, or request manual takeover mode.

[0094] By using a lightweight digital twin model to perform multi-dimensional simulation verification of the optimal action sequence (minimum safe distance, collision, overturning, slippage, excessive contact force, and close proximity to personnel), this invention quantifies potential risks before actual execution, effectively avoiding accidents such as aircraft structural scratches, cargo falls, or personnel injuries caused by perception errors, model simplification, or sudden environmental changes. When risks exceed limits, the system can automatically degrade or request manual intervention, supporting both remote operation and teaching modes for human-machine collaboration, ensuring safety redundancy in extreme situations. After intervention, an automatic map refresh and pose recalibration mechanism based on a dynamic Bayesian estimation algorithm enables the robot to quickly return to autonomous operation without manual map maintenance, significantly improving the continuity and robustness of on-site operations. This process constitutes a key safety gate in the closed loop of "generation → screening → twin verification → safe execution → anomaly recovery," providing a verifiable and traceable safety assurance mechanism for the large-scale deployment of aviation ground support robots.

[0095] Optionally, the underlying impedance controller outputs a command force according to a preset control law; the command force is sent to the robot's servo driver and converted into joint torque or end effector force to drive the robot's chassis, lifting mechanism and arms to execute the optimal motion sequence; During execution, the contact force, slip state, and aircraft structural safety distance field are monitored in real time, and rolling temporal replanning is performed based on the real-time updates of the dynamic semantic map. When force exceeding limits, slippage, or personnel entering a dangerous area are detected, local replanning is immediately triggered to generate an avoidance path.

[0096] Specifically, after the optimal action sequence is verified through digital twin simulation, it is sent to the robot's underlying impedance controller for execution. The underlying impedance controller adopts a compliant control law, enabling the robot to maintain high-precision trajectory tracking in free space and actively exhibit low-stiffness, controllable compliant characteristics when in contact with cargo or the environment, in order to avoid rigid impacts.

[0097] The preset control law of the underlying impedance controller is: ; in: This indicates the command force / torque output by the controller to the robot joint or end effector; This represents the damping coefficient matrix (or stiffness coefficient matrix, as defined by the impedance model), used to adjust the end velocity deviation. The response gain; Indicates the desired end-effector pose (position and attitude); This indicates the actual feedback end pose; This represents the stiffness coefficient matrix (or damping coefficient matrix), used to adjust end-position deviation. The response gain; Indicates the desired terminal velocity; Indicates the actual feedback terminal velocity; This represents the feedforward force / torque term, used to compensate for known gravity, friction, or inertial forces, thereby improving dynamic tracking performance.

[0098] This command force The data is sent to the robot's servo drivers, which convert it into actual torque or end effector force at each joint, thereby driving the robot's chassis, lifting mechanism, and arms to move according to the optimal motion sequence. In free space (e.g., during unloaded movement), position and velocity deviations are small, and the controller mainly outputs feedforward force and a small damping force to achieve high-precision trajectory tracking. When the end effector contacts a cargo package or aircraft structure, the position deviation increases, and the controller actively limits the output force to avoid excessive contact impact.

[0099] During execution, the following key states will be monitored in real time: Contact force: The actual contact force between the end effector and the cargo or environment is collected by a six-dimensional force sensor and compared with a preset safety force threshold. Slippage status: The presence of minute slippage on the surface of the cargo package is detected by a tactile array, and the risk of slippage is assessed by combining the friction cone model; Aircraft structural safety distance field: continuously calculates the minimum distances from each link, end effector, and cargo outline of the robot to restricted areas such as bulkheads, door frames, and roller rails.

[0100] Simultaneously, rolling temporal replanning is performed based on real-time updates of the dynamic semantic map (e.g., personnel movement, goods removal, or new stacking). That is, in each control cycle or every few cycles, starting from the current actual state (pose, contact force, map), the strategy is re-diffused to generate candidate actions, optimize and adjust them, and verify them through digital twin simulation, thereby continuously generating new optimal action sequences to replace the original sequences that have not yet been executed, in order to adapt to environmental changes.

[0101] When any of the following abnormal events are detected, local replanning is immediately triggered to generate an avoidance path, without waiting for a full rolling cycle: Force Exceeds Limit: The actual contact force exceeds the preset safety threshold (e.g., the baggage protection force limit or the contact force allowed by the aircraft structure). Slip: The tactile array detects continuous micro-slip or slip acceleration exceeding a threshold; Personnel entering a hazardous area: The dynamic semantic map shows personnel entering the robot's motion envelope or the safe distance field.

[0102] Local replanning can quickly generate an avoidance path from the current location to a safe state (such as retreating to the initial point or the paused position), prioritizing the safety of personnel and equipment.

[0103] The underlying impedance control law achieves a balance between high-precision trajectory tracking and compliant contact, ensuring pose accuracy during heavy-load handling while preventing baggage damage or aircraft structural scratches caused by rigid contact through force / position hybrid control. Real-time monitoring of contact force, slippage state, and safe distance field, combined with rolling temporal replanning using a dynamic semantic map, enables the robot to adaptively respond to dynamic environmental changes (such as personnel movement and cargo shifting), significantly improving operational robustness and safety. When emergency situations such as excessive force, slippage, or personnel intrusion are detected, local replanning is immediately triggered to generate an avoidance path, ensuring high safety in human-robot collaborative operations. This process constitutes a closed-loop end-effector of "perception → planning → execution → monitoring → replanning," providing a stable, safe, and adaptive control foundation for heavy-load handling in aircraft belly cargo holds.

[0104] Optionally, the control method for the aircraft belly cargo / logistics dual-arm heavy-load handling robot further includes suction-grip composite gripping control, the process of which includes: Before grasping, the vacuum adsorption unit on the end effector of the robot is brought into contact with the surface of the cargo package and a negative pressure is established. The compliance of the cargo package surface is evaluated based on the negative pressure establishment speed and stability value. When the vacuum adsorption is determined to be unreliable, the robot, without retracting the robotic arm, controls the flexible gripper fingers to slightly open to release the initial limit on the cargo package, and then coordinates the adjustment of the chassis position and lifting and pitching posture so that the lifting palm surface is inserted into the bottom of the cargo package. Finally, the robot controls the flexible gripper fingers to close again, expanding the contact area with the side wall of the cargo package, and switches to pure lifting mode to complete the grasping.

[0105] Specifically, before the robot performs the gripping action, the suction-grip combined gripping control process is first initiated. The purpose of this process is to adaptively select the optimal gripping mode (adsorption-assisted gripping or pure lifting gripping) based on the surface characteristics of the package, and to smoothly switch between modes if adsorption is unreliable.

[0106] 1. Vacuum Adsorption and Surface Compliance Assessment: The robot's end effector is equipped with a vacuum adsorption unit (including a suction cup, negative pressure sensor, and vacuum generator). After the suction cup contacts the surface of the package, the negative pressure build-up process is initiated, and two key indicators are recorded simultaneously: Negative pressure build-up rate: The time required from the start of contact until the negative pressure reaches the target value (e.g., -60 kPa); Negative pressure stability value: The range of negative pressure fluctuation within a preset time after the target value is reached.

[0107] Based on these two indicators, assess the conformity of the cargo package surface. For example: Rigid trolley cases or plastic cases: The negative pressure build-up speed is fast (<0.5 seconds), the stability value is high and the fluctuation is small, and it is judged as "reliable adsorption"; Woven bags, soft packaging, or cardboard boxes: If the negative pressure builds up slowly (>2 seconds), and the stable value is low or fluctuates greatly (e.g., >±10 kPa), it is judged as "unreliable adsorption".

[0108] 2. Grasping when adsorption is reliable: If adsorption is determined to be reliable, maintain vacuum adsorption directly, while closing the flexible gripper fingers to provide auxiliary limiting, and then perform subsequent lifting and transfer actions.

[0109] 3. Switching to gripping mode when adsorption is unreliable (pure lifting mode): If adsorption is determined to be unreliable, the robot will not force its way into adsorption or retract the arm significantly for a retry. Instead, it will immediately execute the following sequence of actions to switch to pure lifting mode in place: Step 1: Slightly open the flexible finger clamp: Slightly open the flexible gripper fingers (e.g., open them 5-10 mm) to release any slight compression or restriction that may have been applied to the package, making room for subsequent insertion of the palm.

[0110] Step 2: Coordinated adjustment of chassis and lifting mechanism: Adjust the chassis position (e.g., move it forward slowly by 3-8 cm) and the posture of the lifting and tilting mechanism (e.g., lower the shoulder platform height by 2-5 cm, or tilt it slightly) so that the lifting palm of the end effector is aligned with the gap between the bottom of the cargo package and the support surface (conveyor belt, cargo compartment floor, or lower cargo).

[0111] Step 3: Support the palm with the insertion at the bottom: Continue fine-tuning to smoothly insert the palm of the hand into the bottom of the bag until the palm fully supports the main weight of the bag (this can be determined by the six-dimensional force sensor sensing that the normal force reaches more than 80% of the expected gravity value).

[0112] Step 4: Softly close the finger clamp again: The flexible gripper fingers are controlled to close again, forming a large area of ​​flexible contact with the side wall of the cargo bag (rather than point gripping), which serves to limit positioning and prevent slippage. At this time, the vacuum adsorption unit can be turned off or used only as a posture aid, and the weight of the cargo bag is entirely borne by the supporting palm surface, with the gripper fingers providing lateral stability.

[0113] The entire switching process is completed within seconds, with the robotic arm's end effector always close to the package, eliminating the need for retraction or repositioning.

[0114] 4. Grasping Completed: After the switch is completed, the robot continues to hold the object in a lifting state and begins to perform subsequent transfer and stacking actions. During the transfer process, the tactile array and six-dimensional force data are continuously monitored. If a slippage trend is detected, the finger gripping pressure and the lifting palm angle can be further fine-tuned.

[0115] Optionally, the control method for the aircraft belly cargo / logistics dual-arm heavy-load handling robot further includes a data flywheel strategy, which includes: After each operation is completed, the operation data, including the success rate of grasping, force and tactile waveform, reasons for failure, placement photos and manual intervention records, will be uploaded to the edge-cloud collaborative system. When the cumulative number of failed samples in the same scenario or the same type of cargo reaches a preset threshold, the conditional diffusion strategy model fine-tuning training in the offline environment is triggered.

[0116] Specifically, the data flywheel strategy aims to drive offline optimization and online updates of the model by continuously collecting real-world operational data, forming a self-evolving closed loop of "data collection → model training → deployment optimization → re-collection".

[0117] After each operation, the robot system automatically packages and uploads the following operational data to the edge-cloud collaborative system (edge ​​server or cloud platform): Success rate: The ratio of the number of successful captures to the total number of attempts in this task; Force-tactile waveform: The complete time-domain signal of the six-dimensional force sensor and tactile array throughout the entire grasping, lifting, transporting, and stacking process; Reasons for failure: When a capture fails or an anomaly occurs, the recorded fault codes (such as "negative pressure establishment timeout", "slippage overlimit", "collision warning", "torque overlimit", etc.) are recorded. Stacking photos: On-site images before and after the operation and after the stacking is completed, used for manual verification and visual documentation; Manual takeover record: If manual takeover is triggered during the operation, record the takeover time, takeover method (remote operation or teaching), manually input action commands, and execution results after recovery.

[0118] The edge-cloud collaborative system categorizes, stores, and statistically analyzes uploaded data. Specifically, it clusters failed samples according to scenario type (e.g., belly compartments of different aircraft models, conveyor belt areas, ULD pallet areas) and package type (e.g., hard cases, soft cases, cardboard boxes, and membrane-wrapped packages). When the cumulative number of failed samples in the same scenario or package type reaches a preset threshold (e.g., the number of failures for the same package type exceeds 20), it automatically triggers fine-tuning training of the conditional diffusion strategy model in an offline environment.

[0119] The specific process for fine-tuning the training is as follows: Extract all failure samples and some success samples for this scenario or package type from the data warehouse to build a dedicated fine-tuning dataset; In the offline training environment, the original pre-trained model is used as the initial weights, and several rounds of iterative training are performed on the fine-tuning dataset with a small learning rate. This allows the model to focus on the state-action distribution corresponding to the failure samples, thereby improving its adaptability to difficult scenarios. After training, the new model is first validated in a digital twin environment in typical scenarios (such as test cases simulating narrow passages in the cargo compartment, sliding boundaries of soft packs, and heavy-load eccentricity) to ensure that performance does not degrade and the failure rate is significantly reduced. Once the verification is successful, the new model will be deployed to the robot's online inference module to replace the original model for subsequent tasks.

[0120] In addition, the data flywheel strategy also supports active learning with human intervention: when staff teach the robot in human-controlled mode, the teaching trajectory is added to the dataset as a positive sample for the next fine-tuning, enabling the robot to learn better grasping strategies from human-robot collaboration.

[0121] By employing a data flywheel strategy, successful and unsuccessful experiences from each operation are transformed into quantifiable training samples, enabling continuous model optimization and scenario adaptation after actual deployment. The edge-cloud collaborative system clusters and statistically analyzes failed samples by scenario and cargo type. When the cumulative failures reach a threshold, offline fine-tuning is automatically triggered, avoiding ineffective and frequent training while ensuring that model updates occur precisely in scenarios where real-world problems are concentrated. The offline-trained model is verified and redeployed via digital twin, forming a safe and controllable model upgrade closed loop, effectively preventing new risks that may be introduced during training. This strategy enables the robot to learn from its own failures and human intervention experiences, gradually improving its generalization ability to grasp challenging cargo packages (such as soft-sided bags and off-center luggage). This significantly reduces the failure rate and the need for human intervention in long-term operation, serving as a key mechanism for achieving sustainable evolution from "deployment → data collection → learning → optimization → redeployment," and also providing a data foundation for rapid migration across airports and cargo types.

[0122] like Figure 2 As shown in the figure, an embodiment of the present invention provides a control system for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, used to implement the aforementioned control method for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics. The system includes: The acquisition unit is used to acquire task instructions and multimodal perception data, construct and update a dynamic semantic map based on the multimodal perception data and robot state data, and parse the task instructions into executable task graph data. The processing unit is used to encode the current dynamic semantic map, the task graph data, and the historical contact state to obtain three-dimensional observation features, input the three-dimensional observation features into a pre-trained conditional diffusion strategy model, generate one or more candidate double-arm action sequences, and filter and locally adjust the candidate double-arm action sequences through a whole-body optimization control algorithm under preset comprehensive constraints to obtain the optimal action sequence. The simulation unit is used to call the lightweight digital twin model to perform simulation verification before execution based on the optimal action sequence; The execution unit is used to send the optimal action sequence to the robot's underlying impedance controller for execution when all risks verified by the simulation are lower than the preset safety permission threshold.

[0123] This invention provides a control device for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, comprising a memory and a processor; the memory is used to store computer programs; the processor is used to implement the control method for the dual-arm heavy-load handling robot for aircraft cargo hold / logistics as described above when the computer programs are executed.

[0124] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described above.

[0125] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics, characterized in that, include: Acquire task instructions and multimodal perception data, construct and update a dynamic semantic map based on the multimodal perception data and robot state data, and parse the task instructions into executable task graph data; The dynamic semantic map, the task graph data, and the historical contact state at the current moment are encoded to obtain three-dimensional observation features. The three-dimensional observation features are then input into a pre-trained conditional diffusion strategy model to generate one or more candidate bi-arm action sequences. The optimal action sequence is obtained by screening and locally adjusting the candidate dual-arm action sequences through a whole-body optimization control algorithm under preset comprehensive constraints. Based on the optimal action sequence, a lightweight digital twin model is invoked for pre-execution simulation verification; When all risks verified by the simulation are lower than the preset safety threshold, the optimal action sequence is sent to the robot's underlying impedance controller for execution. The control method for the aircraft belly cargo / logistics dual-arm heavy-load handling robot also includes suction-grip composite gripping control, the process of which includes: Before grasping, the vacuum adsorption unit on the end effector of the robot is brought into contact with the surface of the cargo package and a negative pressure is established. The compliance of the cargo package surface is evaluated based on the negative pressure establishment speed and stability value. When the vacuum adsorption is determined to be unreliable, the robot, without retracting the robotic arm, controls the flexible gripper fingers to slightly open to release the initial limit on the cargo package, and then coordinates the adjustment of the chassis position and lifting and pitching posture so that the lifting palm surface is inserted into the bottom of the cargo package. Finally, the robot controls the flexible gripper fingers to close again, expand the contact area with the side wall of the cargo package, and switches to pure lifting mode to complete the grasping. The step of inputting the three-dimensional observation features into a pre-trained conditional diffusion strategy model to generate one or more candidate bi-arm movement sequences includes: The three-dimensional observation features are used as conditions and input into a pre-trained conditional diffusion strategy model. Through an iterative denoising process, one or more candidate double-arm action sequences are gradually recovered from random noise. Each candidate double-arm action sequence includes the left and right arm grasping posture, end pose, relative configuration of the arms and motion trajectory. The relative configuration of the arms includes the switching trajectory between narrow-space passage state and wide-space transport state. The pre-trained conditional diffusion strategy model is as follows: ； in, Indicates parameterization Conditional diffusion strategy model; This represents the candidate sequence of arm movements from the current time t to H future steps; This represents the multimodal observation data at the current moment; The task conditions include the target package identity, stacking location, and handling sequence constraints in the task graph data. Denoising network for conditional diffusion model; This represents the multimodal observation data at the current moment. The three-dimensional observation features obtained after three-dimensional encoding.

2. The control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described in claim 1, characterized in that, The construction and updating of the dynamic semantic map based on the multimodal perception data and robot state data includes: The RGB-D image, laser point cloud, and robot pose in the multimodal perception data are spatiotemporally aligned to generate a three-dimensional occupied grid map, which serves as the geometric layer data. The RGB-D image is semantically segmented to identify cargo packages, personnel, and aircraft structure. The identification results are then superimposed onto the geometric layer data to form semantic layer data. Based on the preset dynamic Bayesian estimation algorithm, the dynamic semantic map of the previous moment, the semantic layer data and geometric layer data of the current moment, the current pose in the robot state data, and the action executed in the previous moment are taken as inputs to perform recursive state updates and output the dynamic semantic map of the current moment. In the dynamic semantic map at the current moment, each aircraft structure semantic node is expanded outward by a preset distance to generate an aircraft structure safety distance field. This safety distance field is then superimposed onto the dynamic semantic map at the current moment to obtain the updated dynamic semantic map.

3. The control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described in claim 1, characterized in that, The process of parsing the task instructions into executable task graph data further includes: Extract the package weight, handling frequency, and target location from the task instructions; An efficiency risk score is obtained based on the cargo package quality, the handling frequency, and the target location. Subtasks whose work efficiency risk scores exceed a preset threshold are assigned a higher execution priority in the task graph data.

4. The control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described in claim 1, characterized in that, The process involves using a whole-body optimization control algorithm to filter and locally adjust the candidate dual-arm movement sequences under preset comprehensive constraints to obtain the optimal movement sequence, including: Calculate the comprehensive cost for each candidate dual-arm motion sequence, and select the candidate dual-arm motion sequence with the lowest comprehensive cost as the initial motion sequence, provided that the preset joint speed upper limit constraint, joint torque upper limit constraint, chassis stability margin constraint and aircraft structural safety distance constraint are met in the comprehensive constraints. On the execution trajectory of the initial action sequence, the optimal load distribution coefficient is solved online, where the solution formula is: ； in: This represents the optimal load-sharing factor for the left arm; The left arm load-sharing coefficient is the independent variable, and the right arm load-sharing coefficient is... ; This indicates that the left arm has a sharing coefficient of 1. The joint torque vector at that time; This indicates that the right arm has a sharing coefficient of 1. The joint torque vector at that time; It represents the norm weighted by the weight matrix Q, where Q is the weight matrix of the joint torques; Preset weighting coefficients for the slippage risk item; This indicates the slippage risk item, calculated based on the friction cone safety margin at the current gripping point and the tactile slippage detection results. Based on the optimal load distribution coefficient obtained by the solution, the lifting force of the two arms, the chassis position, and the lifting and pitching attitude are locally fine-tuned to generate the optimal action sequence.

5. The control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described in claim 1, characterized in that, The lightweight digital twin model is generated from a preset model parameter template and real-time scanning data. Using the optimal action sequence as input, the lightweight digital twin model is invoked for simulation verification. The simulation verification includes: Calculate the minimum safe distance from the outline of the robot and cargo to the restricted area of ​​the aircraft structure, as well as the risks of collision, overturning, slippage, excessive contact force, and close proximity to personnel; When any risk exceeds the corresponding preset threshold, the system will enter a slowdown, withdrawal, hold, or request manual takeover mode. Specifically, the manual takeover mode includes: receiving action instructions from staff via a handheld terminal or remote control device, or receiving teaching actions from staff within a safe area; after the takeover is completed, the robot's pose is recalibrated using the multimodal perception data, and the dynamic semantic map is automatically refreshed based on a preset dynamic Bayesian estimation algorithm, without the need for manual repair.

6. The control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described in claim 1, characterized in that, The underlying impedance controller outputs command force according to a preset control law; The command force is sent to the robot's servo drive and converted into joint torque or end effector force to drive the robot's chassis, lifting mechanism and arms to execute the optimal motion sequence; During execution, the contact force, slip state, and aircraft structural safety distance field are monitored in real time, and rolling temporal replanning is performed based on the real-time updates of the dynamic semantic map. When force exceeding limits, slippage, or personnel entering a dangerous area are detected, local replanning is immediately triggered to generate an avoidance path.

7. The control method for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics as described in claim 1, characterized in that, The control method for the aircraft belly cargo / logistics dual-arm heavy-load handling robot also includes a data flywheel strategy, which includes: After each operation is completed, the operation data, including the success rate of grasping, force and tactile waveform, reasons for failure, placement photos and manual intervention records, will be uploaded to the edge-cloud collaborative system. When the cumulative number of failed samples in the same scenario or the same type of cargo reaches a preset threshold, the conditional diffusion strategy model fine-tuning training in the offline environment is triggered.

8. A control system for a dual-arm heavy-load handling robot for aircraft belly cargo / logistics, characterized in that, For implementing the control method of the aircraft belly cargo / logistics dual-arm heavy-load handling robot as described in claims 1 to 7, the system includes: The acquisition unit is used to acquire task instructions and multimodal perception data, construct and update a dynamic semantic map based on the multimodal perception data and robot state data, and parse the task instructions into executable task graph data. The processing unit is used to encode the current dynamic semantic map, the task graph data, and the historical contact state to obtain three-dimensional observation features, input the three-dimensional observation features into a pre-trained conditional diffusion strategy model, generate one or more candidate double-arm action sequences, and filter and locally adjust the candidate double-arm action sequences through a whole-body optimization control algorithm under preset comprehensive constraints to obtain the optimal action sequence. The simulation unit is used to call the lightweight digital twin model to perform simulation verification before execution based on the optimal action sequence; The execution unit is used to send the optimal action sequence to the robot's underlying impedance controller for execution when all risks verified by the simulation are lower than the preset safety permission threshold.

9. A control device for a dual-arm heavy-load handling robot for aircraft cargo hold / logistics, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the control method for the aircraft belly cargo / logistics dual-arm heavy load handling robot as described in any one of claims 1 to 7.

Images