VLA model-based intelligent autonomous cargo handling control method

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a VLA model-based intelligent autonomous cargo handling control method, which comprises the following steps of: acquiring visual observation information containing target cargo and environment, and receiving a natural language task instruction; inputting the visual observation information and the natural language task instruction into a pre-trained VLA model, and analyzing to generate a structured semantic task description; based on the structured semantic task description and the real-time environment state, mapping and outputting a computable motion safety constraint set; taking the current state of the intelligent body as an initial point, taking the motion safety constraint set as a hard boundary condition, and generating a feasible track meeting all safety constraints; the feasible track is calculated into a direct control instruction of a robot driving unit, the direct control instruction is issued to an execution system, structural safety constraint is achieved through a VLA model, and the problems that a traditional robot system is poor in adaptability and low in safety when facing a complex environment are solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an embodied intelligent autonomous cargo handling control method based on a VLA model. Background Technology

[0002] In smart logistics and smart factories, enabling robots to understand natural language commands and safely execute handling tasks is a core challenge for improving system flexibility and achieving efficient human-machine collaboration. Current mainstream technical solutions fall into three main categories: systems based on precise environment modeling and pre-programming are reliable but suffer from poor environmental adaptability, task rigidity, and an inability to respond to non-preset commands; systems based on traditional vision and rule-based reasoning have weak semantic understanding capabilities and insufficient generalization, making it difficult to handle complex scenarios and semantic constraints; and solutions based on end-to-end deep reinforcement learning, while possessing some adaptive capabilities, have a decision-making process that is like a "black box," lacking explicit safety mechanisms, and task assignment relies on complex reward function design, making it difficult to directly respond to natural language commands.

[0003] In recent years, the development of large-scale visual language models has provided a new path for solving semantic understanding problems. However, when directly applied to physical robot control, a "semantic gap" emerges: there is a serious disconnect between the high-level instructions output by the model and the robot's underlying continuous, safe, and precise motion control requirements. Directly outputting action instructions suffers from high latency, large uncertainty, and completely ignores the dynamics and safety constraints of the physical world, resulting in low reliability and high risk when deployed in real physical environments. Therefore, how to organically integrate the open-world understanding capabilities of large-scale models with the safety and reliability assurance mechanisms of classical control theory has become a key technical bottleneck that urgently needs to be overcome in this field. Summary of the Invention

[0004] This application provides an embodied intelligent autonomous cargo handling control method based on a VLA model. By constructing a technical framework that uses a VLA model for high-level semantic understanding, a dedicated neural network to map semantics into formal safety constraints, and an optimization controller to generate trajectories under strict constraints, this method solves the technical problem of severe disconnect between semantic understanding and physical control in existing robot systems when dealing with dynamic unstructured environments and complex natural language instructions. It achieves the technical effect of automatically converting high-level semantic instructions into mathematically calculable motion safety constraints, and using these constraints as hard boundaries to generate safe and feasible trajectories. Thus, while ensuring operational safety and reliability, it significantly improves the robot's understanding and execution capabilities for handling tasks.

[0005] To achieve the above objectives, this invention provides an embodied intelligent autonomous cargo handling control method based on a VLA model, comprising: Acquire visual observation information including the target cargo and its environment, and receive natural language task instructions; The visual observation information and the natural language task instructions are input into a pre-trained VLA model to parse and generate a structured semantic task description; Based on the structured semantic task description and real-time environment state, the mapping output is a computable set of motion safety constraints; Starting from the current state of the embodied intelligence, the set of motion safety constraints is used as hard boundary conditions to generate a feasible trajectory that satisfies all safety constraints. The feasible trajectory is calculated into direct control commands for the robot drive unit and sent to the execution system.

[0006] Furthermore, the mapping output is a computable set of motion safety constraints, specifically including: The structured semantic task description is encoded into a semantic feature vector, and the real-time environment state is encoded into a spatial feature vector; The semantic feature vector and the spatial feature vector are fused in a multimodal manner to generate a joint feature representation; The joint feature representation is input into a constraint parameter prediction network, which predicts and outputs the parameterized mathematical expression coefficients or constraint boundary values ​​required to define the set of motion safety constraints.

[0007] Furthermore, the constrained parameter prediction network specifically includes: A shared feature encoder is used to perform a nonlinear transformation on the input joint feature representation to extract deep abstract features related to the generation of security constraints; Multiple parallel constraint parameter prediction heads, each associated with a type of security constraint, independently predict specific mathematical parameters used to define the corresponding type of security constraint, taking the deep abstract features as input.

[0008] Furthermore, the semantic feature vector and the spatial feature vector are fused in a multimodal manner, specifically including: The semantic feature vector and spatial feature vector are respectively subjected to feature alignment and dimension normalization. The cross-modal interaction module enables feature interaction between the aligned semantic feature vectors and spatial feature vectors. The interacting features are fused to generate a joint feature representation that includes consistency information between semantic and spatial features.

[0009] Furthermore, the aligned semantic feature vector and spatial feature vector are interacted through a cross-modal interaction module, specifically including: The semantic feature vector is used as the query vector and input into the first input port of the attention module; The spatial feature vectors are input as key vectors and value vectors into the second and third input ports of the attention module, respectively. By calculating the attention weight between semantic feature queries and spatial feature keys, spatial feature values ​​are weighted and aggregated to generate semantically guided spatial context features. The semantically guided spatial context features are residually connected or spliced ​​with the original semantic features to form the interactive features.

[0010] Furthermore, the attention weights between the semantic feature query and the spatial feature key are calculated, specifically including: Perform a matrix dot product operation between the semantic feature query vector and the spatial feature key vector to obtain the original similarity matrix; Divide the original similarity matrix by a scaling factor, where the scaling factor is the square root of the dimension of the spatial feature key vector; The scaled similarity matrix is ​​input into the Softmax function for normalization to obtain the final attention weight distribution.

[0011] Furthermore, the feature is that the set of motion safety constraints is used as hard boundary conditions to generate a feasible trajectory that satisfies all safety constraints, specifically including: Initial conditions for trajectory optimization are constructed based on the current state; The set of motion safety constraints is analyzed as hard boundary conditions for the trajectory optimization problem. The hard boundary conditions include motion space obstacle avoidance inequality constraints, operation process mechanical equality constraints, and system state inequality constraints. By using a constrained trajectory optimization solver, a trajectory that meets the task performance indicators is solved under the hard boundary conditions, generating a feasible trajectory from the current state to the target state.

[0012] Furthermore, a constrained trajectory optimization solver is used to solve for a trajectory that meets the task performance indicators under the hard boundary conditions, specifically including: Within the current control cycle, based on the current state and the hard boundary conditions, a constrained optimization problem is constructed and solved within a finite prediction time domain; The control command at the current moment is extracted from the optimal control sequence obtained from the solution and used as an intermediate output for trajectory generation, and then applied to the robot. In the next control cycle, based on the updated robot state and environmental feedback, the above prediction and optimization process is re-executed to achieve rolling trajectory generation and closed-loop correction.

[0013] Furthermore, the feasible trajectory is calculated into direct control commands for the robot drive unit, specifically including: The feasible trajectory is discretized in the time dimension to obtain a series of robot desired states at discrete time points. Based on the robot's kinematics and dynamics model, the inverse kinematics module is used to calculate the action commands of each drive unit at each discrete time point. The sequence of action instructions is smoothed and frequency adapted to generate a direct control instruction stream that meets the communication protocol and real-time requirements of the actuator. The direct control command stream is sent to the controller of the execution system via the communication bus.

[0014] Furthermore, the inverse calculation module calculates the action commands of each driving unit corresponding to each discrete time point, specifically including: Based on the robot's configuration and the desired state, the angle commands of each joint or the linear and angular velocity commands of the wheeled chassis are calculated using inverse kinematics methods. Based on the angle or speed command, and combined with the robot's dynamic parameters, the desired torque or current command for each drive unit is calculated using inverse dynamics or feedforward control law.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: by combining the VLA model with the field of robot control, the robot can understand complex instructions rich in semantics and constraints, and can automatically convert these high-level intentions into computable mathematical constraints such as precise grasping force range, dynamic obstacle avoidance corridors, and joint motion limits. These constraints serve as inviolable hard boundaries, and by optimizing the solution, motion trajectories that absolutely satisfy all safety conditions are generated in real time. This solves the problem of safety hazards caused by control fragmentation in intelligent robots in dynamic and complex environments, and significantly improves the adaptability and safety of autonomous handling systems. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 The diagram shows a flowchart of an embodied intelligent autonomous cargo handling control method based on a VLA model according to an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0018] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0019] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0021] The following is a description of preferred embodiments of the present invention in conjunction with the accompanying drawings.

[0022] like Figure 1 As shown, an embodiment of the present invention discloses an embodied intelligent autonomous cargo handling control method based on a VLA model, comprising: S110: Acquire visual observation information including the target cargo and environment, and receive natural language task instructions; In this embodiment, the visual observation information including the target cargo and the environment includes the appearance, three-dimensional position and placement of the target cargo, the geometric structure of static obstacles in the environment, and the motion state of dynamic obstacles.

[0023] In this embodiment, the natural language task instructions include: Target indication elements specify the target cargo to be operated and its characteristics; Operation instruction elements specify the types of actions and operating parameters required to perform the handling task; Spatial indication elements define the transport path, destination, or placement posture of goods; Constraints specify the safety, priority, or operational restrictions during task execution.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: by acquiring visual observation information and receiving natural language task instructions, it provides underlying data for the subsequent VLA model understanding and control module, ensuring that the high-level task intent can be unambiguously parsed and ultimately transformed into robot actions that meet specific environmental constraints and safety requirements, thereby improving the reliability and adaptability of the embodied intelligence system in understanding and executing handling tasks in complex dynamic scenarios.

[0025] S120: Input the visual observation information and the natural language task instructions into the pre-trained VLA model to parse and generate a structured semantic task description; In this embodiment, the pre-trained VLA model is a multimodal large language model jointly trained based on large-scale Internet image and text data and robot operation data. Its core capability lies in integrating visual observation information with natural language task instructions and performing reasoning oriented towards physical operations. The parsing process is as follows: First, the visual encoder extracts scene and target features from the visual observation, while the text encoder extracts semantic features from the natural language instructions. Then, the model utilizes its powerful internal cross-modal attention mechanism to perform deep alignment and interaction between visual and text features, identifying the specific visual entities referred to in the instructions, parsing the action intent and its modifiers, and inferring spatial relationships. Finally, the model organizes the above understanding results into a clear, structured semantic task description through a structured output head, providing a machine-parsable semantic representation for subsequent safety constraint generation.

[0026] In this embodiment, the structured semantic task description includes: The target object description includes the bounding box of the target cargo in the image coordinate system, its semantic category, and the inference of key physical attributes. Operation intent description, including the parsing results of the task instructions and the corresponding standardized operation sequence; Spatial relationship description, including the relative pose relationship between the target cargo and other objects in the environment, preset landmarks, or the robot body; Task constraint descriptions include performance constraints, security constraints, or timing logic requirements that are explicitly or implicitly resolved from task instructions.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows: by introducing a pre-trained VLA model and designing a structured output parsing process, visual observation information and received natural language task instructions are decoded into machine-parsable semantic representations, laying a solid foundation for the subsequent accurate mapping from semantics to computable security constraints, and greatly improving the system's understanding of complex and implicit task intentions and execution reliability.

[0028] S130: Based on the structured semantic task description and real-time environment state, the mapping output is a computable set of motion safety constraints; In some embodiments of the present invention, the mapping output is a computable set of motion safety constraints, specifically including: The structured semantic task description is encoded into a semantic feature vector, and the real-time environment state is encoded into a spatial feature vector; The semantic feature vector and the spatial feature vector are fused in a multimodal manner to generate a joint feature representation; The joint feature representation is input into a constraint parameter prediction network, which predicts and outputs the parameterized mathematical expression coefficients or constraint boundary values ​​required to define the set of motion safety constraints.

[0029] In this embodiment, the neural network architecture of the constraint parameter prediction network is a shared-branch architecture.

[0030] In this embodiment, the constraint parameter prediction network specifically includes: A shared feature encoder is used to perform a nonlinear transformation on the input joint feature representation to extract deep abstract features related to the generation of security constraints; Multiple parallel constraint parameter prediction heads, each associated with a type of security constraint, independently predict specific mathematical parameters used to define the corresponding type of security constraint, taking the deep abstract features as input.

[0031] In this embodiment, the key feature is that the semantic feature vector and the spatial feature vector are fused in a multimodal manner, specifically including: The semantic feature vector and spatial feature vector are respectively subjected to feature alignment and dimension normalization. The cross-modal interaction module enables feature interaction between the aligned semantic feature vectors and spatial feature vectors. The interacting features are fused to generate a joint feature representation that includes consistency information between semantic and spatial features.

[0032] In this embodiment, the training of the constraint parameter prediction network is based on the physical simulation environment of PyBullet. Training samples are generated by running a large number of randomized transport tasks. Each sample is input by a joint feature representation generated based on the current scene and instructions, and the real constraint parameters that can ensure the safe and collision-free execution of the task are calculated based on the classic robot safety rule algorithm as supervision labels. During training, the weighted mean square error between the parameters output by multiple prediction heads of the network and the corresponding real labels is used as the loss function. The network weights are continuously adjusted by the gradient descent optimization algorithm until the network can accurately predict the formal safety constraint parameters from the joint features, thereby establishing a reliable mapping relationship from open-world semantic understanding to precise mathematical constraints.

[0033] In this embodiment, the feature is that the aligned semantic feature vector and spatial feature vector are interacted through a cross-modal interaction module, specifically including: The semantic feature vector is used as the query vector and input into the first input port of the attention module; The spatial feature vectors are input as key vectors and value vectors into the second and third input ports of the attention module, respectively. By calculating the attention weight between semantic feature queries and spatial feature keys, spatial feature values ​​are weighted and aggregated to generate semantically guided spatial context features. The semantically guided spatial context features are residually connected or spliced ​​with the original semantic features to form the interactive features.

[0034] In this embodiment, the key feature is that calculating the attention weight between the semantic feature query and the spatial feature key specifically includes: Perform a matrix dot product operation between the semantic feature query vector and the spatial feature key vector to obtain the original similarity matrix; Divide the original similarity matrix by a scaling factor, where the scaling factor is the square root of the dimension of the spatial feature key vector; The scaled similarity matrix is ​​input into the Softmax function for normalization to obtain the final attention weight distribution.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows: by encoding structured semantic tasks and fusing multimodal approaches, the high-level structured semantic description is deeply integrated with the low-level real-time environment. Furthermore, by utilizing a dedicated neural network based on a shared-branch architecture, the computable mathematical constraint parameters are predicted end-to-end from the joint features. This enables the transformation of semantic instructions into specific, directly executable hard safety boundaries, ensuring the operational safety and task reliability of the embodied intelligent system in complex dynamic environments.

[0036] S140: Taking the current state of the embodied intelligence as the initial point, the set of motion safety constraints is used as hard boundary conditions to generate a feasible trajectory that satisfies all safety constraints; In some embodiments of the present invention, the set of motion safety constraints is used as hard boundary conditions to generate feasible trajectories that satisfy all safety constraints, specifically including: Initial conditions for trajectory optimization are constructed based on the current state; The set of motion safety constraints is analyzed as hard boundary conditions for the trajectory optimization problem. The hard boundary conditions include motion space obstacle avoidance inequality constraints, operation process mechanical equality constraints, and system state inequality constraints. By using a constrained trajectory optimization solver, a trajectory that meets the task performance indicators is solved under the hard boundary conditions, generating a feasible trajectory from the current state to the target state.

[0037] In this embodiment, the task performance index is specified as a comprehensive quantitative requirement for trajectory execution efficiency and quality. The core of this index is to simultaneously minimize the deviation between the robot's actual motion trajectory and the expected path of the task, and to minimize the amplitude and energy consumption of control actions, thereby generating a smooth and energy-efficient optimal or suboptimal trajectory while satisfying all safety constraints. Furthermore, the motion safety constraint set is parsed into hard boundary conditions by directly substituting the specific parameter values ​​output by the constraint parameter prediction network into and instantiating a series of predefined mathematical inequalities associated with the robot's dynamics model and environment model. For example, the "joint velocity upper limit" value predicted by the network is directly used as the right boundary value of the velocity inequality constraint, and the predicted "real-time safe distance of dynamic obstacles" value is directly used as the threshold that the distance between the robot's end position and the obstacle must be greater than or equal to. Thus, these abstract safety requirements derived from semantics are transformed into concrete and computable mathematical constraints that each optimization variable in the trajectory optimization solver must strictly satisfy at each time point.

[0038] In this embodiment, a constrained trajectory optimization solver is used to solve for a trajectory that meets the task performance indicators under the hard boundary conditions, specifically including: Within the current control cycle, based on the current state and the hard boundary conditions, a constrained optimization problem is constructed and solved within a finite prediction time domain; The control command at the current moment is extracted from the optimal control sequence obtained from the solution and used as an intermediate output for trajectory generation, and then applied to the robot. In the next control cycle, based on the updated robot state and environmental feedback, the above prediction and optimization process is re-executed to achieve rolling trajectory generation and closed-loop correction.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows: by setting hard boundary conditions and driving trajectory generation with comprehensive quantitative indicators within this framework, the optimal synergy between safety and motion performance is achieved; it ensures that the robot can execute all specific safety restrictions derived from abstract instructions; and through the rolling optimization and closed-loop correction mechanism of model predictive control, the system can adapt to dynamic environmental changes and model uncertainties online, thereby performing normal work tasks in complex unstructured handling scenarios.

[0040] S150: The feasible trajectory is calculated into direct control commands for the robot drive unit and sent to the execution system.

[0041] In some embodiments of the present invention, the feasible trajectory is calculated into direct control commands for the robot drive unit, specifically including: The feasible trajectory is discretized in the time dimension to obtain a series of robot desired states at discrete time points. Based on the robot's kinematics and dynamics model, the inverse kinematics module is used to calculate the action commands of each drive unit at each discrete time point. The sequence of action instructions is smoothed and frequency adapted to generate a direct control instruction stream that meets the communication protocol and real-time requirements of the actuator. The direct control command stream is sent to the controller of the execution system via the communication bus.

[0042] In this embodiment, the inverse calculation module is the core converter in the robot control system that transforms high-level trajectory command planning into specific driving commands of the low-level actuators. Its working principle is based on the robot's geometric model and physical models such as mass distribution and inertia. Through mathematical calculation, it reverse-calculates the precise angle, speed, and even the torque or current value that needs to be output in order to accurately track each planning point for each joint or wheel. In this way, it "translates" and executes the motion intention of the command at the physical level.

[0043] In this embodiment, the inverse calculation module calculates the action commands of each driving unit corresponding to each discrete time point, specifically including: Based on the robot's configuration and the desired state, the angle commands of each joint or the linear and angular velocity commands of the wheeled chassis are calculated using inverse kinematics methods. Based on the angle or speed command, and combined with the robot's dynamic parameters, the desired torque or current command for each drive unit is calculated using inverse dynamics or feedforward control law.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows: by designing an inverse calculation module, feasible trajectories are converted into torque or current commands that can be directly executed by the underlying drive unit, and by smoothing and communication adaptation processing, the real-time performance and stability of the command stream are ensured.

[0045] In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0046] Although the invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the embodiments disclosed in this invention can be combined with each other in any way. The fact that not all of these combinations are described in this specification is merely for the sake of brevity and resource conservation.

[0047] It will be understood by those skilled in the art that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for embodied intelligent autonomous cargo handling control based on a VLA model, characterized in that, include: Acquire visual observation information including the target cargo and its environment, and receive natural language task instructions; The visual observation information and the natural language task instructions are input into a pre-trained VLA model to parse and generate a structured semantic task description; Based on the structured semantic task description and real-time environment state, the mapping output is a computable set of motion safety constraints; Starting from the current state of the embodied intelligence, the set of motion safety constraints is used as hard boundary conditions to generate a feasible trajectory that satisfies all safety constraints. The feasible trajectory is calculated into direct control commands for the robot drive unit and sent to the execution system.

2. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 1, characterized in that, The mapping output is a computable set of motion safety constraints, specifically including: The structured semantic task description is encoded into a semantic feature vector, and the real-time environment state is encoded into a spatial feature vector; The semantic feature vector and the spatial feature vector are fused in a multimodal manner to generate a joint feature representation; The joint feature representation is input into a constraint parameter prediction network, which predicts and outputs the parameterized mathematical expression coefficients or constraint boundary values ​​required to define the set of motion safety constraints.

3. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 2, characterized in that, Constrained parameter prediction networks, specifically including: A shared feature encoder is used to perform a nonlinear transformation on the input joint feature representation to extract deep abstract features related to the generation of security constraints; Multiple parallel constraint parameter prediction heads, each associated with a type of security constraint, independently predict specific mathematical parameters used to define the corresponding type of security constraint, taking the deep abstract features as input.

4. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 2, characterized in that, Multimodal fusion of the semantic feature vector and the spatial feature vector specifically includes: The semantic feature vector and spatial feature vector are respectively subjected to feature alignment and dimension normalization. The cross-modal interaction module enables feature interaction between the aligned semantic feature vectors and spatial feature vectors. The interacting features are fused to generate a joint feature representation that includes consistency information between semantic and spatial features.

5. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 4, characterized in that, The cross-modal interaction module facilitates feature interaction between the aligned semantic feature vectors and spatial feature vectors, specifically including: The semantic feature vector is used as the query vector and input into the first input port of the attention module; The spatial feature vectors are input as key vectors and value vectors into the second and third input ports of the attention module, respectively. By calculating the attention weight between semantic feature queries and spatial feature keys, spatial feature values ​​are weighted and aggregated to generate semantically guided spatial context features. The semantically guided spatial context features are residually connected or spliced ​​with the original semantic features to form the interactive features.

6. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 5, characterized in that, Calculating the attention weights between semantic feature queries and spatial feature keys specifically includes: Perform a matrix dot product operation between the semantic feature query vector and the spatial feature key vector to obtain the original similarity matrix; Divide the original similarity matrix by a scaling factor, where the scaling factor is the square root of the dimension of the spatial feature key vector; The scaled similarity matrix is ​​input into the Softmax function for normalization to obtain the final attention weight distribution.

7. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 1, characterized in that, Using the set of motion safety constraints as hard boundary conditions, feasible trajectories satisfying all safety constraints are generated, specifically including: Initial conditions for trajectory optimization are constructed based on the current state; The set of motion safety constraints is analyzed as hard boundary conditions for the trajectory optimization problem. The hard boundary conditions include motion space obstacle avoidance inequality constraints, operation process mechanical equality constraints, and system state inequality constraints. By using a constrained trajectory optimization solver, a trajectory that meets the task performance indicators is solved under the hard boundary conditions, generating a feasible trajectory from the current state to the target state.

8. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 7, characterized in that, A constrained trajectory optimization solver is used to solve for a trajectory that meets the task performance indicators under the hard boundary conditions, specifically including: Within the current control cycle, based on the current state and the hard boundary conditions, a constrained optimization problem is constructed and solved within a finite prediction time domain; The control command at the current moment is extracted from the optimal control sequence obtained from the solution and used as an intermediate output for trajectory generation, and then applied to the robot. In the next control cycle, based on the updated robot state and environmental feedback, the above prediction and optimization process is re-executed to achieve rolling trajectory generation and closed-loop correction.

9. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 1, characterized in that, The feasible trajectory is calculated into direct control commands for the robot drive unit, specifically including: The feasible trajectory is discretized in the time dimension to obtain a series of robot desired states at discrete time points. Based on the robot's kinematics and dynamics model, the inverse kinematics module is used to calculate the action commands of each drive unit at each discrete time point. The sequence of action instructions is smoothed and frequency adapted to generate a direct control instruction stream that meets the communication protocol and real-time requirements of the actuator. The direct control command stream is sent to the controller of the execution system via the communication bus.

10. The embodied intelligent autonomous cargo handling control method based on the VLA model according to claim 9, characterized in that, The inverse calculation module calculates the action commands of each driving unit corresponding to each discrete time point, specifically including: Based on the robot's configuration and the desired state, the angle commands of each joint or the linear and angular velocity commands of the wheeled chassis are calculated using inverse kinematics methods. Based on the angle or speed command, and combined with the robot's dynamic parameters, the desired torque or current command for each drive unit is calculated using inverse dynamics or feedforward control law.

Images