Space robot semantic-control instruction conversion method based on large language model

Abstract

The application discloses a space robot semantic-control instruction conversion method based on a large language model, and belongs to the field of spacecraft decision planning and artificial intelligence. The application solves the problem that existing space robots are difficult to directly convert natural language instructions into accurate actions meeting physical constraints and are highly dependent on manual parameter adjustment. The application first analyzes the natural language instruction into a structured intention through a large language model, extracts a semantic feature vector, and adaptively configures a task execution mode and optimization weight; then constructs a pseudo three-dimensional voxel space in a base coordinate system and generates a composite value graph, and extracts a path point set through gradient ascent; finally, trajectory optimization is performed in combination with the optimization weight and time domain parameters, and when the verification fails, autonomous recovery is performed through path point sparsification, direct solution of an edge value problem or dynamic adjustment of a total planning time, so that control instructions meeting the dynamics constraints of the space robot are finally generated. The application is suitable for natural language command conversion of a free-flying space robot.

Description

Technical Field

[0001] This invention belongs to the field of spacecraft decision-making and planning and artificial intelligence, specifically relating to a space robot intelligent agent system based on a large language model. Background Technology

[0002] With the increasing frequency of human space activities, tasks such as on-orbit servicing and space debris removal place higher demands on the autonomy and intelligence of space robots. Free-Flying Space Robots (FFSRs), as electromechanical devices operating in the microgravity environment of space, are fundamentally different from ground-based robots. Because their base is floating, the system operates strictly according to the law of conservation of momentum, resulting in extremely strong dynamic coupling between the base and the robotic arm. In actual operation, any movement of the robotic arm will disturb the attitude of the base, and the displacement of the base will in turn affect the working accuracy of the end effector. This mutually interfering characteristic makes the precise planning and coordinated control of FFSRs extremely challenging.

[0003] Currently, trajectory planning and control methods for FFSRs mainly focus on motion primitives, reinforcement learning, and nonlinear optimization. However, traditional methods face significant bottlenecks when dealing with increasingly complex on-orbit tasks. First, methods based on motion primitives are usually designed for specific tasks, lack generalization ability, and rely heavily on manually pre-defined logic. Second, while reinforcement learning exhibits some adaptability, its safety is difficult to guarantee mathematically under complex coupled dynamic constraints, and its training cost is extremely high. Furthermore, although methods based on nonlinear programming or model predictive control can handle physical constraints, the weight coefficients and penalty parameters in their cost functions often require engineers with specialized backgrounds to repeatedly tune them for different task scenarios, which greatly limits the robot's autonomous operation capabilities in unpredictable environments.

[0004] In recent years, Large Language Models (LLMs) have made groundbreaking progress in semantic reasoning and natural language interaction, enabling robots to make advanced decisions. However, in the highly safety-critical field of aerospace, the direct application of LLMs faces a serious semantic-physical gap. While LLMs excel at logical deduction, they lack an intuitive understanding of FFSR dynamics, easily leading to a "semantic illusion" that ignores momentum conservation or physical constraints. Furthermore, large models trained on three-dimensional corpora often struggle to generate physically consistent spatial coordinates when faced with a workspace with specific dimensional constraints. Summary of the Invention

[0005] This application aims to address the problem that existing space robot decision-making and planning methods struggle to directly convert unstructured natural language instructions into precise actions that meet physical constraints, and heavily rely on manual parameter tuning, resulting in poor autonomy and insufficient fault tolerance when performing diverse tasks. Therefore, this application provides a space robot semantic-control instruction conversion method based on a large language model.

[0006] The present invention discloses a semantic-control instruction conversion method for space robots based on a large language model, the method comprising:

[0007] Step 1: Receive natural language instructions, parse the natural language instructions into structured intents through the base big language model, extract three-dimensional semantic feature vectors from the structured intents, determine the current task execution mode based on the three-dimensional semantic feature vectors through priority logic, and generate corresponding optimization weight vectors and time-domain parameters based on the current task execution mode to form a structured configuration parameter set;

[0008] Step 2: According to the structured intent, the task space under the base coordinate system is discretized in a single layer in the height direction to construct a pseudo-three-dimensional voxel space. In the pseudo-three-dimensional voxel space, a composite value map is generated by weighted superposition of the target attraction field and the obstacle avoidance field. Based on the composite value map, the path point set is recursively extracted from the starting point using the gradient ascent algorithm.

[0009] Step 3: Using the structured configuration parameter set, perform code-level safety verification, kinematic reachability verification, and dynamic feasibility verification on the path point set in sequence. After the verification is passed, input the path point set as the initial path and the structured configuration parameter set together into the nonlinear programming solver to perform trajectory optimization, generate a nominal trajectory that satisfies the dynamic constraints of the space robot, and convert the nominal trajectory into control instructions to be executed by the space robot.

[0010] Step 4: If any check in Step 3 fails or the nonlinear programming solver fails to generate a nominal trajectory that satisfies the space robot's dynamic constraints, execute:

[0011] The path point set is sampled a second time to generate a sparse path point subset. The sparse path point subset is used as the new path point set, and step three is repeated until all verifications pass and the nonlinear programming solver generates a nominal trajectory that satisfies the dynamic constraints of the space robot.

[0012] Alternatively, a boundary value problem from the initial state to the target state can be directly constructed, and a feasible path can be found using dynamic constraints; the feasible path can be used as the nominal trajectory, and then the nominal trajectory can be converted into control commands to be executed by the space robot.

[0013] Alternatively, the total planning time can be dynamically adjusted based on the displacement span required by the task and the physical limitations of the actuator. This will adjust the time-domain parameters to form a new set of structured configuration parameters, and step three can be re-executed until all verifications pass and the nonlinear programming solver generates a nominal trajectory that satisfies the dynamic constraints of the space robot.

[0014] Furthermore, in this invention, the structuring intent in step one is as follows:

[0015] ;

[0016] in, This is the decomposed sequence of serial subtasks. , The set of spatial target objects and their attributes involved in the task; Hard geometric constraints (such as specific orientations and boundary limitations) are required for task execution. A set of action primitives; This represents the final terminal state threshold for the task.

[0017] Furthermore, in this invention, in step one, the three-dimensional semantic feature vector extracted from the structured intent is as follows:

[0018] ;

[0019] In the formula, Represents a three-dimensional semantic feature vector. Indicates collaboration metrics. Indicates the number of semantic path points. Indicators representing movement diversity It represents a three-dimensional real vector space.

[0020] Furthermore, in this invention, the method for determining the current task execution mode based on the three-dimensional semantic feature vector using priority logic in step one is as follows:

[0021] when If so, the previous task execution mode is determined to be serial mode. This is a preset sequence depth threshold;

[0022] when and This is then determined to be parallel mode. Preset thresholds for diversity indicators;

[0023] If it is not determined to be in parallel mode or serial mode, it is determined to be in simplified mode.

[0024] Furthermore, in this invention, the method for discretizing the task space in the base coordinate system in the height direction in step two to construct a pseudo-three-dimensional voxel space is as follows:

[0025] Using continuous coordinates To voxel index Conversion formula:

[0026] ;

[0027] Perform the transformation, where, This represents the minimum coordinates of the starting boundary of the workspace; The preset voxel resolutions are for the x and y directions; Indicates in Only a single layer of discretization is performed in the axial direction. These represent the minimum and maximum values ​​of the task space in the vertical direction (z-axis), respectively.

[0028] Furthermore, in this invention, in step two, the composite value map is as follows:

[0029] ;

[0030] In the formula, Represents a composite value map. Basic value map type; The weight coefficients generated by the large language model satisfy... Value Map Obtained by weighted average of target attraction field and obstacle avoidance field, target attraction field and obstacle avoidance field All are constructed using a Gaussian kernel function:

[0031] ;

[0032] ;

[0033] In the formula, These represent the coordinate vectors of the target and the obstacle, respectively. These are the kernel width parameters representing the influence range of the control potential fields for the target attraction field and the obstacle avoidance field, respectively. This is the gain coefficient.

[0034] Furthermore, in this invention, the method for extracting the path point set in step two is as follows:

[0035] Using the gradient ascent algorithm, starting from the current starting point... Begin recursively extracting the path point set. During the extraction process, the selection of candidate points in each iteration follows the following rules:

[0036] ;

[0037] In the formula, The radius around the current point The set of neighboring sampling points within; This represents the candidate point for each iteration step;

[0038] When value increment The search stops when the target's coordinate vector falls below a preset threshold or reaches the maximum number of iterations. Add to path point set end.

[0039] Furthermore, in this invention, in step three,

[0040] Code-level security checks are as follows:

[0041] The structured intent is converted into environment modeling code using an abstract syntax tree and then statically parsed to enforce an API whitelist system, prohibiting risky functions that include operating system calls and dynamic execution.

[0042] The kinematic accessibility check is as follows:

[0043] Verification of path point set using analytical inverse kinematics algorithm Does the point in the equation lie within the physical working envelope of the robotic arm? Does the physical working envelope of the robotic arm constrain the workspace? for:

[0044] ;

[0045] In the formula, The target position vector; The coordinates of the shoulder joint of the robotic arm base; These are the lengths of the first and second links of the robotic arm, respectively.

[0046] The kinetic feasibility verification is as follows:

[0047] Verification using a nonlinear programming solver on the condition that the system dynamics equations are satisfied The existence of its trajectory under controlled amplitude limiting conditions;

[0048] The verification process needs to explicitly consider the momentum coupling effect between the base and the robotic arm, satisfying the following:

[0049] ;

[0050] In the formula, For the system's generalized coordinate vector; For configuration-dependent inertia matrices; For Coriolis force and centripetal force terms; To control the input vector, For the system's generalized velocity vector, Let be the system's generalized acceleration vector.

[0051] Furthermore, in step three of this invention, after converting the nominal trajectory into control commands to be executed by the space robot, a control law integrating feedback regulation and model predictive dynamics feedforward is used. :

[0052] ;

[0053] In the formula, The nominal input is the feedforward input calculated based on the dynamic model; and These are the proportional and differential feedback gain matrices, respectively; To address real-time status tracking errors, The derivative of the real-time status tracking error.

[0054] Furthermore, in this invention, in step four, the sparsification of the path point subset... :

[0055] ;

[0056] In the formula, To reduce the number of path points, the optimization dimensionality is lowered by reducing decision variables and inter-segment continuity constraints.

[0057] This invention balances computational efficiency and reliability. The pseudo-3D mapping technique, while retaining the 3D reasoning capabilities of large models, significantly reduces computational overhead compared to full 3D space and effectively eliminates the "illusion" of large language models in spatial coordinate generation. Strong robustness and security are ensured by a three-tiered security defense and recovery strategy at the code layer, kinematics, and dynamics levels, guaranteeing that the generated trajectory is not only mathematically optimal but also physically safe and feasible. This improves the on-orbit servicing autonomy and environmental adaptability of space robots, achieving end-to-end mapping from natural language commands to complex dynamic actions, greatly reducing reliance on manual parameter tuning, and enabling robots to autonomously handle diverse tasks. Experimental results show that, with this method, space robots achieve good levels of success rate, planning error, and planning solution time in complex task decision-making and planning. Attached Figure Description

[0058] Figure 1 This is a block diagram illustrating the execution process of a semantic-control instruction conversion method for space robots based on a large language model.

[0059] Figure 2 This is a diagram showing the geometric connections and physical representation of the space robot in Specific Implementation Method 1;

[0060] Figure 3This is a flowchart of the semantic-driven decision execution logic in Implementation Method 1.

[0061] Figure 4 This is a schematic diagram of the pseudo-3D mapping value map generation in Implementation Method 1.

[0062] Figure 5 This is a snapshot of the hardware experimental platform and task execution in Implementation Method 1.

[0063] Figure 6 Figure (a) shows the motion planning error and tracking error comparison of multiple tasks in Implementation Method 1, and Figure (b) shows the motion planning error curve of multiple tasks. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0065] Specific implementation method one: Refer to Figure 1 This embodiment specifically describes a semantic-control instruction conversion method for a space robot based on a large language model. The method includes:

[0066] Step 1: Receive natural language instructions, parse the natural language instructions into structured intents through the base big language model, extract three-dimensional semantic feature vectors from the structured intents, determine the current task execution mode based on the three-dimensional semantic feature vectors through priority logic, and generate corresponding optimization weight vectors and time-domain parameters based on the current task execution mode to form a structured configuration parameter set;

[0067] Step 2: According to the structured intent, the task space under the base coordinate system is discretized in a single layer in the height direction to construct a pseudo-three-dimensional voxel space. In the pseudo-three-dimensional voxel space, a composite value map is generated by weighted superposition of the target attraction field and the obstacle avoidance field. Based on the composite value map, the path point set is recursively extracted from the starting point using the gradient ascent algorithm.

[0068] Step 3: Using the structured configuration parameter set, perform code-level safety verification, kinematic reachability verification, and dynamic feasibility verification on the path point set in sequence. After the verification is passed, input the path point set as the initial path and the structured configuration parameter set together into the nonlinear programming solver to perform trajectory optimization, generate a nominal trajectory that satisfies the dynamic constraints of the space robot, and convert the nominal trajectory into control instructions to be executed by the space robot.

[0069] Step 4: If any check in Step 3 fails or the nonlinear programming solver fails to generate a nominal trajectory that satisfies the space robot's dynamic constraints, execute:

[0070] The path point set is sampled a second time to generate a sparse path point subset. The sparse path point subset is used as the new path point set, and step three is repeated until all verifications pass and the nonlinear programming solver generates a nominal trajectory that satisfies the dynamic constraints of the space robot.

[0071] Alternatively, a boundary value problem from the initial state to the target state can be directly constructed, and a feasible path can be found using dynamic constraints; the feasible path can be used as the nominal trajectory, and then the nominal trajectory can be converted into control commands to be executed by the space robot.

[0072] Alternatively, the total planning time can be dynamically adjusted based on the displacement span required by the task and the physical limitations of the actuator. This will adjust the time-domain parameters to form a new set of structured configuration parameters, and step three can be re-executed until all verifications pass and the nonlinear programming solver generates a nominal trajectory that satisfies the dynamic constraints of the space robot.

[0073] Furthermore, in this invention, the structuring intent in step one is as follows:

[0074] ;

[0075] in, This is the decomposed sequence of serial subtasks. , The set of spatial target objects and their attributes involved in the task; Hard geometric constraints (such as specific orientations and boundary limitations) are required for task execution. A set of action primitives; This represents the final terminal state threshold for the task.

[0076] Furthermore, in this invention, in step one, the three-dimensional semantic feature vector extracted from the structured intent is as follows:

[0077] ;

[0078] In the formula, Represents a three-dimensional semantic feature vector. Indicates collaboration metrics. Indicates the number of semantic path points. Indicators representing movement diversity It represents a three-dimensional real vector space.

[0079] Furthermore, in this invention, the method for determining the current task execution mode based on the three-dimensional semantic feature vector using priority logic in step one is as follows:

[0080] when If so, the previous task execution mode is determined to be serial mode. This is a preset sequence depth threshold;

[0081] when and This is then determined to be parallel mode. Preset thresholds for diversity indicators;

[0082] If it is not determined to be in parallel mode or serial mode, it is determined to be in simplified mode.

[0083] Furthermore, in this invention, the method for discretizing the task space in the base coordinate system in the height direction in step two to construct a pseudo-three-dimensional voxel space is as follows:

[0084] Using continuous coordinates To voxel index Conversion formula:

[0085] ;

[0086] Perform the transformation, where, This represents the minimum coordinates of the starting boundary of the workspace; The preset voxel resolutions are for the x and y directions; Indicates in Only a single layer of discretization is performed in the axial direction. These represent the minimum and maximum values ​​of the task space in the vertical direction (z-axis), respectively.

[0087] Furthermore, in this invention, in step two, the composite value map is as follows:

[0088] ;

[0089] In the formula, Represents a composite value map. Basic value map type; The weight coefficients generated by the large language model satisfy... Value Map Obtained by weighted average of target attraction field and obstacle avoidance field, target attraction field and obstacle avoidance field All are constructed using the Gaussian kernel function:

[0090] ;

[0091] ;

[0092] In the formula, These represent the coordinate vectors of the target and the obstacle, respectively. These are the kernel width parameters representing the influence range of the control potential fields for the target attraction field and the obstacle avoidance field, respectively. This is the gain coefficient.

[0093] Furthermore, in this invention, the method for extracting the path point set in step two is as follows:

[0094] Using the gradient ascent algorithm, starting from the current starting point... Begin recursively extracting the path point set. During the extraction process, the selection of candidate points in each iteration follows the following rules:

[0095] ;

[0096] In the formula, Radius around the current point The set of neighboring sampling points within; This represents the candidate point for each iteration step;

[0097] When value increment The search stops when the target's coordinate vector falls below a preset threshold or reaches the maximum number of iterations. Add to path point set end.

[0098] Furthermore, in step three of this invention,

[0099] Code-level security checks are as follows:

[0100] The structured intent is converted into environment modeling code using an abstract syntax tree and then statically parsed to enforce an API whitelist system, prohibiting risky functions that include operating system calls and dynamic execution.

[0101] The kinematic accessibility check is as follows:

[0102] Verification of path point set using analytical inverse kinematics algorithm Does the point in the equation lie within the physical working envelope of the robotic arm? Does the physical working envelope of the robotic arm constrain the workspace? for:

[0103] ;

[0104] In the formula, The target position vector; The coordinates of the shoulder joint of the robotic arm base; These are the lengths of the first and second links of the robotic arm, respectively.

[0105] The kinetic feasibility verification is as follows:

[0106] Verification using a nonlinear programming solver on the condition that the system dynamics equations are satisfied The existence of its trajectory under controlled amplitude limiting conditions;

[0107] The verification process needs to explicitly consider the momentum coupling effect between the base and the robotic arm, satisfying the following:

[0108] ;

[0109] In the formula, For the system's generalized coordinate vector; For configuration-dependent inertia matrices; For Coriolis force and centripetal force terms; To control the input vector, For the system's generalized velocity vector, Let be the system's generalized acceleration vector.

[0110] Furthermore, in step three of this invention, after converting the nominal trajectory into control commands to be executed by the space robot, a control law integrating feedback regulation and model predictive dynamics feedforward is used. :

[0111] ;

[0112] In the formula, The nominal input is the feedforward input calculated based on the dynamic model; and These are the proportional and differential feedback gain matrices, respectively; To address real-time status tracking errors, The derivative of the real-time status tracking error.

[0113] Furthermore, in this invention, in step four, the sparsification of the path point subset... :

[0114] ;

[0115] In the formula, To reduce the number of path points, the optimization dimensionality is lowered by reducing decision variables and inter-segment continuity constraints.

[0116] In specific implementation of the present invention,

[0117] Step 1: Design a semantic-driven decision-making module to implement adaptive configuration of task modes and optimization weights;

[0118] This step transforms unstructured text into a set of structured configuration parameters that the control system can recognize by constructing a mapping model between a large language model and underlying physical parameters. Specifically, it includes the following steps:

[0119] Step 11: Multi-level intent parsing and structured mapping.

[0120] The system receives natural language commands. First, the preset system prompt word template is invoked. Task prompt template The base-based large-scale model inference engine performs multi-level parsing to output structured intent. This representation can be abstractly defined as a quintuple:

[0121] (1)

[0122] In the formula, This is the decomposed sequence of serial subtasks. , The set of spatial target objects and their attributes involved in the task; Hard geometric constraints (such as specific orientations and boundary limitations) are required for task execution. A set of action primitives; This represents the final terminal state threshold for the task. During inference, the object extraction module is responsible for... and The code is converted into environment modeling code; the semantic parsing module uses context learning to accurately map ambiguous references in the text to numerical coordinates in the robot's base coordinate system; the task configuration module uses inference chain technology to determine... The evolutionary path.

[0123] Steps 1 and 2: Quantization extraction of semantic feature vectors.

[0124] The structured intent generated step by step The agent extracts three-dimensional semantic feature vectors that can characterize the physical task through semantic evaluation. :

[0125]

[0126] In the formula, the collaboration index By identification Calculate the weight values ​​of synchronization keywords and multi-actuator coupled actions; motion diversity index. The ratio of different action types in a task is calculated to quantify the task's complexity; the number of semantic path points. statistics The total number of path points that the trajectory tends to pass through determines the segment size of the subsequent planned trajectory.

[0127] Step 13: Decision-making on execution mode based on priority hierarchy.

[0128] The system obtains the results from steps one and two. The optimal planning system for the task is determined using a logical decision matrix. This system decision follows the following priority logic:

[0129] 1) If detected ( If the preset sequence depth threshold is met, the system is classified as being in serial mode. In this mode, the system uses the path point continuity constraint operator. splicing the trajectory segments:

[0130]

[0131] In the formula, Mark the start and end states of the trajectory ends respectively, with superscripts. Indicates the first Segment trajectory.

[0132] 2) If And diversity If the threshold is exceeded, it is determined to be in parallel mode. This mode focuses on the coordination between the base and the robotic arm in the workspace and requires triggering strong coupling dynamic compensation.

[0133] 3) If none of the above conditions are met, the system is classified as simplified mode, and trajectory optimization is performed using fixed parameters confirmed by expert experience.

[0134] Step 14: Optimize the adaptive generation of parameters and weights.

[0135] Based on the pattern determined in steps one and three, the system generates a vector containing penalty weights. and time domain parameters configuration set The specific calculation method for each component of the penalty weight vector is as follows:

[0136]

[0137] In the formula, For execution mode, Prioritize the task objectives. To invoke an agent inference mapping once, and Transformed into specific trajectory optimizer constraint weight values For example, in parallel mode, the weight of the terminal state error term is set to an ultra-high-order penalty value, using soft constraints to approximate hard boundaries, thereby achieving high-precision cooperative positioning control.

[0138] Step 2: Design a pseudo-3D voxel space mapping module to generate a smooth value map to guide the robot's movement;

[0139] This module serves as the interface between the semantic layer and the physical execution layer, aiming to eliminate the randomness in spatial coordinate resolution of large language models and transform discrete intents into a continuous guidance field. Specifically, it includes the following steps:

[0140] Step 21: Construction and Mapping of the Pseudo-3D Voxel Space. To enable a language model with 3D reasoning capabilities to adapt to 2D planar task constraints, the system will construct and map the task space. Discretized into voxel mesh And define a single-layer discretization mapping, continuous coordinates To voxel index The conversion is as follows:

[0141]

[0142] In the formula, The starting boundary coordinates (minimum value) of the workspace; The preset voxel resolution; Indicates in The axis direction is discretized in a single layer, thereby forcing 3D inference instructions to be projected onto the 2D physical plane, ensuring architectural consistency and reducing search complexity.

[0143] Step 22: Value Graph Generation Based on Semantic Weights. Based on the structured intent generated in Step 1, the system dynamically synthesizes a composite value graph using code instructions. This serves as the potential field foundation for path planning. Its mathematical expression is:

[0144]

[0145] In the formula, Basic value map type; The weight coefficients generated by the large language model satisfy... In the value map, the target attraction field and obstacle avoidance field are constructed using a Gaussian kernel function:

[0146]

[0147]

[0148] In the formula, These represent the coordinate vectors of the target and the obstacle, respectively. The kernel width parameter is used to control the range of influence of the potential field; This represents the gain coefficient. The Gaussian form ensures that the value map is valid across the entire domain. (Infinite order continuity) Continuity effectively avoids the gradient discontinuity problem in the traditional potential field method.

[0149] Steps 2 and 3: Path point extraction based on greedy gradient ascent.

[0150] In obtaining the composite value map Then, the system uses the gradient ascent algorithm to ascend from the current starting point. Start recursively extracting the critical path point set The candidate point selection in each iteration follows the following rules:

[0151]

[0152] In the formula, Radius around the current point The set of neighborhood sampling points within the range. When the value increment... The search stops when the number of iterations falls below a preset threshold or reaches the maximum number of iterations, and is forced to stop. Add to point set The end is used to ensure terminal positioning accuracy. This point set... This will serve as the initial path for subsequent motion testing and nonlinear programming optimization. Based on the initial path, task mode, and parameter settings, the optimizer can quickly calculate the motion trajectory that satisfies the strongly coupled dynamics of the space robot.

[0153] Step 3: Design a multi-level defense and autonomous recovery module to achieve closed-loop security verification and feasibility recovery;

[0154] This module aims to ensure the physical security and mathematical solvability of instructions generated by large language models in complex spatial environments through multi-dimensional logical filtering and redundancy planning strategies. Specifically, it includes the following steps:

[0155] Step 3: Multi-level security verification and defense mechanism. Before execution by the underlying controller, the system performs three levels of security filtering to identify and block potential conflicting outputs.

[0156] 1) Code-level security verification. The generated program code is statically analyzed using an abstract syntax tree, enforcing an API whitelist system to prohibit risky functions related to the operating system, system calls, and dynamic execution, ensuring secure isolation of the code's runtime environment.

[0157] 2) Kinematic reachability verification. Path points are verified using an analytical inverse kinematics algorithm. Whether it is located within the physical working envelope of the robotic arm. Its workspace constraints. The expression is as follows:

[0158]

[0159] In the formula, The target position vector; The coordinates of the shoulder joint of the robotic arm base; These are the lengths of the first and second links of the robotic arm, respectively.

[0160] 3) Dynamic feasibility verification. A nonlinear programming solver is used to verify that the system's dynamic equations are satisfied. The existence of the trajectory under controlled amplitude limiting conditions. This verification process needs to explicitly consider the momentum coupling effect between the base and the robotic arm:

[0161]

[0162] In the formula, For the system's generalized coordinate vector; For configuration-dependent inertia matrices; For Coriolis force and centripetal force terms; To control the input vector.

[0163] Step 3.2: Hierarchical Autonomous Recovery and Rollback Strategy. If any verification step in Step 3.1 fails, or if the underlying trajectory optimization fails to converge, the system will automatically trigger a recovery mechanism to improve the task success rate by gradually relaxing the constraint complexity.

[0164] When the sequential execution of a task becomes too complex, causing the solver to be unable to find a feasible solution within the time limit, the system considers the candidate path point set. Perform secondary sampling to generate a sparsed subset of path points. :

[0165]

[0166] In the formula, To reduce the number of path points, the optimization dimensionality is lowered by reducing decision variables and inter-segment continuity constraints.

[0167] When the environment is relatively simple but semantic parsing is ambiguous, the system bypasses the path point guidance logic and directly constructs the system from the initial state. To the target state For boundary value problems, feasible paths are found using purely dynamic constraints.

[0168] The total planning time is dynamically adjusted based on the required displacement span of the task and the physical limitations of the actuator. This provides sufficient execution capability for the control system.

[0169]

[0170] In the formula, These are the estimated motion durations based on the maximum linear velocity of the base and the maximum angular velocity of the joint, respectively. This is a conservative time buffer coefficient; This is a limiting function to ensure that the planning duration is within a safe and reasonable range.

[0171] Step 33: Closed-loop execution and real-time feedback compensation. After the above verification and optimization, the system will generate the nominal trajectory. This is converted into control commands, and then controlled by an integrated feedback regulation and model predictive dynamics feedforward control law. Driver hardware execution:

[0172]

[0173] In the formula, The nominal input is the feedforward input calculated based on the dynamic model; and These are the proportional and differential feedback gain matrices, respectively; This step is for real-time state tracking error. It ensures that the robot can still complete its intended task with millimeter-level accuracy even in the presence of sensor noise and external disturbances.

[0174] In summary, the physical meaning, logical flow, and experimental results of this embodiment are further explained below with reference to the accompanying drawings:

[0175] Figure 1 This is a diagram of the intelligent agent system architecture for a space robot.

[0176] This diagram illustrates the four-layer hierarchical logical structure of the intelligent agent system described in this invention: a language model encoding layer, a value map navigation layer, a trajectory planning layer, and a physical entity execution layer. Each layer interacts through structured parameters, achieving decoupling and mapping from abstract human intentions to the robot's underlying driving commands.

[0177] Figure 2 This is a diagram showing the geometric connections and physical representation of a space robot.

[0178] The diagram clarifies the physical composition of a free-flying space robot, including a spacecraft with a floating base, a solar array, and other components. A planar robotic arm with degrees of freedom. The inertial reference frame is defined in the diagram. With base reference frame This forms the basis for the pseudo-3D coordinate transformation in step two, including the length parameters of each link. This directly constrains the kinematic reachability envelope in step 3.1. Indicates the 1st, 2nd, ..., 3rd robots The center of mass of the joint, Indicates the 1st, 2nd, ..., 1st Joints in an inertial frame of reference coordinates in Indicates the 1st, 2nd, ..., 1st Joints in an inertial frame of reference The corner in the middle. These represent the coordinates of the floating base and the end effector in the inertial reference frame, respectively.

[0179] Figure 3 This is a flowchart of the semantic-driven decision execution logic.

[0180] The figure illustrates the decision-making process described in step one, demonstrating how the system extracts semantic feature vectors from the input of a natural language task. And according to the decision function The system automatically divides the process into sequential, parallel, or simplified execution modes, ultimately transforming it into an optimal control problem involving parameters (optimization weights). Total planning time The process of quantitatively configuring and solving (constrained topology, etc.).

[0181] Figure 4 A schematic diagram for generating a value map for pseudo-3D mapping.

[0182] This diagram corresponds to the result of step two, demonstrating the three-dimensional workspace in... The voxel mesh morphology after single-layer discretization along the axis. The target attraction field is represented by color intensity or height peaks in the figure. Repulsion field with barrier The superposition effect reflects the generation of the Gaussian kernel function. The guiding role of continuous gradients in path point extraction.

[0183] Figure 5 This is a snapshot of the hardware experimental platform and task execution.

[0184] The figure illustrates the air-bearing experimental platform environment and the physical hardware of the space robot used to verify the effectiveness of the invention. The time-series snapshots in the figure show the actual motion trajectories of the robot when performing multipath navigation (complex task 1), high-precision coordinated robot motion (complex task 2), and obstacle avoidance (complex task 3), demonstrating the feasibility and real-time performance of the intelligent agent system in a real physical environment.

[0185] Figure 6 Comparison charts of 20 custom task planning errors, tracking errors, and normalized scores across 3 categories.

[0186] This figure provides quantitative performance verification of the technical solution of this invention. Experimental results show that the planning accuracy is high, thanks to the smoothing value map guidance in step two and the NLP optimization in step three-one. The nominal trajectory planning error generated by the system remains in the sub-millimeter level (0.0002-0.001mm). The tracking robustness is strong; considering sensor noise and non-ideal characteristics of the actuator, the hardware tracking error remains in the centimeter level (approximately 20-30mm), verifying the compensation effect of the feedback-feedforward composite control law in step three-three. Regarding the accuracy-efficiency trade-off, the radar chart shows the system's accuracy performance in different modes (simplified, parallel, sequential), proving that the adaptive configuration strategy in step one optimizes resource consumption while ensuring a task success rate (above 90%).

[0187] While specific embodiments of this application have been described herein with reference to them, it should be understood that these embodiments are merely examples of the principles and applications of this application. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of this application as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A method for semantic-control instruction conversion of a space robot based on a large language model, characterized in that, include: Step 1: Receive natural language instructions, parse the natural language instructions into structured intents through the base big language model, extract three-dimensional semantic feature vectors from the structured intents, determine the current task execution mode based on the three-dimensional semantic feature vectors through priority logic, and generate corresponding optimization weight vectors and time-domain parameters based on the current task execution mode to form a structured configuration parameter set; Step 2: According to the structured intent, the task space under the base coordinate system is discretized in a single layer in the height direction to construct a pseudo-three-dimensional voxel space. In the pseudo-three-dimensional voxel space, a composite value map is generated by weighted superposition of the target attraction field and the obstacle avoidance field. Based on the composite value map, the path point set is recursively extracted from the starting point using the gradient ascent algorithm. Step 3: Using the structured configuration parameter set, perform code-level safety verification, kinematic reachability verification, and dynamic feasibility verification on the path point set in sequence. After the verification is passed, input the path point set as the initial path and the structured configuration parameter set together into the nonlinear programming solver to perform trajectory optimization, generate a nominal trajectory that satisfies the dynamic constraints of the space robot, and convert the nominal trajectory into control instructions to be executed by the space robot. Step 4: If any check in Step 3 fails or the nonlinear programming solver fails to generate a nominal trajectory that satisfies the space robot's dynamic constraints, execute: The path point set is sampled a second time to generate a sparse path point subset. The sparse path point subset is used as the new path point set, and step three is repeated until all verifications pass and the nonlinear programming solver generates a nominal trajectory that satisfies the dynamic constraints of the space robot. Alternatively, a boundary value problem from the initial state to the target state can be directly constructed, and a feasible path can be found using dynamic constraints; the feasible path can be used as the nominal trajectory, and then the nominal trajectory can be converted into control commands to be executed by the space robot. Alternatively, the total planning time can be dynamically adjusted based on the displacement span required by the task and the physical limitations of the actuator. This will adjust the time-domain parameters to form a new set of structured configuration parameters, and step three can be re-executed until all verifications pass and the nonlinear programming solver generates a nominal trajectory that satisfies the dynamic constraints of the space robot.

2. The method for semantic-control instruction conversion of a space robot based on a large language model according to claim 1, characterized in that, In step one, the intended purpose of structuring is: ； in, This is the decomposed sequence of serial subtasks. , The set of spatial target objects and their attributes involved in the task; For hard geometric constraints in task execution; A set of action primitives; This represents the final terminal state threshold for the task.

3. A method for semantic-control instruction conversion of a space robot based on a large language model according to claim 1 or 2, characterized in that, In step one, the three-dimensional semantic feature vector extracted from the structured intent is as follows: ； In the formula, Represents a three-dimensional semantic feature vector. Indicates collaboration metrics. Indicates the number of semantic path points. Indicators representing movement diversity It represents a three-dimensional real vector space.

4. The semantic-control instruction conversion method for space robots based on a large language model according to claim 1, characterized in that, In step one, the method for determining the current task execution mode based on the three-dimensional semantic feature vector using priority logic is as follows: when If so, the previous task execution mode is determined to be serial mode. This is a preset sequence depth threshold; when and This is then determined to be parallel mode. Preset thresholds for diversity indicators; If it is not determined to be in parallel mode or serial mode, it is determined to be in simplified mode.

5. The method for semantic-control instruction conversion of a space robot based on a large language model according to claim 1, characterized in that, In step two, the method for discretizing the task space in the base coordinate system in a single layer along the height direction to construct a pseudo-3D voxel space is as follows: Using continuous coordinates To voxel index Conversion formula: ； Perform the transformation, where, This represents the minimum coordinates of the starting boundary of the workspace; The preset voxel resolutions are for the x and y directions; Indicates in Only a single layer of discretization is performed in the axial direction. These represent the minimum and maximum values ​​of the task space in the vertical direction, respectively.

6. The semantic-control instruction conversion method for space robots based on a large language model according to claim 1, characterized in that, In step two, the composite value map is as follows: ； In the formula, Represents a composite value map. Basic value map type; The weight coefficients generated by the large language model satisfy... Value Map Obtained by weighted average of target attraction field and obstacle avoidance field, target attraction field and obstacle avoidance field All are constructed using a Gaussian kernel function: ； ； In the formula, These represent the coordinate vectors of the target and the obstacle, respectively. These are the kernel width parameters representing the influence range of the control potential fields for the target attraction field and the obstacle avoidance field, respectively. This is the gain coefficient.

7. The method for semantic-control instruction conversion of a space robot based on a large language model according to claim 1, characterized in that, In step two, the method for extracting the path point set is as follows: Using the gradient ascent algorithm, starting from the current starting point... Begin recursively extracting the path point set. During the extraction process, the selection of candidate points in each iteration follows the following rules: ； In the formula, Radius around the current point The set of neighboring sampling points within; This represents the candidate point for each iteration step; When value increment The search stops when the target's coordinate vector falls below a preset threshold or reaches the maximum number of iterations. Add to path point set end.

8. The semantic-control instruction conversion method for space robots based on a large language model according to claim 1, characterized in that, In step three, Code-level security checks are as follows: The structured intent is converted into environment modeling code using an abstract syntax tree and then statically parsed to enforce an API whitelist system, prohibiting risky functions that include operating system calls and dynamic execution. The kinematic accessibility check is as follows: Verification of path point set using analytical inverse kinematics algorithm Does the point in the equation lie within the physical working envelope of the robotic arm? Does the physical working envelope of the robotic arm constrain the workspace? for: ； In the formula, The target position vector; The coordinates of the shoulder joint of the robotic arm base; These are the lengths of the first and second links of the robotic arm, respectively. The kinetic feasibility verification is as follows: Verification using a nonlinear programming solver on the condition that the system dynamics equations are satisfied The existence of its trajectory under controlled amplitude limiting conditions; The verification process needs to explicitly consider the momentum coupling effect between the base and the robotic arm, satisfying the following: ； In the formula, For the system's generalized coordinate vector; For configuration-dependent inertia matrices; For Coriolis force and centripetal force terms; To control the input vector, For the system's generalized velocity vector, Let be the system's generalized acceleration vector.

9. The method for semantic-control instruction conversion of a space robot based on a large language model according to claim 1, characterized in that, In step three, after converting the nominal trajectory into control commands to be executed by the space robot, a control law integrating feedback regulation and model predictive dynamics feedforward is used. : ； In the formula, The nominal input is the feedforward input calculated based on the dynamic model; and These are the proportional and differential feedback gain matrices, respectively; To address real-time status tracking errors, The derivative of the real-time status tracking error.

10. The method for semantic-control instruction conversion of a space robot based on a large language model according to claim 1, characterized in that, In step four, the path point subset is sparsified. : ； In the formula, To reduce the number of path points, the optimization dimensionality is lowered by reducing decision variables and inter-segment continuity constraints.

Images