Robot motion control system based on multi-dimensional context perception and large language model

Abstract

The application discloses a robot motion control system based on multi-dimensional context perception and a large language model, and relates to a robot intelligent control system.The system comprises an information perception and interaction module, an agent decision and generation module, and a bottom motion execution module; each module communicates through a standardized data protocol, and jointly realizes a complete link from information input to physical feedback; the logical reasoning capability of a cloud large language model is utilized, multi-dimensional perception data and long-term memory information of the robot are combined, and an interactive intention is mapped to a bottom executable meta-action sequence.The advantages are that three core technical problems are systematically solved, i.e., motion control logic rigidity and separation from perception, a conversion barrier between high-level semantic intention and bottom physical execution, and a lack of parameter adaptive evolution capability based on natural language feedback.

Description

Technical Field

[0001] This invention relates to intelligent control systems for robots, and more particularly to robot motion control systems based on multidimensional contextual awareness and large language models. Background Technology

[0002] With the evolution of service robot technology, the motion control system of home companion robots is gradually developing from simple point-to-point path planning to embodied intelligence with environmental perception and interactive response capabilities. Existing home robots typically integrate sensors such as LiDAR or millimeter-wave radar to construct two-dimensional or three-dimensional maps of the indoor environment and rely on underlying kinematic models to achieve basic obstacle avoidance and navigation tasks.

[0003] Currently, most mainstream home mobile assistants and companion robots on the market rely on fixed state machines and preset rule engines for their motion control logic. Taking Amazon's Astro home robot as an example, while these products can achieve area movement and visual tracking of specific targets based on Simultaneous Localization and Mapping (SLAM) technology, and even integrate basic voice assistants for everyday conversations, they still face the following significant technical bottlenecks in their actual motion control architecture:

[0004] 1. Decoupling of Motion Control Parameters from Real-Time Context Information: Existing systems primarily rely on physical obstacle avoidance algorithms and kinematic constraints to determine their underlying motion control parameters. The systems lack effective mechanisms to incorporate the user's immediate emotional state and long-term behavioral habits into the motion decision-making model. When performing a following task, parameters such as following distance, speed, and acceleration are fixed factory values. Even when the user is depressed or anxious, the robot will still approach the user at a standard speed and a fixed minimum distance. This singular and rigid motion behavior cannot dynamically adjust its strategy based on different external sensory information, making it difficult to meet the multi-dimensional companionship needs in home settings.

[0005] 2. Technical barriers exist in the conversion between semantic commands and refined motion sequences: Existing control architectures often only respond to macroscopic task assignments, making it difficult to directly translate complex natural language intentions into a series of continuous and precise physical actions at the underlying level. For example, when a voice-controlled mobile vehicle receives the command "go to the living room," the system only translates the semantics into a single target point coordinate and hands it over to a global path planning algorithm for execution. When faced with natural language commands like "step back a little, then slowly turn around," which involve multiple steps and refined parameter control, existing systems cannot handle them. This is because there is a lack of a standardized interface layer between the high-dimensional semantic information output by the large language model and the underlying motor controller. Due to the lack of a defined set of highly decoupled parameterized meta-actions, the large language model cannot generate continuous motion sequences that can be parsed and executed sequentially by the underlying layer through formatted data structures.

[0006] 3. Lack of parameter optimization and evolution mechanisms based on unstructured feedback: In home scenarios, user feedback on robot movement performance is often vague and unstructured natural language. Existing systems typically only respond to hard commands with clear quantifiable indicators. Taking a navigation-enabled robotic vacuum cleaner or companion robot as an example, if a user says "You were moving too fast" or "Move away from me," the system, lacking an adaptive model, cannot effectively recognize the parameter optimization intent in this natural language feedback. Furthermore, existing control systems lack a dynamic profile update mechanism linked to user habits, preventing the system from automatically correcting local default movement parameters based on historical feedback. This one-way command transmission and rigid parameter calling pattern results in the system lacking continuous learning capabilities and failing to achieve personalized adaptation during long-term companionship.

[0007] The primary problem is that the motion control systems of existing home companion robots suffer from highly rigid commands and a disconnect between instruction execution and implementation. They lack a mechanism that can deeply integrate the high-level semantic understanding capabilities of large language models with the underlying, refined, and continuous physical motion. Furthermore, the systems cannot effectively extract the user's immediate emotional state during operation, nor can they adaptively adjust motion parameters or dynamically evolve the local user profile based on the user's daily unstructured natural language feedback. This results in stiff and monotonous motion feedback in the highly dynamic real-world home environment, failing to maintain a personalized and long-term intelligent companionship experience, as specifically demonstrated below.

[0008] (1) Rigid motion feedback and lack of multi-dimensional context perception: The existing underlying motion parameters of mobile robots mainly include forward speed, translation direction, and following distance. These parameters are usually encapsulated as factory-fixed default configurations or simply limited by the underlying obstacle avoidance and path planning logic. When facing complex and ever-changing family interaction scenarios, the system cannot effectively extract and integrate external immediate context and internal memory context. Among them, the external immediate context involves the user's immediate emotional characteristics such as happiness and anger, while the internal memory context includes the user's long-term behavioral preferences and the 3D topological structure of the specific environment. This lack of multi-dimensional perception information causes the robot to be unable to make differentiated and adaptive motion feedback that conforms to the current situation. For example, when it senses that the user is in a low mood, the robot cannot automatically increase the following distance or decrease the movement speed, which seriously weakens the naturalness of human-computer interaction.

[0009] (2) A conversion barrier exists between high-level semantic intent and underlying physical control: Although some products in the industry have introduced cloud-based large language models, they are limited by the underlying architecture and can usually only handle coarse-grained task allocation at the macro level, and can only execute discrete commands such as going to a specific room or stopping in place. Due to the lack of a highly decoupled, standardized and flexibly configurable meta-action interface encapsulation at the bottom layer of the control system, the bottom layer cannot support circling actions with speed and turning radius parameters, following actions with segmented speed adjustment features, and translational actions with time and distance constraints. Therefore, the large language model cannot directly map and convert the refined, multi-step natural language intent it parses into a JSON format sequence of actions and parameters that can be executed sequentially by the underlying motor control board. This gap in semantic mapping greatly limits the robot's ability to perform coherent, complex, and customized embodied behaviors.

[0010] (3) Lack of adaptive evolution capability based on natural language feedback: In actual home use, users often tend to use colloquial and vague natural language to express their needs for fine-tuning the robot's movement state, such as asking the robot to move a little faster or to come around from the left. However, the existing system's preset state machine and rule engine can only recognize and process hard instructions containing precise absolute values, and cannot use the model to accurately identify and extract the parameter optimization intentions in the above-mentioned unstructured natural language. More importantly, the existing system lacks a data closed-loop mechanism, and cannot transform these optimization intentions into updates to the local user profile, nor can it modify the default call parameters of specific local meta-actions accordingly. This unidirectional control flow causes the system to remain in a static operating state during long-term deployment, unable to achieve self-learning and personalized evolution. Summary of the Invention

[0011] The purpose of this invention is to provide a robot motion control system based on multi-dimensional context awareness and a large language model, so as to solve the problems existing in the prior art.

[0012] The robot motion control system based on multidimensional context perception and large language model described in this invention includes: an information perception and interaction module, an agent decision-making and generation module, and a low-level motion execution module; each module communicates through a standardized data protocol to jointly realize a complete link from information input to physical feedback; by utilizing the logical reasoning capability of the cloud-based large language model, combined with the robot's local multidimensional perception data and long-term memory information, the interaction intention is mapped into a low-level executable meta-action sequence; The information perception and interaction module is responsible for the collection, preprocessing and local memory management of multi-source heterogeneous data; The agent decision-making and generation module comprehensively evaluates the global context information summarized by the perception module and generates structured action sequence messages through logical reasoning. The underlying motion execution module is deployed on the robot's local controller. It is responsible for receiving the motion sequence generated by the intelligent agent module and driving the motor actuator to complete the physical action through internal parameter verification and spatiotemporal planning algorithms.

[0013] The robot motion control system based on multi-dimensional context awareness and a large language model described in this invention has the advantage of systematically solving three core technical problems: rigid motion control logic detached from perception, a conversion barrier between high-level semantic intent and low-level physical execution, and the lack of parameter adaptive evolution capability based on natural language feedback. It also produces the following significant technical effects: (1) The system completely abandons the traditional control mode that relies solely on rigid obstacle avoidance algorithms and preset fixed parameters. It deeply integrates the user's transient and real-time emotions with long-term motion preference profiles through the local intelligent agent workflow. The large language model performs reasoning under the overall coordination of the global context, enabling the robot's motion decision to have context awareness capabilities. It can display motion trajectories that conform to the user's real-time psychological state and long-term interaction habits, greatly improving the naturalness of human-computer interaction and the system's empathy.

[0014] (2) By building a highly decoupled and standardized meta-action interface at the bottom layer and using the cloud-based large language model as the central hub for intent translation and parameter decision-making, the system successfully converts the user's fuzzy natural language commands containing multi-step logic directly into a formatted JSON action sequence array that can be parsed by the underlying motor. Combined with the format verification of the local intelligent agent and the finite state machine queue scheduling, the system achieves high-fidelity and high-coherence execution of complex continuous action sequences without the need for manual hard coding.

[0015] (3) An innovative preprocessing gateway mechanism was designed to integrate intent parsing and adaptive judgment. The system can quickly capture users' everyday conversational feedback, fully rely on the generalization reasoning ability of the cloud-based large language model to calculate precise parameter adjustment increments, and complete physical boundary security verification and automatic overwriting of user profiles locally. This enables implicit iteration of the underlying motion call parameters, giving the robot the long-term memory and personalized self-evolution ability to "understand users better with use" without code upgrades.

[0016] (4) This system clearly defines the boundary of responsibilities between the local intelligent agent's central assembly and scheduling and the cloud-based large model's stateless full-scale reasoning. Complex semantic dimensionality reduction and intent parameter calculation tasks are completely handled by the cloud-based large model, greatly reducing the bloat of the local firmware code. At the same time, core sensitive user profile data, legality verification of issued commands, closed-loop monitoring of high-frequency physical execution thresholds, and hardware preemption and emergency stop in case of emergencies are all absolutely controlled by the local intelligent agent. This architecture fully introduces the infinite generalization capability of the large model while achieving physical isolation of user privacy data and setting an insurmountable safety bottom line for robot movement. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall structure of the robot motion control system described in this invention.

[0018] Figure 2 This is a flowchart of the core workflow of the edge-cloud collaborative and adaptive control of the robot motion control system described in this invention. Detailed Implementation

[0019] This invention discloses a robot motion control system based on multi-dimensional context awareness and a large language model, aiming to construct a closed-loop control framework from high-order semantic perception to embodied motion execution. The core design idea of ​​the architecture is to utilize the logical reasoning capabilities of a cloud-based large language model, combined with the robot's local multi-dimensional perception data and long-term memory information, to accurately map ambiguous interaction intentions into low-level executable meta-action sequences. Through a standardized meta-action interface protocol, the system effectively solves the gap between semantic understanding and physical execution in motion control, and endows the robot with the ability to adaptively evolve parameters based on user feedback.

[0020] The system's logical architecture consists of three core functional modules: an information perception and interaction module, an agent decision-making and generation module, and a low-level motion execution module. These modules communicate via standardized data protocols to jointly realize a complete link from information input to physical feedback.

[0021] The information perception and interaction module serves as the system's perception front end, responsible for the acquisition, preprocessing, and local memory management of multi-source heterogeneous data. On one hand, the module receives external information in real time through an open, standardized interface, including real-time emotional characteristics of users output by visual recognition algorithms or emotion computing engines, as well as unstructured natural language commands issued by users. On the other hand, the module manages the local memory system, constructing a 3D environmental map using fused perception data from LiDAR and millimeter-wave radar, and maintaining local user profile data that records user preferences and movement habits.

[0022] The intelligent agent decision-making and generation module, serving as the system's logical hub, comprises a cloud-based large language model and its accompanying workflow logic. Its core function is to comprehensively evaluate global contextual information gathered by the perception module, including real-time emotions, movement commands, user profiles, and environmental topology information, and generate structured action sequence messages through logical reasoning. Furthermore, the module integrates an adaptive model to parse the user's natural language feedback and dynamically adjust the parameter weights in the local user profile accordingly, enabling the autonomous evolution of movement strategies.

[0023] The underlying motion execution module, deployed on the robot's local controller, is the physical carrier for implementing system actions. This module encapsulates a series of highly decoupled and standardized low-level meta-action call interfaces, possessing the capability to parse and execute serialized instructions. The module is responsible for receiving action sequences generated by the agent module and, through internal parameter verification and spatiotemporal planning algorithms, driving the motor actuators to complete physical actions such as forward movement, translation, rotation, following, and navigation. The module's design ensures precise matching between semantic instructions and the robot's underlying kinematic constraints. The overall module communication and architecture logic of the system are attached. Figure 1 As shown.

[0024] The system rigorously divides the contextual information upon which motion decisions depend into transient external information and long-term memory information, and stores and retrieves them uniformly through standardized protocols and data structure formats. External information is defined as transient interactive data received in real-time by the system through its open application programming interface (API), primarily comprising two core elements: user-instantaneous emotion data and user command data. User-instantaneous emotion data is provided by an external emotion computing engine or multimodal recognition module, including emotion category labels and their corresponding confidence levels. User command data refers to unstructured control commands issued by the user through a voice or text terminal. The system opens its API to acquire the aforementioned transient information, ensuring efficient access to transient emotions and command intentions. Its standard input data structure is defined through the following nested hierarchy: Transient interaction data includes a subset of emotional state information and a subset of user command information. The emotional state information subset includes: a primary emotion label (text representing the current primary emotion category); a confidence floating-point value (representing the accuracy of emotion recognition); and a timestamp (recording the absolute time of emotion data collection). The user command information subset includes: the original text string (recording the complete natural language input by the user); and an intent classification label (recording the command classification result after natural language parsing).

[0025] Memory information is defined as global state data that is accumulated and persistently stored during long-term system operation. The system divides memory information into two main parts: environmental topology map and dynamic user profile, and uses a structured data format to ensure compatibility between persistent storage and cross-module calls.

[0026] The robot acquires underlying physical environment information using its onboard LiDAR and millimeter-wave radar, and processes the raw point cloud data using a three-dimensional laser inertial odometry mapping algorithm based on multi-sensor fusion. After mapping, the system projects the 3D point cloud downwards to generate a 2D occupancy grid map. To enable the large language model to understand the physical environment, the system extracts the boundary coordinates of closed connected domains and their internal visual objects from the grid map, serializing them into environmental feature data. This environmental feature data uses physical regions as basic units, each containing: a set of boundary polygon vertices and a set of identified objects. The set of boundary polygon vertices records the sequence of geometric boundary coordinates of the physical region in a 2D coordinate system. The set of identified objects records a list of semantic label characters for stationary objects within the physical region.

[0027] After receiving feature data, the cloud-based large language model assigns semantic labels to each physical region based on common-sense reasoning and calculates the physical connectivity between regions. The cloud-based large language model returns results according to the system-defined topology map data structure. The system directly persists the returned results as a local spatial semantic topology map, serving as the underlying pathfinding basis for regional navigation meta-actions. The topology map data structure includes a node data set and an edge data set. The node data set includes: a node identifier code, used to uniquely identify the topology node; a semantic label, used to record the specific physical room type represented by the node; and a center coordinate point, used to record the absolute physical coordinates of the node in the underlying raster map. Each node in the node data set represents a logical region. The edge data set includes: a starting node identifier code and a target node identifier code, used to define the two connection points of the edge; a connection type, used to characterize the physical channel structure attributes between two nodes; and a passable width value, used to record the actual physical passage distance of the channel. Each edge in the edge data set represents the physical connectivity between regions.

[0028] During the execution of a specific action sequence, the system accepts location marking commands issued by the user via voice. The system captures the current global absolute coordinates and heading angle from the odometry and appends them to the environment memory for direct invocation by navigation meta-actions. The custom record point data set contains multiple subsets of record points. Each subset includes: a record point identifier for uniquely identifying the target location; a set of absolute coordinates for marking the 3D position coordinates of the record point in the underlying map space; and a yaw angle value for recording the robot's desired physical orientation at the record point.

[0029] The system utilizes a nested dictionary structure to construct a user profile database. This database is the sole object for the adaptive model to override motion parameters. The root node is the user identifier, and internally it is strictly divided into basic attributes, motion preference parameters, and interaction history logs. Variable names in the motion preference parameters are strictly bound to the calling parameters of the underlying meta-actions. The standard persistent data structure includes: a basic attribute module, a motion preference parameter module, and an interaction history log list. The basic attribute module stores the user's inherent physical characteristics such as height and age group identifiers. The motion preference parameter module records subsets of follow-up action preferences, rotation action preferences, and translational action preferences according to meta-action categories. The follow-up action preference subset includes a minimum safe distance floating-point value and a list of desired speed ranges; the rotation action preference subset includes a default turning radius floating-point value; and the translational action preference subset includes a default speed limit value. Each historical log entry in the interaction history log list includes: a timestamp, used to record the absolute time when the user initiated feedback; feedback intent type, used to record the specific adjustment direction of the user's motion parameters; applied action module, used to record the name of the specific underlying meta-action targeted by the feedback; and weight adjustment value, used to record the quantitative correction magnitude made by the system to the default action parameters.

[0030] By strictly standardizing the nested data structure of input and output, this invention achieves a significant breakthrough in software and hardware decoupling. Standardized serialization features enable the cloud-based large language model to losslessly read the underlying physical geometry data and directly output a topological map structure that can be used by the robot's underlying planning algorithm. Simultaneously, the multi-layered nested user profile design allows the underlying meta-motion control logic to acquire and apply the latest personalized motion parameters in real time, much like reading system configuration files, greatly improving the engineering feasibility of system deployment and the efficiency of development iteration.

[0031] The underlying motion execution module operates by parsing standardized command messages. These command messages are in the form of a serialized set of actions, with each meta-action containing an action name identifier and a corresponding parameter mapping dictionary. The system sets strict type checks and physical threshold constraints for each parameter to ensure the smoothness and safety of the motion trajectory. All execution thresholds contain two mutually exclusive parameters, one of which can only be specified during system invocation; these two mutually exclusive parameters are duration and distance traveled.

[0032] The basic translation movement is responsible for handling the robot's linear displacement in the horizontal plane, including forward, backward, and lateral translation.

[0033] The system's underlying motion control is based on the kinematic model of a four-wheel independently driven Mecanum wheel chassis. When performing longitudinal or lateral translational movements, the upper-level application converts the set motion velocity into a target linear velocity vector in the chassis's local coordinate system. The controller then uses the inverse kinematic matrix of the Mecanum wheels to convert the longitudinal linear velocity... lateral linear velocity and yaw rate Uncoupled solution for the left front wheel Right front wheel Left rear wheel Right rear wheel The independent target rotational speed. Its inverse kinematics solution matrix equation is as follows: ; in, Let be the physical radius of the Mecanum wheel. This is the longitudinal physical distance from the wheel axle center to the geometric center of the chassis. This refers to the lateral physical distance from the wheel axle center to the chassis geometric center. The controller sends the aforementioned speed command to the underlying motor drive board via the fieldbus. The drive board then uses the underlying closed-loop control to output the corresponding pulse width modulation waveform. In this type of purely linear motion, the yaw rate... The value is always zero. Thanks to the special arrangement of the Mecanum rollers, the above calculation ensures that the robot can move smoothly in a straight line or laterally without changing the heading angle of the robot body.

[0034] Trajectory-based actions are responsible for handling the robot's curvilinear motion and in-place turning logic, achieving refined trajectory control through the coupling of radius and speed.

[0035] When a turning command is received, the controller extracts the linear velocity parameters. With turning radius parameter Based on the principle of rigid body rotation about a fixed axis, the yaw rate of the chassis target... The calculation equation is as follows: ; Subsequently, the system will determine the target's longitudinal linear velocity. lateral linear velocity and the target angular velocity calculated above Substituting the aforementioned inverse kinematics matrix equations into vector superposition, the independent rotational speeds required for each of the four wheels are calculated. When the turning radius is zero, the system forces... Furthermore, the linear velocity parameter is directly mapped to the rotational angular velocity, causing the front and rear wheels on the same side of the chassis and the diagonal wheel sets to generate a specific proportion of reverse driving force, thereby achieving smooth rotation in place based on the robot's geometric center.

[0036] The composite mission action integrates multi-dimensional sensor feedback and dynamic logic processing to achieve complex accompanying and autonomous navigation functions.

[0037] The vision processing thread inputs color image frames into a lightweight YOLOv8 object detection network deployed on a local edge computing unit, extracts candidate bounding boxes for pedestrians in the image, and uses a pedestrian re-identification model to pinpoint the unique following target. After target identification, the system extracts the median depth of the corresponding pixel region in the depth image for the bounding box, and calculates the distance between the target and the robot. The actual physical distance at any given moment The control layer utilizes the parameters of actual distance and minimum safe distance. Calculate real-time distance deviation : ; The system employs a position-based proportional-integral-derivative (PID) control law, combined with the desired feedforward speed at the current moment as set in the speed range list. Calculate the final longitudinal following linear velocity command sent to the chassis. Its continuous domain governing equations are: ; in, For proportional gain, For integral gain, This represents the differential gain. In practical digital control systems, the above equation is discretized, and high-frequency speed commands are output, thereby achieving flexible intelligent following that does not violate safety boundaries.

[0038] The underlying execution of multi-mode autonomous navigation relies on the Nav2 high-order navigation algorithm stack within the robot operating system ecosystem. When the system triggers a navigation command, the upper-level logic first parses the target identifier code to generate the target's absolute pose in the underlying coordinate system. After receiving the target pose, the Nav2 algorithm stack's global path planner generates a reference path without global collisions based on the persistent occupancy grid map in its memory information, using the A* pathfinding algorithm. During actual movement, the local path planner combines the local point cloud obstacle information output in real time by the LiDAR and depth camera, uses a time-elastic band algorithm for rolling optimization to generate a local trajectory, and strictly adheres to the maximum motion speed constraint. The local planner continuously and frequently outputs three-degree-of-freedom velocity control commands adapted to the Mecanum wheels to the chassis drive nodes, achieving high-precision physical space autonomous pathfinding and obstacle avoidance displacement.

[0039] By defining highly decoupled and parameterized meta-action interfaces, a standardized set of robot motion instructions was constructed. This design not only achieves high-fidelity conversion from semantic instructions to physical control but also endows the system with extremely flexible motion orchestration capabilities. Through the design of mutually exclusive parameters, the system avoids logical conflicts at the execution level, greatly improving the robustness of the robot's underlying actuators. Simultaneously, the encapsulation of segmented speed-adjustable following and multimodal navigation allows the upper-level large model to implement complex interactive behaviors through simple JSON sequence calls, significantly reducing the development complexity of embodied intelligence systems.

[0040] The robot motion control integrates multi-dimensional contextual information and drives a cloud-based large language model to generate structured action sequences, ultimately achieving a complete closed-loop process from semantic understanding to physical execution. Figure 2 As shown.

[0041] The system has a built-in, independently running context aggregation daemon responsible for monitoring data changes on the system bus in real time. When the system detects a clear user voice command or when the sentiment confidence output by the emotion computing engine undergoes a significant jump, the aggregation process is triggered by the event, initiating the feature fusion and prompt word construction process of the multi-dimensional context.

[0042] The system first writes strict system role settings and grammatical constraints at the top level of the prompt words, forcing the large language model to act as the robot's underlying action planner, and limiting its response content to contain only JSON arrays that conform to the preset data structure, prohibiting the output of any natural language explanation text.

[0043] The aggregation process extracts corresponding data from the local database and transforms it into structured text for injecting prompts. The structured text assembly hierarchy includes: a transient intent layer, which loads the original instruction text captured by the external information interface and the current emotion category label with the highest confidence; a long-term preference layer, which retrieves the motion preference parameter module from the dynamic user profile and extracts various default call parameters of meta-actions accumulated in the user's history as the decision background; and a spatial topology layer, which reads the spatial semantic topology map and extracts the node identifier code of the current robot, the identifier codes of connected nodes, and the passable width of the channel, providing physical boundary constraints for actions involving navigation.

[0044] The local agent sends the fused prompt message to the cloud-based large language model decision-making system via an encrypted network communication protocol. The cloud-based large language model performs chain-like reasoning based on the physical boundary constraints and user preferences provided in the prompt words. The model decomposes ambiguous intentions in natural language into multi-step discrete tasks and precisely maps them to five predefined categories of meta-action identifiers. The large language model outputs an ordered array consisting of multiple action units. To ensure system stability, the cloud-based decision-making module enforces a JSON-based validity check upon receiving the data from the model. Only when the data structure fully conforms to the following definitions does it proceed to the distribution phase.

[0045] Action sequence messages that pass validity verification are sent from the local workflow to the underlying motion control slave device. The local controller uses a first-in-first-out queue mechanism and a finite state machine to achieve physical closed-loop scheduling of actions.

[0046] The local controller parses the action nodes in the queue sequentially. For each meta-action, the controller first extracts the default preference parameters of the action from the local user profile. Then, the system performs a dynamic comparison: if a certain control value is explicitly issued in the current action parameter dictionary, the controller uses the issued value to overwrite the default parameter; if a parameter in the dictionary is empty, it directly inherits the default value from the user profile, thus forming the final execution instruction.

[0047] After receiving the command, the control system implements low-level scheduling through a state machine. The state machine extracts the action command from the head of the queue and calls the low-level inverse kinematics algorithm to drive the chassis motor. The controller internally allocates a high-frequency timer or odometer integrator to monitor the physical execution threshold of the action in real time with millisecond-level precision. When the set threshold of the current action is exhausted or reached, the low-level system generates a software interrupt, and the state machine actively sends a smooth deceleration command to the motor. At the same time, it automatically loads and starts the execution of the next action command from the queue. If the ultrasonic radar or drop sensor at the low level triggers the highest priority hardware safety alarm during queue execution, the state machine will immediately suspend the current queue and force the chassis to stop abruptly to ensure absolute physical safety.

[0048] This invention achieves the following significant technical effects in the field of robot motion control through a mechanism design of "local workflow assembly and scheduling + cloud-based large model full inference": Completely decoupling business decisions from underlying control logic: The local intelligent agent workflow returns to its pure data flow and hardware-driven essence, no longer containing any complex "parameter overriding" or "rule judgment" business code. All personalized parameter decisions, emotion adaptation, and intent decomposition are handled by a large cloud model with powerful reasoning capabilities. This design greatly reduces the code complexity of the local system and improves the system's maintainability and scalability.

[0049] Giving robots the ability to generate global contextual awareness: By feeding a complete user profile and environmental topology into a large cloud model at once, the system breaks through the limitations of local rule-based judgment. The large model can globally coordinate user habits and real-time emotions in a single reasoning process, directly generating coherent motion sequences with "warmth" and high personalization, making the choreography of complex embodied behaviors more natural and fluid.

[0050] Ensuring physical determinism in high-order intelligent decision-making: Although the decision-making power for action parameters is completely shifted to the cloud-based large model, the system retains the legality verification of the local workflow and the hardware state machine preemption mechanism. This fully leverages the generalization and reasoning capabilities of the cloud-based large language model while defining absolutely safe physical boundaries for the robot with the hard rules of the local code, effectively avoiding dangerous movements that may be caused by model illusions.

[0051] When the system receives a new user natural language command through an external information interface, the local agent workflow intercepts the command and initiates a dedicated intent parsing preprocessing process before generating subsequent action sequences. The local agent workflow packages the original user command, the current system execution state, and associated user profile parameters, and sends them to the cloud-based large language model specifically responsible for intent parsing. Upon receiving the request, the cloud-based intent parsing model forces the following processes to be executed in parallel: Redundant colloquial elements in natural language are stripped away, and the user's core requests are translated into structured intermediate instructions that are easier for subsequent serialization models to understand. For example, "You were too close, move away and then turn around" is translated as "Increase the safe distance, step back, and then perform the turning action." The instructions are examined to see if they contain evaluations or fine-tuning requests for the movement state. If an adjustment intention is identified, the baseline parameters in the prompt are retrieved, and the incremental or absolute value of the parameter modification is directly calculated. A joint parsed message is output, and the cloud model ultimately outputs a joint parsed message back to the local machine. The message contains the cleaned, structured instruction text, as well as a parameter adjustment increment dictionary that may be empty.

[0052] Upon receiving the joint parsing message, the local agent workflow splits it into an action execution flow and a profile update flow. For the profile update flow, which includes an incremental dictionary of parameter adjustments, the local agent implements a robust security takeover through code execution. The workflow extracts the target action identifier, target parameter key name, and adjustment value from the incremental dictionary. The workflow code then enforces a comparison to ensure that the values ​​do not exceed the accuracy limits of the underlying sensors or the maximum and minimum absolute safety thresholds allowed by the motor dynamics. After successful verification, the local agent workflow uses local addressing to directly overwrite the new parameter values, calculated by the large model and verified locally, into the corresponding node of the local user profile database. Simultaneously, the system generates a corresponding adjustment log in the local interaction history module, completing the evolution and iteration of the profile data.

[0053] The adaptive evolution of the user profile and the subsequent generation of specific actions are seamlessly linked, ensuring that the user's fine-tuning feedback can take effect immediately in the current or next action. After confirming that the user profile has been updated, the local agent workflow extracts the "structured intermediate state instruction" generated in the first step, and packages it together with the newly updated user profile and spatial topology map as input, triggering the steps described in Chapter 4. Since the large model receives clear, pre-processed instructions at this point, and references the latest profile parameters that have been adaptively adjusted, the subsequently generated action sequence messages can immediately exhibit kinematic characteristics that meet the user's expectations.

[0054] Unlike traditional systems that rely on post-action evaluation and parameter adjustments only after an action is completed, this invention performs intent parsing and profile updates before generating a specific action sequence. This allows user feedback to be applied to the local database instantly and take effect immediately in subsequent actions, significantly improving the real-time performance and agility of robot-assisted control.

[0055] By establishing a dedicated intent parsing model for front-end "preprocessing," the system successfully separates the complex tasks of spoken audio cleaning, emotional soothing, and incremental calculation from the intricate spatial path topology planning and serialization combination. The structured translated instructions enable the subsequent action sequence generation model to focus more intently and accurately generate the underlying JSON parameter messages, reducing the incidence of model illusions and formatting errors at the architectural level.

[0056] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. A robot motion control system based on multi-dimensional context awareness and a large language model, characterized in that, include: Information perception and interaction module, intelligent agent decision-making and generation module, and underlying motion execution module; Each module communicates through a standardized data protocol to jointly realize a complete link from information input to physical feedback; by utilizing the logical reasoning capabilities of the cloud-based large language model, combined with the robot's local multidimensional perception data and long-term memory information, the interactive intent is mapped into a sequence of underlying executable meta-actions. The information perception and interaction module is responsible for the collection, preprocessing and local memory management of multi-source heterogeneous data; The intelligent agent decision-making and generation module comprehensively evaluates the global context information summarized by the perception module and generates structured action sequence messages through logical reasoning. The underlying motion execution module is deployed on the robot's local controller. It is responsible for receiving the motion sequence generated by the intelligent agent module and driving the motor actuator to complete the physical action through internal parameter verification and spatiotemporal planning algorithms.

2. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 1, characterized in that, The robot acquires underlying physical environment information through its onboard lidar and millimeter-wave radar, and processes the raw point cloud data using a three-dimensional laser inertial odometry mapping algorithm based on multi-sensor fusion. After mapping is completed, the system projects the three-dimensional point cloud downward to generate a two-dimensional occupied grid map.

3. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 2, characterized in that, After receiving feature data, the cloud-based big language model assigns semantic labels to each physical region based on common sense reasoning and calculates the physical connectivity between regions. The cloud-based large language model returns results according to the topology graph data structure specified by the system; the system directly persists the returned results as a local spatial semantic topology graph, which serves as the underlying pathfinding basis for regional navigation meta-actions.

4. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 3, characterized in that, During the execution of a specific action sequence, the system accepts location marking commands issued by the user via voice; the system captures the current global absolute coordinates and heading angle of the odometer and adds them to the environment memory bank for direct invocation by navigation meta-actions.

5. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 4, characterized in that, The system uses a nested dictionary structure to build a user profile database; the user profile database is the only object of operation for the adaptive model to overwrite motion parameters.

6. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 5, characterized in that, The underlying motion execution module works by parsing standardized command messages; the command messages are in the form of a serialized action set, and each meta-action contains an action name identifier and a corresponding parameter mapping dictionary; the system sets strict type checks and physical threshold constraints for each parameter to ensure the smoothness and safety of the motion trajectory; all execution thresholds contain two mutually exclusive parameters, and the system can only specify one of them when calling; the two mutually exclusive parameters are duration and movement distance.

7. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 6, characterized in that, When implementing basic translational movements of the robot based on the kinematic model of the Mecanum wheel chassis with four independent drive, the upper-level application of the system converts the set motion velocity into a target linear velocity vector in the local coordinate system of the chassis when performing longitudinal or lateral translational movements; the controller converts the longitudinal linear velocity into a target linear velocity vector based on the inverse kinematic matrix of the Mecanum wheel. lateral linear velocity and yaw rate Uncoupled solution for the left front wheel Right front wheel Left rear wheel Right rear wheel The independent target rotational speed; its inverse kinematics solution matrix equation is as follows: ； in, Let be the physical radius of the Mecanum wheel. This is the longitudinal physical distance from the wheel axle center to the geometric center of the chassis. The lateral physical distance from the wheel axle center to the chassis geometric center; the controller sends the above speed command to the underlying motor drive board via the fieldbus, and the drive board outputs the corresponding pulse width modulation waveform using the underlying closed-loop control; yaw rate It is always zero.

8. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 7, characterized in that, The robot's trajectory-like movements are achieved by coupling radius and velocity; when a turning command is received, the controller extracts the linear velocity parameter. With turning radius parameter ; Based on the principle of rigid body rotation about a fixed axis, the yaw rate of the chassis target. The calculation equation is as follows: ； Subsequently, the system will determine the target's longitudinal linear velocity. lateral linear velocity and the calculated target angular velocity Substituting the aforementioned inverse kinematics matrix equations into vector superposition, the independent rotational speeds required for each of the four wheels are calculated; when the turning radius is zero, the system forces... Furthermore, the linear velocity parameter is directly mapped to the rotational angular velocity, causing the front and rear wheels on the same side of the chassis and the diagonal wheel sets to generate a specific proportion of reverse driving force, thereby achieving smooth rotation in place based on the robot's geometric center.

9. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 8, characterized in that, When performing accompanying and autonomous navigation functions, composite task actions are achieved based on multi-dimensional sensor feedback and dynamic logic processing; The vision processing thread inputs color image frames into a lightweight YOLOv8 object detection network deployed on a local edge computing unit, extracts candidate bounding boxes for pedestrians in the image, and uses a pedestrian re-identification model to locate the unique following target. After target location, the system extracts the median depth of the corresponding pixel region in the depth image for the bounding box and calculates the distance between the target and the robot. The actual physical distance at any given moment The control layer utilizes the actual distance and minimum safe distance parameters. Calculate real-time distance deviation : ； The system employs a position-based proportional-integral-derivative (PID) control law, combined with the desired feedforward speed at the current moment as set in the speed range list. Calculate the final longitudinal following linear velocity command sent to the chassis. The governing equations for the continuous domain are: ； in, For proportional gain, For integral gain, This is the differential gain.

10. The robot motion control system based on multi-dimensional context awareness and a large language model according to claim 9, characterized in that, By integrating multi-dimensional contextual information and driving a large language model in the cloud to generate structured action sequences, a complete closed loop from semantic understanding to physical execution is ultimately achieved. The system has a built-in independent context aggregation daemon process that is responsible for monitoring data changes on the system bus in real time. When the system detects a clear user voice command or when the sentiment confidence output by the emotion computing engine changes significantly, the aggregation process is triggered by the event, starting the feature fusion and prompt word construction process of the multi-dimensional context. The system first writes strict system role settings and grammatical constraints at the top level of the prompt words, forcing the large language model to act as the robot's underlying action planner, and limiting its response content to contain only JSON arrays that conform to the preset data structure, prohibiting the output of any natural language explanation text; The aggregation process extracts corresponding data from the local database and transforms it into structured text for injecting the prompt words. The local agent sends the fused prompt word message to the cloud-based large-scale model decision-making system via an encrypted network communication protocol. The cloud-based large-scale language model performs chain-like reasoning based on the physical boundary constraints and user preferences provided in the prompt words. It decomposes the fuzzy intent in natural language into multi-step discrete tasks and maps them to five predefined categories of meta-action identifiers. The large-scale language model outputs an ordered array consisting of multiple action units. To ensure the stability of the system, the cloud-based decision-making module enforces a JSON-based validity check after receiving the data returned by the model. The system only proceeds to the distribution stage if the data structure fully conforms to the following definition. Action sequence messages that pass the validity check are sent from the local workflow to the underlying motion control slave device; the local controller uses a first-in-first-out queue mechanism and a finite state machine to realize the physical closed-loop scheduling of actions; The local controller parses the action nodes in the queue in sequence; for each meta-action, the controller first extracts the default preference parameters of the action from the local user profile. The system then performs a dynamic comparison: if a control value is explicitly issued in the current action parameter dictionary, the controller will use the issued value to overwrite the default parameter; if a parameter in the dictionary is empty, it will directly inherit the default value in the user profile, thus forming the final execution instruction. After receiving the instruction, the control system implements low-level scheduling through a state machine; the state machine extracts the action instruction from the head of the queue and calls the low-level inverse kinematics solution algorithm to drive the chassis motor.

Images