Motion control method, system, device and medium for mobile robot

Abstract

The application relates to the technical field of robot control, and discloses a motion control method, system, device and medium for a mobile robot. The method comprises the following steps: acquiring multi-modal observation data and a natural language instruction of the mobile robot; inputting the multi-modal observation data and the natural language instruction into a pre-trained hierarchical vision-language-action model to obtain a structured thinking chain reasoning result and an executable action answer; analyzing the structured thinking chain reasoning result and the executable action answer to generate executable continuous motion control instructions and discrete high-level behaviors; and controlling the mobile robot to perform corresponding embodied actions under a closed loop condition based on the continuous motion control instructions and the discrete high-level behaviors. The application firstly generates a structured thinking chain action plan, and then converts the plan into executable continuous control instructions through an action decoder. The model has improved interpretability, the stability of language understanding is enhanced, and more reliable long-term behavior performance is supported.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a motion control method, system, device, and medium for mobile robots. Background Technology

[0002] Vision-language-motion systems for mobile robots aim to translate natural language commands into embodied behaviors based on perception, reasoning, and executable control. This problem is particularly challenging for quadruped robots because language-guided decisions must ultimately be translated into continuous motion control commands under complex conditions of partial observability, execution noise, and long-term task dependence. Robustly bridging this gap in real-world environments is central to achieving embodied intelligence.

[0003] Recent advances in multimodal foundational models have significantly improved visual perception and language understanding, inspiring extensive research into extending visual-language models to embodied decision-making. However, existing methods still face two practical limitations in quadrupedal visual-language-action scenarios. First, direct language-to-action mapping often blurs the boundary between high-level semantic reasoning and low-level motor execution, resulting in weak interpretability and unstable semantic anchoring in long-term planning. Second, methods that improve stability through latent intermediate representations typically sacrifice transparent reasoning structures, making semantic logic difficult to verify, fault cases difficult to diagnose, and unfavorable for supporting combinatorial decision-making. Summary of the Invention

[0004] This application provides a motion control method, system, device, and medium for mobile robots, which at least solves the technical problems of weak model interpretability and unstable semantic understanding in related technologies.

[0005] According to one aspect of the embodiments of this application, a motion control method for a mobile robot is provided, comprising: Acquire multimodal observation data and natural language commands from mobile robots; The multimodal observation data and natural language instructions are input into a pre-trained hierarchical vision-language-action model to obtain structured thought chain reasoning results and executable action answers. The structured thought chain reasoning results and executable action answers are parsed to generate executable continuous motion control commands and discrete high-level behaviors; Based on the continuous motion control commands and discrete high-level behaviors, the mobile robot is controlled to perform corresponding embodied actions under closed-loop conditions.

[0006] According to another aspect of the embodiments of this application, a motion control system for a mobile robot is also provided, comprising: The acquisition module is used to acquire multimodal observation data and natural language commands from the mobile robot. The reasoning module is used to input the multimodal observation data and natural language instructions into a pre-trained hierarchical vision-language-action model to obtain the structured thought chain reasoning results and executable action answers. The instruction generation module is used to parse the structured thought chain reasoning results and executable action answers to generate executable continuous motion control instructions and discrete high-level behaviors. The control module is used to control the mobile robot to perform corresponding embodied actions under closed-loop conditions based on the continuous motion control commands and discrete high-level behaviors.

[0007] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described motion control method for a mobile robot through the computer program.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described motion control method for a mobile robot at runtime.

[0009] The technical solutions provided in this application embodiment may include the following beneficial effects: This application proposes a hierarchical quadruped robot vision-language-action framework, MobileVLA-R1, which explicitly aligns structured reasoning with continuous control. Instead of directly predicting motion commands from language and observation information, MobileVLA-R1 first generates a structured thought chain reasoning result and action plan based on multimodal observations, and then transforms this plan into executable continuous control commands. This "reasoning-then-execution" design improves the model's interpretability, enhances the stability of language understanding, and supports more reliable long-term behavioral performance. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a method according to an embodiment of this application; Figure 2 This is a general architecture diagram of a motion control scheme for mobile robots according to an embodiment of this application; Figure 3 This is a schematic diagram of a reinforcement learning method according to an embodiment of this application; Figure 4 This is a structural diagram of a motion control system for a mobile robot according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0012] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0013] This application proposes MobileVLA-R1, a hierarchical quadruped robot vision-language-action (VLA) framework that explicitly aligns structured reasoning with continuous control. Instead of directly predicting motion commands from language and observation information, MobileVLA-R1 first generates a structured chain-of-thought (CoT) action plan based on multimodal observations, and then transforms this plan into executable continuous control commands through an action decoder. This "reason-then-execute" design improves the model's interpretability, enhances the stability of language understanding, and supports more reliable long-term behavioral performance. To train this capability, this application constructs MobileVLA-CoT, a large-scale embodied trajectory dataset containing multi-granularity CoT supervision, and employs a two-stage training paradigm: first, CoT alignment is performed through supervised learning to establish structured reasoning capabilities; then, a reinforcement learning method based on group relative policy optimization (GRPO) is used to further optimize the semantic alignment and execution accuracy of actions. This combination significantly improves the consistency from reasoning to control commands while maintaining executability at deployment time.

[0014] The motion control method for mobile robots according to embodiments of this application will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, the method mainly includes the following steps: S101 acquires multimodal observation data and natural language commands from the mobile robot.

[0015] In one implementation, a mobile robot, such as a quadruped robot equipped with an RGB-D camera, LiDAR, and inertial measurement unit, collects multimodal observation data of the current environment in real time through its sensor modules, including RGB images, depth maps, and 3D point clouds. Simultaneously, it receives natural language commands from the user through an interactive interface or task scheduling system, such as "turn right, walk down the corridor, and then lie down in front of the fireplace."

[0016] S102 inputs multimodal observation data and natural language instructions into a pre-trained hierarchical vision-language-action model to obtain structured thought chain reasoning results and executable action answers.

[0017] In one implementation, multimodal observation data and natural language instructions are input into a pre-trained hierarchical vision-language-action model to obtain structured thought chain reasoning results and executable action answers. This includes: encoding the multimodal observation data using a dedicated modality encoder to obtain a label sequence for each modality; projecting the label sequences of each modality onto a shared token space using a learnable linear adapter, and concatenating them with the text label sequence to obtain a multimodal input label sequence. The shared token space refers to the same vector representation space as the text label sequence. After projection, each modality label sequence has the same dimension and semantic alignment with the text token in this space, thereby supporting the subsequent unified attention calculation and fusion reasoning of multimodal information by the backbone network.

[0018] Specifically, a pre-trained visual encoder is used to extract features from the RGB image, a pre-trained depth estimation encoder is used to extract features from the depth map, and a point cloud encoder is used to extract features from the 3D point cloud, thereby obtaining labeled sequences for the RGB modality, depth modality, and point cloud modality, respectively. Then, through a learnable linear adapter, the labeled sequences of each modality are projected into a shared feature space with the same vector dimension as the text labeled sequence, aligning the semantic representation of each modality feature with the text labeled natural language instructions. Finally, the projected modal feature representations are concatenated with the text labeled sequence along the sequence dimension to form a unified multimodal input labeled sequence.

[0019] Furthermore, the multimodal input labeled sequence is input into the backbone network of the pre-trained hierarchical vision-language-action model for inference, generating structured thought chain inference results and executable action answers.

[0020] Specifically, the backbone network, based on the Transformer decoder architecture, deeply fuses multimodal input labeled sequences through a cross-modal attention mechanism to gradually generate structured inference trajectories and corresponding executable action answers. The backbone network first generates inference trajectories describing the decision-making basis based on multimodal context information. Then, based on these inference trajectories, it generates executable action answers containing continuous motion control parameters and discrete high-level behaviors. Finally, it outputs these inference trajectories in a pre-defined structured template format—"inference trajectory + executable action answer"—for use in subsequent deterministic parsing steps.

[0021] S103 parses the results of the structured thought chain reasoning and the executable action answers to generate executable continuous motion control commands and discrete high-level behaviors.

[0022] In one implementation, the structured thought chain reasoning result and the executable action answer are parsed to generate executable continuous motion control instructions and discrete high-level behaviors, including: extracting a structured template output containing reasoning trajectory and executable actions from the structured thought chain reasoning result and the executable action answer; performing deterministic parsing on the structured template output to extract the continuous motion control instructions and discrete high-level behaviors contained in the executable action answer, wherein the discrete high-level behaviors are used to indicate posture or skill switching instructions in the action set.

[0023] Specifically, the structured reasoning results and executable action answers output by the backbone network are presented in the form of a pre-defined structured template, which is a text sequence containing reasoning trajectory labels and executable action labels. During parsing, the reasoning trajectory field and executable action field are first extracted from this structured template to obtain the reasoning trajectory and executable action parameters. Then, the executable action parameters are deterministically parsed, decomposed into two parts: continuous control parameters and discrete high-level behaviors. Among them, the continuous control parameters are converted into planar translational velocity and yaw angular velocity after numerical conversion, which are directly used as inputs to the bottom-level motion controller; the discrete high-level behaviors are mapped to posture switching or skill invocation instructions in a predefined action set, used to trigger the robot to perform non-continuous actions such as sitting down, lying down, and standing up. Finally, the continuous motion control instructions and discrete high-level behaviors together constitute the complete executable action control parameters for use in closed-loop control.

[0024] In an exemplary scenario, at time step t, the agent receives the observation value. And an instruction i∈I, and according to the strategy Output Action We unify continuous motion with discrete behavior using the following representation: , .

[0025] in, This represents the RGB image observation data at time step t. This represents depth map observation data at time step t. This represents the 3D point cloud observation data at time step t. Indicates the plane translation velocity. Yaw angular velocity, These are discrete high-level behaviors derived from a predefined set of actions. These represent the translational velocity components along the x and y axes, respectively. The model output is serialized using a structured template.

[0026] in, Represents the reasoning trajectory, This represents the encoding of executable commands, thereby enabling machine-parseable execution and interpretable decision-making trajectories.

[0027] S104 controls the mobile robot to perform corresponding embodied actions under closed-loop conditions based on continuous motion control commands and discrete high-level behavior.

[0028] In the specific implementation, the continuous motion control commands obtained through deterministic parsing, such as planar translational velocities Vx and Vy and yaw angular velocity ω, along with discrete high-level behaviors, such as posture or skill switching commands, are synchronously sent to the low-level motion controller of the mobile robot. The low-level motion controller converts the continuous motion control commands into torque or position commands for each joint motor, driving the robot to achieve smooth translation and turning movements. At the same time, the discrete high-level behavior commands trigger the corresponding behavior primitives in the predefined action set, controlling the robot to perform non-continuous posture switching such as sitting, lying down, and standing.

[0029] During execution, the robot acquires new multimodal observation data in real time through its sensor module, and regenerates the action command for the next time step based on the updated observation data, forming a closed-loop control loop of perception-reasoning-analysis-execution-reperception until the task described by the natural language command is completed.

[0030] In one implementation, the hierarchical visual-language-action model is trained before inputting multimodal observation data and natural language commands into the pre-trained hierarchical visual-language-action model.

[0031] The hierarchical vision-language-action model of this application includes an input module, a backbone network module, and an output module. The input module includes a modality-specific encoder, a text encoder, and a linear adapter. The modality-specific encoder extracts features from RGB images, depth maps, and 3D point clouds, outputting a labeled sequence for each modality. The text encoder encodes natural language instructions, outputting a text labeled sequence. The linear adapter projects the labeled sequences of each modality onto a shared feature space with the same vector dimension as the text labeled sequences, achieving cross-modal alignment. The backbone network module is initialized with NaVILA model parameters and efficiently fine-tuned using LoRA parameters. The output module is an action decoder, used to extract inference trajectories and executable action answers from the structured output. It deterministically parses the executable action answers into continuous motion control commands and discrete high-level behaviors. During the group-relative policy optimization training phase, the backbone network, as the policy model, is responsible for sampling and generating multiple structured outputs. It is also equipped with a reference model with frozen weights for KL divergence regularization to constrain policy updates from excessively deviating from the reference distribution.

[0032] In one implementation, the method further includes constructing a multi-granularity thought chain training dataset and obtaining a basic embodied trajectory dataset, which includes multimodal observation data, natural language instructions, and state-action history. The basic embodied trajectory dataset is input into the thought chain data engine, and the reasoning process is requested through a structured prompt template to generate multi-granularity thought chain training samples containing intermediate decision-making processes. The multi-granularity thought chain training samples are validated and filtered to obtain a multi-granularity thought chain training dataset. The multi-granularity thought chain training dataset includes fragment-level samples, step-level samples, and navigation-level samples. Each sample is stored in a structured format, including multimodal observation data, natural language instructions, action history, reasoning trajectory, and executable action targets.

[0033] Existing R2R provides command-trajectory pairing data in Matterport3D indoor scenes, serving as a benchmark for language-guided navigation. RxR extends this by adding multilingual commands and longer, semantically richer descriptions, providing denser supervision signals for long-term anchoring. To introduce continuous control signals on legged robot platforms, this application also uses QUARD, a dataset containing motion and manipulation segments of quadruped robots, along with paired observation and action targets. These data sources collectively provide complementary supervision signals, used for language-to-trajectory anchoring and the execution of underlying embodied control, respectively, laying the foundation for synthesizing MobileVLA-CoT.

[0034] Building upon R2R, RxR, and QUARD, MobileVLA-CoT is proposed. This is a large-scale, multi-granular dataset that combines multimodal observations with structured reasoning supervision and executable action goals. Unlike datasets that only contain instructions, which specify goals without intermediate decision-making processes, MobileVLA-CoT provides explicit reasoning steps, connecting linguistic instructions with embodied actions, thereby achieving interpretable supervision of consistent reasoning control.

[0035] The multi-granularity thought chain training dataset MobileVLA-CoT comprises three complementary subsets: fragment-level, summarizing trajectory results and high-level policies; step-level, specifying the next action and its decision-making basis based on current observations and historical information; and navigation-level, providing long-term temporal reasoning that connects global instructions with multi-step decisions along the trajectory. In one implementation, MobileVLA-CoT contains 18K fragment-level samples, 78K step-level samples, and 38K navigation-level samples.

[0036] In the specific implementation, the basic embodied trajectory dataset is first obtained from a public data source. This dataset refers to the raw data obtained from public data sources (such as R2R, RxR, and QUARD), which includes multimodal observation data (such as RGB images, depth maps, and task-specific states), natural language instructions, and optional state-action history. Subsequently, the above basic data is input into the CoT (Co-T) data engine. This application designs a model-independent CoT data engine, a data synthesis tool. This engine uses structured prompt templates to guide a large multimodal model to generate inference trajectories containing intermediate decision-making processes and corresponding executable outputs under given observations and instructions, thereby obtaining multi-granularity CoT training samples. The structured prompt template refers to a pre-designed prompt format that guides the data engine to output content according to a fixed structure of "inference trajectory + executable action answer," ensuring the consistency of the generated results.

[0037] Finally, a four-stage semi-automatic verification process is performed on the generated multi-granularity mind chain training samples, which includes format verification, action consistency verification, security verification, and manual sampling. Samples with format errors, non-executable, unsafe, hallucinatory, mismatched instructions, or visual inconsistencies are removed. After screening, the multi-granularity mind chain training dataset is obtained.

[0038] The dataset contains three complementary subsets: fragment-level samples, step-level samples, and navigation-level samples. Each sample is stored in a structured format and includes at least multimodal observation data, natural language instructions, action history, inference trajectory, and executable action target fields, providing supervision signals for the subsequent two-stage training.

[0039] Furthermore, the model is trained based on the constructed multi-granularity thought chain training dataset. The training adopts a two-stage training paradigm, including: cold start stage: supervised fine-tuning is performed on the pre-constructed multi-granularity thought chain training dataset; reinforcement learning stage: after the cold start stage, the parameters are optimized using an offline group relative policy optimization algorithm.

[0040] In multimodal embodied environments, direct end-to-end reinforcement learning often becomes unstable due to sparse feedback, format drift, and action execution constraints. Therefore, this application introduces a supervised fine-tuning (SFT)-based cold start phase to stabilize the output format and initialize structured reasoning capabilities before reinforcement learning.

[0041] In one implementation, supervised fine-tuning is performed on a pre-constructed multi-granularity thought chain training dataset, including: fine-tuning on fragment-level and navigation-level samples of the multi-granularity thought chain training dataset to enable the model to learn a trajectory-level structured reasoning format and anchor global instructions; and fine-tuning on step-level samples of the multi-granularity thought chain training dataset to enable the executable action answer output by the model to include executable continuous motion control instructions and discrete high-level behaviors.

[0042] Specifically, the MobileVLA-R1 model is first fine-tuned on fragment-level and navigation-level datasets to enable it to learn to generate trajectory-level navigation-level inference in a structured output format. This stage helps the model master the ability to generate coherent long-term decision trajectories and achieve consistent anchoring between natural language instructions and global path planning; that is, the model learns to generate high-level policies and inference chains that run through the entire navigation process, starting from global instructions. Building on the first stage, supervised fine-tuning continues on a step-level dataset, where the samples are within a structured template. <answer>The fields contain executable continuous motion control commands and discrete high-level behaviors. The answer is deterministically resolved as... This is to achieve closed-loop execution, thus enabling a smooth transition from language-guided reasoning to motor execution. This stage primarily focuses on improving format fidelity and action executableness before reinforcement learning.

[0043] The supervised fine-tuning process employs a two-stage strategy. The first stage is based on a multi-granularity thought chain training dataset containing fragment-level and navigation-level samples, while the second stage is based on a step-level training dataset. In each stage, multimodal observation data is input into a hierarchical vision-language-action model. Through forward propagation, the model is forced to autoregressively generate structured thought chain sequences containing explicit reasoning logic and corresponding executable action answers, where the executable action answers encompass continuous motion control commands and discrete high-level behaviors. Subsequently, the cross-entropy loss between the model output and the standard thought chains and action labels labeled in the dataset is calculated. This supervised signal is used to perform gradient backpropagation and weight updates on the LoRA adapter and linear adapter parameters in the model backbone. In minimizing the prediction error, the first stage enables the model to learn long-term temporal reasoning capabilities at the trajectory level, while the second stage improves the model's output format fidelity and action executableness.

[0044] In one implementation, an offline group-relative policy optimization algorithm is used for parameter optimization, including: for a given input pair of multimodal observation data and natural language instructions, a hierarchical vision-language-action model, after supervised fine-tuning, samples and generates multiple candidate structured outputs, which are then parsed into candidate executable actions; for each candidate executable action, motion reward, action reward, and format reward are calculated separately, and weighted summation is performed to obtain the total reward; the total rewards corresponding to multiple candidate executable actions are normalized within the same group to obtain the intra-group relative advantage of each candidate executable action; based on the intra-group relative advantage, a group-relative policy optimization objective function is constructed, and gradient updates are performed based on the objective function to obtain the trained hierarchical vision-language-action model.

[0045] To further enhance the reasoning-based action generation capability, this application employs the GRPO reinforcement learning method, which optimizes the policy by using inter-group relative feedback across multiple sampled responses. Compared to standard PPO-style optimization, GRPO utilizes within-group normalization to reduce sensitivity to absolute reward scaling and stabilizes the training process for structured outputs.

[0046] Specifically, given input , This represents the multimodal observation data at the current time step t. The policy model of the MobileVLA-R1 model to be trained, representing natural language instructions. Generate N candidate outputs . This represents the j-th output, which contains the reasoning trajectory and the executable action answer. Each output is parsed into candidate executable actions. And it is evaluated by a reward function to encourage accurate, executable, and well-formatted predictions. Among them, This represents the planar translation velocity resolved by the j-th candidate output. This represents the yaw rate resolved from the j-th candidate output. This represents the discrete high-level behavior parsed from the j-th candidate output.

[0047] Motion rewards, action rewards, and format rewards are calculated separately, with motion rewards encouraging directional consistency between the prediction and the target's continuous commands. Since translational velocity and yaw rate have different units, cosine similarity is calculated in the normalized command space:

[0048] in, This represents the normalized continuous motion command vector of the j-th candidate output. This represents the translational velocity component in the forward direction (x-axis) resolved from the j-th candidate output. This represents the y-axis translational velocity component resolved from the j-th candidate output. This represents the yaw rate resolved from the j-th candidate output. The normalization factor representing the translational velocity. The normalization factor represents the yaw rate. This represents the normalized vector of continuous motion commands from the target. This represents the y-axis translational velocity component in the target's motion. This represents the translational velocity component in the forward direction (x-axis) of the target's motion. This represents the yaw rate during the target motion. The motion reward is:

[0049] Action rewards use binary matching rewards:

[0050] in, This represents the discrete high-level behavior prediction value obtained after parsing the j-th candidate output. This represents the discrete high-level behavior of the target, that is, the discrete behavior label corresponding to the actual action.

[0051] Formatting rewards ensure machine-parsable structured output by strictly adhering to the inference-execution template:

[0052] in, The label represents the reasoning result, and the label represents the action to be performed.

[0053] Furthermore, the total reward is calculated using a weighted summation:

[0054] in, Indicates the total reward. , , These represent the corresponding weighting coefficients, which can be set according to the actual situation.

[0055] For each group of N samples, calculate the relative advantage within the group:

[0056] in, This represents the intra-group relative advantage value (unnormalized) of the j-th candidate output. This represents the total reward value of the j-th candidate output. This represents the normalized intra-group relative advantage value of the j-th candidate output. Indicates the within-group standard deviation. The constant is represented by k, which is used to iterate from 1 to N and is used to calculate the average reward within the group.

[0057] make This indicates the strategy before the update. This is the reference strategy for freezing. The training objective of the pruned GRPO is:

[0058] in, , This represents the overall objective function for GRPO training. This represents the trainable parameters of the current policy model. This represents the samples in the training dataset D. Find the expected value, where N represents the number of candidate outputs in the group. This represents the probability ratio of the j-th candidate output. Indicates the probability ratio Crop to [ Within the interval, Represents the KL divergence regularization coefficient. Hyperparameters of the cropping range Indicates the current strategy With reference strategy for freezing The KL divergence between them.

[0059] This application employs an offline group-relative policy optimization algorithm to optimize the parameters of the supervised fine-tuned visual-language action model, resulting in a well-trained visual-language action model.

[0060] To facilitate understanding of the methods in the embodiments of this application, the following description is provided in conjunction with the appendix. Figure 2 Further description, such as Figure 2 As shown in the attached figure, this illustrates the system architecture and workflow of the mobile vision-language-action model R1 of this application: The left side is the input end. The 3D scene, images (RGB images), and depth maps collected by the robot are processed by a point cloud encoder, an image encoder, and a depth encoder to extract features, which are then aligned by a linear projection layer. At the same time, natural language commands (such as "turn right, walk along the corridor, then go to the fireplace and lie down in front of it") are encoded by a text encoder. After the multimodal features are input into the mobile vision-language-action model R1 for cross-modal fusion and inference, they are output to the action decoder. The action decoder deterministically parses the structured output into continuous motion control commands and discrete behaviors, including planar translational velocities (Vx, Vy) and yaw angular velocities. Finally, it drives the quadruped robot to complete navigation and action execution in complex indoor scenes according to the commands.

[0061] To facilitate understanding of the methods in the embodiments of this application, the following description is provided in conjunction with the appendix. Figure 3 Further description, such as Figure 3 As shown in the attached figure, this application illustrates the training optimization and reward calculation process based on GRPO: the input data on the left includes RGB and depth visual observations and natural language instructions (such as "walk into the gap between the sofa and the chair, and then sit down"); in the middle GRPO optimization stage, the policy model generates multiple structured outputs (o1 to o) based on input sampling. n Meanwhile, a reference model with frozen weights is used to calculate the KL divergence to constrain policy updates from excessive deviation. Each output is evaluated by the reward function on the right to obtain the corresponding reward value (r1 to r). n The reward function specifically covers three dimensions: exercise reward, action reward, and format reward. These reward values ​​are fed into the group's calculation module and combined with the actions (A1 to A1) output by the strategy model. n It performs advantage estimation and policy gradient update to achieve reinforcement learning optimization of model inference and action generation capabilities.

[0062] The structured thought chain reasoning results output in this application explicitly record the decision-making basis in natural language, ensuring that every robot action has a traceable reasoning trajectory, greatly enhancing the interpretability of the model. These results are then converted into executable, continuous control instructions by an action decoder. This "reasoning before execution" design improves the model's interpretability, enhances the stability of language understanding, and supports more reliable long-term behavioral performance.

[0063] According to another aspect of the embodiments of this application, a motion control system for a mobile robot for implementing the above-described motion control method for a mobile robot is also provided. For example... Figure 4 As shown, the system includes: The acquisition module 401 is used to acquire multimodal observation data and natural language commands of the mobile robot; The reasoning module 402 is used to input multimodal observation data and natural language instructions into a pre-trained hierarchical vision-language-action model to obtain structured thought chain reasoning results and executable action answers. The instruction generation module 403 is used to parse the reasoning results of the structured thought chain and the executable action answers to generate executable continuous motion control instructions and discrete high-level behaviors. The control module 404 is used to control the mobile robot to perform corresponding embodied actions under closed-loop conditions based on continuous motion control commands and discrete high-level behavior.

[0064] It should be noted that the motion control system for mobile robots provided in the above embodiments is only illustrated by the division of the functional modules described above when executing the motion control method for mobile robots. In practical applications, the functions described above can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the motion control system for mobile robots provided in the above embodiments and the motion control method embodiments for mobile robots belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0065] According to another aspect of the present application, an electronic device corresponding to the motion control method for mobile robots provided in the foregoing embodiments is also provided, so as to execute the above-described motion control method for mobile robots.

[0066] Please refer to Figure 5 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 5 As shown, the electronic device includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected via the bus 502. The memory 501 stores a computer program that can run on the processor 500. When the processor 500 runs the computer program, it executes the motion control method for mobile robots provided in any of the foregoing embodiments of this application.

[0067] The memory 501 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0068] Bus 502 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. Memory 501 is used to store programs. After receiving execution instructions, processor 500 executes the program. The motion control method for mobile robots disclosed in any of the aforementioned embodiments of this application can be applied to processor 500, or implemented by processor 500.

[0069] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0070] The electronic device provided in this application embodiment and the motion control method for mobile robots provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0071] According to another aspect of the present application, a computer-readable storage medium corresponding to the motion control method for mobile robots provided in the foregoing embodiments is also provided, wherein a computer program (i.e., a program product) is stored thereon, and when the computer program is run by a processor, it executes the motion control method for mobile robots provided in any of the foregoing embodiments.

[0072] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0073] The computer-readable storage medium provided in the above embodiments of this application and the motion control method for mobile robots provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.< / answer>

Claims

1. A motion control method for mobile robots, characterized in that, include: Acquire multimodal observation data and natural language commands from mobile robots; The multimodal observation data and natural language instructions are input into a pre-trained hierarchical vision-language-action model to obtain structured thought chain reasoning results and executable action answers. The structured thought chain reasoning results and executable action answers are parsed to generate executable continuous motion control commands and discrete high-level behaviors; Based on the continuous motion control commands and discrete high-level behaviors, the mobile robot is controlled to perform corresponding embodied actions under closed-loop conditions.

2. The method according to claim 1, characterized in that, Before inputting the multimodal observation data and natural language commands into the pre-trained hierarchical vision-language-action model, the training of the hierarchical vision-language-action model is further included. The training adopts a two-stage training paradigm, including: Cold start phase: Supervised fine-tuning is performed on a pre-built multi-granularity mind chain training dataset; Reinforcement learning phase: After the cold start phase, an offline group-relative policy optimization algorithm is used for parameter optimization.

3. The method according to claim 2, characterized in that, This also includes building a multi-granularity thought chain training dataset: Obtain a basic embodied trajectory dataset, which includes multimodal observation data, natural language commands, and state-action history; The basic embodied trajectory dataset is input into the thinking chain data engine, and the reasoning process is requested through structured prompt templates to generate multi-granular thinking chain training samples containing intermediate decision-making processes. The multi-granularity thinking chain training samples are validated and screened to obtain the multi-granularity thinking chain training dataset. The multi-granularity thought chain training dataset includes fragment-level samples, step-level samples, and navigation-level samples; each sample is stored in a structured format, including multimodal observation data, natural language instructions, action history, reasoning trajectory, and executable action targets.

4. The method according to claim 3, characterized in that, Supervised fine-tuning is performed on a pre-built multi-granularity thought chain training dataset, including: Fine-tuning is performed on fragment-level and navigation-level samples of the multi-granularity thought chain training dataset to enable the model to learn a structured reasoning format at the trajectory level and anchor it to global instructions; Fine-tuning is performed on step-level samples of the multi-granularity thought chain training dataset so that the executable action answers output by the model include executable continuous motion control commands and discrete high-level behaviors.

5. The method according to claim 2, characterized in that, Parameter optimization is performed using an offline group-relative strategy optimization algorithm, including: For a given input pair of multimodal observation data and natural language instructions, multiple candidate structured outputs are generated by a hierarchical vision-language-action model that has been fine-tuned under supervision, and then parsed into candidate executable actions. For each candidate executable action, calculate the motion reward, action reward, and format reward separately, and then sum them up to obtain the total reward. The total reward corresponding to the multiple candidate executable actions is normalized within the same group to obtain the relative advantage of each candidate executable action within the group; Based on the relative advantage within the group, a group relative strategy optimization objective function is constructed, and gradient updates are performed based on the objective function to obtain a trained hierarchical vision-language-action model.

6. The method according to claim 1, characterized in that, The multimodal observation data and natural language instructions are input into a pre-trained hierarchical vision-language-action model to obtain structured thought chain reasoning results and executable action answers, including: The multimodal observation data are encoded using a modal-specific encoder to obtain a label sequence for each modality; The learnable linear adapter is used to project the token sequences of each modality onto a shared token space and concatenate them with the text token sequence to obtain a multimodal input token sequence. The multimodal input label sequence is input into the backbone network of the pre-trained hierarchical vision-language-action model for inference, generating the structured thought chain inference result and the executable action answer.

7. The method according to claim 1, characterized in that, The structured thought chain reasoning results and executable action answers are parsed to generate executable continuous motion control commands and discrete high-level behaviors, including: Extract a structured template output containing the reasoning trajectory and executable actions from the structured thought chain reasoning results and executable action answers; The structured template output is subjected to deterministic parsing to extract the continuous motion control instructions and discrete high-level behaviors contained in the executable action answer. The discrete high-level behaviors are used to indicate the posture or skill switching instructions in the action set.

8. A motion control system for mobile robots, characterized in that, include: The acquisition module is used to acquire multimodal observation data and natural language commands from the mobile robot. The reasoning module is used to input the multimodal observation data and natural language instructions into a pre-trained hierarchical vision-language-action model to obtain the structured thought chain reasoning results and executable action answers. The instruction generation module is used to parse the structured thought chain reasoning results and executable action answers to generate executable continuous motion control instructions and discrete high-level behaviors. The control module is used to control the mobile robot to perform corresponding embodied actions under closed-loop conditions based on the continuous motion control commands and discrete high-level behaviors.

9. An electronic device, characterized in that, It includes a processor and a memory storing program instructions, the processor being configured to execute the motion control method for a mobile robot as described in any one of claims 1 to 7 when executing the program instructions.

10. A computer storage medium, characterized in that, It stores computer-readable instructions that are executed by a processor to implement the motion control method for a mobile robot as described in any one of claims 1 to 7.

Images