Robot control method and device, electronic device, storage medium, program product

Abstract

Embodiments of the present application disclose a robot control method and device, electronic equipment, storage medium and program product. In the technical solutions provided in the embodiments of the present application, long-range operation control of the robot is realized through an iterative round control manner. Each round of predicted action sequence covers a preset future time step, and the task progress predicted in the previous round is connected between rounds. In this way, on the basis of not changing the reactive action generation paradigm of the existing visual-language-action model architecture, the overall progress perception for a long-range task is realized by introducing a task progress signal, thereby avoiding problems such as repeated issuance of action instructions for completed operations, stage skipping or early termination in the long-range control process of the robot, and realizing stable control of long-range operation of the robot.

Description

Technical Field

[0001] This application relates to the field of robot control technology, specifically to a robot control method and device, electronic equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] In recent years, Vision-Language-Action (VLA) models have made significant progress in the field of robot manipulation, but their applications are still mainly limited to short-term single-step or simple continuous tasks. For long-term manipulation tasks, robots need to continuously execute according to task instructions over dozens or even hundreds of time steps, and correctly switch control strategies between different sub-stages as the task progresses.

[0003] Existing vision-language-action model architectures generally adopt a reactive action chunk paradigm, which directly predicts action sequences within a finite time domain based on current observations. Their control decisions essentially rely solely on current local observations and lack the ability to perceive task progress. This lack of progress awareness leads to problems in long-range control, such as repeatedly issuing action commands for completed operations, skipping stages, or premature termination.

[0004] Therefore, how to achieve stable long-range operation control for robots is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of this application provide a robot control method, a robot control device, an electronic device, a computer-readable storage medium, and a computer program product. Embodiments of this application achieve single-system closed-loop progress perception for long-range tasks without reconstructing the reactive action generation paradigm, enabling stable long-range operation control of the robot.

[0006] One aspect of this application provides a robot control method, which includes: acquiring a long-range task instruction for instructing long-range control of a robot; iteratively executing a robot control process for each round based on the long-range task instruction until the long-range control of the robot ends. Each round of the robot control process includes the following steps: predicting the robot's task progress at a future time based on an observed image of the robot at the current moment and a task progress prompt text corresponding to the current moment, and acquiring a task progress prompt text corresponding to the future time based on the task progress, wherein the future time is determined by the current moment and a preset number of future time steps; predicting a sequence of actions of the robot within the future time steps based on the observed image at the current moment, the long-range task instruction, and the task progress prompt text corresponding to the current moment; controlling the robot to execute the sequence of actions within the future time steps, and triggering entry into the next round when the future time arrives.

[0007] In another exemplary embodiment, the robot's controller is equipped with a robot motion prediction model, which includes a vision-language-action sub-model and a progress prediction module. The vision-language-action sub-model includes a vision-language encoding module and a motion generation module. Predicting the robot's task progress in future moments based on the robot's observed image at the current moment and the corresponding task progress prompt text at the current moment includes: extracting a first visual-language joint feature from the observed image at the current moment and the corresponding task progress prompt text at the current moment using the vision-language encoding module; and predicting the robot's task progress in future moments using the first visual-language joint feature using the progress prediction module.

[0008] In another exemplary embodiment, predicting the robot's action sequence within the future time steps based on the observed image at the current moment, the long-range task instruction, and the task progress prompt text corresponding to the current moment includes: extracting second visual-language joint features from the observed image at the current moment, the long-range task instruction, and the task progress prompt text corresponding to the current moment using the visual-language encoding module; and predicting the robot's action sequence within the future time steps using the action generation module based on the second visual-language joint features.

[0009] In another exemplary embodiment, the process by which the progress prediction module predicts the robot's task progress at a future time based on the first visual language joint features includes the following steps: writing the first visual language joint features into fast weights; based on query tokens and multiple stacked progress prediction blocks, aggregating information related to task stage judgment from the fast weights to obtain progress query features; mapping the progress query features to scalars, and mapping the scalars to continuous progress prediction values ​​within a target value range to obtain the robot's task progress at a future time.

[0010] In another exemplary embodiment, the step of converting the first visual language joint features into fast weights includes: saving the visual language joint features obtained by the progress prediction module during the execution of task progress prediction as a historical feature sequence; and converting the historical feature sequence to obtain the fast weights.

[0011] In another exemplary embodiment, during the process of the progress prediction module predicting the robot's task progress at a future time based on the first visual language joint features, the task progress is constrained to advance only one task stage at a time.

[0012] In another exemplary embodiment, the training process of the visual-language-action sub-model includes the following steps: training an initial visual-language-action sub-model to enable the visual-language-action sub-model to learn sub-task-level action prediction, thereby obtaining a trained visual-language-action sub-model; using the trained visual-language-action sub-model as a teacher policy model; initializing a student policy model based on the trained visual-language-action sub-model; training the student policy model through the teacher policy model to enable the student policy model to learn long-term task-level action prediction, thereby obtaining the trained visual-language-action sub-model.

[0013] In another exemplary embodiment, training the student strategy model using the teacher strategy model to enable the student strategy model to learn long-term task-level action prediction includes: acquiring teacher model input signals and student model input signals, wherein the teacher model input signals include observed image samples and sub-task instruction samples, and the student model input signals include the observed image samples, long-term task instruction samples, and task progress prompt samples; inputting the teacher model input signals into the teacher strategy model to obtain first visual-language joint sample features output by the visual-language encoding module included in the teacher strategy model; inputting the student model input signals into the student strategy model to obtain second visual-language joint sample features output by the visual-language encoding module included in the student strategy model, and action prediction results output by the student strategy model; obtaining a training loss value based on the first visual-language joint sample features, the second visual-language joint sample features, and the action prediction results output by the student strategy model, and optimizing the parameters of the student strategy model based on the training loss value.

[0014] In another exemplary embodiment, obtaining the training loss value based on the first visual language joint feature, the second visual language joint feature, and the action prediction result output by the student policy model includes: calculating a feature alignment loss value based on the first visual language joint feature and the second visual language joint feature; calculating an action supervision loss value based on the action prediction result output by the student policy model and the corresponding sample action label; and calculating the training loss value based on the feature alignment loss value and the action supervision loss value.

[0015] In another exemplary embodiment, the step of calculating the feature alignment loss value based on the first visual language joint features and the second visual language joint features includes: extracting a first key feature from the first visual language joint sample features based on a learnable query token, and extracting a second key feature from the second visual language joint sample features; and calculating the feature alignment loss value based on the first key feature and the second key feature.

[0016] In another exemplary embodiment, calculating the feature alignment loss value based on the first key feature and the second key feature includes: calculating the distance similarity and cosine similarity between the first key feature and the second key feature, respectively; and calculating the feature alignment loss value based on the distance similarity and the cosine similarity.

[0017] In another exemplary embodiment, the training process of the progress prediction module includes the following steps: extracting joint visual-language sample features from the observed image samples and task progress prompt samples using the visual-language encoding module included in the trained visual-language-action sub-model; inputting the extracted joint visual-language sample features into the initial progress prediction module to obtain the task progress prediction result output by the progress prediction module; calculating the training loss value based on the task progress prediction result and the corresponding sample progress label, and optimizing the parameters of the progress prediction module based on the training loss value to obtain the trained progress prediction module.

[0018] In another exemplary embodiment, the step of calculating the training loss value based on the task progress prediction result and the corresponding sample progress label includes: calculating the mean squared error for the task progress prediction result and the actual progress label corresponding to the training sample; obtaining the sample weight corresponding to the training sample, wherein the sample weight of the training sample representing the stage of stagnation is less than the sample weight of the training sample representing the stage of progress; and calculating the training loss value based on the mean squared error corresponding to each training sample and the sample weight corresponding to each training sample.

[0019] In another aspect of this application, a robot control device is provided, comprising: an acquisition module configured to acquire a long-range task instruction for instructing long-range control of a robot; and an iteration module configured to iteratively execute each round of robot control flow based on the long-range task instruction until the long-range control of the robot ends. Each round of robot control flow includes the following steps: predicting the robot's task progress at a future time based on an observed image of the robot at the current moment and a task progress prompt text corresponding to the current moment, and acquiring the task progress prompt text corresponding to the future time based on the task progress, wherein the future time is determined by the current moment and a preset number of future time steps; predicting the robot's action sequence within the future time steps based on the observed image at the current moment, the long-range task instruction, and the task progress prompt text corresponding to the current moment; controlling the robot to execute the action sequence within the future time steps, and triggering entry into the next round when the future time arrives.

[0020] Another aspect of this application provides an electronic device, including: one or more processors; and a memory for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the robot control method as described above.

[0021] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to perform the robot control method as described above.

[0022] Another aspect of this application provides a computer program product, including a computer program that, when executed by a processor of an electronic device, implements the robot control method described above.

[0023] In the technical solution provided in the embodiments of this application, long-term operation control of the robot is achieved through iterative round control. The predicted action sequence of each round covers a preset number of future time steps, and the rounds are connected by the task progress predicted in the previous round. In this way, without changing the existing reactive action generation paradigm of the vision-language-action model architecture, the overall progress perception of long-term tasks is achieved by introducing task progress signals (task progress prompt text). This avoids problems such as repeatedly issuing action commands for completed operations, stage skipping, or premature termination during the long-term control of the robot, and achieves stable control of the robot's long-term operation.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of an implementation environment involved in this application.

[0026] Figure 2 A flowchart of an exemplary robot control method is shown.

[0027] Figure 3 A schematic diagram of an exemplary robot motion prediction model is shown.

[0028] Figure 4 A schematic diagram of a three-stage training process for an exemplary robot motion prediction model is shown.

[0029] Figure 5 A schematic diagram illustrating an exemplary training process for a vision-language-action sub-model is shown.

[0030] Figure 6 A schematic diagram of the training process for another exemplary vision-language-action sub-model is shown.

[0031] Figure 7 A schematic diagram of the training process for an exemplary progress prediction module is shown.

[0032] Figure 8A schematic diagram of an exemplary schedule prediction module based on the TTT mechanism is shown.

[0033] Figure 9 A block diagram of an exemplary robot control device is shown.

[0034] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0037] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0038] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0039] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0040] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0041] As mentioned earlier, existing vision-language-action model architectures generally adopt a reactive action generation paradigm, which directly predicts the sequence of actions (also known as action blocks) within a finite time domain based on the robot's current observations. The control decisions essentially rely only on the current local observations and do not have the ability to perceive the progress of task execution. Therefore, it is suitable for short-term, closed-loop control tasks, but it is prone to stage confusion or forgetting in long-term, multi-stage tasks. Consequently, in long-term robot control, it is easy to issue action commands for completed operations repeatedly, easily generate stage skipping steps, or premature termination.

[0042] To address this issue, existing technologies employ a dual-system approach for controlling long-range robot operations. This approach uses a separate architecture for a high-level Vision-Language Model (VLM) planner and a low-level Vision-Language-Motion Model (VLM) actuator. The high-level VLM planner extracts semantic memory from the robot's historical observations, performs stage-specific reasoning, and outputs sub-task labels or keyframes. The low-level VLM actuator then generates fine-grained actions based on the output of the high-level VLM planner. While this approach demonstrates good performance in long-range task completion, it suffers from high system complexity. Furthermore, the high-level VLM planner's low inference frequency coupled with high latency limits end-to-end response speed, making it difficult to meet the demands of real-time robot control.

[0043] To address this, embodiments of this application provide a novel long-range robot control scheme. Through iterative round-by-round control, in each round of the control process, the predicted action sequence covers a preset number of future time steps, and the task progress predicted in the previous round is seamlessly connected between rounds. In this way, without changing the reactive action generation paradigm of the existing vision-language-action model architecture, the introduction of task progress signals (i.e., task progress prompt text) enables overall progress perception of long-range tasks. This avoids problems such as repeatedly issuing action commands for completed operations, skipping stages, or premature termination during long-range robot control, thus achieving stable control of long-range robot operations. Moreover, since the iteration trigger between rounds depends on the action prediction of the previous round, that is, the robot enters the next round after executing the action sequence within the predicted future time steps. This can be understood as the entire long-range control process relying on a single vision-language-action model architecture that follows the reactive action generation paradigm. Only a progress prediction branch (also called a progress prediction head) is added to the vision-language-action model architecture of the reactive action generation paradigm. There is no need to introduce an external high-level vision-language model planner. As a result, it has a high end-to-end response speed, which can meet the needs of real-time robot control.

[0044] The following will describe in detail the robot long-range control scheme proposed in the embodiments of this application:

[0045] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment involved in this application. Specifically, the implementation environment is a robot control system, which includes a terminal 110 and a robot 120, and the terminal 110 and the robot 120 can communicate wirelessly.

[0046] Terminal 110 is used to run the robot control client. Users can trigger relevant operations through the user interface of this client, such as inputting long-range task commands in natural language. Terminal 110 sends the user-input long-range task commands in natural language to the controller on the robot 120 via the network, serving as task description conditions for subsequent long-range control processes. Terminal 110 can be any electronic device, such as a smartphone, tablet, laptop, desktop computer, smart TV, smartwatch, vehicle terminal, or aircraft; no restrictions are imposed here.

[0047] Robot 120 is an autonomous operating platform with visual perception and joint execution capabilities, including but not limited to mobile robotic arms, humanoid robots, or desktop dual-arm operating robots. Robot 120 is equipped with at least one visual sensor (such as a wrist camera, head camera, depth camera, multi-view camera, or panoramic camera) facing the operating area for acquiring observational images. Robot 120 has a built-in controller, which may be, for example, an embedded computing unit equipped with a GPU (Graphics Processing Unit) or a TPU (Tensor Processing Unit), without limitation.

[0048] The robot 120's built-in controller houses a pre-trained robot motion prediction model. This model can also be understood as a single-system vision-language-action model. Building upon the existing reactive motion generation paradigm's vision-language-action model architecture (including a vision-language encoding module and a motion generation module), this model adds a progress prediction module to perform long-term task semantic understanding, task progress prediction, and motion sequence generation. Specifically, the vision-language encoding module receives observed images and text commands, extracting joint vision-language features from the observed images; the progress prediction module extracts information related to task stage judgment based on the joint features output by the vision-language encoding module, outputting continuous task progress values ​​within the range [0,1]; and the motion generation module uses the current observed image features, long-term task commands, and the task progress prompt text corresponding to the current round as joint conditions to output a sequence of motions within future time steps. During inference, there is no need to call any model independent of this single-system vision-language-action model. The controller performs forward inference once in each control round, sequentially obtaining the task progress prediction for future time steps and the action sequence within the future time steps. After the robot 120 completes its execution, it automatically triggers the next round until the long-term task is completed. Based on this single-system vision-language-action model, problems such as repeatedly issuing action commands for completed operations, stage skipping, or premature termination during long-term robot control can be effectively avoided, achieving stable control of the robot's long-term operations.

[0049] It should also be noted that, Figure 1The robot 120 in the illustrated implementation environment can function as a household cleaning robot and kitchen cooking robot in the home service sector, a sorting and packaging robot and express delivery loading and unloading robot in the logistics and warehousing sector, and a pharmacy dispensing robot and laboratory table cleaning robot in the medical field. The application areas achievable in this implementation environment are not limited here. For example, if a user inputs the long-range natural language task command "clean the living room" into the household cleaning robot, the robot, under the control of the controller, will sequentially perform tasks such as tidying up items, wiping tables, vacuuming, emptying trash, and finishing up. Similarly, if a user inputs the long-range natural language task command "pack and ship the ordered goods" into the sorting and packaging robot, the robot, under the control of the controller, will sequentially perform tasks such as order recognition, shelf retrieval, goods inspection, filling buffers, sealing boxes, labeling, and conveying to the shipping outlet. The specific actions performed by the robot 120 in each task stage are continuously predicted by its built-in controller based on the user's input long-range natural language task command and the images observed by the robot 120 in real time.

[0050] Please continue reading. Figure 2 , Figure 2 A flowchart illustrating an exemplary robot control method is shown. This method can be applied to... Figure 1 The implementation environment shown, and by Figure 1 The implementation environment shown is executed by robot 120, specifically by the controller built into robot 120. This method can also be applied to other implementation environments and executed by the controller built into the robot in other implementation environments to achieve long-range operation control of the robot. The embodiments of this application do not limit this.

[0051] like Figure 2 As shown, in an exemplary embodiment, the robot control method includes steps S210-S250, which are described in detail below: S210, Obtain long-range task instructions for instructing long-range control of the robot.

[0052] First, it should be noted that the long-range control of the robot mentioned in the embodiments of this application refers to the robot's built-in controller continuously driving the robot to execute a task spanning multiple time periods and containing multiple ordered sub-stages after receiving a long-range task instruction in natural language or other equivalent structured semantic form, in an iterative round manner. Each control round only generates a sequence of actions covering future time steps, without planning or generating the complete trajectory of the entire long-range task in a single inference. The controller transmits task progress information between rounds, ensuring that the generation of underlying actions is constrained by the overall stage progress of the task at any given time, until all sub-stages corresponding to the long-range task instruction are completed.

[0053] Therefore, long-range task instructions can also be understood as instructions input before the start of the long-range control process, expressed in natural language text or equivalent structured semantic form, describing the overall goal of the task to be executed. Long-range task instructions cover multiple ordered or conditionally ordered sub-task phases, but do not include step-by-step specifications for the execution time, specific action sequences, or intermediate keyframes of each sub-task phase. In each subsequent round of control flow, the long-range task instructions, as resident semantic conditions, work together with the currently observed image and task progress prompt text to predict task progress and generate actions. The text content of the long-range task instructions remains unchanged throughout a single long-range control process.

[0054] The S220, based on long-range task instructions, iteratively executes each round of robot control procedures until the long-range control of the robot ends.

[0055] The embodiments of this application iteratively execute the robot control process for each round based on the obtained long-range task instructions until the long-range control of the robot ends. This can be understood as follows: after the robot receives the long-range task instructions, it iteratively executes the robot control process at the round-by-round granularity until the long-range task is completed.

[0056] Specifically, each round constitutes the basic time unit for the robot controller to perform long-range control in this embodiment. When the controller receives a long-range task instruction, it performs a task progress prediction and action sequence generation based on the robot's current observations and task progress indicators, and drives the robot to execute the corresponding action sequence. Then, it automatically triggers the next round of control at a preset future time. This cyclical process forms the basic framework of the robot's long-range control architecture. Under this architecture, the controller does not need to plan the complete trajectory at the start of the task, nor does it need to call external models to reissue instructions between rounds. Instead, it achieves continuous closed-loop control across multiple sub-stages through self-consistent progress perception within rounds and state transfer between rounds, thereby realizing stable long-range control of the robot.

[0057] like Figure 2 As shown, the robot control process for each round includes S221-S223, which are detailed below: S221, based on the robot's observed image at the current moment and the corresponding task progress prompt text at the current moment, predict the robot's task progress at future moments, and obtain the corresponding task progress prompt text at future moments based on the task progress.

[0058] The robot's observed image refers to image data acquired by the robot's onboard vision sensors at a specific control moment, reflecting the robot's current environment and operating area. This control moment refers to the moment corresponding to the current control cycle, i.e., the start moment of each control cycle; embodiments of this application also refer to this control moment as the current moment. The image data is intended to provide the robot with visual observation at the current moment and can be a single image of the current frame, a collection of multi-view images, a collection of recent multi-frame images, or a feature tensor preprocessed by an encoder; no limitation is imposed here. The robot's onboard vision sensors include, but are not limited to, RGB cameras, depth cameras, event cameras, multi-view cameras, wrist cameras, head cameras, or panoramic cameras, etc., and no limitation is imposed here either.

[0059] The task progress prompt text at the current moment refers to the text expressed in natural language or a similar natural language form, generated according to a predetermined mapping rule based on the task progress value corresponding to the start time of that round within any control round. Its content is updated dynamically as the rounds progress. For example, the task progress prompt text used in round i is converted from the task progress value predicted in round i-1, while the task progress prompt text used in the first round is usually converted from the default starting progress value, which is typically 0.

[0060] The future time is determined by the current time (i.e., the start time of the current control round) and the number of future time steps. For example, if the current time is represented as t, and the number of future time steps is represented as... Future moments are represented as It is understandable that the future timeframe represents the termination boundary of the action sequence generated in the current round and the starting timeframe of the next round. Within each round, at the future timeframe, the controller executes the action sequence within the predicted future time steps of that round.

[0061] In each round of the robot control flow, the embodiments of this application predict the robot's task progress in future moments based on the robot's observed image at the current moment and the corresponding task progress prompt text. Based on this task progress, the corresponding task progress prompt text for future moments is obtained. This allows for the autonomous deduction of the future progress state of long-term tasks in each round, and the transformation of this future progress state into text conditions that can be directly invoked in the next round. Task progress information is passed in text form across different rounds, providing continuously updated contextual information for subsequent action generation, while avoiding the system complexity and inference latency introduced by an independent high-level planner.

[0062] S222, based on the observed image and long-range task instructions at the current moment, as well as the task progress prompt text corresponding to the current moment, predicts the robot's action sequence within a future number of time steps.

[0063] In each round of the robot control process, the embodiments of this application predict the robot's action sequence within a future number of time steps based on the observed image and long-range task instructions at the current moment, as well as the task progress prompt text corresponding to the current moment. This control process fully follows the existing reactive action generation paradigm. Therefore, without changing the original reactive action generation paradigm, the embodiments of this application combine the long-range task semantics, the current visual observation, and the current progress state as conditions for action generation. This allows the underlying action output to respond to real-time environmental changes while being constrained by the overall task progress, thereby effectively avoiding problems such as repeatedly issuing action instructions for completed operations, skipping stages, or premature termination.

[0064] It is also important to emphasize that in each round of the robot control process, the task progress prompt text used is the task progress prompt text corresponding to the current moment, not the task progress prompt text corresponding to the future moment obtained in S221. In other words, action generation is always conditionalized based on the currently known task progress state, while the result of future progress prediction is only used for condition updates in the next round.

[0065] S223 controls the robot to execute a sequence of actions within a future time step, and triggers the next round when the future moment arrives.

[0066] In the embodiments of this application, in each round of robot control flow, the controller sends the action sequence predicted in S222 frame by frame to the robot's joint controller, driving the robot to execute each action sequentially until all future time steps are covered. Furthermore, during the execution of actions in the future time steps, the controller does not perform new inference calculations, thus maintaining the low-latency, high-frequency execution characteristics of the reactive action sequence. When the future time arrives, the controller executes the action sequence predicted in S222 for the future time steps and automatically proceeds to the next round. At the start of the next round, the system time is marked as the new current time, i.e., the current time in the current round, and the task progress prompt text corresponding to the future time obtained in S221 is used as the task progress prompt text corresponding to the new current time. An observation image of the new current time is acquired, and then S221-S223 are executed to predict the task progress, generate and execute the action sequence, and trigger the next round.

[0067] The process of iterating through steps S221-S223 continues until, in the last round, the predicted task progress value for future moments reaches a preset completion threshold (e.g., approaching 1), indicating that all sub-stages corresponding to the long-term task instruction have been completed. When the last round ends, the controller terminates the iteration, and the long-term control process ends.

[0068] Therefore, the embodiments of this application organize long-range control into an iterative closed-loop architecture consisting of several interconnected rounds, using a preset number of future time steps as the granularity. Each round generates only a sequence of actions within a finite time domain, rather than planning a complete trajectory all at once, and state transfer between rounds is achieved through the rolling updates of task progress prompt text. This design allows the controller to perceive the overall progress of the task at any time, obtaining step-by-step historical context while ensuring real-time performance. This avoids problems such as repeatedly issuing action commands for completed operations, stage skipping, or premature termination during long-range robot control, thereby achieving stable long-range robot control.

[0069] In another exemplary embodiment, the robot's controller is equipped with a trained robot motion prediction model for performing long-range task semantic understanding, task progress prediction, and motion sequence generation.

[0070] It is understandable that the robot motion prediction model is a single-system vision-language-motion model. Task progress prediction and motion sequence generation are both completed by different prediction branches within the same end-to-end network. Moreover, the two prediction branches share the same vision-language encoding backbone network. In a single forward inference, the future progress and motion sequence are predicted in parallel. There is no high-level vision-language model planner independent of this network. The output of the progress prediction branch is only used as one of the text conditions of the motion generation branch, rather than a planning result that has the effect of instructing the underlying execution.

[0071] Figure 3 A schematic diagram of an exemplary robot motion prediction model is shown. Figure 3 As shown, the robot motion prediction model includes a vision-language-action sub-model and a progress prediction module. The vision-language-action sub-model adopts a vision-language-action model architecture based on a reactive motion generation paradigm, including a vision-language encoding module and a motion generation module. Specifically, the vision-language encoding module receives observed images and text commands, extracting joint vision-language features from the observed images; the progress prediction module extracts information related to task stage judgment based on the joint features output by the vision-language encoding module, outputting continuous task progress values; and the motion generation module uses the features of the current observed image, long-term task commands, and task progress prompt text as joint conditions to output a sequence of actions within a future time step.

[0072] In each round of robot control flow, the visual-language encoding module and the progress prediction module execute the process in S221, which predicts the robot's task progress in the future based on the robot's observed image at the current moment and the corresponding task progress prompt text at the current moment. The visual-language encoding module and the action prediction module execute the process in S222, which predicts the robot's action sequence within the future time steps based on the observed image at the current moment, long-range task instructions, and the corresponding task progress prompt text at the current moment.

[0073] For example, the process of predicting the robot's task progress in the future based on the robot's observed image at the current moment and the corresponding task progress prompt text at the current moment, executed by the vision-language encoding module and the progress prediction module in S221, includes the following steps (not shown in the figures): S221-1, through the vision-language encoding module, extracts the first visual-language joint features from the observed image at the current moment and the corresponding task progress prompt text at the current moment; S221-2, through the progress prediction module, predicts the robot's task progress at future moments based on the joint features of the first visual language.

[0074] The process of predicting the robot's action sequence within the future time steps by executing the observation image and long-range task instructions based on the current moment in S222, as well as the task progress prompt text corresponding to the current moment, through the visual-language encoding module and the action prediction module includes the following steps (not shown in the attached figures): S222-1, through the vision-language encoding module, extracts the second visual-language joint features from the observed image and long-range task instructions at the current moment, as well as the corresponding task progress prompt text at the current moment; S222-1, through the action generation module, predicts the robot's action sequence within future time steps based on the joint features of second visual language.

[0075] Therefore, the embodiments of this application do not change the existing visual-language-action model architecture, nor do they change the reactive action generation paradigm of the action generation module. Instead, based on the visual-language-action model architecture of the reactive action generation paradigm, a progress prediction module is added to introduce task progress prompt text as conditional input, injecting the visual-language-action model architecture with the ability to understand the stages of long-term tasks. This enables the visual-language-action model architecture to adjust the action output according to the current task progress, thereby avoiding common problems in long-term tasks such as repeated execution, skipping steps, or premature termination.

[0076] Figure 4 A schematic diagram illustrating a three-stage training process for an exemplary robot motion prediction model is shown. Figure 4 As shown, the training objective of the first stage is to train a teacher policy model that can stably predict the action distribution corresponding to the current subtask, given the current subtask, thus possessing basic subtask-level action generation capabilities. The training objective of the second stage is to train a student policy model that can stably predict the action distribution corresponding to the current task progress based on long-term task instructions and the current task progress. In other words, compared to the teacher policy model, the student policy model no longer relies on explicit subtask instructions; it can execute long-term tasks based solely on the externally provided current task progress. The training objective of the third stage is to train a progress prediction module that makes the model no longer dependent on the externally provided current task progress.

[0077] In the first stage of training, the initial visual-language-action sub-model is trained to learn sub-task-level action prediction, resulting in the trained visual-language-action sub-model.

[0078] As is understandable, a subtask is a task concept that corresponds to a long-term task. A long-term task can include several subtasks. For example, suppose a long-term task is "pouring water," its corresponding subtasks include getting up, going to the kitchen, picking up the kettle and pouring water, and handing the water to the user. Typically, in real-world applications, a long-term task may include dozens or even hundreds of subtasks; this is merely an illustrative explanation of the difference between long-term tasks and subtasks.

[0079] Figure 5 This diagram illustrates an exemplary training process for a vision-language-action sub-model, corresponding to the first stage of training. The vision-language-action sub-model can employ a pre-trained reactive action generation paradigm architecture, without limiting the specific model type. Training samples include observation image samples and sub-task instruction samples, as well as robot state samples. These robot state samples describe the robot's body state and typically include information such as joint positions and velocities. The real action labels for the training samples contain multi-dimensional vectors of the robot's joint poses; however, different models may have different action space representations, such as joint space representations or end effector (EEF) space representations, which are not limited here.

[0080] The visual-language-action sub-model outputs action sequences for future time steps corresponding to the training samples. The first-stage training loss is calculated based on the deviation between the action sequences output by the visual-language-action sub-model and the actual action labels of the corresponding training samples. This loss is then used to optimize the parameters of the visual-language-action sub-model. The first-stage training loss is essentially a supervised loss, such as cross-entropy loss. Through this first-stage training, the visual-language-action sub-model learns to stably learn the action distribution corresponding to each sub-task. The resulting trained visual-language-action sub-model possesses basic sub-task-level action prediction capabilities. This trained visual-language-action sub-model is then used as a teacher policy model to guide the second-stage training process.

[0081] In the second stage of training, the trained visual-language-action sub-model is used as the teacher policy model, and the student policy model is initialized based on the trained visual-language-action sub-model. The student policy model is trained through the teacher policy model, enabling the student policy model to learn long-term task-level action prediction, so as to obtain a well-trained visual-language-action sub-model.

[0082] Since the teacher strategy model has already been trained in the first phase, the parameters of the teacher strategy model need to be frozen in the second phase of training. Only the parameters of the student strategy model need to be optimized, but the teacher strategy model is needed to guide the parameter optimization of the student strategy model. The initial parameters of the student strategy model come from the trained visual-language-action sub-model obtained in the first phase of training. In other words, at the beginning of the second phase of training, the parameters of the student strategy model and the teacher strategy model correspond to each other.

[0083] Figure 6 A schematic diagram of another exemplary training process for a vision-language-action sub-model is shown, corresponding to the second stage of training. For example... Figure 6 As shown, the second phase of training involves two paths, corresponding to the teacher's strategy model and the student's strategy model, respectively. Figure 6 The training process shown may include the following steps in the second phase (not shown in the attached diagram): S610, acquire the teacher model input signal and the student model input signal. The teacher model input signal includes observation image samples and subtask instruction samples. The student model input signal includes observation image samples, long-term task instruction samples and task progress prompt samples. S620, Input the teacher model input signal into the teacher strategy model to obtain the first visual-language joint sample features output by the visual-language encoding module contained in the teacher strategy model; S630, input the student model input signal into the student policy model, obtain the second visual-language joint sample features output by the visual-language encoding module contained in the student policy model, and the action prediction results output by the student policy model; S640 obtains the training loss value based on the joint sample features of the first visual language, the joint sample features of the second visual language, and the action prediction results output by the student policy model, and optimizes the parameters of the student policy model based on the training loss value.

[0084] It should be noted that the input signals for the teacher model and the student model are different. The teacher model signal includes observation image samples and subtask instruction samples, while the student model input signal includes observation image samples, long-term task instruction samples, and task progress prompt samples. It should also be noted that the task progress mentioned in the embodiments of this application is a discretized representation based on the current subtask stage, typically a percentage. For example, assuming a long-term task contains 5 subtasks, after completing 2 subtasks, the task progress would be 40% or 0.4%.

[0085] In the second phase of training, the training loss value includes two parts: feature alignment loss value and action supervision loss value. The feature alignment loss value is calculated by performing loss calculation on the first visual-language joint sample features output by the visual-language encoding module of the teacher's policy model and the second visual-language joint sample features output by the visual-language encoding module of the student's policy model. The action supervision loss value is calculated by performing loss calculation on the action prediction results output by the student's policy model and the real action labels corresponding to the training samples.

[0086] For example, the training loss value in the second stage can be expressed as the following formula:

[0087] in, This represents the training loss value in the second stage. This represents the action supervision loss value. This represents the feature alignment loss value. This represents the preset hyperparameters used to adjust the relative proportions of action supervision loss and feature alignment loss in the joint parameter optimization process. This ensures that while the student policy model reproduces the action distribution of the teacher policy model, the visual-language joint representation output by its visual-language encoding module is also aligned with the visual-language joint representation of the teacher policy model. It is usually greater than 0.

[0088] The visual-language joint sample features output by the visual-language encoding modules of both the teacher and student policy models are typically long sequences, high-dimensional, and contain a large amount of redundant feature information, such as object texture, lighting, background, and language token alignment noise. One of the training objectives in the second stage is to selectively distill the stage meaning of the teacher policy model into the student policy model. If the alignment loss is directly calculated for the first and second visual-language joint sample features, the student policy model would be forced to align with low-level visual noise unrelated to action prediction, resulting in diluted distilled signals and requiring very high computational resources. Therefore, in an exemplary embodiment, a first key feature is extracted from the first visual-language joint features based on a learnable query token, and a second key feature is extracted from the second visual-language joint features. An alignment loss value is then calculated based on the first and second key features. The alignment loss value can be the cosine similarity or L1 distance between the first and second key features, and is not limited here.

[0089] In this context, the first and second key features refer to feature information related to action prediction, and their extraction can be understood as a feature compression process. The learnable query tokens are those that need to be updated during the second stage of training, thus making the feature compression process more accurate. The learnable query tokens can be provided by a lightweight Q-Former module containing 32 query tokens, or by other network modules; no limitation is imposed here. Therefore, the embodiments of this application do not affect the backbone network (visual-language encoding module) of the visual-language-action sub-model, and have a low parameter count, ensuring no gradient overflow after freezing the parameters of the visual-language-action sub-model.

[0090] In another exemplary embodiment, the distance similarity and cosine similarity between the first key feature and the second key feature are calculated respectively, and the feature alignment loss value is calculated based on the calculated distance similarity and cosine similarity. The distance similarity is used to constrain and ensure the consistency of feature activation amplitude, while the cosine similarity is used to constrain and ensure the consistency of semantic orientation. Their synergy allows the student policy model to reproduce the staged visual semantic representation of the teacher policy model without relying on explicit subtask instructions.

[0091] The calculation process of the feature alignment loss value can be expressed as the following formula:

[0092] in, This represents the feature alignment loss value. This represents the first key feature extracted from the first visual-language joint sample features output from the visual-language encoding module of the teacher strategy model based on the query token. This represents the second key feature extracted from the second visual-language joint sample features output from the visual-language encoding module of the student policy model based on the query token. This represents the preset hyperparameters. It is usually greater than 0.

[0093] Therefore, in the second stage of training, by jointly optimizing the action supervision loss and feature alignment loss, the student policy model, which initializes the self-teacher policy model, not only learns action generation capabilities comparable to the teacher policy model, but also, through the dual constraints of L1 distance and cosine similarity, enhances the teacher policy model's deep visual understanding of the sub-task stage. This is achieved by using only long-term task instruction samples and task progress cue samples, completely eliminating dependence on explicit sub-task instruction samples. The language semantic representation was successfully distilled into itself, thereby effectively avoiding the problem of using incorrect semantics to predict the correct action that may be caused by simple action supervision, while maintaining the lightweight architecture of a single system. This endows the visual-language-action sub-model of the reactive action generation paradigm with the ability to autonomously understand the long-term task stage, laying a key foundation for the closed-loop self-updating of the subsequent progress prediction module and the final autonomous execution of the long-term task.

[0094] The third phase of the training process aims to train an independent progress prediction module that can understand the correspondence between task progress and task stages.

[0095] Figure 7 A schematic diagram of the training process for an exemplary progress prediction module is shown, corresponding to the third stage of training. For example... Figure 7 As shown, the third stage of training no longer updates the visual-language-action sub-model, but freezes the parameters of the visual-language-action sub-model obtained in the second stage of training, and only uses the visual-language encoding module included in the visual-language-action sub-model to extract the visual-language joint sample features corresponding to the observed image samples and task progress prompt samples.

[0096] Corresponding to Figure 7 The training process shown, specifically the third stage, may include the following steps (not shown in the attached diagram): S710 extracts joint visual-language sample features from observed image samples and task progress prompt samples using the visual-language encoding module contained in the trained visual-language-action sub-model. S720: Input the extracted visual language joint sample features into the initial progress prediction module to obtain the task progress prediction result output by the progress prediction module; S730 calculates the training loss value based on the task progress prediction result and the corresponding sample progress label, and optimizes the parameters of the progress prediction module based on the training loss value to obtain a trained progress prediction module.

[0097] It should be noted that the task progress prediction module outputs a prediction of the task progress corresponding to a future time based on the joint visual-language sample features. If we assume that the observed image sample corresponds to time t, and the task progress prompt sample also corresponds to time t, the progress prediction module will output... The predicted task progress value at any given time. This indicates the preset number of future time steps. Similarly, the sample progress label should also be... The actual progress value corresponding to each moment. Therefore, the progress prediction module obtained through the third stage of training can predict the task progress at the future moment corresponding to the next number of time steps, based on the robot's current observed image and current task progress. In other words, the progress prediction module realizes a closed-loop update of the task progress, ultimately enabling the robot's motion prediction model to actively track task progress without external progress input, relying instead on the progress prediction module trained in the third stage. This results in a high end-to-end response speed, thus meeting the needs of real-time robot control.

[0098] The progress prediction module can use an attention mechanism to predict task progress values ​​based on the joint visual-language sample features, or it can use a TTT (Test-Time Training) mechanism to predict task progress values ​​based on the joint visual-language sample features; neither is limited here. However, since the TTT mechanism has fast weights, the joint visual-language sample features can be written into the fast weights, so that the joint visual-language sample features are compressed and stored in parameter form. This method has the advantages of constant memory usage and constant single-frame computational overhead during long-term control, thus making it more suitable for the architecture goal of the single-system vision-language-action model in this application embodiment.

[0099] Figure 8 A schematic diagram of an exemplary schedule prediction module based on the TTT (Time-to-Time) mechanism is shown. Figure 8As shown, the progress prediction module includes fast weights, a feature extraction module, a regression layer, and an activation layer. Fast weights refer to the dynamic memory carrier under the TTT mechanism; their capacity is usually fixed. The process of writing the joint visual-language sample features into the fast weights is to convert the joint visual-language sample features into linear mapping parameters, which can also be understood as a feature compression process. The feature extraction module, based on learnable query tokens, aggregates information related to task stage judgment from the fast weights and outputs progress query features. The regression layer maps the progress query features to a scalar, and the activation layer maps this scalar to continuous progress prediction values ​​within a target value range. The target value range can be [0,1] or [0,100%], without restriction.

[0100] The feature extraction module consists of multiple stacked progress prediction blocks. Within each block, based on a query token, subspace representations related to task stage judgment are further aggregated from the input signal. The progress query features output by the last progress prediction block are highly aggregated features relevant to task stage judgment. This can also be understood as follows: after multiple stacking layers, the query token gradually evolves from an initial general random vector into a progress-aware latent vector, whose direction is highly aligned with the task completion progress in the feature space. During the third stage of training, the query token is optimized through backpropagation. Therefore, the query token does not originate from any input image or text, but rather from a fixed question within the model: what is the current task progress?

[0101] Therefore, during inference, the progress prediction module based on the TTT mechanism writes the first visual-language joint features extracted by the visual-language encoding module for the observed image at the current moment and the corresponding task progress prompt text at the current moment into the fast weights. Then, based on the query token obtained from training and multiple stacked progress prediction blocks, it aggregates information related to task stage judgment from the fast weights to obtain progress query features. The progress query features are then mapped to scalars, and the scalars are mapped to continuous progress prediction values ​​within the target value range, thereby obtaining the robot's task progress at future moments.

[0102] In an exemplary embodiment, during inference, the progress prediction module can also be constrained to advance the task progress by only one task stage at a time, avoiding skipping multiple sub-task stages at once. This allows the task progress variable, predicted by the robot motion prediction model itself and updated in the closed loop, to be obtained during inference.

[0103] In another exemplary embodiment, during inference, the visual-language joint features obtained by the progress prediction module during the task progress prediction process can also be saved as a historical feature sequence, and a fast weight can be obtained by converting the historical feature sequence. In this way, the fast weight can be used to form a parametric morphological historical memory, avoiding the loss of distant history due to context truncation or attention decay.

[0104] In yet another exemplary embodiment, during inference, fast weights can also be applied to the action generation module included in the vision-language-action sub-model, so that progress prediction and action generation share the same weight memory. When generating action sequences for future moments, the action generation module not only depends on the currently observed image and the current task progress, but also refers to the historical prediction state, thereby further reducing repeated actions, stage jumps and target localization errors after occlusion, and further improving the stability of long-term operation control of the robot.

[0105] In another exemplary embodiment, the process of S730 calculating the training loss value based on the task progress prediction result and the corresponding sample progress label may further include the following steps: S730-1 calculates the mean squared error for the task progress prediction results and sample progress labels corresponding to the training samples; S730-2, Obtain the sample weights corresponding to the training samples, wherein the sample weights of the training samples that remain in the representation phase are less than the sample weights of the training samples that advance in the representation phase. S730-3, calculate the training loss value based on the mean square error corresponding to each training sample and the sample weight corresponding to each training sample.

[0106] It should be noted that progress is essentially a continuous scalar between [0,1], representing the overall completion rate of the task. If it is discretized into stage labels (such as stage 1, stage 2, stage 3) before calculating the classification loss, subtle progress changes within stages and the continuity information between adjacent stages will be lost. Therefore, S730-1 calculates the continuous regression mean squared error based on the task progress prediction results and sample progress labels corresponding to the training samples. The embodiments of this application use continuous regression mean squared error, which allows each decimal in the model output to directly correspond to the true physical meaning of task progress and provides a smoother and more refined gradient signal for backpropagation.

[0107] Furthermore, in long-duration task datasets, the robot spends most of its time executing the same subtask, and each subtask typically requires many frames. The training samples that actually undergo stage transitions usually only account for about 4% of the total training samples. If a standard mean squared error is directly calculated, the average loss will be extremely low as long as the predicted progress never advances. This would cause the trained progress prediction module to never advance the task during inference. Therefore, S730-2 needs to set different weights based on whether the training samples advance in a stage. The weight of training samples representing stage pauses should be less than the weight of training samples representing stage advancements. S730-3 needs to calculate the training loss value based on the mean squared error and weight of each training sample to force the progress prediction module to accurately capture those 4% of stage transitions.

[0108] The formula for calculating the training loss value in the third stage can be expressed as follows:

[0109] in, This represents the schedule loss value in the third phase. Indicates the first The sample weights corresponding to each training sample Indicates the first The actual progress label corresponding to each training sample. This indicates the task progress prediction results output by the progress prediction module based on the training samples. This represents the magnification factor, which is usually set to 100.

[0110] Therefore, the third stage involves freezing the trained student policy model and using only its visual... The language encoder extracts joint features to train the progress prediction module based on the TTT mechanism, and calculates the training loss using the mean squared error of continuous regression with weighted pause / advancement samples. This forces the model to learn to keenly capture progress transitions on only about 4% of the stage advancement samples, thereby transforming the originally externally injected discrete progress cues into continuous progress that the model can autonomously predict based on the current observations. This progress is then legally mapped to the progress conditions for the next round of action generation, thus realizing a closed loop of self-perception and self-updating of progress. This allows the visual-language-action model architecture of the reactive action generation paradigm to break free from dependence on external progress signals. A single system can autonomously track long-term task stages and significantly reduce the risk of skipping steps and repetitive executions.

[0111] The embodiments of this application achieve efficient transformation from a visual-language-action model based on a reactive action generation paradigm to a single-system visual-language-action model with autonomous execution capabilities for long-term tasks through a three-stage progressive training process. The first stage, through sub-task-level supervised training, provides the pre-trained visual-language-action model with a stable foundation for sub-task-level action prediction, offering a high-quality teacher policy module for subsequent distillation. The second stage introduces a lightweight network module for feature alignment, distilling the teacher policy model's staged semantic understanding of sub-tasks into a student policy model conditioned only by long-term task instructions and progress, while maintaining the original action generation architecture. This frees the model from dependence on explicit sub-task instructions. The third stage further constructs a progress prediction module based on the TTT mechanism, transforming external progress input into state variables that the model can autonomously perceive and update in a closed loop. Ultimately, without introducing an independent high-level planner or increasing system complexity, the model possesses accurate stage tracking and long-term advancement capabilities. While significantly reducing the risk of skipping steps and repeated executions, it achieves a task success rate approaching that of complex dual-system models with a relatively low parameter count, providing a lightweight and efficient solution for long-term robot operation.

[0112] Offline experiments have verified that the robot motion prediction model trained using the three-stage progressive training process proposed in this application exhibits excellent performance in long-range robot operation tasks. Test results show that compared to the existing visual-language-action model π0.5 based on the reactive motion generation paradigm, the robot motion prediction model obtained in this application achieves significant improvements of 15.7% and 16.7% in average task success rate (TSR) and task completion success rate (CSR), respectively. This fully demonstrates that the progress awareness capability injected into the visual-language-action model architecture in this application can effectively overcome the stage forgetting and confusion problems of the reactive motion generation paradigm in long-range tasks. Notably, when compared with the PrediMem model, which relies on a complex dual-system architecture, the robot motion prediction model obtained in this application, while maintaining a lightweight single-system architecture, achieves average performance very close to that of the PrediMem model, and even slightly surpasses it in task completion rate. In terms of the number of parameters, the total number of parameters of the robot motion prediction model obtained by the embodiments of this application is only 3.3B, of which the progress prediction model based on the TTT mechanism introduced in the third stage of training only adds 26.8M additional parameters, achieving an average performance close to that of the PrediMem dual system.

[0113] The experimental data above strongly confirms that the three-stage training process proposed in the embodiments of this application not only significantly enhances the semantic understanding and long-term execution capability of the visual-language-action model based on the reactive action generation paradigm for the task stage, but also achieves efficient transformation of the existing visual-language-action model based on the reactive action generation paradigm with extremely low computational and parameter overhead, providing a lightweight and highly competitive single-system solution for long-range robot operation.

[0114] Figure 9 A block diagram of an exemplary robot control device is shown. This device can be applied to... Figure 1 The implementation environment shown is configured in... Figure 1 Specifically, the device is configured on the controller built into the robot 120 in the illustrated implementation environment. This device can also be applied to other implementation environments and configured on the controllers built into robots in other implementation environments to achieve long-range operation control of the robot; the embodiments of this application do not limit this.

[0115] like Figure 9 As shown, in an exemplary embodiment, the robot control device includes an acquisition module 910 and an iteration module 920.

[0116] The acquisition module 910 is configured to acquire long-range task instructions for instructing long-range control of the robot; the iteration module 920 is configured to iteratively execute the robot control process for each round based on the long-range task instructions until the long-range control of the robot ends.

[0117] The robot control process executed by iteration module 920 in each round includes the following steps: Based on the robot's observation image at the current moment and the corresponding task progress prompt text at the current moment, the robot's task progress at future moments is predicted, and the task progress prompt text at future moments is obtained based on the task progress. The future moments are determined by the current moment and the preset number of future time steps. Based on the observed image and long-range task instructions at the current moment, as well as the corresponding task progress prompt text at the current moment, predict the robot's action sequence within the next number of time steps; Control the robot to execute a sequence of actions within a future time step, and trigger the next round when the future moment arrives.

[0118] In another exemplary embodiment, based on the aforementioned scheme, a robot motion prediction model is deployed in the robot's controller. The robot motion prediction model includes a vision-language-action sub-model and a progress prediction module. The vision-language-action sub-model includes a vision-language encoding module and a motion generation module. In the iteration module 920, the vision-language encoding module extracts first vision-language joint features from the observed image at the current moment and the corresponding task progress prompt text. The progress prediction module then predicts the robot's task progress at future moments based on the first vision-language joint features. Furthermore, the vision-language encoding module extracts second vision-language joint features from the observed image at the current moment, the long-range task instruction, and the corresponding task progress prompt text. The motion generation module then predicts the robot's motion sequence within future time steps based on the second vision-language joint features.

[0119] In another exemplary embodiment, based on the foregoing scheme, the device further includes a training module configured to perform the following steps: training an initial visual-language-action sub-model to enable the visual-language-action sub-model to learn sub-task-level action prediction, thereby obtaining a trained visual-language-action sub-model; using the trained visual-language-action sub-model as a teacher policy model; initializing a student policy model based on the trained visual-language-action sub-model; training the student policy model through the teacher policy model to enable the student policy model to learn long-term task-level action prediction, thereby obtaining a trained visual-language-action sub-model.

[0120] In another exemplary embodiment, based on the foregoing scheme, the training module is further configured to perform the following steps: acquiring teacher model input signals and student model input signals, wherein the teacher model input signals include observation image samples and subtask instruction samples, and the student model input signals include observation image samples, long-term task instruction samples, and task progress prompt samples; inputting the teacher model input signals into the teacher policy model to obtain the first visual-language joint sample features output by the visual-language encoding module contained in the teacher policy model; inputting the student model input signals into the student policy model to obtain the second visual-language joint sample features output by the visual-language encoding module contained in the student policy model, and the action prediction result output by the student policy model; obtaining the training loss value based on the first visual-language joint sample features, the second visual-language joint sample features, and the action prediction result output by the student policy model, and optimizing the parameters of the student policy model based on the training loss value.

[0121] In another exemplary embodiment, based on the foregoing scheme, the training module is further configured to perform the following steps: extracting joint visual-language sample features from the observed image samples and task progress prompt samples using the visual-language encoding module included in the trained visual-language-action sub-model; inputting the extracted joint visual-language sample features into the initial progress prediction module to obtain the task progress prediction result output by the progress prediction module; calculating the training loss value based on the task progress prediction result and the corresponding sample progress label, and optimizing the parameters of the progress prediction module based on the training loss value to obtain the trained progress prediction module.

[0122] It should be noted that the apparatus and method provided in the above embodiments belong to the same concept, and the specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the robot control device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0123] Embodiments of this application also provide an electronic device, including: one or more processors; and a memory for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the robot control methods provided in the above embodiments.

[0124] Figure 10 A schematic diagram of a computer system suitable for implementing an electronic device according to embodiments of this application is shown. It should be noted that the electronic device can be... Figure 1 The robot 120 in the illustrated implementation environment can also be a terminal or server in other implementation environments; no restrictions are imposed here. It should also be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0125] like Figure 10As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on a computer program stored in Read-Only Memory (ROM) 1002 or a computer program loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. Various computer programs and data required for system operation are also stored in RAM 1003. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004.

[0126] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0127] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0128] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0131] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor of an electronic device, implements the robot control method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0132] Another aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the robot control methods provided in the various embodiments described above.

[0133] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A robot control method, characterized in that, The robot's controller is equipped with a robot motion prediction model, which includes a vision-language-action sub-model and a progress prediction module. The vision-language-action sub-model includes a vision-language encoding module and a motion generation module. The method includes: Obtain long-range task instructions for instructing long-range control of the robot; Based on the long-range task instructions, the robot control process is iteratively executed in each round until the long-range control of the robot ends. The robot control process for each round includes the following steps: The visual-language encoding module extracts first visual-language joint features from the robot's observed image at the current moment and the corresponding task progress prompt text at the current moment. The progress prediction module predicts the robot's task progress at future moments based on the first visual-language joint features and obtains the corresponding task progress prompt text at the future moments based on the task progress. The future moments are determined by the current moment and a preset number of future time steps. The visual-language encoding module extracts second visual-language joint features from the observed image at the current moment, the long-range task instructions, and the task progress prompt text corresponding to the current moment. The action generation module then predicts the robot's action sequence within the future time steps based on the second visual-language joint features. The robot is controlled to execute the action sequence within the future time steps, and when the future time arrives, it is triggered to enter the next round; The training process of the vision-language-action sub-model includes the following steps: Based on training the initial visual-language-action sub-model, the visual-language-action sub-model learns sub-task-level action prediction, thus obtaining the trained visual-language-action sub-model. The trained visual-language-action sub-model is used as the teacher policy model, and the student policy model is initialized based on the trained visual-language-action sub-model. The student policy model is trained through the teacher policy model to enable the student policy model to learn long-term task-level action prediction, thereby obtaining the trained visual-language-action sub-model.

2. The method according to claim 1, characterized in that, The process by which the progress prediction module predicts the robot's task progress at future moments based on the first visual language joint features includes the following steps: Write the first visual language joint features into the fast weights; Based on the query token and multiple stacked progress prediction blocks, information related to task stage judgment is aggregated from the fast weights to obtain progress query features; The progress query features are mapped to scalars, and the scalars are mapped to continuous progress prediction values ​​within the target value range to obtain the robot's task progress at future times.

3. The method according to claim 2, characterized in that, The step of writing the first visual language joint features into the fast weights includes: The visual-language joint features obtained by the progress prediction module during the task progress prediction process are saved as a historical feature sequence. The fast weights are obtained by transforming the historical feature sequence.

4. The method according to claim 1, characterized in that, In the process of the progress prediction module predicting the robot's task progress at future moments based on the first visual language joint features, the task progress is constrained to advance only one task stage at a time.

5. The method according to claim 1, characterized in that, The step of training the student strategy model using the teacher strategy model to enable the student strategy model to learn long-term task-level action prediction includes: Acquire the input signals of the teacher strategy model and the student strategy model. The input signals of the teacher strategy model include observation image samples and subtask instruction samples. The input signals of the student strategy model include the observation image samples, long-term task instruction samples, and task progress prompt samples. The input signal of the teacher strategy model is input into the teacher strategy model to obtain the first visual-language joint sample features output by the visual-language encoding module contained in the teacher strategy model; The input signal of the student strategy model is input into the student strategy model to obtain the second visual-language joint sample features output by the visual-language encoding module contained in the student strategy model, and the action prediction result output by the student strategy model. Based on the joint visual language sample features, the joint visual language sample features, and the action prediction results output by the student policy model, a training loss value is obtained, and the parameters of the student policy model are optimized based on the training loss value.

6. The method according to claim 5, characterized in that, The step of obtaining the training loss value based on the action prediction results output by the first visual language joint sample features, the second visual language joint sample features, and the student policy model includes: The feature alignment loss value is calculated based on the joint sample features of the first visual language and the joint sample features of the second visual language, and the action supervision loss value is calculated based on the action prediction result output by the student policy model and the corresponding sample action label. The training loss value is calculated based on the feature alignment loss value and the action supervision loss value.

7. The method according to claim 6, characterized in that, The calculation of the feature alignment loss value based on the first visual language joint sample features and the second visual language joint sample features includes: Based on a learnable query token, a first key feature is extracted from the first visual language joint sample features, and a second key feature is extracted from the second visual language joint sample features, wherein the first key feature and the second key feature refer to feature information related to action prediction. The feature alignment loss value is calculated based on the first key feature and the second key feature.

8. The method according to claim 7, characterized in that, The step of calculating the feature alignment loss value based on the first key feature and the second key feature includes: Calculate the distance similarity and cosine similarity between the first key feature and the second key feature, respectively; The feature alignment loss value is calculated based on the distance similarity and the cosine similarity.

9. The method according to claim 1, characterized in that, The training process of the progress prediction module includes the following steps: By using the visual-language encoding module included in the trained visual-language-action sub-model, visual-language joint sample features are extracted from observed image samples and task progress prompt samples; The extracted visual language joint sample features are input into the initial progress prediction module to obtain the task progress prediction result output by the progress prediction module. The training loss value is calculated based on the task progress prediction result and the corresponding sample progress label, and the parameters of the progress prediction module are optimized based on the training loss value to obtain the trained progress prediction module.

10. The method according to claim 9, characterized in that, The calculation of the training loss value based on the task progress prediction result and the corresponding sample progress label includes: Calculate the mean squared error between the predicted task progress and the actual progress label corresponding to the training samples; Obtain the sample weights corresponding to the training samples, wherein the sample weights of the training samples representing the stage of stagnation are less than the sample weights of the training samples representing the stage of advancement. The training loss value is calculated based on the mean squared error corresponding to each training sample and the sample weight corresponding to each training sample.

11. A robot control device, characterized in that, The robot's controller is equipped with a robot motion prediction model, which includes a vision-language-action sub-model and a progress prediction module. The vision-language-action sub-model includes a vision-language encoding module and a motion generation module. The device includes: The acquisition module is configured to acquire long-range task instructions for instructing long-range control of the robot; The iteration module is configured to iteratively execute the robot control process for each round based on the long-range task instructions until the long-range control of the robot ends. The robot control process for each round includes the following steps: The visual-language encoding module extracts first visual-language joint features from the robot's observed image at the current moment and the corresponding task progress prompt text at the current moment. The progress prediction module predicts the robot's task progress at future moments based on the first visual-language joint features and obtains the corresponding task progress prompt text at the future moments based on the task progress. The future moments are determined by the current moment and a preset number of future time steps. The visual-language encoding module extracts second visual-language joint features from the observed image at the current moment, the long-range task instructions, and the task progress prompt text corresponding to the current moment. The action generation module then predicts the robot's action sequence within the future time steps based on the second visual-language joint features. The robot is controlled to execute the action sequence within the future time steps, and when the future time arrives, it is triggered to enter the next round; The training process of the vision-language-action sub-model includes the following steps: Based on training the initial visual-language-action sub-model, the visual-language-action sub-model learns sub-task-level action prediction, thus obtaining the trained visual-language-action sub-model. The trained visual-language-action sub-model is used as the teacher policy model, and the student policy model is initialized based on the trained visual-language-action sub-model. The student policy model is trained through the teacher policy model to enable the student policy model to learn long-term task-level action prediction, thereby obtaining the trained visual-language-action sub-model.

12. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by one or more processors, cause the electronic device to perform the method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the processor of the electronic device, causes the electronic device to perform the method of any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor of the electronic device, it implements the method as described in any one of claims 1-10.

Images