Robot circulation task control system and method based on effective historical perception and understanding

Abstract

The invention discloses a robot cyclic task control system and method based on effective history perception and understanding, and the system comprises a CycleManip framework, and the core comprises an effective history perception module, an effective history understanding module, a feature fusion and task output module and a CycleManip reference module. The effective history sensing module balances the time view and the calculation cost through high and low overhead observation data differential sampling; the effective history understanding module adopts multi-task learning, and a task progress prediction auxiliary task is added so as to learn loop stage discrimination features; and the CycleManip reference module provides a multi-class cyclic task data set and an automatic evaluation system. According to the method, the success rate and accuracy of the cycle task are greatly improved, low overhead, high generalization and multi-platform adaptability are achieved, the standard blank in the cycle task control field is filled, and the method can be widely applied to the scenes of family service, industrial manufacturing and the like.

Description

Technical Field

[0001] This invention belongs to the technical field of machine learning, specifically relating to a robot cyclic task control system and method based on effective historical perception and understanding. Background Technology

[0002] With the continuous development of robotics and computer vision, enabling robots to autonomously handle various tasks in daily life has become a long-term goal pursued by the industry. In practical application scenarios such as home services and industrial production, many tasks involve repetitive and periodic actions, such as shaking a bottle to mix its contents, pressing a syrup pump a preset number of times, or using a hammer to knock nails. These tasks not only require robots to stably execute cyclical actions, but also require them to accurately terminate the action after completing the expected number of cycles based on user instructions or the real-time status of the task.

[0003] Current mainstream robot manipulation methods, including imitation learning (IL) and vision-language-action (VLA) models, have significant advantages in modeling complex data distributions. They can effectively predict subsequent actions based on current observation information and have achieved good application results in sequential task execution.

[0004] However, the aforementioned methods often face execution failures when dealing with repetitive or periodic tasks. The core issue lies in the fact that these strategies typically rely on observation data within a short time window for action prediction. In cyclical tasks, the observation information, especially visual observation data, at different stages of the cycle often exhibits high similarity. For example, in the task of "shaking a bottle five times," the visual input after the first shake is almost identical to that after the third shake. This similarity can cause confusion in models that rely on short-term observations, making it difficult to accurately track the number of completed cycles. Consequently, it becomes difficult to decide whether to continue the action or terminate, ultimately causing the robot to get stuck in an infinite loop or prematurely terminate the task.

[0005] To address this issue, existing technologies extend the model's observation history window, enabling the model to acquire earlier observation information. However, this approach has significant cost drawbacks: encoding and fusing high-dimensional visual observation history at each time step incurs enormous computational overhead and inference latency, failing to meet the application requirements of real-time robot control.

[0006] Therefore, how to enable robots to effectively utilize historical observation information and accurately track the number of cycles and complete cyclic tasks without significantly increasing computational overhead has become a technical challenge that urgently needs to be overcome in the field of robot control. Summary of the Invention

[0007] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a robot cyclic task control system and method based on effective historical perception and understanding. Through the designed CycleManip cyclic task control framework, without significantly increasing computational overhead, the end-to-end imitation learning model can accurately execute cyclic actions a specified number of times by utilizing the two core strategies of "effective historical perception" and "effective historical understanding".

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a robot cyclic task control system based on effective historical perception and understanding, including an effective historical perception module, an effective historical understanding module, a feature fusion and task output module, and a CycleManip benchmark module;

[0010] The effective history perception module is used to classify and sample robot observation data, and the effective history understanding module is used to achieve in-depth interpretation of historical information through multi-task learning.

[0011] The feature fusion and task output module is used to fuse encoded features and output action prediction results and task progress prediction results.

[0012] The CycleManip benchmark module provides a dataset for recurring tasks and an automatic evaluation mechanism.

[0013] As a preferred technical solution, the effective historical perception module is implemented in the following ways:

[0014] Robot observation data is divided into high-overhead observation data and low-overhead observation data. The high-overhead observation data is RGB images or point clouds, and the low-overhead observation data is robot proprioception. A dense sampling strategy covering the entire history is adopted for low-overhead observation data, while a sparse and heuristic sampling strategy is adopted for high-overhead observation data. The low-overhead observation data is represented by the temporal pose difference of the end effector as the core input.

[0015] As a preferred technical solution, the effective history understanding module is implemented in the following ways:

[0016] A multi-task learning framework is constructed, comprising a main task and an auxiliary task. The main task is to predict the robot expert's actions, and the auxiliary task is to predict the current task progress stage. The task progress stage is represented by the number of loops or the percentage of task completion, and its true value is calculated by the ratio of the current frame number to the total number of task frames. The multi-task learning framework is trained using a combined loss function, which is a weighted sum of action prediction loss and progress prediction loss.

[0017] As a preferred technical solution, the implementation process of the feature fusion and task output module includes:

[0018] User language commands are encoded using a CLIP encoder, high-overhead observation data is encoded using a point cloud encoder, and low-overhead historical observation data is encoded using a Transformer encoder. The encoded language features and observation features are fused using an MLP network. The fused features are then input into a diffusion model and an MLP network, respectively. The diffusion model is used to predict the robot's actions in the next n steps, and the MLP network is used to output the task progress prediction results.

[0019] As a preferred technical solution, the CycleManip benchmark module includes:

[0020] The RoboTwin 2.0 platform is used to build a dataset of eight types of cyclic tasks, including hammering, shaking bottles, rolling, chopping vegetables, double-knife cutting, beating eggs, chemical mixing, and Morse code. An automatic cycle evaluation system is also included. This system calculates the number of cycles performed by the robot by tracking the pose and contact state of objects in the simulation. For contact tasks, a state machine-based collision detection counting algorithm is used, while for non-contact tasks, a peak detection algorithm based on the object's pose trajectory is used.

[0021] Secondly, the present invention provides a method for controlling a robot in a cyclical task based on effective historical perception and understanding, comprising the following steps:

[0022] S1. Acquire user language commands and robot observation data, and classify and sample the observation data through an effective historical perception strategy to obtain historical observation data that expands the time horizon and controls computational costs.

[0023] S2. Encode user language commands and historical observation data separately, and obtain a unified feature representation through feature fusion;

[0024] S3. The unified feature representation is processed based on a multi-task learning framework, and the robot action prediction result and task progress prediction result are output simultaneously. The robot performs a cyclic task based on the action prediction result.

[0025] S4. The automatic evaluation system of the CycleManip benchmark module evaluates the results of the robot's cyclic task execution and determines whether the number of cycles is consistent with the user's instructions.

[0026] As a preferred technical solution, the specific implementation of the effective historical perception strategy in step S1 includes:

[0027] The observation data is divided into high-overhead observation data and low-overhead observation data. All historical data are sampled for low-overhead observation data, while heuristic sparse sampling with a fixed number of frames is used for high-overhead observation data. The low-overhead observation data is converted into temporal pose difference data of the end effector to eliminate absolute position deviation.

[0028] As a preferred technical solution, the training process of the multi-task learning framework in step S3 includes:

[0029] Using expert action ground truth as the supervision signal, an action prediction loss function is constructed; using task progress ground truth as the supervision signal, a progress prediction loss function is constructed; the action prediction loss function and the progress prediction loss function are weighted and combined to form a combined loss function, and the model is trained end-to-end through the combined loss function.

[0030] As a preferred technical solution, the evaluation logic of the automatic evaluation system in step S4 is as follows:

[0031] If the robot completes the specified operation and the automatically counted number of loops matches the number of loops in the user's instruction, the task is considered successful; otherwise, the task is considered to have failed.

[0032] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the robot cyclic task control method.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] (1) Low overhead and high efficiency in perceiving historical information: Innovative “cost-aware sampling strategy” adopts differentiated sampling methods for observation data with different overhead, which not only expands the model’s time horizon to track the progress of the cycle, but also avoids the huge computation and delay caused by high-dimensional data encoding and fusion, meets the needs of real-time robot control, and overcomes the traditional dilemma of “expanding history increases costs”.

[0035] (2) Deep understanding of the progress of cyclic tasks: By adding a task progress prediction auxiliary task through a multi-task learning framework, the model is forced to learn to distinguish the discriminative features of different cyclic stages under similar visual observations, which solves the core problem of traditional imitation learning that "perceives history but cannot understand progress", avoids the robot from getting into an infinite loop or terminating prematurely, and greatly improves the accuracy and success rate of cyclic task execution.

[0036] (3) Modular design combines versatility and adaptability: The effective historical perception and historical understanding modules have the characteristics of "plug and play" and can be integrated into mainstream imitation learning / vision-language-action models. They can improve the performance of cyclic tasks without reconstructing the overall framework. At the same time, they can be adapted to various heterogeneous hardware platforms such as single-arm grippers, dual-arm grippers, dexterous hands, and humanoid robots, reducing the threshold for cross-scenario and cross-device applications.

[0037] (4) Building industry benchmarks to promote field development: Through the CycleManip benchmark and automatic evaluation system that includes 8 types of tasks, the number of loops can be accurately and automatically counted, filling the gap in the field of loop task manipulation where there is a lack of standardized datasets and evaluation tools, and providing a unified development and verification platform for subsequent research.

[0038] (5) Generalization capability covers multiple task scenarios: It not only performs well in recurring tasks, but also maintains good performance in general non-recurring control tasks, breaking the limitation that "specialized optimization is only suitable for a single task", and can be widely used in many fields such as home service, industrial manufacturing, scientific research experiments. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a robot cyclic task control system based on effective historical perception and understanding, according to an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram illustrating the effect of deploying the present invention on a real machine;

[0042] Figure 3 This is a schematic diagram of diverse cyclical tasks in an embodiment of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0044] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0045] Definitions of abbreviations and key terms

[0046] CycleManip: The name of the robot cyclic task control framework proposed in this invention. This framework enables the robot to accurately control and terminate cyclic tasks after receiving natural language instructions by performing steps such as historical data sampling, cross-modal feature encoding, multi-task learning and reasoning, and diffusion-based action generation.

[0047] Cycle-based manipulation: The core problem this invention aims to solve is the ability of a robot to execute periodic or repetitive operations based on natural language instructions, and to autonomously terminate the action after reaching a specified number of cycles. This invention focuses on solving problems such as the robot's identification of cycle phases, perception of historical progress, and determination of termination timing.

[0048] Effective Historical Perception: One of the core strategies of this invention refers to a "cost-aware sampling strategy" that classifies observation streams according to overhead type. By using sparse long-term sampling for high-overhead observations and dense full-history sampling for low-overhead observations, the strategy model can obtain sufficiently long task history information while keeping computational resources under control.

[0049] Effective Historical Understanding: One of the core strategies of this invention refers to the use of a "multi-task learning" strategy to enable the strategy model not only to "see" history but also to "understand" history. The "multi-task learning" strategy of this invention introduces a task progress prediction auxiliary task in addition to the main task of action prediction, so that the model can automatically learn discriminative features that distinguish different cycle stages when encoding historical information, thereby improving the robot's ability to understand the cycle progress.

[0050] Cost-aware sampling strategy: A specific method employed to achieve effective historical awareness. This strategy categorizes observation data into high-overhead (e.g., RGB images, point clouds) and low-overhead (e.g., robot proprioception, end-effector pose). Sparse sampling is used for high-overhead data, while dense, extensive sampling is used for low-overhead data.

[0051] like Figure 1 As shown, this embodiment provides a robot cyclic task control system based on effective historical perception and understanding. The system includes an effective historical perception module, an effective historical understanding module, a feature fusion and task output module, and a CycleManip benchmark module. The modules work together to complete the precise control and evaluation of the robot's cyclic tasks.

[0052] Furthermore, the effective historical perception module is used to classify and sample robot observation data, with the core objective of resolving the "cost-viewpoint" contradiction. Robot observation data is divided into high-overhead and low-overhead observation data, where high-overhead observation data consists of RGB images or point clouds. For high-overhead observation data, this invention employs a sparse and heuristic sampling strategy to control computational costs; low-overhead observation data consists of robot proprioception (such as joint angles, torques, and end effector poses). For low-overhead observation data, this invention employs a dense sampling strategy that covers the entire history, ensuring no key historical information is missed. Simultaneously, low-overhead observation data uses the temporal pose differences of the end effector as the core input representation, eliminating the influence of absolute position deviations on model judgment.

[0053] Furthermore, this effective history awareness module employs a "cost-aware sampling strategy," specifically:

[0054] Observation Classification: First, the robot's observation data is divided into two categories: high-overhead observations ( ): Refers to high-dimensional, expensive-to-encode data, such as RGB images or point clouds. Low-overhead observations ( ): refers to low-dimensional, inexpensive data, such as a robot's proprioception (joint angles, end effector pose, etc.).

[0055] Differential sampling: for low-overhead observations ( ): Employ a dense and extensive sampling strategy Due to its low coding cost, this invention can include all low-overhead observations from the start of the task to the current time; for high-overhead observations (… ): Employs a sparse and heuristic sampling strategy. For example, by adopting a sparse sampling strategy with a fixed upper limit on the number of frames, and by performing binary time segment extraction and exponential interval extraction on historical frames, the selected frames can cover a longer historical time range, while avoiding the computational burden caused by encoding a large amount of high-dimensional visual data.

[0056] Specific representation of low-overhead observations: To enable low-overhead observations to compactly represent the cyclic process, this invention preferably uses the pose difference of the end effector as input. Compared to joint positions, the motion pattern of the end effector more directly reflects the cyclic characteristics, and using the difference values ​​helps to eliminate the bias caused by absolute positions, allowing the model to focus more on the cyclic task itself.

[0057] Through this strategy, the strategy model The input is This both expands the time horizon and controls computational costs.

[0058] Furthermore, the effective historical understanding module is used to achieve deep interpretation of historical information through multi-task learning, solving the problem of the disconnect between "action" and "progress". A multi-task learning framework is constructed, which includes a main task and auxiliary tasks. The main task is to predict the robot expert's actions, and the auxiliary task is to predict the task progress stage at the current moment. The task progress stage is represented by the number of loops or the percentage of task completion. If predicting the number of loops, the true value is provided by the dataset; if predicting the percentage of completion, the true value is calculated by the ratio of the current frame number to the total number of frames in the task. A combined loss function is used to train the multi-task learning framework. The combined loss function is a weighted sum of the action prediction loss and the progress prediction loss. The model is forced to learn discriminative features that distinguish the loop stages through the supervision signals of the auxiliary tasks.

[0059] Furthermore, the effective historical understanding module employs a "multi-task learning strategy," specifically:

[0060] Introducing auxiliary tasks: In standard imitation learning (predicting expert actions) In addition to the main task, an auxiliary task is introduced.

[0061] Predicting Task Progress: The goal of this auxiliary task is to predict the current stage of progress within the entire task (e.g., the current loop count or task completion percentage). In a specific implementation, progress is characterized by the percentage of task completion; the true value of this progress... This can be obtained by (current frame rate / total frame rate of the task).

[0062] Learning discriminative features: the supervision signal for this auxiliary task This changes over time (i.e., cyclical progression), forcing the model to learn features from history that distinguish different cyclical phases. For example, the model must differentiate between the first hammer blow (progression). ) and the third hammer blow (progress) Even if their visual observations and target actions are similar, this solves the problem of fuzzy supervision signals in traditional imitation learning.

[0063] Furthermore, the feature fusion and task output module is used to fuse encoded features and output action prediction results and task progress prediction results. User language commands are encoded using a CLIP encoder to obtain language features; high-overhead observation data is encoded using a point cloud encoder to obtain visual features; low-overhead historical observation data is encoded using a Transformer encoder to obtain historical temporal features; the encoded language features, visual features, and historical temporal features are fused using an MLP network to obtain a unified feature representation; the fused features are then input into a diffusion model and an MLP network, respectively. The diffusion model is used to predict the robot's actions in the next n steps, and the MLP network is used to output task progress prediction results (such as the current loop count and task completion percentage).

[0064] For further details, please refer to [link / reference]. Figure 3 The specific content of feature fusion and task output is as follows:

[0065] Input: User language instructions (e.g., “Hit the black object 8 times with a hammer”) and observation history after “cost-aware sampling” (Module 1).

[0066] Encoding: Language instructions are encoded using encoders such as CLIP, high-overhead observations are encoded using point cloud encoders, and low-overhead (dense) historical observations are encoded using Transformer encoders.

[0067] Feature fusion: fusing encoded linguistic features and observational features (through MLP).

[0068] Multi-task output: The fused features are used in two parallel heads: Main task (action prediction): As conditional input to a diffusion model, it predicts the robot's actions for the next n steps. Auxiliary task (progress prediction): Input is given to an MLP to predict the current task progress. .

[0069] Loss function: A combined loss function is used. The network is trained end-to-end, and the function is a weighted sum of the action prediction loss (MSE) and the progress prediction loss (CE), expressed as follows:

[0070] ,

[0071] in and These are the expert action truth value and the task progress truth value, respectively.

[0072] Furthermore, the CycleManip benchmark module provides a controllable collection of cyclic task data and a high-confidence automatic strategy evaluation mechanism, filling a gap in the field of benchmarks. This module includes eight types of cyclic task environments built on the RoboTwin 2.0 platform. These cyclic tasks include hammering, shaking, rolling, chopping, double-knife cutting, egg beating, chemical mixing, and Morse code. Using the environment designed in this invention, users can generate any number of data points with any number of cycles through configuration files. To provide the benchmark module, this invention has collected 200 high-quality expert demonstration data points for each of the eight tasks, with the number of cycles for each task evenly distributed between 1 and 8. The benchmark module also includes an automatic cycle evaluation system, which calculates the number of cycles performed by the robot by tracking the object pose and contact state in the simulation. For contact-based tasks (such as hammering), a state machine-based collision detection counting algorithm is used; for non-contact tasks (such as shaking a bottle), a peak detection algorithm based on object pose trajectory is used. The automatic evaluation mechanism provided by this invention, after manual testing, achieved an accuracy rate of over 97% in 100 tests, demonstrating high confidence.

[0073] The method of this invention significantly outperforms existing state-of-the-art (SOTA) baseline methods (such as DP3, RDT, and Pi-0) in both success rate (Suc.) and cycle count deviation (Cyc.) across all eight cyclic tasks. For example, it achieves a 95% success rate on the shake-bottle task. Simultaneously, the average cycle deviation (Cyc.) remains consistently below 1.0, far less than the baseline methods, demonstrating that this invention accurately executes the required number of instruction loops. The effectiveness of this invention when deployed on a real machine is as follows: Figure 2 As shown.

[0074] Another embodiment of the present invention provides a robot cyclic task control method based on effective historical perception and understanding. This method is implemented based on the above-described system and includes the following steps:

[0075] S1: Acquire user language commands and robot observation data, classify and sample the observation data through an effective historical perception strategy, and obtain historical observation data that expands the time horizon while controlling computational costs.

[0076] Specifically, this includes: dividing the observation data into high-overhead and low-overhead observation data; sampling all historical data for low-overhead observation data; and using heuristic sparse sampling with a fixed number of frames for high-overhead observation data; and converting low-overhead observation data into temporal pose difference data of the end effector to eliminate absolute position deviation.

[0077] S2: Encode user language commands and historical observation data separately, and obtain a unified feature representation through feature fusion.

[0078] Specifically, this involves encoding language instructions using a CLIP encoder, high-overhead observation data using a point cloud encoder, and low-overhead historical observation data using a Transformer encoder, and then fusing the three types of encoded features using an MLP network.

[0079] S3: Based on a multi-task learning framework, the unified feature representation is processed, and the robot action prediction result and task progress prediction result are output simultaneously. The robot executes cyclical tasks based on the action prediction result.

[0080] The training process of the multi-task learning framework includes: constructing an action prediction loss function using expert action ground truth as the supervision signal; constructing a progress prediction loss function using task progress ground truth (calculated by the ratio of the current frame number to the total number of frames in the task) as the supervision signal; weighting and combining the action prediction loss function and the progress prediction loss function to form a combined loss function, and completing end-to-end training of the model through the combined loss function.

[0081] S4: The automatic evaluation system of the CycleManip benchmark module evaluates the results of the robot's cyclic task execution and determines whether the number of cycles is consistent with the user's instructions.

[0082] The evaluation logic is as follows: if the robot successfully completes the specified operation and the number of loops counted automatically matches the number of loops in the user's instruction, the task is considered successful; if none of the above conditions are met, the task is considered to have failed.

[0083] In one embodiment, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, the computer program being executed by a processor as follows:

[0084] Acquire user language commands and robot observation data, classify and sample the observation data through an effective historical perception strategy, and obtain historical observation data that expands the time horizon while controlling computational costs;

[0085] User language commands and historical observation data are encoded separately, and a unified feature representation is obtained through feature fusion.

[0086] The unified feature representation is processed based on a multi-task learning framework, and robot action prediction results and task progress prediction results are output simultaneously. The robot performs cyclic tasks based on the action prediction results.

[0087] The automatic evaluation system of the CycleManip benchmark module evaluates the results of the robot's cyclic task execution to determine whether the number of cycles matches the user's instructions.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A robot cyclic task control system based on effective historical perception and understanding, characterized in that, It includes an effective history perception module, an effective history understanding module, a feature fusion and task output module, and a CycleManip benchmark module; The effective history perception module is used to classify and sample robot observation data, and the effective history understanding module is used to achieve in-depth interpretation of historical information through multi-task learning. The feature fusion and task output module is used to fuse encoded features and output action prediction results and task progress prediction results. The CycleManip benchmark module provides a dataset for recurring tasks and an automatic evaluation mechanism.

2. The robot cyclic task control system according to claim 1, characterized in that, The effective history perception module is implemented in the following ways: Robot observation data is divided into high-overhead observation data and low-overhead observation data. The high-overhead observation data is RGB images or point clouds, and the low-overhead observation data is robot proprioception. A dense sampling strategy covering the entire history is adopted for low-overhead observation data, while a sparse and heuristic sampling strategy is adopted for high-overhead observation data. The low-overhead observation data is represented by the temporal pose difference of the end effector as the core input.

3. The robot cyclic task control system according to claim 1, characterized in that, The effective history understanding module is implemented in the following ways: A multi-task learning framework is constructed, comprising a main task and an auxiliary task. The main task is to predict the robot expert's actions, and the auxiliary task is to predict the current task progress stage. The task progress stage is represented by the number of loops or the percentage of task completion, and its true value is calculated by the ratio of the current frame number to the total number of task frames. The multi-task learning framework is trained using a combined loss function, which is a weighted sum of action prediction loss and progress prediction loss.

4. The robot cyclic task control system according to claim 1, characterized in that, The implementation process of the feature fusion and task output module includes: User language commands are encoded using a CLIP encoder, high-overhead observation data is encoded using a point cloud encoder, and low-overhead historical observation data is encoded using a Transformer encoder. The encoded language features and observation features are fused using an MLP network. The fused features are then input into a diffusion model and an MLP network, respectively. The diffusion model is used to predict the robot's actions in the next n steps, and the MLP network is used to output the task progress prediction results.

5. The robot cyclic task control system according to claim 1, characterized in that, The CycleManip benchmark module includes: The RoboTwin 2.0 platform is used to build a dataset of eight types of cyclic tasks, including hammering, shaking bottles, rolling, chopping vegetables, double-knife cutting, beating eggs, chemical mixing, and Morse code. An automatic cycle evaluation system is also included. This system calculates the number of cycles performed by the robot by tracking the pose and contact state of objects in the simulation. For contact tasks, a state machine-based collision detection counting algorithm is used, while for non-contact tasks, a peak detection algorithm based on the object's pose trajectory is used.

6. A method for controlling a robot in a cyclical task based on effective historical perception and understanding, characterized in that, Includes the following steps: S1. Acquire user language commands and robot observation data, and classify and sample the observation data through an effective historical perception strategy to obtain historical observation data that expands the time horizon and controls computational costs. S2. Encode user language commands and historical observation data separately, and obtain a unified feature representation through feature fusion; S3. The unified feature representation is processed based on a multi-task learning framework, and the robot action prediction result and task progress prediction result are output simultaneously. The robot performs a cyclic task based on the action prediction result. S4. The automatic evaluation system of the CycleManip benchmark module evaluates the results of the robot's cyclic task execution and determines whether the number of cycles is consistent with the user's instructions.

7. The robot cyclic task control method according to claim 6, characterized in that, The specific implementation of the effective history perception strategy in step S1 includes: The observation data is divided into high-overhead observation data and low-overhead observation data. All historical data are sampled for low-overhead observation data, while heuristic sparse sampling with a fixed number of frames is used for high-overhead observation data. The low-overhead observation data is converted into temporal pose difference data of the end effector to eliminate absolute position deviation.

8. The robot cyclic task control method according to claim 6, characterized in that, The training process of the multi-task learning framework described in step S3 includes: Using expert action ground truth as the supervision signal, an action prediction loss function is constructed; using task progress ground truth as the supervision signal, a progress prediction loss function is constructed; the action prediction loss function and the progress prediction loss function are weighted and combined to form a combined loss function, and the model is trained end-to-end through the combined loss function.

9. The robot cyclic task control method according to claim 6, characterized in that, The evaluation logic of the automatic evaluation system in step S4 is as follows: If the robot completes the specified operation and the automatically counted number of loops matches the number of loops in the user's instruction, the task is considered successful; otherwise, the task is considered to have failed.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the robot cyclic task control method according to any one of claims 6-9.

Images