Generalized robot reward generation method based on vision-language model

Abstract

The invention discloses a generalizable robot reward generation method based on a vision-language model, and the method comprises the steps: accessing a human data set containing videos and text tags for human to execute different tasks, and pre-training a video-language model through video-language comparative learning; collecting data of successful task execution and data of unsuccessful task execution of the robot to form a robot data set; and inputting the failed video into a visual encoder to extract visual features of the failed video. A corresponding failure prompt is further generated; robot task data and human task data are aligned through cross-domain comparative learning, video-language comparative learning is improved, and videos and task descriptions are aligned; and training the video-language model, taking the trained model as a robot reward function, and combining the reward function with a reinforcement learning method to generate a task execution reward value. By means of the method and device, the success rate of the robot executing wide tasks in diversified environments can be increased.

Description

Technical Field

[0001] The present invention belongs to the field of machine control and reinforcement learning, and in particular relates to a generalizable robot reward generation method based on a vision-language model. Background Art

[0002] In recent years, the field of general robotics research has grown rapidly, with the goal of enabling robots to perform a wide range of tasks based on natural language instructions. This emerging field has been driven by breakthroughs in multimodal models that have demonstrated superior capabilities in vision-language tasks, and their success stems from training on rich datasets covering a wide range of human data. However, collecting data on robots executing instructions from a variety of languages ​​in multiple environments is more complex and labor-intensive than collecting computer vision and natural language processing datasets.

[0003] A promising solution is to develop a general reward function that can evaluate the success or failure of a robot's behavior. Some traditional methods, such as inverse reinforcement learning proposed in the 2004 ICML conference paper "Apprenticeship learning via inverse reinforcement learning", can usually only generate reward functions specifically for a limited range of tasks.

[0004] In the present invention, an attempt is made to convert the pre-trained visual-language model into a reward function. Such models have a wide range of generalization capabilities, but it is not easy to apply them to robot reward learning. First, due to the scarcity of robot interaction data, fine-tuning these models on limited robot datasets may lead to overfitting and catastrophic forgetting of the models, thereby weakening their generalization ability. Secondly, there is a significant domain difference problem from human data to robot applications. In addition, unlike diverse human videos, robot videos usually come from more controlled and static environments, and the mechanical nature of robot movements is very different from the smoothness of human movements. Some previous studies, such as the DVD method proposed in the article "Learning generalizable robotic reward functions from "in-the-wild" human videos." included in the 2021 RSS conference, attempted to determine whether two videos performed the same task by training a discriminator, but these methods ignored the binary classification requirements of success and failure in the reward function and the importance of introducing task failure data.

[0005] Unlike typical robotics datasets used to train reward functions, human video-language datasets usually contain few failure videos. This difference often results in models that perform well on task classification but poorly on distinguishing whether a task was successfully performed.Figure 1 As shown in Figure 2, although the video-language model can effectively distinguish between the "close drawer" task and the "open drawer" task, it may not be able to identify failure videos in the task, such as incomplete closure or accidental reopening. Therefore, training only with successful videos is not enough for the model to accurately identify and classify diverse failure scenarios. Summary of the invention

[0006] The present invention provides a generalizable robot reward generation method based on a vision-language model, which can provide effective rewards for general robots when facing diverse environments and performing a wide range of tasks, thereby improving the success rate of robots in performing a wide range of tasks in diverse environments.

[0007] A generalizable robot reward generation method based on a vision-language model, characterized by comprising the following steps:

[0008] (1) Access to human datasets containing videos and text labels of humans performing different tasks Use video-language contrastive learning to pre-train the video-language model;

[0009] (2) Collect data on the robot's successful and unsuccessful tasks in a virtual environment to form a robot dataset

[0010] (3) Input the failure video data in the robot task into the visual encoder to obtain the corresponding failure visual features; then use the K-means clustering algorithm to perform cluster analysis on the failure visual features of each task to identify and classify different failure modes; generate a specific learnable failure prompt for each failure mode and align it with the corresponding failure video;

[0011] (4) Conduct cross-domain comparative learning between robot success data and human success data to align the features of similar tasks in different domains. At the same time, based on the original video-language comparative learning, improve and introduce failure prompts to distinguish the features of successful and failed task execution videos;

[0012] (5) Training the video-language model. The trained model will be used as the robot reward function. The reward function will be combined with the reinforcement learning method to generate the reward value for task execution, guide the robot to select the optimal action sequence or provide learning signals to improve the robot's success rate in completing the task.

[0013] The present invention first identifies different failure modes by clustering failure videos, and assigns a unique identifier to each cluster to represent the corresponding failure prompt. This strategy enables the model to form a detailed understanding of different failure modes and transfer knowledge through learnable failure prompts, thereby improving initial performance on new tasks. In addition, the present invention aligns text and video representations in the human and robotics domains through cross-domain contrastive learning and domain-specific prompt learning. The reward generation method of the present invention can adapt to different environmental settings and similar new tasks, and effectively distinguish between success and failure of task execution.

[0014] In step (1), the human dataset It contains human demonstration videos of N different tasks is the corresponding task description, that is, the text label.

[0015] Video-language contrastive learning is used to pre-train the video-language model. The specific process is as follows:

[0016] The visual encoder encodes the video of the i-th L frame in the human dataset independently into a batch of images and provides these frame-level features to the temporal visual encoder to obtain the video representation v i ; The text encoder encodes the i-th task description into the corresponding text representation t i ; After that, the video-language contrastive learning pre-training model is used, and the training loss function is for:

[0017]

[0018] In the formula, Indicates that the video is represented by v i Match its corresponding task description text representation t i The loss function based on contrastive learning, Represents the task description text representation t i Match its corresponding video representation v i The loss function based on contrastive learning is , and B represents the number of video-text pairs in each training.

[0019] is defined as follows:

[0020]

[0021] Where τ is the temperature coefficient.

[0022] In step (2), the robot dataset in, They represent the video and the corresponding text instruction executed by the jth robot, respectively, and M represents the number of data in the robot dataset; when the robot successfully completes the text instruction When describing the task, r j =1, otherwise, r j =0.

[0023] In step (3), for each robot task For each task Create a task-specific failure prompt pool It consists of K learnable failure prompts composition:

[0024]

[0025] in, Indicates the task The kth failure prompt in the failure prompt pool;

[0026] Each task corresponds to K failure prompts. In each round of model training, each task gets K text encodings of failure contexts.

[0027] In step (4), when performing cross-domain contrastive learning between robot data and human data, the cross-domain contrastive loss as follows:

[0028]

[0029] In the formula, A cross-domain contrast loss that uses samples from the robotics dataset as anchors;

[0030] A cross-domain contrast loss that uses samples from the human dataset as anchors; is defined as follows:

[0031]

[0032] in, B h represents the number of successful human videos in each training, B r Represents the number of successful videos of the robot in each training.

[0033] In step (4), the failure prompt is improved and introduced to distinguish the features of successful and failed task execution videos, specifically:

[0034] For each video sample in the robot failure dataset, a contrastive learning objective is formed with the corresponding failure prompt, the contrast loss as follows:

[0035]

[0036] Where, the failed robot dataset is Represents the failure video feature and text feature pair of the i-th failure video; Indicates the kth corresponding to the i-th failed video * failure context text encodings, and K represents the number of text encodings of the failure context of the current task.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The failure prompt proposed in the present invention can effectively improve the model's ability to identify and classify failed tasks, so that the robot can accurately distinguish between success and failure when performing tasks.

[0039] 2. Through cross-domain comparative learning and video-language comparative learning, the method of the present invention realizes the effective fusion of robot data and human data, and improves the generalization performance of the model in unknown tasks and environments.

[0040] 3. The reward function generated by the present invention is suitable for reinforcement learning, especially in multi-task and multi-environment robot tasks, which helps to reduce the task failure rate and improve the success rate of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Comparison chart of the human video dataset and the robotics dataset for reward learning.

[0042] Figure 2 This is a model architecture diagram of the present invention.

[0043] Figure 3 Schematic diagram of the MetaWorld experimental environment and part of the Concept2Robot experimental environment in an embodiment of the present invention.

[0044] Figure 4 This is the result of the environmental generalization experiment of the present invention.

[0045] Figure 5 This is the result of the ablation experiment on the failure data of the present invention.

[0046] Figure 6 This is the result of the ablation experiment on the number of clusters in the present invention.

[0047] Figure 7 These are the results of ablation experiments of the present invention under different failure data sources.

[0048] Figure 8 This is the result of the reward distribution of different trajectories of the present invention.

[0049] Fig. 9These are the experimental results of the present invention in the Concept2Robot experimental environment. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0051] like Figure 2 As shown in the figure, the present invention first identifies different failure modes by clustering failure videos, and assigns a unique identifier to each cluster to represent the corresponding failure prompt. This strategy enables the model to form a detailed understanding of different failure modes and transfer knowledge through learnable failure prompts, thereby improving the initial performance on new tasks. In addition, the present invention also aligns the text and video representations of the human and robot domains through cross-domain contrastive learning and domain-specific prompt learning.

[0052] Specifically, a generalizable robot reward generation method based on a vision-language model includes the following steps:

[0053] S01, access to datasets containing videos and text labels of humans performing different tasks Each video shows a human completing a task with a specific language instruction. Video-language contrastive learning is used to pre-train the video-language model, giving it a basic understanding of human tasks and visual features.

[0054] In step S01, the human video dataset It contains human demonstration videos of N different tasks is the corresponding task description.

[0055] The visual encoder encodes the i-th L frames of video independently into a batch of images and feeds these frame-level features to the temporal visual encoder to obtain a fine-grained temporal video representation v i The text encoder encodes the i-th text describing the corresponding behavior or task into the corresponding text representation t i After that, the video-language contrastive learning pre-training model is adopted, which is defined as follows:

[0056]

[0057] where τ is the temperature coefficient, Indicates that for video v i Match the corresponding task description text t i The loss function based on contrastive learning is , where B represents the batch size, i.e., the number of video-text pairs in each training. At the same time, we calculate That is, for the task description text ti Match its corresponding video v i The loss function based on contrastive learning.

[0058] The sum of the two modality contrast losses is then minimized to obtain the final video-language contrastive learning loss As shown below:

[0059]

[0060] S02, collects data on successful and unsuccessful tasks performed by the robot in a virtual environment to form a robot dataset

[0061] The present invention considers a robot agent to complete the task of task distribution in represents the set of tasks contained in the robot environment, The present invention collects a limited robot dataset in a virtual environment. in They represent the video and the corresponding text instructions executed by the jth robot, respectively, and M represents the number of data in the robot dataset. When describing the task, r j =1, otherwise, r j = 0. In addition, the human dataset is sampled in the task set Data in in Including N human demonstration data, represent the i-th human demonstration video, represents the textual instructions for the corresponding task. While human data is widely available, robotics data only has a limited number of tasks in a few environments, namely Therefore, there are more videos of humans performing the same and different tasks than there are videos of robots performing the same tasks.

[0062] like Figure 1 As shown in the figure, we collect video datasets of robots successfully completing tasks, including pushing a cup from left to right, opening a drawer, etc., to serve as examples of standard task behaviors for the model. At the same time, we collect video datasets of robots failing to perform tasks, covering different failure modes such as incomplete task execution and failure to find the target object, for generating failure prompts.

[0063] S03, input the failure video data in the robot task into the pre-trained visual encoder to obtain the corresponding visual features. Then, use the K-means clustering algorithm to perform cluster analysis on the failure video features of each task, identify and assign failure mode identifiers. For each failure mode, a specific learnable failure prompt is generated and integrated into the text encoder of the model to align with the corresponding failure video.

[0064] For each task The corresponding text description is The present invention is for each task Create a task-specific hint pool It consists of K learnable cues composition:

[0065]

[0066] in, Indicates the task The kth failure prompt in the failure prompt pool.

[0067] For each task’s failed video, spherical K-means clustering is used to iteratively update the cluster center at the end of each training cycle and assign new pseudo labels to the failed video. The present invention will is represented as the failed video features encoded by the current visual encoder, where Indicates the number of videos that failed the task in the training set. The pseudo label q of the i-th video i and cluster center C are obtained by minimizing the following problem:

[0068]

[0069] Then, the failure hint is selected according to the corresponding pseudo-label k. The corresponding input text embedding of the robot failure context is defined as follows:

[0070]

[0071] Among them, k∈{1...K},; represents concatenation along the word embedding length, Indicates the task Text embedding. Each task in the training set corresponds to K failure prompts. In each round of model training, each task can get the text encoding of K failure contexts.

[0072] S04, conducts cross-domain contrastive learning between robot and human data, aligning the features of similar tasks across different domains to achieve consistent feature representation between the robot domain and the human domain. This step is achieved through cross-domain contrastive loss, which helps the model adapt to the differences brought about by domain migration.

[0073] Define a cross-domain contrastive loss with samples from the robotics domain as anchors as follows:

[0074]

[0075] in, It uses samples from the robot domain as anchors, and positive samples are videos from different domains that complete the same task. At the same time, human samples can also be used as anchors to calculate Afterwards, and Combined, we get the cross-domain contrast loss as follows,

[0076]

[0077] S05, based on the original video-language contrastive learning, improves and introduces failure prompts to distinguish the features of successful and failed task execution videos. Specifically, for each video sample in the robot failure dataset, a contrastive learning goal is formed with the corresponding failure prompt, so that the model can use these prompts to more accurately identify the task status.

[0078] We further define each video-text pair (v i ,t i ) video language contrast loss as follows:

[0079]

[0080] And from the failed robot data Define the pair of failed video features and failed context features in The failure video language contrast loss as follows:

[0081]

[0082] S06, training the model until the model reaches a stable convergence state. The final trained model will be used as the robot reward function. The reward function can be combined with visual model predictive control (VMPC) or other reinforcement learning methods to generate reward values ​​for task execution, and select the optimal action sequence through these rewards, or provide reward signals for the learning process.

[0083] For a given task T, the robot agent of the present invention runs a Markov decision process in a fixed range in, Represented as a state space (image or short video), is the robot’s action space, p(s t+1 |s t ,a t ) represents the stochastic dynamics of the robot’s environment, Indicates that for the task Rewards, Represents the corresponding moment. The visual input (a short video) representing the state of the world x i , and a corresponding text description l i Input the trained video-language model to obtain the video feature v i and text feature t i , and is input into the multimodal encoder to obtain a video-level prediction score as a reward.

[0084] The present invention conducts experiments on two typical robot simulation platforms, MetaWorld and Concept2Robot, to test the generalization ability of the reward generation method based on the video-language model in different tasks and environments. These experimental environments contain a variety of task types and changing conditions to evaluate the adaptability and robustness of the model in multi-task scenarios.

[0085] MetaWorld is a robot operation simulation environment based on the MuJoCo physics engine. It provides a Sawyer robot arm and multiple interactive objects to simulate a series of operation tasks in reality. The design of the experimental tasks in MetaWorld covers a variety of daily operation scenarios, including common tasks such as closing drawers, pushing cups, and turning faucets. In order to verify the generalization ability of the model under different visual conditions, the MetaWorld environment sets up a standard training environment and three changing test environments. In the training environment, the color, perspective, and arrangement of all objects are kept consistent as the basic training conditions of the model; in the test environment, the color, perspective, and object arrangement are added to evaluate the adaptability of the model under visual deviation conditions.

[0086] Concept2Robot is another simulation environment for complex multi-task operations. It is built on Pybullet and contains 68 operation tasks, covering a variety of complex operation types such as handling, stacking and pushing. The robot is a simulated 7-DOF Franka Pandas robot arm with a double-layer robot 2F-85 gripper. To further verify the performance of the model under changing conditions, the Concept2Robot test environment simulates more complex operation scenarios by adjusting the viewing angle and adding object interference to test the model's task recognition and execution effects.

[0087] The present invention utilizes the video language model Singularity, which is first pre-trained on 17M human visual language data and then fine-tuned on the Something-Something V2 dataset. For the human data in the training set of the present invention, the present invention uses the Something-Something V2 dataset, which includes 220,837 videos, 174 categories of basic human behaviors, each category containing different objects and scenes. The present invention selects videos from different manual tasks. For the robot demonstration data, there are 560 videos for each task, including the trajectories of success and failure in the training environment. For convenience, the present invention sets the weights of all losses to 1.

[0088] In the MetaWorld environment, we adopt visual model predictive control (VMPC) to select actions, which uses a learned visual dynamics model to plan a sequence of actions. We train an action-conditioned video prediction model p using the FitVid framework. φ (s t+1:t+H |s t ,a t:t+H ). Given an input image s t , sample the G action trajectory of length H, and use p φ To predict their future trajectories t+1:t+H} g Then, the trained model is used to calculate the similarity between the task instruction l and each predicted video for scoring, and the action trajectory with the highest score is executed. In this environment, the method of the present invention is evaluated on four simulated tasks: (1) closing a drawer, (2) moving a cup away from the camera, (3) moving the handle of a faucet, and (4) pushing a cup from left to right.

[0089] In Concept2Robot, we follow the approach used in Concept2Robot and utilize an open-loop trajectory generator that combines Deep Deterministic Policy Gradient (DDPG) and Cross Entropy Method (CEM) to solve this problem.

[0090] Figure 3 The MetaWorld environment (left) is shown consisting of the original training environment and three test environments with modifications to color, viewpoint, and object arrangement. The Concept2Robot environment (right) consists of the training environment and the training and test environments with altered viewpoints or distractor objects.

[0091] Figure 4 The success rates of target tasks in MetaWorld training and test environments are compared for the VMPC method combined with random strategies, Concept2Robot, DVD, LIV, and the proposed Adapt2Reward method. The bar graphs depicted represent the average success rates of the four target tasks calculated from three seeds over 100 trials. In all environments, Adapt2Reward significantly outperforms the three comparison methods on the target tasks, outperforming the best performing method by more than 28% on average.

[0092] Table 1 shows that Adapt2Reward achieves significant generalization performance in new tasks compared to random policies, Concept2Robot, DVD, and LIV. This example reports the average success rate of the four target tasks, calculated for 3 seeds of 100 trials, with standard deviations in parentheses. "Robot Only" in the table means that the reward model is trained only with robot data from the three training tasks, without any data from the four target tasks or any human data. "Robot+3Human Tasks" means training with robot and human videos from the three training tasks. "Robot+6HumanTasks" further integrates human data from three additional tasks (different from the training and target tasks).

[0093] Table 1

[0094]

[0095] Figure 5 To evaluate the impact of failure data on our invention, we compared two training methods: one lacking failure data and another using binary cross entropy (BCE) loss to distinguish between successful and unsuccessful videos. The results show that the generalization ability of the model is significantly improved when failure data is included, which may be due to learning from a wider range of features and failure modes. In addition, using failure cues to understand the cause of failure is more effective than using only BCE loss.

[0096] Figure 6 The influence of different clustering numbers K on the experimental results was evaluated, and the results were optimal when K=3. Figure 7We evaluate whether failure data from different sources affects generalization. We use two methods to collect failure data for training: (1) random exploration and (2) close to success scenarios (achieved by adding noise to successful trajectories or collecting from pre-trained RL algorithms). The results show that combining failure data from different sources is better than using data from a single source, likely due to the wider range of failure modes captured from different sources.

[0097] Figure 8 The reward distribution of different trajectories under different reward functions is shown in Figure 2. The model using BCE loss incorrectly assigns high rewards to failed trajectories. In contrast, the present invention accurately assigns rewards and clearly distinguishes between successful and failed trajectories.

[0098] Fig. 9 Figure 2 shows the success rates of 68 tasks in the Concept2Robot environment under different reward functions. The results show that policies based on the reward function trained with the present invention often equal or exceed policies using "hand-crafted rewards". This advantage may stem from the dense reward scores provided by the present invention, which is beneficial for evaluating attempts that are close to success compared to binary ground truth rewards. In further comparisons, the present invention outperformed in multiple tasks, especially Task 12 (Putting the box on the book) and Task 95 (Pushing the box off the book). This advantage may be due to the fact that Concept2Robot represents objects as "[Something]" during training, which affects the object interaction recognition of specific tasks and hinders accurate success state and object relationship recognition.

[0099] Table 2 evaluates the robustness of the reward function under different viewpoints and interfering objects in the Concept2Robot environment. As shown in Table 2 (a), changing the camera angle significantly affects Concept2Robot (the success rate drops by 5%-37%), while the present invention shows greater stability (the success rate drops by less than 12%). As shown in Table 2 (b), under this new visual arrangement, the success rate of the present invention is 20-46% higher than that of Concept2Robot, demonstrating its robustness to visual changes.

[0100] Table 2

[0101]

[0102] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A generalizable robot reward generation method based on a vision-language model, characterized in that: The following steps are involved: (1) Access to human datasets containing videos and text labels of humans performing different tasks Use video-language contrastive learning to pre-train the video-language model; (2) Collect data on successful and unsuccessful tasks performed by the robot in a virtual environment to form a robot dataset (3) Inputting the failure video data in the robot task into the visual encoder to obtain the corresponding failure visual features; Then, the K-means clustering algorithm is used to perform cluster analysis on the visual features of each task failure, identify and classify different failure modes; for each failure mode, a specific learnable failure prompt is generated and aligned with the corresponding failure video; (4) Conduct cross-domain comparative learning between robot success data and human success data to align the features of similar tasks in different domains. At the same time, based on the original video-language comparative learning, improve and introduce failure prompts to distinguish the features of successful and failed task execution videos; (5) Training the video-language model. The trained model will be used as the robot reward function. The reward function will be combined with the reinforcement learning method to generate the reward value for task execution, guide the robot to select the optimal action sequence or provide learning signals to improve the robot's success rate in completing the task.

2. The generalizable robot reward generation method based on the vision-language model according to claim 1, characterized in that: In step (1), the human dataset It contains human demonstration videos of N different tasks is the corresponding task description, that is, the text label.

3. The generalizable robot reward generation method based on the vision-language model according to claim 2, characterized in that: In step (1), video-language contrastive learning is used to pre-train the video-language model. The specific process is as follows: The visual encoder encodes the video of the i-th L frame in the human dataset independently into a batch of images and provides these frame-level features to the temporal visual encoder to obtain the video representation v i ; The text encoder encodes the i-th task description into the corresponding text representation t i ; After that, the video-language contrastive learning pre-training model is used, and the training loss function is for: In the formula, Indicates that the video is represented by v i Match its corresponding task description text representation t i The loss function based on contrastive learning, Represents the task description text representation t i Match its corresponding video representation v i The loss function based on contrastive learning is , and B represents the number of video-text pairs in each training.

4. The generalizable robot reward generation method based on a vision-language model according to claim 3, characterized in that: is defined as follows: Where τ is the temperature coefficient.

5. The generalizable robot reward generation method based on a vision-language model according to claim 1, characterized in that: In step (2), the robot dataset in, They represent the video and the corresponding text instruction executed by the jth robot, respectively, and M represents the number of data in the robot dataset; when the robot successfully completes the text instruction When describing the task, r j =1, otherwise, r j =0.

6. The generalizable robot reward generation method based on a vision-language model according to claim 1, characterized in that: In step (3), for each robot task For each task Create a task-specific failure prompt pool It consists of K learnable failure prompts composition: in, Indicates the task The kth failure prompt in the failure prompt pool; Each task corresponds to K failure prompts. In each round of model training, each task gets K text encodings of failure contexts.

7. The generalizable robot reward generation method based on a vision-language model according to claim 1, characterized in that: In step (4), when performing cross-domain contrastive learning between robot data and human data, the cross-domain contrastive loss as follows: In the formula, A cross-domain contrast loss that uses samples from the robotics dataset as anchors; is a cross-domain contrast loss that uses samples from the human dataset as anchors; B h represents the number of successful human videos in each training, B r Represents the number of successful videos of the robot in each training.

8. The generalizable robot reward generation method based on a vision-language model according to claim 7, characterized in that: is defined as follows: in, B h represents the number of successful human videos in each training, B r Represents the number of successful videos of the robot in each training.

9. The generalizable robot reward generation method based on a vision-language model according to claim 1, characterized in that: In step (4), the failure prompt is improved and introduced to distinguish the features of successful and failed task execution videos, specifically: For each video sample in the robot failure dataset, a contrastive learning objective is formed with the corresponding failure prompt, the contrast loss as follows: In the formula, the failed robot dataset is Represents the failure video feature and text feature pair of the i-th failure video; represents the k*th failure context text encoding corresponding to the ith failure video, and K represents the number of text encodings of the failure context of the current task.

Images