A Counterfactual Control System Based on Action-Recognizable Joint Embedded World Model

By introducing residual latent dynamics and action identifiability constraints into the latent world model, the problem of insufficient action identifiability in counterfactual programming of the joint embedded world model is solved, improving the success rate of model predictive control and planning performance, and is suitable for post-training of pre-trained visual encoders.

CN123113384APending Publication Date: 2026-11-17NANJING UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611500884.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-24
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing joint embedded latent world models lack action identifiability in counterfactual programming, resulting in low prediction error but insufficient control capability, inability to effectively compare the consequences of different actions, and lack of a counterfactual evaluation mechanism consistent with the real environment.

Method used

By employing a residual latent dynamics prediction module and an action identifiable constraint module, the supervision object of the dynamics predictor is transformed from the absolute successor state to the action-induced potential change through residual parameterization. Furthermore, inverse dynamics and conditional mutual information constraints are applied during prediction transitions to ensure that the model maintains action identifiability in counterfactual programming.

Benefits of technology

It improves the success rate of model predictive control, enhances the searchability of counterfactual optimizers, significantly reduces local minimum bias and elite candidate diffusion, improves planning success rate in difficult and out-of-distribution states, and is suitable for post-training of pre-trained visual encoders.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113384A_ABST
    Figure CN123113384A_ABST
Patent Text Reader

Abstract

This invention discloses a counterfactual control system based on an action-identifiable joint embedded world model. It encodes current and subsequent observations into a unified latent space and reparameterizes absolute subsequent state prediction as a residual displacement prediction based on the current latent state and action. Inverse dynamics and conditional mutual information constraints are applied to the predicted transition, ensuring that action information remains identifiable and recoverable in the predicted subsequent state. The world model is trained jointly with the latent prediction loss and a representation regularization term. During deployment, the model is used to roll candidate action sequences, evaluating them based on the potential distance between the predicted terminal state and the target state. The action distribution and rolling replanning are iteratively optimized using the cross-entropy method. This invention improves the distinguishability and ranking reliability of different counterfactual action consequences, enhancing the success rate and out-of-distribution robustness of model predictive control for robots.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of embodied intelligence, robot learning, latent world models and model predictive control, and specifically to a counterfactual control system based on action-identifiable joint embedded world models. Background Technology

[0002] World models, by learning how environmental states change with actions, provide robots with prediction, planning, and control capabilities without requiring extensive direct interaction with the real environment. Compared to directly generating future images in pixel space, latent world models encode visual observations into low-dimensional or structured latent representations and predict future states in the latent space, significantly reducing the computational overhead of generation and planning. In recent years, the Joint-Embedding Predictive Architecture (JEPA) has further adopted a method of "predicting future representations rather than reconstructing pixels" to learn environmental dynamics, demonstrating high application value in offline visual control, target-condition planning, and robot manipulation.

[0003] Existing joint embedding world models typically use the current latent representation and action as input to directly regress the absolute next-time latent representation, and are trained using the mean squared error or similarity loss of the latent space. This approach achieves low factual trajectory prediction error, but low prediction error does not necessarily imply sufficient model control. Offline robot trajectories exhibit significant short-term autocorrelation; changes between consecutive observations are often minimal, and a large portion of the visual dimensions are dominated by static background, object appearance, and slowly changing factors. Therefore, the predictor can primarily rely on the persistence of the current state to approximate the next state, while downplaying the utilization of fine-grained actions.

[0004] The aforementioned problems are particularly prominent in counterfactual planning based on model predictive control (MMDC). MMDC does not simply evaluate the actual actions executed in the dataset; instead, it requires starting from the same current state, rolling predictions of a large number of candidate action sequences that have not actually been executed, and comparing their relative costs to reach the goal. If different actions produce nearly identical or incorrectly ordered potential consequences in the model, even if the prediction error of the next state in reality is very low, the planner still cannot obtain an effective candidate ordering signal. This leads to problems in sampling optimizers such as cross-entropy methods, including overly flat cost terrain, dispersed elite sets, local optimum shifts, or inconsistencies between the model and the actual environment's ordering.

[0005] Existing inverse dynamics, self-supervised action representation, or mutual information methods are often used to improve visual encoders or policy representations. However, imposing action-aware constraints on the encoder does not guarantee that the subsequent states predicted by the world model itself will still be action-identifiable. For model predictive control, what is truly repeatedly rolled over and used to compare candidate actions is the predicted transition output by the predictor. Therefore, it is necessary to establish action information constraints directly at the predicted transition level.

[0006] Therefore, existing technologies have at least the following shortcomings: First, absolute successor representation regression easily utilizes state inertia and static background to form prediction shortcuts, failing to highlight action-induced local changes; second, the lack of action-identifiable constraints that directly affect prediction transitions leads to the overlapping of different counterfactual action consequences in the potential space; third, the factual one-step prediction error may be significantly decoupled from the downstream planning success rate, lacking a counterfactual evaluation mechanism consistent with the actual needs of model predictive control; fourth, when using a stronger pre-trained visual encoder, there is still a lack of a general method to improve counterfactual planning capabilities without changing the encoder itself, simply through post-training with dynamics. Summary of the Invention

[0007] Purpose of the invention: This invention proposes a counterfactual control system based on an action-identifiable joint embedded world model, aiming to solve the problem of "predictable but uncontrollable" joint embedded latent world model, and to provide a clearer, more stable, and more realistic potential cost terrain for counterfactual action comparison and model predictive control.

[0008] Technical solution: The present invention discloses a counterfactual control system based on action-recognizable joint embedded world model, applied to robotic arm vision operation tasks, comprising:

[0009] Latent State Encoding Module: Acquires the current observation from the robot's offline vision-motion trajectory. ,action and subsequent observations A shared visual encoder is used. The current potential state is obtained by encoding them separately. and subsequent potential states Furthermore, during end-to-end joint training of the visual encoder, a representation regularization term is applied to the latent representation to prevent representation collapse or maintain representation variance; The residual latent dynamics prediction module employs residual parameterization to predict the current latent state. With action Input residual dynamics predictor Output action-induced potential increment and in accordance with Construct a predictable subsequent potential state; Action-identifiable constraint module: This module assigns actions to... via motion encoder Mapping to action embedding And based on the current potential state With predicting subsequent potential states Imposing at least one action information constraint on the predicted transition, such that the predicted transition retains action information used to distinguish the consequences of different actions; Joint training module: to predict subsequent potential states. With subsequent potential states The potential prediction error is used as the main loss, and it is weighted and combined with the action information constraint and the representation regularization term to train the visual encoder, residual dynamics predictor, and action information constraint module; Counterfactual planning and control module: encodes the current observation at deployment time. With target observation get and Starting from the same current potential state, multiple candidate action sequences are sampled. Based on the residual potential dynamics prediction module, the predicted terminal potential state corresponding to each candidate action sequence is recursively obtained. The prediction is then compared with the target potential state. The candidate action sequences are sorted by distance, and the action is selected by the model prediction control optimization process of sampling-evaluation-elite update. After the first action block of the selected action sequence is executed, new observations are obtained and replanning is performed.

[0010] Furthermore, the visual encoder in the latent state encoding module employs a visual representation network with a joint embedding prediction architecture; the joint embedding prediction architecture is trained by minimizing the distance between the predicted latent representation and the target latent representation.

[0011] Furthermore, the prediction target of the residual latent dynamics prediction module is the local latent displacement of the subsequent state relative to the current state. The residual parameterization transforms the direct supervision object of the dynamics predictor from an absolute successor potential representation to an action-induced potential change, thereby reducing the dominance of static background, state inertia, and slowly changing characteristics on dynamics learning.

[0012] Furthermore, the action information constraints include inverse dynamics constraints and conditional mutual information constraints; both the inverse dynamics constraints and the conditional mutual information constraints are based on model predictions. Instead of relying solely on the actual subsequent representation output by the visual encoder, this allows motion information to be directly written into the dynamic state of the rolling control model during the deployment phase.

[0013] Furthermore, the inverse dynamics constraint includes: a constraint imposed by the inverse dynamics head. Receive the current potential state With predicting subsequent potential states Output action embedding estimation ;use As an inverse dynamic loss, sg(·) denotes stopping gradient operations, making the action embedding a stable supervision target, and making the optimization gradient mainly act on the predicted transition to improve its action recognizability.

[0014] Furthermore, the conditional mutual information constraint is based on To optimize the objective, a posterior distribution is constructed. The target is formed after batch normalization of the action embedding and stopping the gradient. By minimizing the posterior pair The negative log-likelihood is used to improve the degree to which the action embedding can be recovered from the predicted subsequent latent state given the current latent state; and the KL divergence between the posterior and the unit Gaussian prior is added as a stabilizing term.

[0015] Furthermore, the overall training objective of the joint training module is:

[0016]

[0017] in, , To characterize the regularization term, For inverse dynamic loss, The conditional mutual information constraint loss includes the normalized action recovery negative log-likelihood term and the posterior KL stability term relative to a unit Gaussian reference distribution. , and The weights are non-negative.

[0018] Furthermore, the visual encoder can be jointly trained end-to-end with the residual dynamics predictor, or it can be post-trained only on the motion condition latent dynamics, motion encoder, inverse dynamics head, and conditional mutual information posterior head using a pre-trained and frozen visual video encoder.

[0019] Furthermore, the action employs a continuous motion vector that includes Cartesian position increments, end-effector attitude increments, and gripper control quantities.

[0020] Furthermore, after the counterfactual planning and control module outputs an action, the robot control end safely trims the position increment and converts it into a target pose that the robotic arm can execute. After each execution, visual observations are reacquired and potential encoding and planning are performed.

[0021] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] 1. This invention reparameterizes the supervision object of the dynamics predictor from "absolute successor potential state" to "action-induced local potential displacement", which structurally weakens the prediction shortcuts caused by state persistence, static background and slowly changing visual features, and allows the model capacity to be allocated more to the dynamic components that change with the action.

[0023] 2. This invention directly applies inverse dynamics and conditional mutual information constraints to the prediction of subsequent potential states. Instead of merely improving the encoder’s motion-aware representation, it ensures that the predicted transfer of actual rolling in the model predictive control during the deployment phase remains motion-identifiable.

[0024] 3. This invention avoids degenerate collaboration between the auxiliary head and the action encoder by stopping the gradient action target and standardizing the action embedding, and controls the scale by using a lightweight posterior contraction term, so that the action information regularization can be stably used as a plug-in training target for the latent world model.

[0025] 4. This invention improves the searchability of potential cost terrains seen by counterfactual optimizers such as CEM. In experiments, on OGBench-Cube hard and out-of-distribution starting points P00-P04, the baseline LeWM average success rate is 3.7%, while the default configuration of this invention reaches 52.0%, and the enhanced mutual information configuration reaches 61.5%. It also significantly reduces local minimum offset, elite candidate diffusion, and model-environment cost difference in CEM.

[0026] 5. This invention can be transferred to a strongly pre-trained visual backbone without retraining the encoder for the target experimental scenario; in a real Franka robotic arm experiment with the V-JEPA2 visual encoder frozen and trained using only DROID offline data for dynamics, the basic grasping and placement success rate increased from 42.2% to 71.1%, without the need for special fine-tuning for the experimental scenario. Attached Figure Description

[0027] Figure 1 This is a framework diagram of the present invention;

[0028] Figure 2 This is a schematic diagram of the training structure of the action-identifiable latent world model of the present invention, showing latent encoding, residual dynamics prediction, and inverse dynamics and conditional mutual information constraints;

[0029] Figure 3 This is a schematic diagram of the model prediction control process of the present invention, showing the process of candidate action sampling, potential rolling, target cost calculation, elite update, action execution and replanning;

[0030] Figure 4 Example diagrams of trajectory responses corresponding to different facts and counterfactual actions starting from the same current state;

[0031] Figure 5 For rolling response consistency of different latent world models Statistical results chart;

[0032] Figure 6 This chart compares the planning success rates of the present invention under the original test protocols in TwoRoom, Reacher, Push-T, and OGBench-Cube environments.

[0033] Figure 7 This is a comparison chart of the results of the present invention under the following conditions: Cube difficulty and distributed initial state, Scene environment, and counterfactual response index.

[0034] Figure 8 CEM candidate cost topographic maps generated for different potential dynamic models;

[0035] Figure 9 A statistical comparison chart of local minimum deviation, elite set dispersion, and model-environment cost difference;

[0036] Figure 10 This is a diagram of a real Franka robotic arm experimental setup;

[0037] Figure 11 This is a schematic diagram showing the setup of tasks such as basic grasping and placement, complex object manipulation, and multi-object manipulation under target conditions. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings.

[0039] like Figure 1As shown, this invention proposes a counterfactual control system based on an action-identifiable joint embedded world model, mainly comprising a latent state encoding module, a residual latent dynamics prediction module, an action-identifiable constraint module, a joint training module, and a counterfactual planning and control module. In the training phase, the latent state encoding, residual latent dynamics, and action-identifiable constraints are jointly trained using current observations, actions, and subsequent observations to obtain an action-identifiable latent world model. In the deployment phase, the trained latent world model is used for counterfactual latent rolling of candidate action sequences, and actions are selected and executed through target cost evaluation and model predictive control. This invention addresses the problem that joint embedded latent world models tend to rely on state persistence and weaken the influence of actions in offline visual trajectories by constructing an action-identifiable latent world model oriented towards counterfactual planning. For the same current state, model predictive control (MPC) needs to compare the future consequences produced by multiple different candidate actions or action sequences, rather than simply fitting the factual subsequent states corresponding to the actual actions executed in the dataset. Therefore, this invention highlights the local latent changes induced by action through residual latent dynamics, and directly applies inverse dynamic constraints and conditional mutual information constraints to the subsequent latent states predicted by the latent world model, so that different candidate actions form distinguishable and comparable consequences in the prediction latent space; after the action-identifiable latent world model is trained, the learned latent dynamics are used for counterfactual action rolling and model prediction control under target image conditions.

[0040] This invention addresses an application scenario where robots are trained based on offline visual trajectories and controlled under target conditions using a world model during the deployment phase. The training data consists of robot visual observations and corresponding actions, with each adjacent observation representing a specific action. ,action and subsequent observations A shared visual encoder is used. Mapping the current observations and subsequent observations to a unified latent space yields the current latent state. With subsequent potential states The visual encoder employs a joint embedding prediction architecture, which does not require reconstructing the original pixels from the latent states, but instead learns future state predictions directly in the representation space. To prevent representation collapse during joint embedding training and to maintain effective statistical changes across different latent dimensions, a representation regularization term can be applied to the visual encoder output. .in, Indicates a time index. This represents the parameters of the visual encoder.

[0041] In this embodiment, the regularization term is SIGReg; however, this regularization term may not be used if a pre-trained and frozen visual encoder is employed. The visual encoder can be jointly trained end-to-end with the subsequent dynamics predictor, or a pre-trained visual or video representation network can be used with its parameters frozen, training only the motion conditional latent dynamics and corresponding motion-identifiable modules. Therefore, this invention is applicable both to training a joint embedded world model directly from offline visual trajectories and to post-training dynamics based on existing large-scale pre-trained visual models.

[0042] Traditional joint embedded world models typically predict the absolute successor latent state directly based on the current latent state and action. However, robot offline trajectories usually exhibit strong temporal continuity between adjacent time points, with a significant amount of background, object appearance, and low-frequency state information remaining largely unchanged. Therefore, the predictor tends to rely on the persistence of the current state to achieve low successor state prediction errors, while underutilizing action inputs. Although such joint embedded world models can accurately fit the actual factual transitions occurring in the training trajectory, when different candidate actions are input from the same current state, the resulting predictions may be too similar or incorrectly ordered, making it difficult to provide a reliable basis for predictive control based on candidate action comparison using joint embedded world models.

[0043] In response to the above problems, such as Figure 2 As shown, this invention integrates latent dynamics prediction and action information constraints during the training phase, applying them to the actual predicted transitions generated by the latent world model. Instead of having the dynamics predictor directly output the absolute successor latent state, this invention allows it to predict local latent state changes jointly determined by the current state and the executed action. Specifically, the current latent state... With action Input residual dynamics predictor The predicted potential increment of action-induced behavior is obtained:

[0044]

[0045] Where θ represents the parameters of the residual latent dynamics predictor. And based on the current latent state, the predicted successor state is constructed:

[0046]

[0047] The latent world model is trained by minimizing the distance between the predicted successor state and the corresponding latent state in the actual successor observation:

[0048]

[0049] From the perspective of equivalent supervision relationships, the residual dynamics predictor actually learns what... The representation focuses on local potential displacements, rather than the complete subsequent state itself. Therefore, many static or slowly changing components in the current state that do not require action interpretation do not need to be explicitly generated again by the dynamics predictor, allowing the predictor to allocate more modeling capabilities to describing action-related local state changes. This residual form is not a simple network skip connection, but rather alters the learning object and inductive bias directly faced by the latent dynamics predictor. In this embodiment, the visual input resolution is... The visual encoder uses a visual Transformer with a latent dimension of 192; the residual dynamics predictor uses a Transformer structure with a depth of 6, 16 attention heads, and a multilayer perceptron with a hidden dimension of 2048. During training, three consecutive frames of historical visual input are used, while one frame is used during testing, with data augmentation techniques such as random scaling and cropping applied to the visual images.

[0050] Learning local changes solely through residuals cannot guarantee sufficient identifiability of different actions in the prediction latent space. To ensure that the latent world model retains action information during actual counterfactual rolling, this invention further directly modifies the prediction transfers generated by the latent world model itself. Apply action-recognizable constraints, rather than just constraining the realistic representation obtained by the visual encoder. First, employ an action encoder. Continuous action Mapping to action embedding Then through the inverse dynamics head Recover the corresponding action embedding based on the current potential state and the predicted subsequent state:

[0051]

[0052] in, and These represent the parameters of the motion encoder and the inverse dynamics head, respectively.

[0053] Inverse dynamic loss is expressed as:

[0054]

[0055] in, This represents the stopping gradient. By embedding actions as the supervisory target for stopping the gradient, we can avoid the action encoder and inverse dynamics head reducing the loss by jointly altering the action representation. This allows the optimization process to primarily drive the predictions obtained by the residual latent dynamics predictor. It retains information that distinguishes actual actions. In other words, given a fixed current state, different actions need to exhibit sufficient differences in the predicted transition to be recognizable in reverse engineering. In a specific implementation, the inverse dynamics head employs a multilayer perceptron with a hidden dimension of 1024, and the inverse dynamics loss weights can be set to... .

[0056] Building upon this, the present invention further enhances the recoverability of actions in predicted subsequent states by using normalized action recovery constraints based on conditional mutual information motivation. The recovery term aims to improve... The conditional mutual information describes how much action-related information is still contained in the predicted successor state given the current latent state. If this amount is too low, it means that the latent world model's predicted state cannot reflect the action performed given the current state, and different candidate actions are likely to produce similar predicted consequences, making it impossible to form an effective counterfactual comparison. To achieve the conditional mutual information constraint, an action posterior distribution conditioned on the current latent state and the predicted successor state can be constructed. Furthermore, by utilizing variational lower bounds, maximizing mutual information is transformed into an action embedding recovery objective. In a specific implementation, a conditional posterior distribution is constructed. First, embed the action. Standardize within the current training batch. ;in, This represents the parameters of the conditional action post-procedure header. Among them, and These are the mean and standard deviation of the intra-batch action embeddings, respectively, and a stopping gradient is applied to the standardized action target.

[0057] The conditional mutual information loss is expressed as:

[0058]

[0059] The first term improves the retention of conditional action information in the predicted transition by restoring the target through variational actions; the second term is a posterior stabilization term, used to limit the scale of the posterior mean and avoid unconstrained amplification of the posterior distribution during training; the KL term is used as a numerical stabilization mechanism, rather than as a lower bound of the conditional mutual information itself. This represents the non-negative weights of the KL stable term. Represents the identity matrix with the same dimension as the action embedding; when the state or action representation and target are normalized, the first term corresponds to the action recovery term in the variational lower bound of the conditional mutual information.

[0060] Both inverse dynamic constraints and conditional mutual information constraints directly depend on the latent world model's prediction of the state. This allows motion information to be directly written into the potential dynamic state that requires recursive rolling for subsequent model prediction and control.

[0061] Taking into account the above constraints, the joint training objective of the latent world model is expressed as:

[0062]

[0063] in, Used to ensure basic potential dynamic prediction capabilities. Used to maintain the stability of joint embedding representation, and The transition is constrained from two perspectives: deterministic action recovery and preservation of conditional action information. In this implementation, the AdamW optimizer is used, with a learning rate of [missing information]. The weight decays to The batch size is 128, and the training is performed for 10 epochs using bf16 precision. The weight of the representation regularization term can be set to 0.09, the weight of the inverse dynamics loss can be set to 0.1, and the weight of the conditional mutual information loss can be set to 0.01.

[0064] The strength of the conditional mutual information constraint can be adjusted according to the task's requirements for action sensitivity. For example, in a setting that emphasizes the ability to distinguish counterfactual actions in difficult initial states and out-of-distribution states, the weight of the conditional mutual information loss can be increased. The strength of the action information constraint can be set according to the specific task to balance action identifiability and potential scroll fidelity.

[0065] Through the above joint optimization, the residual latent dynamics predictor is used to focus on predicting local action-induced changes, the inverse dynamics constraint enables the execution action to be identified from the predicted transition, and the conditional mutual information constraint further ensures that the action information can still be recovered from the predicted subsequent state after the current state is given. The three together form an action-identifiable latent dynamics model for counterfactual programming.

[0066] like Figure 3 As shown, after the action-identifiable latent world model is trained, the action-identifiable latent dynamics are directly used as the world model in model predictive control, and a closed-loop control process is formed through "candidate action sampling—counterfactual latent rolling—target cost evaluation—candidate distribution update—action execution—replanning". The robot acquires the current visual observation. and target observations representing the desired task state The current latent state is obtained using the same visual encoder. and target potential state The planner starts from the same current potential state. Trigger sampling of multiple candidate action sequences:

[0067] in, To plan the time domain, denoted as the number of candidate action sequences, and i represents the index of the candidate action sequence.

[0068] For each candidate action sequence, recursive counterfactual rolling is performed using the trained residual latent dynamics:

[0069]

[0070] During the first scroll τ represents the rolling time index; for any candidate sequence i, the initial rolling initialization is that the predicted current state is equal to the encoded current state.

[0071] Obtain the predicted terminal latent state corresponding to each candidate action sequence. Then, the cost of the candidate action is calculated based on its distance from the target potential state. In a specific implementation, the following is used:

[0072]

[0073] Thus, different candidate action sequences, starting from the same current state, produce different counterfactual potential rolling results in the world model. The planner compares and selects based on these predicted consequences rather than a single factual trajectory.

[0074] In this embodiment, the Cross-Entropy Method (CEM) is used to optimize the action sequence. First, samples are taken from the parameterized action distribution. Given a set of candidate action sequences, calculate the target cost for each candidate action using the aforementioned latent world model; select the action with the lowest cost. Each candidate is treated as an elite set. The action distribution parameters are re-estimated based on the elite action sequence. The process of "sampling-potential rolling-cost evaluation-elite update" is repeated until the predetermined number of iterations is reached.

[0075] After optimization, the robot executes the first action or the first action block in the obtained optimal action sequence, acquires new visual observations, and executes the latent encoding and candidate action planning again, thus forming a closed-loop rolling time-domain control.

[0076] CEM samples 300 candidate action sequences each time, selects 30 of them as elite candidates, iterates 30 times, and plans the temporal domain. The action block length is 5. All candidate actions are pruned to the range of actions allowed by the task environment. All potential world models to be compared use the same model prediction control parameters so that the differences in planning performance mainly come from the differences in the learned potential dynamics.

[0077] like Figure 4 As shown, starting from the same current state, different factual and counterfactual actions will produce different trajectory responses; the underlying dynamics with strong action identifiability should be able to produce corresponding and orderly predictive consequences as candidate actions change. Furthermore, as... Figure 5 As shown, this can be achieved through consistent rolling response. The consistency between the world model's predicted response and the actual environmental response is crucial. Since the cross-entropy method inherently relies on the relative cost ranking among a large number of candidate actions, the world model's ability to generate correctly relative predicted responses to different factual and counterfactual actions from the same current state directly impacts planning quality. To evaluate this capability, the current state can be fixed, and multiple sets of candidate actions or action sequences can be generated. The consistency between the world model's predicted costs and the costs corresponding to the actual visual results after execution can be compared. The Spearman rank correlation coefficient can be used to define a counterfactual action identifiability index.

[0078]

[0079] in, Represents the Spearman rank correlation coefficient. Indicates the index of the shared candidate action sequence. The target cost predicted by the potential world model. The target cost is the cost obtained after actually executing the candidate actions. The actual execution cost can be directly calculated using the distance between the latent state obtained from the visual observation after execution and the target latent state obtained through the same encoder. Therefore, it does not need to rely on privileged state information such as the precise coordinates of objects, which are difficult to obtain during the actual deployment of the robot. In addition to the overall ranking correlation, we can further examine the quality of the elite candidate set, the pairwise ranking accuracy of candidate actions, the Top-k regret value, and the local cost terrain formed by model predictive control to determine whether the world model provides a discriminative planning signal for candidate action search.

[0080] This invention was trained and tested in the OGBench-Cube environment. The standard absolute successor latent state prediction model achieved a high success rate under the original testing protocol, but its performance significantly decreased in difficult initial states and out-of-distribution tests with progressively increasing initial state perturbations. By employing the residual latent dynamics of this invention, the planning success rate under difficult and out-of-distribution states was significantly improved; further, by adding inverse dynamics and conditional mutual information constraints, the responses and rankings of different counterfactual actions in the latent space were further improved.

[0081] like Figure 6 and Figure 7As shown, in the OGBench-Cube environment, the original test protocol and the difficult and out-of-distribution initial state tests exhibit significantly different model differentiation results. The success rate of the absolute successor latent prediction baseline under the original protocol is 73.3%, while the average success rate on the difficult and out-of-distribution starting points P00-P04 is only 3.7%; the model of this invention with enhanced conditional mutual information constraints achieves an average success rate of 61.5% on P00-P04. Further ablation experiments show that the success rate is 34.7% when using only residual latent dynamics, 37.1% after adding inverse dynamics constraints, and 52.0% after using complete action identifiable joint targets.

[0082] like Figure 8 As shown, when analyzing the CEM candidate cost terrain formed by different latent world models in the shared candidate action subspace, the candidate cost structure formed by the absolute prediction baseline is relatively flat or deviates significantly from the actual environmental consequences. However, after applying this invention, the low-cost regions of candidate actions are more concentrated. Furthermore, as... Figure 9 As shown, the one-step local minimum bias, elite set dispersion, and model-environment cost difference of the absolute prediction baseline are approximately 0.827, 0.448, and 121.1, respectively. After adopting the default configuration of this invention, they are reduced to approximately 0.130, 0.082, and 78.1, respectively, indicating that the latent dynamics after action recognition training can provide a more concentrated and stable target cost structure for candidate action optimization.

[0083] like Figure 6 and Figure 7 As shown, this invention can also be applied to Reacher, TwoRoom, PushT, and Scene environments containing interactive types such as buttons, squares, drawers, and windows. Specifically, in the Reacher and TwoRoom environments, performance is improved relative to the absolute prediction baseline of matching training; in the PushT environment where the baseline planning success rate is close to saturation, the improvement is not significant; and in multi-object Scene environments, there is an overall improvement trend, mainly reflected in drawer and window interaction tasks. These results indicate that this invention primarily addresses the problem of insufficient response to action-induced changes in joint embedded latent dynamics, and is not intended to replace long-range visual localization, complex contact dynamics modeling, or high-level task planning.

[0084] To verify the applicability of this invention in pre-trained visual world models and real robots, this invention uses a pre-trained V-JEPA2 video encoder as the visual backbone and keeps its parameters frozen, only post-training the motion condition latent dynamics predictor and the inverse dynamics and conditional mutual information modules. The post-training data uses DROID offline robot operation data; data collected from the target experimental scene is not used for scene-specific fine-tuning of the visual encoder or world model. In this embodiment, the visual input uses images from a camera located on the robot's external side and rear, which are pre-processed to form 8 frames at 4fps video input; the robot's motion uses a 7-dimensional continuous incremental vector, including Cartesian position increments (dx, dy, dz), end-effector pose increments (droll, dpitch, dyaw), and gripper control variables (dgripper).

[0085] The action-conditional latent dynamics predictor has a dimension of 1024, a depth of 24, and 16 attention heads; the inverse dynamics and conditional mutual information modules employ a multilayer perceptron with a hidden dimension of 1024; and the AdamW optimizer is used for training with a learning rate of [missing information]. The training was performed on four NVIDIA A800 GPUs with a batch size of 8 GPUs for 315 epochs, with 300 iterations per epoch. The inverse dynamics loss had a weight of 0.1, and the conditional mutual information loss had a weight of [missing value]. The KL stability term coefficient is 0.1.

[0086] When deploying the actual robot, CEM-based rolling control is also used. In one specific implementation, 800 candidate actions are sampled for each planning iteration, the top 10 candidates are selected as the elite set, and 10 CEM updates are performed. Each planning iteration rolls one potential state transition, and replanning is performed after execution. For the three stages of grasping, moving, and placing, 10, 10, and 4 rolling execution steps can be assigned respectively.

[0087] The robot control end performs a safe range clipping of the Cartesian position increment selected by the planner and converts the incremental motion into the target pose of the absolute end effector in the robot arm's base coordinate system; at the same time, it sets opening and closing thresholds and width limits for the gripper motion and then sends the corresponding control commands to the robot arm.

[0088] like Figure 10 As shown, in a real robot embodiment, a Franka robotic arm, an external vision acquisition device, and a closed-loop model predictive control system based on target image conditions are used for robot operation. Figure 11As shown, the real-world robot evaluation includes basic grasping and placement, complex object manipulation, and target-conditional multi-object manipulation tasks. In 45 basic grasping and placement experiments consisting of yellow squares, orange squares, and orange cups, the absolute motion-conditional world model using the same frozen pre-trained visual encoder and the same offline training data completed 19 trials with a success rate of 42.2%. After applying the action-identifiable latent dynamics of this invention, 32 trials were completed, increasing the success rate to 71.1%. In complex object and multi-object target designation tasks, this invention also improves the performance of target object following and final lifting and placement, indicating that the residual latent dynamics and predicted transfer action information constraints can be used as a post-training module for the pre-trained visual world model without retraining the visual backbone for the target experimental scenario.

[0089] The specific embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific network architecture, loss weights, visual encoder type, action representation form, and model predictive control optimizer described above. Equivalent substitutions for the encoder, predictor, action information constraint method, latent distance form, and candidate action optimization method, without departing from the technical concept of "using action-induced local latent changes as the dynamic prediction object and directly maintaining action information in the latent world model prediction transition to improve the distinguishability and comparability of different counterfactual action consequences," are all within the scope of protection of the present invention.

Claims

1. A counterfactual control system based on action recognizable joint embedding world model, applied to a visual operation task of a robot arm, characterized in that, include: latent state encoding module: obtaining current observation in robot offline vision-action trajectory , action and successor observation , using shared vision encoder to encode current latent state and successor latent state respectively, and imposing representation regularization term on latent representation for preventing representation collapse or maintaining representation variance when vision encoder participates in end-to-end joint training; residual latent dynamics prediction module: using residual parameterization, inputting current latent state and action into residual dynamics predictor , outputting action-induced latent increment , and constructing predicted successor latent state according to ; action recognizable constraint module: mapping actions action encoder to action embeddings and based on the current latent state with the predicted successor latent state imposes at least one action information constraint on the predicted transition, such that the predicted transition preserves action information useful for distinguishing between different action outcomes; joint training module: training the visual encoder, residual dynamics predictor, and action information constraint module jointly with a main loss based on a latent prediction error between the predicted successor latent state and the successor latent state and a weighted combination of the action information constraint and a representation regularization term. Counterfactual planning and control module: Encodes current observations at deployment time. With target observation get and Starting from the same current potential state, multiple candidate action sequences are sampled. Based on the residual potential dynamics prediction module, the predicted terminal potential state corresponding to each candidate action sequence is recursively obtained. The prediction is then compared with the target potential state. The candidate action sequences are sorted by distance, and the action is selected by the model prediction control optimization process of sampling-evaluation-elite update. After the first action block of the selected action sequence is executed, new observations are obtained and the plan is re-planned.

2. The counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, The visual encoder in the latent state encoding module employs a visual representation network with a joint embedding prediction architecture; the joint embedding prediction architecture is trained by minimizing the distance between the predicted latent representation and the target latent representation.

3. The counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, The prediction objective of the residual latent dynamics prediction module is the local latent displacement of the subsequent state relative to the current state. The residual parameterization transforms the direct supervision object of the dynamics predictor from an absolute successor potential representation to an action-induced potential change, thereby reducing the dominance of static background, state inertia, and slowly changing characteristics on dynamics learning.

4. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, The action information constraints include inverse dynamics constraints and conditional mutual information constraints; both inverse dynamics constraints and conditional mutual information constraints are obtained based on model predictions. Instead of relying solely on the actual subsequent representation output by the visual encoder, this allows motion information to be directly written into the dynamic state of the rolling control model during the deployment phase.

5. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 4, characterized in that, The inverse dynamics constraint includes: a contingent of inverse dynamics head Receive the current potential state With predicting subsequent potential states Output action embedding estimation ;use As an inverse dynamic loss, sg(·) denotes stopping gradient operations, making the action embedding a stable supervision target, and making the optimization gradient mainly act on the predicted transition to improve its action recognizability.

6. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 4, characterized in that, The conditional mutual information constraint To optimize the objective, a posterior distribution is constructed. The target is formed after batch normalization of the action embedding and stopping the gradient. By minimizing the posterior pair The negative log-likelihood is used to improve the degree to which the action embedding can be recovered from the predicted subsequent latent state given the current latent state; and the KL divergence between the posterior and the unit Gaussian prior is added as a stabilizing term.

7. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, The overall training objective of the joint training module is: in, , To characterize the regularization term, For inverse dynamic loss, The conditional mutual information constraint loss includes the normalized action recovery negative log-likelihood term and the posterior KL stability term relative to a unit Gaussian reference distribution. , and The weights are non-negative.

8. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, The visual encoder can be jointly trained end-to-end with the residual dynamics predictor, or it can be post-trained only on the motion condition latent dynamics, motion encoder, inverse dynamics head, and conditional mutual information posterior head using a pre-trained and frozen visual video encoder.

9. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, The action is a continuous motion vector that includes Cartesian position increment, end-effector attitude increment, and gripper control quantity.

10. A counterfactual control system based on an action-identifiable joint embedded world model according to claim 1, characterized in that, After the counterfactual planning and control module outputs an action, the robot control end safely trims the position increment and converts it into a target pose that the robotic arm can execute. After each execution, visual observations are reacquired and potential encoding and planning are performed.