A physical world model system with embodied intelligence and a control method thereof

Abstract

The application discloses a physical world model system with a body intelligence and a control method thereof, and comprises a mode switcher, based on preset switching indicators, selects to activate an end-to-end neural network architecture or a hierarchical architecture; the two architectures share the following homologous modules: an input encoding module, uniformly encodes multi-modal input; a coordinate specification module, maps the encoding result to a world coordinate system; a physical field construction module, generates a plurality of physical fields based on the output result of the coordinate specification module; a relationship mapping module, abstracts the plurality of physical fields into a structured world state; a causal reasoning module, completes the causal constraints of the structured world state; a prediction module, predicts based on the current state, action constraints and task targets; an action generation module, generates world semantic actions based on the prediction result, and compiles into control instructions. The application realizes dynamic switching between the neural network architecture or the hierarchical architecture according to different task scenarios based on the quantitative mode switcher.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a physical world model system with embodied intelligence and its control method. Background Technology

[0002] Embodied intelligent systems need to understand, reason, and act in the physical world. Current mainstream technologies mainly include vision-language-action models, generative world models, and latent representation world models. Among them, the latent representation action-conditional world model framework, represented by the V-JEPA series, demonstrates high efficiency in robot control tasks by pre-training encoders on large-scale internet videos and then training action-conditional predictors in the latent representation space for planning. However, existing solutions still have the following shortcomings: Insufficient generalization of physical attributes: Although the representations learned by the model have semantic information, the modeling of dense physical attributes that directly affect interaction, such as depth, contact, support, and friction, is not explicit and stable enough, leading to performance degradation when changing scenes or perspectives. Lack of unified physical coordinates and action semantics: V-JEPA2 explicitly states that its action inference is sensitive to camera position because its action semantics are implicitly bound to specific camera-robot configurations, lacking a unified world coordinate normalization and action semantic definition across platforms and perspectives. Weak long-term causal reasoning ability: The model focuses on short-term autoregressive prediction and energy minimization of the target image, lacking explicit modeling and reasoning of common-sense causal structures such as "preconditions-consequences-taboo constraints," making it difficult to handle long-term tasks involving multiple steps and implicit physical constraints. Rigid architecture, difficulty in balancing efficiency and reliability: Existing solutions typically employ a single architecture. While end-to-end strategies like VLA offer low latency, they lack explicit world models and online planning, limiting interpretability and generalization ability to unseen tasks. Hierarchical systems, while highly interpretable, suffer from inconsistent interfaces between modules, low collaboration efficiency, and an inability to dynamically adjust strategies based on the scenario. Therefore, there is an urgent need for an embodied intelligent world model architecture that can unify dense physical understanding and common-sense causal reasoning, and flexibly adapt between efficiency and reliability.

[0003] The "Intelligent Driving Control Method and System for Embossed Unmanned Surface Vessels Based on World Model" disclosed in Chinese patent literature, publication number CN121158158A, and publication date December 19, 2025, includes: collecting real-time visual images, navigation status data, and control action data of the unmanned surface vessel; preprocessing and synchronizing the data to obtain standardized visual, state, and action inputs and standardized data packets; mapping the standardized visual, state, and action inputs to latent representations; performing autoregressive calculations through a predictive backbone network to predict multi-step latent state and latent action sequences; and finally obtaining future state and future action sequences based on the predicted latent state and latent action sequences. However, this technology is merely a simple technical improvement using a latent representation world model, and still suffers from insufficient generalization of physical attributes, a lack of unified physical coordinates and action semantics, weak causal reasoning ability, and an architecture that is rigid and struggles to balance efficiency and reliability. Summary of the Invention

[0004] This invention aims to overcome the problems of insufficient generalization of physical attributes, lack of unified physical coordinates and action semantics, weak causal reasoning ability, and rigid architecture that makes it difficult to balance efficiency and reliability in existing embodied intelligent physical world models. It provides an embodied intelligent physical world model system and its control method.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: An embodied intelligence physical world model system includes a mode switcher that selects to enable either an end-to-end neural network architecture or a hierarchical architecture based on preset switching criteria; both architectures share the following common modules: The input encoding module performs unified encoding on multimodal inputs; The coordinate standardization module maps the encoding results to the world coordinate system; The physics field construction module generates several physics fields based on the output of the coordinate normalization module. The relation mapping module abstracts several physical fields into structured world states; The causal reasoning module completes the causal constraints for the structured world state; The prediction module makes predictions based on the current state, action constraints, and task objectives. The action generation module generates world semantic actions based on the prediction results and compiles them into control instructions.

[0006] The core of this invention lies in supporting two scalable implementation architectures within the same physical world model, ensuring their essential identity through a unified sharing mechanism, and combining this with a quantitative mode switcher to achieve dynamic switching between neural network architectures or hierarchical architectures based on different task scenarios. At the perception and understanding level, the model utilizes a physics field construction module and a relationship graphing module to build stable internal representations of physical attributes such as scene geometry and interaction relationships, thereby improving the generalization ability of physical attributes. At the reasoning and planning level, by introducing a dedicated causal reasoning module, the model internalizes physical constraint knowledge about preconditions and postconditions, thus enabling it to handle complex combinatorial tasks requiring multi-step logical reasoning. At the system architecture level, a switchable design of end-to-end neural network architecture and hierarchical architecture is adopted, coupled with a mode switcher based on switching metrics, enabling the system to achieve an optimal balance between high real-time direct response and high reliability, taking into account both efficiency and security. At the engineering deployment level, thanks to the unified world coordinate system, world semantic actions decoupled from the ontology, and the two-level compilation architecture of "world actions-ontology instructions", the same model core can be efficiently adapted to robot platforms with different structures (such as robotic arms and mobile chassis), greatly reducing the migration cost from model to actual application scenarios.

[0007] Preferably, the neural network architecture includes an input encoding subnetwork, a coordinate normalization subnetwork, a physics field construction subnetwork, a relation graphing subnetwork, a causal reasoning subnetwork, a prediction subnetwork, an action generation subnetwork, and a security correction subnetwork connected in sequence; each subnetwork is derived from the corresponding module. The safety correction subnetwork performs risk assessment and constraint projection on world semantic actions, and outputs the final control command to the robot execution end, while the feedback data from the execution end flows back to the input encoding subnetwork.

[0008] Preferably, the layered architecture includes: a system layer that provides unified data and a controlled execution environment for agents and world models; The platform's intelligent agent layer acts as a bridge between tasks and world model calls, invoking the extensible world model layer to complete tasks. The extensible world model layer includes an input encoding module, a coordinate normalization module, a physics field construction module, a relationship graphing module, a causal reasoning module, a prediction module, an action generation module, and a safety correction module.

[0009] Preferably, the mode switcher is configured to dynamically select and enable an end-to-end neural network architecture or a hierarchical architecture based on at least one switching metric among task time domain, model uncertainty, physical risk and task complexity, combined with preset switching rules.

[0010] Preferably, the neural network architecture and the hierarchical architecture share a unified state space and physical semantics; A state space containing several sub-states required for modeling the physical world is predefined. Both neural network architectures and hierarchical architectures map the input to the state space through a unified encoding protocol. A unified physical semantic ontology library is pre-established to ensure that the neural network architecture and the hierarchical architecture output the same set of semantics by unifying the semantic heads and consistency constraints.

[0011] Preferably, the neural network architecture and the hierarchical architecture also share the backbone parameter family and establish a unified distillation mapping relationship; The neural network architecture directly deploys the backbone parameter family to form the overall model, while the hierarchical architecture divides, freezes, prunes, fine-tunes, or loads adapters to form several collaborative sub-models from the backbone parameter family. By employing at least one of state distillation, semantic distillation, state transition distillation, planning distillation, and control distillation, the two architectures are made to remain aligned and transferable in terms of state space and physical semantics.

[0012] Preferably, the system is trained using a joint loss function, which includes at least one of the following losses: Dense physics field loss for supervised physics field construction; object relationship graph loss for supervised object and relationship construction; event prediction loss for supervised model identification and prediction of events; common sense causality loss for supervised causal reasoning; compilation and execution loss to ensure that the execution result is consistent with the prediction of the world model; and security constraint loss for applying security constraints.

[0013] A control method for an embodied intelligence physical world model system, comprising: Acquire multimodal data, the robot's current state, and task objectives; Multimodal data is encoded and normalized to a unified world coordinate system to construct a dense physical field; Abstracting a structured world state containing objects, relationships, and events from a dense physical field; Perform causal reasoning on structured world states; make multi-step predictions for the future based on the current state, action constraints, and task objectives, and generate candidate world semantic actions; The candidate world semantic actions are risk-assessed and constraint-projected, compiled into specific control instructions for the robot, and then executed. Collect feedback after execution, update the status, and enter the next control cycle.

[0014] Preferably, the training method for the physical world model system includes: First, train an end-to-end neural network architecture using a joint loss function, then distill the trained neural network architecture into the various modules of a hierarchical architecture; or First, train the hierarchical architecture, and then train each module in the hierarchical architecture into an end-to-end neural network architecture through joint fine-tuning.

[0015] Preferably, the steps for switching between neural network architecture and layered architecture include: Calculate the preset switching indicators and make hard switching rule judgments. If the hard conditions for layered switching are met, start the layered architecture. If the hard conditions for end-to-end switching are met, start the neural network architecture. If the hard handover rule is not triggered, the comprehensive handover score is calculated based on the handover index. When the comprehensive handover score is greater than or equal to the score threshold, the hierarchical architecture is activated; otherwise, the neural network architecture is activated.

[0016] This invention offers the following advantages: Strong generalization ability of physical properties: By explicitly constructing and supervising dense physical fields, object relationships, and events, the model achieves a stable and generalizable understanding of physical scenes, reducing sensitivity to changes in appearance and perspective; Endogenous common-sense causal reasoning ability: Through causal reasoning modules and corresponding losses, the model can understand and comply with the implicit constraints of the physical world, perform counterfactual inferences, and handle more complex combinatorial tasks; Flexible architecture, balancing efficiency and reliability: Through a dual-structure switchable design, it can achieve low-latency response with an end-to-end neural network architecture, and handle high-risk, long-term complex tasks with a hierarchical architecture. A quantitative mode switcher automates and optimizes switching decisions; Friendly cross-platform deployment: A unified world coordinate system, action semantics, and a two-level action architecture of "world action-ontology compilation" enable the same world model core to adapt to different robot bodies (robotic arms, mobile chassis, humanoid robots, etc.), significantly reducing migration costs; Comprehensive training objectives: Multiple loss functions constitute a complete supervision system from low-level physical perception to high-level safety decision-making, guiding the model to learn a physical world model truly suitable for embodied control. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a physical world model system with embodied intelligence, as described in this invention.

[0018] Figure 2 This is a schematic diagram of the end-to-end neural network architecture in this invention.

[0019] Figure 3 This is a schematic diagram of the layered architecture in this invention.

[0020] Figure 4 This is a flowchart of a control method for an embodied intelligence physical world model system according to the present invention.

[0021] Figure 5 This is a flowchart of the mode switcher's switching judgment in this invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be noted that all data in the present invention were collected under the premise of legality and compliance.

[0023] like Figure 1 As shown, an embodied intelligence physical world model system includes a mode switcher that selects to enable either an end-to-end neural network architecture or a hierarchical architecture based on preset switching criteria; the two architectures share the following common modules: The input encoding module performs unified encoding on multimodal inputs; The coordinate standardization module maps the encoding results to the world coordinate system; The physics field construction module generates several physics fields based on the output of the coordinate normalization module. The relation mapping module abstracts several physical fields into structured world states; The causal reasoning module completes the causal constraints for the structured world state; The prediction module makes predictions based on the current state, action constraints, and task objectives. The action generation module generates world semantic actions based on the prediction results and compiles them into control instructions. The safety correction module performs risk assessment and constraint projection on global semantic actions and outputs the final control command.

[0024] The core of this invention lies in supporting two scalable implementation architectures within the same physical world model, ensuring their essential identity through a unified sharing mechanism, and combining this with a quantitative mode switcher to achieve dynamic switching between neural network architectures or hierarchical architectures based on different task scenarios. At the perception and understanding level, the model utilizes a physics field construction module and a relationship graphing module to build stable internal representations of physical attributes such as scene geometry and interaction relationships, thereby improving the generalization ability of physical attributes. At the reasoning and planning level, by introducing a dedicated causal reasoning module, the model internalizes physical constraint knowledge about preconditions and postconditions, thus enabling it to handle complex combinatorial tasks requiring multi-step logical reasoning. At the system architecture level, a switchable design of end-to-end neural network architecture and hierarchical architecture is adopted, coupled with a mode switcher based on switching metrics, enabling the system to achieve an optimal balance between high real-time direct response and high reliability, taking into account both efficiency and security. At the engineering deployment level, thanks to the unified world coordinate system, world semantic actions decoupled from the ontology, and the two-level compilation architecture of "world actions-ontology instructions", the same model core can be efficiently adapted to robot platforms with different structures (such as robotic arms and mobile chassis), greatly reducing the migration cost from model to actual application scenarios.

[0025] Furthermore, the mode switcher is configured to dynamically select and enable either an end-to-end neural network architecture or a hierarchical architecture based on at least one of the following switching metrics: task time domain, model uncertainty, physical risk, task complexity, and real-time load metrics, combined with preset switching rules.

[0026] Among them, the task time domain index is obtained by multiplying the remaining event chain length and the average duration of the event; the model uncertainty index is obtained by multi-head state prediction variance, candidate action distribution entropy or teacher-student bias; the physical risk index is composed of the collision probability, fall probability, instability probability, over-force probability, human intrusion probability and out-of-distribution probability weighted; the task complexity index is composed of the number of objects, the number of relations, the number of preconditions and the number of event branches; and the real-time load index is obtained by the allowable control budget, the current inference latency and the computing power usage.

[0027] When the task time domain index exceeds the upper limit threshold, the uncertainty index exceeds the upper limit threshold, the physical risk index exceeds the upper limit threshold, the collision probability or human intrusion probability exceeds the corresponding hard upper limit threshold, or the number of preconditions and event branches exceeds the upper limit threshold of complexity, the mode selector control system switches to a hierarchical architecture; when the task time domain, uncertainty, physical risk and task complexity are all below the corresponding lower limit threshold and the control budget meets the low latency requirement, the mode selector control system switches to an end-to-end neural network architecture.

[0028] As a specific implementation, the neural network architecture and the hierarchical architecture share a unified state space and physical semantics, share a backbone parameter family and a unified distillation mapping relationship; thus enabling two engineering implementations of the same physical world model under different deployment conditions.

[0029] To ensure that the neural network architecture and the layered architecture belong to the same physical world model in terms of technical essence, a state space containing several sub-states required for physical world modeling is predefined. The state space includes at least scene geometric state, object state, relation state, event state, embodied state, target state, and memory state. Both the neural network architecture and the layered architecture map the input to the state space through a unified encoding protocol and perform state updates, state queries, and task solving based on a unified read and write interface.

[0030] It's important to note that the state space is not a simple feature vector, but rather contains multiple sub-state components required for physical world modeling. For example, scene geometry sub-states include depth, occupancy, traversable area, supporting surface, and boundary; object sub-states include object instance slots, category, pose, scale, material, and availability; relation sub-states include contact, support, occlusion, constraint, adjacency, and relative motion; event sub-states include temporal events such as grasping, placing, approaching, leaving, collision, and opening / closing; embodied sub-states include robot body pose, joint states, end effector states, and force / contact states; target sub-states include task target, sub-targets, constraints, and reward preferences; and memory sub-states include historical observation summaries, historical action summaries, and long-term environmental memory. In other words, neither architecture directly deals with arbitrary "pixel → action" or "module → module" data; instead, both first encode the input data into the same state space St.

[0031] Specifically, a unified encoding protocol maps different modalities to the same state space. Inputs such as images, videos, depth, haptic feedback, language commands, and robot states first pass through their respective front-end encoders; then, through a unified state projection layer, they are mapped to states with a unified dimension, a unified coordinate system, and a unified temporal index. All state units have fixed semantic positions; for example, slot 1 always represents an object instance, slot 2 always represents a relationship graph, and slot 3 always represents an embodied state. In an end-to-end neural network architecture, St is an intermediate hidden variable within the neural network; in a layered architecture, St is the standard state carrier for exchange between the system layer, the platform agent layer, and the scalable world model layer.

[0032] This invention pre-establishes a unified physical semantic ontology library, which uniformly defines object categories, contact relationships, support relationships, temporal events, action primitives, availability, constraints, and risks, so that the neural network architecture and the hierarchical architecture output the same semantic set with unified semantic heads and consistency constraints.

[0033] Specifically, the physical semantic ontology library includes at least the following: geometric semantics: depth, normal, boundary, reachable region, traversable region; object semantics: object category, component, graspable region, supportable region; relational semantics: contact, support, occlusion, containment, connection, constraint; dynamic semantics: stationary, motion, acceleration, stability, slip tendency; behavioral semantics: grasp, push, pull, rotate, place, obstacle avoidance; and task semantics: target object, target location, priority, risk level.

[0034] In practical implementation, shared physical semantics includes a unified semantic head and semantic label space, namely a unified object detection / object slot head; a unified relation prediction head; a unified event recognition head; a unified availability prediction head; a unified risk / constraint prediction head (regardless of whether it's a neural network architecture or a layered architecture, the same semantic set must ultimately be decoded from St); a unified coordinate system, unit system, and action semantics, unifying the transformation method between the world coordinate system / robot base coordinate system / camera coordinate system; unified units of length, angle, velocity, and force; and unified action semantic encoding, for example, "grasp" is not a specific control pulse, but an action primitive with defined stages such as initiation, contact, closure, and lifting. Semantic consistency constraints are added, and a consistency loss is added during the training phase: under the same data input, the object semantics derived by the two architectures are consistent; relations... Figure 1 Consistent; consistent event labels; consistent risk assessments.

[0035] In terms of model training and deployment, this invention adopts a unified backbone parameter family, which includes multimodal coding parameters, state projection parameters, world model coding parameters, physical semantic decoding parameters, and state transition prediction parameters. The neural network architecture directly deploys the backbone parameter family to form an overall model, while the hierarchical architecture divides, freezes, prunes, fine-tunes, or loads adapters to form several collaborative sub-models.

[0036] Specifically, there are three ways to share backbone parameter families. The first is a shared backbone with independent output headers, meaning the backbone network parameters are shared, and different architectures only differ at the output adaptation layer. The second is a shared backbone with lightweight adapters, meaning the backbone parameters are frozen or partially frozen, and the two architectures load lightweight adaptation parameters such as LoRA / Adapter / Prefix respectively. The third is shared parameter namespace and version inheritance, meaning the same parent model checkpoint is used, and different deployment architectures only select different subgraphs, different pruning depths, or different routing branches.

[0037] This invention establishes a unified distillation mapping relationship between neural network architecture and hierarchical architecture. Through at least one of the following methods, namely state distillation (aligning a unified state space St), semantic distillation (aligning objects, relations, events, availability, and risk outputs), state transition distillation (aligning the prediction results of the state space from time t to time t+1), planning distillation (aligning sub-objective sequences, cost functions, and action trajectories), and control distillation (aligning the final action or action distribution), the two architectures remain alignable and transferable in terms of state space and physical semantics. This enables flexible deployment of the same physical world model under different requirements for computing power, latency, reliability, and interpretability.

[0038] Specifically, when the neural network architecture has large capacity and high performance, it acts as the teacher model, while the modules in the hierarchical architecture act as student models. When the hierarchical architecture has stronger interpretable intermediate semantics, it can also be used as the teacher model, distilling the intermediate physical semantics back to the neural network architecture. Both architectures address the inconsistency between different structural dimensions through mapping heads, including a state mapping head Hs, which maps the teacher's state to the student's state dimension; a semantic mapping head Hm, which maps the teacher's semantics to the student's semantic space; and an action mapping head Ha, which maps the teacher's planning results to the student's control representation. Losses can include: state alignment loss; semantic KL divergence loss; action distillation loss; and temporal rollout consistency loss.

[0039] As a specific example, such as Figure 2 As shown, the neural network architecture includes an input encoding subnetwork, a coordinate normalization subnetwork, a physics field construction subnetwork, a relational graphing subnetwork, a causal reasoning subnetwork, a prediction subnetwork, an action generation subnetwork, and a safety correction subnetwork connected in sequence. Each subnetwork is derived from the corresponding module. The safety correction subnetwork performs risk assessment and constraint projection on the world semantic actions and outputs the final control command to the robot execution end. The feedback data from the execution end flows back to the input encoding subnetwork.

[0040] In this architecture, perceptual encoding, physical state modeling, common-sense causal reasoning, next spatiotemporal prediction, cross-ontology action compilation, and security control are all unified within a neural network computation graph that can be jointly trained, jointly distilled, and adaptively closed-loop. From multimodal input to world state and then to action output, a single differentiable closed loop is formed; the model is not a loosely pieced together "first perception, then rules, then control", but a neural native system "centered on the physical world state"; it is suitable for edge deployment, low-latency inference, fast closed-loop control, and unified training.

[0041] It should be noted that the neural network structure aims to solve the following problems: the traditional perception, planning, and control links are fragmented, and errors cannot be propagated across modules; rule systems are separated from neural networks, and common sense and control cannot be optimized together; multiple sensors and multiple robot bodies are difficult to share unified action semantics; online planning has high overhead and cannot meet the real-time closed-loop requirements of embodied systems. The entire embodied world modeling and control process is unified into a neural native closed loop (ut, Z). t+1:t+H ,Πt,Ωt)=ΦΘ(X 0:t B 0:t (,Gt,Kt), where: X 0:t B represents the multimodal data up to the current moment; 0:t Gt represents the robot's state sequence; Kt represents the current goal or constraint; Z represents common-sense causal memory; t+1:t+HΠt represents the predicted state at several points in the future; Ωt represents the planning result of the event chain or action block; Ωt represents the risk and safety assessment; and ut represents the current actual control output.

[0042] Specifically, the neural network architecture includes the following connection order: Xt, Bt, Gt, M t-1 -N1-N2-N3-N4-N5-N6-N7-N8-ut, where Xt is the current multimodal data; Bt is the current ontology state; Gt is the task objective / constraint; M t-1 : Historical memory state; N1 is the input encoding subnet; N2 is the coordinate normalization subnet; N3 is the physical field construction subnet; N4 is the relational graph construction subnet; N5 is the causal reasoning subnet; N6 is the prediction subnet; N7 is the action generation subnet; N8 is the safety self-correction subnet; ut is the control output.

[0043] In the input encoding subnet, images, videos, depth data, point clouds; force sensing, tactile sensing, IMU data; proprioceptive states (joint angles, end-effector pose, chassis state); historical actions; language commands, target images, target videos, and rule-based text are encoded into a unified input Tt. Images are embedded using 2D patch embedding, videos using 3D tubelet embedding, point clouds using sparse voxel encoding, tactile and force sensing using 1D sequence encoding, and proprioceptive states using vector mapping encoding. After encoding each branch using its own method, the resulting unified input Tt is concatenated.

[0044] The coordinate normalization subnet unifies the inputs from different sensor perspectives, different extrinsic parameters, and different robot bodies into a unified world coordinate system. This subnet performs at least the following transformations: sensor coordinates to world coordinates; body execution coordinates to world coordinates; motion semantic normalization; gravity axis and support direction normalization; and viewpoint change consistency constraints. This ensures that the same grasping action will not experience semantic drift due to changes in camera position. Its output is world system unified data Tt' and world system body state Bt'.

[0045] The physics construction subnet is used to recover multi-scale physics field representations from the outputs Tt' and Bt' of the coordinate normalization subnet. The physics construction subnet includes a shared backbone coding unit, a multi-layer feature extraction unit, a feature fusion unit, a recombination unit, and a parallel physics decoding unit. The shared backbone coding unit performs spatiotemporal encoding on the unified input; the multi-layer feature extraction unit extracts at least two intermediate layer features and one final layer feature; the feature fusion unit concatenates and reduces the dimensionality of the multi-layer features; the recombination unit restores the fused sequence features to spatial grid features; and the parallel physics decoding unit outputs depth fields, occupancy fields, contact probability fields, support relationship fields, slip risk fields, graspability fields, drivability fields, occlusion fields, stability fields, and containment fields at at least one or more scales. This subnet does not perform general semantic segmentation but generates a distribution of physical attributes that directly and effectively control the embodiment. Its output is a combination of multiple physics fields, Ft.

[0046] Relational graph subnetting is constructed based on the input (Ft, Tt', Bt', M). t-1 This process further abstracts the continuous physical field into a structured world state, including object slots, object relationship graphs, event tokens, and preconditions and postconditions. For example, a cup-tabletop relationship represents support; a cup-gripper relationship represents contact / grasping; a drawer-cabinet relationship represents closure; and an object-container relationship represents a candidate object to be placed in. First, object slots Ot are extracted based on the spatial connectivity, boundary consistency, motion consistency, and physical attribute similarity of the continuous physical field. Then, an object relationship graph Rt is established based on the contact, support, occlusion, containment, adjacency, and relative motion relationships between objects. Next, event boundary recognition Et is performed using historical memory. Finally, the structured world state Zt = {Ot, Rt, Et} is output.

[0047] The causal reasoning subnet is based on the input (Ot, Rt, Et, Gt, M). t-1 The algorithm completes the causal constraints of the current object relationship state by retrieving a template from a commonsense template library that matches the current relationship state. Each template includes at least a precondition, an action, a post-result, and a taboo constraint. The current object relationship state is then matched with the retrieved template to obtain the currently executable event, the currently missing precondition, the expected post-result, and the taboo actions that need to be masked. This can be achieved using a neural causal memory unit; template retrieval attention; a graph neural causal propagator; and a counterfactual differential prediction head. The final output causal constraint vector Kt contains at least precondition constraints, postcondition constraints, and taboo constraints.

[0048] The prediction subnet is based on the input (Ft, Ot, Rt, Et, Kt, Bt', Gt, M). t-1The system makes predictions by sharing a temporal backbone and using parallel multi-scale prediction heads. Through a micro-dynamics head, it predicts instantaneous contact, local displacement, and changes in local stability; a meso-event head predicts changes in events such as grasping, placing, opening doors, bypassing obstacles, and avoiding collisions; and a macro-task phase head predicts the progress of sub-tasks, target distance, and global success probability. Finally, it outputs the micro-future state, the meso-future event chain, the macro-task phase progression, and the value / risk prior of candidate actions.

[0049] The motion generation subnet generates unified world semantic actions based on the output of the prediction subnet and Zt and Bt', and then compiles them into specific robot control instructions. World semantic actions can be represented as world-system displacement, rotation, contact intention, gripping state, support establishment / release, dual-arm cooperative relationship, and combined movement and manipulation action blocks. Subsequently, compilation is completed based on the robot ontology description vector, outputting joint trajectories, chassis speed, gripper opening and closing, torso balance compensation, etc., and outputting candidate original control instructions ut'. The motion generation subnet simultaneously feeds back motion candidates to the prediction subnet for counterfactual reasoning. This invention does not directly output the low-level control actions of a certain robot ontology, but first generates world semantic actions under unified world coordinates and unified physical semantics, and then compiles the world semantic actions into specific robot control instructions based on ontology parameters, thereby realizing the reuse of the same world model for different ontology.

[0050] The safety correction subnet performs risk assessment, constraint projection, and error correction on the action based on the input (ut', Ft, Rt, Kt), and finally outputs an executable control command ut. It includes at least a collision prediction head, an over-torque risk head, a human / sensitive area avoidance head, an execution deviation compensation head, and a failure recovery action head.

[0051] The data flow of a neural network structure includes input multimodal data Xt, ontology state Bt, target Gt, and historical memory M. t-1 .

[0052] Step 1: Xt, Bt, and Gt are input into the input encoding subnet N1, and visual token, depth token, tactile token, ontology token, and target token are obtained through parallel encoders of each modality. These tokens are then stitched together by a unified projection layer to form a unified observation token sequence Tt.

[0053] Step 2: Tt, together with camera extrinsic parameters, ontological extrinsic parameters, and IMU gravity information, is input into the world coordinate normalization subnet N2, and the normalized token sequence Tt' and normalized ontological state Bt' under the unified world coordinate system are output.

[0054] Step 3: Tt' and Bt' are input into the dense physical field to construct subnet N3. The hidden state Ht is extracted through the shared physical backbone network, and the depth field, occupation field, contact field, support field, graspability field and stability field Ft are output by multiple parallel physical heads respectively.

[0055] Step 4, Ft, Tt', Bt' and historical memory M t-1 Input the relational graph subnet N4, and generate the object slots Ot, relational graph Rt, and events Et in sequence.

[0056] Step 5: Ot, Rt, Et, target Gt, and historical memory M t-1 Input the causal reasoning subnet N5, and generate the causal constraint vector Kt after fusing the memory retrieval branch and the graph reasoning branch.

[0057] Step 6, Ft, Ot, Rt, Et, Kt, Bt', Gt and M t-1 The common input is the hierarchical spatiotemporal prediction subnet N6, which generates future microstates, future event chains, and future task phases.

[0058] Step 7: Input the current state Zt, future prediction results and causal constraints Kt into the world action generation and ontology action compilation subnet N7, first generate the unified world action At, and then compile it into the original control instruction ut' based on the robot ontology description vector; Step 8: ut', together with the physical field Ft, the relationship graph Rt, the causal constraint Kt, and the future prediction results, is input into the safety correction subnet N8. After risk assessment, constraint projection, and execution correction, the final control command ut is obtained.

[0059] Step 9: ut is sent to the robot actuator, and the robot feeds back multi-dimensional data and body state at the next time step t+1, which is then re-inputted into N1 to form a closed loop.

[0060] As a specific embodiment, the system of the present invention is trained using a joint loss function, which includes at least one of the following losses: Dense physics field loss for supervised physics field construction; object relationship graph loss for supervised object and relationship construction; event prediction loss for supervised model identification and prediction of events; common sense causality loss for supervised causal reasoning; compilation and execution loss to ensure that the execution result is consistent with the prediction of the world model; and security constraint loss for applying security constraints.

[0061] Specifically, the joint loss function can be expressed as a weighted sum of the individual loss functions, i.e., L = λ1*Ltok + λ2*Lcoord + λ3*Lfield + λ4*Lgraph + λ5*Levent + λ6*Lcausal + λ7*Lrollout + λ8*Laction + λ9*Lsafety + λ10*Ldistill. Where Ltok is the representation learning loss; Lcoord is the coordinate invariance loss; Lfield is the dense physics field loss; Lgraph is the object graph loss; Levent is the event prediction loss; Lcausal is the commonsense causality loss; Lrollout is the multi-step state rolling loss; Laction is the compilation and execution loss; Lsafety is the safety constraint loss; and Ldistill is the teacher-student distillation loss.

[0062] The representation learning loss is used to learn a unified representation. It can be directly obtained from the dense predictive loss of V-JEPA 2.1, that is, Ltok is Lpredict + Lctx, where Lpredict is the L1 prediction loss for the masked token, and Lctx is the weighted L1 loss for the visible / context token.

[0063] Coordinate invariance loss is used to constrain the mapping of the same physical scene to a consistent state under different camera views and different body coordinates. It can be defined as the state difference of the same scene after two coordinate transformations, where Lcoord is ||Γ(St)-S||. t，ref ||1, where Γ represents the normalized state of the coordinates, S t，ref This is for reference to the world coordinate system.

[0064] The dense physics loss Lfield is used to monitor the outputs of depth field, occupancy field, contact field, support field, and graspability field, and can be expressed as a weighted sum of the losses corresponding to each physics field. Each physics field loss can be L1, BCE, or focal loss.

[0065] Object relationship graph loss is used to supervise object slots and the edges of relationships between objects. That is, Lgraph is Lobj + Lrel, where Lobj is the object location / category / attribute loss; Lrel is the relationship classification loss such as contact, support, occlusion, containment, etc. Hungarian matching + classification / regression or graph edge BCE / CE can be used.

[0066] Event prediction loss is used to supervise events such as contact establishment, grip establishment, placement completion, and door opening completion. Specifically, Levent is Leventcls + Leventtime, where Leventcls is the event category cross-entropy, and Leventtime is the regression loss for event occurrence time or duration.

[0067] Common sense causality loss is used to supervise preconditions, postconditions, and taboo constraints. That is, Lcausal is Lpre + Lpost + Lforbid, where Lpre is the BCE of whether the precondition is satisfied; Lpost is the L1 / CE of the post-execution state and the expected post-state; and Lforbid is the taboo action mask loss.

[0068] Multi-step state rolling loss is an existing technique. Lrollout uses the predictor's output as input to train multi-step future predictions. Teacher-student distillation loss is also an existing technique. Ldistill not only distills the last layer representation, but also the state space, semantic head, and action head.

[0069] The compilation and execution losses are used to ensure that the execution result of the world semantic action, after being compiled into an ontology action, is consistent with the prediction of the world model. That is, Laction is the sum of the compilation loss from the world semantic action to the ontology action and the state consistency loss after the ontology action is executed. Its core is to make the two-level structure of "unified world semantic action - ontology action compilation" trainable.

[0070] Safety constraint loss is used to penalize dangerous actions. That is, Lsafety is αLcol + βLforce + γLhuman + δLunstable. These correspond to: collision risk, excessive torque / force risk, intrusion risk into the human safety zone, and fall / instability risk. If a constrained projector is used, an additional term for motion deviation before and after projection is added.

[0071] As a specific implementation, the layered architecture includes: a system layer that provides a unified data and controlled execution environment for intelligent agents and world models, and provides basic capabilities such as data, computing power, digital twins, security constraints, and edge-cloud deployment; The platform's intelligent agent layer acts as a bridge between tasks and world model invocations, calling the scalable world model layer to complete tasks. It is responsible for task understanding, sub-goal generation, common sense invocation, skill orchestration, and strategy scheduling. The extensible world model layer includes an input encoding module, a coordinate normalization module, a physics field construction module, a relationship graphing module, a causal reasoning module, a prediction module, an action generation module, and a safety correction module.

[0072] In this invention, the physical world model is not directly equivalent to the entire system, but rather serves as the core of the system's underlying physical reasoning, invoked by the upper-level platform agents, and supported by data, computing power, and security infrastructure provided by the system layer. In addition to the aforementioned three-layer architecture, one or more of the following can be added: a security supervision layer, a knowledge service layer, a robot execution and feedback layer, an edge collaboration layer, or a multi-machine collaboration layer.

[0073] like Figure 3 As shown, this layered architecture, from top to bottom, includes a system layer, a platform intelligent agent layer, a scalable world model layer, and a robot execution and feedback layer. The system layer provides data management, training deployment, digital twin, and security constraint capabilities; the platform intelligent agent layer is used to complete task understanding, sub-goal decomposition, common sense binding, and skill routing; the scalable world model layer is used to complete multimodal encoding, coordinate normalization, dense physics field construction, object relationship event modeling, causal reasoning, spatiotemporal prediction, world semantic action generation and ontology action compilation, and the generation of final control commands; the robot execution and feedback layer (which can be a robotic arm, mobile chassis, dual-arm, humanoid robot, or other embodied intelligent robot) is used to receive the final control commands and execute actions, while simultaneously transmitting perceptual feedback and execution results back to the upper-layer modules, forming a closed loop.

[0074] In this invention, the system layer is responsible for providing basic conditions, rather than directly performing task inference. It includes at least: (1) a multi-source data engine, responsible for aggregating Internet images / videos, robot operation data, sensor logs, digital twin data, rule texts and process documents, scene maps and historical trajectories. (2) a training and deployment engine, responsible for model pre-training, post-training, distillation, quantization, edge-cloud distribution, parameter version management; and A / B deployment switching. (3) a digital twin engine, responsible for automatically generating material change samples, occlusion change samples, instability samples, door opening failure samples, grasping slip samples, navigation obstacle avoidance samples, and intervention action samples. (4) a safety engine, responsible for constraining restricted areas, speed limits, torque limits, human-machine co-operation safety zones, executing emergency stop strategies, and storing execution logs.

[0075] The platform's intelligent agent layer includes task understanding intelligent agents, subtask orchestration intelligent agents, common sense binding intelligent agents, and skill routing intelligent agents.

[0076] The task-understanding agent is an additional component built upon existing video representation pre-training, language alignment, and sub-objective planning. It first utilizes a pre-trained video encoder to obtain spatiotemporal representations, then trains the task-understanding head through language / objective supervision. This maps the input to task tokens, sub-objective sequences, success criteria, and constraints. Furthermore, it fine-tunes the planning consistency using robot trajectories or teaching data. If the same task is broken down into multiple stages in the trajectory, the stage boundaries supervise the sub-objective sequence, the task completion state supervises the success criteria, and unexecutable or failed trajectories supervise the constraints.

[0077] The subtask orchestration agent is used to break down long tasks into a chain of phased events. It generates event chain labels by using pre-trained world representations and automatically segmenting phase boundaries from taught or successful trajectories according to state changes and target switching positions. Then, it trains a sequence prediction head or event graph prediction head with "current task description + current state + historical phase" as input and "next phase subtask / event chain" as supervision, thereby learning the ability to break down long tasks into a chain of phased events.

[0078] The common sense binding agent first retrieves applicable causal templates based on the current object slots, relationship graph, event state, and task goal (the templates must contain at least preconditions, actions, post-results, and taboo constraints). Then, based on the object category, relative position, contact / support relationship, availability features, and current task goal, it matches the role slots in the templates with the current object slots one by one. For successfully matched templates, it outputs the bound preconditions, postconditions, candidate events, and taboo constraints for subsequent causal reasoning and planning.

[0079] The skill-routing agent reads the current ontology description vector (including at least ontology type, number of joints, workspace, end effector type, mobility, control frequency, and safety constraints), task objective, and current state. It then filters candidate skills compatible with the ontology from a predefined skill library (including at least arrival, grasping, placement, door opening, push-pull, navigation, dual-arm collaboration, and failure recovery skill strategies, each with an applicable condition label). The routing network outputs scores for each candidate skill, selecting the execution strategy with the highest score as the current ontology's control strategy. During training, "task-onlogy-action segment" triples are extracted from the robot's trajectory. Successfully executed action segments are mapped to skill labels, and the routing network is trained using supervised learning. If the same task uses different skills on different ontologists, the labels are distinguished by the ontology description vector. Alternatively, task success rate can be used as feedback to refine and enhance the routing network.

[0080] The scalable world model layer is the core layer in the layered architecture. Its modules share the same origin as the various subnetworks in the neural network architecture and can be decomposed into independent services or independent containers. It includes at least: an input encoding module, a coordinate normalization module, a physics field construction module, a relationship graphing module, a causal reasoning module, a prediction module, an action generation module, and a safety correction module. Scalability means that tasks are scalable: from grasping to placement, navigation, opening and closing doors, assembly, and dual-arm collaboration; the ontology is scalable: from single-arm to dual-arm, mobile robots, and humanoid robots; modalities are scalable: from RGB to RGB-D, force sensing, tactile sensing, language, and mapping; and deployment is scalable: from edge-device integrated machines to edge-cloud collaboration and multi-robot sharing.

[0081] The specific functions and methods of each module in this embodiment are consistent with the corresponding subnets in the neural network architecture, and therefore will not be described in detail. For the action generation module, actions are first generated in the unified world semantic space, and then compiled according to the ontology description. This is an important module for the cross-platform generalization of this invention.

[0082] First, in a unified world coordinate and unified physical semantic space, based on the current structured state, target state, and future prediction results, world semantic actions (including at least position increments, posture increments, gripping / releasing states, contact intentions, or support establishment / de-establishment states) are generated, decoupled from the robot body. Then, the world semantic actions are compiled in conjunction with the body description vector (specifically, the world coordinate actions are transformed to the body coordinate system, the end-effector trajectory or joint trajectory is solved, and the corresponding control variables are generated according to the body type), and converted into specific robot control commands (e.g., outputting joint angle increments or end-effector pose trajectories for robotic arms; outputting linear velocity / angular velocity for mobile chassis; outputting opening and closing commands for grippers; and outputting dual-arm / whole-body coordinated trajectories for humanoid robots), thereby enabling the reuse of the same world model for different robot platforms.

[0083] The existing V-JEPA 2 has demonstrated that freezing the video encoder and training the motion condition predictor enables closed-loop planning in the latent representation space; however, its published content also clearly exposes two types of bottlenecks: one is camera position sensitivity due to the lack of explicit coordinate normalization; the other is long-term temporal error accumulation due to autoregressive rollout and short-term target-driven approaches. V-JEPA 2.1 further explains that when predicting only masked tokens, high-quality dense spatio-temporal features do not emerge stably. Only by extending the loss to all tokens and superimposing deep self-supervision and a modality-specific tokenizer can dense features be significantly improved, and the grasping and navigation planning capabilities are further enhanced. Therefore, this invention, by unifying "coordinate normalization + dense physics field + object relationship events + common-sense causality" into the same state space, can fundamentally enhance the modeling capabilities for geometry, contact, support, occlusion, and availability, thus making it more suitable for embodied intelligent world models with physical property generalization capabilities.

[0084] Furthermore, compared to the VLA approach that only directly outputs actions, or the worldfoundation model approach that focuses on generating future videos, this invention does not primarily consume computational power on action fitting or video reconstruction. Instead, it concentrates computation on the physical variables truly needed for embodied control through "next spatiotemporal state prediction + event-level planning + world semantic action compilation + mode switching." Therefore, structurally, this invention, on the one hand, can more easily explain "why the next step should be done" compared to the pure VLA approach, and on the other hand, can more easily reduce online planning latency and cross-platform migration costs compared to the video generation-heavy approach.

[0085] In addition to providing an embodied intelligence physical world model system, this invention also provides a control method for the embodied intelligence physical world model system, comprising: Acquire multimodal data, the robot's current state, and task objectives; Multimodal data is encoded and normalized to a unified world coordinate system to construct a dense physical field; Abstracting a structured world state containing objects, relationships, and events from a dense physical field; Perform causal reasoning on structured world states; make multi-step predictions for the future based on the current state, action constraints, and task objectives, and generate candidate world semantic actions; The candidate world semantic actions are risk-assessed and constraint-projected, compiled into specific control instructions for the robot, and then executed. Collect feedback after execution, update the status, and enter the next control cycle.

[0086] Training methods for physical world model systems include: First, train an end-to-end neural network architecture using a joint loss function, then distill the trained neural network architecture into the various modules of a hierarchical architecture; or First, train the hierarchical architecture, and then train each module in the hierarchical architecture into an end-to-end neural network architecture through joint fine-tuning.

[0087] Specifically, such as Figure 4 The overall training process shown includes: Collect data from multiple sources, including internet videos, images, robot interaction data, digital twin data, rule text, and multimodal sensor data; Modal cleaning and alignment are performed, including time synchronization, scene deduplication, viewpoint alignment, robot body identification, and pseudo-label construction. Basic representation pre-training: Perform full-token self-supervised training on images and videos to obtain unified visual-spatiotemporal basic coding capabilities; Physics field learning applies supervision or weak supervision to physical properties such as depth, contact, support, occupation, occupancy, and gripability. Object-relation event learning trains continuous physical field state sequences into object slots, relation edges, and event boundaries; Common sense causal learning uses rule samples, intervention samples, and counterfactual samples to train preconditions, postconditions, and taboo constraints. Ontology action compilation and learning: training the mapping from unified world semantic actions to different robot ontology actions; End-to-end joint alignment, in the same computation graph, jointly optimizes world state, event prediction, action output, and security loss; Layered deployment distillation distills the end-to-end neural network architecture into modular sub-models suitable for layered invocation, or conversely, feeds the knowledge of the sub-modules of the layered architecture back into the end-to-end neural network architecture. Mode switching training trains a mode switcher, enabling the system to automatically select a neural network architecture or a hierarchical architecture based on complexity and risk.

[0088] The specific reasoning process of neural network architecture has already been implemented. Figure 4 The text explicitly states that the reasoning should proceed step-by-step according to the specific functions of each subnet in the aforementioned system; therefore, a detailed explanation is omitted. The specific reasoning process for the layered architecture is as follows: The system layer completes data access, environment state caching, and security boundary initialization; The platform's intelligent agent layer analyzes the task and generates candidate sub-targets; Invoke the scalable world model layer to perform spatiotemporal predictions of the current state and candidate sub-objectives; The scalable world model layer outputs physically feasible event chains, failure risks, and candidate action blocks; The platform's intelligent agent layer reorders candidate world semantic actions based on target benefits, risks, and the skill library; The action generation module converts unified world semantic actions into ontology actions; The system-level safety engine issues control commands after performing constraint projection. The robot execution and feedback layer performs execution feedback and then returns to the platform intelligent agent layer and the scalable world model layer for updates.

[0089] Specifically, the steps by which the platform's intelligent agent layer reorders semantic actions in the world include: Generate a set of candidate execution strategies. The skill routing agent first outputs several candidate skills / strategies {π1,π2,...,πn}. Calculate three types of scores respectively. For each candidate strategy πi, calculate the target benefit score Vi, which is the predicted value of the target completion after executing the strategy; the risk score Ri, which is the weighted value of risks such as collision, fall, instability, and excessive force; and the skill matching score Ki, which is the degree of matching between the strategy and the current entity, the current task, and the current state. Calculate the overall ranking score, and calculate the weighted score as αVi-βRi+γKi, where α, β, and γ are weighting coefficients; Sort and select by score: Sort all candidate strategies by score from highest to lowest, and select the strategy with the highest score as the current execution strategy; if the risk of the highest score strategy exceeds the threshold, discard it and select the next candidate strategy.

[0090] The system-level safety engine performs constraint projection as follows: After receiving the original control command, it first constructs a set of safe executable commands based on the following constraints: motion amplitude constraints (limiting single-step displacement, rotation, velocity, and gripper changes to not exceed thresholds), joint actuator constraints (limiting joint angle, angular velocity, torque, and current to not exceed the allowable range of the body), spatial safety constraints (the end-effector trajectory must not enter restricted areas, collision areas, or human safety zones), contact safety constraints (when the predicted contact force, fall risk, or instability risk exceeds the threshold, the motion amplitude is reduced or execution is prohibited), and timing consistency constraints (the current motion must not conflict with the motion that was not completed at the previous moment). Then, the original control command is projected onto this set of safe executable commands, and the safe control command closest to the original control command is issued for execution. If no control command meets the conditions, deceleration, stopping, or replanning is triggered.

[0091] As a specific example, such as Figure 5 As shown, the steps for switching between neural network architecture and layered architecture include: Calculate the preset switching indicators and make hard switching rule judgments. If the hard conditions for layered switching are met, start the layered architecture. If the hard conditions for end-to-end switching are met, start the neural network architecture. If the hard handover rule is not triggered, the comprehensive handover score is calculated based on the handover index. When the comprehensive handover score is greater than or equal to the score threshold, the hierarchical architecture is activated; otherwise, the neural network architecture is activated.

[0092] Specifically, the switching metrics include task time-domain metrics, model uncertainty metrics, physical risk metrics, task complexity metrics, and real-time load metrics.

[0093] The task time-domain indicator Ht is used to determine whether the current task is a "short-time closed-loop task" or a "long-time multi-stage task". Let Ne be the remaining shortest event chain length estimated from the world model for the current task, and Δte be the average duration of a single event, then the task time-domain indicator Ht is Ne*Δte; alternatively, at the control step level, it is defined as the number of future planning steps Na multiplied by the control period Δt.

[0094] The switching rule is: when Ht ≤ the lower time-domain threshold (e.g., 2s), the task is determined as a short-time task, and the end-to-end neural network architecture is preferentially adopted; when Ht > the upper time-domain threshold (e.g., 8s), the task is determined as a long-time task, and the layered architecture is forcibly adopted; when the lower time-domain threshold < Ht ≤ the upper time-domain threshold, the task is determined as a medium-time task, which needs to be judged in combination with other indicators.

[0095] The model uncertainty indicator Ut is used to judge whether the current state prediction is sufficiently credible. The future state S output by M parallel prediction heads can be selected t+1 to calculate the variance to obtain Ut, or the entropy is calculated on the softmax score distribution pk of the first K candidate action blocks to obtain Ut, or the teacher-student deviation is calculated from the prediction output of the neural network architecture and the prediction output of the layered architecture to obtain Ut. Then normalization is performed.

[0096] The switching rule is: when Ut ≤ the lower uncertainty threshold (e.g., 0.20), it is low uncertainty, and the neural network architecture is allowed to be retained; when the lower uncertainty threshold < Ut ≤ the upper uncertainty threshold, it is medium uncertainty, which needs to be judged in combination with other indicators; when Ut > the upper uncertainty threshold (e.g., 0.45), it is high uncertainty, and the switch to the layered architecture is forced.

[0097] The physical risk indicator Rt is composed of multiple sub-items in a weighted manner, and Rt is w1*Pcol + w2*Pdrop + w3*Punstable + w4*Pforce + w5*Phuman + w6*Poutdist, where Pcol is the predicted collision probability within T future steps (given by the overlap rate between the future trajectory and the obstacle occupancy field); Pdrop is the probability of the target object falling; Punstable is the probability of support instability / slippage (calculated from the proportion of the center of gravity projection exceeding the support polygon or the mean value of the slippage risk field); Pforce is the risk probability of actuator over-torque / over-force (given by the probability that the predicted torque or end contact force exceeds the threshold); Phuman is the probability of human body approaching or sensitive area intrusion (increases when the minimum distance between the human body key point or safety area and the future trajectory of the robot is lower than the threshold); Poutdist is the probability that the action exceeds the training distribution (determined by the nearest neighbor distance between the current state and the training library).

[0098] The switching rules are as follows: when Rt < the lower risk threshold (e.g., 0.25), it is considered low risk, and the neural network architecture can be retained; when the lower risk threshold ≤ Rt < the upper risk threshold (e.g., 0.55), it is considered medium risk, requiring a joint judgment based on other indicators; when Rt ≥ the upper risk threshold, it is considered high risk, and a forced switch to a hierarchical architecture is imposed. Simultaneously, hard thresholds are set, and if any one of them is met, a forced switch to a hierarchical architecture is imposed: Pcol ≥ collision probability threshold (e.g., 0.10); or Pdrop ≥ fall probability threshold (e.g., 0.15); or Phuman ≥ intrusion probability threshold (e.g., 0.05); or Pforce ≥ excessive force probability threshold (e.g., 0.10).

[0099] The task complexity metric Ct is used to quantitatively determine whether a task is a simple single-step closed loop or a complex task involving multiple preconditions, multiple event chains, and multiple object constraints. Ct is denoted as α1*Nobj + α2*Nrel + α3*Npre + α4*Nbranch, where Nobj is the number of objects currently participating in the task; Nrel is the number of relationships that need to be explicitly maintained; Npre is the number of preconditions for the current candidate event; and Nbranch is the number of branches in the event graph.

[0100] The switching rule is as follows: if Nobj ≤ the lower limit threshold for the number of objects (e.g., 2), Npre ≤ the lower limit threshold for the number of preconditions (e.g., 1), and Nbranch = 1, it is considered low complexity, and a neural network architecture is allowed; if Nobj ≥ the upper limit threshold for the number of objects (e.g., 4), or Npre ≥ the upper limit threshold for the number of preconditions (e.g., 3), or Nbranch ≥ the upper limit threshold for the number of branches (e.g., 2), it is considered high complexity, and a layered architecture is directly adopted. For example, "grabbing a single cup on the table and lifting it" is usually low complexity; "opening the cabinet door, grabbing, moving it in, placing it securely, and closing the door" is high complexity.

[0101] The real-time load metric Lt is used to avoid slow hierarchical inference or insufficient accuracy of the neural network architecture. Lt is defined as β1*Thier / Tbudget+β2*ρgpu, where Tbudget is the allowable latency for the current control cycle, Te2e is the estimated inference time for the neural network architecture, Thier is the estimated inference time for the hierarchical architecture, and ρgpu is the current load rate of the device.

[0102] The switching rules are as follows: if Tbudget ≤ the lower latency threshold (e.g., 100ms), the neural network architecture is preferred; if Tbudget > the upper latency threshold (e.g., 500ms), the hierarchical architecture is allowed; if ρgpu > the upper load rate threshold (e.g., 0.85), the hierarchical architecture is prohibited, and the neural network architecture is enabled instead.

[0103] For hard handover rule judgment, the hierarchical hard handover conditions are: Ht > time domain upper limit threshold; or Ut > uncertainty upper limit threshold; or Rt ≥ risk upper limit threshold; or Pcol ≥ collision probability threshold; or Phuman ≥ intrusion probability threshold; or Npre ≥ upper limit threshold for the number of preconditions or Nbranch ≥ upper limit threshold for the number of branches. The end-to-end hard handover conditions are: Ht ≤ time domain lower limit threshold; and Ut ≤ uncertainty lower limit threshold; and Rt < risk lower limit threshold; and Nobj ≤ lower limit threshold for the number of objects; and Npre ≤ lower limit threshold for the number of preconditions; and Tbudget ≤ lower limit threshold for latency.

[0104] If no hard handover rule is triggered, the comprehensive handover score Qt is calculated as γ1*Ht' plus γ2*Ut plus γ3*Rt plus γ4*Ct' minus γ5*Lt', where Ht' is the normalized value of the task's time-domain metric; Ut is the model uncertainty metric; Rt is the physical risk metric; Ct' is the normalized value of the task complexity metric; Lt' is the normalized value of the real-time load metric; and γ1 to γ5 are weights, which can be selected as γ1 = 0.20, γ2 = 0.25, γ3 = 0.30, γ4 = 0.15, and γ5 = 0.10. If Qt ≥ the score threshold (e.g., 0.50), a hierarchical architecture is adopted; if Qt < the score threshold, a neural network architecture is adopted.

[0105] Example 1: Applicable to low-latency, high-frequency closed-loop grasping scenarios, it starts a neural network architecture for single-arm grasping.

[0106] The system adopts an end-to-end neural network architecture; The inputs are an RGB image, gripper state, end effector pose, and target image; The model simultaneously performs physics reconstruction, causal judgment, action generation, and safety correction within a single neural graph.

[0107] Example 2: Utilizing the interpretability and long-duration task capability of the layered architecture, perform the task of "opening the cabinet door and putting in an object".

[0108] The system adopts a layered architecture; The platform's intelligent agent layer first breaks down the task into sub-events such as opening the door, grabbing, moving in, and placing. The scalable world model layer determines that "the event of placing items inside the cabinet is not allowed if the cabinet door is not open"; The system layer is responsible for security and execution.

[0109] Example 3: Mobile operation in a hybrid mode that can switch between neural network architecture and hierarchical architecture.

[0110] The navigation phase employs a hierarchical architecture for global sub-objective planning. Upon reaching the control panel, it switches to an end-to-end neural network architecture to complete near-field grasping and placement.

[0111] Example 4: Humanoid robot with both hands working together to perform complex tasks such as sorting, assembly, etc.

[0112] The platform's intelligent agent layer is responsible for the division of labor between the two hands and the timing arrangement; The scalable world model layer is responsible for predicting object relationships, contacts, and stability. The motion generation module is responsible for mapping unified world motions to the trajectory of a human body.

[0113] The above embodiments are further elaborations and descriptions of the present invention to facilitate understanding, and are not intended to limit the present invention in any way. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A physical world model system with embodied intelligence, characterized in that, Includes a mode switcher that, based on preset switching metrics, selects to enable either an end-to-end neural network architecture or a hierarchical architecture; both architectures share the following common modules: The input encoding module performs unified encoding on multimodal inputs; The coordinate standardization module maps the encoding results to the world coordinate system; The physics field construction module generates several physics fields based on the output of the coordinate normalization module. The relation mapping module abstracts several physical fields into structured world states; The causal reasoning module completes the causal constraints for the structured world state; The prediction module makes predictions based on the current state, action constraints, and task objectives. The action generation module generates world semantic actions based on the prediction results and compiles them into control instructions.

2. The embodied intelligence physical world model system according to claim 1, characterized in that, The neural network architecture includes, in sequence, an input encoding subnetwork, a coordinate normalization subnetwork, a physics field construction subnetwork, a relation graphing subnetwork, a causal reasoning subnetwork, a prediction subnetwork, an action generation subnetwork, and a security correction subnetwork; each subnetwork is derived from the corresponding module. The safety correction subnetwork performs risk assessment and constraint projection on world semantic actions, and outputs the final control command to the robot execution end, while the feedback data from the execution end flows back to the input encoding subnetwork.

3. The embodied intelligence physical world model system according to claim 1, characterized in that, The layered architecture includes: a system layer that provides unified data and a controlled execution environment for agents and world models; The platform's intelligent agent layer acts as a bridge between tasks and world model calls, invoking the extensible world model layer to complete tasks. The extensible world model layer includes an input encoding module, a coordinate normalization module, a physics field construction module, a relationship graphing module, a causal reasoning module, a prediction module, an action generation module, and a safety correction module.

4. A physical world model system of embodied intelligence according to claim 1, 2, or 3, characterized in that, The mode switcher is configured to dynamically select and enable an end-to-end neural network architecture or a hierarchical architecture based on at least one switching metric among task time domain, model uncertainty, physical risk and task complexity, combined with preset switching rules.

5. A physical world model system of embodied intelligence according to claim 1, 2, or 3, characterized in that, The neural network architecture and the hierarchical architecture share a unified state space and physical semantics; A state space containing several sub-states required for modeling the physical world is predefined. Both neural network architectures and hierarchical architectures map the input to the state space through a unified encoding protocol. A unified physical semantic ontology library is pre-established to ensure that the neural network architecture and the hierarchical architecture output the same set of semantics by unifying the semantic heads and consistency constraints.

6. The embodied intelligence physical world model system according to claim 5, characterized in that, The neural network architecture and the hierarchical architecture also share the backbone parameter family and establish a unified distillation mapping relationship; The neural network architecture directly deploys the backbone parameter family to form the overall model, while the hierarchical architecture divides, freezes, prunes, fine-tunes, or loads adapters to form several collaborative sub-models from the backbone parameter family. By employing at least one of state distillation, semantic distillation, state transition distillation, planning distillation, and control distillation, the two architectures are made to remain aligned and transferable in terms of state space and physical semantics.

7. The embodied intelligence physical world model system according to claim 1, characterized in that, The system is trained using a joint loss function, which includes at least one of the following losses: Dense physics field loss for supervised physics field construction; object relationship graph loss for supervised object and relationship construction; event prediction loss for supervised model identification and prediction of events; common sense causality loss for supervised causal reasoning; compilation and execution loss to ensure that the execution result is consistent with the prediction of the world model; and security constraint loss for applying security constraints.

8. A control method for an embodied intelligence physical world model system, applicable to the physical world model system as described in any one of claims 1-7, characterized in that, include: Acquire multimodal data, the robot's current state, and task objectives; Multimodal data is encoded and normalized to a unified world coordinate system to construct a dense physical field; Abstracting a structured world state containing objects, relationships, and events from a dense physical field; Perform causal reasoning on structured world states; make multi-step predictions for the future based on the current state, action constraints, and task objectives, and generate candidate world semantic actions; The candidate world semantic actions are risk-assessed and constraint-projected, compiled into specific control instructions for the robot, and then executed. Collect feedback after execution, update the status, and enter the next control cycle.

9. The control method for an embodied intelligence physical world model system according to claim 8, characterized in that, The training methods for the physical world model system include: First, train an end-to-end neural network architecture using a joint loss function, then distill the trained neural network architecture into the various modules of a hierarchical architecture; or First, train the hierarchical architecture, and then train each module in the hierarchical architecture into an end-to-end neural network architecture through joint fine-tuning.

10. A control method for an embodied intelligence physical world model system according to claim 8 or 9, characterized in that, The steps for switching between neural network architectures and layered architectures include: Calculate the preset switching indicators and make hard switching rule judgments. If the hard conditions for layered switching are met, start the layered architecture. If the hard conditions for end-to-end switching are met, start the neural network architecture. If the hard handover rule is not triggered, the comprehensive handover score is calculated based on the handover index. When the comprehensive handover score is greater than or equal to the score threshold, the hierarchical architecture is activated; otherwise, the neural network architecture is activated.

Images