A PHYSICAL REALITY-SUPPORTED ROBOTIC CONTROL SYSTEM

Abstract

This invention relates to an AI-powered robotic control system (1) that enables robots to apply the movements they have learned in a simulation environment more safely, consistently and successfully in the real world.

Description

1 TARIFF A ROBOTIC CONTROL SUPPORTED BY PHYSICAL REALITY SYSTEM Technical Area This invention allows robots to perform movements in the real world that they have learned in a simulation environment. artificial intelligence that enables it to be implemented in a safer, more consistent and successful way It relates to an assisted robotic control system. Previous Technique With the current technology, a robot model trained in a simulation environment is then brought into the real world. The aim is to resolve the "reality gap" issue that occurs during the transfer, and to improve the robot task model. the difference between simulation and the real world through a visual-based learning process reducing the robot's free-form natural language input and Enables the generation of robotic actions from image data, 15 from image data visual embedding involves generating linguistic embedding from natural language input and the representation of these inputs. enabling the conversion of action prediction layers into robotic action, also, natural language input is related to the attention mechanism and visual context. Various solutions have been found that make it possible to process. Furthermore, in the current state of the technology, the use of a robotic joint sensor involves the load. using a momentum observer to extract the external torque and the robot estimating the position of the body, Jacobian matrix mapping and The mass of the non-constant dynamic load through moment balance solution, 25 different methods that enable real-time estimation of position and speed Solutions have been found. 2 However, in the current state of the technique, with the differentiable physics constraint gate (DPCL) the robot's physical state, including physical factors such as torque, friction, momentum, balance, and contact. that its parameters directly affect the attention mechanism of the VLA model; Thus, only the subsequent rejection of physically inappropriate actions 5 instead, the visual-linguistic model enables suppression during the decision-making process. Representation of the real physical state in a Physically Compatible Hidden Embedding Space unification and safe production of action across the Physical Border Wall The pre-definition of the sub-space, the different physical trajectories of the generated motion testing under these conditions, comparing the physical state expected in the simulation with the actual 10 the difference between the state measured by the sensors using Residual Physics Learning learning and modeling the physical properties, predicting physical characteristics, and understanding physical constraints. solutions that enable the online updating of mechanisms It has not been encountered. In the Chinese patent document numbered CN122143040, which is included in the known state of the art... Machine body load position estimation based on robot body sensing and related to the technical field of bipedal robot control, providing a collaborative control method. An invention is described. According to the method in the invention, a robot joint sensor and Using an airborne IMU, leg self-weight and 20 through dynamic preprocessing Eliminating inertial interference, extracting the external torque of the load. Using the momentum observer and estimating the position of the robot body fixed by Jacobian matrix mapping and moment balance solution Real-time estimation of the mass, position, and velocity of a non-dynamic load. This involves correcting the prediction distortion caused by the fuselage tilt, and the rotation matrix 25 Correction and elimination through projection and Alpha-Beta filtering It includes the steps involved in obtaining a proper output. In the control linkage, a mechanism is used to convert the load movement into body posture instructions. The backstepping method is used, inverse kinematics, second-order tracing. Differential controller and second-order superbending slip mode controller 30 3 joint torque output is obtained through this method, and the robot's self-balancing and Collaborative control of the load is provided. This invention involves the robot body itself. Estimating load position and providing coordinated control based on perception. This invention presents a method for in-depth analysis of the robot's internal sensor data. by examining and combining it with a dynamic decomposition strategy, external visual or 5 High-precision load detection and control without the need for haptic sensors. This provides, which significantly improves system integration and hardware. It reduces costs and system complexity. However, in the document numbered CN122143040, it is stated that the action produced is actually 10 from being tested in a physical simulation before being sent to the robot, the same motion variations include friction, mass, ground, contact, delay, and sensor noise. by simulating the motion multiple times under the same conditions, thus ensuring that the motion is not only not under ideal conditions, but in different physical situations that might be encountered in the real world. There is no mention of measuring whether it is safe or not. 15 Brief Description of the Invention The aim of this invention is to enable the robot's physical constraints to be measured using a differentiable physics constraint gate (DPCL). physical parameters such as state, torque, friction, momentum, balance and contact The VLA model directly influences the attention mechanism; thus, it affects physical... Instead of merely rejecting inappropriate actions afterwards, the model visual-linguistic representation and reality that enables suppression during the decision-making process. the unification of the physical state in a physically compatible hidden embedding space and physical 25 a secure sub-space in which the action can be produced across the border wall beforehand identification, testing of the generated motion trajectory under different physical conditions the physical condition expected in the simulation and measured by the actual sensors the difference between the two situations can be learned through residual physics learning and physics 30 of the model, physical property predictions and physical constraint mechanisms 4 a physical reality-supported robot that enables online updates The goal is to implement a control system. Detailed Description of the Invention 5 The "Physical Reality Supported" project was carried out to achieve the purpose of this invention. A "Robotic Control System" is shown in the figure; Figure 1. The subject of the invention is a physical reality-supported robotic control system. This is a schematic view. The parts shown in the figure are individually numbered, and the corresponding numbers correspond to these numbers. It is given below. 1. System 2. Robet body 3. Central Processing Unit Robots can perform the movements they learned in the simulation environment in the real world within 20 years. the subject of the invention, which enables its safe, consistent and successful implementation a physical reality supported robotic control system (1); - walking, running, bending, jumping, touching an object, grasping an object an articulated structure that can move in a way that allows it to perform physical movements having sensors such as a camera, microphone, and external force / torque sensor 25 at least one robot body (2) configured to have components and - all the hardware in the robot body located inside the robot body (2) and multi-modal data input and sensing that connects with software components running the merging module, multi-mode data input and sensor merging With its module, first 30 from the surroundings of the robot body (2) and its internal state. To ensure the collection of all incoming data, and to enable the robot to process that data; RGB-D images, stereo images from the cameras (2) on its body, depth maps, audio data from ambient microphones, operator's Natural language commands, position, speed and from the joints (2) in the robot body torque data, end-impact unit position and orientation data, external force / torque 5 sensor measurements, inertial unit data, and the mass of surrounding objects, from its physical properties such as friction, rebound, and center of gravity to ensure the formation of visual data from the sensors (2) on the robot body visual encoders, audio data with speech encoders, text commands with language model-based encoders and physical sensor data physical state 10 to enable processing and representation with vectors, the different Data from sources / sensors is processed using a single standard "Multimode Physical". By combining the input structure, it is possible to see what the robot body (2) sees and what it does. what you heard, what command you received, and what your physical condition was To enable them to be evaluated together within the same decision-making structure, Physics-15 Information-Added Visual-Language Encoder with image and language related to the robot body (2) data not only in terms of semantics but also in terms of physical suitability processing, so that the robot body (2) will move with the objects To enable the calculation of whether it is physically possible, Physics-Knowledge With the Enhanced Vision-Language Encoder, the AI ​​model's attention mechanism incorporates physical 20 Adding a limiting gate, this gate; joint limits of the robot body (2), torque limits, gravitational effect, friction, momentum, equilibrium and contact forces to ensure that physical conditions such as these are taken into account, so that the model, It should not focus solely on the visual areas associated with the command; it should also consider physical 25 More emphasis should be placed on relationships that are meaningful and applicable. to ensure that the output of this step is physically limited and subsequent to have a more reliable covert representation structure that can be used in the stages to provide the physical world model on the robot's body (2) and enabling the simulation of the physical behavior of objects in its environment, robot the joint movements, torques, external forces, and objects of the body (2) 6 motion, contact forces, friction, center of gravity, and balance Predicting the physical outcome of an action by taking the circumstances into account to ensure that the robot’s body (2) “if I do this movement, it is physical To enable it to generate a real-time prediction of the question "what will happen in the world?", thanks to the integrated operation of the artificial intelligence model with the decision-making process and 5 By reflecting errors in action production backward, the model becomes more physically consistent. to help the robot body (2) to make decisions, so that the robot body (2) physically considering the possible consequences of an action before applying it in the real world. to enable it to be evaluated, the Hidden Physically Compatible Embedded Space on it with Physics-Information-Enhanced Visual-Language Encoder and meaning representation of physical 10 to enable the integration of physical state information from the world model, this in this way, objects and tasks are understood not only visually or linguistically, but also physically. its features should be represented, a glass not just as a "glass" its inability to recognize; the fragility of the glass, its center of gravity, its contact with the surface, the possibility of slipping and the 15 that may occur when held by the robot body (2). the involvement of forces in the decision-making process, thereby creating physically inconsistent bringing the representation points closer to the valid physical space, so that the model is both He understood the command and also knew under what physical conditions this command would work in the real world. to ensure that he / she learns that it can be applied, on and / or in The Physical Boundary Wall Network will analyze hidden physical representations to generate 20 actions. Determining the physical limits within which the robot body must remain. (2) joint angles, movement speeds, torque values, balance state, collision distance, energy limit, and contact conditions with surrounding objects to ensure that it is evaluated here, so that the robot body (2) is safe the pre-determination of non-movement zones and the action production process of these 25 to prevent it from going outside the regions, with this structure and action defining the safe zone in which it can be produced from the outset, so that in the next step the movements to be produced should be created in a physically safer area to provide a physically compatible representation with the action analyzer on it, and Using safety limits, the movement that the robot body (2) will perform is 30 7 the production of the sequence, the production of the action, in the form of a one-time command production no; as a continuous, smooth and controlled trajectory of motion over time to ensure that the robot body (2) is made to perform the movement from start to finish. taking into account its speed, direction, contact, force, and equilibrium conditions 5 by applying methods such as diffusion principle and flow matching. ensuring that the movement becomes more natural, continuous and feasible, at every interval the step of the movement should be corrected so that it remains within physical limits, thus the robot body (2) sudden, unbalanced, excessively strong or physically to ensure it doesn't produce actions that cannot be performed, the Physics Engine on it The action generated through juxtaposition and real-time simulation is reflected in the real robot 10 It should be tested in physical simulation before being sent to the body (2), the same motion variations include friction, mass, ground, contact, delay, and sensor noise. to enable the simulation of the motion multiple times under the given conditions, thereby allowing the motion to be observed. not only under ideal conditions, but also in the real world, different physical characteristics can be encountered. To ensure that it is safe in these situations as well, each simulation 15 As a result, proximity to the target, balance, collision risk, contact stability, and safety. evaluating the criteria for staying in the region, the most resilient and safest by selecting the movement and transferring it to the next step, at this stage, the robot body (2) Before acting in the real world, mentally consider the outcome of the action. to enable testing, with its Simulation-to-Reality module 20 To reduce the difference between the simulation environment and the real world, the expected result in the simulation and the result from the real sensors (2) on the robot body to enable comparison of the results and to learn the difference between them, this difference, now modeling it with a structure called physics learning, so that the robot 25 of the physics model after each real world test of its body (2) updating and making more accurate predictions in subsequent movements, thus robot body (2) to different environments, different objects and changing physical conditions to ensure compliance is possible, secure approval on it The final safety check performed before sending the motion to the robot body (2) with the inspector the control is carried out, 30 selected and adapted from simulation to reality 8 the action; torque limits, joint speeds, balance conditions, support area, collision re-evaluation in terms of distance, energy limit and contact force, if If the movement passes all security checks, permission should be granted for its implementation; otherwise... If some limits are violated, the robot body (2) moves to the nearest safe movement. attempt to transform, if this correction is not possible, the robot body (2) 5 to ensure a safe positioning and the production of an emergency flag, Thus, a physically risky or impractical move can be performed directly by the robot. to prevent the sending of the robot to its body (2) High-level action verified as secure via the Management Interface by the robot 10 low-level control commands that can be applied to the motors of its body (2) conversion, end-effector control according to task type, joint space performing control, torque control or impedance control actions, Smoother and more force-sensitive control in tasks requiring precise contact. implementation, more direct motion control in position-oriented tasks to enable its use, thereby enabling AI-generated motion 15 the decision depends on hardware such as motor current, joint target, torque command and encoder target. to enable the translation of commands into applicable commands at the robot level, so that the robot the physical safe and task-compatible movement of the body (2) to ensure its implementation worldwide, finally, Closed Loop Recycle Binding on it With the Feed Cycle module, the movement of the robot body (2) in the real world is 20 then new sensor data will be collected again and compared with previous predictions. To enable comparison, if the real-world result is different from the simulation... If it is different, this difference means that the physical model of the robot body (2) is no longer in the learning layer. and to ensure its use in updating physical restriction gates, so that over time, its own physical world model will be made more accurate, robot 25 its body (2) with new objects, different surfaces or changing environmental conditions to enable them to adapt more quickly when faced with the situation, and also adapting to new conditions without completely forgetting the tasks learned previously For this to be possible, continuous learning mechanisms must be used, this closed-loop system Thanks to the structure, the robot body (2) only performs the task result after each movement 30 9 not only that, but also that he learns the difference between himself and the physical world, and that his subsequent decisions will be more... at least one center structured to ensure its security It contains processor (3). Multimode Data 5 in the central processor (3) in the system (1) that is the subject of the invention With the Input and Sensory Convergence module, the image data is transferred to the robot body (2) color-depth (RGB-D) video frames, stereo images and from its cameras To ensure that depth maps are available, this data can be processed by Vision Converter (Vision Transformer enables the conversion of patch embedding vectors with ViT; audio raw sound waves received from ambient microphones (2) on the robot body data 10 and to ensure that the audio data in question has spectrogram characteristics; Audio Audio Spectrogram Transformer (AST) It is structured to enable conversion into tokens. Multimode Data 15 in the central processor (3) in the system (1) that is the subject of the invention With the Input and Sensory Convergence module, text data is transmitted to the robot body (2) natural language commands from the operator and physical property definitions of the object to provide the data in question using the Large Language Model (LLM) Enabling the projection of text embeddings into vectors using a marker; sensor proprioceptive data, position, velocity, 20 from the joints (2) in the robot body Torque values ​​are obtained from the position and orientation of the impact unit and external force / torque sensors. to ensure that the incoming force vector has six degrees of freedom; these sensory data Enabling the transfer to the physical embedding space via linear projection, inertial measurement. to ensure that the unit data includes acceleration and angular velocity measurements It is being structured. 25 Multimode Data within the central processor (3) in the system (1) which is the subject of the invention With the Input and Sensory Convergence module, all sensory units of the robot body (2) To enable the processing of incoming raw sensor data, image data “I_t ∈ R^{H×W×4} (RGB-D)”, audio data “S_t ∈ R^{F×T} (spectrogram)”, text 30 The command “C (token array), physical state vector [q(t), q̇(t), τ(t), x_ee(t), The processing of object properties using formulas such as "[F_ext(t)] and {m, μ, e, r_COM}" to provide, in this way, comparable and measurable heterogeneous sensory data. to ensure that it is transformed into a physically meaningful form It is structured as follows: 5 Multimode Data within the central processor (3) in the system (1) which is the subject of the invention With the Input and Sensory Convergence module, all sensory units of the robot body (2) During the processing of the incoming raw sensor data, “V_i = ViT_Encoder(I_t) ∈ The processing of visual cues is done using the formula "R^{n×d_v}", "A_j = 10 The processing of audio markers is done using the formula "AST_Encoder(S_t) ∈ R^{m×d_a}", Processing text specifiers in the form of “T_k = Text_Embed(C) ∈ R^{k×d_t}”, The physical state is represented by the formula “P = W_p · [q; q̇; x_ee; F_ext] + b_p ∈ R^{d_p}” enabling embedding in the multimode physical input (MMPI) data structure The visual tokens are in the form of “[V_1, ..., V_n] ∈ R^{n×d_v}”, “[A_1, ..., A_m] ∈ 15 Auditory tokens in the form of "R^{m×d_a}" are "{q, q̇, x_ee, F_ext, object_properties} ∈ The physical state is recorded as "R^{d_p}" with a timestamp and task ID. to ensure that the processing is carried out, and that the output of these processes is Standardized Multi-Criteria. Enabling the generation of the Multimodal Physical Input (MMPI) data structure, Standardized Multimodal Physical Input data also have Physics-Information-Added 20 To enable the transfer of the Vision-Language Encoder and the Physical World Model. It is being structured. The physics-infused central processor (3) in the system (1) that is the subject of the invention Differentiable Physics Constraint Gate 25 for the attention mechanism with visual-language encoder By integrating the Differentiable Physics Constraint Layer (DPCL), every detail is taken into consideration. ensuring that the title complies with physical consistency limitations to provide, Standard Multi-Head Self-Attention The addition of a physical restraint arm parallel to the Attention (MHSA) block, 11 Here, the physical state embedding vector is represented by the query and key matrices. combining them to produce a physical door and the attention weights of this door to ensure adjustment according to physical consistency, the total loss of the model in question the function, Newton-Euler dynamics, conservation of momentum and equilibrium to ensure the inclusion of physical loss, which includes limitations, as well as the aforementioned 5 Multimodal Data Input and Sensory in Standard Multi-Head Self-Attention Block The use of the MMPI data structure from the concatenation module, especially for visuals. the markers are “V_i”, the phonetic markers are “A_j”, the text markers are “T_k”, and the physical markers are the use of the state embedding vector “P”, and the model by using this data the internalization of physical laws during education; model 10 learning not only statistical relationships but also physical equations It is structured to ensure its implementation. The physics-infused central processor (3) in the system (1) that is the subject of the invention Mass 15 in the Standard Multi-Head Self-Attention block found in the visual-language encoder. the matrix (mass matrix - M(q)), the Coriolis and centripetal forces matrix (C(q, q̇), Gravity vector (G(q)), Insertion torques (τ), Friction coefficient (μ), To ensure that the Zero Point of Moment (ZMP) is taken into account together, thereby Outputting the physically constrained hidden representation (Z ∈ R^{d_z}), Transferring the obtained output data to the Hidden Physically Compatible Embedding Space 20 to ensure that each hidden vector conforms to the dynamics of the physical world. guaranteeing; representation of physically impossible situations in hidden space It is structured to prevent such actions from occurring. The physical world model within the central processing unit (3) is integrated with the neural network, 25 to ensure that the gradient is calculated within the physical world model The dynamics of the robot body (2) can be determined using Newton-Euler equations of motion. modeling, the object in solid body dynamics during the said modeling the use of equations of motion, Coulomb and viscous friction models By operating it, it is possible to distinguish between kinetic and static friction conditions, Explicit Euler 30 12 instead of integration, an implicit differentiable integrator should be used; thus The equations can be derived using automatic derivation in PyTorch / TensorFlow. to ensure that the contact dynamic is based on a soft rather than complementary condition. to make it differentiable using the contact model It is being structured. 5 Multimode Data Input and the physical world model inside the central processor (3) The physical state vector (joint positions q) coming from the Sensory Augmentation module, velocities q̇, accelerations q̈, end-impact unit posture x_ee, external forces F_ext) and object properties (mass m, coefficient of friction μ, rebound coefficient e, center of mass 10 r_COM) enables joint evaluation, and the robot is designed using this data. accurate physical behavior of the body (2) and the objects around it To enable simulation, output "S_t = [q, q̇, q̈, x_objects, v_objects, Obtaining the physical state vector in the form of "F_contact] ∈ R^{d_s}", The subject is the physical state vector in the Hidden Physically Compatible Embedding Space and Physics 15 It is configured to transmit the information to the Motor Joining module. Robot with Hidden Physically Compatible Embedding Space inside the central processor (3) (2) physical consistency projection of the body, physically in hidden space It should be used as a projection layer that eliminates impossible situations, 20 the physical properties analyzer inside, the physical properties of objects from the hidden space to enable it to function as a discrete parser that extracts its features It is being structured. In the Hidden Physically Compatible Embedding Space within the central processor (3) there were previously 25 calculated physically limited covert representation and physical condition the use of the vector, thereby combining visual-linguistic meaning with physical meaning. achieving the unification of global dynamics into a single coherent representation to ensure the Physical Compatibility Network (Cross-) in the aforementioned merging processes Note), the physical consistency projection, physical property analyzer 30 13 its execution, with the consistent hidden representation “Z_valid ∈ R^{d_z}” as output data and obtaining the physical property estimation “{m, μ, e, r_COM}”, the obtained transmitting the outputs to the Physical Boundary Wall Network and the Action Analyzer It is structured to provide this. Hidden Physical Coherence Network in the Physical Boundary Wall Network within the central processor (3) By analyzing the latent representation from the Embedding Space, the physical nature of the action to be produced can be determined. a "firewall" function that determines whether it violates the restrictions to enable visualization, joint restriction in the Physical Boundary Wall Network (minimum and maximum joint angles, speed limits, torque limits), balance 10 limitation (the zero moment point remaining within the support polygon), collision limitation (self-collision and perimeter collision prevention), torque limitation (engine torque limits, power consumption limits), energy limitation (kinetic and types of constraints such as conservation of potential energy and damped friction to ensure its use, the production of the restraining mask, from covert representation each 15 the creation of a possibility of violation for the restriction, safe zone the projection, a convex polygon defining the safe zone in the action space by using and projecting the action so that it remains within this area It is structured to ensure its implementation. Hidden Physical Coherence Network in the Physical Boundary Wall Network within the central processor (3) The physically consistent hidden representation “Z_valid ∈ R^{d_z}” comes from the Embedding Space. and enable the use of “{m, μ, e, r_COM}” for physical property estimation, this 25 to guarantee that the robot body (2) is physically impossible or To prevent or restrict dangerous actions before they occur. Safe Zone Projection (Convex Polygon) in masking operations, Action The combined use of Space Constraints creates a "Constraint Mask ∈" 14 [0,1]^{n_c}”, “Safe Action Flag ∈ {0,1}”, “Safe Action Subspace” It is structured to enable the production of outputs in this form. maintaining physical continuity with the action analyzer inside the central processor (3) The combination of Diffusion Policy and Flow Matching 5 its use, the principle of propagation, and the reverse propagation of action from noise to noise. its production, at each step of its spread, according to the loss of physical consistency of the action. adjustment and addition of Gaussian noise, action with flow matching generating its trajectory using the optimal transport (Kantorovich-Rubinstein) principle the implementation of the action trajectory using a neural network-based velocity field 10 to enable modeling as a continuous flow, with physically constrained inference a containment mask from the Physical Boundary Wall Network at each spreading step by ensuring that the action remains in a safe subspace. It is structured accordingly. Hidden Physically Compatible Embedding in Action Analyzer within Central Processing Unit (3) The physically consistent hidden representation from the space is “Z_valid ∈ R^{d_z}” and The constraint mask from the Physical Boundary Wall Network is “Constraint_Mask ∈ [0,1]^{n_c}” can be used, thus only being physically realized, the production of a continuous and smooth sequence of motion, the principle of propagation and flow 20 The matching must guarantee continuity in the action space, limiting Ensuring the physical safety of the mask is achieved, Physical in the output data The verified sequence of actions is obtained as “A = [a_1, a_2, ..., a_T] ∈ R^{T×d_a}” to ensure that the resulting output data is also processed by Physics Engine Convergence It is configured to ensure that it is passed to the module. 25 Action with the Physics Engine Concatenation module in the central processor (3) The sequence of actions from your analyzer is “A ∈ R^{T×d_a}” and the Physical World Using the physical state vector “S_t ∈ R^{d_s}” from the model, performing a mental simulation cycle, during the simulation cycle the application of the action to the robot body dynamics (2) as an insertion torque vector, The physics engine calculates the next state; the target of the simulated state. comparison with the situation, and if the simulation result is satisfactory, the action is taken. approval is required; otherwise, feedback is sent to the Action Analyzer, parallel 5 Simulation with numerous different physical parameters (field randomization) Ensuring the selection of the most robust action by running parallel simulations, output to ensure that the data is transferred to the Simulation-to-Reality module It is being structured. Physics in Simulation-to-Reality module in central processor (3) The simulated state sequence from the Engine Joining module is “S_sim ∈ R^{T×d_s}” and the observed situation from the real world is “S_real ∈ R^{d_s} (Real-time sensor telemetry: joint positions, speeds, torques, end-acting unit) to enable the neural network to use "posture, external forces", modeling 15 the field learns about the differences in error, sensor noise and actuator dynamics. randomization of physical parameters during training sampling should be done so that the model can withstand different physical conditions. the realization of this, through the extraction of compatible physical features in the real world online estimation of the physical properties of objects from data 20 to provide, in this way, the friction, elasticity, and delay that the Physics engine idealizes. Modeling errors such as those related to encoder quantization, IMU drift, and F / T sensor bias can occur. Sensor noise in the form of motor inertial noise, backslash, and torque ripple. To enable the understanding of actuator dynamics, as a result of the procedures performed. Simulation-to-reality compatible actions are “a_adapted ∈ R^{d_a}” and 25 Obtaining the updated physical parameters “{μ_real, m_real, I_real}” It is structured to provide this. 16 The action robot body (2) with the secure confirmation controller inside the central processor (3) it must undergo security verification before being implemented on it, the promise The subject of safety verification is the engine data sheet values, safety limit, The zero moment point remains within the support polygon, between the robot body (2) and the environment. The distance between them being greater than the safety distance means that the kinetic and potential 5 operations are performed such that the total energy does not exceed the maximum energy. to ensure, by optimizing the online trajectory, if any action is taken If it violates the limitation, it should be corrected with a second-order program solver. the process involves generating an emergency stop command, if a correction is needed. If this is not possible, ensure that the "safe pose" command is generated. 10 The commands generated as a result of the operations are sent to the Robot Control Interface. It is structured to ensure transmission. The action approved by the Robot Management Interface in the central processor (3), robot transmitting the body to the (2) low-level controller 15 to provide both the end-influence unit (Cartesian) space and the joint space to enable support for end-effect unit control (x, y, z, roll, slope, Deviation - 6 degrees of freedom), joint space control (change in joint angles), task control modes such as operational space control (OSC) to enable the use of a low-level controller, proportional-derivative (PD) 20 control, robot using direct torque control or impedance control to enable it to send commands to the motors (2) in its body, output data to transmit to the Closed-Loop Feedback Loop module on it It is being structured. In the Closed Loop Feedback Loop module in the central processor (3) Real-world sensor telemetry of the robot body (2) (joint positions q, velocities q̇, torques τ, end-impact unit posture x_ee, external forces F_ext, images I_t) and Physics The simulated state sequence from the Engine Joining module is “S_sim ∈ The use of data in the form of "R^{T×d_s}", with 30 in the learning algorithm within it. 17 running the data in question, online adaptation in the learning algorithm (each rapid adaptation with 1-2 gradient steps after action), meta-learning (Model-Agnostic) Meta-learning, MAML (rapid adaptation to new objects and environments), continuous learning. (Elastic Weight Consolidation, EWC: New tasks without forgetting the old ones) to ensure that the steps in the form of (learning) are carried out, with 5 as output. Updated residual network weights “θ_residual”, updated physical properties Estimates “{μ_real, m_real, I_real}” and updated DPCL weights “W_g, Obtaining information in the form of "b_g", and the output data / information being Physics-Knowledge From Enhanced Vision-Language Encoder to Physical World Model, From Simulation to Reality To ensure it is transferred as a feed to the compatibility module, 10 It is being structured. Industrial application of the invention The invention system includes (1) a robot body capable of performing humanoid movements (2) It has numerous motion, acceleration, microphone, camera inside the robot body (2). Sensors in the form of are located. The sensors in the robot body (2) The data is sent to the central processor located in the robot body (2). The central processor (2) processes the sensor data it receives into numerous and different 20 By working with modules and layers that perform tasks, the robot body (2) before any action is taken in the real world There is no problematic action for the robot body (2) in performing the simulation. When understood, the relevant action is the command to move the robot body (2) It is sent to the components, to the motors. 25 In addition, the following codes and formulas are executed in the central processing unit (3): In the Multimode Data Input and Sensor Convergence module 18 Visual markers: V_i = ViT_Encoder(I_t) ∈ R^{n×d_v} Sound tokens: A_j = AST_Encoder(S_t) ∈ R^{m×d_a} Text specifiers: T_k = Text_Embed(C) ∈ R^{k×d_t} Physical state embedding: P = W_p · [q; q̇; x_ee; F_ext] + b_p ∈ R^{d_p} 5 Multimode Physical Input (MMPI) data structure: plain MMPI = { visual_tokens: [V_1, ..., V_n] ∈ R^{n×d_v} audio_tokens: [A_1, ..., A_m] ∈ R^{m×d_a} 10 text_tokens: [T_1, ..., T_k] ∈ R^{k×d_t} physics_state: {q, q̇, x_ee, F_ext, object_properties} ∈ R^{d_p} timestamp: t task_id: UUID } 15 Formulas and codes of this form are executed. In the Physics-Integrated Visual-Language Encoder Standard Multi-Headed Self-Attention: 20 plain Attention(Q, K, V) = softmax(QK^T / √d_k) V Differentiable Physics Constraint Gate (DPCL): plain Physics_Gate(Q, K, P) = σ(W_g · [Q; K; P] + b_g) 25 Physics_Mask = Physics_Gate ⊙ Attention_Weights Constrained_Attention = Physics_Mask V Loss of Physical Coherence (L_physics): plain 19 L_total = L_vlm + λ_physics · L_physics + λ_safety · L_safety L_physics = || M(q)·q̈ + C(q,q̇)·q̇ + G(q) - τ||² + || F_friction - μ·N·sign(v) ||² + || p_total - Σ m_i·v_i ||² 5 L_safety = max(0, |τ| - τ_max)² + max(0, |q̈| - q̈_max)² + max(0, ZMP_x - ZMP_boundary)² bimindeki codes, formulas are executed. Physical World Model Robot Newton-Euler Dynamics: 15 plain M(q)·q̈ + C(q,q̇)·q̇ + G(q) = τ + J^T(q)·F_ext Solid Body Dynamics: plain m·ẍ = F_gravity + F_contact + F_friction 20 I·ω̇ = τ_contact + τ_friction - ω × (I·ω) Coulomb + Viscous Friction: plain F_friction = -μ_k·N·sign(v) - μ_v·v (kinetic) F_friction ∈ [-μ_s·N, +μ_s·N] (static, v=0) 25 Implicit Differentiable Integrator: plain q_{t+1} = q_t + Δt·q̇_t + (Δt² / 2)·q̈_t q̇_{t+1} = q̇_t + Δt·q̈_t Soft Touch Model: 30 plain F_contact = k·penetration - d·penetration_velocity Codes and formulas in this format are being executed. Within a concealed, physically compatible burial space. Combining: plain Z_physics = Concat(Z_vlm, W_s · S_t) ∈ R^{d_z + d_s} 10 Z_aligned = Physics_Alignment_Network(Z_physics) ∈ R^{d_z} Physical Adaptation Network (Cross-Attention): plain Z_aligned = LayerNorm(Z_vlm + CrossAttention(Z_vlm, S_t, S_t)) Physical Consistency Projection: 15 plain Z_valid = Z_aligned - α · ∇L_physics(Z_aligned) Physical Property Analyzer: plain [mass, friction, restitution, COM] = Physics_Property_Decoder(Z_valid) 20 Codes and formulas in this format are being executed. Within the Physical Boundary Wall Network 25 Restriction Mask: plain Constraint_Mask = σ(W_c · Z_valid + b_c) ∈ [0, 1]^{n_constraints} 21 Safe Zone Projection (Convex Polygon): plain A · a ≤ b a_projected = Projection(a, Safe_Polyhedron) Action Space Constraints: 5 plain q_min ≤ q ≤ q_max |q̇| ≤ q̇_max |τ| ≤ τ_max ZMP ∈ Support_Polygon 10 Distance(robot, env) ≥ d_safe E_kinetic + E_potential ≤ E_max Codes and formulas in this format are being executed. In Action Analyzer The Principle of Propagation (Reverse Propagation): plain a_0 ~ N(0, I) 20 a_{k-1} = a_k - η · ∇_a [ ||a_k - a_target||² + λ · L_physics(a_k) ] + σ_k · z Flow Matching (Optimal Transport): plain v_t(a) = Neural_Network(a, t, Z_valid) a_1 = a_0 + ∫_0^1 v_t(a_t) dt 25 Physically Restricted Inference: plain a_k ← Projection(a_k, Safe_Action_Subspace) a_k ← a_k ⊙ Constraint_Mask 22 Codes and formulas in this format are being executed. In the Physics Engine Joining module Simulation Cycle: plain τ_t = Action_to_Torque(a_t) S_{t+1} = Physics_Step(S_t, τ_t) L_sim = ||S_{t+1} - S_target||² 10 Parallel Simulation (Field Randomization): plain S_{t+1}^{(i)} = Physics_Step(S_t, τ_t; μ_i, m_i, f_i) for i = 1..N a_robust = argmax_a Σ_i 1[S_{t+1}^{(i)} ∈ Safe_Region] Durable Action Selection: 15 plain Robustness_Score(a) = Σ_i exp(-||S_{t+1}^{(i)} - S_target||²) a_robust = argmax_a Robustness_Score(a) Codes and formulas in the following format are executed. 20 In the Simulation-to-Reality module Now for Learning Physics: 25 plain ΔS = S_real - S_sim = Residual_Network(S_sim, θ_residual) Network Loss: plain L_residual = || S_real - (S_sim + Residual_Network(S_sim, θ_residual)) ||² 30 23 Learned Differences: Modeling error: Friction, elasticity, and delay idealized by the physics engine. Sensor noise: Encoder quantization, IMU drift, F / T sensor bias. Actuator dynamics: Motor inertia, backslash, torque ripple. Field Randomization: 5 plain μ ~ Uniform(μ_min, μ_max) m ~ Uniform(m_min, m_max) delay ~ Uniform(0, τ_max) Inference of Consistent Physical Traits: 10 plain [μ_real, m_real, I_real] = Online_Property_Estimator(S_history, τ_history) Codes and formulas in this format are being executed. Within Secure Approval Auditor Security Check: plain 20 |τ| ≤ τ_max (engine datasheet) |q̇| ≤ q̇_max (safety limit) ZMP ∈ Support_Polygon (balance) Distance(robot, env) ≥ d_safe (collision) E_kin + E_pot ≤ E_max (energy) 25 Orbit Correction with Second Order Programming (QP): plain minimize ||a_corrected - a_original||² subject to A · a_corrected ≤ b Emergency Safe Stance: 30 24 plain a_safe = Inverse_Kinematics(q_safe) Codes and formulas in this format are being executed. Within the Robot Management Interface Control Modes: plain 10 End-Effector: Δx, Δy, Δz, Δroll, Δpitch, Δyaw ∈ R⁶ Joint Space: Δq_1, Δq_2, ..., Δq_n ∈ R^n Task Space: Operational Space Control (OSC) Low-Level Controllers: plain 15 PD Control: τ = K_p · (q_des - q) + K_d · (q̇_des - q̇) Torque Control: τ = τ_des Impedance: F = K · (x_des - x) + D · (ẋ_des - ẋ) Codes and formulas in the following format are executed. 20 In Closed-Loop Feedback Online Compatibility: 25 plain θ_residual ← θ_residual - α · ∇_θ ||S_real - (S_sim + Residual_Network(S_sim, θ_residual))||² Meta-Learning (MAML): plain θ* = θ - β · ∇_θ L_task(θ) (inner loop: task matching) θ_global ← θ_global - γ · ∇_θ L_meta(θ*) (external loop: meta-learning) Continuous Learning (EWC): plain 5 L_EWC = L_task + Σ_i (λ / 2) · F_i · (θ_i - θ*_i)² Fisher Information Matrix: plain F_i = E[ (∂L / ∂θ_i)² ] Codes and formulas in this format are being executed. The invention system (1) and the robot with differentiable physics constraint gate (DPCL) physical state, torque, friction, momentum, balance and contact, etc. The fact that its parameters directly affect the attention mechanism of the VLA model, 15 thus only the subsequent rejection of physically inappropriate actions instead, the model is suppressed during the decision-making phase, through visual-linguistic representation. the integration of the real physical state in a Physically Compatible Hidden Embedding Space and The secure sub-space in which action can be produced across the Physical Boundary Wall is determined in advance. Identification, testing of the generated motion trajectory under different physical conditions, 20 The expected physical state in the simulation versus the state measured by the actual sensors. The difference between them can be learned through Residual Physics Learning and the physics model, Physical property predictions and physical constraint mechanisms are available online. It is being updated. The system in question (1) is within the scope of the Personal Data Protection Law (KVKK). It operates. All the data to be used in this system (1) is the robot's obtained from their own sensory units or from open-source physical simulation environments These are anonymous technical telemetry data. They do not contain personal data. 26 The subject of the invention is a physical reality-supported robotic control system (1) for a wide variety of It is possible to develop its applications, and the invention is illustrated with the examples described here. It cannot be restricted, it is essentially as stated in the claims.

Claims

27 REQUESTS 1. Robots can perform the movements they learn in a simulation environment more effectively in the real world. enabling it to be implemented in a safe, consistent and successful manner; - walking, running, bending, jumping, touching an object, grasping an object (5 types) an articulated structure that can move in a way that allows it to perform physical movements having sensors such as a camera, microphone, and external force / torque sensor at least one robot body (2) configured to have components and - all the hardware in the robot body located inside the robot body (2) and multi-mode data input and sensor 10 that connects with software components running the merging module, multi-mode data input and sensor merging with its module, first from the surroundings (2) of the robot body and its internal state To ensure the collection of all incoming data, and to enable the robot to process that data; RGB-D images, stereo images from the cameras (2) on its body, depth maps, audio data from ambient microphones, operator's 15 Natural language commands, position, speed and from the joints (2) in the robot body torque data, end-impact unit position and orientation data, external force / torque sensor measurements, inertial unit data, and the mass of surrounding objects, from its physical properties such as friction, rebound, and center of gravity to ensure that visual data from the sensors (2) on the robot body is generated 20 visual encoders, audio data with speech encoders, text commands with language model-based encoders and physical sensor data in the physical state to enable processing and representation with vectors, the different Data from sources / sensors is processed using a single standard "Multimode Physical". By combining the input structure, so that the robot body (2) knows what it sees and what it doesn't see. what you heard, what command you received, and what your physical condition was To enable them to be evaluated together within the same decision-making structure, Physics- Information-Added Visual-Language Encoder with image and language related to the robot body (2) data not only in terms of semantics but also in terms of physical suitability processing, so that the movement of the robot body (2) with the objects will be 30 28 To enable the calculation of whether it is physically possible, Physics-Knowledge The Enhanced Visual-Language Encoder integrates physical elements into the attention mechanism of an AI model. Adding a limiting gate, this gate; joint limits of the robot body (2), torque limits, gravitational effect, friction, momentum, equilibrium and contact forces to ensure that physical conditions such as those described are taken into account, so that the model, 5 It should not focus solely on the visual areas associated with the command; it should also consider physical more emphasis should be placed on relationships that are meaningful and applicable. to ensure that the output of this step is physically limited and subsequent to have a more reliable covert representation structure that can be used in the stages to provide the physical world model on the robot's body (2) and 10 enabling the simulation of the physical behavior of objects in its environment, robot the joint movements, torques, external forces, of objects of the body (2) motion, contact forces, friction, center of gravity, and balance Predicting the physical outcome of an action by taking the circumstances into account to ensure that the robot’s body (2) “if I do this movement, I will physically 15 To enable it to generate a real-time prediction of the question "what will happen in the world?", this allows the artificial intelligence model on it to work integrated with the decision-making process and By reflecting errors in action production backward, the model becomes more physically consistent. to help the robot body (2) to make decisions, so that the robot body (2) Physically consider the possible consequences of the action before applying it in the real world. to enable it to be evaluated, the Hidden Physically Compatible Embedded Space on it with Physics-Information-Enhanced Visual-Language Encoder and meaning representation from physical. to enable the integration of physical state information from the world model, this in this way, objects and tasks are understood not only visually or linguistically, but also physically. its features should be represented, a glass not just as a "glass" 25 its inability to recognize; the fragility of the glass, its center of gravity, its contact with the surface, the possibility of slipping and what may occur when held by the robot body (2) the involvement of forces in the decision-making process, thereby creating physically inconsistent bringing the representation points closer to the valid physical space, so that the model is both He understood the command and also knew under what physical conditions this command could be used in the real world. 29 to ensure that he / she learns that it can be applied, on and / or in Action will be generated by analyzing the hidden physical representation through the Physical Boundary Wall Network. Determining the physical limits within which the robot body must remain. (2) joint angles, movement speeds, torque values, balance state, collision distance, energy limit and contact conditions with surrounding objects 5 to ensure that it is evaluated here, so that the robot body (2) is safe the pre-determination of non-movement zones and the action production process to prevent it from going outside the regions, with this structure and action defining the safe zone in which it can be produced from the outset, so that in the next step 10 The movements to be produced should be created in a physically safer area. to provide a physically compatible representation with the action analyzer on it, and The movement that the robot body (2) will perform using safety limits the production of the sequence, the production of the action, in the form of a one-time command production no; as a continuous, smooth and controlled trajectory of motion over time to ensure that the robot body (2) 15 is made from the beginning to the end of the movement taking into account its speed, direction, contact, force, and equilibrium conditions by applying methods such as diffusion principle and flow matching ensuring that the movement becomes more natural, continuous and feasible, at every interval the step of the movement should be corrected so that it remains within physical limits, thus If the robot body (2) is suddenly, unbalanced, too strong or physically 20 to ensure it doesn't produce actions that cannot be performed, the Physics Engine on it Action generated through juxtaposition and real-time simulation, using a real robot. It should be tested in physical simulation before being sent to the body (2), the same motion variations include friction, mass, ground, contact, delay, and sensor noise. to enable the simulation of the motion multiple times under the given conditions, thus allowing the motion to be performed 25 times. not only under ideal conditions, but also in the real world, different physical characteristics can be encountered. In these situations, it is possible to measure whether it is safe or not, in each simulation. As a result, proximity to the target, balance, collision risk, contact stability, and safety. evaluating the criteria for staying in the region, the most resilient and safest by selecting the movement and transferring it to the next step, at this stage, the robot body (2) 30 Before acting in the real world, mentally consider the outcome of the action. to enable it to run, with its Simulation-to-Reality module To reduce the difference between the simulation environment and the real world, the expected result in the simulation and the result from the real sensors (2) on the robot body To enable comparison of the results and to learn the difference between them, this difference is 5 now modeling it with a structure called physics learning, so that the robot after every real world test of the body (2) the physics model updating and making more accurate predictions in subsequent movements, thus robot body (2) to different environments, different objects and changing physical conditions to ensure compliance is possible, secure approval on it 10 The final safety check performed before sending the motion to the robot body (2) with the inspector the control is carried out, selected and adapted from simulation to reality the action; torque limits, joint speeds, balance conditions, support area, collision re-evaluation in terms of distance, energy limit and contact force, if If the movement passes all security checks, permission to implement it should be granted, if 15 If some limits are violated, the robot body (2) moves to the nearest safe movement. attempt to transform the robot body (2) if this correction is not possible to ensure a safe positioning and the production of an emergency flag, Thus, a physically risky or impractical move can be performed directly by the robot. To prevent the sending of Robot 20 to its body (2) High-level action verified as secure via the Management Interface by the robot low-level control commands that can be applied to the motors of its body (2) conversion, end-effector control according to task type, joint space performing control, torque control or impedance control actions, 25% smoother and more force-sensitive control for tasks requiring precise contact. implementation, more direct motion control in position-oriented tasks to enable its use, thereby allowing AI-generated movement. the decision depends on hardware such as motor current, joint target, torque command and encoder target. to enable the translation of commands into applicable commands at the robot level, so that the robot the actual 30 of the body's (2) physically safe and task-compatible movement 31 to ensure its implementation worldwide, finally, Closed Loop Recycle Binding on it With the Feed Cycle module, the robot body's (2) movement in the real world then new sensor data will be collected again and compared with previous predictions. To enable comparison, if the real-world result is different from the simulation... If it is different, this difference means that the physical model of the robot body (2) is now in the learning layer 5 and to ensure its use in updating physical restriction gates, so that over time, the robot's own model of the physical world is made more accurate. its body (2) with new objects, different surfaces or changing environmental conditions to enable them to adapt more quickly when faced with the situation, and also 10 For this to be possible, continuous learning mechanisms must be used, this closed-loop system Thanks to the structure, the robot body (2) only reports the task result after each movement. not only that, but also that he learns the difference between himself and the physical world, and that his subsequent decisions will be more... at least one center structured to ensure its security Physical reality-supported robotic control characterized by processor (3) 15 system (1).

2. With its Multi-Mode Data Input and Sensor Convergence module, image data, Color-depth (RGB-D) video from the cameras (2) on the robot body To ensure that the data consists of frames, stereo images and depth maps, these 20 Using Vision Transformer (ViT) to embed patch vectors to ensure the conversion of the sound data in the robot body (2) environment raw sound waves and spectrogram characteristics obtained from microphones to provide; the audio data in question is processed by the Audio Spectrogram Converter (Audio Spectrogram Transformer (AST) enables the conversion of audio signals into audio markers. 25 As in Claim 1, characterized by the central processor (3) configured to do so. a physical reality supported robotic control system (1).

3. With its Multimodal Data Input and Sensory Concatenation module, it can convert text data into speech. The subject is the natural language commands from the operator to the robot body (2) and the physical object 30 32 To ensure that feature definitions exist, the data in question is presented in the Large Language Model (LNG). The Language Model (LLM) token is projected into text embedding vectors. to ensure that sensory data (proprioceptive) is obtained in the robot body (2) position, velocity, torque values ​​from the joints, end-action unit position and posture 5. There should be six degrees of freedom force vectors from external force / torque sensors. to provide; to project this sensory data into the physical embedding space via linear projection. to enable the transfer of inertial unit data, acceleration and angular velocity measurements characterized by the central processor (3) which is configured to include A physical reality-supported robotic control system as in claim 1 or 2 (1).

4. With its Multi-Mode Data Input and Sensor Convergence module, the robot body (2) to enable the processing of raw sensor data from all sensory units, The image data is “I_t ∈ R^{H×W×4} (RGB-D)”, and the audio data is “S_t ∈ R^{F×T} (spectrogram)”, the text command “C (token sequence), physical state vector [q(t), 15 in the form of object properties {m, μ, e, r_COM}” q̇(t), τ(t), x_ee(t), F_ext(t)] and object properties {m, μ, e, r_COM}” to enable the processing of heterogeneous sensory data using formulas. transforming it into a comparable, measurable and physically meaningful form with the central processor (3) configured to enable its implementation physical as in any of the above characterized claims a reality-supported robotic control system (1). 20 5. Robot body with multi-mode data input and sensor convergence module inside (2) During the processing of raw sensor data from all sensory units, “V_i = The processing of visual cues is done with the formula “ViT_Encoder(I_t) ∈ R^{n×d_v}”, “A_j The processing of sound markers is done with the formula "= AST_Encoder(S_t) ∈ R^{m×d_a}", 25 Processing text specifiers in the form of “T_k = Text_Embed(C) ∈ R^{k×d_t}”, The physical state is represented by the formula “P = W_p · [q; q̇; x_ee; F_ext] + b_p ∈ R^{d_p}” enabling embedding in the multimode physical input (MMPI) data structure The visual tokens are in the form of “[V_1, ..., V_n] ∈ R^{n×d_v}”, “[A_1, ..., A_m] ∈ 33 Auditory tokens in the form of "R^{m×d_a}" are "{q, q̇, x_ee, F_ext, object_properties} ∈ The physical state is recorded as "R^{d_p}" with a timestamp and task ID. to ensure that the processing is carried out in a standardized manner in the output of these processes To enable the acquisition of the multimodal physical input (MMPI) data structure, Standardized multimodal physical input data also exhibit physics-infused insights. to enable the language to be transferred to the encoder and the physical world model from the above requests characterized by the configured central processor (3) a robotic control system supported by physical reality like any other (1).

6. With its physics-infused visual-language encoder, it helps the attention mechanism 10 Differentiable Physics Constraint Layer (DPCL) is integrated, with each attention span having physical consistency limitations. to ensure compliance, the standard on it is multifaceted. a physical restriction parallel to attention (Multi-Head Self-Attention, MHSA) block the addition of the arm, where the physical state embedding vector, query and key 15 the creation of a physical gate by combining matrices and the attention of this gate enabling the model to adjust its weights according to physical consistency. the total loss function, Newton-Euler dynamics, momentum the addition of physical loss including preservation and balance limitations to provide, also, the aforementioned standard multi-head self-attention block multi-mode 20 The use of the MMPI data structure from the data input and sensor concatenation module; Specifically, visual markers are “V_i”, audio markers are “A_j”, and text markers are “A_j”. The use of "T_k" and the physical state embedding vector "P", these data by using the model to internalize physical laws during training the implementation; the model should not only examine statistical relationships but also physical 25 structured to enable the learning of equations as well. any of the above requests characterized by the central processor (3) a robotic control system supported by physical reality like one (1). 34 7. Standard multi-heading found in the physics-infused visual language encoder within it. In the self-attention block, the mass matrix (M(q)), Coriolis and centripetal the matrix of forces (C(q, q̇), the gravity vector (G(q)), the insertion torques (τ), The coefficient of friction (μ) and the zero moment point (ZMP) are considered together. to ensure that it is obtained, thereby enabling the physically restricted covert representation (Z ∈ 5 The output of R^{d_z}) is obtained, and the resulting output data is stored in a secure physical compatible manner. to enable the transfer of each hidden vector to the embedding space, so that each hidden vector is in the physical world. ensuring that it is in line with its dynamics; physically impossible to prevent the representation of situations in a hidden space Physical 10 as in Claim 6 characterized by the configured central processor (3). a reality-supported robotic control system (1).

8. A gradient-capable model of the physical world within it, integrated with a neural network. to ensure that it is in the structure, Newton-Euler motion within the physical world model Modeling the dynamics of the robot body (2) using the equations, word 15 the subject is the equations of motion of objects in rigid body dynamics during modeling. The use of Coulomb and viscous friction models to analyze kinetic and static friction. distinguishing between friction situations, using implicit instead of explicit Euler integration. the use of differentiable integrators; thus, the equations The fact that it is differentiable with automatic differentiation in PyTorch / TensorFlow is 20 to provide, contact dynamics, a soft contact model instead of the complementarity requirement a center structured to enable it to be made derivable by using as in any of the above requests characterized by processor (3) a physical reality supported robotic control system (1).

9. Multimodal Data Input and Sensory Convergence in the Physical World Model Within the physical state vector from the module (joint positions q, velocities q̇, accelerations) q̈, end-impact unit posture x_ee, external forces F_ext) and object properties (mass m, friction coefficient μ, rebound coefficient e, center of mass r_COM) together to enable evaluation of the robot's body (2) and 30 using this data 35 accurate simulation of the physical behavior of objects in the environment providing “S_t = [q, q̇, q̈, x_objects, v_objects, F_contact] ∈ R^{d_s}” as output obtaining the physical state vector in the form of, the physical state in question Vector's Hidden Physically Compatible Embedding Space and Physics Engine Joining central processor (3) configured to transmit to module 5 physical as in any of the above characterized claims a reality-supported robotic control system (1).

10. Physical compatibility of the robot body (2) with its hidden physical embedding space inside. consistency projection, physically impossible situations in hidden space 10 its use as a projection layer that distances the physical within it a feature analyzer that extracts physical properties of objects from a hidden space using discrete features. a central processing unit configured to function as an analyzer Physical as in any of the above demands characterized by (3). a reality-supported robotic control system (1). 15 11. Pre-calculated physical within the hidden physically compatible embedding space. the use of limited latent representation and physical state vector, In this way, visual-linguistic meaning-making and the dynamics of the physical world are combined into a single system. to ensure that consistent representation is achieved, the said unification 20 Physical Consistency Network (Cross-Attention) in processes, physical consistency its projection, running the physical feature analyzer, as output data consistent latent representation “Z_valid ∈ R^{d_z}” and physical property estimation “{m, μ, e, obtaining the "r_COM}", and connecting the obtained outputs to the Physical Boundary Wall Network and Central processing unit 25 configured to ensure that the action is passed to the Action Analyzer. Physical as in any of the above demands characterized by (3). a reality-supported robotic control system (1).

12. From the hidden physically compatible embedding space within the physical boundary wall network inside. By analyzing the incoming hidden representation, it is determined whether the action to be produced violates physical limitations and 30 36 to ensure it acts as a "firewall" that determines that it is not doing so, Joint restriction in the physical boundary wall network (minimum and maximum joints) angles, speed limits, torque limits), equilibrium limitation (zero moment point) (remaining within the support polygon), collision restriction (self-collision and perimeter) collision prevention), torque limiting (engine torque limits, power consumption 5 limits), energy limitation (conservation of kinetic and potential energy, damped to enable the use of restraint types such as friction, the production of the restraint mask, the violation of each restraint from covert representation the possibility of generating and implementing safe zone projection in the action space Using a convex polygon that defines the safe zone, the action must take place within this zone 10 to ensure that it is realized by projecting it in such a way that it will remain from the above requests characterized by the configured central processor (3) a robotic control system supported by physical reality like any other (1).

13. Concealed Physically Compatible Embedding Within the Internal Physical Boundary Wall Network 15 The physically consistent hidden representation "Z_valid ∈ R^{d_z}" coming from its space and the physical Enabling the use of “{m, μ, e, r_COM}” for feature prediction, thereby generating ensuring that the action is physically safe and feasible; thus, physically impossible or dangerous movements of the robot body (2), 20 in the containment mask processes to ensure it is prevented before production. Safe Zone Projection (Convex Polygon), Action Space Constraints When used together, “Restriction Mask ∈ [0,1]^{n_c}”, “Safe Action It should generate outputs in the form of "Flag ∈ {0,1}", "Safe Action Subspace". The above is characterized by the central processor (3) configured to provide a robotic control supported by physical reality, as in any of the requests 25 system (1).

14. The Diffusion Principle, which maintains physical continuity with its internal action analyzer. Using a combination of Diffusion Policy and Flow Matching 37 The principle of propagation suggests that action is produced from noise through reverse propagation, with each propagation... In this step, the action is adjusted according to the loss of physical consistency and Gaussian the addition of noise, flow matching and optimal transport of the action trajectory The generation of neural networks is realized using the Kantorovich-Rubinstein principle. 5 The trajectory of action is shown as a continuous flow using a velocity field based on a vector. to enable modeling, physically constrained inference at each propagation step The action is masked by applying a restriction from the Physical Boundary Wall Network. structured to ensure that it remains in a secure subspace any of the above requests characterized by the central processor (3) a robotic control system supported by physical reality, like one (1). 10 15. The action analyzer inside comes from the Hidden Physically Compatible Embedding Space. physically consistent hidden representation “Z_valid ∈ R^{d_z}” and Physical Boundary Wall The constraint mask coming from the network is “Constraint_Mask ∈ [0,1]^{n_c}” its use, thus making it possible to achieve only physically, continuously and 15 the generation of a smooth motion sequence, the principle of propagation and flow matching, the physical nature of the limiting mask guarantees continuity in the space of action. to ensure security is implemented physically in the output data. The verified sequence of actions obtained is “A = [a_1, a_2, ..., a_T] ∈ R^{T×d_a}” to ensure that the resulting output data is also processed by Physics Engine Convergence 20 central processor (3) configured to ensure transmission to the module physical as in any of the above characterized claims a reality-supported robotic control system (1).

16. Action 25 from the action analyzer with the internal physics engine concatenation module. The sequence “A ∈ R^{T×d_a}” and the physical state derived from the physical world model the use of the vector “S_t ∈ R^{d_s}”, the mental simulation cycle the action is performed during the simulation cycle, as an insertion torque vector. the application of the robot body (2) dynamics to the next state of the physics engine 38 calculation; comparison of the simulated state with the target state, if If the simulation result is satisfactory, the action should be approved; otherwise, the action should be rejected. sending feedback to the analyzer, different physical parameters through parallel simulation by running numerous parallel simulations with parameters (field randomization) Ensuring the selection of the most robust action, output data from Simulation-5 The center is configured to enable the transfer to the Reality Compliance module. as in any of the above requests characterized by processor (3) a physical reality supported robotic control system (1).

17. Physics engine integration in the simulation-to-reality module 10 The simulated state sequence from the module is “S_sim ∈ R^{T×d_s}” and the actual state is... The observed situation from the world is defined as “S_real ∈ R^{d_s} (real-time sensor telemetry: joint positions, velocities, torques, end-impacting unit stance, external forces) enabling its use in a neural network, this network's modeling error, sensor noise, and 15 during training to learn actuator dynamics differences, using field randomization. performing random sampling of physical parameters, thereby enabling the model ensuring resistance to different physical conditions, compatible physical feature extraction to extract the physical properties of objects from real-world data enabling online prediction, thereby idealizing the physics engine. Modeling error caused by friction, elasticity, and delay, Encoder 20 quantization, IMU drift, F / T sensor bias, sensor noise, and motor noise. Learning actuator dynamics in terms of inertia, backslash, torque, and ripple. to ensure that the actions performed result in simulations that are consistent with reality. “a_adapted ∈ R^{d_a}” and the updated physical parameters “{μ_real, m_real, 25 with central processor (3) configured to enable obtaining "I_real}". physical as in any of the above characterized claims a reality-supported robotic control system (1). 39 18. With the secure approval controller inside, the action is on the robot body (2) it must undergo security verification before implementation, During safety verification, engine datasheet values, safety limit, zero The moment point must remain within the support polygon, between the robot body (2) and the environment. The distance between them being greater than the safety distance means that the kinetic and potential 5 operations are performed such that the total energy does not exceed the maximum energy. to ensure, by optimizing the online trajectory, if any action is taken If it violates the limitation, it should be corrected with a second-order program solver. the process involves generating an emergency stop command, if a correction is needed. If this is not possible, ensure that the "safe pose" command is generated. 10 The commands generated as a result of the operations are sent to the Robot Control Interface. characterized by the central processor (3) configured to enable transmission. as in any of the above requests, supported by physical reality a robotic control system (1).

19. The action confirmed by the robot management interface inside, the robot body (2) to ensure that it is transmitted to the low-level controller, both end- To ensure support for both the domain (Cartesian) space and the joint space, End-effect unit control (x, y, z, roll, pitch, yaw - 6 degrees of freedom), joint space control (joint angle variation), task space control (operational 20 To enable the use of control modes such as space control (OSC), low-level controller, proportional-derivative (PD) control, direct torque control or by using impedance control, commands to the motors (2) in the robot body. enabling the transmission of output data via closed-loop feedback. 25 characterized by the central processor (3) configured to transmit to the loop module. as in any of the above requests, supported by physical reality a robotic control system (1).

20. The robot body (2) in the closed-loop feedback loop module inside Real-world sensor telemetry (joint positions q, velocities q̇, torques τ, end-effect 30) 40 (unit orientation x_ee, external forces F_ext, images I_t) and Physics Engine Joining The simulated state sequence from the module is in the form “S_sim ∈ R^{T×d_s}”. the use of data, the learning algorithm within it execution, online adaptation in the learning algorithm (1-2 after each action) (Rapid adaptation with gradient step), meta-learning (Model-Agnostic Meta-Learning, 5 MAML enables rapid adaptation to new objects and environments, and continuous learning (Elastic Weight). Consolidation (EWC: Learning new tasks without forgetting old ones) Ensuring that the steps outlined are carried out, with an updated output. Network weights “θ_residual”, updated physical property estimates “{μ_real, The updated DPCL weights are in the form of 10: 'm_real, I_real}” and 'W_g, b_g'. obtaining information, output data / information with physics-based input. to the encoder, to the physical world model, to the simulation-to-reality conversion module with the central processor (3) configured to ensure that it is transferred as a feed physical as in any of the above characterized claims a reality-supported robotic control system (1). 15

Images