Body intelligence physical illusion prediction and pre-intervention method and body intelligence system

Abstract

This application discloses an embodied intelligent method and system for predicting and intervening in physical illusions. It replaces ordinary Markov chains with a hidden Markov risk transmission model involving nonlinear coupling of multiple risk sources, introduces Riemann geodesic distance for physical consistency verification, and employs the absorption Markov first-arrival time theory and risk geodesic entropy to quantify risk evolution. Before action execution, it obtains the state probability distribution, absorption state first-arrival probability, expected risk arrival time, and risk geodesic entropy for each step, thereby achieving forward-looking quantification of physical illusion risks. Based on this, risk levels are classified according to the absorption state first-arrival probability, and pre-intervention is achieved through action parameter correction, local action sequence replanning, and backtracking action sequence execution. This proactively reduces physical illusion risks before the risk reaches an irreversible absorption state, improving the operational safety and task execution continuity of the embodied intelligent system in unstructured environments and distributed working conditions.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to embodied intelligence physical illusion prediction and intervention methods and embodied intelligence systems. Background Technology

[0002] Embodied intelligence, as a core technological paradigm for the interaction between artificial intelligence and the physical world, relies on a vision-language-action model to achieve an end-to-end closed loop of perception, reasoning, and execution. It has been widely applied in scenarios such as industrial precision assembly, warehousing and logistics sorting, humanoid robot services, and human-machine collaborative integration. As the complexity of work scenarios increases and the level of task refinement deepens, intelligent agents need to complete dynamic interactive tasks in unstructured environments and distributed working conditions, which places extremely high demands on the safety and reliability of action execution.

[0003] However, existing vision-language-action models are essentially data-driven statistical fitting paradigms, failing to internalize core prior knowledge such as rigid body dynamics, spatial geometric constraints, contact force mechanisms, and causal evolution logic of the real physical world. This leads to frequent physical illusions in non-standard interaction scenarios, operations on unknown objects, and execution under boundary conditions—the semantic logic of the action commands output by the model is consistent, but in reality, they violate physical rules and spatial constraints, manifesting as missed grasps, trajectory collisions, inaccurate force control, and joint over-limits. Once physical illusions are triggered, they can directly cause material damage, equipment collisions, tissue injuries, and even personal safety accidents, representing a core safety bottleneck restricting the large-scale deployment of embodied intelligence.

[0004] Current protection solutions for embodied intelligent physical safety all fall under the "real-time detection-passive intervention" model: sensors such as force, vision, and torque monitor the execution status in real time, triggering an emergency stop when anomalies such as collisions, slippage, or exceeding limits are detected. This model has inherent flaws: risk detection and action execution occur simultaneously; by the time a risk is detected, the action has already been partially executed, and physical damage has already occurred partially or irreversibly. It can only passively respond to anomalies that have already occurred, lacking the ability to proactively predict risks in subsequent action sequences and failing to actively intervene and avoid risks in their early stages. Therefore, there is an urgent need to break through the existing passive protection technical framework and construct a physical illusion safety protection system with forward-looking prediction and proactive intervention capabilities, minimizing the impact on task efficiency while ensuring operational safety. Summary of the Invention

[0005] This application aims to propose an embodied intelligence method and embodied intelligence system for predicting and intervening in physical illusions, which can solve the technical problems in the prior art where the detection of physical illusion risks lags behind the execution of actions, lacks the ability to predict risks in subsequent action sequences, and can only passively block rather than actively intervene in the early stages of risks.

[0006] In a first aspect, embodiments of this application provide a method for predicting and intervening in embodied intelligence physical illusions, applied to an embodied intelligence system. The method includes: Acquire the multimodal perception observation data, action sequence data to be executed, physical prior rule base data, and historical risk transmission sample base data of the embodied intelligence system; Based on a hidden Markov risk transmission model with nonlinear coupling of multiple risk sources, a physical illusion risk transmission field is calculated using the multimodal sensing observation data, the action sequence data to be executed, the physical prior rule base data, and the historical risk transmission sample base data. The physical illusion risk transmission field includes the state probability distribution, the first arrival probability of the absorption state, the expected risk arrival time, and the risk geodesic entropy for each action step. The state probability distribution represents the probability that the system is in each hidden risk state under each action step. The hidden risk states include safe state, intermediate state, and absorption state. The absorption state is an irreversible risk state. The first arrival probability of the absorption state represents the probability of triggering the absorption state for the first time. The expected risk arrival time represents the remaining expected time for triggering the absorption state for the first time. The risk geodesic entropy represents the uncertainty and severity of risk evolution. A pre-intervention strategy is executed based on the physical illusion risk transmission field and the action sequence data to be executed, in order to reduce the physical illusion risk before reaching the absorption state. The pre-intervention strategy includes action parameter correction, local action sequence replanning, and action sequence rollback.

[0007] Secondly, embodiments of this application provide an embodied intelligent system, including an electronic device, the electronic device including a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the embodied intelligent physical illusion prediction and pre-intervention method as described in the first aspect embodiment above.

[0008] The embodied intelligent physical illusion prediction and pre-intervention method and embodied intelligent system of this application replace ordinary Markov chains with a hidden Markov risk transmission model with nonlinear coupling of multiple risk sources, introduces Riemann geodesic distance to complete physical consistency verification, and uses the absorption Markov first-arrival time theory and risk geodesic entropy to quantify risk evolution. Before the action is executed, the state probability distribution, absorption state first-arrival probability, expected risk arrival time, and risk geodesic entropy of each step are obtained, thereby achieving forward quantification of physical illusion risk. On this basis, risk levels are divided according to the absorption state first-arrival probability, and pre-intervention is carried out through action parameter correction, local action sequence replanning, and backtracking action sequence execution. Before the risk reaches the irreversible absorption state, the physical illusion risk is actively reduced, reducing problems such as grasping failure, trajectory collision, force control inaccuracy, and joint over-limit, thereby improving the operational safety and task execution continuity of the embodied intelligent system in unstructured environments and distributed working conditions.

[0009] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing this application. Attached Figure Description

[0010] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of an embodiment of the method for predicting and intervening in embodied intelligent physical illusions according to this application. Detailed Implementation

[0011] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0012] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0013] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0014] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0015] Existing research on technologies related to physical safety protection and hallucination detection in embodied intelligence mainly focuses on three technical directions: physical hallucination detection, robot safety control, and risk transmission assessment, as detailed below: In the field of physical illusion detection technology, existing solutions can be divided into two categories: single-step prior verification and real-time feedback detection. Transformer-based visual agent physical prior verification mechanisms perform millisecond-level feasibility verification before instruction execution, identifying physical illusions of single-step actions from three dimensions: geometric compatibility, dynamic rationality, and state consistency. Real-time detection methods based on multi-sensor fusion compare real-time data from six-dimensional force sensors, joint encoders, and visual observations to identify abnormal states such as collisions, slippage, and exceeding limits during execution. Visual consistency verification methods detect perceptual physical illusions by comparing the semantic differences between the actual observed image and the model's expected image. All three methods perform independent detection for single-step actions or the current moment, only outputting the illusion judgment result at the current moment. They cannot predict the risk evolution trend in subsequent action sequences, and the detection lags behind action execution, lacking forward-looking predictive capabilities.

[0016] In the field of robot safety control technology, existing solutions primarily rely on passive triggering for protection. Traditional industrial robots employ mechanisms such as safety distance protection, speed limits, and hard limits, defining safe operating boundaries through preset thresholds. Collaborative robots introduce technologies such as collision detection emergency stops, force limiting, and compliant control, triggering a stop or retreat upon contact with obstacles or human bodies. Layered safety gate mechanisms like SafeFlow filter risk commands layer by layer at three levels: input semantics, generation process, and output results. All of these solutions are threshold-triggered passive response modes, only able to perform passive operations such as stopping and retreating after a risk event is triggered, unable to proactively correct motion trajectories to avoid risks before they occur; furthermore, the safety strategies are coarse-grained, easily causing frequent task interruptions and significant losses in operational efficiency.

[0017] In the field of risk transmission and assessment technology, Markov chain models have been applied in areas such as industrial system safety and fault prediction. SafetyDrift uses Markov chains to analyze the safety drift trajectory of intelligent agents and quantifies the evolutionary trend of the system deviating from a safe state; industrial process risk assessment uses Markov models to model the accumulation and transmission process of faults and calculate the probability of system failure; some robot motion planning methods introduce Markov decision processes to solve for the optimal path in uncertain environments. The above-mentioned Markov risk models are all aimed at macro-level industrial systems or information system scenarios, and do not specifically model the transmission path of perception deviation-action inaccuracy-physical violation-collision damage caused by embodied intelligence physical illusion; moreover, the model output is only a risk assessment conclusion, which is not connected with the robot's execution control link, cannot directly drive hierarchical pre-intervention actions, and does not have the closed-loop capability of prediction-intervention.

[0018] In summary, existing technologies suffer from three common core deficiencies: First, physical illusion detection is entirely single-step real-time, lacking a multi-step forward-looking prediction mechanism. By the time risk is detected, the action has already been partially executed, making it difficult to avoid physical damage. Second, safety controls are all passive blocking mechanisms, lacking proactive pre-intervention capabilities and only capable of reactive responses, unable to proactively correct and resolve risks at their nascent stage. Third, risk assessment and execution control are disconnected, failing to consider the cumulative effect of risk transmission in the action sequence and unable to translate risk prediction conclusions into end-to-end hierarchical safety control strategies. This application addresses these deficiencies by proposing an embodied intelligent physical illusion prediction and pre-intervention method based on Markov risk transmission. It constructs an action-driven risk transmission model, enabling multi-step forward-looking risk deduction and hierarchical proactive pre-intervention, forming a closed-loop safety protection mechanism of prediction-intervention-feedback.

[0019] The purpose of this application is to overcome the shortcomings of the existing technology and provide an embodied intelligent physical illusion prediction and pre-intervention method based on Markov risk transmission, which solves the following technical problems: 1. Existing illusion detection is all real-time detection during execution. When a risk is detected, the action has already been partially executed and may still cause physical damage; 2. It lacks the ability to predict the risk of subsequent action sequences and cannot predict the transmission, accumulation and amplification effects of risk; 3. It can only passively block risks and cannot actively pre-intervene and correct trajectories at the risk incubation stage; 4. The single-step independent judgment mode ignores the state dependence and risk chain reaction between actions.

[0020] The technical solution of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are some embodiments of this application, not all embodiments.

[0021] See Figure 1 , Figure 1 This is a flowchart of a method for predicting and intervening in embodied intelligence physical illusions according to an embodiment of this application. This method is applied to an embodied intelligence system and includes, but is not limited to, steps S100 to S300: Step S100: Obtain multimodal perception observation data, action sequence data to be executed, physical prior rule base data, and historical risk transmission sample base data from the embodied intelligence system; Step S200: Based on the Hidden Markov Risk Transmission Model with Nonlinear Coupling of Multiple Risk Sources, calculate the physical illusion risk transmission field according to multimodal sensing observation data, action sequence data to be executed, physical prior rule base data, and historical risk transmission sample base data. The physical illusion risk transmission field includes the state probability distribution, first arrival probability of absorption state, expected risk arrival time, and risk geodesic entropy for each action step. The state probability distribution represents the probability of observing the actual observed value under each hidden risk state. Hidden risk states include safe state, intermediate state, and absorption state. The absorption state is an irreversible risk state. The first arrival probability of absorption state represents the probability of triggering the absorption state for the first time. The expected risk arrival time represents the remaining expected time for triggering the absorption state for the first time. The risk geodesic entropy represents the uncertainty and severity of risk evolution. Step S300: Execute a pre-intervention strategy based on the physical illusion risk transmission field and the action sequence data to be executed, in order to reduce the physical illusion risk before reaching the absorption state. The pre-intervention strategy includes action parameter correction, local action sequence replanning, and action sequence rollback.

[0022] In this embodiment, a Hidden Markov Risk Transmission Model with nonlinear coupling of multiple risk sources is used instead of a regular Markov chain. Riemann geodesic distance is introduced to complete physical consistency verification. The first-arrival time theory of the Absorbed Markov and the risk geodesic entropy are used to quantify risk evolution. Before the action is executed, the state probability distribution, the first-arrival probability of the absorbing state, the expected risk arrival time, and the risk geodesic entropy of each step are obtained, thereby achieving forward quantification of physical illusion risk. Based on this, the risk level is divided according to the first-arrival probability of the absorbing state. Pre-intervention is carried out through action parameter correction, local action sequence replanning, and backtracking action sequence execution. Before the risk reaches the irreversible absorbing state, the physical illusion risk is actively reduced, reducing problems such as grasping failure, trajectory collision, force control inaccuracy, and joint over-limit, thereby improving the operational safety and task execution continuity of the embodied intelligent system in unstructured environments and distributed working conditions.

[0023] In step S100 above, the multimodal sensing observation data includes, but is not limited to, the target pose estimation value output by the visual sensor, the contact force / torque value output by the force sensor, and the joint angle / angular velocity / angular acceleration data output by the joint encoder. The sampling frequency of the multimodal sensing observation data is 100Hz. The multimodal sensing observation data serves as the basis for risk state observation and is used for physical consistency assessment and launch probability calculation.

[0024] The action sequence data to be executed includes, but is not limited to, the N-step trajectory point sequence output by the motion planning module, which includes the end pose, joint angle, desired velocity, desired torque, and execution timing for each step. The typical sequence length of the action sequence data to be executed is 5 to 10 steps. The action sequence data to be executed serves as the driving input for risk transmission and is used to calculate the transfer excitation of the risk state by the action.

[0025] The physical prior rule base data includes, but is not limited to, robot rigid body dynamic parameters (mass, inertia, link length), workspace geometric constraints, collision detection thresholds, force safety thresholds, and joint limit parameters. This physical prior rule base data serves as a benchmark for verifying physical consistency, used to determine whether sensory observations violate physical laws.

[0026] The historical risk transmission sample database includes, but is not limited to, "observation-state-action-consequence" quadruple samples recorded in historical operations, covering all scenarios from normal operating conditions to mild hallucinations, moderate hallucinations, and severe failures. The sample size requirement is at least 10,000 valid samples. This historical risk transmission sample database serves as the prior knowledge for initializing model parameters, used to estimate the initial values ​​of the transition probability and the emission probability.

[0027] The above step S200 includes, but is not limited to, steps S210 to S240: Step S210: Calculate the consistency deviation coefficient based on the multimodal sensing observation data and the physical prior rule base data. The consistency deviation coefficient characterizes the degree of agreement between the sensing observation data and the physical prior rules. Step S220: Based on the hidden Markov risk transmission model with nonlinear coupling of multiple risk sources, construct a multi-level hidden risk state, which includes a safe state, an intermediate state and an absorption state, and the absorption state is an irreversible risk state. Step S230: Calculate the initial state distribution, action-dependent transition probability matrix, and corrected launch probability based on the consistency deviation coefficient, the action sequence data to be executed, and the historical risk transmission sample library data. The initial state distribution represents the initial probability that the system is in each hidden risk state at the time of task start-up. The action-dependent transition probability matrix represents the directed transition prior probability between each pair of hidden risk states. The corrected launch probability represents the probability of observing different observation values ​​under each hidden risk state. Step S240: Based on the initial state distribution, action-dependent transition probability matrix and modified emission probability, forward recursive calculation is performed to obtain the state probability distribution, absorption state first arrival probability, expected risk arrival time and risk geodesic entropy for each action step.

[0028] The above step S210 includes, but is not limited to, steps S211 to S214: Step S211: Based on the dynamic parameters and spatial constraints of the physical prior rule base data, construct the physical feasible state manifold for robot operation. The geodesic distance on the physical feasible state manifold corresponds to the degree of violation of physical rules. Step S212: Project the multimodal sensing observation data onto the physical feasible state manifold to obtain the projection points in the physical feasible domain corresponding to different observation points, and calculate the Riemann geodesic distance between the observation points and the projection points. The observation dimensions of the observation points include vision, force perception and joints. Step S213: Calculate the consistency scores for the three dimensions of visual, force and joint based on the Riemann geodesic distance. The consistency scores characterize the degree of physical consistency. Step S214: Calculate the consistency deviation coefficient based on the consistency scores of the three dimensions: visual, force, and joint.

[0029] In step S211 above, the physically feasible state manifold This is a high-dimensional Riemannian manifold constructed based on robot kinematic constraints and dynamic equations. Specifically, the robot's joint space is mapped to the task space, and a set of legal states satisfying physical laws is defined by combining geometric constraints such as link lengths and joint limits, as well as dynamic parameters such as mass and inertia. On this manifold, the shortest path between any two points is the geodesic distance, the length of which reflects the physical constraint cost that must be overcome to transition from one state to another. This is achieved by constructing a physically feasible state manifold. This can transform the degree of violation of abstract physical rules into a geometric distance metric on a manifold, providing a mathematical basis for subsequent consistency checks.

[0030] In step S212 above, the current observation value of the multimodal sensing observation data is... ( For observations in the visual dimension, For the observations in the force dimension, Projecting the joint-dimensional observations onto the physically feasible state manifold The projection points within the physically feasible region are obtained. Calculate the Riemann geodetic distance between the observation point and the projection point: ;Formula (1) in, The Riemann geodetic distance between the observation point and the projection point. For physically feasible manifolds The Riemannian metric tensor The shortest geodesic line between two points. For physically feasible manifolds Local coordinate vector on, superscript and It is the dimension index of the coordinate components and is the standard notation for tensor analysis.

[0031] For a A 3D manifold, the complete coordinate vector is denoted as . ,in, Represents the first point on the manifold Each coordinate component Represents the first point on the manifold Each coordinate component , It is the differential increment of the corresponding coordinate component, representing the geodesic. The previous infinitesimal displacement, in the th , No. A tiny change in each coordinate direction.

[0032] In the physically constrained manifold scenario of this application, coordinates Each component corresponds to a different dimension of the physical state, such as the end-effector pose coordinates, joint angles, and contact force components. That is the first one The values ​​of each physical quantity.

[0033] The above and It is a tensor summation index, following Einstein's summation convention: when the same index appears as both a superscript and subscript in an expression, it defaults to all possible values ​​of that index (from 1 to the manifold dimension). Automatic summation eliminates the need to write additional summation symbols. Simply put: and It is a "counter" used to traverse all dimensions of a manifold, by measuring tensors. By weighted summing of the differential increments in each coordinate direction, a true distance metric that conforms to physical constraints is obtained.

[0034] In step S213 above, the consistency scores for the three observation dimensions are calculated respectively. A lower consistency score indicates poorer physical consistency and a higher risk of hallucination. , ,in, The Riemann geodetic distance between the observation point and the projection point. The noise standard deviation for the corresponding dimension is given by the prior parameters of the physical prior rule base data.

[0035] In some implementations, a consistency qualification threshold is set. For consistency scores below the consistency pass threshold The dimension is marked as "physical inconsistency" and its weight is increased in subsequent launch probability corrections, so it should be given special attention.

[0036] In step S214 above, the consistency deviation coefficient .

[0037] It should be noted that the perception-physical consistency assessment in step S210 is not an independent hallucination detection step, but a necessary preliminary step for risk calculation. Its output directly corrects the observation likelihood of the subsequent risk model, thus preventing the transmission of perception hallucinations into risk misjudgment from the source.

[0038] In step S220 above, a 5-level hidden risk status is defined. ,in, For a safe state, , and It is an intermediate state. It is in the absorption state (risk of irreversible collision / damage), where, , , and As a transient state, the hidden risk state is specifically defined as follows: : Safe condition, no physical hallucinations; Mild perceptual deviation, the beginnings of perceptual hallucination; Moderate motor inaccuracies, with executive hallucinations. Severe physical violation, worsening of physical hallucinations; Dangerous collision / damage, irreversible risk.

[0039] The above step S230 includes, but is not limited to, steps S231 to S237: Step S231: Obtain the pre-stored initial state distribution under the system initialization state; Step S232: Based on the historical risk transmission sample database, the initial state transition probability matrix is ​​obtained using the maximum likelihood estimation method; Step S233: Based on the action features of the action sequence data to be executed, extract the risk factors for each action. The risk factors are obtained by weighted normalization of action speed, acceleration, and contact force level. Step S234: Based on historical risk transmission sample database data, the risk increase transfer increment matrix and risk decrease transfer increment matrix are obtained by statistical difference method. The risk increase transfer increment matrix represents the probability of high-risk actions amplifying to a worse state, and the risk decrease transfer increment matrix represents the probability of low-risk actions improving to a better state. Step S235: Calculate the action-dependent transition probability matrix based on the initial state transition probability matrix, risk factor, risk-increasing transition increment matrix, and risk-decreasing transition increment matrix. Step S236: Based on historical risk transmission sample database data, a connection function is used to characterize the nonlinear dependency relationship of the three observation dimensions of vision, force, and joint, and the joint launch probability is constructed. Step S237: The joint launch probability is corrected using the consistency deviation coefficient to obtain the corrected launch probability.

[0040] In step S231 above, the initial state distribution It is the time when the task starts ( The initial probability distribution of the system in 5 hidden risk states is a 5-dimensional vector that sums to 1. There are three main engineering methods to obtain this distribution: 1. Prior Steady-State Method (Cold Start Scenario). When the system is running for the first time and there is no prior observation data, the initial state distribution is obtained by directly calculating the frequency of state occurrence at the initial moment of all tasks based on historical samples from the historical risk transmission sample library. Typical calibration results are as follows: This corresponds to a 95% probability of being in a safe state, a 4% chance of mild perception deviation, a 1% chance of moderate movement inaccuracy, and no severe or higher risks.

[0041] 2. Observation-Estimation Method (most commonly used in warm-start scenarios). After the task starts, the first frame of multimodal sensing observation data is acquired. Combined with the sensing-physical consistency assessment results (i.e., consistency deviation coefficient), the initial state distribution is calculated through Bayesian inference: ,in, As the prior initial distribution, The initial observations after correction for consistency deviation coefficient. Corresponding state The emission probability, i.e., in state The initial observation value was observed. The probability, and then for Normalization yields the final initial state distribution. .

[0042] 3. Task Continuation Method (for continuous tasks): If there are multiple consecutive tasks, the state probability distribution at the end of the previous task is directly used as the initial state distribution for the next task. This enables the continuous transmission of risk status without the need for repeated initialization.

[0043] In step S232 above, the initial state transition probability matrix is ​​obtained using the maximum likelihood estimation method based on historical risk transmission sample data. ,in, Indicates from state One step to The prior probabilities satisfy the probability constraint that the sum of the rows is 1.

[0044] In step S233 above, the risk factors for each action are... Used to quantify the potential driving force of this action on the evolution of the system's risk state. Risk factors It is obtained by weighted normalization of action speed, acceleration, and contact force level. The corresponding weights of action speed, acceleration, and contact force level are determined based on the statistical distribution of actions of different risk levels in the historical risk transmission sample database, or by empirical calibration.

[0045] In step S234 above, the risk escalation transition increment matrix and the risk descent transition increment matrix are sensitivity increment matrices of action risk to state transition probability. They are used to characterize the directional shift effect of actions of different risk levels on the evolution of risk states. Both are calibrated based on historical risk transmission sample data through statistical differencing methods. The specific construction process is as follows: 1. Data stratification and grouping statistics Based on historical action-state quadruples from the historical risk transmission sample database, the samples are divided into three groups according to the magnitude of the risk factor of the action, and the empirical transition probability matrix of each group is calculated: Benchmark group: Selected risk factors within the median range ( From the sample, the baseline transition probability matrix is ​​obtained statistically. This corresponds to the state transition pattern under medium-risk actions; High-risk group: Selected groups have risk factors in the high range ( From samples of high-speed movement, forceful insertion, and high-acceleration actions, a high-risk transition probability matrix is ​​obtained. ; Low-risk group: Selected groups have risk factors in the low range ( From samples (such as low-speed fine-tuning, light contact, and stationary standby actions), a low-risk transition probability matrix is ​​obtained. .

[0046] 2. Incremental matrix difference calculation Two increment matrices are obtained by inter-group matrix differencing: Risk escalation transfer increment matrix The difference between the high-risk group matrix and the baseline group matrix, i.e. Its physical meaning is: the change in the probability of transition to each state when the action risk factor increases from the baseline level to the highest level. Elements in the matrix that transition to a worse risk state are positive (e.g., ...). , , (Position), elements that transition to a better state have negative values, reflecting the effect of high-risk actions accelerating risk deterioration.

[0047] Risk reduction transfer increment matrix The difference between the low-risk group matrix and the benchmark group matrix, i.e. Its physical meaning is: the change in the probability of transition to each state when the action risk factor drops from the baseline level to the minimum level. Elements in the matrix that transition to a better risk state are positive (e.g., ...). , , (Position), elements that transition to a worse state have negative values, reflecting the effect of low-risk actions promoting risk convergence.

[0048] 3. Probability conservation constraint calibration After calibration, the increment matrix needs to be constrained and calibrated to ensure that the corrected transition matrix still satisfies the probability axiom that the sum of the elements in each row is 1: the sum of the elements in each row of the two increment matrices must be 0.

[0049] It should be noted that if there are insufficient historical samples in the initial stage, state transition samples under different risk levels can be generated through physical simulation (such as rigid body dynamics simulation or Monte Carlo simulation) to complete the initial calibration. After the system goes online, the values ​​of the two incremental matrices are continuously iteratively optimized using actual operating data through the Bayesian online update mechanism to make them constantly adapt to the actual working conditions.

[0050] In step S235 above, the constraint formula for the action-dependent transition probability matrix is: ;Formula (2) in, The action-dependent transition probability matrix, Let be the initial state transition probability matrix. As a risk factor, This is a risk escalation transfer increment matrix. This is the risk reduction transition increment matrix. The action-dependent transition probability matrix. It is necessary to ensure that the sum of each row is always 1.

[0051] Formula (2) forms the linear interpolation framework: when At that time, the transition increment matrix takes the high-risk boundary. ;when At that time, take the low-risk boundary. The action of any risk factor in the middle corresponds to the linear interpolation between the two boundaries, realizing a smooth mapping from action risk to transfer probability.

[0052] The above step S236 includes, but is not limited to, steps S2361 and S2362: Step S2361: Based on the historical risk transmission sample database, group the data according to the hidden risk state, and statistically analyze the conditional marginal distribution of the three observation dimensions of vision, force perception, and joints under each hidden risk state. Step S2362: A three-dimensional joint distribution is constructed by using a connection function to construct the conditional marginal distributions of the three observation dimensions of vision, force, and joint, and the joint emission probability is obtained.

[0053] To address the shortcomings of traditional HMMs that assume each observation dimension is independent, steps S2361 and S2362 above use a pair-copula function to characterize the nonlinear dependencies between the three observation dimensions of vision, force, and joint, thus constructing a joint emission probability that better reflects the actual risk coupling characteristics.

[0054] The conditional marginal distributions of the three observation dimensions—visual, force, and joint—are as follows: , and A three-dimensional joint distribution is constructed by using pairwise connection functions to examine the conditional marginal distributions of the three observation dimensions: visual, force, and joint. ;Formula (3) in, For a three-dimensional joint distribution constructed based on pairwise connection functions, , For the probability integral transform of the conditional marginal distribution, Let be the cumulative distribution function of the three-dimensional standard normal distribution. It is the inverse function of the standard normal distribution function. The correlation coefficient matrix is ​​estimated from samples in the historical risk transmission sample database.

[0055] The three-dimensional joint distribution is the core underlying mathematical foundation for calculating the joint emission probability. Its key function is to address the strong assumptions of traditional Hidden Markov Models (HMMs): traditional HMMs assume that the visual, force, and joint observation dimensions are independent, and the joint emission probability is directly equal to the product of the conditional marginal distributions of each dimension. However, in real physical systems, perceptual biases are coupled. For example, a shift in visual positioning is often accompanied by abnormal end-effector contact force and joint angle synchronization deviations, exhibiting a nonlinear correlation among the three. Pairwise connection functions can concatenate the three one-dimensional conditional marginal distributions, accurately characterizing the nonlinear dependencies between the three observation dimensions. This yields a three-dimensional joint probability density that better reflects real physical characteristics, ultimately outputting a more accurate joint emission probability and avoiding the risk of misjudgment caused by the independence assumption.

[0056] The joint launch probability is calculated using the following steps: Hidden risk status The conditional marginal distribution function in three dimensions was obtained statistically. , and and the corresponding probability density function , and ; For the current observation value Perform a probability integral transformation (homogenization mapping): , , ; Calculate the probability density using the Gaussian Copula function: ;Formula (4) in, For probability density, It is a one-dimensional standard normal density. The three-dimensional standard normal joint probability density. It is the inverse function of the standard normal distribution function. The correlation coefficient matrix is ​​estimated from samples in the historical risk transmission sample database.

[0057] Through probability density function , and and probability density The hidden risk state is calculated. Joint launch probability under : ;Formula (5) The above step S237 includes, but is not limited to, steps S2371 and S2372: Step S2371: Construct corrected weights based on the consistency deviation coefficient; Step S2372: The joint launch probability is weighted and corrected based on the corrected weights to obtain the corrected launch probability.

[0058] In step S2371 above, the consistency deviation coefficient A larger component indicates poorer physical consistency, less reliable observation, and a higher risk of perceptual hallucination in the corresponding dimension. Constructing corrected weights. The joint launch probability is weighted and corrected, with dimensions showing greater deviation having higher weight in risk assessment. ;Formula (6) in, To conceal the risk status The corrected emission probability is as follows. For joint launch probability, For visual dimension weights, For force perception dimension weights, For joint dimension weights, For dimension Hidden risk status The sensitivity coefficient is determined by historical samples from the historical risk transmission sample library: the higher the risk level, the greater the sensitivity coefficient to the deviation dimension.

[0059] It should be noted that the sensitivity coefficient is used to characterize the difference in importance of each observation dimension under different risk levels, and is used as an index factor to directly participate in the numerical calculation of the corrected emission probability (see formula (6)).

[0060] Five hidden risk states Normalization is performed to ensure the relative reasonableness of the probability distribution, and finally, a 5×1 corrected emission probability is assembled. : .

[0061] The above step S240 includes, but is not limited to, steps S241 to S245: Step S241: Based on the initial state distribution, action-dependent transition probability matrix and corrected emission probability, perform forward recursive calculation to obtain the state probability distribution of each action step; Step S242: Decompose the action-dependent transition probability matrix into a transient submatrix and an absorbing state transition submatrix. The transient submatrix represents the directed transition prior probability of a hidden risk state transitioning to a safe state or an intermediate state, and the absorbing state transition submatrix represents the directed transition prior probability of a hidden risk state transitioning to an absorbing state. Step S243: Calculate the first-arrival probability of the absorption state based on the initial state distribution, the transient submatrix, and the absorption state transition submatrix; Step S244: Calculate the expected risk arrival time based on the initial state distribution and the transient submatrix; Step S245: Calculate the risk geodesic entropy based on the state probability distribution.

[0062] In step S241 above, the forward variables are first initialized. ,in, To conceal the risk status The initial state distribution is as follows. To conceal the risk status The probability of observing the initial observation is obtained by adjusting the emission probability. Then, the forward variable is recursively calculated for each action step. : ;Formula (7) in, For the first The forward variables of the step action are obtained based on the initial state distribution; Hiding risk states in the action-dependent transition probability matrix A sudden change to a state of hidden risk The directed transition prior probability; For the first The predicted observations of the step action are obtained by forward dynamic estimation from the action sequence data of the action sequence to be executed; To conceal the risk status The next observation The probability of the predicted observation of the step action is obtained based on the corrected launch probability.

[0063] Forward variables Normalization is performed to obtain the first... Probability distribution of step action states : ;Formula (8) in, For the first Forward variables of the step action For the first Forward variables of the step action.

[0064] The embodiments in this application use predicted observations, which is determined by the characteristics of multi-step forward-looking risk extrapolation: we extrapolate the future before the action is executed. Due to the lack of real sensor feedback data, the probability of launch can only be calculated using predicted observations corresponding to the actions.

[0065] Specifically, it is based on the sequence data of actions to be executed. The motion command for each step is used to predict the expected observations in three dimensions (such as the predicted end pose, expected contact force, and expected joint angle) after the robot's forward kinematics and rigid body dynamics model is executed.

[0066] In step S242 above, the action-dependent transition probability matrix is ​​decomposed into transient submatrices. (correspond and absorption state transition matrix (corresponding to the transfer) ).

[0067] In step S243 above, the first arrival probability of the absorption state ,in, The initial state distribution, For transient submatrices, Let be the absorption state transition submatrix.

[0068] In step S244 above, the expected risk arrival time is... ,in, The initial state distribution, For transient submatrices, It is the identity matrix. It is a vector consisting entirely of 1s.

[0069] The expected risk arrival time is the core time benchmark parameter of the tiered pre-intervention strategy. It is a key quantitative basis that distinguishes "proactive pre-intervention" from "passive blocking." Its specific role is reflected in the following four aspects: 1. Basis for calculating the intervention timing of Level III active blockade This is the core application of the expected risk arrival time. During a high-risk warning, shutdown cannot be triggered randomly; the latest intervention initiation time must be calculated to ensure safe braking is completed before the risk reaches the absorption state. The corresponding formula is: ,in, This is the pre-intervention response period. It is the shortest time required for the equipment to brake safely (for example, it takes 0.3 seconds for a robotic arm to decelerate and stop).

[0070] like It can smoothly and safely stop according to the optimal deceleration curve, avoiding sudden stop impact; like : Immediately trigger the highest level of emergency braking; like The risk is determined to be unavoidable, triggering secondary protection (such as energy absorption buffering and torque unloading).

[0071] Without an expected risk arrival time, proactive blocking degenerates into a traditional threshold-triggered emergency stop, losing the core value of the "predictive time window".

[0072] 2. Two-dimensional assessment of the urgency of risk The first-arrival probability of the absorption state only answers the question of "the first state". What is the probability of triggering a risk? This answers the question of "how long on average will it take to trigger the risk," and the two constitute two complementary dimensions for risk assessment: both have a first-arrival probability of 0.05. This step involves high-urgency risk and requires immediate intervention. The current situation is of low urgency and risk; it is advisable to observe first or make minor adjustments.

[0073] By matching the two dimensions of "probability + time", we can avoid misintervention and insufficient intervention caused by a single probability threshold, and accurately match the triggering conditions of the three-level pre-intervention strategy.

[0074] 3. Effectiveness verification indicators for primary and secondary pre-interventions After performing motion parameter correction (Level 1) and local motion sequence replanning (Level 2), the corrected expected risk arrival time needs to be recalculated. Quantitatively assess the intervention effect: like A significant increase (e.g., from 5 steps to 15 steps) indicates that the intervention was effective and the risk has been mitigated; like The lack of significant change indicates insufficient correction and necessitates an upgrade in the intervention level.

[0075] The expected risk arrival time is the core feedback indicator for optimizing the closed-loop of pre-intervention strategies.

[0076] 4. Engineering design basis for system safety margin When the project is implemented, it can be based on The statistical values ​​are matched with the controller cycle, sensor sampling frequency, and security processor response latency to ensure that the system response speed is always faster than the risk evolution speed, providing a quantitative time margin benchmark for the overall security architecture design.

[0077] In step S245 above, based on the first Step state probability distribution Calculate the geodesic entropy of risk, and quantify the uncertainty and severity of risk evolution: ;Formula (9) in, For risk geodesy entropy, For the first The probability distribution of the state of the step action. To conceal the risk status to safe state The geodesic distance is a weighted indicator of the severity of risk. Risk geodesic entropy, as an amplifying factor of risk uncertainty, indicates a stronger risk penalty with higher entropy.

[0078] The state probability distribution of each action, the first arrival probability of the absorption state, the expected risk arrival time, and the risk geodesic entropy together constitute the physical illusion risk transmission field.

[0079] The above step S300 includes, but is not limited to, steps S310 and S320: Step S310: Determine the risk level based on the first arrival probability of the absorption state; Step S320: Based on the risk level, a pre-intervention strategy is executed according to the state probability distribution, the first arrival probability of the absorption state, the expected risk arrival time, the risk geodesic entropy, and the action sequence data to be executed, in order to reduce the risk of physical illusion before reaching the absorption state. The pre-intervention strategy includes action parameter correction, local action sequence replanning, and action sequence rollback.

[0080] In some implementations, in step S310 above, a lower first-reach probability of the absorption state indicates a lower likelihood of risk propagation to the absorption state and a higher probability that the system is in a relatively safe state; conversely, a higher first-reach probability of the absorption state indicates a more significant risk accumulation effect and a stronger tendency for the system to deviate from the safety baseline. Based on this, this embodiment sets a three-level risk threshold. , , Based on the first-arrival probability of the absorption state at each step in the risk transmission field of physical illusion Matching risk level: No risk: ; Low risk: ; Medium risk: ; High risk: .

[0081] It should be noted that the specific size of the three-level risk thresholds mentioned above can be adjusted according to the actual situation and should not be regarded as a limitation on this application.

[0082] In step S320 above, the optimal intervention action is solved with task completion efficiency as the objective function and risk not exceeding the corresponding level threshold as the constraint. Step S320 includes, but is not limited to, steps S321 to S323: Step S321: If the risk level is low risk, take the intermediate state as the correction target point, calculate the correction amount according to the state probability distribution and risk geodesic entropy, add the correction amount to the original action parameters of the action sequence data to be executed, and obtain the target action parameters. Execute the action according to the target action parameters. Step S322: If the risk level is medium risk, starting from the current state, solve the constrained Markov decision process based on the state probability distribution, the first arrival probability of the absorption state, the risk geodesic entropy and the action sequence data to be executed, regenerate the local action sequence that satisfies the preset risk constraints, and execute the action according to the local action sequence. Step S323: If the risk level is high risk, determine the target state for reversal based on the state probability distribution, calculate the reversal action sequence based on the target state for reversal, the expected risk arrival time, the risk geodesic entropy, and the action sequence data to be executed, and execute the action according to the reversal action sequence.

[0083] In step S321 above, if the risk level is low, the overall trajectory and timing of the original action sequence are not changed. Only a small correction is added to the dominant risk dimension, including four types of micro-operations: perception offset compensation, speed micro-reduction, attitude micro-adjustment, and force control margin tightening, so as to suppress the risk below the threshold at the lowest cost.

[0084] First, select the unsafe state with the highest probability from the intermediate states as the correction target. If the correction target is... (Perceptual bias dominant): The correction amount acts on the visual localization compensation dimension; if the correction target is... (Motion inaccuracy is dominant): Correction measures are applied to the velocity and posture dimensions; if the correction target is... (Physical violation dominant): The correction amount acts on the contact force control dimension.

[0085] Calculating the joint correction coefficient: The correction weight is determined by the proportion of the state probability distribution of the dominant state, and the correction margin is determined by the risk geodesic entropy. The joint correction coefficient is obtained by multiplying the two. : ;Formula (10) in, The baseline micro-correction factor is obtained from historical operating conditions and typically ranges from 0.1 to 0.2. To correct the target point, To correct the target The probability distribution of the state under the following conditions To conceal the risk status The probability distribution of the state under the given conditions For risk geodesic entropy.

[0086] Calculate the final correction amount: Apply the correction in the opposite direction of the risk gradient to obtain the correction amount superimposed on the original action parameters. : ,in, First arrival probability of absorption state The gradient vector of the action parameters of the action sequence data to be executed is derived from the sensitivity of the action-dependent transition probability matrix to the action; the negative sign indicates that the correction direction is the direction of risk reduction.

[0087] Target action parameters ,in, The original action parameters are the action sequence data to be executed.

[0088] In step S322 above, if the risk level is medium risk, discard the first step in the original action sequence. For the trajectory segment after the step, starting from the current state, solve the constrained Markov decision process (CMDP), regenerate the local action sequence that satisfies the risk constraints, and adjust the range to cover all dimensions of path, speed, attitude, and contact force, balancing task efficiency and safety margin.

[0089] Constructing a bivariate risk-sensitive reward function: Incorporating the risk severity weighted by state probabilities and the uncertainty represented by geodesic entropy into the penalty term, and solving for the most efficient action sequence. : ; Formula (11) in, The task completion efficiency reward is calculated by weighting the action time, path length, and execution accuracy in the action sequence data to be executed. The baseline risk weighting coefficient is determined by the equipment safety level requirements; To conceal the risk status The probability distribution of the state under the given conditions; To conceal the risk status to safe state The geodetic distance; For risk geodesic entropy.

[0090] Set risk constraints: the first-reach probability of the absorbing state in all steps after reprogramming must not exceed the lower limit of the medium-risk threshold, ensuring that the risk falls back to the low-risk range. Solve the above CMDP problem through value iteration or policy iteration, and output the corrected first-step probability. Step to the first Step-by-step local action sequence.

[0091] In step S323 above, if the risk level is high, the active safety docking procedure is immediately initiated, and three actions are performed: graded deceleration, attitude correction, and safe docking, to avoid secondary impacts and equipment damage caused by emergency braking, and to ensure that a safe state is reached before irreversible risks occur.

[0092] The target state for backoff is determined based on the state probability distribution, and the optimal backoff target is determined by the cumulative distribution of transient probabilities, minimizing efficiency loss while ensuring safety. ;Formula (12) in, To revert to the target state, To conceal the risk status The probability distribution of the states.

[0093] The physical meaning of formula (12) is: the safest state with a cumulative probability of 90% is taken as the backoff target.

[0094] like Only attitude correction and moderate deceleration are performed; complete return to zero is not required. like Perform full braking and complete retraction to the baseline safe posture.

[0095] Calculate the braking strength coefficient The degree of braking aggression is adjusted by the risk geodesic entropy; the higher the entropy, the greater the risk uncertainty, and the greater the braking deceleration. ,in, The reference braking strength coefficient is calibrated based on the rigid body dynamics parameters and load characteristics of the equipment; For risk geodesic entropy.

[0096] Calculate the optimal braking trajectory parameters: Combine the expected risk arrival time and braking intensity coefficient to calculate the deceleration and timing of smooth braking. : ;Formula (13) in, For the first The nominal speed of the original step action is obtained from the action sequence data to be executed; The expected arrival time of the risk; This is the braking strength coefficient; For safety time margin, a typical value of 0.2~0.5s is used, which is determined by engineering safety specifications.

[0097] Deceleration and Timing of Smooth Braking Synchronous matching The attitude recovery rate is calculated, and the final output is a complete sequence of backtracking actions.

[0098] The following is a detailed description of the embodied intelligent physical illusion prediction and intervention method of this application, using a specific embodiment.

[0099] This embodiment is applied to the precision PCB board insertion process of a six-degree-of-freedom collaborative robotic arm in a 3C electronics production line. The task involves the precise insertion of 0805 surface mount capacitors onto PCB pads, with stringent positional accuracy requirements. 0.02mm, contact force safety threshold 5N. The robotic arm hardware configuration includes: a six-axis collaborative robotic arm body, a six-dimensional force / torque sensor at the end effector, a monocular structured light vision positioning system, and a joint encoder assembly. Traditional protection solutions employ real-time force collision detection + emergency stop mechanisms, which suffer from problems such as component breakage due to insertion deviations, PCB pad scratches from collisions, and high rates of job interruption due to accidental shutdowns.

[0100] Step 1: Input data acquisition (data A~data D) Data A: Multimodal sensing observation data, including component / pad pose estimation values ​​output by structured light vision (sampling frequency 30Hz), three-dimensional contact force and torque values ​​output by a six-dimensional force sensor (sampling frequency 100Hz), and joint angle / angular velocity values ​​output by a 6-channel joint encoder (sampling frequency 200Hz). Current reference observation values: visual positioning lateral deviation 0.015mm, force sensing no-load baseline deviation 0.2N, and all joint angles within the nominal range.

[0101] Data B: Action sequence data to be executed, including a 5-step insertion action sequence output by the action planning module: grab and lift, horizontal transfer, deceleration and alignment, vertical approach and pad insertion. Each step includes end pose, motion speed, expected contact force and timing nodes. The total execution time of a single task is 1.8s.

[0102] Data C: Physical prior rule base data, including robotic arm link parameters (arm span 600mm, end effector rated load 0.5kg), joint movement limits, PCB board geometric constraint boundaries, and insertion force safety threshold of 5N; it also includes prior noise for each modality: visual positioning noise standard deviation is 0.02mm, force noise standard deviation is 0.3N, and joint angle noise standard deviation is 0.01°.

[0103] Data D: Historical risk transmission sample database data, including 12,600 operation samples accumulated from the production line in the past 3 months. Each sample is labeled with a corresponding safety status tag, covering 5 types of working conditions: normal operation, positioning deviation, insertion tilt, contact force exceeding limit, and component damage. It is used for model parameter initialization.

[0104] Step 2: Calculation of the risk transmission field of physical illusion (Data E) Sub-step 2.1: Perception-Physical Consistency Assessment Calculation process: 1. Based on the dynamic parameters and spatial constraints of data C, construct the physically constrained Riemannian manifold for the insertion operation. The metric tensor is generated by jointly mapping the joint space inertia matrix and the task space geometric constraints.

[0105] 2. Project the current visual, force, and joint 3D observations onto the manifold. Within the feasible region, the Riemann geodesic distance between the observation point and the projection point is calculated: the geodesic distance for visual observation is 0.018 mm, the geodesic distance for force observation is 0.22 N, and the geodesic distance for articular observation is 0.003°.

[0106] 3. Calculate the three-dimensional consistency score: , , .

[0107] Set a consistency pass threshold If all three-dimensional scores are above the threshold, it is marked as mild perceptual bias, and the consistency bias coefficient is output. .

[0108] Sub-step 2.2: Construction of a Pair-Copula Coupled Hidden Markov Risk Transmission Model Calculation process: 1. Define 5 levels of hidden risk status: safe state Mild perceptual bias Moderate movement inaccuracy, Severe physical violations Component / PCB damage (absorption state).

[0109] 2. Based on 12,600 labeled samples from data D, the initial state transition probability matrix is ​​obtained through maximum likelihood estimation: ; 3. Extract the risk factors for each action in data B. (After normalization:) , , , , The transition probability is adjusted according to the action excitation formula; the inserted action is used. For example, after correction → The transition probability increased by 0.08. → The transfer probability is increased by 0.05, while the low-risk transfer probability is simultaneously reduced, keeping the row sum at 1.

[0110] 4. Based on data D, the conditional marginal distributions of visual, force, and joint 3D observations are estimated, and the correlation coefficient matrix is ​​obtained by fitting using Gaussian Copula: ; Combination By increasing the weights of the visual and force dimensions, which have significant deviations, the corrected emission probability is obtained. .

[0111] Sub-step 2.3: Multi-step forward-looking risk evolution simulation (generating data E) Input data includes: PC-HMM model parameters; The 5-step action sequence of data B; Initial state distribution .

[0112] Calculation process: The forward algorithm is used to recursively calculate the state probability distribution at each step, and the first arrival probability of the absorption state and the risk geodesic entropy are calculated simultaneously to finally generate a 5-step physical illusion risk transmission field (data E).

[0113] In one specific embodiment, the content of the physical illusion risk transmission field includes: Capture lifting motion The probability is 0.892. The probability is 0.087. The probability is 0.021. The probability is 0.000. The probability of first arrival is 0.000, and the risk geodesic entropy is 0.124; Horizontal transfer action The probability is 0.825. The probability is 0.131. The probability is 0.042. The probability is 0.002. The probability of first arrival is 0.000, and the risk geodesic entropy is 0.217; Deceleration and alignment action The probability is 0.768. The probability is 0.164. The probability is 0.063. The probability is 0.005. The probability of first arrival is 0.001, and the risk geodesic entropy is 0.302; Vertical approach action The probability is 0.681. The probability is 0.205. The probability is 0.097. The probability is 0.015. The probability of first arrival is 0.002, and the risk geodesic entropy is 0.415; Inserting pad action The probability is 0.542. The probability is 0.243. The probability is 0.156. The probability is 0.047. The first-to-arrive probability is 0.012, and the risk geodesic entropy is 0.598.

[0114] Simultaneously calculate the expected risk arrival time of the absorbed state: Each step, meaning it continues to execute along the current original trajectory, triggers irreversible damage on average after about 12.7 steps.

[0115] Step 3: Generate a tiered pre-intervention strategy based on data E Calculation and execution process: Set three risk thresholds: low risk threshold medium risk threshold High risk threshold .

[0116] 1. Risk level matching: Step 5 inserts the action. The probability of first-time arrival is 0.012, exceeding the low-risk threshold. This triggers a primary pre-intervention (micro-correction).

[0117] 2. Optimal intervention solution: Taking the minimum action correction amount as the objective function and the risk reduction to below the threshold as the constraint, the micro-correction strategy is obtained: add a 0.3s low-speed probing phase before the insertion action, reduce the insertion speed from 2mm / s to 1.2mm / s, tighten the force control margin by 20%, and compensate for the end attitude with 0.01mm lateral offset.

[0118] 3. Re-verification after correction: Re-input the corrected action sequence into the model for deduction, step 5. The probability of first-time arrival dropped to 0.003, below the low-risk threshold, and the risk was eliminated.

[0119] Implementation results: This embodiment operated continuously on the production line for 30 days, completing a total of 180,000 insertion operations. Compared with traditional real-time detection solutions: The component failure rate decreased from 0.12% to 0.015%, a reduction of 87.5%. The average warning lead time has been increased from the traditional 0.08s (after a collision and the vehicle stops) to 1.32s, enabling intervention at the early stage of risk. The unexpected mission interruption rate decreased from 2.1% to 0.58%, a reduction of 72.4%. The average execution time per task increased from 1.8s to 1.87s, with an efficiency loss of only 3.9%, far lower than the 18% efficiency loss of traditional solutions; The system self-optimizes through a Bayesian online update mechanism, and after 10,000 runs, the risk prediction accuracy has increased from the initial 79% to 94.2%.

[0120] This application addresses the core shortcomings of existing embodied intelligent physical illusion protection technologies, such as detection time lag, isolated risk assessment, coarse control strategies, perception error propagation, and weak scene adaptability. Through core technologies including Pair-Copula coupled Hidden Markov Modeling, multi-step look-ahead risk extrapolation, perception-physical pre-calibration, hierarchical proactive pre-intervention, and Bayesian online adaptive methods, it achieves significant technological advancements and performance improvements across multiple dimensions, including prediction lead time, risk identification accuracy, proactive prevention capabilities, scene robustness, and engineering feasibility. Specific beneficial effects are as follows: 1. This application achieves proactive risk prediction, upgrading the paradigm from passive detection during execution to proactive prediction before execution. It abandons the traditional reactive approach of relying on sensor feedback to trigger abnormal shutdowns. Instead, it uses a Pair-Copula coupled Hidden Markov Model to extrapolate the risk evolution trajectory across multiple steps based on the sequence of actions to be executed. It quantifies the probability of the first arrival of the absorption state and the expected risk arrival time, enabling identification and early warning even before the action is executed and the risk is still in its nascent stage. In typical scenarios such as precision industrial assembly, this increases the average early warning lead time, reduces the rate of physical collisions and material damage, and achieves proactive risk prevention rather than trigger-based remediation. This overcomes the core limitations of existing technologies where detection lags behind action and damage has already occurred by the time risks are detected.

[0121] 2. A multi-risk source coupling and transmission model is constructed to significantly improve the accuracy and recall of identifying chain-like physical illusions. On the one hand, breaking through the traditional single-step independent verification framework, this method models physical illusion risk for the first time as a state transition process of "perceptual deviation → inaccurate movement → physical violation → collision damage." A Pair-Copula function is used to characterize the nonlinear coupling effect of multi-dimensional risk sources (visual, force, and joint), accurately quantifying the transmission, accumulation, and amplification patterns of risk in the action sequence. On the other hand, risk geodesic entropy is introduced to simultaneously measure the severity and evolutionary uncertainty of the risk, effectively distinguishing between normal operational fluctuations and the deterioration trend of illusion risks. Benchmark tests show that, compared to traditional single-step threshold detection schemes, this application improves the accuracy of chain-like risk prediction and the recall rate of severe physical violation events.

[0122] 3. Establish a tiered proactive pre-intervention mechanism to achieve the optimal balance between safety protection and operational efficiency. This application abandons the traditional, crude safety control logic of "stop and go" and matches a three-tiered differentiated pre-intervention strategy based on risk prediction results: in the low-risk stage, gradient micro-correction is adopted to mitigate potential hazards through attitude and speed fine-tuning without interrupting the task; in the medium-risk stage, local trajectory replanning is triggered to avoid risk transmission paths at the lowest efficiency cost; in the high-risk stage, proactive safety docking is executed to complete braking protection before the risk becomes irreversible. Compared with traditional emergency shutdown schemes, this application reduces the accidental interruption rate of the task and the overall loss of operational efficiency, improves the task completion efficiency under the same safety margin, and solves the technical contradiction of the incompatibility between safety protection and operational efficiency in existing technologies.

[0123] 4. Pre-sensing and physical consistency calibration: preventing the transmission of perceptual illusions to execution risks at the source. This application places physical consistency verification as a necessary input step in risk calculation. Based on physical constraints, Riemannian manifolds, and geodesic distance metrics, it quantifies the deviation between perceptual observations and physical prior rules, and corrects the observation emission probability of the risk model in real time, avoiding the transmission of perceptual biases into risk misjudgments. In complex perceptual scenarios such as illumination interference, occlusion, and low texture, it reduces the false alarm rate and significantly improves the robustness of risk prediction in unstructured environments, breaking through the technical bottlenecks of traditional detection schemes that are susceptible to perceptual noise interference and have a high false trigger rate.

[0124] 5. A closed-loop adaptive evolution mechanism is established during runtime, significantly reducing engineering implementation and scenario adaptation costs. This application employs a Bayesian online update mechanism, continuously iteratively optimizing the transition probability matrix based on actual sensor feedback data. It can adapt to different equipment parameters, operating conditions, and task scenarios without offline retraining. After 1000 system runs, the risk prediction accuracy is improved, and the single-step computation time of the core inference module is reduced. It can be directly embedded into existing embodied intelligent control architectures without modifying the original VLA model and hardware configuration, shortening the engineering implementation cycle and demonstrating strong generalization ability and deployment cost-effectiveness.

[0125] This application also provides an embodied intelligence system, including an electronic device. The electronic device includes a processor and a memory. The memory stores a program or instructions that can run on the processor. When the program or instructions are executed by the processor, they implement the various steps of the above-described embodied intelligence physical illusion prediction and pre-intervention method embodiments and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0126] This application also provides a computer-readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiments of the embodied intelligent physical illusion prediction and pre-intervention method, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0127] It should be clarified that this application is not limited to the specific configurations and processes described above. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of this application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0128] The functional blocks described above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0129] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0130] It should be understood that the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in this application, and such modifications or substitutions should be covered within the scope of protection of this application.

Claims

1. A method for predicting and intervening in embodied intelligence physical illusions, applied to embodied intelligence systems, characterized in that, The method for predicting and intervening in embodied intelligent physical illusions includes: Acquire the multimodal perception observation data, action sequence data to be executed, physical prior rule base data, and historical risk transmission sample base data of the embodied intelligence system; Based on a hidden Markov risk transmission model with nonlinear coupling of multiple risk sources, a physical illusion risk transmission field is calculated using the multimodal sensing observation data, the action sequence data to be executed, the physical prior rule base data, and the historical risk transmission sample base data. The physical illusion risk transmission field includes the state probability distribution, the first arrival probability of the absorption state, the expected risk arrival time, and the risk geodesic entropy for each action step. The state probability distribution represents the probability that the system is in each hidden risk state under each action step. The hidden risk states include safe state, intermediate state, and absorption state. The absorption state is an irreversible risk state. The first arrival probability of the absorption state represents the probability of triggering the absorption state for the first time. The expected risk arrival time represents the remaining expected time for triggering the absorption state for the first time. The risk geodesic entropy represents the uncertainty and severity of risk evolution. A pre-intervention strategy is executed based on the physical illusion risk transmission field and the action sequence data to be executed, in order to reduce the physical illusion risk before reaching the absorption state. The pre-intervention strategy includes action parameter correction, local action sequence replanning, and action sequence rollback.

2. The method for predicting and intervening in embodied intelligent physical illusions according to claim 1, characterized in that, The Hidden Markov Risk Transmission Model based on nonlinear coupling of multiple risk sources calculates the physical hallucination risk transmission field according to the multimodal sensing observation data, the action sequence data to be executed, the physical prior rule base data, and the historical risk transmission sample database data, including: The consistency deviation coefficient is calculated based on the multimodal sensing observation data and the physical prior rule base data. The consistency deviation coefficient characterizes the degree of agreement between the sensing observation data and the physical prior rules. Based on the hidden Markov risk transmission model with nonlinear coupling of multiple risk sources, a multi-level hidden risk state is constructed. The hidden risk state includes a safe state, an intermediate state, and an absorptive state. The absorptive state is an irreversible risk state. The initial state distribution, action-dependent transition probability matrix, and corrected launch probability are calculated based on the consistency deviation coefficient, the action sequence data to be executed, and the historical risk transmission sample library data. The initial state distribution represents the initial probability that the system is in each hidden risk state at the time of task startup. The action-dependent transition probability matrix represents the directed transition prior probability between each pair of hidden risk states. The corrected launch probability represents the probability of observing different observations under each hidden risk state. Based on the initial state distribution, the action-dependent transition probability matrix, and the modified emission probability, a forward recursive calculation is performed to obtain the state probability distribution, the first-arrival probability of the absorption state, the expected risk arrival time, and the risk geodesic entropy for each action step.

3. The method for predicting and intervening in embodied intelligent physical illusions according to claim 2, characterized in that, The consistency deviation coefficient calculated based on the multimodal sensing observation data and the physical prior rule base data includes: Based on the dynamic parameters and spatial constraints of the physical prior rule base data, a physically feasible state manifold for robot operation is constructed, and the geodesic distance on the physically feasible state manifold corresponds to the degree of violation of physical rules. The multimodal sensing observation data is projected onto the physical feasible state manifold to obtain the projection points in the physical feasible domain corresponding to different observation points. The Riemann geodesic distance between the observation points and the projection points is calculated. The observation dimensions of the observation points include visual, force, and joint. Based on the Riemann geodesic distance, consistency scores are calculated for the three dimensions of visual, force, and joint, respectively, and the consistency scores characterize the degree of physical consistency. The consistency deviation coefficient is calculated based on the consistency scores of the three dimensions: visual, force, and joint.

4. The method for predicting and intervening in embodied intelligent physical illusions according to claim 2, characterized in that, The calculation of the initial state distribution, action-dependent transition probability matrix, and corrected emission probability based on the consistency deviation coefficient, the action sequence data to be executed, and the historical risk transmission sample database includes: Obtain the pre-stored initial state distribution under the system initialization state; Based on the historical risk transmission sample database, the initial state transition probability matrix is ​​obtained using the maximum likelihood estimation method; Based on the action characteristics of the action sequence data to be executed, the risk factors of each action are extracted. The risk factors are obtained by weighted normalization of action speed, acceleration and contact force level. Based on the historical risk transmission sample database, the risk escalation transfer increment matrix and the risk descent transfer increment matrix are obtained by statistical difference method. The risk escalation transfer increment matrix represents the probability of amplifying a high-risk action into a worse state, and the risk descent transfer increment matrix represents the probability of improving a low-risk action into a better state. The action-dependent transition probability matrix is ​​calculated based on the initial state transition probability matrix, the risk factor, the risk-increasing transition increment matrix, and the risk-decreasing transition increment matrix. Based on the historical risk transmission sample database, a connection function is used to characterize the nonlinear dependency relationship of the three observation dimensions of vision, force, and joint, and a joint launch probability is constructed. The joint launch probability is corrected using the consistency deviation coefficient to obtain the corrected launch probability.

5. The method for predicting and intervening in embodied intelligent physical illusions according to claim 4, characterized in that, Based on the historical risk transmission sample database, a connection function is used to characterize the nonlinear dependencies of the three observation dimensions of vision, force, and joint, and a joint emission probability is constructed, including: Based on the historical risk transmission sample database, the conditional marginal distribution of the three observation dimensions of vision, force, and joints is statistically analyzed for each hidden risk state, grouped by hidden risk state. A three-dimensional joint distribution is constructed by using a connection function to assess the conditional marginal distributions of the three observation dimensions of vision, force, and joint, and the joint emission probability is obtained.

6. The method for predicting and intervening in embodied intelligent physical illusions according to claim 4, characterized in that, The step of correcting the joint launch probability using the consistency deviation coefficient to obtain the corrected launch probability includes: Construct corrected weights based on the consistency deviation coefficient; The joint launch probability is weighted and corrected based on the corrected weights to obtain the corrected launch probability.

7. The method for predicting and intervening in embodied intelligent physical illusions according to claim 2, characterized in that, The forward recursive calculation based on the initial state distribution, the action-dependent transition probability matrix, and the modified emission probability yields the state probability distribution, the first-arrival probability of the absorption state, the expected risk arrival time, and the risk geodesic entropy for each action step, including: The state probability distribution for each action step is obtained by forward recursion calculation based on the initial state distribution, the action-dependent transition probability matrix, and the corrected emission probability. The action-dependent transition probability matrix is ​​decomposed into a transient submatrix and an absorptive state transition submatrix. The transient submatrix represents the directed transition prior probability of a hidden risk state transitioning to a safe state or an intermediate state, and the absorptive state transition submatrix represents the directed transition prior probability of a hidden risk state transitioning to an absorptive state. The first-reach probability of the absorption state is calculated based on the initial state distribution, the transient submatrix, and the absorption state transition submatrix. The expected risk arrival time is calculated based on the initial state distribution and the transient submatrix. The risk geodesic entropy is calculated based on the state probability distribution.

8. The method for predicting and intervening in embodied intelligent physical illusions according to claim 1, characterized in that, The step of executing a pre-intervention strategy based on the physical hallucination risk transmission field and the sequence of actions to be performed, in order to reduce the risk of physical hallucination before reaching the absorption state, includes: The risk level is determined based on the first-arrival probability of the absorption state. Based on the risk level, a pre-intervention strategy is executed according to the state probability distribution, the first arrival probability of the absorption state, the expected risk arrival time, the risk geodesic entropy, and the action sequence data to be executed, in order to reduce the risk of physical illusion before reaching the absorption state. The pre-intervention strategy includes action parameter correction, local action sequence replanning, and action sequence rollback.

9. The method for predicting and intervening in embodied intelligent physical illusions according to claim 8, characterized in that, The risk levels include low risk, medium risk, and high risk; the pre-intervention strategy, based on the risk levels and according to the state probability distribution, the first-arrival probability of the absorption state, the expected risk arrival time, the risk geodesic entropy, and the action sequence data to be executed, is implemented to reduce the risk of physical hallucination before reaching the absorption state, including: If the risk level is low risk, the intermediate state is used as the correction target. The correction amount is calculated based on the state probability distribution and the risk geodesic entropy. The correction amount is superimposed on the original action parameters of the action sequence data to be executed to obtain the target action parameters. The action is executed according to the target action parameters. If the risk level is medium risk, starting from the current state, a constrained Markov decision process is solved based on the state probability distribution, the first arrival probability of the absorption state, the risk geodesic entropy, and the action sequence data to be executed. A local action sequence that satisfies the preset risk constraints is regenerated, and the action is executed based on the local action sequence. If the risk level is high risk, the target state for reversal is determined according to the state probability distribution. The reversal action sequence is calculated based on the target state for reversal, the expected risk arrival time, the risk geodesic entropy, and the action sequence data to be executed. The action is then executed according to the reversal action sequence.

10. An embodied intelligence system, characterized in that, The method includes an electronic device, which includes a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the embodied intelligent physical illusion prediction and intervention method as described in any one of claims 1 to 9.

Images