Body intelligence illusion hierarchical autonomous recovery and man-machine collaborative takeover method and system

Abstract

The application discloses a method and system for hierarchical autonomous recovery and human-machine collaborative takeover of body intelligent illusion, wherein according to multi-modal perception original data, task reasoning semantic sequence data and physical world prior constraint data of a target body intelligent, semantic confidence deviation features, physical consistency deviation features and action feasibility deviation features are respectively calculated; based on historical sample correlation parameters in a historical decision sample library data, the semantic confidence deviation features, the physical consistency deviation features and the action feasibility deviation features are nonlinearly fused to obtain a comprehensive illusion risk measure and a dominant illusion type label; through a preset hierarchical error correction decision tree, a target execution strategy is generated according to the comprehensive illusion risk measure and the dominant illusion type label; based on the target execution strategy, hierarchical autonomous recovery and human-machine collaborative takeover matching the comprehensive illusion risk measure and the dominant illusion type label are executed to obtain a collaborative execution result, which can reduce artificial intervention and improve human-machine collaborative efficiency.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and robot control technology, and in particular to a method and system for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusion. Background Technology

[0002] Embodied intelligence, by integrating large language model reasoning, multimodal environmental perception, and robot motion control technologies, enables task understanding, decision-making, planning, and action execution in the physical world, and has been widely applied in scenarios such as industrial manufacturing assembly, domestic services, and special operations inspection. However, embodied intelligence systems may experience illusions throughout the perception, reasoning, and action chain. These illusions can directly translate into erroneous operations in the physical world, leading to tasks that fail or, in severe cases, equipment damage or personal safety risks. This is a crucial problem that embodied intelligence must address as it transitions from the digital world to the physical world.

[0003] Currently, the mainstream approach to handling hallucinations in embodied intelligence systems largely adopts a binary model of "detection and alarm - manual intervention." This means that once the system detects a hallucination risk, it immediately suspends the task and switches to manual remote operation. This approach delegates hallucination handling to humans, lacking differentiation based on the type and severity of hallucinations. All types of hallucinations are treated with the same handling logic, resulting in many mild hallucinations that could be corrected autonomously being uniformly handled by humans. This leads to a high frequency of human intervention, severely straining operational resources, and high hallucination handling costs, making it difficult to meet the needs of large-scale deployment of embodied intelligence. Summary of the Invention

[0004] The main purpose of this application is to propose a hierarchical autonomous recovery and human-machine collaborative takeover method and system for embodied intelligent illusion, which can reduce human intervention and improve the efficiency of human-machine collaboration.

[0005] The first aspect of this application provides a method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions, the method comprising: Acquire multimodal perception raw data of the target embodied intelligence, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data; Based on the multimodal perception raw data, the task reasoning semantic sequence data, and the physical world prior constraint data, semantic confidence bias features, physical consistency bias features, and action feasibility bias features are calculated respectively. Based on the historical sample correlation parameters in the historical decision sample database, the semantic confidence bias feature, the physical consistency bias feature and the action feasibility bias feature are nonlinearly fused to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. By using a pre-defined hierarchical error correction decision tree, a target execution strategy is generated based on the comprehensive hallucination risk measure and the dominant hallucination type label. Based on the target execution strategy, a hierarchical autonomous recovery and human-machine collaborative takeover are performed, matching the comprehensive hallucination risk measure and the dominant hallucination type label, to obtain the collaborative execution result.

[0006] The first aspect of this application provides a method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligence illusions. This method calculates semantic confidence bias, physical consistency bias, and action feasibility bias based on the target embodied intelligence's multimodal perception raw data, task reasoning semantic sequence data, and prior physical world constraint data. Using historical sample correlation parameters from a historical decision sample database, the method nonlinearly fuses these features to obtain a comprehensive illusion risk measure and a dominant illusion type label. A target execution strategy is generated based on the comprehensive illusion risk measure and dominant illusion type label using a pre-set hierarchical error-correction decision tree. The hierarchical autonomous recovery and human-machine collaborative takeover, matching the comprehensive illusion risk measure and dominant illusion type label, are then executed according to the target execution strategy to obtain a collaborative execution result. This method reduces human intervention and improves human-machine collaboration efficiency.

[0007] To achieve the above objectives, a second aspect of this application provides an embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover system, the system comprising: The acquisition module is used to acquire the target's embodied intelligence's multimodal perception raw data, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data; The calculation module is used to calculate semantic confidence bias features, physical consistency bias features, and action feasibility bias features based on the multimodal perception raw data, the task reasoning semantic sequence data, and the physical world prior constraint data, respectively. The fusion module is used to perform nonlinear fusion of the semantic confidence bias feature, the physical consistency bias feature and the action feasibility bias feature based on the historical sample correlation parameters in the historical decision sample database, so as to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. The generation module is used to generate a target execution strategy based on the comprehensive hallucination risk measure and the dominant hallucination type label by using a preset hierarchical error correction decision tree; The execution module is used to perform hierarchical autonomous recovery and human-machine collaborative takeover based on the target execution strategy, matching the comprehensive hallucination risk measure and the dominant hallucination type label, to obtain collaborative execution results.

[0008] To achieve the above objectives, a third aspect of this application provides an electronic device, comprising: at least one processor and a memory for communicatively connecting to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method.

[0009] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusion.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating a hierarchical autonomous recovery and human-machine collaborative takeover method for embodied intelligent illusion provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover system provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0013] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0014] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0015] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0016] This application provides a method and system for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions, which will be specifically described through the following embodiments. First, the method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions in this application is described.

[0017] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0018] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0019] The embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method provided in this application relates to the fields of artificial intelligence and robot control technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method, but is not limited to the above forms.

[0020] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0021] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0022] Therefore, referring to Figure 1This application provides a method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusion, which includes the following steps S110 to S150.

[0023] Step S110: Obtain the target embodied intelligence's multimodal perception raw data, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data; Step S120: Based on the multimodal perception raw data, task reasoning semantic sequence data, and physical world prior constraint data, calculate the semantic confidence bias feature, physical consistency bias feature, and action feasibility bias feature, respectively. Step S130: Based on the historical sample correlation parameters in the historical decision sample database, perform nonlinear fusion of semantic confidence bias features, physical consistency bias features and action feasibility bias features to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. Step S140: Generate a target execution strategy based on the comprehensive hallucination risk measure and the dominant hallucination type label by using a preset hierarchical error correction decision tree; Step S150: Based on the target execution strategy, perform hierarchical autonomous recovery and human-machine collaborative takeover that match the comprehensive hallucination risk measurement and the dominant hallucination type label to obtain the collaborative execution result.

[0024] In this step, we first acquire four types of input data for the target's embodied intelligence, providing a unified input data source for the subsequent calculation of dimensional bias features.

[0025] Among them, the target embodied intelligence refers to the embodied intelligence equipment used in this hallucination detection and classification process, including but not limited to industrial six-axis robotic arms, collaborative robots, mobile operation composite robots, domestic service robots, and special operation inspection robots. Multimodal perception raw data refers to the low-level observation data, which is not semantically interpreted, and is synchronously collected by the sensing devices carried by the target embodied intelligent entity during task execution. It is denoted as the set. ,in Red, green, and blue depth point cloud data collected by a binocular camera. Force and torque data collected by the end-effector six-dimensional force sensor. The sampling frequency of the joint pose data and joint velocity data collected by the encoders of each joint is strictly synchronized with the control cycle of the robot to ensure that the subsequent feature calculation and control command issuance are strictly aligned on the time axis. Task reasoning semantic sequence data refers to the mid-level semantic results output by the embodied big model carried by the target embodied intelligence during the process of completing task understanding and action planning, denoted as a set. ,in Plan action chains for tasks. For each action in the action chain of a task, a knowledge graph triple is generated. A sequence of semantic embedding vectors consisting of the semantic embedding vectors corresponding to each of several inference steps, and satisfying the following conditions: , The total number of reasoning steps. The dimension of the semantic embedding vector; Prior constraint data in the physical world refers to prior data, denoted as a set, that is pre-constructed and stored in the system's prior knowledge base to describe the objective physical laws of the operational scenario and the boundaries of equipment capabilities. ,in This is a scene topology map. A library of physical properties for objects. For robot kinematic parameters, These are the threshold values ​​for various safety constraints; Historical decision sample database data refers to the accumulated experience data of the system in previous hallucination treatments, denoted as the set. ,in To pre-determine the historical error correction success rate of each branch of the hierarchical error correction decision tree, To average processing time, For manual operator load data, Label the effectiveness of each strategy.

[0026] Specifically, the four types of data—multimodal perception raw data, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data—differ in data dimensions and update frequencies. The multimodal perception raw data is updated in real-time according to the robot's control cycle; the task reasoning semantic sequence data is updated segment by segment according to the progress of the task planning actions; the physical world prior constraint data remains stable when the operational scenario remains unchanged and is updated as a whole when the scenario changes; and the historical decision sample database data is incrementally updated after each hallucination treatment. By aligning and caching the four types of data with timestamps under the same time reference, distortion in feature calculation caused by data time misalignment can be avoided, providing a consistent sample basis for subsequent fusion calculations.

[0027] Furthermore, based on the original multimodal perception data, task reasoning semantic sequence data, and prior constraint data of the physical world, semantic confidence bias features, physical consistency bias features, and action feasibility bias features are calculated respectively. Semantic confidence bias features are quantitative indicators used to characterize the degree of deviation between the output of the reasoning layer and the objective facts represented by the knowledge graph, corresponding to reasoning-related hallucinations. Physical consistency bias features are quantitative indicators used to characterize the degree of alignment between the output of the perception layer and objective physical rules, corresponding to perception-related hallucinations. Action feasibility bias features are quantitative indicators used to characterize the executability of the action layer planning results in terms of kinematic singularity and collision safety, corresponding to action-related hallucinations. The values ​​of the three types of bias features are all normalized to the range of 0 to 1; the larger the value, the higher the degree of deviation of the corresponding link and the greater the probability of hallucination. By decomposing the hallucination representation of the entire link into three independent but jointly observable dimensions, the sources of hallucination at the reasoning layer, perception layer, and action layer can be covered simultaneously, avoiding false negatives and false positives caused by single-dimensional detection.

[0028] Furthermore, based on the historical sample correlation parameters in the historical decision sample database, semantic confidence bias features, physical consistency bias features, and action feasibility bias features are nonlinearly fused to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. The historical sample correlation parameters are parameters obtained from the historical decision sample database that characterize the nonlinear correlation structure between the bias features of each dimension, reflecting historical statistical patterns such as perceptual bias synchronously triggering action bias and inference bias propagating along the action chain. The comprehensive hallucination risk measure is a scalar value ranging from 0 to 1, used to uniformly characterize the comprehensive severity of the target's embodied intelligence experiencing hallucinations at the current moment. The dominant hallucination type label refers to the hallucination category corresponding to the dimension with the largest value among the three dimensions, including inference, perception, and action types. Because nonlinear fusion is used instead of simple weighted summation, it can characterize the synergistic deviation effect between dimensions; that is, when moderate deviations occur simultaneously in multiple dimensions, the output risk measure is significantly higher than that for single-dimensional deviations, thereby improving the sensitivity of comprehensive judgment.

[0029] Furthermore, by using a pre-defined hierarchical error-correction decision tree, a target execution strategy is generated based on the comprehensive hallucination risk measure and the dominant hallucination type label. The pre-defined hierarchical error-correction decision tree refers to a multi-layered tree-like strategy index structure that is pre-built and updated online with historical samples. Its first layer uses severity level as the splitting condition, and the second layer uses hallucination type as the splitting condition under the branch pointed to by the first layer. Leaf nodes store the set of treatment strategies corresponding to their respective branches. The target execution strategy refers to the set of treatment strategies output by the matched leaf nodes, which can be directly issued to the execution unit. It includes strategy type, execution parameters, expected success rate, and expected time consumption. Because the hierarchical structure of the decision tree corresponds one-to-one with the comprehensive hallucination risk measure and the dominant hallucination type label, the strategy routing process is entirely driven by the measure values, completing the end-to-end mapping from hallucination quantification to treatment strategy without manual intervention.

[0030] Furthermore, based on the target execution strategy, a tiered autonomous recovery and human-machine collaborative takeover process, matching the comprehensive hallucination risk measure and the dominant hallucination type label, is executed to obtain the collaborative execution result. Tiered autonomous recovery refers to the process of progressively calling error correction algorithms from weak to strong according to severity level, and recalculating the comprehensive hallucination risk measure after each execution to verify the recovery effect and determine whether to escalate the treatment level. Human-machine collaborative takeover refers to the process of gradually transferring some or all operational permissions from the system to a human operator according to a tiered permission transition method. The collaborative execution result refers to the final output of this treatment process, which includes either the recovery execution result or the human-machine collaborative takeover result. Because the recovery strategy and the takeover strategy share the same severity level classification criteria, the system can achieve a balance between autonomous capabilities and human intervention, transferring only samples requiring human judgment to the operator.

[0031] For example, taking the bolt tightening operation of a six-axis collaborative robotic arm at the automotive body-in-white chassis assembly station as an example, the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method provided in this application embodiment will be described in general. The execution subject of this station is a six-axis collaborative robotic arm, whose end effector is equipped with a binocular vision camera and a torque tightening actuator, which undertakes the fully automated operation of bolt gripping, hole positioning and torque tightening, with a robot control cycle of 10 milliseconds.

[0032] Within a certain work cycle, the system acquires the following four types of data through step S110: data That is, the raw data of multimodal sensing This includes red, green, and blue depth point clouds of bolt workstations captured by a wrist-worn binocular camera. (Resolution: 1024 x 768, containing approximately 120,000 3D points), real-time data from the six-dimensional force sensor of the end-tightening gun. The six-axis joint encoder outputs six joint angles and angular velocities. ; data That is, semantic sequence data for task reasoning This includes a 3-step task action chain output by the embodied large model. (In sequence, grab the M10 flange bolt, move it to the front left mounting hole of the chassis, and tighten it to 45 Nm), corresponding knowledge graph triplet. (The bolt is M10, the torque is 45 Nm, and the location is a hole in the left front chassis), and the 768-dimensional semantic embedding sequence corresponding to each step of reasoning. ; data That is, prior constraint data of the physical world This includes pre-calibrated 3D topology maps of workstations. Standard bolt point cloud template and object physical attribute library (Including the geometry, mass, and thread friction coefficient of the M10 bolt), and the De Navet-Hartenberg kinematic parameters of the robot. and safety constraint thresholds (Tightening torque limit 50 Nm, collision safety distance 5 mm); data That is, historical decision sample database data This includes the historical error correction success rate of each branch of the pre-defined hierarchical error correction decision tree. Average processing time On-duty operator load rate and the effectiveness labels of each strategy .

[0033] Furthermore, the above data are processed in step S120 to calculate the semantic confidence bias features. The physical consistency deviation characteristic is 0.28. The deviation characteristic of action feasibility is 0.39. The value is 0.40; the three bias characteristics are nonlinearly fused in step S130 to obtain the comprehensive hallucination risk measure. The value is 0.42, and the category corresponding to the physical consistency deviation feature with the largest value among the three dimensions is taken as the dominant hallucination type label, that is, the dominant hallucination type is determined to be the perception type; in step S140, the comprehensive hallucination risk measure of 0.42 is compared with the severity level classification threshold. , Comparison, by The hallucination is determined to be of moderate severity. The second layer then matches the leaf node for moderate perception, outputting a target execution strategy that includes 3D point cloud template matching and local viewpoint resampling correction, while simultaneously pushing hallucination status monitoring information to the human-machine collaborative control panel. This strategy has an expected success rate of 82% and an estimated execution time of 0.8 seconds. In step S150, this target execution strategy is executed. The robotic arm adjusts the wrist camera to a 30-degree lateral viewpoint to re-acquire the local point cloud and performs iterative nearest-point registration. After correcting the bolt positioning coordinates, the comprehensive hallucination risk measure is recalculated, resulting in an updated measure. The value was below the mild threshold of 0.35, indicating successful autonomous recovery. The task then proceeded to the tightening process. The actual processing time was 0.72 seconds, and no manual intervention was triggered throughout the process.

[0034] Specifically, if the hallucinations within the aforementioned work cycle were handled using the traditional dual-mode of detection alarm and manual intervention, the system would immediately shut down upon risk triggering and await remote operator intervention. Based on the historical average manual intervention time of 3.37 seconds for this workstation, the current work cycle would be forced to extend by approximately 2.65 seconds. However, using the method provided in this application's embodiment, the system autonomously corrected the perceptual hallucination within 0.72 seconds, without interrupting the work cycle. Therefore, this application's embodiment, by classifying hallucinations in two dimensions according to severity and type and matching them with differentiated autonomous recovery strategies, can significantly reduce the frequency and duration of manual intervention while ensuring operational safety.

[0035] In this embodiment, the target embodied intelligence's multimodal perception raw data, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data are acquired. Based on the multimodal perception raw data, task reasoning semantic sequence data, and physical world prior constraint data, semantic confidence bias features, physical consistency bias features, and action feasibility bias features are calculated respectively. Based on the historical sample correlation parameters in the historical decision sample database data, the semantic confidence bias features, physical consistency bias features, and action feasibility bias features are nonlinearly fused to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. Through a pre-set hierarchical error correction decision tree, a target execution strategy is generated based on the comprehensive hallucination risk measure and the dominant hallucination type label. Based on the target execution... The system implements a strategy that matches the comprehensive hallucination risk measurement and the dominant hallucination type label with hierarchical autonomous recovery and human-machine collaborative takeover, resulting in a collaborative execution outcome. By decomposing the entire hallucination chain into three separately quantifiable dimensions—reasoning, perception, and action—and nonlinearly fusing them to obtain a unified risk measurement, a hierarchical error correction decision tree directly maps this risk measurement to differentiated hierarchical handling strategies. This constructs an end-to-end linkage link from hallucination detection to hierarchical handling, enabling the system to autonomously correct and handle most mild and moderate hallucinations, with only severe hallucinations and recovery failure samples triggering manual takeover. This significantly reduces the frequency of manual intervention and maintenance costs, alleviating the problem of severely strained maintenance resources caused by the disconnect between detection and handling and the one-size-fits-all error correction strategy in existing technologies.

[0036] In some embodiments, an embodiment model is constructed using the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method. The model consists of six core modules, which are not parallel and isolated from each other, but form a strict temporal sequence and feedback loop. Specifically, these modules are: a multimodal perception unit, an embodied task reasoning big model, a motion control unit, a hierarchical error correction decision tree engine, a human-machine collaborative operation platform, and an online optimization unit.

[0037] Among them, the multimodal perception unit is responsible for the synchronous acquisition and preprocessing of raw observation data; the embodied task reasoning big model is responsible for task understanding and action chain planning and outputs semantic embedding sequences; the motion control unit is responsible for trajectory interpolation and safety braking; the hierarchical error correction decision tree engine is responsible for multidimensional deviation feature calculation, nonlinear fusion and policy routing; the human-machine collaborative operation platform is responsible for task status presentation and human permission access; and the online optimization unit is responsible for handling sample refeeding and Bayesian update of decision tree parameters. The six modules together constitute a complete progressive data flow of input data, feature calculation, fusion measurement, policy routing, execution feedback and closed-loop optimization.

[0038] Specifically, the output of the multimodal perception unit flows to the feature calculation submodule of the hierarchical error correction decision tree engine and the pose feedback channel of the motion control unit, respectively. The output of the embodied task reasoning big model is sent to the motion control unit to generate the planned trajectory, and simultaneously fed into the hierarchical error correction decision tree engine to calculate the semantic confidence bias feature. After generating the target execution strategy, the hierarchical error correction decision tree engine sends the strategy to the motion control unit for execution, and pushes a takeover request to the human-machine collaborative operation platform when human intervention is required. After execution, the hierarchical error correction decision tree engine feeds back the sample data of this handling, the handling result label, the actual time consumption and the human intervention time to the online optimization unit, which updates the historical decision sample library and decision tree parameters, thereby realizing the continuous evolution of the system's self-healing capability.

[0039] In some embodiments, the calculation of semantic confidence bias features in step S120 may include, but is not limited to, steps S210 to S260: Step S210: Map the semantic embedding vector sequence to the Riemann semantic manifold space to obtain the semantic embedding vector in the manifold space; Step S220: Calculate the geodesic distance between the semantic embedding of each of the multiple inference steps in the manifold space and the standard entity embedding of the knowledge graph in the prior constraint data of the physical world; Step S230: Construct a probability distribution based on the geodesic distances corresponding to each of the multiple inference steps; Step S240: Calculate the Riemann entropy of the entire inference chain corresponding to the semantic embedding vector sequence based on the probability distribution; Step S250: Compare the Riemann entropy with the preset normal inference baseline value and maximum entropy threshold in the prior constraint data of the physical world to obtain the comparison result; Step S260: Normalize the comparison results to obtain semantic confidence bias features.

[0040] In this step, the semantic embedding vector sequence in the task reasoning semantic sequence data is first mapped to the Riemann semantic manifold space to obtain the semantic embedding vector in the manifold space.

[0041] Here, the Riemannian semantic manifold space refers to the curved geometric space pre-trained from a large number of normal inference samples, used to carry semantic embedding vectors, denoted as... It uses Riemannian metric tensors to describe the local distance relationship between two adjacent points in space; semantic embedding vector sequences refer to the sequences in data B. Its output is derived from the embodied large model at each inference step. The semantic embedding vectors are arranged in the order of reasoning, satisfying ,in The total number of reasoning steps. is the dimension of the semantic embedding vector; the semantic embedding vector in the manifold space refers to the vector representation obtained by projecting the semantic embedding vector originally located in Euclidean space onto the Riemann semantic manifold space through exponential mapping or geodesic projection.

[0042] Since the semantic embeddings output by large language models are objectively distributed on a curved manifold with a hierarchical structure, directly measuring the deviation between inference steps by straight-line distance in Euclidean space would introduce distance distortions that are irrelevant to semantics. Therefore, this application first maps the semantic embedding vector sequence to the Riemannian semantic manifold space so that the subsequent distance measurement is consistent with the intrinsic structure of the inference semantics.

[0043] Specifically, the mapping process can be implemented using pre-trained manifold parameters and a mapping network. In the offline phase, the Riemannian metric tensor is estimated using a set of historical normal inference samples. In the online phase, the semantic embedding vector sequence acquired in real-time is input into the mapping network, which outputs the corresponding coordinates of the sequence in the manifold space. In some embodiments, the mapping network can be configured as a structure of a multilayer perceptron and an exponential mapping layer cascaded together, with its output dimension consistent with the dimension of the input semantic embedding vector, thereby completing the spatial transformation without changing the vector dimension.

[0044] Furthermore, the geodesic distance between the semantic embedding of each reasoning step in the manifold space and the standard entity embedding of the knowledge graph in the prior constraint data of the physical world is calculated. Here, the standard entity embedding of the knowledge graph refers to the embedding of the knowledge graph in the prior constraint data of the physical world. of The identified standard entity, in the same Riemannian semantic manifold space as the semantic embedding vector sequence, corresponds to an embedding vector that represents the objective semantic position that the reasoning step should correspond to at the factual level; geodesic distance refers to the length of the shortest smooth curve connecting two points in a given manifold space, which is used to measure the true degree of deviation between two semantic embedding vectors in curved space, as opposed to the straight-line distance in Euclidean space.

[0045] Specifically, no. The semantic embedding of each reasoning step in the manifold space is denoted as . The knowledge graph standard entity embedding corresponding to this reasoning step is denoted as . The geodetic distance between the two is calculated using the following formula: ; in, Connecting in manifold space and The set of all smooth curves, For any one of them, smooth curves and satisfying , , For this curve in parameters The tangent vector at that point, Let Riemannian metric tensor be the point on the curve where the manifold is located. For Riemannian metric tensors on manifolds. Since geodesic distance measures the degree of deviation along the shortest path of the manifold, it can avoid the overestimation or underestimation of distance components due to spatial curvature in Euclidean space, thus providing a more realistic characterization of the semantic deviation between the reasoning step and the standard entity.

[0046] Furthermore, a probability distribution is constructed based on the geodesic distances corresponding to each of the multiple inference steps. Here, the probability distribution refers to a discrete distribution whose probability quality is represented by the proportion of each inference step's geodesic distance to the total geodesic distance of the entire inference chain. It characterizes the contribution of each inference step to the total deviation of the entire inference chain and is calculated according to the following formula: ; in, For the first The probability mass corresponding to each inference step Let be the total number of reasoning steps, and satisfy . It should be noted that in Riemannian manifold spaces, the absolute value of geodesic distance varies with the manifold curvature and embedding dimension, and its absolute magnitude is not directly equivalent to the severity of inference deviation. This application constructs a probability distribution based on the relative proportion of geodesic distances of each inference step, and uses the shape of this probability distribution to characterize the dispersion of deviations along the entire inference chain. When only a few inference steps contribute the vast majority of geodesic distances in the entire inference chain, the probability distribution exhibits a concentrated shape, indicating that the deviation is a local semantic drift that can be corrected by local replanning. When the contributions of each inference step to the total geodesic distance are nearly equal, the probability distribution exhibits a uniform shape, indicating that the inference chain as a whole deviates from the standard entity represented by the knowledge graph, which is a systematic inference disorder.

[0047] Furthermore, the Riemann entropy of the entire inference chain corresponding to the semantic embedding vector sequence is calculated based on the probability distribution. Here, Riemann entropy refers to the information entropy calculated in the Riemann semantic manifold space based on the aforementioned probability distribution, used to characterize the semantic confusion of the entire inference chain as a single scalar, and is calculated according to the following formula: ; in, The Riemann entropy of the entire reasoning chain, Let be the total number of reasoning steps. A smaller Riemann entropy indicates that the deviation is more concentrated in individual reasoning steps, while the overall reasoning chain still conforms to the standard entity; a larger Riemann entropy indicates that the deviation is more evenly distributed across all reasoning steps, and the overall semantic confusion of the reasoning chain is higher. Therefore, this application uses geodesic entropy on the Riemann manifold instead of the traditional semantic entropy based on Euclidean distance, which can accurately characterize the inherent confusion of semantic reasoning and avoid misjudgments caused by inappropriate spatial measurement methods.

[0048] Furthermore, the Riemann entropy is compared with the preset normal inference baseline value and maximum entropy threshold in the prior constraint data of the physical world to obtain the comparison results. Here, the normal inference baseline value refers to the benchmark value obtained from the statistical analysis of the Riemann entropy of historical normal inference samples, denoted as […]. It characterizes the degree of dispersion of the reasoning chain deviation under normal reasoning conditions; the maximum entropy threshold refers to the pre-set entropy value used to limit the upper bound of the normalization interval, denoted as . It is determined by the Riemann entropy distribution of severe hallucination samples in the historical sample; the comparison results refer to Compared to and The relative positional relationship is expressed in the form of normalized deviation.

[0049] Furthermore, the comparison results are normalized to obtain semantic confidence bias features. Normalization refers to the process of compressing the comparison results to the 0-1 interval using a monotonic mapping function; this application uses the Sigmoid function for this purpose. The semantic confidence bias features... The larger the value, the greater the deviation of the reasoning from the facts, according to the following formula: ; in, This is the baseline entropy value for normal inference. The maximum entropy threshold, For the Sigmoid function. Since normalization is based on the baseline entropy value of normal inference and the maximum entropy threshold is the upper bound, the semantic confidence bias features under different task types and different inference steps have a unified dimension and comparability, and the value is constrained to the range of 0 to 1, which can be directly input into the subsequent nonlinear fusion process.

[0050] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the task planning action chain output by the large model is embodied. It contains three inference steps: grabbing the M10 flange bolt, moving it to the front left mounting hole of the chassis, and tightening it to 45 Nm. The semantic embedding dimension corresponding to each inference step is 768, and the semantic embedding sequence satisfies... After mapping the semantic embedding vectors of the three inference steps to the Riemannian semantic manifold space, geodesic distances are calculated between these vectors and the corresponding standard entity embeddings in the knowledge graph, yielding the results. , , .

[0051] Furthermore, based on the aforementioned geodesic distance distribution, the Riemann entropy of the entire inference chain is calculated, yielding... Take the baseline entropy of normal inference. Maximum entropy threshold Substituting into the normalization formula, we get The semantic confidence bias feature of this task's beat is 0.28, which corresponds to a low risk of reasoning-related hallucinations.

[0052] Specifically, in this inference chain, the geodesic distances between the three inference steps and the standard entities embedded in the knowledge graph are all relatively small, ranging from 0.08 to 0.15. The entire inference chain maintains a high degree of consistency with the objective facts represented by the knowledge graph, and the probability distribution does not show extreme concentration towards individual inference steps. Therefore, the semantic confidence bias feature value is low. If the geodesic distances of each inference step generally increase and the distribution tends to be more balanced in a certain work cycle, the uniformity of the probability distribution increases, the Riemann entropy increases accordingly, and the semantic confidence bias feature obtained after normalization also increases accordingly. This indicates that the inference chain as a whole deviates from the standard entities of the knowledge graph, and the risk of inference-related illusions increases, which needs to be addressed in the subsequent fusion and routing stages.

[0053] This embodiment first maps the semantic embedding vector sequence to the Riemann semantic manifold space, then constructs a probability distribution using geodesic distance and calculates the Riemann entropy of the entire inference chain, and finally performs Sigmoid normalization using the normal inference baseline entropy and the maximum entropy threshold. This allows the degree of illusion in the inference layer to be represented by a scalar quantity with uniform dimensions. Since geodesic distance measures the degree of deviation along the shortest path of the manifold, it avoids the distance distortion of Euclidean distance in curved semantic space. Since the geodesic entropy form on the Riemann manifold is used to characterize the dispersion of deviation, it can distinguish between local drift of individual inference steps and systematic disorder of the entire inference chain. Since the normalization is based on the baseline entropy and the maximum entropy threshold obtained from historical sample statistics, the deviation features under different tasks and different numbers of inference steps are comparable, thus directly supporting subsequent multi-dimensional nonlinear fusion and improving the accuracy and recall of inference-type illusion recognition.

[0054] In some embodiments, the calculation of physical consistency deviation characteristics in step S120 may include, but is not limited to, steps S310 to S370: Step S310: Perform point cloud registration between the real-time point cloud and the prior standard point cloud to obtain the average geodesic distance deviation between the real-time point cloud and the prior standard point cloud. Step S320: Determine the constraint set based on the object physical attribute library and the scene topology map; Step S330: Construct a constraint satisfaction problem based on the constraint set; Step S340: Based on the constraint satisfaction problem, verify the perception results corresponding to the real-time point cloud to obtain the number of constraint violations; Step S350: Calculate the violation ratio based on the number of violated constraints and the constraint set; Step S360: Based on the registration deviation allowable threshold, normalize the average geodetic distance deviation to obtain the registration deviation components; Step S370: Fuse the registration deviation component and the violation ratio to obtain the physical consistency deviation characteristics.

[0055] In this step, the real-time point cloud in the original multimodal perception data is first registered with the prior standard point cloud in the prior constraint data of the physical world to obtain the average geodesic distance deviation between the real-time point cloud and the prior standard point cloud. Here, the real-time point cloud refers to the 3D point cloud acquired by the binocular camera mounted on the target embodied intelligence within the current control cycle and obtained after preprocessing; denoted as data. In Prior standard point cloud refers to a standard 3D point cloud template corresponding to the target object, which is pre-collected and stored in the object's physical attribute library, denoted as... Point cloud registration refers to the process of aligning real-time point clouds with prior standard point clouds to the same coordinate system through rigid body transformation, which is implemented using the iterative nearest point algorithm. The average geodesic distance deviation refers to the average distance from each point in the real-time point cloud to the nearest point in the prior standard point cloud after registration, calculated according to the following formula: ; in, For point cloud points, For the first in real-time point cloud One point, For prior standard point clouds Candidate points in the model. The average geodesic distance deviation characterizes the overall deviation between the perceived result and the prior standard model from the perspective of spatial positioning alignment, and is the first type of basis for identifying perceptual illusions of spatial positioning deviation.

[0056] Furthermore, based on the object physical attribute database and the scene topology map, a constraint set is determined. The object physical attribute database refers to the data... In It is a pre-built database used to record the mass, friction coefficient, geometric dimensions, and material properties of each object in the work scene; the scene topology map refers to the data. In It is a pre-constructed and calibrated 3D map used to describe the spatial positional relationships, support relationships, and passable areas between objects in the work scene; the constraint set refers to a set of physical rule constraints that the perception results must satisfy, which are derived from the object physical attribute library and the scene topology map. These constraints include, but are not limited to, gravity constraints, collision constraints, and support relationship constraints.

[0057] Among them, gravity constraints are used to limit the object from being suspended when there is no support, collision constraints are used to limit the space occupied by any two rigid bodies from overlapping each other, and support relationship constraints are used to limit the upper object from being provided with an effective support surface by the lower object.

[0058] Furthermore, a constraint satisfaction problem is constructed based on the constraint set. The constraint satisfaction problem refers to a mathematical problem that uses the poses and dimensions of each object in the perception results as variables to be solved, and the physical rules in the constraint set as constraints that must be satisfied simultaneously among the variables. Its objective is to determine whether there exists a set of variable assignments that simultaneously satisfies all constraints. By explicitly modeling the physical rules as a constraint satisfaction problem, the perceptual rationality judgment, which originally relied on human experience, can be transformed into a formal verification process that can be automatically solved.

[0059] Furthermore, based on the constraint satisfaction problem, the perception results corresponding to the real-time point cloud are verified to obtain the number of constraint violations. Here, the perception results refer to the category, pose, and size of each object obtained after instance segmentation and pose estimation of the real-time point cloud; verification refers to substituting the perception results into each constraint in the constraint satisfaction problem for a truth value determination; the number of constraint violations refers to the number of constraints that are not satisfied, denoted as . .

[0060] Furthermore, based on the number of violated constraints and the constraint set, the violation ratio is calculated. The violation ratio refers to the proportion of violated constraints to the total number of constraints in the constraint set, calculated using the following formula: ; in, For the number of constraints violated, The total number of constraints. Violation of proportion, from the perspective of physical rule constraints, characterizes the degree to which the perceived result violates objective physical common sense, and is the second type of basis for identifying perceptual hallucinations that violate physical common sense.

[0061] Furthermore, based on the registration deviation tolerance threshold, the average geodetic distance deviation is normalized to obtain the registration deviation component. Here, the registration deviation tolerance threshold refers to a pre-set distance threshold used to define the acceptable upper limit of point cloud registration deviation, denoted as […]. The registration deviation component is determined by the accuracy requirements of the operation process and the camera calibration error. The registration deviation component is the normalized ratio of the average geodetic distance deviation to the allowable registration deviation threshold, which reflects the proportion of the current registration deviation in the allowable threshold.

[0062] Furthermore, the registration deviation component and the violation ratio are fused to obtain the physical consistency deviation characteristics. Fusion refers to the process of weighting and summing the registration deviation component and the violation ratio according to preset weights, calculated using the following formula: ; in, This is a characteristic of physical consistency deviation. For the fusion weights of the registration deviation components, The fusion weights violate the proportions; their sum is 1 and their values ​​are obtained from training with historical samples. A larger value indicates a worse alignment between the perceived result and the physical world, and a greater risk of perceptual hallucinations. Therefore, the physical consistency deviation feature is specifically used to anchor the alignment between the embodied system and the objective physical world, and is the core basis for identifying embodied perceptual hallucinations. It, along with the semantic and action dimensions, supports the subsequent risk classification.

[0063] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the system performs iterative nearest-point registration between the real-time point cloud acquired by the wrist-worn binocular camera and the standard bolt point cloud template in each control cycle, and calculates the average geodetic distance deviation. Millimeters, take the registration deviation allowable threshold. Millimeters, normalized to obtain registration bias .

[0064] Furthermore, the system consists of a 3D topology map of the workstations. With the physical property library of objects The constraint set was exported. A total of 8 constraints were constructed under this job cycle, covering three main categories: gravity support constraints, bolt size matching constraints, and spatial position constraints. One hole position matching constraint was determined to be violated due to positioning deviation. The number of violated constraints is... Total number of constraints Violation of proportion .

[0065] Furthermore, take the fusion weight. , Substitute into the fusion to obtain That is, the physical consistency deviation characteristic of this task's rhythm is 0.39, corresponding to a moderate deviation in perceptual hallucinations, which is the current dominant deviation dimension.

[0066] This embodiment measures the deviation of perception results from two levels: spatial positioning alignment and physical rule constraints. These two factors are then weighted and fused into a unified-dimensional physical consistency deviation feature, enabling quantitative characterization of perceptual illusions. By introducing a constraint satisfaction problem based on a scene topology map and an object physical attribute library, perceptual illusions such as object recognition errors and spatial positioning deviations can be transformed into automatically solvable constraint violation ratios, avoiding the false negatives caused by traditional methods that rely solely on point cloud registration deviations. Furthermore, since the registration deviation is normalized based on the allowable registration deviation threshold, different accuracy levels of work scenarios can share the same set of criteria, improving the portability of the solution across different workstations.

[0067] In some embodiments, the calculation of the action feasibility deviation characteristics in step S120 may include, but is not limited to, steps S410 to S480: Step S410: Determine the planned trajectory based on the task planning action chain. The planned trajectory includes multiple pose points. Step S420: Construct the Jacobian matrix based on joint pose data, task planning action chain, and robot kinematic parameters; Step S430: Calculate the Jacobian condition number for each pose point among multiple pose points; Step S440: Compare the Jacobian condition number of each pose point with the condition number threshold to obtain the motion singularity. Step S450: Construct a symbolic distance field based on the scene topology map and the object physical attribute library; Step S460: Based on the symbolic distance field, calculate the minimum distance between the planned trajectory and the obstacle; Step S470: Compare the minimum distance with the safe distance threshold to obtain the collision risk; Step S480: Integrate motion singularity and collision risk to obtain motion feasibility deviation characteristics.

[0068] In this step, the planned trajectory is first determined based on the task planning action chain in the task reasoning semantic sequence data. Here, the task planning action chain refers to the data... In It is a sequence of actions output by the embodied large model, consisting of several ordered actions. Each action includes the target pose, motion mode, and process parameters. The planned trajectory refers to the continuous motion path obtained by interpolating the action chain planned according to the task in joint space or Cartesian space. The pose point refers to the discrete pose sampling point selected on the planned trajectory at fixed interpolation intervals or according to action segments. Each pose point corresponds to a set of joint angles. ,in Let be the index of the pose point, and let be the total number of pose points. Each pose point contains the position and orientation of the end effector.

[0069] Furthermore, a Jacobian matrix is ​​constructed based on the joint pose data, task planning action chain, and robot kinematic parameters in the prior constraint data of the physical world from the original multimodal perception data.

[0070] Among them, joint pose data refers to data In These are the joint angles and angular velocities acquired in real time by the encoders of each joint; robot kinematic parameters refer to the data... In The parameters are the pre-calibrated linkage parameters of the Denavitt-Hartenberg robot; the Jacobian matrix is ​​the first-order transformation matrix that maps the joint velocity space to the end effector velocity space, denoted as . Where 6 represents the velocity degrees of freedom of the end effector. This represents the number of robot joints, whose elements are determined by the current joint angles and link parameters.

[0071] Joint pose data provides the real-time joint angles required to construct the Jacobian matrix. The task planning action chain provides the target pose and the joint angle sequence obtained from the inverse kinematics for each action. Robot kinematic parameters provide the length, offset, and torsion angle of each link. Thus, the construction of the Jacobian matrix depends on the real-time state, the planning objective, and the prior parameters, ensuring that the subsequent feasibility strictly corresponds to the current operational situation.

[0072] Furthermore, the Jacobian condition number is calculated for each of the multiple pose points. The Jacobian condition number is the ratio of the maximum singular value to the minimum singular value of the Jacobian matrix, and it characterizes the degree of kinematic isotropy of the robot at that pose point. It is calculated according to the following formula: ; in, For the first Jacobian condition number for each pose point and These are the maximum and minimum singular values ​​of the Jacobian matrix, respectively. The larger the Jacobian condition number, the closer the pose point is to the kinematic singular configuration. The worse the motion controllability of the end effector at that point, the more likely it is to experience situations where the velocity tends to infinity or the torque amplifies in a certain direction.

[0073] Furthermore, the Jacobian condition number of each pose point is compared with a condition number threshold to obtain the motion singularity. Here, the condition number threshold is a pre-set upper limit of the Jacobian condition number used to define whether a pose point is in an acceptable kinematic state, denoted as [insert threshold here]. It is obtained from the structural parameters of the robot body and the statistics of historical instability samples; motion singularity refers to the statistical value of the Jacobian condition number of all pose points on the planned trajectory. With condition number threshold The scalar obtained after comparison and normalization using the Sigmoid function takes values ​​between 0 and 1; a larger value indicates a higher degree of kinematic singularity in the overall planned trajectory. In some embodiments, Take the average value of the Jacobian condition number of all pose points on the planned trajectory.

[0074] Furthermore, a symbolic distance field is constructed based on the scene topology map and object physical attribute database from the prior constraint data of the physical world. The symbolic distance field refers to the distance between the scene topology map and the object physical attribute database. Using spatial reference and object physical property database The geometric dimensions of each object recorded are based on a scalar field constructed according to the occupancy criteria. The scalar field defines the directed distance from any point in the space to the nearest obstacle surface, with positive values ​​outside the obstacle, zero values ​​on the obstacle surface, and negative values ​​inside the obstacle. By constructing a signed distance field, obstacles with complex geometries can be uniformly represented as distance functions that can be quickly queried, avoiding the computational overhead of finding intersections of triangular faces one by one.

[0075] Furthermore, based on the symbolic distance field, the minimum distance between the planned trajectory and the obstacle is calculated. The minimum distance is the minimum value obtained by substituting each pose point on the planned trajectory into the symbolic distance field for querying, denoted as [missing value]. The smaller the minimum distance, the closer the planned trajectory is to the obstacle, and the greater the possibility of a collision.

[0076] Furthermore, the minimum distance is compared with a safe distance threshold to obtain the collision risk. The safe distance threshold refers to the data... Safety constraint threshold The lower limit of the distance used to define the safe clearance between the planned trajectory and obstacles is denoted as . The value of this parameter is determined by the robot's body dimensions, the end effector dimensions, and the process safety margin; in this station, it is set to 5 mm. Collision risk is a scalar value obtained by comparing the difference between the safety distance threshold and the minimum distance relative to the safety distance threshold, and then normalizing it using the Sigmoid function. Its value is between 0 and 1. When the minimum distance is not less than the safety distance threshold, the difference is non-positive, and the normalized result is low, indicating that there is sufficient safety clearance between the planned trajectory and the obstacle. When the minimum distance is less than and approaches the safety distance threshold, the normalized result increases, indicating an increased collision risk.

[0077] Furthermore, by fusing motion singularity and collision risk, the feasibility deviation characteristics of the action are obtained. Here, fusing refers to the process of weighting and summing motion singularity and collision risk according to preset weights, calculated using the following formula: ; in, This refers to the characteristics of action feasibility deviation. The fusion weights for motion singularity. The fusion weights for collision risk are equal to 1, and their values ​​are obtained from training with historical samples. This is the Sigmoid function. The larger the value, the higher the impossibility of the action and the greater the risk of action-related hallucinations.

[0078] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the three-step task action chain output by the large model is shown below. After trajectory interpolation, the planned trajectory is obtained. Three key pose points are selected on the planned trajectory according to the action segments: the bolt gripping point, the hole positioning transition point, and the torque tightening point. The six joint angles are collected in real-time by a six-axis joint encoder, and the parameters of the Denavitt-Hartenberg robot are compared. Construct the Jacobian matrix at each key pose point Calculate the average Jacobian condition number of the three key poses on the planned trajectory. Take the condition number threshold Normalization yields singularity scores .

[0079] Furthermore, the system constructs a symbolic distance field using a 3D topological map of the workstation and a physical attribute database of objects. By substituting the three key pose points into the query, the minimum distance between the planned trajectory and the surrounding tooling is obtained. Millimeters, taking the safe distance threshold Millimeters, normalized collision risk score .

[0080] Furthermore, take the fusion weight. , Substitute into the fusion to obtain This means that the action feasibility deviation characteristic of this task's rhythm is 0.40, corresponding to a low risk of action-related hallucinations.

[0081] This embodiment measures the feasibility of planned actions from two perspectives: kinematic singularity and collision risk. These two factors are then weighted and fused after Sigmoid normalization to form a unified dimension of action feasibility deviation features, enabling quantitative characterization of action-related illusions. Since the Jacobi condition number is constructed based on real-time joint poses and prior kinematic parameters, it can identify singular configurations implicit in the planned trajectory, avoiding sudden changes in joint velocity and torque amplification caused by singular poses. The introduction of a symbolic distance field to uniformly represent complex geometric obstacles allows for collision distance queries to be completed within millisecond-level control cycles, meeting the computational latency requirements of embodied intelligent real-time control. Because both components are Sigmoid normalized, the fusion result is stably constrained within the 0-1 range, allowing it to directly participate in subsequent nonlinear fusion along with deviation features from the semantic and perceptual dimensions.

[0082] In some embodiments, the nonlinear fusion of semantic confidence bias features, physical consistency bias features, and action feasibility bias features in step S130 may include, but is not limited to, steps S510 to S560: Step S510: Based on the historical decision sample database, perform marginal distribution fitting on the semantic confidence bias feature, physical consistency bias feature, and action feasibility bias feature respectively to obtain the semantic marginal distribution, physical marginal distribution, and action marginal distribution; Step S520: Based on the semantic edge distribution, physical edge distribution, action edge distribution and historical sample correlation parameters, construct a vine-like joint distribution structure to decompose the three-dimensional dependency relationship between semantic confidence bias features, physical consistency bias features and action feasibility bias features, and obtain the decomposed joint distribution structure. Step S530: Based on the joint distribution structure, the three-dimensional dependency relationship is characterized to obtain the three-dimensional joint distribution function; Step S540: Perform probability integral transformation on the semantic edge distribution, physical edge distribution and action edge distribution respectively to obtain the semantic probability integral value, physical probability integral value and action probability integral value; Step S550: Substitute the semantic probability integral value, physical probability integral value, and action probability integral value into the three-dimensional joint distribution function to calculate the comprehensive hallucination risk measure; Step S560: Based on semantic confidence bias features, physical consistency bias features, and action feasibility bias features, determine the dominant hallucination type label.

[0083] In this step, firstly, based on historical decision-making sample database data, marginal distributions are fitted to the semantic confidence bias feature, physical consistency bias feature, and action feasibility bias feature, respectively, to obtain semantic marginal distributions, physical marginal distributions, and action marginal distributions. Marginal distribution fitting refers to using historical decision-making sample database data... The historical values ​​of the accumulated deviation features of each dimension are used as samples, and a parameterized probability distribution is used to estimate the distribution pattern of the values ​​of each dimension. Since the values ​​of the deviation features of the three dimensions are all in the interval between 0 and 1 and the distribution pattern is skewed, this application uses the beta distribution as the marginal distribution form of each dimension, and respectively... , , The beta edge distribution was obtained by fitting. , , Its distribution parameters are obtained from historical samples through maximum likelihood estimation.

[0084] Furthermore, based on semantic marginal distribution, physical marginal distribution, action marginal distribution, and historical sample correlation parameters, a vine-like joint distribution structure is constructed to decompose the three-dimensional dependencies between the three dimensions, resulting in the decomposed joint distribution structure. Here, historical sample correlation parameters refer to parameters obtained from historical decision sample database data, used to characterize the correlation structure and strength between each dimension, including parameters used to characterize lower-tail cooperativity. Parameters used to characterize conditional correlation The vine-like joint distribution structure refers to the pairwise connection function decomposition structure organized using the C-vine structure, which decomposes a three-dimensional joint distribution into a combination of several binary connection functions, thereby reducing the difficulty of directly modeling high-dimensional joint distributions. The three-dimensional dependency relationship refers to the objectively existing nonlinear correlation between the three-dimensional deviation features. For example, when perceptual deviation occurs, it often triggers action deviation simultaneously, and reasoning deviation is transmitted down the action chain to trigger action deviation.

[0085] Specifically, because the simple weighted summation assumes that the dimensions are independent, it cannot characterize the aforementioned nonlinear correlations, leading to a significant underestimation of the overall risk when multiple dimensions deviate moderately simultaneously. This application employs a pairwise connection function with a C-vine structure to decompose the three-dimensional dependency into a product of two binary connection functions. One binary connection function characterizes the lower-tail coherence between the first and second dimensions, while the other characterizes the conditional correlation between the first and third dimensions given the second dimension. This allows for a complete characterization of the three-dimensional dependency structure while preserving computational solvability.

[0086] Furthermore, based on the joint distribution structure, the three-dimensional dependencies are characterized to obtain the three-dimensional joint distribution function. The characterization process involves multiplying the decomposed binary link functions according to their combination relationships and determining the parameter values ​​of each link function using historical sample correlation parameters, thereby obtaining an explicit functional expression that can output the three-dimensional joint probability. The three-dimensional joint distribution function is a multivariate function that takes the probability integrals of the three dimensions as inputs and outputs the joint distribution value of the three dimensions. Its values ​​are located in the interval between 0 and 1, and it is constructed according to the following formula: ; in, A binary connection function to characterize the lower-tail coherence between the first and second dimensions, To characterize the conditional correlation between the first and third dimensions given a second dimension, a binary connection function is proposed. and These are the corresponding conditional distribution functions. and The parameters of the pairwise join function are derived from historical samples. It is obtained through maximum likelihood estimation.

[0087] Furthermore, probability integral transformations are performed on the semantic marginal distribution, physical marginal distribution, and action marginal distribution respectively to obtain semantic probability integral values, physical probability integral values, and action probability integral values. The probability integral transformation refers to the process of substituting a deviation feature value of a certain dimension into its corresponding marginal distribution function to obtain the cumulative probability position of that value in the historical sample distribution. Calculate according to the following formula: ; in, The integral value of semantic probability. The physical probability integral value. These are the integral values ​​of the action probability. All three values ​​range from 0 to 1. A larger value indicates that the deviation in that dimension is further back in the historical sample distribution, meaning the deviation is more severe. Through probability integral transformation, the three deviation characteristics with different dimensions and distribution patterns are uniformly mapped to a comparable probability scale, providing a unified input for subsequent calculations of the connection function.

[0088] Furthermore, the semantic probability integral, physical probability integral, and action probability integral are substituted into the three-dimensional joint distribution function to calculate the comprehensive hallucination risk measure. That is, the function value of the three-dimensional joint distribution function at the current probability integral value is taken as the comprehensive hallucination risk measure, calculated according to the following formula: ; in, The comprehensive hallucination risk measure is the joint probability of three dimensions simultaneously deviating from normal levels, ranging from 0 to 1. A higher value indicates a higher overall severity of hallucination. Since this measure is the joint probability of the deviation levels of the three dimensions under historical correlation, it naturally has the ability to amplify coordinated deviations. That is, the measure output when there is a moderate deviation in multiple dimensions simultaneously is significantly higher than the measure output when there is an equal deviation in a single dimension.

[0089] Furthermore, based on semantic confidence bias, physical consistency bias, and action feasibility bias features, a dominant hallucination type label is determined. Specifically, the hallucination category corresponding to the dimension with the largest value among the three dimensions is taken as the dominant hallucination type label. This label can be one of three categories: reasoning, perception, or action, corresponding to the cases where the semantic confidence bias, physical consistency bias, and action feasibility bias features have the largest values, respectively. The dominant hallucination type label is used to determine the splitting direction of the second layer in subsequent decision tree routing, ensuring that the treatment strategy directly targets the primary source of the current hallucination.

[0090] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the semantic confidence bias features are calculated from steps S210 to S480. Physical consistency deviation characteristics Characteristics of deviation in action feasibility Perform a beta distribution probability integral transform on each of the three deviation values ​​to obtain the probability integral values. , , ; From historical samples The parameters of the fitted C-vine connection function are as follows: , .

[0091] Furthermore, by substituting the three probability integral values ​​into the pairwise connection function of the C-vine structure, the three-dimensional joint distribution value is calculated. Therefore, the overall hallucination risk measure for this task's rhythm is 0.42. Meanwhile, based on the values ​​of the three-dimensional deviation characteristics, the physical consistency deviation characteristic of 0.39 is the maximum value; therefore, the dominant hallucination type is determined to be perceptual.

[0092] Specifically, the deviation characteristics of the three dimensions are all at a moderate level, and the physical dimension and the action dimension show a synchronous deviation. If a simple weighted summation method is used for fusion, the correlation between the dimensions will be ignored, and the superimposed risk brought about by the synchronous deviation will not be reflected. The resulting comprehensive value will be significantly lower than the 0.42 obtained in this application. This application incorporates this synergistic effect through the pairwise connection function of the C-vine structure, making the comprehensive risk measurement closer to the actual severity, so as to be able to match the appropriate handling strategy in subsequent routing.

[0093] This embodiment achieves nonlinear fusion of the three-dimensional deviation features by first fitting the marginal distributions of each dimension with a beta distribution, then decomposing the three-dimensional dependencies with a pairwise connection function of a C-vine structure, and finally substituting the result of the probability integral transformation into the three-dimensional joint distribution function to calculate the comprehensive hallucination risk measure. Because the fusion process explicitly incorporates the lower-tail collaboration and conditional correlation between dimensions, it overcomes the deficiency of simple weighted summation in failing to characterize the coupling effect between dimensions. Since the probability integral transformation uniformly maps each dimension to a probability scale, the fusion result is unaffected by differences in the dimensions and distribution patterns of each dimension. Because the dominant hallucination type label and the comprehensive hallucination risk measure are output simultaneously, subsequent decision tree routing can simultaneously obtain the criteria for both the intensity and type dimensions, providing complete input for differentiated strategy matching.

[0094] In some embodiments, the generation of the target execution strategy through a preset hierarchical error correction decision tree in step S140 may include, but is not limited to, steps S610 to S650: Step S610: Compare the comprehensive hallucination risk measure with the preset severity level classification threshold to determine the severity level, which includes mild, moderate and severe. Step S620: Match the dominant hallucination type label with the preset hallucination type to determine the hallucination type. The hallucination type includes reasoning type, perception type and action type. Step S630: Input the severity level into the first layer of the preset hierarchical error correction decision tree, and the first layer outputs the first routing result according to the severity level. The first routing result includes a mild branch, a moderate branch, or a severe branch. Step S640: Input the hallucination type into the second layer of the preset hierarchical error correction decision tree. The second layer outputs the second routing result according to the hallucination type under the branch pointed to by the first routing result. The second routing result includes reasoning sub-branch, perception sub-branch or action sub-branch. Step S650: Input the second routing result into the leaf node of the preset hierarchical error correction decision tree, and output the set of handling strategies corresponding to the second routing result as the target execution strategy.

[0095] In this step, the comprehensive hallucination risk measure is first compared with a preset severity level threshold to determine the severity level. The preset severity level threshold refers to the threshold data pre-stored in a preset hierarchical error correction decision tree and derived from historical data. The splitting threshold obtained during training includes and And satisfy The values ​​of both are updated iteratively online; the severity level refers to the level obtained by dividing the range according to the comprehensive hallucination risk measure, which includes three levels: mild, moderate and severe. The division rule is: in order to meet the following conditions... Under these circumstances, it is determined to be a mild hallucination, provided that the following conditions are met. Under these circumstances, it is determined to be a moderate hallucination, provided that the following conditions are met. In this case, it was determined to be a severe hallucination; this workstation was initially selected... , .

[0096] Specifically, to ensure that severe deviations in a single dimension are not masked by the grading system, this application also establishes an alignment score red line rule. This rule states that if any value in the semantic confidence bias feature, physical consistency bias feature, or action feasibility bias feature exceeds 0.8, the severity level of the hallucination is directly increased by one level, with the severe level remaining severe even after the increase. This red line rule prevents the dilution of severe single-dimensional biases caused by multi-dimensional averaging, thus ensuring task safety.

[0097] Furthermore, the dominant hallucination type label is matched with the preset hallucination type to determine the hallucination type. The preset hallucination type refers to the set of categories stored in the preset hierarchical error correction decision tree for the second-level split, which includes reasoning, perception, and action categories; the hallucination type refers to the category determined by comparing the dominant hallucination type label with the preset hallucination type item by item, which is used to indicate the main source link of the hallucination, so that subsequent strategies can directly target the main contradiction rather than generalized treatment.

[0098] Furthermore, the severity level is input into the first layer of a pre-defined hierarchical error correction decision tree, which then outputs a first routing result based on the severity level. Here, the first layer refers to the root node layer in the pre-defined hierarchical error correction decision tree where severity level is the splitting attribute; the first routing result refers to the branch direction output by the first layer, which includes mild, moderate, and severe branches, corresponding to autonomous recovery, assisted recovery, and manual takeover branches, respectively. Because the first layer splits only based on severity level, the intensity of treatment strictly corresponds to the intensity of risk, avoiding both under- and over-treatment.

[0099] Furthermore, the hallucination type is input into the second layer of a pre-defined hierarchical error correction decision tree. The second layer, under the branch pointed to by the first routing result, outputs a second routing result based on the hallucination type. Here, the second layer refers to the intermediate layer in the pre-defined hierarchical error correction decision tree located below each severity level branch, with hallucination type as the splitting attribute; the second routing result refers to the branch pointed to by the output of the second layer, which includes reasoning sub-branches, perception sub-branches, and action sub-branches. Since the second layer only splits under the branch pointed to by the first routing result, the treatment intensity corresponding to the same hallucination type is not the same under different severity levels, thus forming a two-dimensional policy matrix of severity and hallucination type.

[0100] Furthermore, the second routing result is input into the leaf nodes of the preset hierarchical error correction decision tree. The leaf nodes output a set of handling strategies corresponding to the second routing result, which serves as the target execution strategy. Here, a leaf node refers to a terminal node in the preset hierarchical error correction decision tree that no longer splits. Each leaf node stores a set of handling strategies corresponding to its current branch. This includes the strategy type, execution parameters, expected success rate, and estimated processing time. The strategy type indicates the type of error correction algorithm used this time, the execution parameters indicate the parameter values ​​required for the error correction algorithm to run, the expected success rate is given by the statistical success rate of historical samples of this branch, and the estimated processing time is given by the average processing time of historical samples of this branch.

[0101] Specifically, the preset hierarchical error correction decision tree provided in this application embodiment contains a total of 9 leaf nodes, which are distributed according to the combination of severity level and hallucination type as follows: Mild reasoning leaf nodes output local replanning plus knowledge graph constraint reverification strategy; mild perception leaf nodes output multi-view re-acquisition plus point cloud re-registration strategy; mild action leaf nodes output local trajectory re-interpolation plus singular pose avoidance strategy. All of the above mild branch strategies are executed completely autonomously by the system without manual monitoring; moderate reasoning leaf nodes output full-chain replanning plus retrieval-enhanced fact verification strategy; moderate... The leaf nodes for low-level perception output a 3D point cloud template matching plus local view resampling correction strategy; the leaf nodes for medium-level motion output a global trajectory replanning plus symbolic distance field obstacle avoidance strategy. The above medium-level branch strategies are executed by the system while pushing hallucination state monitoring information to the human-machine collaborative operation console; the leaf nodes for high-level reasoning output a task pause plus manual task replanning strategy; the leaf nodes for high-level perception output a state monitoring plus manual pose annotation strategy; the leaf nodes for high-level motion output a safety braking plus manual teleoperation strategy. All of the above high-level branch strategies trigger a four-level gradient takeover process.

[0102] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the comprehensive hallucination risk measure obtained from steps S510 to S560 is 0.42, and the dominant hallucination type label is perception. The comprehensive hallucination risk measure is compared with the splitting threshold... The severity level is determined to be moderate; at the same time, the red line rule is checked, and the values ​​of the three dimensions of deviation characteristics are 0.28, 0.39 and 0.40 respectively, none of which exceed 0.8, so the level will not be upgraded.

[0103] Furthermore, the moderate severity level is input into the first layer, and the moderate branch output by the first layer is used as the first routing result. The hallucination type perception class is input into the second layer, and the second layer continues to split under the moderate branch, outputting the perception class sub-branch as the second routing result. The second routing result points to the leaf node of the moderate perception class. This leaf node outputs a set of handling strategies that implement the 3D point cloud template matching plus local viewpoint resampling correction strategy and push hallucination state monitoring information to the human-machine collaborative operation platform. This set serves as the target execution strategy. The execution parameters include a wrist camera deflection angle of 30 degrees, a resampling point density of 4 points per square millimeter, and a maximum number of iterations for iterative nearest point registration of 50. The expected success rate is 82% based on the historical samples of this branch, and the expected time is 0.8 seconds based on the historical samples of this branch.

[0104] This embodiment inputs the comprehensive hallucination risk measure and the dominant hallucination type label into the first and second layers of a preset hierarchical error correction decision tree, respectively, to form a two-dimensional policy routing structure based on severity and hallucination type. Since the first layer is split by level and the second layer by type, the same type of hallucination is matched with policies of varying strengths at different levels, avoiding the redundancy in mild hallucination correction and the insufficient response to severe hallucinations caused by a one-size-fits-all approach to error correction policies. Because the leaf nodes directly output a complete set of policies containing execution parameters, expected success rate, and estimated time, policies can be directly issued and executed without manual configuration, achieving end-to-end linkage from hallucination quantification to treatment execution.

[0105] In some embodiments, the execution of hierarchical autonomous recovery and human-machine collaborative takeover in step S150 may include, but is not limited to, steps S710 to S740: Step S710: According to the target execution strategy, execute the corresponding recovery strategy to obtain the corresponding recovery execution result, and obtain the updated multimodal perception raw data, updated task reasoning semantic sequence data and updated physical world prior constraint data of the target embodied intelligence. The recovery strategy includes embodied intelligence autonomous recovery and artificial assisted recovery. Step S720: Based on the updated multimodal perception raw data, the updated task reasoning semantic sequence data, and the updated physical world prior constraint data, calculate the updated comprehensive hallucination risk measure. Step S730: Compare the updated comprehensive hallucination risk measure with the mild threshold in the severity level classification threshold in the preset hierarchical error correction decision tree to obtain the comparison result; Step S740: Based on the comparison results, generate collaborative execution results, which include either the recovery execution results or the human-machine collaborative takeover results.

[0106] In this step, the corresponding recovery strategy is first executed according to the target execution strategy to obtain the corresponding recovery execution result, and the updated multimodal perception raw data, updated task reasoning semantic sequence data, and updated physical world prior constraint data of the target embodied intelligence are obtained simultaneously. Among them, the recovery strategy refers to the error correction algorithm specified by the target execution strategy to eliminate the current illusion, which includes two categories: embodied intelligence autonomous recovery and human-assisted recovery. Embodied intelligence autonomous recovery refers to the error correction process that is completed independently by the system without human intervention, which corresponds to the leaf node strategies under the light branch. Human-assisted recovery refers to the error correction process that pushes status monitoring information to the human operator while the system executes the enhanced error correction algorithm, and the operator intervenes to confirm or correct when necessary, which corresponds to the leaf node strategies under the medium branch. The recovery execution result refers to the intermediate result obtained after the recovery strategy is executed, which describes the current error correction execution status. It includes the strategy identifier used, whether the execution is completed, and the pose or semantic correction amount after the execution. The updated multimodal perception raw data, the updated task reasoning semantic sequence data, and the updated physical world prior constraint data refer to the same type of data that is re-collected or regenerated in the new control cycle after the recovery strategy is executed. They are used to reflect the actual state of the target embodied intelligence after the error correction.

[0107] Specifically, since the work scenario and robot state have changed after the recovery strategy is executed, directly using the deviation features before execution as subsequent criteria will lead to distorted verification conclusions. Therefore, this application re-collects the above three types of data after each recovery strategy is executed to form an updated dataset that is isomorphic to the data before execution, providing a consistent input basis for the calculation of the updated comprehensive hallucination risk measure.

[0108] Furthermore, based on the updated multimodal perception raw data, the updated task reasoning semantic sequence data, and the updated physical world prior constraint data, an updated comprehensive hallucination risk measure is calculated. Here, the updated comprehensive hallucination risk measure refers to the comprehensive risk measure recalculated using the updated three types of data as input, following the same method as steps S120 and S130, and is denoted as... Since the calculation process is completely consistent with the initial calculation, the measurements before and after the update are strictly comparable and can objectively reflect the actual effect of this recovery strategy in numerical form.

[0109] Furthermore, the updated comprehensive hallucination risk measure is compared with the mild threshold in the severity level classification threshold of the preset hierarchical error correction decision tree to obtain the comparison result. Here, the mild threshold is the splitting threshold. The comparison result refers to and The size relationship between them includes the following two cases: the first case is This indicates that the hallucination has been eliminated and the system state has returned to within the mild range; the second scenario is... This indicates that the current recovery strategy has not reduced the risk to an acceptable level.

[0110] Furthermore, based on the comparison results, a collaborative execution result is generated. This collaborative execution result refers to the final output of this handling process, which includes either the recovery execution result or the human-machine collaborative takeover result; when the comparison result is... In the case of a system-wide recovery execution result, the system generates a recovery execution result as the collaborative execution result, indicating that the hallucination was autonomously recovered by the system and the task continues to proceed according to the original plan; in the case of a comparison result of In the case of a hallucination, a human-machine collaborative takeover result is generated as the collaborative execution result, indicating that the treatment level needs to be upgraded. That is, the current severity level is automatically upgraded to the previous severity level and the policy routing of step S140 is re-executed. If it is already at a severe level, the hierarchical takeover process is triggered.

[0111] Specifically, the progressively upgraded verification mechanism operates according to the following rules: at the mild level, the system calls the corresponding type of self-correcting algorithm, and after execution, it recalculates the result. ,like If recovery is successful, the task continues; otherwise, it automatically escalates to moderate hallucination and reroutes. At the moderate level, the system invokes an enhanced error correction algorithm and simultaneously pushes status monitoring information to the human-machine collaborative console. After execution, the result is recalculated. ,like If the recovery is successful, it is considered successful; otherwise, it automatically escalates to severe hallucination and triggers manual takeover. At the severe level, a smooth transition is achieved through four levels: status monitoring, operational suggestions, partial takeover, and full takeover. The scope of manual permissions is automatically adjusted based on the dynamic changes in the comprehensive hallucination risk measure. Because the measure is recalculated and compared with the mild threshold for each execution result before deciding whether to escalate, the potential for autonomous recovery is fully explored, avoiding the resource waste caused by directly jumping to manual takeover upon a single recovery failure.

[0112] Furthermore, in cases of severe severity, this embodiment of the application implements human-machine collaborative takeover according to the following four-level gradient: The first level is the status monitoring level, where the system triggers a safety brake to stop the movement, and simultaneously displays a red warning on the human-machine collaborative operation console, showing the hallucination type, positioning deviation value, and impact range. The operator only assumes monitoring responsibilities and does not intervene in the operation, and the system maintains the safety protection of the lower-level machine; The second level is the operation suggestion level, where the system automatically generates candidate handling solutions and pushes them to the operation console for the operator to select and confirm. For example, in the scenario of incorrect bolt hole position recognition, it generates a regression to the previous safe posture to re-perceive the global position and call the preset hole position. The system directly executes two candidate solutions based on coordinates. After the operator clicks to confirm one of them, the system automatically executes it. The third level is the partial takeover level. If the overall illusion risk measure is still higher than 0.7 after the solution is executed, the operator takes over the target pose setting permissions and marks the actual bolt hole position through a drag-and-drop interface. The system automatically generates the motion trajectory and tightening parameters based on the marking results. The fourth level is the full takeover level. If there are unconventional tooling deviations in the scene or if partial takeover still cannot eliminate the risk, the system switches to the force feedback teleoperation mode. The operator directly controls the robot arm's movements through the handle, and the system provides real-time collision warnings and torque limiting assistance.

[0113] Specifically, the aforementioned four-level gradient takeover authorization transition is driven by the dynamic changes in the comprehensive hallucination risk metric. The system recalculates the metric after the previous level is completed, and only grants authorization for the next level if the metric has not fallen back to the expected level. This achieves a smooth transition from system-controlled autonomy to manual remote operation, avoiding task interruptions and operational shocks caused by sudden changes in authorization. Simultaneously, when executing manual takeover, the system synchronously reads human operator load data from the historical decision sample database. When the on-duty operator's load rate exceeds the preset load limit, operators with lower load rates are given priority to take over the takeover request, and the mild threshold is relaxed accordingly to expand the system's autonomous handling scope and achieve optimal scheduling of human resources.

[0114] Furthermore, this process, while issuing the collaborative execution results, also outputs a handling result label. The data includes: actual processing time, human intervention time, and final comprehensive hallucination risk measurement. The outcome is tagged as success or failure. Actual processing time is the duration from hallucination detection to the generation of the collaborative execution result. Human intervention time is the cumulative time the operator actually holds operational authority. The final comprehensive hallucination risk measurement is the value measured at the end of the processing. These four data points serve as a process record for this processing, supporting subsequent online closed-loop optimization.

[0115] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the system, based on the target execution strategy output by the leaf node of the moderate perception class, adjusts the robotic arm's wrist camera to a 30-degree side view, re-acquires the local point cloud of the bolt station, and performs iterative nearest-point registration with the standard bolt point cloud template to correct the bolt positioning coordinates, completing this autonomous recovery. After the recovery is completed, the system re-acquires the multimodal perception raw data and task inference semantic sequence data in the new control cycle, and recalculates the updated comprehensive hallucination risk measure according to the same process. .

[0116] Furthermore, 0.29 was compared with the mild threshold. The comparison results are as follows: Therefore, the system generated a recovery execution result as the collaborative execution result for this operation, determining that the autonomous recovery was successful, and the task continued into the tightening process. The actual processing time for this operation was 0.72 seconds, the manual intervention time was 0 seconds, the final comprehensive hallucination risk measure was 0.29, and the processing result was labeled as successful. In comparison, if the traditional dual mode of detection alarm and manual takeover were used, this perceptual hallucination would have been directly determined as unmanageable and switched to manual remote operation. Based on the historical average manual intervention processing time of 3.37 seconds for this workstation, the cycle time for this operation would have been extended by approximately 2.65 seconds and would have required one operator to perform the entire operation.

[0117] For example, in another cycle at the same workstation, the robotic arm misidentifies a process weight reduction hole as a bolt mounting hole, resulting in a complete bolt hole location identification error. In this case, the system calculates the physical consistency deviation characteristics. The combined hallucination risk measure after fusion is 0.81, falling into... In the severe phase, a level four gradient takeover was triggered. The system first triggered a safety brake to stop the robotic arm's movement. A red warning popped up on the control panel, displaying the hallucination type, positioning deviation value, and affected area. The operator entered status monitoring. Subsequently, the system generated two candidate handling schemes: reverting to the previous safe pose for global re-perception and directly executing the preset hole coordinates. After the operator clicked to confirm the first scheme, the system automatically executed the reversion and global re-perception. After re-perception, the overall hallucination risk measure was still higher than 0.7. The system granted partial takeover permissions. The operator marked the actual bolt hole positions in the 3D view through a drag-and-drop interface. The system automatically generated the motion trajectory and tightening parameters and completed the remaining tightening work. The actual handling time was 8.6 seconds, and the manual intervention time was 5.2 seconds.

[0118] This embodiment re-collects data and recalculates the comprehensive hallucination risk measure after each recovery strategy is executed, performing closed-loop verification of the treatment effect using values ​​of the same dimension. The system then determines whether to continue the task or escalate the treatment level based on the comparison with the mild threshold. Because of the step-by-step escalation verification mechanism, the system can fully attempt autonomous recovery at mild and moderate levels, significantly reducing the frequency of human intervention. At the severe level, a four-level gradient takeover is used, with human permissions gradually opened according to the change in measure, avoiding task interruptions and operational shocks caused by hard switching. Furthermore, the relaxation and tightening of the takeover threshold are linked to the real-time operator load, allowing for optimal scheduling of human resources in multi-workstation cluster scenarios.

[0119] In some embodiments, after step S740, the embodied intelligent illusion graded autonomous recovery and human-machine collaborative takeover method provided in this application embodiment may also include, but is not limited to, steps S810 to S830: Step S810: Add the handling result label, actual time consumed, duration of human intervention, and final comprehensive hallucination risk measurement of this handling as new samples to the historical decision sample database and update the sample distribution; Step S820: Using the Bayesian posterior update method, the correlation parameters between the splitting threshold of the preset hierarchical error correction decision tree and the vine-like joint distribution structure are re-estimated based on the updated historical decision sample database data. Step S830: If the error correction success rate of any branch in the preset hierarchical error correction decision tree is continuously lower than the preset success rate threshold, the split threshold corresponding to that branch is automatically lowered, the samples in the boundary interval are upgraded to the next severity level, and the effective strategies for manual handling are extracted as new leaf nodes.

[0120] In this step, the recorded handling result tags, actual time consumption, human intervention time, and final comprehensive hallucination risk measurement are first added as new samples to the historical decision sample database to update the sample distribution. New samples refer to the complete records generated throughout the entire handling process, which, in addition to the four handling result data mentioned above, also include the deviation feature values ​​of each dimension, the leaf node identifier, the recovery strategy identifier, and the operational scenario identifier. Updating the sample distribution means re-counting the sample quantity, error correction success rate, and average handling time for each branch after adding the new samples, ensuring the historical decision sample database is updated. The statistical patterns reflected are consistent with the current operating scenario. Since the operating scenario of embodied intelligence will drift with changes in vehicle model, tool wear and tear, and lighting, if the historical samples are not updated for a long time, the criteria obtained based on them will gradually become invalid. Therefore, this application performs sample reinjection at the granularity of each disposal.

[0121] Furthermore, a Bayesian posterior update method is employed to re-estimate the splitting threshold of the preset hierarchical error correction decision tree based on the updated historical decision sample database. , With the parameters of the pairwise join function Update according to the following formula: ; in, The prior distribution is taken from the posterior distribution obtained from the previous update; This is the likelihood function for observing newly added samples under the current parameter values; The expected value of the updated posterior distribution is taken as the updated parameter value. Through Bayesian posterior update, the parameter estimation can converge continuously with the accumulation of samples, and there is no need to retrain on all historical samples, so the computational cost is controllable.

[0122] Furthermore, if the error correction success rate of any branch in the preset hierarchical error correction decision tree continuously falls below the preset success rate threshold, the split threshold corresponding to that branch is automatically lowered. Samples in the boundary interval are upgraded to the next higher severity level, and effective manual handling strategies are extracted as new leaf nodes. Here, the preset success rate threshold refers to a pre-set lower limit for determining whether the handling capability of a branch is sufficient; the boundary interval refers to the range between the split threshold before and after the reduction, where samples within this interval are upgraded from the original severity level to the next higher severity level after the threshold reduction, thus receiving stronger handling; and the effective manual handling strategy refers to the sequence of operations actually adopted by the operator during manual takeover and judged as successfully handled. After being solidified as a new leaf node, it can be directly invoked by the system when similar scenarios recur without manual intervention.

[0123] For example, taking the bolt tightening operation at the automotive body-in-white chassis assembly station as an example, the system offline summarizes all hallucination handling samples of the day and updates the historical decision sample database daily. Subsequently, a Bayesian posterior update method was used to re-estimate the split threshold and the parameters of the pairwise connection function. In a 7-day statistical analysis, the error correction success rate of the moderately perceptual branch decreased from 82% to 68%, falling below the preset success rate threshold. Based on this, the system automatically adjusted the split threshold. The threshold was lowered from 0.35 to 0.32, causing samples that were originally in the 0.32 to 0.35 boundary range to be upgraded from mild to moderate and processed according to the moderate perception class strategy. At the same time, the system extracted the side-view plus top-view dual-view registration operation commonly used by operators from the manual takeover records, solidified it into a new leaf node strategy under the moderate perception class branch and included it in the strategy library, so that the system can directly perform dual-view registration in subsequent similar scenarios without manual intervention.

[0124] Specifically, the above-mentioned online closed-loop optimization demonstrated the following effects during a 30-day trial run at the automotive body-in-white chassis assembly station: the frequency of manual intervention decreased by 64.2% compared to the original binary scheme of detection and takeover, with 78.3% of hallucinations being resolved through autonomous recovery at levels one and two; the F1 score for hallucination detection reached 0.91, an improvement of 26.4% compared to the original single-dimensional semantic entropy detection scheme, and the false negative rate for physical perception hallucinations decreased by 43.7%; the average time for handling a single hallucination was 0.95 seconds, a reduction of 71.8% compared to the original manual takeover mode, and the overall work efficiency of the station increased by 18.6%; after 30 days of continuous system operation, the overall autonomous error correction success rate increased from the initial 72.1% to 87.4%, and the strategy calibration cost during new model replacement decreased by 65.9%.

[0125] This embodiment constructs a complete capability closed loop of detection, handling, feedback, and optimization by feeding back process data from each handling process into the historical decision sample library and using a Bayesian posterior update method to iterate the splitting threshold and pairwise connection function parameters online. Combined with statistical monitoring of branch success rate, it automatically adjusts the hierarchical boundary and leaf node strategy library. Because the branch handling capability declines due to shifts in the work scenario or changes in task type, the system can automatically adjust the threshold and expand the strategy without manual offline recalibration, significantly reducing scenario adaptation costs. Furthermore, since effective manual handling experience is solidified into new leaf nodes, the system's autonomous recovery capability continuously increases with runtime, overcoming the limitations of existing technical solutions where strategies are statically fixed and capability iteration relies on manual parameter tuning.

[0126] The embodiments of this application construct a multi-dimensional nonlinear fusion grading and quantification system for two-dimensional classification of severity and hallucination type through the above steps, and achieve the following technical effects in terms of hallucination treatment accuracy, human resource utilization rate, human-machine collaboration fluency, scene adaptability and task execution security.

[0127] First, a two-dimensional hierarchical self-healing system based on severity and hallucination type is constructed, significantly reducing the frequency of manual intervention and fully releasing operational resources. This application's embodiment abandons the binary, crude handling mode of detection-based alarm and alarm-based takeover. Instead, it uses a pre-set hierarchical error correction decision tree to classify hallucinations into three severity levels and three hallucination types, matching differentiated autonomous and assisted recovery strategies. Manual intervention is only triggered for severe hallucinations and failed recovery samples. In typical industrial robotic arm assembly scenarios, over 76% of minor perception deviations and motion offsets can be autonomously corrected and handled by the system. The frequency of manual intervention is reduced by over 62% compared to traditional solutions, and the labor cost per workstation is reduced by 47%, achieving a tiered management system where autonomous operation is possible.

[0128] Second, a multi-dimensional nonlinear fusion detection mechanism using pairwise connection functions is proposed, significantly improving the accuracy and recall rate of embodied scene hallucination recognition. This application's embodiments construct a three-dimensional measurement system of semantic Riemann entropy, physical consistency deviation, and action feasibility deviation. Relying on mathematical tools such as Riemannian manifold geodesic distance, constraint satisfaction problem modeling, and Jacobi singularity analysis, it characterizes the deviation degrees of the reasoning layer, perception layer, and action layer, respectively. Furthermore, it employs a vine-like pairwise connection function structure to model the nonlinear dependencies between the three dimensions and calculate the joint risk probability, overcoming the information bottleneck of traditional linear weighting in capturing inter-dimensional coupling effects. Simultaneously, it anchors to physical world constraints for alignment verification. Joint testing on publicly available benchmark datasets and industrial scenarios shows that, compared to traditional single-dimensional semantic entropy detection and simple weighted fusion schemes, this application's embodiments improve the F1 score for hallucination detection by 24% to 31%, increase the recall rate for physical perception hallucinations by over 41%, and reduce the false negative rate for motion-related hallucinations by 46%.

[0129] Third, a four-level gradient human-machine collaborative takeover mechanism is designed to achieve a smooth transition of permissions and significantly reduce the risk of task interruption. This application's embodiments break through the technical path of one-click switching hard takeover, designing a four-level gradient takeover mode: status monitoring, operation suggestions, partial takeover, and full takeover. It dynamically adjusts manual operation permissions based on the perceived risk level, achieving a smooth transition from system autonomous control to manual remote operation. Simultaneously, it supports dynamic adjustment of the takeover threshold and granularity based on the operator's real-time load and task priority. Actual tests show that compared to the traditional hard switching mode, the average interruption time of a single takeover is reduced by more than 72%, the peak operational impact during the takeover process is reduced by 63%, and the number of devices that can be monitored per person in multi-workstation cluster management scenarios is increased by 54%.

[0130] Fourth, a Bayesian-driven online closed-loop optimization mechanism for decision trees is established to achieve continuous self-evolution of self-healing capabilities. This application's embodiments construct a complete capability closed loop of detection, handling, feedback, and optimization. Based on the result data of each hallucination handling, a Bayesian posterior update method is used to iterate online the decision tree splitting threshold, connection function parameters, and leaf node policy library. When the success rate of branch error correction decreases due to shifts in the work scenario or changes in task type, the system automatically adjusts the hierarchical boundaries and policy priorities without requiring manual offline recalibration. Long-term operational test data shows that after 90 days of continuous operation, the overall autonomous error correction success rate increased from the initial 71% to 88%, and the calibration cost for adapting to new scenarios decreased by 68%.

[0131] Fifth, it adopts a fully interconnected data flow architecture to ensure traceability of the handling logic and excellent engineering feasibility. This application's embodiment strictly follows a progressive data flow: original multi-source input, multi-dimensional feature calculation, nonlinear fusion measurement, decision tree strategy routing, and execution feedback iteration. The output of each step is a direct input for the next, resulting in a clear and seamless link logic. The pre-set hierarchical error correction decision tree is fully interpretable, allowing for the traceability of the complete decision path for each action. Simultaneously, the architecture employs a modular design, requiring no modification to the underlying large model and robot hardware control logic, and can be quickly embedded into existing embodied intelligence systems. Compared to black-box error correction schemes based on end-to-end neural networks, this application's embodiment reduces the fault diagnosis cycle by 83% and the engineering deployment cycle by more than 55%.

[0132] like Figure 2 As shown, some embodiments of this application provide a hierarchical autonomous recovery and human-machine collaborative takeover system for embodied intelligent illusions, specifically: The acquisition module 210 is used to acquire the target embodied intelligence's multimodal perception raw data, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data; The calculation module 220 is used to calculate semantic confidence bias features, physical consistency bias features, and action feasibility bias features based on the multimodal perception raw data, task reasoning semantic sequence data, and physical world prior constraint data, respectively. The fusion module 230 is used to perform nonlinear fusion of semantic confidence bias features, physical consistency bias features and action feasibility bias features based on historical sample correlation parameters in the historical decision sample database, to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. The generation module 240 is used to generate a target execution strategy based on a comprehensive hallucination risk measure and a dominant hallucination type label by using a preset hierarchical error correction decision tree. The execution module 250 is used to perform hierarchical autonomous recovery and human-machine collaborative takeover based on the target execution strategy and matched with the comprehensive hallucination risk measure and the dominant hallucination type label, so as to obtain the collaborative execution result.

[0133] It should be noted that the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover system provided in this embodiment is based on the same inventive concept as the above-mentioned embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method. Therefore, the relevant content of the above-mentioned embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method is also applicable to the content of the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover system. Therefore, it will not be repeated here.

[0134] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method.

[0135] like Figure 3 The diagram shown is a hardware structure schematic of an electronic device provided in an embodiment of this application. The electronic device includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method.

[0136] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0137] The electronic devices according to embodiments of this application will now be described in detail.

[0138] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method of this application.

[0139] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0140] This application embodiment also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method.

[0141] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0143] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0146] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any related variations, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0147] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0151] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

[0153] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions, characterized in that, The method includes: Acquire multimodal perception raw data of the target embodied intelligence, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data; Based on the multimodal perception raw data, the task reasoning semantic sequence data, and the physical world prior constraint data, semantic confidence bias features, physical consistency bias features, and action feasibility bias features are calculated respectively. Based on the historical sample correlation parameters in the historical decision sample database, the semantic confidence bias feature, the physical consistency bias feature and the action feasibility bias feature are nonlinearly fused to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. By using a pre-defined hierarchical error correction decision tree, a target execution strategy is generated based on the comprehensive hallucination risk measure and the dominant hallucination type label. Based on the target execution strategy, a hierarchical autonomous recovery and human-machine collaborative takeover are performed, matching the comprehensive hallucination risk measure and the dominant hallucination type label, to obtain the collaborative execution result.

2. The method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions according to claim 1, characterized in that, The task reasoning semantic sequence data includes a sequence of semantic embedding vectors for multiple reasoning steps, and the semantic confidence bias feature is calculated through the following steps: The semantic embedding vector sequence is mapped to the Riemann semantic manifold space to obtain the semantic embedding vector in the manifold space; Calculate the geodesic distance between the semantic embedding of each of the plurality of inference steps in the manifold space and the standard entity embedding of the knowledge graph in the prior constraint data of the physical world; A probability distribution is constructed based on the geodesic distances corresponding to each of the multiple inference steps; Calculate the Riemann entropy of the entire inference chain corresponding to the semantic embedding vector sequence based on the probability distribution; The Riemann entropy is compared with the preset normal inference baseline value and maximum entropy threshold in the prior constraint data of the physical world to obtain the comparison result; The comparison results are normalized to obtain the semantic confidence bias features.

3. The method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions according to claim 1, characterized in that, The multimodal perception raw data includes real-time point clouds, and the physical world prior constraint data includes prior standard point clouds, object physical attribute databases, scene topology maps, and registration deviation allowable thresholds. The physical consistency deviation features are calculated through the following steps: The real-time point cloud is registered with the prior standard point cloud to obtain the average geodesic distance deviation between the real-time point cloud and the prior standard point cloud. Based on the object physical attribute library and the scene topology map, determine the constraint set; Construct a constraint satisfaction problem based on the set of constraints; Based on the constraint satisfaction problem, the perception results corresponding to the real-time point cloud are verified to obtain the number of constraint violations; Calculate the violation ratio based on the number of violated constraints and the set of constraints; Based on the registration deviation allowable threshold, the average geodetic distance deviation is normalized to obtain the registration deviation component; The physical consistency deviation characteristics are obtained by fusing the registration deviation component and the violation ratio.

4. The method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions according to claim 1, characterized in that, The task reasoning semantic sequence data includes task planning action chains; the multimodal perception raw data includes joint pose data; the physical world prior constraint data includes robot kinematic parameters, condition number thresholds, safety distance thresholds, scene topology maps, and object physical attribute databases; and the action feasibility deviation features are calculated through the following steps: The planned trajectory is determined based on the task planning action chain, and the planned trajectory includes multiple pose points; Based on the joint pose data, the task planning action chain, and the robot kinematic parameters, construct the Jacobian matrix; Calculate the Jacobian condition number for each of the plurality of pose points; The Jacobian condition number of each of the plurality of pose points is compared with the condition number threshold to obtain the motion singularity. Based on the scene topology map and the object physical attribute library, a symbolic distance field is constructed; Based on the symbolic distance field, calculate the minimum distance between the planned trajectory and the obstacle; The collision risk is obtained by comparing the minimum distance with the safe distance threshold. By integrating the motion singularity and the collision risk, the action feasibility deviation characteristics are obtained.

5. The method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions according to claim 1, characterized in that, The semantic confidence bias feature, the physical consistency bias feature, and the action feasibility bias feature are nonlinearly fused based on the historical sample correlation parameters in the historical decision sample database to obtain a comprehensive hallucination risk measure and a dominant hallucination type label, including: Based on the historical decision sample database, the semantic confidence bias feature, the physical consistency bias feature, and the action feasibility bias feature are fitted with marginal distributions to obtain semantic marginal distributions, physical marginal distributions, and action marginal distributions. Based on the semantic edge distribution, the physical edge distribution, the action edge distribution, and the historical sample correlation parameters, a vine-like joint distribution structure is constructed to decompose the three-dimensional dependency relationship between the semantic confidence bias feature, the physical consistency bias feature, and the action feasibility bias feature, thereby obtaining the decomposed joint distribution structure. Based on the joint distribution structure, the three-dimensional dependency relationship is characterized to obtain the three-dimensional joint distribution function; The semantic edge distribution, the physical edge distribution, and the action edge distribution are respectively subjected to probability integral transformation to obtain semantic probability integral value, physical probability integral value, and action probability integral value; Substitute the semantic probability integral value, the physical probability integral value, and the action probability integral value into the three-dimensional joint distribution function to calculate the comprehensive hallucination risk measure; The dominant hallucination type label is determined based on the semantic confidence bias feature, the physical consistency bias feature, and the action feasibility bias feature.

6. The method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions according to claim 1, characterized in that, The step of generating a target execution strategy by using a preset hierarchical error correction decision tree, based on the comprehensive hallucination risk measure and the dominant hallucination type label, includes: The comprehensive hallucination risk measure is compared with a preset severity level classification threshold to determine the severity level, which includes mild, moderate and severe. The dominant hallucination type label is matched with a preset hallucination type to determine the hallucination type, which includes reasoning, perception, and action types. The severity level is input into the first layer of the preset hierarchical error correction decision tree, and the first layer outputs a first routing result based on the severity level. The first routing result includes a mild branch, a moderate branch, or a severe branch. The hallucination type is input into the second layer of the preset hierarchical error correction decision tree. The second layer outputs a second routing result based on the hallucination type under the branch pointed to by the first routing result. The second routing result includes a reasoning sub-branch, a perception sub-branch, or an action sub-branch. The second routing result is input into the leaf node of the preset hierarchical error correction decision tree, and the leaf node outputs the set of handling strategies corresponding to the second routing result as the target execution strategy.

7. The method for hierarchical autonomous recovery and human-machine collaborative takeover of embodied intelligent illusions according to claim 1, characterized in that, Based on the target execution strategy, a graded autonomous recovery and human-machine collaborative takeover are executed, matching the comprehensive hallucination risk measure and the dominant hallucination type label, to obtain the collaborative execution result, including: According to the target execution strategy, the corresponding recovery strategy is executed to obtain the corresponding recovery execution result, and the updated multimodal perception raw data, updated task reasoning semantic sequence data, and updated physical world prior constraint data of the target embodied intelligence are obtained. The recovery strategy includes autonomous recovery of embodied intelligence and artificial assisted recovery. Based on the updated multimodal perception raw data, the updated task reasoning semantic sequence data, and the updated physical world prior constraint data, the updated comprehensive hallucination risk measure is calculated. The updated comprehensive hallucination risk measure is compared with the mild threshold in the severity level classification threshold of the preset hierarchical error correction decision tree to obtain the comparison result; Based on the comparison results, the collaborative execution result is generated, which includes either the recovery execution result or the human-machine collaborative takeover result.

8. A hierarchical autonomous recovery and human-machine collaborative takeover system for embodied intelligent illusions, characterized in that, The system includes: The acquisition module is used to acquire the target's embodied intelligence's multimodal perception raw data, task reasoning semantic sequence data, physical world prior constraint data, and historical decision sample database data; The calculation module is used to calculate semantic confidence bias features, physical consistency bias features, and action feasibility bias features based on the multimodal perception raw data, the task reasoning semantic sequence data, and the physical world prior constraint data, respectively. The fusion module is used to perform nonlinear fusion of the semantic confidence bias feature, the physical consistency bias feature and the action feasibility bias feature based on the historical sample correlation parameters in the historical decision sample database, so as to obtain a comprehensive hallucination risk measure and a dominant hallucination type label. The generation module is used to generate a target execution strategy based on the comprehensive hallucination risk measure and the dominant hallucination type label by using a preset hierarchical error correction decision tree; The execution module is used to perform hierarchical autonomous recovery and human-machine collaborative takeover based on the target execution strategy, matching the comprehensive hallucination risk measure and the dominant hallucination type label, to obtain collaborative execution results.

9. An electronic device, characterized in that, It includes at least one processor and a memory for communicatively connecting to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the embodied intelligent illusion hierarchical autonomous recovery and human-machine collaborative takeover method as described in any one of claims 1 to 7.

Images